Our client is the best technological consultant company, specialist in Microsoft, as well as Dynamics 365 FO, Dynamics 365, AX, Power BI, CRM, EPR, Office 365 and Azure. With offices in Madrid, Barcelona, Valencia and Castellon with specialized professional experts, the company has aided in the development of small businesses through automatization and implantation of Microsoft solutions.THE ROLEAs a Data Engineer/Big Data Developer you will develop and enhance data systems and pipelines, support the process of optimization in extraction, transformation and loading of large data sets, and design solutions with strategic focus on code quality, reliability, reusability, maintainability and dataops, all of which will have an immediate impact on the day-to-day functioning of the company.Data reporting and visualization to support business operations and decisions.Design data structures and write code to combine and transform datasets to meet business needs.Building required infrastructure for optimal extraction, transformation and loading of data from various data sources using AWS and SQL technologiesPerform statistical analyses on a wide range of datasets.Convert SAS based pipelines into languages like PySpark, Scala to execute on Hadoop and non-Hadoop ecosystemsBuild new datasets proactively; seek out, develop, and share new algorithms and best practices.Build and maintain the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using Python and SQL (Snowflake & SQL Server) in both on premise and cloud environmentsManage and improve the user facing server infrastructure (web + api) including authorization and load balancingWork with Cloud Computing Platforms (GCP/AWS), Luigi, Kafka and other open-source technologiesImplement both batch and streaming workflows that combine, transform and enrich data utilizing Microsoft AzureREQUIREMENTSBachelor/Master´s in Computer Sc./App or Information TechnologyExperience of more 3+ years in a similar roleDesirable experience on version control (Git), Data Modelling, Data Pipelines, Solution Development, Product Industrialization, CI/CD (Jenkins or Git hub)Experience using analytics programming languages (Python, Spark, Scala, Java…) to manipulate data and draw insights from large data sets.Knowledge in machine learning techniques.Experience in cloud infrastructure such as MS AzureExperience with Spark, SQL, NoSQL, DatabricksKnowledge in distributed systems and frameworks such as Hadoop, HBase, NoSQL. Spark SQL, Spark Streaming, HDFS, Hive.Minimum 2-year experience in: Spark with Scala, SQL, Azure, Power BI, Agile WorkingAbility to work individually without much direction while also working as part of a team towards a common goal.Good written and oral communication skills.Results-oriented, self-directed ability to drive change in an unstructured environment.International experience and multicultural skills are highly valueTravel availability: Yes, 10% maximum of total working timeUpload your resume and our tool will compare it to the requirements for this job like recruiters do.#J-18808-Ljbffr
