On-siteMidData Engineerposted 3w ago

✦Role summaryAI-generated

The Data Engineer will design and develop scalable data platforms and pipelines using Apache Spark/pySpark and Azure technologies like Data Factory, Synapse, and Databricks. They will ensure data consistency, quality, and reliability while collaborating with analysts and data scientists to support analytical and operational use cases.

About this role

Responsibilities

Design and develop scalable and reliable data platforms and/or data pipelines for analytical and operational use cases
Design and implement data processing workflows using distributed processing frameworks such as Apache Spark/pySpark (Databricks)
Contribute to the evolution of the organization’s cloud data platform using Azure technologies (e.g., Azure Data Factory, Synapse, Microsoft Fabrik, Databricks, Event Hub, Azure Data Lake Storage)
Strong background in data modelling to support analytics, reporting, and downstream data consumption
Integrate and process data from multiple internal and external sources while ensuring data consistency, quality, and reliability
Take ownership of data pipeline reliability, monitoring, and performance, ensuring scalable and maintainable data processing workflows
Collaborate with data analysts, data scientists, and engineering teams to translate business and analytical requirements into robust data solutions
Drive improvements in data engineering practices, including pipeline design standards, testing approaches, and operational reliability
Participate in evaluating and introducing new tools and technologies within the data engineering ecosystem

Qualifications

Required Qualifications

Strong hands-on experience (4+ years) in data engineering or building data processing platforms
Strong SQL skills and solid experience with Python for building and maintaining data pipelines
Experience working with distributed data processing frameworks such as Apache Spark/pySpark (Databricks experience is a strong advantage)
Practical experience designing and implementing data pipelines in cloud environments, preferably within the Azure ecosystem
Experience designing data pipelines and data models for production-scale analytical systems
Strong understanding of data engineering concepts including ETL/ELT design, dimensional modeling, and data warehousing principles
Experience working with modern data lake architectures and data formats such as Parquet and JSON
Experience with workflow orchestration tools (e.g., Airflow)
Experience with database development and performance optimization
Ability to design scalable and maintainable data solutions with minimal supervision
Strong collaboration and communication skills
Professional working proficiency in English

Nice to Have

Experience with modern cloud data warehouses such as Snowflake, BigQuery, or Redshift
Experience with streaming data technologies (e.g., Event Hub, Kafka, Spark Streaming)
Experience with Delta Lake or other lakehouse technologies
Familiarity with CI/CD practices and Git-based development workflows
Experience working in agile development environments