On-siteSeniorData Engineerposted 10mo ago

✦Role summaryAI-generated

This senior data engineer role at EXL in Gurugram requires hands-on expertise in Azure-based ETL pipelines and cloud data infrastructure, with a focus on optimizing large-scale data workflows and collaborating with cross-functional teams to resolve complex data challenges. The position emphasizes leadership in Azure cloud environments and deep technical problem-solving for enterprise-grade data systems.

Skills required

Azure data lake storage Synapse analytics Etl pipelines Azure sql database Blob storage Api integration Data pipeline optimization Cloud etl leadership Data engineering Data pipelines ETL Azure SQL Databricks Spark Delta Lake

About this role

Data Engineers design and build data systems and pipelines. Responsibilities include developing data processing workflows, optimizing data storage, and ensuring data accuracy. You will collaborate with data scientists and analysts to meet data requirements and resolve data issues. Strong experience in data engineering and problem-solving skills are required.

Responsibilities

Design and implement data pipelines, Optimize data processing and storage, Ensure data solutions meet performance standards, Provide technical support, Collaborate with stakeholders.

Qualifications

8+ years in data engineering/ETL roles, with at least 4+ years in Azure cloud ETL leadership. Azure certifications (e.g., Azure Analytics Specialty, Solutions

Required Qualifications:

Azure Data Sources: Azure Data Lake Storage (ADLS), Blob Storage, Azure SQL Database, Synapse Analytics. External Sources: APIs, on-prem databases, flat files (CSV, Parquet, JSON).
Tools: Azure Data Factory (ADF) for orchestration, Databricks connectors.
Apache Spark: Strong knowledge of Spark (PySpark, Spark SQL) for distributed processing.
Data Cleaning & Normalization: Handling nulls, duplicates, schema evolution.
Performance Optimization: Partitioning, caching, broadcast joins.
Delta Lake: Implementing ACID transactions, time travel, and schema enforcement.
Azure Data Factory (ADF): Building pipelines to orchestrate Databricks notebooks.
Azure Key Vault: Secure credential management.
Azure Monitor & Logging: For ETL job monitoring and alerting.
Networking & Security: VNET integration, private endpoints.