SS
Sigma Software
Data Engineer

Data Engineer (DATABRICKS / PYSPARK)

HybridMidData Engineerposted 2w ago
✦Role summaryAI-generated

The candidate will design, build, and maintain scalable data pipelines using Databricks and PySpark to process large batch and near real-time datasets. They will also ensure data quality and reliability while collaborating with cross-functional teams to deliver lakehouse solutions.

Skills required

DatabricksSparkSQLPythonETLBig dataData lakehouseBatch processingReal-time streamingData qualityData pipelinesData engineeringDelta LakeAWS

About this role

  • Develop and maintain data pipelines for processing large volumes of data in batch and near real-time environments
  • Work with Databricks, PySpark, and SQL to transform, validate, and optimize data
  • Support implementation of Data Lakehouse solutions following engineering standards and best practices
  • Build, test, and automate ETL processes and data workflows
  • Improve data quality, organization, and reliability across platforms
  • Collaborate with engineers, analysts, and stakeholders to deliver scalable solutions
  • Troubleshoot, optimize, and monitor data pipelines
  • Stay up to date with modern data engineering technologies
  • Ensure deliverables meet quality standards and project requirements

Qualifications

  • 2+ years of experience in Big Data development and database design
  • Strong hands-on experience with SQL
  • Strong Python development skills
  • Hands-on experience with PySpark
  • Experience developing and maintaining data pipelines in Databricks
  • Experience with Delta Lake
  • Experience with Medallion Architecture (Bronze, Silver, Gold layers)
  • Strong analytical and problem-solving skills
  • Intermediate or higher English level

WILL BE A PLUS

  • Experience with AWS services, including S3
score your resume against this role

Similar open roles

Z
NEW

D365 Data Engineer

Zensar·India
On-siteSeniorData Engineer
today
UG
NEW

Senior Data Engineer

UBDS Group·London, England, United Kingdom; Manchester, England, United Kingdom
On-siteSeniorData Engineer
today
C
NEW

Senior Data Engineer

CodeLink·Da Nang, Hai Chau, Vietnam; Hanoi, Ha Noi, Vietnam; Ho Chi Minh City, Ho Chi Minh, Vietnam
On-siteSeniorData Engineer
Master data managementData integrationETL
today
C
NEW

Data Engineer (Machine Learning & Data Platform)

CodeLink·Da Nang, Hai Chau, Vietnam; Hanoi, Ha Noi, Vietnam; Ho Chi Minh City, Ho Chi Minh, Vietnam
On-siteMidData Engineer
today
B
NEW

Data Architect - R01573922

Brillio·Bangalore, Karnataka, India
On-siteSeniorData Engineer
today
Apply on Sigma Software →