SS
Sigma Software
Data Engineer
Data Engineer (DATABRICKS / PYSPARK)
HybridMidData Engineerposted 2w ago
Role summaryAI-generated
The candidate will design, build, and maintain scalable data pipelines using Databricks and PySpark to process large batch and near real-time datasets. They will also ensure data quality and reliability while collaborating with cross-functional teams to deliver lakehouse solutions.
Skills required
DatabricksSparkSQLPythonETLBig dataData lakehouseBatch processingReal-time streamingData qualityData pipelinesData engineeringDelta LakeAWS
About this role
- Develop and maintain data pipelines for processing large volumes of data in batch and near real-time environments
- Work with Databricks, PySpark, and SQL to transform, validate, and optimize data
- Support implementation of Data Lakehouse solutions following engineering standards and best practices
- Build, test, and automate ETL processes and data workflows
- Improve data quality, organization, and reliability across platforms
- Collaborate with engineers, analysts, and stakeholders to deliver scalable solutions
- Troubleshoot, optimize, and monitor data pipelines
- Stay up to date with modern data engineering technologies
- Ensure deliverables meet quality standards and project requirements
Qualifications
- 2+ years of experience in Big Data development and database design
- Strong hands-on experience with SQL
- Strong Python development skills
- Hands-on experience with PySpark
- Experience developing and maintaining data pipelines in Databricks
- Experience with Delta Lake
- Experience with Medallion Architecture (Bronze, Silver, Gold layers)
- Strong analytical and problem-solving skills
- Intermediate or higher English level
WILL BE A PLUS
- Experience with AWS services, including S3
Apply on Sigma Software →Opens in new tab
score your resume against this role