Data Platform Engineer

Support the design, build, and operation of the Data Platform, blending infrastructure engineering with data engineering practices to keep the Databricks/Azure environment reliable, automated, and self-service for platform users.

Responsibilities

• Design and maintain CI/CD pipelines (Azure DevOps/GitHub Actions) for infrastructure and application deployments.
• Write and maintain Terraform IaC for Azure services and Azure Databricks (Unity Catalog, workspaces, networking, compute policies).
• Develop and support application code in Python and/or .NET for platform tooling and automation.
• Build and maintain shared libraries/modules used by data engineers and analysts across the platform.
• Design and implement platform management processes covering cost tracking, governance, provisioning, and monitoring.
• Collaborate with data engineers, analysts, and security/infrastructure teams to translate requirements into platform capabilities.
• Troubleshoot infrastructure, pipeline, and integration issues, including networking, authentication, and connectivity.
• Contribute to platform documentation and best-practice standards.
• Required Skills & Experience
• Strong experience with Microsoft Azure infrastructure, including Entra ID, Key Vault, networking, and storage.
• Hands-on experience with Azure Databricks, including Unity Catalog and cluster/job management.
• Terraform expertise, preferably in multi-environment deployments.
• Proficiency in Python and SQL; .NET experience is an advantage.
• Experience building and maintaining CI/CD pipelines.
• Understanding of data platform governance and cost management.
• Ability to work effectively across infrastructure and data engineering domains.
• Nice to Have
• Experience with Medallion Architecture (Bronze/Silver/Gold).
• Exposure to FinOps and cost attribution practices.

Qualifications

• Bachelor's or Master's degree in Computer Science, Information Technology, Data Engineering, Software Engineering, or a related field.
• 5+ years of experience in Data Engineering, Data Platform Engineering, or Big Data development.
• Proven experience designing, building, and supporting scalable enterprise data platforms.
• Strong understanding of data architecture, data modeling, and distributed computing concepts.
• Experience working in Agile/Scrum development environments.
• Excellent analytical, problem-solving, and troubleshooting skills.