Lead / Principal Data Engineer - DBT & Databricks

Job Title: Lead / Principal Data Engineer - DBT & Databricks
Key Skills: DBT, Azure Databricks, SQL, Python, Delta Lake, AI-Assisted Development
Experience: 5+ YOE.
Location: Costa Rica, Peru, Colombia, and Bolivia.
Mode: Remote.

We at Coforge are hiring Lead / Principal Data Engineer - DBT & Databricks (#22555) with the following skill set.

Key Responsibilities
· Design, build, and maintain DBT models on Azure Databricks using Delta Lake, Unity Catalog, and Medallion Architecture principles.
· Deliver curated Gold-layer data domains to production standards in regulated enterprise environments.
· Leverage AI-assisted development tools such as Claude Code for model generation, refactoring, documentation, debugging, and engineering acceleration.
· Establish comprehensive DBT testing strategies, including schema validation, data quality testing, unit testing, and automated quality controls.
· Work within fast-paced, CI/CD-driven delivery environments using pull request workflows and automated quality gates.
· Curate datasets, semantic definitions, and business terminology to support Databricks Genie and natural-language analytics initiatives.
· Define and enforce data contracts, domain ownership, and governance standards.
· Implement data observability, monitoring, and root-cause analysis practices across data platforms.
· Drive modernization and migration initiatives through AI-assisted Data Engineering approaches.
· Contribute to engineering playbooks, standards, and enablement programs related to DBT, Databricks, and AI-powered development.
· Measure and improve AI-generated code adoption, quality, productivity, and delivery outcomes.

Required Skills & Qualifications
· 5+ years of experience in Data Engineering.
· At least 2 years of hands-on experience building and operating production-grade DBT projects, including models, tests, macros, snapshots, and large-scale refactoring.
· Strong SQL skills and solid Python programming experience.
· Hands-on experience with Azure Databricks, including Delta Lake, Unity Catalog, and Medallion Architecture.
· Strong knowledge of DBT testing frameworks, data validation, and quality assurance best practices.
· Familiarity with tools such as dbt-expectations, Great Expectations, or Delta Live Tables expectations.
· Experience working with Git-based pull request workflows, code reviews, and CI/CD deployment pipelines.
· Experience using AI-powered development tools such as Claude Code, GitHub Copilot, Cursor, or similar solutions.
· Ability to independently own data domains and collaborate directly with business stakeholders.
· Strong communication, analytical, and problem-solving skills.

Preferred Skills
· Experience curating Databricks Genie environments or building semantic layers for natural-language analytics.
· Experience within Insurance or Financial Services domains.
· Experience working with policy, claims, exposure, or regulatory reporting data.
· Hands-on experience with PySpark for large-scale data transformations.
· Experience with Databricks Workflows, Airflow, or similar orchestration platforms.
· DBT and/or Databricks certifications.
· Experience implementing AI-driven modernization and migration programs.

Posted On: 14-08-2026

At Coforge, we hire professionals based solely on their skills and qualifications and do not discriminate based on age, disability, religion, gender, sexual orientation, socioeconomic status, or nationality.