Lead Data Engineer

Design and deliver robust, production-grade data-intensive applications and services that power business-critical use cases at scale.
This role combines strong software engineering fundamentals with deep data engineering expertise and technical leadership.

Qualifications

• • Bachelor’s degree in computer science, Computer Engineering, relevant technical field, or equivalent; Master’s degree preferred.

Additional Information

Roles & Responsibility:

• Operates effectively with minimal supervision, demonstrating strong ownership and accountability.
• Translates high-level business requirements into well-designed, scalable, and robust systems and applications.
• Exhibits exceptional data and software engineering expertise, drives architectural decisions and technology stack selection, establishes high-quality engineering standards, and ensures adherence through thorough code reviews and best practices.
• Independently leads discussions with stakeholders, gathers and refines requirements, and develops deep domain expertise to deliver effective technical solutions.
• Design end-to-end systems (ingestion, processing, storage, serving) with minimal supervision.
• Build and maintain high-quality data products and pipelines on Spark and Databricks.
• Design event-driven architectures using Kafka and related technologies for real-time data processing.
• Build backend services and APIs that expose data capabilities reliably and securely.
• Build and optimize data models for analytical workloads (for example, dimensional modeling, data vault)
• Write clean, testable, and maintainable software in Scala (preferred) or Java.
• Drive architecture and design discussions with engineers, architects, and stakeholders.
• Coach and mentor junior engineers through code reviews, design sessions, and hands-on pairing.
• Improve engineering standards across testing, observability, CI/CD, reliability, and documentation.
• Deliver production-quality software with strong automated test coverage and clear quality gates.
• Use design reviews and lightweight ADR-style documentation for major technical decisions.
• Build observability into systems from day one (logs, metrics, tracing, alerts).
• Deliver through CI/CD with safe release and rollback strategies.

Skills

• 8+ years of professional software engineering experience, including substantial backend and data platform work.
• Strong programming background in Scala or Java (Scala preferred).
• Proven experience building distributed data pipelines with Spark and Databricks.
• Hands-on experience with Kafka in production systems.
• Strong understanding of distributed systems concepts and failure handling.
• Solid experience designing and developing APIs (REST and/or event-driven service interfaces).
• Strong knowledge of relational databases (for example PostgreSQL), data modeling, and query optimization.
• Demonstrated ability to design high-performance, concurrent systems.
• Experience leading technical design and architecture decisions across teams.
• Experience mentoring or coaching junior engineers.

Good To Have:

• Practical experience with Flink and advanced stream processing patterns.
• Experience with cloud-native data platforms and infrastructure-as-code.
• Experience with building & deploying applications on Kubernetes
• Experience with data governance, security, and compliance in data platforms.
• Familiarity with table formats and query engines in modern lakehouse ecosystems.
• Experience with performance profiling and JVM tuning.

Collaboration Scope:

• Partner with product, platform, and domain teams to translate requirements into resilient technical solutions.
• Work closely with architects and technical leadership to shape platform direction.
• Support incident response and root-cause analysis for critical production issues.
• Help define roadmaps that align business priorities with engineering sustainability.