R
robusta
Data Engineer

Senior Data Engineer - (GCP & Lakehouse Architecture) - On Site - Octopus by RTG

On-siteSeniorData EngineerJust posted
✦Role summaryAI-generated

The Senior Data Engineer will design and maintain a GCP-based Lakehouse platform using Apache Iceberg, ensuring transactional consistency and time-travel capabilities. They will also build and optimize high-throughput ETL/ELT pipelines with Python and Apache Airflow to support analytics, machine learning, and BI workloads.

Skills required

About this role

We are seeking an experienced Senior Data Engineer to lead the design, development, and scaling of our next-generation, cloud-native Data Lakehouse platform on Google Cloud Platform (GCP). In this role, you will be instrumental in modernizing our data infrastructure, building robust ELT/ETL pipelines, and establishing enterprise-grade data modeling standards to empower analytics, machine learning, and business intelligence. If you are passionate about modern open-source table formats, high-performance distributed computing, and data reliability, we would love to have you on the team.

Key Responsibilities

• Lakehouse Architecture & Development: Design, deploy, and maintain a scalable GCP-based Lakehouse ecosystem leveraging Apache Iceberg for transactional consistency, time-travel, and efficient storage management.

• Pipeline Engineering: Architect, build, and optimize high-throughput ETL/ELT data pipelines using Python and Apache Airflow for orchestrating complex dependency workflows.

• Big Data Processing: Implement large-scale distributed data processing workloads and transformation pipelines using Apache Spark.

• Data Warehousing & Analytics: Leverage BigQuery for high-performance analytical queries, cost optimization, partition/cluster management, and serving structured data to downstream consumers.

• Transformation & Quality: Write clean, modular, and testable transformation code using dbt (Data Build Tool), embedding automated data testing, documentation,and lineage tracking.

•Data Modeling: Lead dimensional and semantic data modeling efforts (Star/Snowflake schemas, One-Big-Table, and Lakehouse medallion architectures) to ensure high query performance and seamless business usability.

• Performance Tuning & SQL Mastery: Write, optimize, and troubleshoot complex SQL queries to ensure low latency and cost-efficient processing across large datasets.

• Collaboration: Partner closely with data scientists, analytics teams, and product managers to understand data requirements and deliver robust, production-ready data products.

Requirements

•3+ years of hands-on experience designing and implementing enterprise-grade data platforms and pipelines.

• Cloud Platform: Proven production experience with Google Cloud Platform (GCP) core data services (BigQuery, Cloud Storage, Dataflow/Dataproc, IAM).

• Lakehouse & Table Formats: Deep understanding of modern lakehouse concepts and hands-on experience with Apache Iceberg.

• Transformation & Orchestration: Advanced proficiency in dbt for pipeline transformations and Apache Airflow for workflow orchestration.

• Big Data & Compute: Strong working knowledge of Apache Spark for distributed data processing.

• Programming & Querying:

o Advanced Python programming skills for data engineering tasks and custom operator development.

o Expert-level SQL writing and performance tuning capabilities.

• Data Modeling: Demonstrated expertise in dimensional modeling, data warehousing concepts, and schema design for analytical workloads.

• Methodologies: Strong grasp of modern ETL/ELT design patterns, CI/CD for data pipelines, data governance, and data quality best practices.

Apply on robusta →Opens in new tab
score your resume against this role

Similar open roles

Z
NEW

D365 Data Engineer

Zensar·India
On-siteSeniorData Engineer
today
UG
NEW

Senior Data Engineer

UBDS Group·London, England, United Kingdom; Manchester, England, United Kingdom
On-siteSeniorData Engineer
today
C
NEW

Senior Data Engineer

CodeLink·Da Nang, Hai Chau, Vietnam; Hanoi, Ha Noi, Vietnam; Ho Chi Minh City, Ho Chi Minh, Vietnam
On-siteSeniorData Engineer
Master data managementData integrationETL
today
C
NEW

Data Engineer (Machine Learning & Data Platform)

CodeLink·Da Nang, Hai Chau, Vietnam; Hanoi, Ha Noi, Vietnam; Ho Chi Minh City, Ho Chi Minh, Vietnam
On-siteMidData Engineer
today
B
NEW

Data Architect - R01573922

Brillio·Bangalore, Karnataka, India
On-siteSeniorData Engineer
today
Apply on robusta →