SS
Sigma Software
Data Engineer

Senior Data Engineer

HybridSeniorData Engineerposted 1mo ago
✦Role summaryAI-generated

The Senior Data Engineer will design, build, and maintain large‑scale ingestion and transformation pipelines on BigQuery, ensuring data freshness and quality for advertising analytics. They will also create experimentation infrastructure, manage schema contracts, and collaborate with customer engineers to document operational processes.

Skills required

About this role

  • Write and defend diagnostic SQL queries against large-scale production datasets
  • Build and maintain ingestion pipelines for bid, win, and impression logs into BigQuery
  • Harmonize fields across independently designed datasets and maintain versioned field mappings
  • Develop point-in-time-correct feature tables and aggregation pipelines
  • Design and maintain conversion and labeling pipelines with delayed label handling
  • Own the data serving write path, schema contracts, publishing flows, and freshness SLOs
  • Build experimentation infrastructure including traffic splitting and reporting pipelines
  • Perform large-scale historical backfills and safe reprocessing after mapping changes
  • Implement data isolation and safe-aggregation controls for advertiser data protection
  • Develop automated data quality validation frameworks
  • Collaborate closely with Customer engineers and prepare operational documentation
  • Contribute to architecture discussions and platform scalability improvements

Qualifications

  • 5+ years of experience in Data Engineering
  • At least 2 years of experience working with production ML or large-scale analytics pipelines
  • Expert-level SQL skills including window functions and incremental processing patterns
  • Strong Python skills for production-grade pipeline development
  • Hands-on experience with Spark or PySpark
  • Experience designing ETL / ELT pipelines with Airflow, Cloud Composer, Dagster, or similar tools
  • Experience working with cloud data warehouses at scale, preferably BigQuery
  • Strong understanding of data modeling and point-in-time correctness
  • Experience working with event-driven or clickstream datasets at very large scale
  • Experience supporting business-critical production pipelines
  • Upper-Intermediate English level or higher

WILL BE A PLUS

  • Experience with GCP services including Dataflow, Pub/Sub, GCS, and Beam
  • Experience building streaming or near-real-time ingestion systems
  • Understanding of feature stores, train/serve skew, and label leakage prevention
  • Experience in AdTech or auction-based environments
  • Experience handling delayed or incomplete labels in ML systems
  • Experience with dbt or similar transformation frameworks
  • Experience delivering solutions into Customer-owned infrastructure
  • Knowledge of GDPR/CCPA-related privacy engineering practices
  • Experience with experimentation infrastructure and statistical validation pipelines
  • Experience working in hybrid cloud/on-prem Linux environments
  • Terraform and Kubernetes experience
  • Experience optimizing warehouse cost and performance

Additional Information

PERSONAL PROFILE

  • Strong analytical and problem-solving skills
  • Ownership-oriented mindset
  • Ability to work independently in a client-facing environment
  • Strong communication and documentation skills
  • Comfortable working in a fast-paced engineering environment
  • Collaborative and proactive attitude
score your resume against this role

Similar open roles

KG
NEW

Data & Analytics Engineer - Consultant

KPMG Global Services·Bangalore, Karnataka, India; Pune, Maharashtra, India
On-siteMidData Engineer
yesterday
KG
NEW

Data & Analytics Engineer - Consultant

KPMG Global Services·Bangalore, Karnataka, India; Pune, Maharashtra, India
On-siteMidData Engineer
yesterday
KG
NEW

Data & Analytics Engineer - Consultant

KPMG Global Services·Bangalore, Karnataka, India
On-siteMidData Engineer
yesterday
MU
NEW

Data Engineering- Senior Developer/Analyst

Mitsubishi UFJ Financial Group·MUFG Global Service Private Ltd. - Bengaluru (BCIT)
On-siteSeniorData Engineer
yesterday
S
NEW

Senior Staff Data Platform Software Engineer

Servicenow·San Diego, CALIFORNIA, United States
On-siteStaffData Engineer
2d ago
Apply on Sigma Software →