GCP Data Engineer

Data Engineering

• SQL
• Data Warehousing Concepts
• ETL/ELT Development
• Data Validation and Reconciliation
• Data Pipeline Development

Cloud Technologies

• Google Cloud Platform (GCP)
• BigQuery
• Cloud Storage
• Cloud Composer (Airflow)

Development Tools

• DBT (Data Build Tool)
• Git/GitHub
• Python (Basic to Intermediate)

SAS Knowledge

• Understanding of SAS datasets
• PROC SQL
• SAS ETL concepts
• SAS code analysis

Data Modeling Understanding

• Star Schema
• Dimension and Fact Tables
• Source-to-Target Mapping
• Data Lineage Concepts

Responsibilities

Manage data engineering projects. Provide strategic guidance. Oversee data engineering team. Ensure continuous improvement of data processes.

Qualifications

SAS Migration Support

• Analyze SAS programs, PROC SQL code, and SAS datasets.
• Assist in converting SAS transformation logic into DBT models and SQL transformations.
• Support migration of data from legacy SAS environments to GCP.
• Participate in code conversion, testing, and reconciliation activities.

Data Pipeline Development

• Develop and maintain ELT pipelines using DBT and BigQuery.
• Build reusable transformation models following DBT best practices.
• Implement data ingestion and processing workflows.
• Support batch and incremental data processing requirements.

Data Transformation & Modeling

• Develop SQL-based transformations for Bronze, Silver, and Gold layers.
• Implement business rules and data quality validations.
• Support dimensional models, fact tables, and dimension tables.
• Assist Data Modelers and Architects in implementing target-state data models.

Data Quality & Testing

• Perform source-to-target validation and reconciliation.
• Support automated testing using DBT tests.
• Investigate and resolve data quality issues.
• Ensure completeness, accuracy, and consistency of migrated data.

GCP Development

• Work with GCP services including:
  • BigQuery
  • Cloud Storage
  • Dataproc
  • Cloud Composer (Airflow)