BC
Bitful Consulting
Data Engineer

Senior Data Engineer

RemoteSeniorData Engineerposted 1mo ago
✦Role summaryAI-generated

This role focuses on building and maintaining scalable data pipelines in a Databricks lakehouse environment, ingesting and transforming structured and unstructured data with Python, PySpark, and SQL. You will also develop reporting datasets for Tableau and Spotfire, improve pipeline reliability, and implement automated testing, monitoring, and CI/CD practices.

Skills required

About this role

  • You’ll build and maintain data pipelines and datasets that support reporting, visualization, machine learning and data-driven decision making across finance, sales, supply chain, marketing and other business functions.
  • Your responsibilities will include:
    • Developing and maintaining scalable data pipelines and reusable datasets within a Databricks lakehouse environment
    • Ingesting, validating, cleaning and transforming structured and unstructured data from multiple sources using Python, PySpark and SQL
    • Building data models and reporting datasets that support Tableau and established analytics tools such as Spotfire
    • Integrating established data warehouse and reporting systems with modern cloud analytics platforms
    • Improving pipeline reliability, data quality and processing performance
    • Implementing automated testing, monitoring, documentation and CI/CD practices
    • Partnering with analysts, architects, developers and business stakeholders to deliver practical data solutions
    • Providing technical guidance, reviewing code and mentoring other data engineer

Requirements

    • 7+ years of experience in data engineering, including building and supporting production data pipelines
    • Strong SQL skills and a solid understanding of data modeling, database design and data warehousing
    • Hands-on experience with Databricks, Apache Spark and cloud data platforms
    • Strong proficiency in Python and PySpark
    • Experience developing ETL/ELT pipelines and integrating data from multiple source systems
    • Experience preparing reliable datasets for Tableau or comparable business intelligence tools
    • Familiarity with source control, automated testing and CI/CD practices
    • Ability to troubleshoot complex data issues and optimize pipeline performance
    • Strong communication skills and experience collaborating with technical teams and business stakeholders
    • Experience with Spotfire, Oracle, Informatica, R or data virtualization preferred
    • Databricks certification preferred
score your resume against this role

Similar open roles

FG
NEW

Data Engineer

Ford Global·Chennai, Tamil Nadu, India
On-siteSeniorData Engineer
today
D
NEW

Data Platform Engineer

Domino's·Ann Arbor, MI, United States
On-siteSeniorData Engineer
today
A
NEW

Staff Data Platform Engineer, Knowledge Platform

Airwallex·US - Seattle
HybridStaffData Engineer
$200k – $260k USD
today
A
NEW

Data Platform Principal Architect, Seattle/San Francisco

Airwallex·US - Seattle, US - San Francisco
On-siteStaffData Engineer
$220k – $300k USD
today
R
NEW

Senior Data Engineer - (GCP & Lakehouse Architecture) - On Site - Octopus by RTG

robusta·Riyadh, Riyadh Province, Saudi Arabia
On-siteSeniorData Engineer
GCPApache icebergPython
yesterday