Software Engineer – Data Platform
This role focuses on designing and operating secure, scalable data ingestion pipelines from diverse sources such as SAP, Salesforce, and legacy manufacturing platforms. You will build production‑grade metadata‑driven components and APIs, ensuring end‑to‑end ownership and adherence to CI/CD best practices.
Skills required
About this role
Job Description
Job Description
This is a software engineering role focused on building production-grade data platform components.
You will design and operate services, APIs, and data pipelines in a CI/CD-driven environment.
This is NOT a notebook-driven or analytics-focused role.
Job Description:
- Design, implement, and maintain secure, scalable and fault-tolerant data ingestion pipelines from a wide variety of source systems, including SAP, Salesforce, SharePoint, APIs, and (legacy) manufacturing platforms.
- Build and enhance production-grade metadata-driven data handling components and inbound-/outbound-interfaces.
- Own production data interfaces end-to-end.
- Develop reusable, version-controlled platform components, APIs, or data interfaces, following software engineering best practices (CI/CD, automated testing, version control, monitoring).
- Contributes to a global data engineering team delivering to all major business domains.
- Establish and maintain holistic data quality management, monitoring and reporting.
- Collaborates across business IT, cybersecurity, infrastructure, and architecture teams to ensure secure and sustainable delivery.
Main Tasks:
- Design and implement fault-tolerant, observable data pipelines, including logging, alerting, and failure handling strategies.
- Build and maintain Python- or Scala-based extraction services (e.g., Debezium Server, custom APIs, rclone)
- Implement CDC, delta, and event-based patterns.
- Build and enhance push-based HTTP, Kerberos-authenticated custom data lake transfer delivery component.
- Integrate with systems such as Salesforce, SharePoint, and other API- or file-based endpoints.
- Establish, operate and troubleshoot extraction from SAP using tools like Theobald Extract Universal.
- Establish and maintain a web-accessible data catalog application as part of the data platform, focusing on backend services, metadata management, and integration with platform components.
- Integrate dataset discoverability, preview/exploration options, and lineage information using Unity Catalog as a backend metadata system.
- Design and implement structured access request workflows including request submission, approval chains, audit trail, and enablement triggers.
- Perform design reviews with Cybersecurity.
- Ensure documentation and compliance for all interfaces and data ingress points.
- Manage audit and traceability requirements.
- Collaborate closely with IT and business users to translate requirements into scalable technical patterns.
- Serve as technical escalation point for complex source integration.
- Define and implement a multi-layered data quality framework, including unit-level, integration-level, and cross-pipeline validation rules.
- Establish centralized and version-controlled storage of DQ rules, with integration into orchestration and CI/CD pipelines.
- Implement automatic DQ monitoring with severity levels (Critical, High, Medium, Low) and enable flagging, filtering, and quarantining logic at relevant stages of the pipeline.
- Collaborate with source system owners and business stakeholders to define meaningful and actionable DQ thresholds.
Qualifications
Qualifications
Degree in Computer Science, Data Engineering, or a related field.
- 5–8 years in software development, with hands-on experience with inbound- and outbound data exchange interfaces, using a variety of technologies, such as APIs, CDC, managing endpoints.
- Hands-on experience with Scala / Java, Python, and advanced SQL, using Git for collaborative development and CI/CD methodologies.
- Experience building robust ingestion pipelines and ETL-jobs in a software engineering context.
- Track record with Azure or other hyperscaler resource management.
- Familiarity with data lake housing and data warehousing, Databricks and Azure knowledge is a plus.
- Comfortable working across geographies and time zones; collaborates effectively with global teams and enterprise stakeholders.
- Candidates with a strong backend/software engineering background (e.g. Java, Scala, distributed systems) who want to move into data platform engineering are highly encouraged to apply.
NOT a fit if your experience is primarily:
▪ Notebook-based data processing (e.g. Jupyter, ad-hoc PySpark) without production deployment
▪ Dashboarding / analytics / BI tools
▪ SQL-only roles without software engineering experience
Additional Information
Additional information
The well-being of our employees is important to us. That's why we offer exciting career prospects and support you in achieving a good work-life balance with additional benefits such as:
- Training opportunities
- Mobile and flexible working models
- Sabbaticals
and much more...
Sounds interesting for you? Click here to find out more.
Diversity, Inclusion & Belonging are important to us and make our company strong and successful. We offer equal opportunities to everyone - regardless of age, gender, nationality, cultural background, disability, religion, ideology or sexual orientation.
Ready to drive with Continental? Take the first step and fill in the online application.
Ready to drive with Continental? Take the first step and fill in the online application.