I
Inetum
Data Engineer

DATA ENGINEER (Scala/Spark)

On-siteSeniorData Engineerposted 1w ago
✦Role summaryAI-generated

The candidate will lead the migration of the existing Hadoop platform to a cloud‑native stack on Kubernetes, leveraging Spark‑as‑a‑service and Airflow for scalable data processing. They will also design, implement, and automate robust Scala/SQL data pipelines with CI/CD, testing, and documentation to ensure data quality and reliability.

Skills required

About this role

Hybrid work mode in Madrid. This service could imply a willingness to travel occasionally within Italy.

The main responsibilities are:

  • Migrate the existing Hadoop infrastructure to cloud infrastructure on Kubernetes Engine, COS, Spark as a service, and Airflow as a service.
  • Implement data transformation and quality to ensure data consistency and accuracy. Utilize programming languages such as Scala and SQL and tools like Spark for data transformation and enrichment operations.
  • Set up CI/CD pipelines to automate deployments, unit testing and development management.
  • Write and conduct unit and validation tests to ensure accuracy and integrity of code developed.
  • Automate data pipelines and streamline data ingestion through the implementation of different orchestrators and scheduling processes (Airflow as a Service mainly).
  • Writing technical documentation (specifications, operational documents) to ensure knowledge capitalization.
  • Collaborate with cross-functional teams to understand data requirements and deliver solutions.
  • Foster a culture of continuous learning and improvement within the team.

Qualifications

Good knowledge of:

  • Spark on Scala
  • CI/CD tools (Gitlab, Jenkins…)
  • HDFS and structured databases (SQL)
  • Apache Airflow
  • S3 storage/COS and parquet (and ORC) format
  • Strong knowledge of SQL and NoSQL databases
  • Hadoop

Full understanding of:

  • Oozie
  • Shell script
  • Software Development Lifecycle (SDLC) awareness
  • Agile principles and ceremonies
  • Design effective prompts to leverage Gen AI tools across IT domains (e.g., development, testing, data generation, documentation) during the development stage.

Some knowledge of:

  • Kubernetes containerization
  • Dremio as tool to virtualize data

Optionally/ as a plus:

  • Elasticsearch and Kibana
  • Streaming process (Kafka, event steam…)
  • HVault

Additional Information

BUSINESS AND TRANSVERSAL SKILLS

  • Knowledge of banking industry and processes
  • Problem-Solving and Decision-Making person
  • Business / IT relationship (including IT OPS)
  • Ability to understand, explain and support change
  • Ability to Deliver / Results driven
  • Ability to collaborate / Teamwork with data squads and business teams


LANGUAGES

  • English level C1
  • Optionally Fench
Apply on Inetum →Opens in new tab
score your resume against this role

Similar open roles

G
NEW

Data Platform Engineer

GoTymeX·Ho Chi Minh City, Ho Chi Minh, Vietnam
On-siteSeniorData Engineer
today
O
NEW

Senior Data Engineer

Oddball·Remote
RemoteSeniorData Engineer
$130k – $160k USD
yesterday
O
NEW

Data Engineer

Oddball·Remote
RemoteMidData Engineer
$100k – $130k USD
yesterday
S
NEW

Staff Data Platform Software Engineer

Servicenow·Santa Clara, CALIFORNIA, United States
On-siteStaffData Engineer
$176k – $308k USD
Apache luceneOpensearchJava
yesterday
SS
NEW

Data Engineering - Assistant Vice President

State Street·Bangalore
On-siteSeniorData Engineer
PythonSQLAlteryx
2d ago
QG
NEW

Data Engineer with expertise in Snowflake Data Engineering

Quest Global·Bengaluru, Karnataka, India
On-siteSeniorData Engineer
SnowflakeData modelingData integration
2d ago
Apply on Inetum →