I
Inetum
Data Engineer

DATA ENGINEER

On-siteSeniorData Engineerposted 1w ago
✦Role summaryAI-generated

The candidate will lead the migration of the existing Hadoop environment to a cloud-native stack using Kubernetes Engine, Spark as a service, and Airflow as a service. They will also design, automate, and test robust data pipelines while maintaining documentation and ensuring data quality.

Skills required

About this role

Hybrid work mode in Madrid. This service could imply a willingness to travel occasionally within Italy.

The main responsibilities are:

  • Migrate the existing Hadoop infrastructure to cloud infrastructure on Kubernetes Engine, COS, Spark as a service, and Airflow as a service.
  • Implement data transformation and quality to ensure data consistency and accuracy. Utilize programming languages such as Scala and SQL and tools like Spark for data transformation and enrichment operations.
  • Set up CI/CD pipelines to automate deployments, unit testing and development management.
  • Write and conduct unit and validation tests to ensure accuracy and integrity of code developed.
  • Automate data pipelines and streamline data ingestion through the implementation of different orchestrators and scheduling processes (Airflow as a Service mainly).
  • Writing technical documentation (specifications, operational documents) to ensure knowledge capitalization.
  • Collaborate with cross-functional teams to understand data requirements and deliver solutions.
  • Foster a culture of continuous learning and improvement within the team.

Qualifications

Good knowledge of:

  • Spark on Scala
  • CI/CD tools (Gitlab, Jenkins…)
  • HDFS and structured databases (SQL)
  • Apache Airflow
  • S3 storage/COS and parquet (and ORC) format
  • Strong knowledge of SQL and NoSQL databases
  • Hadoop

Full understanding of:

  • Oozie
  • Shell script
  • Software Development Lifecycle (SDLC) awareness
  • Agile principles and ceremonies
  • Design effective prompts to leverage Gen AI tools across IT domains (e.g., development, testing, data generation, documentation) during the development stage.

Some knowledge of:

  • Kubernetes containerization
  • Dremio as tool to virtualize data

Optionally/ as a plus:

  • Elasticsearch and Kibana
  • Streaming process (Kafka, event steam…)
  • HVault

Additional Information

BUSINESS AND TRANSVERSAL SKILLS

  • Knowledge of banking industry and processes
  • Problem-Solving and Decision-Making person
  • Business / IT relationship (including IT OPS)
  • Ability to understand, explain and support change
  • Ability to Deliver / Results driven
  • Ability to collaborate / Teamwork with data squads and business teams


LANGUAGES

  • English level C1
  • Optionally Fench
Apply on Inetum →Opens in new tab
score your resume against this role

Similar open roles

G
NEW

Data Platform Engineer

GoTymeX·Ho Chi Minh City, Ho Chi Minh, Vietnam
On-siteSeniorData Engineer
today
O
NEW

Senior Data Engineer

Oddball·Remote
RemoteSeniorData Engineer
$130k – $160k USD
yesterday
O
NEW

Data Engineer

Oddball·Remote
RemoteMidData Engineer
$100k – $130k USD
yesterday
S
NEW

Staff Data Platform Software Engineer

Servicenow·Santa Clara, CALIFORNIA, United States
On-siteStaffData Engineer
$176k – $308k USD
Apache luceneOpensearchJava
yesterday
SS
NEW

Data Engineering - Assistant Vice President

State Street·Bangalore
On-siteSeniorData Engineer
PythonSQLAlteryx
2d ago
QG
NEW

Data Engineer with expertise in Snowflake Data Engineering

Quest Global·Bengaluru, Karnataka, India
On-siteSeniorData Engineer
SnowflakeData modelingData integration
2d ago
Apply on Inetum →