Data Engineer
This role focuses on designing scalable data platforms for GenAI applications, requiring expertise in building and maintaining high-performance batch/streaming pipelines while administering databases and ensuring data reliability for retrieval and chatbot workloads. The candidate will own end-to-end data infrastructure, including embeddings and vector data, with ownership over cloud-agnostic systems and platform performance.
Skills required
About this role
Job Summary
We are seeking an IC3 Data Engineer (5-8 years) to design, build, and operate the data platforms that power our GenAI applications. This is a hands-on, pureplay data engineering role with strong database and systems administration ownership: you will build and tune batch and streaming pipelines, administer databases and jobs, and keep the platform reliable and performant. You will supply clean, governed data - including embeddings and vector data - to retrieval, chatbot, and agent workloads. Our stack is cloud-agnostic; experience with any major cloud is welcome.
Job Requirements
- 5-8 years in data engineering, platform engineering, or analytics engineering.
- Strong SQL and hands-on database/query performance optimization (OLTP and OLAP).
- Strong Python, Spark/PySpark, ETL/ELT, Kafka streaming, and multi-source data integration.
- Database administration: access control, backups, replication, high availability, and job management.
- Systems administration: Linux, OS patching, certificates, monitoring, backup, and disaster recovery.
- Hands-on MongoDB; familiarity with Cassandra and vector databases (Milvus, Qdrant, or pgvector).
- Workflow orchestration (Apache Airflow or similar); data modeling and warehousing.
- Power BI (Power Query, DAX) and Git.
Education
- Bachelor's in Computer Science or a related field (or equivalent experience).
- 5-8 years of professional experience in data engineering and data platform operations.
