AI Data Engineer (Generative AI & Scalable Data Pipelines)

• Build and optimize ETL pipelines for large-scale AI applications, integrating APIs, web scraping, and real-time data processing.
• Develop and maintain scalable AI data infrastructure using PySpark, Pandas, SQL, and cloud services (Azure, AWS, GCP).
• Implement retrieval-augmented generation (RAG) pipelines using FAISS, ChromaDB, and Pinecone for AI-driven insights.
• Deploy and monitor AI applications using FastAPI, Streamlit, Docker, for real-time performance.
• Work with cross-functional teams to ensure data security, compliance, and AI model reliability in production environments.

Requirements

• Bachelor's or Master's degree in Computer Science, Data Engineering, Artificial Intelligence, or a related field.
• Atleast 2-4 years of experience in data engineering, AI/ML, or cloud-based AI infrastructure.
• Expertise in Python, PySpark, and SQL for data transformation and large-scale processing.
• Experience with cloud platforms (AWS, Azure, GCP) for AI model deployment and data pipeline automation.
• Hands-on experience with vector databases (FAISS, ChromaDB, Pinecone) for efficient data retrieval.
• Proficiency in containerization and orchestration tools like Docker and Kubernetes.
• Strong understanding of retrieval-augmented generation (RAG) and real-time AI model deployment.
• Knowledge of API development and AI service integration using FastAPI and Streamlit/Dash.
• Ability to optimize AI-driven automation processes and ensure model efficiency.
• Strong analytical and problem-solving skills. • Excellent communication and collaboration abilities.