IO
Institute of Foundation Models
MLOps

Senior MLOps Engineer

On-siteSeniorMLOpsposted 10mo ago
Role summaryAI-generated

You will design and maintain the cloud, orchestration, and deployment layers that power large-scale AI systems, working closely with researchers to productionize LLMs, voice, and multimodal models at MBZUAI. The role requires deep expertise in Kubernetes, Docker, and CI/CD pipelines to ensure scalable, secure, and reliable model serving at scale.

Skills required

About this role

About the Institute of Foundation Models (IFM)

The Institute of Foundation Models is a dedicated research lab for building, understanding, deploying, and risk-managing large-scale AI systems. We drive innovation in foundation models and their operationalization, empowering research, education, and industry adoption through scalable infrastructure and real-world applications.

As part of our engineering team, you will operate at the intersection of machine learning and systems design — building the cloud, orchestration, and deployment layers that power the next generation of intelligent applications at MBZUAI. You’ll work alongside world-class AI researchers and engineers to productionize LLMs, voice models, and multimodal systems at scale.

The Role

As a Senior MLOps Engineer, you will design, build, and maintain robust ML(Machine Learning) infrastructure across training, inference, and deployment pipelines. You will take ownership of the model lifecycle — from data ingestion to real-time serving — and ensure our LLM and speech models are deployed efficiently, securely, and reproducibly in Kubernetes-based environments.

This position requires deep hands-on experience with Kubernetes (EKS), Helm, AWS cloud infrastructure, and modern MLOps toolchains (e.g., vLLM, SGLang, OpenWebUI, Weights & Biases, MLflow). Familiarity with speech/voice AI frameworks like ElevenLabs, Whisper, and RVC is also valuable.

Key Responsibilities

  • Design and manage scalable ML infrastructure on AWS using EKS, EC2, RDS, S3, and IAM-based access control.
  • Build and maintain Kubernetes deployments for LLM and TTS inference using Helm, ArgoCD, and Prometheus/Grafana monitoring.
  • Implement and optimize model serving pipelines using vLLM, SGLang, TensorRT, or similar frameworks for high-throughput inference.
  • Develop CI/CD and MLOps automation for data versioning, model validation, and deployment (GitHub Actions, Jenkins, or AWS CodePipeline).
  • Integrate OpenWebUI, Gradio, or similar UIs for user-facing model demos and internal evaluation tools.
  • Collaborate with ML researchers to productize models — including TTS (e.g., ElevenLabs API), ASR (Whisper), and LLM-based chat systems.
  • Ensure observability, cost optimization, and reliability of cloud resources across multiple environments.
  • Contribute to internal tools for dataset curation, model monitoring, and retraining pipelines.
  • Maintain infrastructure-as-code using Terraform and Helm charts for reproducibility and governance.
  • Support real-time multimodal workloads (voice, text, vision) across inference clusters.
  • Academic Qualifications

  • 4+ years of experience in MLOps, DevOps, or Cloud Infrastructure Engineering for ML systems.
  • Strong proficiency in Kubernetes, Helm, and container orchestration.
  • Experience deploying ML models via vLLM, SGLang, TensorRT, or Ray Serve.
  • Proficiency with AWS services (EKS, EC2, S3, RDS, CloudWatch, IAM).
  • Solid experience with Python, Docker, Git, and CI/CD pipelines.
  • Strong understanding of model lifecycle management, data pipelines, and observability tools (Grafana, Prometheus, Loki).
  • Excellent collaboration skills with ML researchers and software engineers.
  • Professional Experience – Preferred

  • Extensive Experience with vLLM, K8s, Elevenlabs, Whisper, Gradio/OpenWebUI, or custom TTS/ASR model hosting.
  • Familiarity with multi-GPU scheduling, NCCL optimization, and HPC cluster integration.
  • Knowledge of security, cost management, and network policy in multi-tenant Kubernetes clusters and cloudflare systems.
  • Prior work in LLM deployment, fine-tuning pipelines, or foundation model research.
  • Exposure to data governance and responsible AI operations in research or enterprise settings.
  • Similar open roles

    M
    NEW

    Senior Engineer, ML Data Services

    Motional·Boston, Massachusetts, United States; Las Vegas, Nevada, United States; Pittsburgh, Pennsylvania, United States; Remote U.S.
    HybridSeniorMLOps
    $149k – $199k USD
    2d ago
    Z
    NEW

    ML Infrastructure Engineer

    Zipline·South San Francisco, California, USA
    On-siteMidMLOps
    $160k – $250k USD
    2d ago
    C
    NEW

    Software Engineer, Inference AI/ML

    CoreWeave· Sunnyvale, CA / Bellevue, WA
    HybridMidMLOps
    $92k – $135k USD
    2d ago
    SK
    Sponsored

    Land 5x More Interviews - Resume & Strategy

    Shaqeeq Khan·Built this board, coached engineers from Netflix, Google, IBM, Amazon. 100+ grads placed.
    CV ReviewLive One on One CallsStrategy
    Book A Call