LLM Ops Engineer

Job Description We are hiring an LLM Ops Engineer to join our AI Research team, a highly technical group working on cutting-edge advancements in the AI industry. The team focuses on building scalable, production-grade LLM systems, fine-tuning strategies, evaluation frameworks, and next-generation deployment architectures. This role requires hands-on experience operating LLMs beyond simple API integration. The ideal candidate understands the architectural, operational, and evaluation complexities that differentiate LLMOps from traditional MLOps. Responsibilities:- - Manage the end-to-end lifecycle of LLMs: registry, packaging, versioning, deployment, monitoring, and rollback. - Deploy and operate self-hosted / open-source LLMs (not limited to OpenAI API usage). - Design and manage scalable inference infrastructure, including GPU-aware deployments. - Implement CI/CD pipelines for LLM deployment and continuous evaluation. - Monitor system performance including latency, throughput, token usage, cost, drift (model and data), and hallucinations. - Ensure secure, compliant, and resilient cloud-based model deployments. - Collaborate with research and engineering for deployments. Skills:- - Strong hands-on experience with LLM handling, hosting, and operationalization. - Clear understanding of how LLMOps differs from traditional MLOps (prompt management, non-deterministic outputs, semantic evaluation, token economics, guardrails etc.). - Experience with Kubernetes, Docker, and containerized deployments. - Cloud expertise (AWS / Azure / GCP) including compute, storage, IAM, networking, and monitoring. - Experience building scalable inference and model-serving architectures. - Familiarity with tools such as MLflow, Kubeflow etc. (good to have). - Understanding of vector databases, RAG systems, and evaluation frameworks (preferred). - Knowledge of GenAI security considerations (prompt injection, data leakage prevention).