Staff Technical Product Manager – AI Quality and Evaluation

Job Description We are looking for a highly technical, strategic, and outcome-oriented Staff Technical Product Manager to establish and scale AI Evaluation as a core engineering capability across the DT organization. The Eval Technical Product Manager will lead this transformation by making trusted Evals the evidence layer for AI. This is a dual-role position with responsibility for both: - Platform Eval Strategy & Adoption Partner closely with Servicenow AI platform and product teams to understand, implement, validate, and drive adoption of out-of-the-box Evals, while influencing the platform roadmap based on enterprise needs. - Enterprise Eval Innovation Identify gaps in platform evaluation capabilities and drive the definition and development of custom Evals, metrics, rubrics, datasets, and evaluation strategies for enterprise-specific AI applications and agentic workflows. The role will establish Eval-Driven Development as a standard engineering practice—moving AI quality from point-in-time testing to continuous measurement throughout the AI lifecycle - Custom Eval Innovation - Identify gaps where OOTB Evals don't adequately measure enterprise AI quality. - Define and drive custom metrics, rubrics, datasets and evaluation strategies. - Develop reusable Evals for domain-specific behaviors and business outcomes. - Establish evaluation approaches for AI agents, task completion, tool usage and multi-step workflows. - Eval-Driven Development - Embed Evals into the full AI lifecycle: Define → Measure → Build → Evaluate → Improve → Release → Monitor - Require AI teams to define expected behavior and measurable quality criteria. - Establish evaluation coverage and quality thresholds. - Integrate trusted Evals into development, regression and release workflows. - Establish Eval-based quality gates for appropriate AI applications. - Enable continuous evaluation in production.