Quality Engineering Consultant AI and Software Testing

QA Test Consultant We are seeking a highly motivated Quality Engineering QE Consultant with 46 years of experience in software testing automation and emerging AI technologies The ideal candidate will have handson experience testing AI GenAI applications APIs and web applications along with strong expertise in Playwright automation and AI evaluation frameworks This role will focus on ensuring the quality reliability safety and performance of AIpowered solutions by designing comprehensive test strategies implementing automation frameworks validating AI model behavior and establishing evaluation mechanisms for large language model LLM based applications The candidate will collaborate closely with Product Owners Developers Data Scientists and AI Engineers to drive quality throughout the software development lifecycle AI GenAI Testing Design and execute test strategies for AI GenAI applications including LLMpowered solutions AI agents copilots and conversational systems Validate AI outputs for Accuracy Relevance Groundedness Consistency Hallucination detection Toxicity and safety compliance Create and maintain prompt test suites and benchmark datasets Perform functional regression performance and reliability testing for AIenabled applications Define and execute AI evaluation metrics and quality gates Automation Quality Engineering Develop and maintain test automation frameworks using Playwright Automate UI API and endtoend business workflows Create reusable automation assets and testing accelerators Integrate automated tests into CI CD pipelines Support shiftleft testing and quality engineering practices API Testing Design and execute API test cases using tools such as Postman REST Assured Playwright API Testing Validate API contracts authentication authorization error handling and performance Conduct integration testing across distributed systems and thirdparty services AI Evaluation Quality Metrics Establish AI evaluation frameworks and quality scorecards Measure model quality using evaluation techniques such as Precision Recall Relevancy scoring Semantic similarity Groundedness validation Humanintheloop evaluation Analyze AI quality trends and recommend improvements Support Responsible AI and model governance requirements Collaboration Stakeholder Management Work closely with engineering product and AI teams to identify quality risks early Participate in requirement reviews and design discussions Communicate testing progress risks and quality metrics to stakeholders Contribute to QE best practices standards and reusable playbooks Experience 46 years of experience in Quality Engineering Software Testing or Test Automation Handson experience testing AI Generative AI applications Experience with modern web application testing and API testing Technical Skills Strong expertise in Playwright automation Handson experience with API testing and automation Experience validating AI LLMbased applications and AI agents Good understanding of AI evaluation methodologies and testing techniques Familiarity with M365 Copilot OpenAI Azure OpenAI Claude Gemini Agentic AI frameworks Knowledge of test management tools such as Azure DevOps ADO Jira or similar platforms Experience working with CI CD pipelines and DevOps practices Preferred Skills Exposure to Python or TypeScript for automation and AI testing Experience with AI evaluation frameworks such as RAG evaluation Prompt evaluation LLM benchmarking Human feedbackbased evaluation Knowledge of Responsible AI principles AI governance and model risk management Experience testing RetrievalAugmented Generation RAG systems and AI agents Education Bachelors degree in Computer Science Information Technology Engineering or a related discipline Key Competencies Analytical thinking and problemsolving Strong communication and stakeholder management Qualityfirst mindset Continuous learning and innovation Collaboration and teamwork Attention to detail Adaptability to emerging AI technologies Experience Level Senior Level