Spoke at a Ministry of Testing meetup
Ministry of Testing Coventry
Thursday, May 8, 2025
Bill Matthews
• Owner & Consultant - Target Testing Ltd
• Technical Lead - Voy Finance
• Strategic Business Consultant
Thursday, May 8, 2025
Bill Matthews
• Owner & Consultant - Target Testing Ltd
• Technical Lead - Voy Finance
• Strategic Business Consultant
About the talk:
As Large Language Models (LLMs) become integral to modern applications, software testers must adapt their strategies to assess these AI-driven systems effectively. This talk explores the unique challenges of testing LLM-based applications, including their probabilistic nature, hallucinations, bias, and prompt sensitivity.
As Large Language Models (LLMs) become integral to modern applications, software testers must adapt their strategies to assess these AI-driven systems effectively. This talk explores the unique challenges of testing LLM-based applications, including their probabilistic nature, hallucinations, bias, and prompt sensitivity.
We will discuss key capabilities and risks that testers should focus on, methodologies for designing robust evaluation frameworks, and automation techniques to streamline testing. Attendees will also learn how to leverage LLMs as judges for qualitative assessments and explore tools that support scalable, systematic testing. Whether you're new to AI testing or looking to enhance your approach, this session will equip you with practical insights to ensure the reliability and fairness of LLM-based applications.
