A Tester's Guide to Testing AI Applications

23rd January 2018
  • Locked
Bill Matthews's profile
Bill Matthews

A Tester's Guide to Testing AI Applications image
Talk Description
Self-driving cars, intelligent digital assistants, making parole decisions, detecting, developing new treatments and drugs, and reviewing legal contracts and documents – the use of Artificial Intelligence (AI) technology is increasingly finding its way into mainstream software development. If you are not currently working on a product or project that includes AI technology, you will likely be within the next five years.

While there are a great many resources covering AI from a development perspective, there are few that deal with testing AI-based software despite the challenges and risks this poses. For example:

 How do we test software… 
  • When the decision logic is not clearly defined?
  • When the results may be wrong, and that’s OK sometimes
  • That learns and adapts based on interactions
  • To find the problems that will matter when the input domain is complex and massive
  • That may need to collaborate or compete with other AI software.

These are just some of the challenges of testing AI-based software and while traditional software test design ideas can help but are generally not enough to explore the capabilities of AI-Based software.

In this Masterclass, Bill will take you beyond the AI hype and give you a pragmatic and useful perspective of what AI is. You'll then be shown how testing an AI-based application differs from more traditional applications and explore some strategies to help you tackle testing of AI-based applications.

By the end of this session, you'll be able to:

  • Define what AI is
  • Compare how testing AI-based applications differs from more traditional applications
  • List key risks that are unique to AI-based applications
  • Use unique strategies to test AI-based applications

Bill Matthews has been a freelance test consultant for over 20 years working mainly on complex integration and migration as a Test Architect and as a Technical Lead. He champions the use of modern and effective approaches to development and testing.

He is a regular contributor to the testing community at both local and international levels through conference speaking, coaching/mentoring and delivering workshops and training focusing on automation, performance, reliability, security testing and more recently artificial intelligence.

Bill Matthews has been a freelance test consultant for over 20 years working mainly on complex integration and migration as a Test Architect and as a Technical Lead. He champions the use of modern and effective approaches to development and testing.

He is a regular contributor to the testing community at both local and international levels through conference speaking, coaching/mentoring and delivering workshops and training focusing on automation, performance, reliability, security testing and more recently artificial intelligence.

Suggested Content
TestBash Brighton 2025 image
On the 1st & 2nd of October 2025 we're back in Brighton for TestBash: the largest software testing conference in the UK
Explore MoT
Castelo Branco Meetup image
Tue, 6 May
The Future of Testing in an Automated World: Embracing Continuous Learning and A
Beginner's Guide To Mobile Testing image
Begin your testing journey in mobile by learning the different aspects of testing mobile
Leading with Quality
A one-day educational experience to help business lead with expanding quality engineering and testing practices.
This Week in Testing image
Debrief the week in Testing via a community radio show hosted by Simon Tomes and members of the community
Subscribe to our newsletter
We'll keep you up to date on all the testing trends.