In this episode of Software Testing Live, Ben Dowen is joined by Principal Quality Engineer Emily O’Connor for a live session exploring the balance between automation and human review in modern software testing.
As AI-powered coding and test generation tools become more widespread, the session focuses on why testers must review everything, even when it’s generated automatically, and avoid the trap of automating everything by default.
Ben and Emily explore a ticketing application in real time, combining exploratory testing with AI-generated test planning and writing using Playwright MCP. Along the way, they uncover usability issues, accessibility concerns, and gaps in functionality, while also demonstrating how AI can quickly produce overly complex or misaligned test plans if left unchecked.
The session highlights the importance of critical thinking, context awareness, and intentional test design, showing that while AI can accelerate test creation, it often requires refinement to ensure tests are meaningful, maintainable, and aligned with business value.
You’ll learn how to:
- Balance exploratory testing and automation to focus on what really matters to users and the business
- Decide what should and should not be automated, avoiding low-value or overly granular tests
- Critically review AI-generated test plans and identify unnecessary complexity or incorrect assumptions
- Apply good test design principles (Arrange–Act–Assert, small focused tests, clear intent)
- Recognise common pitfalls of AI-generated automation, including over-generation and hidden assumptions
- Evaluate usability, accessibility, and missing features during exploratory testing
- Ensure automated tests remain maintainable, readable, and valuable within a broader test strategy
- Understand why human review is more important than ever in AI-assisted development workflows
Comments