Stability is a system, not a metric: lessons from scaling mobile test automation

Many teams assume more automation leads to better quality, but at scale, this often breaks, resulting in flakiness, instability, and a loss of trust in test results.

In this talk, I’ll share lessons from scaling a large mobile automation ecosystem (~2000 Appium tests) where stability hovered around 40–50% and became a bottleneck for release confidence. The turning point wasn’t improving individual tests, but recognising stability as a system problem.

By rethinking test purpose, reducing unnecessary complexity, and improving failure handling, we shifted focus from fixing tests to fixing the surrounding system, which helped move stability toward ~90%.

This session explores the practical decisions, trade-offs, and lessons behind that shift, and how it changes the way teams approach automation, ownership, and quality at scale.

What you’ll learn:

• Practical strategies for improving automation stability at scale
• Lessons on balancing coverage vs reliability
• Insights into modern quality engineering beyond “write more tests”
• Reflections on how the QE role evolves when scaling complex systems