The great thing about humanity is that we love to talk.

The terrible thing about humanity is that we love to talk.

Yes, I say that in jest, but we must apply our critical thinking skills to industry chatter about test automation and AI.

In 2025, I did 20+ recorded conversations with leaders in quality. At the back of my mind, I was curious as to whether test automation was all it cracked up to be. So I asked the question, time and time again. The responses varied so much, it validated what I had felt for years: so much talk about test automation, and the reality doesn't quite match up to it.

The stories I heard varied so much, from devs doing the test automation, to a collaborative approach, to not doing any at all, and even trying and failing so many times to implement any kind of proper test automation.

Melissa Fisher says we have an obsession with test automation, and I agree:

On a similar topic, Jason Gorman of Codemanship in his blog post 'Why Almost Nothing You Read Online About AI Is True' created a Social Transmission Accumulated Accuracy Risk  theory. So much talk. Not enough fact-checking, but also not enough real-world practical implementation.

The Jason's theory of Social Transmission Accumulated Accuracy Risk (STAAR) posits that almost nothing you read about AI online is true because initial inaccuracies rapidly compound and propagate across social networks as influencers repeatedly extrapolate and share claims without fact-checking them.

This is not me dissing on test automation. But I think that we are falling into the same trap with AI. It's all about AI now. Everyone is doing. Everyone must have the skills. Agents everywhere. And all of this has an impact. The most obvious is that these skills become a 'necessary thing' in job descriptions. But there are also knock-on effects, like mismatched expectations when we actually show up to do the work.

We must remind ourselves this is not the reality it seems. Under the hood, how we show up every day to our work, it looks different, for most of us. And so much of the time we cannot talk publicly about all the challenges we truly face.

AI is not the point. Test automation is not the point. Getting attention for a great idea or model is not the point either. Building a good product is. And most of the time, the only way that can be taught is through doing the work.

What does test automation, or AI, actually look like where you work? Tell me in the comments