Evolving quality with AI testing, great for those that have struggled and given up on test automation

A part of me wants to quit my CEO day job and spend time exploring all the AI testing tools. It feels like such an interesting time to explore how to do quality better. If only I had the time. 🫠

Instead, I feel my role is to paint an honest picture through connection, conversations, with the people, the leaders and the companies.

I am feeling optimistic about what the tools can do for us. And yes, there are no magic solutions, and as with all things, there are likely trade offs, but it's interesting to problem solve around.

I've not come to this thinking overnight. My work is to study the quality and tech space and surface what we discover.

Over the years, we were sold test automation as the solution to better testing...which would hopefully lead to better quality. I'm not saying that it has failed, but what I have felt is that the balance of conversation has weighed too much in favour of it. Test automation is great, sometimes. We talk about test automation and the successes, but often what gets lost are those that haven't succeed with it (and the impact that has). We can be made out to be failures for not implementing test automation to some expected standard.

It was interesting having conversations with Leaders in Quality, I kept asking them what does actually test automation look like? And the biggest thing became clear. How test automation happens in reality rarely looks how we talk publicly about it.

I've heard many times, that "yes, that test automation practice seems really interesting, but that's not how we do it."

And it's not saying that those who talk about it are lying, I am definitely not saying that. I feel it's more that it's just one angle of the story. Along the way other things don't get spoken about, for many reasons. I was pleasantly surprised to discover testers often didn't own the test automation, developers did. I was pleasantly surprised to see a whole range of quality practices that weren't spoken about. It was super interesting to see quality roles split with support or product.

The point is, people don't want to talk about what didn't work, or what they've given up. Talks and articles are normally biased towards success stories. But it's important we talk more openly about these things, when we can clearly see that certain practices aren't helpful, then we can come together to problem solve for better ways.

My hope for AI and testing is that we explore this deeper, further, honestly. The models need to be rewritten, and the ones that are going to work are the ones we create for ourselves, for our own systems, not one that is preached down upon us.

What feels clearer by the day, is that everything is changing, it is up to us to create a better quality engineering industry, which i believe we can do. And that perhaps things like "documentation as source of truth" becomes more important than our coding skills. It's certainly interesting to ponder upon.

[The image is for attention and is a Bug painting I made, no AI was used in the making of it]