The bedrock of consistency is gone

I just had a great conversation with Anupam about the testing, the future of what quality means. And of course, how this ties in with the MoTaverse.

There were many things he said that got me thinking, but one thing that stood out was how our past work was very much based on a bedrock of consistency. We knew the expected inputs and outputs. It made verifying the results easy to say if they have passed or failed.

However, now the world is changing because of AI, and increasingly, pass and fail does not capture our work. The output of LLMs could be so varied ,yet determining if it is of quality is a total shift in how we approach our quality work. Whilst this is not necessarily new news, there have certainly been discussions about it, I do feel we should discuss the consequences of it, both good and bad. And because I'm a solutions focused person, I'd personally love to explore how quality work may already be changing on a day to day basis.

With developers, for example, does their work mostly stay the same? They are still producing code, sure with the help of AI, but their role is mostly the same as what it was: producing code. (If I'm simplifying this too much, or I am wrong, please correct me!)

However, when it comes to quality, when the bedrock of consistency is gone, our work changes, potentially massively. We can't just check if things pass or fail, there are so many other angles of quality that we must now explore. How do we adapt to that? How will our work actually change? What new things will be end up doing? What old ways will we discard?

Feel free to drop a comment with any thoughts.

And thank you, Anupam, for your time today and inspiring this post.