We love a good TWiQ episode that celebrates the wins. And in today's episode we did just that. So here goes, we celebrated the following wins... Diana Dromey ran her first ever 10k, goin...

A celebratory 'wins' episode full of community milestones, personal breakthroughs, bug catches, career confidence, and the many small moments that keep the MoTaverse moving.

A hot-takes episode on AI dependency, community knowledge sharing, silos, and why exploratory testing matters more than ever as software teams move faster.

Upon my AMA about how baking a cake is the same as developping software moment Simon Tomes asked "While the recipe instructions and ingredients are consistent, how is it that the quality outcome of...

Upon my AMA about how baking a cake is the same as developping software moment Gary Hawkes asked "How do the teams responsibilities in a bakery, align with a quality engineering team?". Immediately...

Behaviour where the same input can produce different outputs across multiple runs. LLMs exhibit this by design, requiring testers to shift from exact-match checks to presence/absence assertions. So what? Non-determinism means a regression test can "fail" without the answer being wrong — a fundamental difference from testing traditional software. Example: Running the same summarisation prompt twice may return the same meaning in different words. Editorial note: This definition has been inferred from how the term was used in the source material. "The maximum time you expect your application to take to respond to a specific request or action." — Demi Van Malcot

A reference point used to decide whether a test has passed or failed. For LLM testing this becomes unreliable because multiple valid outputs can exist for the same input. So what? The absence of a stable oracle is one of the central challenges of AI testing. Techniques like metamorphic testing exist partly to work around it by checking consistency rather than correctness. Example: "Who was the first president of the USA?" has a clear oracle. A summarisation request does not. "The 'expected result' can be determined, but will always have some ambiguity. Comparing it to the 'actual result' won't be as straightforward as you are used to." — Demi Van Malcot

A machine learning technique that uses layered neural networks to find patterns across large volumes of data. LLMs are built on deep learning to make connections across billions of words and generate contextually relevant responses. So what? LLMs work by statistical pattern-matching rather than reasoning — a foundational insight for anyone designing tests. Example: An LLM predicts the most statistically likely next word or phrase, not the most factually accurate one. "They are trained on billions of words from different sources. Using deep learning, they make connections between all the words they are trained on to answer whatever questions we ask of them." — Demi Van Malcot

A category of AI model that produces new content: text, images, code, or other outputs, in response to input prompts, rather than returning a fixed or pre-programmed answer. LLMs are the most widely used type of generative AI. So what? Because outputs are generated fresh each time, the testing approaches used for traditional deterministic software don't transfer cleanly. Concepts like "expected result" and pass/fail need to be rethought. Example: ChatGPT, Claude, Bard, and Copilot are all generative AI applications built on large language models. "It won't give an answer based on what is logically correct, but on what is statistically most likely. The sentences can be completely correct, while the answer is completely wrong." — Demi Van Malcot

Learn how to evaluate LLM quality and limitations using a range of testing techniques, from unit and regression testing to bias, adversarial and explainability testing.

We have recreated the import screens users need to fill in for declaration in our test environment. Although I appreciate the devs wanting to make it as close to real as possible, I have to check t...

In TWiQ today, Aj raised the idea of self-service infrastructure and it got me thinking about how that is related to platform engineering. Something I realise I don't know enough about, but a recen...