Explore all our content
Dive into our back catalogue of content. With over 2000 pieces of content to digest, there will definitely be some software testing content of interest to you.
Searching...
I'm coming from the Netherlands
I'm a quality engineer for Dutch Railways
I'm coming to TestBash as a host!
I'm super interested in and w...
Building trustworthy AI requires a partnership between the two
You've heard the cliché, it's not what you know, it's who you know. When people see you as a leader and look to you for answers, you can'...
Can you find Bug, Space Duck and Space Seagull?
About me:
I'm coming from: Bury St Edmunds, Suffolk, UK
My role is: QA Lead
I'd love to meet others who are into: Leadership and anyone i...
About Me:
I’m coming from: Cambridge, UK
My role is: Quality and Test Engineer
I’d love to meet others who are into: Testing AI systems |...
![A screenshot from the paper: The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity
Showing a composite figure illustrating how reasoning models solve the Tower of Hanoi problem, and how their performance varies with problem complexity.
Top Section – LLM Response Workflow:
On the left is a code-like LLM response showing a section with a list of disk moves (e.g., [1, 0, 2], [2, 0, 1], etc.) and a section referencing the final moves list. Arrows indicate:
Moves are extracted from the section for analysis.
Final answer is extracted from the section for measuring accuracy.
To the right, a sequence of three Tower of Hanoi diagrams represents:
Initial State: All disks stacked on peg 0.
Middle State: Disks distributed across pegs.
Target State: All disks correctly stacked on peg 2.
Each disk is color-coded and numbered for clarity.
Bottom Row – Three Line Graphs:
Left Graph: Accuracy vs. Complexity
Y-axis: Accuracy (%)
X-axis: Problem complexity (number of disks, from 1 to 20)
Two lines: Claude 3.7 (red circles) and Claude 3.7 with “thinking” mode (blue triangles).
Accuracy drops sharply for both as disk number increases, with “thinking” performing slightly better up to 8 disks.
Middle Graph: Response Length vs. Complexity
Y-axis: Token count
X-axis: Number of disks
“Thinking” responses grow rapidly in length with complexity, peaking near 8 disks.
Right Graph: Position of Error in Thought Process
Y-axis: Normalized position in the LLM’s reasoning (0 to 1)
X-axis: Complexity (1 to 15 disks)
Shows where correct vs. incorrect reasoning paths diverge; incorrect solutions typically fail earlier in the thoughts.
Background colors across all graphs denote complexity bands: yellow (easy), blue (moderate), red (hard).](https://d8iqbmvu05s9c.cloudfront.net/fl6kouav8tlcxxgcdr19foooovrt)
Apple just tested the smartest "reasoning" AI Models out there: Claude 3.7 Sonnet, DeepSeek-R1, OpenAI’s o1/o3.
The verdict?
They didn’t...
A few things I’m proud of:
🧪 I break things for a living (so they don’t break in the wild).
📍 Based in Jaipur, India’s Pink City, where ...
About Me:
I’m coming from: Surrey/London, England
My role is: Lead Quality Engineer
I’d love to meet others who are into: quality engi...
What impact can following people in the MoTaverse have on your career and those around you? Read a roundup of news, events and actionable ideas in this week's MoT Weekly.
You can imagine my delight when I spotted this bloke in my local Waterstones bookstore.
"I'm a tester, can I get a photo of your t-shirt...
We’re thrilled to welcome Postman as a Gold Sponsor for TestBash Brighton 2025!
A leader in API collaboration, Postman helps testers an...
Elizabeth Zagroba shares what it really looks like to be a Quality Lead.
See the work behind the title, how to make your impact visible...
A reminder that one size does not fit all and that time moves on
About Me:
I’m coming from: Bristol, UK
My role is: QA Professional
I’d love to meet others who are into: Passionate about Software testi...
On the 1st & 2nd of October 2025 we're back in Brighton for TestBash: the largest software testing conference in the UK
Explore MoT
Wed, 17 Sep
with Ivan Fillipov, Solution Architect at Xray
Boost your career in software testing with the MoT Software Testing Essentials Certificate. Learn essential skills, from basic testing techniques to advanced risk analysis, crafted by industry experts.
A one-day educational experience to help business lead with expanding quality engineering and testing practices.
Debrief the week in Testing via a community radio show hosted by Simon Tomes and members of the community
