How do you test that something is AI generated?

May 22, 2025

Create Memory
One man is presenting on a large stage. Behind him, there is a huge screen. A host is standing to the right of the stage.
In this memory: Jarsto van Santen

"There are hints. There are signatures. There are probabilities that are unique to this set of data. The same thing goes for something like ChatGPT. For the different GPT variants, they have statistical correlations which you can work out probability-based. The idea was that in post-processing, we force the AI to prefer certain synonyms that humans tend not to prefer. So you generate a fingerprint which says β€” and again this is probabilistic β€” if you have these choices in language, if you have these coincidences, these words linking up in certain frequencies, that makes it very likely this was AI generated."

Brighton, September 2024 at TestBash. Jarsto van Santen answers a question from the audience after introducing how Generative AI works.

Talk available at: https://www.ministryoftesting.com/testbash-sessions/how-generative-ai-works-a-very-rough-guide

Community Lead at Ministry of Testing
he/him
Hello, I'm Simon. Since 2003 I've had various roles in testing, tech leadership and coaching. I believe in the power of collaboration, creativity and community. πŸŽ“ MoT-STEC qualified.
Sign in to comment
Explore MoT
MoTaCon 2026 image
Thu, 1 Oct 2026
Previously known as TestBash, MoTaCon is the new name for our annual conference. It's where quality people gather.
Advanced prompting for testers image
Advanced prompting skills to turn AI into your trusted testing companion.
This Week in Quality image
Debrief the week in Quality via a community radio show hosted by Simon Tomes and members of the community
Subscribe to our newsletter
We'll keep you up to date on all the testing trends.