How do you test that something is AI generated?

22 May 2025

Create Moment
One man is presenting on a large stage. Behind him, there is a huge screen. A host is standing to the right of the stage.
In this moment: Jarsto van Santen
"There are hints. There are signatures. There are probabilities that are unique to this set of data. The same thing goes for something like ChatGPT. For the different GPT variants, they have statistical correlations which you can work out probability-based. The idea was that in post-processing, we force the AI to prefer certain synonyms that humans tend not to prefer. So you generate a fingerprint which says — and again this is probabilistic — if you have these choices in language, if you have these coincidences, these words linking up in certain frequencies, that makes it very likely this was AI generated."

Brighton, September 2024 at TestBash. Jarsto van Santen answers a question from the audience after introducing how Generative AI works.

Talk available at: https://www.ministryoftesting.com/testbash-sessions/how-generative-ai-works-a-very-rough-guide
Simon Tomes
Community Lead at Ministry of Testing
he/him

Hello, I'm Simon. Since 2003 I've had various roles in testing, tech leadership and coaching. I believe in the power of collaboration, creativity and community. 🎓 MoT-STEC qualified.

MoTaverse Team
Sign in to comment
Explore MoT
Thunders Connect image
Tue, 17 Mar
Thunders Connect – Mar 17, Station F, Paris: QA & testing future, certification, product updates, panel, Station F tour & first-ever Thunders Awards. RSVP if in Paris!
Advanced prompting for testers image
Advanced prompting skills to turn AI into your trusted testing companion.
This Week in Quality image
Debrief the week in Quality via a community radio show hosted by Simon Tomes and members of the community
Subscribe to our newsletter
We'll keep you up to date on all the testing trends.