How do you test that something is AI generated?

"There are hints. There are signatures. There are probabilities that are unique to this set of data. The same thing goes for something like ChatGPT. For the different GPT variants, they have statistical correlations which you can work out probability-based. The idea was that in post-processing, we force the AI to prefer certain synonyms that humans tend not to prefer. So you generate a fingerprint which says — and again this is probabilistic — if you have these choices in language, if you have these coincidences, these words linking up in certain frequencies, that makes it very likely this was AI generated."

Brighton, September 2024 at TestBash. Jarsto van Santen answers a question from the audience after introducing how Generative AI works.

Talk available at: https://www.ministryoftesting.com/testbash-sessions/how-generative-ai-works-a-very-rough-guide