Detecting AI-Generated Text

2023-09-19 11:08

There are no reliable ways to distinguish text written by a human from text written by an large language model.

While some have released tools that purport to detect AI-generated content, none of these have proven to reliably distinguish between AI-generated and human-generated content.

ChatGPT has no "Knowledge" of what content could be AI-generated.

To elaborate on our research into the shortcomings of detectors, one of our key findings was that these tools sometimes suggest that human-written content was generated by AI. Even if these tools could accurately identify AI-generated content, students can make small edits to evade detection.

There is some good research in watermarking LLM-generated text, but the watermarks are not generally robust.

I don't think the detectors are going to win this arms race.

News URL

https://www.schneier.com/blog/archives/2023/09/detecting-ai-generated-text.html

#AI