Paged Out! Institute

"𝗜𝘁 𝘀𝗲𝗲𝗺𝘀 𝘁𝗵𝗮𝘁 𝗔𝗜 𝗶𝘀 𝗹𝗶𝗺𝗶𝘁𝗶𝗻𝗴 𝗵𝘂𝗺𝗮𝗻 𝗲𝘅𝗽𝗿𝗲𝘀𝘀𝗶𝗼𝗻 𝗷𝘂𝘀𝘁 𝗯𝘆 𝘁𝗵𝗲 𝘀𝗵𝗲𝗲𝗿 𝗳𝗮𝗰𝘁 𝗶𝘁 𝗲𝘅𝗶𝘀𝘁𝘀" The problem we're currently tackling at Paged Out! magazine is the... well, the same one as every other publication media—what to do with AI generated texts. However, after digging into this problem a bit, I'm not sure if the problem is even phrased correctly. On the surface level the problem seems simple: "if an article is generated with AI, reject the article". And then the real world kicks in with its typical chaotic nature making the problem complicated. Let's start with the problem of detecting AI text. First of all, I think we can agree that generic AI detectors are untrustworthy—we've all heard enough stories about AI detectors saying that texts written decades or centuries before ChatGPT existed were "written by AI". Just recently, some vendors like Anthropic started introducing the long discussed statistical watermarks into the text it generates, but these do have their own shortcomings (as documented in the Limitations section on Claude's support page). And there's still the problem of open models, as well as vendors who care little about the EU AI Act. What about AI text detection by humans? I mean, a lot of us have this "spider sense" kicking in when we read AI slop that tells us "ooh, this reads like it's been generated by AI". Apparently there are studies which say that heavy LLM users fare pretty well when classifying whether a text was written by a human or AI—up to 90% accuracy. This accuracy is down to a coin flip for regular folks though. But wait... what is it that actually triggers us? Most of the time it's the typical AI-favorite patterns like the classic "It's not X, it's Y". It's also the usage of correct punctuation marks like — or “ ” instead of - and " ". The problem with the last one is pretty obvious—anyone who ever worked with a professional editor to get their text into good shape has a habit of using the correct punctuation marks anyway, so their texts get misclassified as AI generated slop (FWIW: most keyboard layouts have "—" under AltGr+Shift+Minus; and if your doesn't, you can always modify it, e.g. using Microsoft MSKLC on Windows). Going back to typical patterns, here's the deeper problem: our human language is flexible and we do change our speech (writing) patterns from time to time. Ever hung out with a new friend group for some time only to realize that some of their catch phrases started to show up in the way you speak? Ever noticed you've started using some new slang words you've read online? Or heard your favorite youtuber use? Well, this is also true for AI generated text (chats, articles, etc)—the more of these we read, the more exposed to certain patterns we are... and the more we start writing like AIs ourselves. <continued in the comments>