bt_bb_section_bottom_section_coverage_image

OpenAI won’t watermark ChatGPT text because its users could get caught

OpenAI has had a text watermarking system created by ChatGPT (watermark ChatGPT) and a tool to detect watermarks for about a year, the Wall Street Journal reports. But the company is divided internally over whether to release it. On one hand, it seems like the responsible thing to do; on the other, it could hurt its financial results.

OpenAI’s watermarking is described as a tweak to how the model predicts the most likely words and phrases after previous words and phrases, creating a detectable pattern. (This is a simplification, but you can check out Google’s in-depth explanation of the Gemini text watermark to learn more.)

Providing a way to detect AI-generated content is a potential boon for teachers trying to discourage students from assigning homework to AI. The Journal reports that the company found that watermarks did not affect the quality of the chatbot’s text rendering. “In a survey commissioned by the company, people around the world supported the idea of ​​an AI detection tool by a margin of 4:1,” the Journal writes.

After the Journal published its paper, OpenAI confirmed that it was working on text watermarking in a blog post update today spotted by TechCrunch. In the post, the company claims that its method is highly accurate (“99.9% effective,” according to documents the Journal referenced) and resistant to “changes such as paraphrasing.” But techniques like rephrasing with a different model allow bad actors to bypass “the mundane,” she said. The company also said it was concerned about stigmatizing the usefulness of AI tools for non-native speakers.

But OpenAI also seems concerned that using watermarks could turn off ChatGPT users it surveyed, nearly 30% of whom apparently told the company they would use the software less if tattoos were applied.

Some employees still believe tattoos are effective, though. Given the lingering sentiment among users, the Journal says some have suggested trying methods that “may be less controversial among users but are unproven.” In an update to its blog post today, the company said it’s “in the early stages” of exploring the integration of metadata. It’s still “too early” to know how well it will work, she said, but since it’s cryptographically signed, there shouldn’t be any false positives.

Share
× WhatsApp