OpenAI will begin adding invisible watermarks to eligible text generated by ChatGPT and Codex in the European Union, introducing a machine-readable signal intended to help identify content produced by its artificial intelligence systems.
The company plans to roll out the watermark across eligible ChatGPT and Codex users on all plans in the EU over the coming weeks. The move comes in response to transparency requirements under the European Union's AI Act, which require providers of generative AI systems to make certain AI-generated content identifiable.
OpenAI's technology, called textGrain, does not add a visible label, unusual punctuation or hidden characters to generated text. Instead, it subtly changes how a model selects between possible words or word pieces while producing a response. Across a longer passage, those choices create a statistical pattern that can be detected using specialised tools.
Since the watermark is embedded in the wording itself, it can remain with a passage when the text is copied and pasted. OpenAI said the signal does not identify the person who generated the content, reveal their prompts or provide information about their conversations.
The company, however, has acknowledged limitations with the technology. Detection becomes less reliable when passages are short, heavily edited, paraphrased or translated. In one evaluation cited by OpenAI, replacing 10% of words with synonyms reduced detection from about 92% to 66%.
Short passages, mathematical answers and translated content can also be more difficult to identify reliably. OpenAI has cautioned that the absence of a detectable watermark should not be interpreted as proof that content was written by a human.
Because of the possibility of false positives and missed watermarks, OpenAI is not making its text watermark detector publicly available at launch. Access will initially be provided to approved researchers and expert organisations to evaluate the technology and its responsible use.
The company is also making watermarking available to API customers globally for select models. Unlike the EU rollout for eligible ChatGPT and Codex outputs, API watermarking will be optional and disabled by default. OpenAI is not making text watermarking a global default at launch.
OpenAI said it also plans to make textGrain available as open source, allowing researchers and developers to build on the technology.
The rollout expands OpenAI's broader content provenance work, which already includes mechanisms for identifying supported AI-generated images and audio. For text, the company is taking a phased approach as regulators and AI developers continue to assess how reliably machine-generated writing can be identified after it has been edited or reused.
Disclaimer: This article may include information derived from interviews, press releases, public statements, research, company communications and other publicly available or third-party sources. Such material may be summarised, paraphrased or contextualised for journalistic and editorial purposes. All rights in third-party content remain with their respective owners.