It may soon become common to encounter a tweet, essay or news article and wonder if it was written by artificial intelligence software. There could be questions over the authorship of a given piece of writing, like in academic settings, or the veracity of its content, in the case of an article. There could also be questions about authenticity: If a misleading idea suddenly appears in posts across the internet, is it spreading organically, or have the posts been generated by A.I. to create the appearance of real traction?
Tools to identify whether a piece of text was written by A.I. have started to emerge in recent months, including one created by OpenAI, the company behind ChatGPT. That tool uses an A.I. model trained to spot differences between generated and human-written text. When OpenAI tested the tool, it correctly identified A.I. text in only about half of the generated writing samples it analyzed. The company said at the time that it had released the experimental detector “to get feedback on whether imperfect tools like this one are useful.”
Identifying generated text, experts say, is becoming increasingly difficult as software like ChatGPT continues to advance and turns out text that is more convincingly human. OpenAI is now experimenting with a technology that would insert special words into the text that ChatGPT generates, making it easier to detect later. The technique is known as watermarking.
The watermarking method that OpenAI is exploring is similar to one described in a recent paper by researchers at the University of Maryland, said Jan Leike, the head of alignment at OpenAI. Here is how it works.
r.json()) ]).then(function(data) { var target = document.querySelector(‘[data-component=”preview-2″]’); var url = NYTG.ASSETS + ‘/build/js/scroller.js’; var props = { amp: false, exampleText: data[0], graphicId: “bottom”, items: [{“text”:”Imagine a list of every word you know, every unique word you might use when writing an essay, email or text message.”,”position”:”max”,”promptName”:”allwords”,”icon”:”Human”,”prevPos”:0,”isMax”:true,”pos”:378,”responseSoFar”:””,”responseToEnd”:”work she pages program both know related are date how listing show rating car that site before life service three current had search details press mail compare around found house want great each text prices item he not place another code help video map index first long reply data health project us best day art money any course games contact those take terms should get where pictures post buy now am product group local per movies software quality check as children for then care website many research some which message using may shopping than print list after type next time all into times by member there business read their visit big process jan subject well source his form it december were think order photo same end down so case is at version because pm during will college based love article canada black home old her country could shop center if page use january a previous support on to was comments forum states family this would between free posts him usa rights special section very content said view training add report return since do music results link policy private have state here travel city guide control online we be books my you go up reserved number north value way game office been address computer when more its line phone point change our history must review members real today profile american media who class access in students web email network learn internet water under news what personal these own your other being shipping sales property file credit book and systems but two area sign hotel made open company image has name science items click hours john small still can school days did size send sports national also system stock info even they links with services security shall digital make reviews advanced good without title need me dvd download user top posted through years no high county only from right board public united women or tools set most out last people privacy rate account little sites law ebay used about them year find full store price like include of world general white back new please provide location see one products does south part over while just much off level the design hotels author customer power every events within an such total”,”responseFromLast”:”work she pages program both know related are date how listing show rating car that site before life service three current had search details press mail compare around found house want great each text prices item he not place another code help video map index first long reply data health project us best day art money any course games contact those take terms should get where pictures post buy now am product group local per movies software quality check as children for then care website many research some which message using may shopping than print list after type next time all into times by member there business read their visit big process jan subject well source his form it december were think order photo same end down so case is at version because pm during will college based love article canada black home old her country could shop center if page use january a previous support on to was comments forum states family this would between free posts him usa rights special section very content said view training add report return since do music results link policy private have state here travel city guide control online we be books my you go up reserved number north value way game office been address computer when more its line phone point change our history must review members real today profile american media who class access in students web email network learn internet water under news what personal these own your other being shipping sales property file credit book and systems but two area sign hotel made open company image has name science items click hours john small still can school days did size send sports national also system stock info even they links with services security shall digital make reviews advanced good without title need me dvd download user top posted through years no high county only from right board public united women or tools set most out last people privacy rate account little sites law ebay used about them year find full store price like include of world general white back new please provide location see one products does south part over while just much off level the design hotels author customer power every events within an such total”,”nextToken”:null,”alternatives”:null,”wAlts”:null,”matchIndex”:null,”watermarkWords”:null,”allWatermarkWords”:[]},{“text”:”Now imagine that half of those words are on a special list.”,”position”:”max”,”promptName”:”allwords”,”greenList”:”0.5″,”icon”:”Human”,”prevPos”:378,”isMax”:true,”pos”:378,”responseSoFar”:”work she pages program both know related are date how listing show rating car that site before life service three current had search details press mail compare around found house want great each text prices item he not place another code help video map index first long reply data health project us best day art money any course games contact those take terms should get where pictures post buy now am product group local per movies software quality check as children for then care website many research some which message using may shopping than print list after type next time all into times by member there business read their visit big process jan subject well source his form it december were think order photo same end down so case is at version because pm during will college based love article canada black home old her country could shop center if page use january a previous support on to was comments forum states family this would between free posts him usa rights special section very content said view training add report return since do music results link policy private have state here travel city guide control online we be books my you go up reserved number north value way game office been address computer when more its line phone point change our history must review members real today profile american media who class access in students web email network learn internet water under news what personal these own your other being shipping sales property file credit book and systems but two area sign hotel made open company image has name science items click hours john small still can school days did size send sports national also system stock info even they links with services security shall digital make reviews advanced good without title need me dvd download user top posted through years no high county only from right board public united women or tools set most out last people privacy rate account little sites law ebay used about them year find full store price like include of world general white back new please provide location see one products does south part over while just much off level the design hotels author customer power every events within an such total”,”responseToEnd”:””,”responseFromLast”:””,”nextToken”:null,”alternatives”:null,”wAlts”:null,”matchIndex”:null,”watermarkWords”:null,”allWatermarkWords”:[“work”,”pages”,”program”,”both”,”related”,”are”,”listing”,”show”,”car”,”site”,”before”,”life”,”service”,”three”,”current”,”details”,”press”,”mail”,”compare”,”house”,”great”,”each”,”item”,”he”,”another”,”code”,”video”,”reply”,”health”,”project”,”best”,”day”,”art”,”money”,”games”,”terms”,”get”,”buy”,”now”,”am”,”product”,”group”,”software”,”quality”,”as”,”children”,”care”,”website”,”many”,”research”,”some”,”which”,”message”,”shopping”,”type”,”time”,”all”,”into”,”times”,”by”,”member”,”there”,”business”,”read”,”their”,”visit”,”big”,”jan”,”well”,”it”,”were”,”pm”,”during”,”college”,”love”,”black”,”old”,”her”,”country”,”center”,”if”,”page”,”use”,”january”,”a”,”support”,”on”,”was”,”comments”,”forum”,”states”,”this”,”between”,”posts”,”usa”,”special”,”section”,”very”,”content”,”view”,”add”,”since”,”policy”,”private”,”have”,”state”,”city”,”guide”,”online”,”we”,”my”,”you”,”number”,”value”,”game”,”been”,”phone”,”change”,”our”,”review”,”real”,”today”,”american”,”media”,”who”,”in”,”students”,”web”,”network”,”learn”,”water”,”news”,”personal”,”your”,”shipping”,”property”,”and”,”systems”,”but”,”area”,”made”,”open”,”company”,”name”,”science”,”click”,”john”,”still”,”can”,”national”,”stock”,”even”,”links”,”security”,”make”,”good”,”without”,”need”,”user”,”top”,”years”,”no”,”high”,”county”,”only”,”public”,”united”,”women”,”tools”,”set”,”last”,”people”,”sites”,”about”,”them”,”find”,”full”,”store”,”like”,”include”,”white”,”new”,”please”,”provide”,”one”,”does”,”south”,”just”,”much”]},{“text”:”If you wrote a couple of paragraphs, up to half of the words you used would probably be on the special list, statistically speaking. (This text is from a New York Times article about Serena Williams from 2022.)
When a language model or chatbot writes, it can insert a watermark by choosing more of the words on the special list than a person would be expected to use.”,”position”:”max”,”promptName”:”human”,”greenList”:”0.25″,”icon”:”Human”,”prevPos”:378,”isMax”:true,”pos”:93,”responseSoFar”:”On Friday, a few weeks before her 41st birthday, Serena Williams lost to Ajla Tomljanovic in the third round of the U.S. Open, 7-5, 6-7 (4), 6-1, in what is very likely the end of a career that forever changed the world’s perception and understanding of women — especially Black women — in sports.
Williams dominated generation after generation of opponents for 27 years and changed the way women’s tennis is played, winning 23 Grand Slam singles titles, one shy of the record, and cementing her reputation as the queen of comebacks.”,”responseToEnd”:””,”responseFromLast”:””,”nextToken”:null,”alternatives”:null,”wAlts”:null,”matchIndex”:null,”watermarkWords”:null,”allWatermarkWords”:[“few”,”weeks”,”before”,”Williams”,”Tomljanovic”,”third”,”is”,”likely”,”end”,”a”,”career”,”in”,”Williams”,”dominated”,”way”,”women’s”,”is”,”winning”,”Slam”,”her”,”as”]},{“text”:”The text here was generated by the researchers at the University of Maryland who wrote the watermarking paper. They used a technique that essentially bumped up the scores of the words on the special list, making the generator more likely to use them.
When the generator got to this point in the text, it would have chosen the word “the” …”,”position”:”3″,”promptName”:”serena”,”icon”:”A.I.”,”prevPos”:93,”isMax”:false,”pos”:3,”responseSoFar”:”Serena Williams,”,”responseToEnd”:” who played the final Grand Slam in her 27-year career on Saturday, has always been an absolute force of nature.
Whether it was winning the Australian Open title at age 17 in 1999, becoming the world’s first all-American No. 1 player in 2001, or having a baby less than six months ago, Williams has always been an athlete who doesn’t follow the normal rules.”,”responseFromLast”:””,”nextToken”:” who”,”alternatives”:{“the”:21,”who”:16,”a”:6,”right”:4,”one”:3,”left”:3,”in”:3,”center”:3,”winner”:2,”whose”:2,”with”:1,”pictured”:1,”at”:1,”seen”:1,”as”:1,”of”:1,”after”:1,”an”:1,”her”:1,”on”:1},”wAlts”:{“who”:22,”the”:11,”a”:8,”right”:6,”one”:5,”left”:4,”whose”:2,”with”:2,”at”:2,”seen”:2,”in”:1,”center”:1,”after”:1,”on”:1,”winner”:1,”tennis”:1,”during”:1,”pictured”:1,”now”:1},”matchIndex”:0,”watermarkWords”:[“who”,”a”,”right”,”one”,”left”,”whose”,”with”,”at”,”seen”,”after”,”on”,”37″,”tennis”,”during”,”now”],”allWatermarkWords”:[]},{“text”:”… but the word “who” was on the special list, and its score was artificially increased enough to overtake the word “the.””,”position”:”3″,”promptName”:”serena”,”showAlts”:”yes”,”blueCheck”:”yes”,”icon”:”A.I.”,”prevPos”:3,”isMax”:false,”pos”:3,”responseSoFar”:”Serena Williams,”,”responseToEnd”:” who played the final Grand Slam in her 27-year career on Saturday, has always been an absolute force of nature.
Whether it was winning the Australian Open title at age 17 in 1999, becoming the world’s first all-American No. 1 player in 2001, or having a baby less than six months ago, Williams has always been an athlete who doesn’t follow the normal rules.”,”responseFromLast”:””,”nextToken”:” who”,”alternatives”:{“the”:21,”who”:16,”a”:6,”right”:4,”one”:3,”left”:3,”in”:3,”center”:3,”winner”:2,”whose”:2,”with”:1,”pictured”:1,”at”:1,”seen”:1,”as”:1,”of”:1,”after”:1,”an”:1,”her”:1,”on”:1},”wAlts”:{“who”:22,”the”:11,”a”:8,”right”:6,”one”:5,”left”:4,”whose”:2,”with”:2,”at”:2,”seen”:2,”in”:1,”center”:1,”after”:1,”on”:1,”winner”:1,”tennis”:1,”during”:1,”pictured”:1,”now”:1},”matchIndex”:0,”watermarkWords”:[“who”,”a”,”right”,”one”,”left”,”whose”,”with”,”at”,”seen”,”after”,”on”,”37″,”tennis”,”during”,”now”],”allWatermarkWords”:[]},{“text”:”When the generator got here, the words “Tuesday,” “Thursday” and “Friday” were on the special list …”,”position”:”16″,”promptName”:”serena”,”icon”:”A.I.”,”prevPos”:3,”isMax”:false,”pos”:16,”responseSoFar”:”Serena Williams, who played the final Grand Slam in her 27-year career on”,”responseToEnd”:” Saturday, has always been an absolute force of nature.
Whether it was winning the Australian Open title at age 17 in 1999, becoming the world’s first all-American No. 1 player in 2001, or having a baby less than six months ago, Williams has always been an athlete who doesn’t follow the normal rules.”,”responseFromLast”:” played the final Grand Slam in her 27-year career on”,”nextToken”:” Saturday”,”alternatives”:{“Saturday”:72,”Thursday”:8,”Sunday”:8,”Friday”:4,”Monday”:3,”Tuesday”:1,”the”:1,”Wednesday”:1,”Sept”:”<1","a":"<1","June":"<1","Day":"<1","September":"<1","an":"<1","this":"<1","Aug":"<1","Wim":"<1","Centre":"<1","her":"<1","what":"<1"},"wAlts":{"Saturday":58,"Thursday":18,"Friday":8,"Sunday":6,"Tuesday":3,"Monday":2,"Wednesday":2,"the":1,"Sept":"<1","a":"<1","Day":"<1","September":"<1","June":"<1","Centre":"<1","an":"<1","what":"<1","day":"<1","this":"<1","July":"<1","Aug":"<1"},"matchIndex":0,"watermarkWords":["Thursday","Friday","Tuesday","Wednesday","Day","September","Centre","what","day","July"],"allWatermarkWords":[]},{"text":"… but their scores were not increased so much that they overtook “Saturday,” which was by design. For watermarking to work well, it should not overrule an A.I. on its choice of words when it comes to dates or names, to avoid inserting falsehoods. (Although, in this case, the A.I. was wrong: Ms. Williams’s final match was indeed on a Friday.)","position":"16","promptName":"serena","showAlts":"yes","icon":"A.I.","prevPos":16,"isMax":false,"pos":16,"responseSoFar":"Serena Williams, who played the final Grand Slam in her 27-year career on","responseToEnd":" Saturday, has always been an absolute force of nature.
Whether it was winning the Australian Open title at age 17 in 1999, becoming the world’s first all-American No. 1 player in 2001, or having a baby less than six months ago, Williams has always been an athlete who doesn’t follow the normal rules.”,”responseFromLast”:””,”nextToken”:” Saturday”,”alternatives”:{“Saturday”:72,”Thursday”:8,”Sunday”:8,”Friday”:4,”Monday”:3,”Tuesday”:1,”the”:1,”Wednesday”:1,”Sept”:”<1","a":"<1","June":"<1","Day":"<1","September":"<1","an":"<1","this":"<1","Aug":"<1","Wim":"<1","Centre":"<1","her":"<1","what":"<1"},"wAlts":{"Saturday":58,"Thursday":18,"Friday":8,"Sunday":6,"Tuesday":3,"Monday":2,"Wednesday":2,"the":1,"Sept":"<1","a":"<1","Day":"<1","September":"<1","June":"<1","Centre":"<1","an":"<1","what":"<1","day":"<1","this":"<1","July":"<1","Aug":"<1"},"matchIndex":0,"watermarkWords":["Thursday","Friday","Tuesday","Wednesday","Day","September","Centre","what","day","July"],"allWatermarkWords":[]},{"text":"In the end, about 70 percent of the words in the generated text were on the special list — far more than would have been in text written by a person. A detection tool that knew which words were on the special list would be able to tell the difference between generated text and text written by a person.
That would be especially helpful for this generated text, as it includes several factual inaccuracies.”,”position”:”max”,”promptName”:”serena”,”icon”:”A.I.”,”greenList”:”0.7″,”prevPos”:16,”isMax”:true,”pos”:79,”responseSoFar”:”Serena Williams, who played the final Grand Slam in her 27-year career on”,”responseToEnd”:”, has always been an absolute force of nature.
Whether it was winning the Australian Open title at age 17 in 1999, becoming the world’s first all-American No. 1 player in 2001, or having a baby less than six months ago, Williams has always been an athlete who doesn’t follow the normal rules.”,”responseFromLast”:”, has always been an absolute force of nature.
Whether it was winning the Australian Open title at age 17 in 1999, becoming the world’s first all-American No. 1 player in 2001, or having a baby less than six months ago, Williams has always been an athlete who doesn’t follow the normal rules.”,”nextToken”:null,”alternatives”:null,”wAlts”:null,”matchIndex”:null,”watermarkWords”:null,”allWatermarkWords”:[“Williams”,”,”,”who”,”played”,”the”,”final”,”Grand”,”Slam”,”in”,”her”,”27″,”year”,”career”,”on”,”,”,”has”,”always”,”been”,”an”,”absolute”,”force”,”of”,”nature”,”.”,”Whether”,”it”,”was”,”winning”,”the”,”Australian”,”Open”,”title”,”at”,”age”,”17″,”in”,”1999″,”,”,”becoming”,”the”,”world’s”,”first”,”all”,”-“,”American”,”No”,”1″,”player”,”in”,”2001″,”or”,”having”,”a”,”baby”,”less”,”six”,”months”,”ago”,”always”,”been”,”an”,”who”,”normal”,”rules”]}] } let component; // TODO capture props. requires compilation in dev mode NYTG.watch(url, (mod) => { const Component = mod.default; try { if (component) component.$destroy(); console.log(‘destroyed component preview-2’); } catch (err) { console.error(err); target.innerHTML = ”; } component = new Component({ target, hydrate: !component, props }); }); }); ]]>
If someone tried to remove a watermark by editing the text, they would not know which words to change. And even if they managed to change some of the special words, they would most likely only reduce the total percentage by a couple of points.
Tom Goldstein, a professor at the University of Maryland and co-author of the watermarking paper, said a watermark could be detected even from “a very short text fragment,” such as a tweet. By contrast, the detection tool OpenAI released requires a minimum of 1,000 characters.
Like all approaches to detection, however, watermarking is not perfect, Mr. Goldstein said. OpenAI’s current detection tool is trained to identify text generated by 34 different language models, while a watermark detector could only identify text that was produced by a model or chatbot that uses the same list of special words as the detector itself. That means that unless companies in the A.I. field agree on a standard watermark implementation, the method could lead to a future where questionable text must be checked against several different watermark detection tools.
To make watermarking work well every time in a widely used product like ChatGPT, without reducing the quality of its output, would require a lot of engineering, Mr. Goldstein said. Mr. Leike of OpenAI said the company was still researching watermarking as a form of detection, and added that it could complement the current tool, since the two “have different strengths and weaknesses.”
Still, many experts believe a one-stop tool that can reliably detect all A.I. text with total accuracy may be out of reach. That is partly because tools could emerge that could help remove evidence that a piece of text was generated by A.I. And generated text, even if it is watermarked, would be harder to detect in cases where it makes up only a small portion of a larger piece of writing. Experts also say that detection tools, especially those that do not use watermarking, may not recognize generated text if a person has changed it enough.
“I think the idea that there’s going to be a magic tool, either created by the vendor of the model or created by an external third party, that’s going to take away doubt — I don’t think we’re going to have the luxury of living in that world,” said David Cox, the director of the MIT-IBM Watson A.I. Lab.
Sam Altman, the chief executive of OpenAI, shared a similar sentiment in an interview with StrictlyVC last month.
“Fundamentally, I think it’s impossible to make it perfect,” Mr. Altman said. “People will figure out how much of the text they have to change. There will be other things that modify the outputted text.”
Part of the problem, Mr. Cox said, is that detection tools themselves present a conundrum, in that they could make it easier to avoid detection. A person could repeatedly edit generated text and check it against a detection tool until the text is identified as human-written — and that process could potentially be automated. Detection technology, Mr. Cox added, will always be a step behind as new language models emerge, and as existing ones advance.
“This is always going to have an element of an arms race to it,” he said. “It’s always going to be the case that new models will come out and people will develop ways to detect that it’s a fake.”
Some experts believe that OpenAI and other companies building chatbots should come up with solutions for detection before they release A.I. products, rather than after. OpenAI launched ChatGPT at the end of November, for example, but did not release its detection tool until about two months later, at the end of January.
By that time, educators and researchers had already been calling for tools to help them identify generated text. Many signed up to use a new detection tool, GPTZero, which was built by a Princeton University student over his winter break and was released on Jan. 1.
“We’ve heard from an overwhelming number of teachers,” said Edward Tian, the student who built GPTZero. As of mid-February, more than 43,000 teachers had signed up to use the tool, Mr. Tian said.
“Generative A.I. is an incredible technology, but for any new innovation we need to build the safeguards for it to be adopted responsibly, not months or years after the release, but immediately when it is released,” Mr. Tian said.
