Loading...
Distinguishing academic science writing from humans or ChatGPT with over 99% accuracy using off-the-shelf machine learning tools
Desaire, Heather Chua, Aleesa E. Isom, Madeline Jarosova, Romana Hua, David
Desaire, Heather
Chua, Aleesa E.
Isom, Madeline
Jarosova, Romana
Hua, David
Citations
Altmetric:
Abstract
ChatGPT has enabled access to artificial intelligence (AI)-generated writing for the masses, initiating a culture shift in the way people work, learn, and write. The need to discriminate human writing from AI is now both critical and urgent. Addressing this need, we report a method for discriminating text generated by ChatGPT from (human) academic scientists, relying on prevalent and accessible supervised classification methods. The approach uses new features for discriminating (these) humans from AI; as examples, scientists write long paragraphs and have a penchant for equivocal language, frequently using words like “but,” “however,” and “although.” With a set of 20 features, we built a model that assigns the author, as human or AI, at over 99% accuracy. This strategy could be further adapted and developed by others with basic skills in supervised classification, enabling access to many highly accurate and targeted models for detecting AI usage in academic writing and beyond.
Description
Date
2023-06-21
Journal Title
Journal ISSN
Volume Title
Publisher
Elsevier
Collections
Research Projects
Organizational Units
Journal Issue
Keywords
ChatGPT, AI, Machine learning, Text analysis, XGBoost, Plagiarism
Citation
Desaire, H., Chua, A. E., Isom, M., Jarosova, R., & Hua, D. (2023). Distinguishing academic science writing from humans or ChatGPT with over 99% accuracy using off-the-shelf machine learning tools. Cell reports. Physical science, 4(6), 101426. https://doi.org/10.1016/j.xcrp.2023.101426