KEY TAKEAWAYS

• Findings of a recent study suggest that accurate detection of AI-generated text can be achieved.
• Researchers propose that accuracy is dependent on tailoring detectors to specific fields and writing types.

The meteoric rise of large language models, such as ChatGPT, is likely to result in a rapid increase in the use of generative artificial intelligence (AI) in academic publishing. This presents a quandary for journal publishers and editorial teams as they strive to develop guidance and ‘stay ahead’ of the technology. Currently, attitudes vary somewhat between journals, ranging from The Lancet limiting AI use to improving readability, to Nature adopting a firm stance against the use of generative AI to create images. Regardless of the detail in individual guidelines, enforcement is reliant on accurate detection of AI-generated content; technology which, to date, has been viewed as flawed. A recent Nature News article by McKenzie Prillaman spotlights research on a potential solution, namely, the development of more specialist detectors.

Developing a specialist AI detector

As Prillaman reports, a recent study published in Cell Reports Physical Science suggests that tailoring AI detectors so that they are trained to check specific types of writing may result in more reliable detection methods.

Tailoring AI detectors so that they are trained to check specific types of writing may result in more reliable detection methods.

The research group, Desaire et al., used 100 published (ie, human-created) introductions from articles in various chemistry journals to train ChatGPT 3.5 to develop 200 introductions that followed similar styles. These documents were used to train their machine learning algorithm. The model was then used to test more articles, checking for AI- vs human-generated content via 20 different features of writing style. The group found that:

• the detector identified AI-generated documents with 98–100% accuracy
• human-written documents were detected with 96% accuracy
• the model outperformed other more general detectors, such as OpenAI’s AI classifier and ZeroGPT, in detecting AI-generated documents
• the model performed similarly when tested on writing from chemistry journals beyond those it was trained on, but not when tested on more general science magazine writing.

Implications for scientific publishers

The group concluded that their detector outperformed its contemporaries because it was trained specifically on academic publications. They propose that this tailored approach is vital for the development of accurate AI detectors suitable for use by academic publishers.

————————————————

In what co-author Dr Rebeccah Lijek describes as “one of the worst kept secrets in academia”, a recent survey published in eLife has shown that it is common practice for early career researchers to be involved in the peer review of manuscripts for which they are not the invited reviewer.

The survey made a distinction between ‘co-reviewing’ (making significant contributions to a peer review report as a non-invited reviewer) and ‘ghostwriting’ (co-reviewing without identifying the contributor to journal editorial staff). Nearly three quarters of respondents had co-reviewed a manuscript. Importantly, around half of respondents had ghostwritten a peer review report on behalf of their research group leader, despite 81% viewing ghostwriting as unethical.

Around half of respondents had ghostwritten a peer review report.

Most respondents felt that co-reviewing is a beneficial (95%) and ethical (73%) form of training in peer review. However, in a recent opinion piece, Dr James Sherley argues that the place to learn peer review skills is in open working forums, such as departmental journal clubs and graduate courses. He reasons that inexperienced reviewers may do a disservice to submitting authors, as deficient reviews could result in manuscript rejection, impacting researchers’ career success, research funding and even scientific progress. It may also exploit early career researchers if they are denied credit. Furthermore, Dr Sherley believes that co-reviewing erodes academic integrity by violating the professional ethics of peer review.

In contrast, the study’s authors highlight potential benefits for journals if barriers preventing early career co-reviewers being named as contributors are removed: this could help to enlarge reviewer pools, while addressing ethical concerns surrounding ghostwriting. While the study’s authors put the onus on journals to take responsibility for fair and ethical peer review, Dr Sherley believes that academic institutions need to begin teaching ethical manuscript review as a core principle of academic life. With many ongoing initiatives aiming to improve peer review processes, we are interested to see how these issues may be tackled in future.

Note: There is a poll embedded within this post, please visit the site to participate in this post's poll.

——————————————————–

Summary by Robyn Foster PhD from Aspire Scientific

——————————————————–

With thanks to our sponsors, Aspire Scientific Ltd and NetworkPharma Ltd

Recorded 4 July 2018 at a MedComms Networking event in Oxford. Produced by NetworkPharma.tv

Jackie’s presentation (PDF format) is available here.