Responsible but Innovative Use of Artificial Intelligence in Scientific Publishing

In late November 2022, what is arguably the first widely accessible large language model (LLM) ChatGPT (chat.openai.com), was released for public use and appraisal. This represents a step on the road towards artificial intelligence by statistically mimicking human generated factual and stylistic content; using large mathematical models which have been trained on web sites, in books, and other media. and further more curated and corrected by humans working with the OpenAI team. There have been many enthusiastically proposed use cases in the medical field and the release of preliminary pilot products, such as “AI medical scribes,” and equally, a number of concerns raised involving accuracy, security, privacy, and ownership. The appeal of AI is in the near-immediate production of large amounts of plausible text, which works well for ubiquitous topics and less so for many others. Numerous articles and reports demonstrate that it is factually incorrect for many critical medical questions (Sallam, 2023Sallam M. ChatGPT Utility in Healthcare Education, Research, and Practice: Systematic Review on the Promising Perspectives and Valid Concerns.Healthcare. 2023 Mar 19; 11: 887Crossref Scopus (333) Google Scholar). Furthermore, by design, it is unexplainable and it cannot trace the factual source of the text it produces (Amann, 2020); a gaping large problem with AI! Both issues arise due to the fundamental nature of the underpinning transformer algorithms: these models are built not through knowledge but by statistically evaluating billions of samples of text to simply answer “given a series of words, what word is most likely to come next?” It’s clear that without additional significant human intervention ChatGPT (and other LLMs) are NOT a source, and you cannot use ChatGPT to suggest sources – there is a real risk they will be imaginary. This risk of imaginary data (also called hallucinations, derived from the “temperature” of the algorithm) is inseparable from what makes the system attractive in the first place. Asking it to summarize an article, to elaborate on a sentence, or to write a story requires a degree of freedom to create text beyond rote plagiarism. In this sense philosophically it is acting as a creative agent through the mechanism of statistics. We need to know more about what AI can do – it’s possible that the statistical models when pressed can produce new considerations regarding known diseases, or novel ways to diagnose or treat conditions, or act as a hypothesis-generator in research. It is true that forming new connections between previously unrelated pieces of data is a fundamental foundation of human creativity! Small groups seeking publication of novel research face many barriers and pitfalls. Submitting to multiple journals with different specific word counts can be one laborious task which LLMs can do instantly. Sound research submitted to journals that are not in the author’s native language can be overlooked when the article does not appropriately match the tone and timbre of the journal – another task which LLMs seem to excel at. These have the potential to unlock and promote new research globally for researchers who could not otherwise afford a team of medical writers and editors. The expectation should be similar to the rigor of clinical trials; that the use of an LLM should be disclosed and the inputs, the queries, and the outputs should be all readily available as appendices for scrutiny or further analysis. Encouraging this form of open communication will further the truthful, accepted use of LLMs. But there are worrisome considerations. Over time new LLMs being trained on new data and information drawn from the Internet run the risk of being trained on LLM-generated material in a vicious feedback cycle of hallucinatory nonsense (Shumailov I, Shumaylov Z, Zhao Y, Gal Y, Papernot N, Anderson R. Model Dementia: Generated Data Makes Models Forget. arXiv preprint arXiv:2305.17493. (2023)). Stated more explicitly, when an LLM is training for “what word should come next in this sequence,” if it is training from text that was already generated by another imperfect LLM generating imperfect sequences, these studies suggest it leads inexorably into absurdity. If we instead want LLMs to improve, especially on a factual basis, the trainers need to know what sources are reliable human generated content and what are not. It is not clear we will be able to rely on the tools to do it: Currently, GPT-4 is not necessarily able to clearly differentiate text it has generated from human text (Bhattacharjee, Amrita, and Huan Liu. "Fighting Fire with Fire: Can ChatGPT Detect AI-generated Text?." arXiv preprint arXiv:2308.01284 (2023)). One reaches further and reimagines how this can be used for guidelines or in information-producing structures such as network meta analyses. As these models become more sophisticated and capable of better factual interpretation we will need a formal system for determining how to safely integrate these into our discourse - should an LLM specifically trained on the acne data be a participatory member in acne consensus meetings? Should there be a Delphi-like consensus of LLMs as the AI representative in guideline creation? This will be an interesting and challenging direction for coming decades. When we assess the current climate surrounding AI-generated material there are certainly many positive and negative extreme opinions. Unless we encourage transparent exploration of these new technologies, we will be driven by competing corporate influences of hype and fear until the future overtakes us – or, if ChatGPT is prompted on that sentence to “please rephrase and shorten to be more neutral” – “without transparent exploration of new technologies, we may be swayed by corporate agendas as the future progresses.” Amann et al., 2020Amann J, Blasimme A, Vayena E, Frey D, Madai VI. Explainability for artificial intelligence in healthcare: a multidisciplinary perspective. BMC Medical Informatics and Decision Making [Internet]. 2020 Nov 30;20(1). Available from: OpenAI. ChatGPT [Internet]. chat.openai.com. 2023. Available from: https://chat.openai.com. Accessed on Sept 1, 2023.Google Scholar.

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.