By Shane Ryan

What’s the point of academic research? An obvious answer is to advance knowledge and understanding in our various subject areas. Indeed, when peer reviewers judge that manuscripts submitted to journals do this, then often they recommend that those manuscripts be published. That a central goal of research is to advance knowledge and understanding in our various subject areas has, as we shall see, some important implications.      

Using Goals to Inform Means 

If our goal is to advance knowledge and understanding and there is a tool that can help us to do that, then we should use that tool. There is a tool that can help us do that, certainly with respect to theoretical work. That tool is generative AI, more precisely Large Language Models (LLMs). Such LLMs are already contributing to manuscripts that are getting published. For example, the Committee on Publication Ethics (COPE) acknowledges the role that LLMs are already playing in content generation. Numerous papers now seek to address whether LLMs should be recognised as an author given the contribution it can make to publishable papers (see, for example, Responsibility is not required for authorship). What’s more, LLMs as generators of publishable materials can be expected to get better and better. We obviously have reason to think that LLMs are a tool that can help us to advance knowledge and understanding. While there are constraints on using tools to advance knowledge and understanding, such as ethical constraints, there is no obvious such constraint on using AI to advance knowledge and understanding.  

An Obstacle to Using LLMs to Advance Research 

Despite the fact that there is no obvious reason why we shouldn’t use LLMs to advance research, at the moment doing so faces a practical barrier. LLMs are currently not accepted as authors by journals and there are recommendations against attributing authorship to a human if their contribution is deemed to be too low. The latter would be the case, say, if a scholar simply submitted an AI generated text for publication. See, for example, the International Committee of Medical Journal Editors (ICMJE) recommended authorship criteria. 

Perhaps such recommendations should be discarded but they do serve a purpose. A scholar submitting an AI generated text and listing themselves as author is making a false claim or, at the least, a very misleading one. Such a scholar didn’t do the research and being transparent about the source of the research doesn’t change that. 

On the other hand, authors using LLMs to produce publishable research will have provided suitable prompts for LLM to produce publishable research. The ability to provide such prompts, will often require a significant understanding of the relevant literature. Scholars will also have to check any answers provided by LLMs, as there is no guarantee that they will produce good or accurate answers. Scholars may take further steps too to improve a submission. Anecdotally, and somewhat muddying the waters with respect to the attributing authorship to LLMs debate, scholars are running LLM outputs through other LLMs as this leads to improved texts.   

If scholars were to put their name to research that, for example, included fake references which weren’t identified as such prior to publication, then their article might be corrected and, in that case, their reputation as a scholar would likely be diminished. More generally, if a scholar puts their name to research that is bad in any of the various ways research can be bad, it will reflect poorly on them as a scholar.  Scholars using LLMs to produce research will have an incentive to work on that research prior to submission to journals to avoid such embarrassments.  

In fact, in the long-run it seems likely that some scholars will become especially skilled at using LLMs to produce outputs in their subject areas, assuming such publications are permitted. Such scholars and their work with LLMs will advance our goal of furthering knowledge and understanding in our various subjects and will deserve credit for doing so.     

The Research-Discovery Report as a Frame for Author Contributions 

So far it has been argued that scholars should be able to use LLMs for research purposes. It has also been argued that such AI-based research faces legitimate obstacles to being published as things are. That’s why things should change. In order to gain the benefits of LLMs, a new category of research submission should be introduced. For now, let’s call it the Research-Discovery Report. Such submissions will credit scholars as authors of reports of research discoveries and detail how the discovery was made. This will require keeping and submitting the relevant records, including of prompts used, detailing the methodology to discover potentially literature advancing research, and listing the LLMs used and describing how they were used. Easily checkable parts of the research should be checked, including the literature used and the accuracy of the references. Naturally, the author would be responsible for responding to reviewer comments and editorial direction.  

The suggested practice for such reports is fixed by our goal of advancing knowledge and understanding in our various subject areas. The Research-Discovery Report can aid further research advances by sharing how good work was produced, as well as accurately framing what we expect AI-based research to look like. Such an accurate framing not only allows authors to be credited for their work but the framing of their work under the Research-Discovery Report category facilitates recognition of what their contribution has been.  

Conclusion 

LLMs can advance our research. We should make use of them. Doing so currently faces some practical obstacles. A solution is the creation of a new category of research article that frames the author’s role as reporting research advancing output from an LLM and what prompted that LLM to generate that research. Such a category facilitates the publication of valuable research and appropriate credit for the reporting author.         

Author: Shane Ryan 

Affiliation: Public and International Affairs, City University of Hong Kong 

 Competing interests: None 

Social media: LinkedIn