Editor’s Note: This is the first part of a two-part discussion on chatbot citation authored by Leticia Antunes Nogueira and Jan Ove Rein, both from the Norwegian University of Science and Technology (NTNU). Leticia is project leader for artificial intelligence at the university library. Jan Ove is Senior Research Librarian and acts as a subject specialist for medicine, providing teaching, counseling and research support for staff and students at The Faculty of Medicine and Health Science. Part two is now available.

With the rapid proliferation of AI-tools in research and higher education, how we treat outputs from chatbots powered by large language models (LLMs) is one area of contention. When it comes to the question of generative AI and authorship, academic journals and publishers have been quick to release policies and position themselves largely against the practice of treating chatbots as (co)authors, since machines cannot bear responsibility for the text they produce (see for instance Nature and Science). Nonetheless, the question of citation is still quite vague. If you use a chatbot in writing a text, and are discouraged from listing it as a coauthor, should you attribute the relevant passages to the tool via citation instead? Is it appropriate to cite chatbots as information sources?

If you use a chatbot in writing a text, and are discouraged from listing it as a coauthor, should you attribute the relevant passages to the tool via citation instead? Is it appropriate to cite chatbots as information sources?

It can be difficult to quantify the extent to which scholars have taken up the practice of citing chatbots. One example from Retraction Watch illustrates an article that has been retracted after the editors learned that the authors had used ChatGPT to update references, which led to incorrect and manufactured citations. We have searched for ChatGPT and OpenAI in Web of Science’s Cited References to try and identify where chatbots might show up. The trouble is that, among the hundreds of results, it can be difficult to sort which might be legitimate references (such as when citing a report or documentation issued by OpenAI) from more questionable ones (such as when authors have used a chatbot as information sources). What we observe from our interactions with other librarians, researchers and students is that there does not seem to be a consensus surrounding chatbot citation, but there is a strong demand for more clear guidance.

The question of whether chatbots should be cited appears to be related to whether AI tools should be seen as information sources. To be clear, we are focused on applications that make foundational models available to the public, such as OpenAI’s ChatGPT or Anthropic’s Claude. We assume that users working with tools that apply GenAI capabilities combined with other sources — such as Perplexity, Scopus AI, and others — would reference the original sources.

In this post (part I), we briefly explore what publishers’ policies say about generative AI and the reasons why we cite other works in general. In part II, we make the case for treating chatbots as tools rather than sources — in other words, for disclosing how they are used without citing them.

Publishers’ policies on generative AI

We consulted the AI policies for seventeen publishers and relevant academic organizations — AAAS (American Association for the Advancement of Science); APA Publishing; British Medical Journal; Cambridge University Press; COPE (Committee on Publication Ethics); Elsevier; Frontiers; IEEE (Institute of Electrical and Electronics Engineers); International Committee of Medical Journal Editors: IOP Publishing; MDPI; Oxford University Press; PLOS; Sage; Springer Nature; Taylor & Francis; and Wiley. Policies were consulted on 21^st March 2024, and once again on 22^nd May 2024.

In general, there is consensus that chatbots do not meet the minimum requirements for authorship, and that the use of tools with generative AI capabilities should be declared and detailed. However, the question of whether they can (or should) be cited as sources is not explicit in many cases. The reasons given for rejecting chatbots as authors can also be grounds to reject them as sources, but a clear connection needs to be made.

The reasons given for rejecting chatbots as authors can also be grounds to reject them as sources, but a clear connection needs to be made.

Contributing to the confusion is the fact that the American Psychological Association (APA) — whose style is used far beyond the discipline of psychology — has issued concrete guidelines for how to cite chatbots as tools. This could be confused with citing them as sources. Additionally, the influential recommendations from the International Committee of Medical Journal Editors (ICMJE), and Elsevier clearly advise against citing chatbots. Below is an excerpt from APA Style Blog — the latest edition of the APA’s Publication Manual does not encompass a clear guideline for AI or Chatbots. APA’s publishing policies point to APA Style Blog, which is our source for this quote and for APA’s guidance on the matter. The recommendation given by APA is illustrated by the quote below:

“If you’ve used ChatGPT or other AI tools in your research, describe how you used the tool in your Method section or in a comparable section of your paper. (…) Unfortunately, the results of a ChatGPT ‘chat’ are not retrievable by other readers, and although non-retrievable data or quotations in APA Style papers are usually cited as personal communications, with ChatGPT-generated text there is no person communicating. Quoting ChatGPT’s text from a chat session is therefore more like sharing an algorithm’s output; thus, credit the author of the algorithm with a reference list entry and the corresponding in-text citation. (…) The in-text citations and references above are adapted from the reference template for software in Section 10.10 of the Publication Manual” (APA Style Blog, 2023, our Emphasis).

APA Style Blog gives a few examples on how to cite from ChatGPT, all of which include informing in the running text not only the response from the chatbot, but also the instructions that prompted it, in addition to their recommending the documentation of the full transcript of the exchange in an appendix. It is clear from the APA Style Blog that although they offer guidelines for citing chatbots, they do not take a stand accepting them as information sources. Nonetheless, contrast APA’s guidelines with the following excerpts from ICMJE and Elsevier:

“At submission, the journal should require authors to disclose whether they used artificial intelligence (AI) assisted technologies (such as Large Language Models [LLMs], chatbots, or image creators) in the production of submitted work. Authors who use such technology should describe, in both the cover letter and the submitted work, how they used it. (…) Authors should carefully review and edit the result because AI can generate authoritative-sounding output that can be incorrect, incomplete, or biased. Authors should not list AI and AI assisted technologies as an author or co-author, nor cite AI as an author. Authors should be able to assert that there is no plagiarism in their paper, including in text and images produced by the AI. Humans must ensure there is appropriate attribution of all quoted material, including full citations” (ICMJE, 2023, p.3, our emphasis).

***

“Authors should not list AI and AI-assisted technologies as an author or co-author, nor cite AI as an author. Authorship implies responsibilities and tasks that can only be attributed to and performed by humans” (Elsevier, 2023, our emphasis).

Both the ICMJE and Elsevier take a clearer stance on the matter. Like APA, they recommend authors to declare their use of AI. But by instructing scholars to not cite AI as an author, they preclude the use of chatbots as information sources.

Why we cite other works

The practice of citation is a sociocultural norm; it is not the result of lawmaking. Laws regulating research ethics — which also involve academic misconduct and plagiarism — emerged in the postwar period, while citation is a much older practice. Thus, research ethics frameworks have formally spelled out and institutionalized attitudes and practices that had already been widespread. The issue with social norms, in contrast with hard rules and laws, is that compliance or non-compliance rely to a greater extent on judgment that is situation dependent. The same goes for deciding when a citation is needed, although the accepted wisdom would say that when in doubt, chances are a reference is needed.

The issue with social norms, in contrast with hard rules and laws, is that compliance or non-compliance rely to a greater extent on judgment that is situation dependent.

Although each academic style has its own specificities, all of them build on shared underlying premises concerning which kinds of statements require a reference and which do not. These premises can be difficult to lay out in unambiguous rules valid for all cases (the very kind that machines can operationalize), but humans’ abilities with tacit knowledge have been able to grasp them with guidance and experience.

The reasons why we cite other works are numerous. We cite to disclose what our sources are and to show support/evidence for a claim. We cite to place our work in the landscape of knowledge, signal where we stand in a debate and inform the reader where they can find out more. We cite to acknowledge those upon whom our ideas build, and to give them their due credit. In practice, these reasons have gone hand in hand, even though they are conceptually distinct.

The rapid development of generative AI capabilities (particularly regarding chatbots) forces us to look to citation anew. That is, when considering whether to cite from chatbots, we are confronted with a split between: (i) reasons that have to do with acknowledging ideas that are not one’s own and attributing credit, and (ii) reasons that have to do with disclosing sources of information.

Questions concerning chatbots and citation practices forces us to grapple with the question of what these AI innovations actually are. Are they sources or are they tools?

Are chatbots sources or are they tools?

Whether you think authors should add a reference to sentences taken from a chatbot depends on how you view chatbots and what reasons for citing have more weight in your view. On the one hand, those who think a citation is required are generally concerned with the attribution of credit and origin. After all, if you are using text you have not written yourself, then the source must be acknowledged; otherwise, you would be engaging in plagiarism. This is consistent with the idea that chatbots can be accepted as sources. For many people, the knee jerk reaction is that just as you credit text and ideas that are not yours through citation, you should also cite chatbots if that is where you got your text and ideas from.

On the other hand, those who think a citation is not only unnecessary, but also should be avoided are concerned with the fact that LLMs generate outputs that are predominantly untraceable, and therefore do not give the reader the possibility of fully checking the statement against the source. After all, how can something be a source if it cannot be traced and double-checked? This is consistent with the view that chatbots are merely tools we apply in intellectual work. That is, just as you do not refer to the multitude of other tools you might use in your research (other than perhaps contextually in the methods section), you do not cite a chatbot.

As these observations exemplify, the emergence of chatbots challenges assumptions we have so far taken for granted about sources, information and knowledge. Competences in source criticism and critical thinking have always been crucial. Nonetheless the challenge with securing the integrity of our information ecosystem has grown significantly in the age of AI. Because coherence and meaning are in the eyes of the beholder, strings of words in grammatical order resulting from probabilistic calculations constitute neither sources, nor information or knowledge. Rather, text generated by chatbots constitutes synthetic text and bring about the idea of non-information.

Keeping in mind publishers’ positions on AI and authorship, as well as the reasons why we cite other works and the different perspectives on chatbots as tools vs sources, in part II, we make the case for not citing chatbots as information sources.

Acknowledgements

We would like to thank Ann Michael, Avi Staiman and Tadeu Fernando Nogueira for their comments on earlier versions of this two-part series and Sindre Andre Pedersen for interesting discussions on this theme. We are also grateful to Inger Hesjevoll Schmidt-Melbye and Alexander Lyngsnes for their assistance with etymological interpretation (part I), and Katrine Aronsen for her advice on searching for articles that employ chatbots as information sources.