Academic Libraries’ Citation Guides to ChatGPT Show Mixed Levels of Accuracy and Currency
Notice bibliographique
Résumé
A Review of: Moulaison-Sandy, H. (2023). What is a person? Emerging interpretations of AI authorship and attribution. Proceedings of the Association for Information Science & Technology, 60(1), 279–290. https://doi.org/10.1002/pra2.788 Objective – To examine how and which academic libraries are responding to emerging guidelines on citing ChatGPT in the American Psychological Association (APA) style through guidance published on the libraries’ websites. Design – Analysis of search results and webpage content. Setting – Websites of academic libraries in the United States. Subjects – Library webpages addressing how ChatGPT should be cited in APA format. Methods – Google search results for academic library webpages providing guidance on citing ChatGPT in APA format were retrieved on a weekly basis using the query “chatgpt apa citation site:.edu” over a six-week period that covered the weeks before and immediately after the APA issued official guidance for citing ChatGPT. The first three pages of relevant search results were coded in MAXQDA and analyzed to determine the type of institution, using the Carnegie Classification and membership in the Association of American Universities (AAU). As this was a period during which APA style recommendations for citing ChatGPT were shifting, the accuracy of the library webpage content was also assessed and tracked across the studied time period. Main Results – During the six-week period, the number of library webpages with guidance for citing ChatGPT in APA format increased. Although doctoral universities accounted for the largest number of webpages each week, baccalaureate colleges, baccalaureate/associate’s colleges, and associates’ colleges were also well-represented in the search results. Institutions belonging to the AAU were represented by a relatively small number throughout the study. Over half of the pages made some mention of APA’s recommendations being interim or evolving, though the exact number fluctuated throughout the period. Prior to the collection period, APA had revised its initial recommendations to cite ChatGPT as a webpage or as personal communication, but 40% to 60% of library webpages continued to offer this outdated guidance. Of the library webpages, 13% to 40% provided verbatim guidance from ChatGPT responses on how it should be cited. The final two weeks of the collection period occurred after April 7, 2023, when APA had published official recommendations for citing ChatGPT. In the week following this change, none of the webpages in the first three pages of results had been updated to fully capture the new recommendations. The study analyzed the nine webpages appearing in the first page of results for the second week after APA’s official recommendations were published, showing that three linked to the APA’s blog, zero provided further explanation on how to apply the recommendations, five included outdated guidance, and three gave guidance from ChatGPT’s responses to questions on how it should be cited. Conclusion – The author sees the results of the study as reflecting three interrelated components: a new technology, gaps in librarians’ knowledge related to large language models (LLMs) and how they are currently being discussed in terms of authorship, and Google’s inability to rank the results in a way that prioritizes correct information. The substantial presence of institutions serving undergraduates leads the author to conclude that this is the population most in need of guidance for citing ChatGPT and the responsiveness on the part of the librarians shows an understanding of this need, even if the guidance itself is inaccurate.
Récupéré en direct depuis OpenAlex et désinversé. Les résumés ne sont pas conservés dans cette base de données : les index inversés représentent 8,6 Go des 9,3 Go de texte de la base, et le serveur dispose de 13 Go libres.
Comment cette classification a été obtenuedéplier
Prédiction machine sur la base complète
Imitation des enseignantsNi prévalence calibrée, ni vérité terrain. Validation humaine à venir. Le volet Gemma est une étiquette directe du modèle pour chaque travail de la base, lue sur la notice réduite au titre. Le volet Codex est un classifieur appris des 10 348 étiquettes directes de Codex et calibré sur les taux pondérés de l'échantillon; les champs sans appui suffisant ne portent aucun appel Codex. Le mode candidate est l'union des deux volets; le consensus est leur intersection. Ces sorties portent le statut machine_predicted_unvalidated et ne sont pas des étiquettes humaines.
Scores du classifieur distillé par catégorie (deux têtes)
| Catégorie | Codex | Gemma |
|---|---|---|
| Métarecherche | 0,054 | 0,348 |
| Méta-épidémiologie (sens strict) | 0,001 | 0,001 |
| Méta-épidémiologie (sens large) | 0,002 | 0,001 |
| Bibliométrie | 0,057 | 0,063 |
| Études des sciences et des technologies | 0,004 | 0,003 |
| Communication savante | 0,014 | 0,013 |
| Science ouverte | 0,003 | 0,006 |
| Intégrité de la recherche | 0,002 | 0,002 |
| Charge utile insuffisante (le modèle a refusé de juger) | 0,013 | 0,010 |
Scores machine (provisoires)
Les deux têtes enseignantes du modèle étudiant, lues sur ce travail. Un score ordonne la base pour la relecture; il n'affirme jamais une catégorie, et le statut de validation accompagne chaque rangée tel quel.
Scores de référence d'un modèle non mature (critères de maturité non atteints, 7 itérations). Un score ordonne; il n'affirme jamais une catégorie.
score_only:v0-immature-baseline · tel quel depuis la passe de notation : score_only signifie que le nombre peut ordonner les travaux, et qu'aucune étiquette de catégorie n'en découleClassification
machine, non validéePrédiction automatique; un appel candidat d’une seule source (Gemma direct ou Codex distillé), pas un consensus.
Le détail, modèle par modèle et score par score, se trouve en fin de page sous « Comment cette classification a été obtenue ».