A checker's career?

For many of us, the ability to conduct reference or citation checking on behalf of prospective authors employed by our institution is rightly regarded as a core competency. Part science, part art, this particular brand of literary detective work is both challenging and, when successful, ultimately rewarding. Skills in reference checking cover the gamut of activity from simple correction of typographical errors through to the pièce de résistance—correctly identifying a source from accidentally anonymized photocopies by such features as the most recently cited reference (for the date), typography (for the publishers) and patterns of citation (for candidate journals). I still have vivid memories of my time as an apprentice librarian of the mid 1980s, painstakingly checking the reference citations of a proposed BMJ article and labouring under the added pressure that this particular article was reporting on reference accuracy!1 If, like Caesar's wife, we as librarians have to be similarly above reproach, technological improvements, such as the PubMed Single Citation Matcher function (http://www.ncbi.nlm.nih.gov/entrez/query/static/citmatch.html) and the greater availability of full text, have at least combined to make the task of citation checking much easier. Nevertheless, checking of incorrect or incomplete references continues to command much time and effort for enquiry desk or inter-library loan staff alike. For the reference checker, three related questions are of practical importance: What is the likelihood that a particular reference is inaccurate? Which elements of a reference are most likely to be incorrect? and Can I develop a reference checking protocol to make the process more efficient and effective? The answers to these questions lie within the evidence base of reference services and enquiry work, one of the six domains of evidence-based information practice.2 We start with a realistic scenario: An NHS Trust librarian notices that he and his staff are spending an inordinate amount of time trying to look up references and finding that ‘the citations given are just plain wrong’. Feeling that the situation is getting beyond a joke, he starts to wonder ‘Do journal editors ever check that the citations that article authors give are actually correct?’. This leads him to explore the not inconsequential evidence base on citation accuracy with a view to developing a working procedure for himself and his staff to follow. Our first question is ‘What is the likelihood that a particular reference is inaccurate?’. Table 1 demonstrates from 36 published studiesw1–w36 that there is a considerable range in the percentages of incorrect references with the highest being 66%w19 and the lowest being 8%w3. Nevertheless, there is a clear trend for between 25% and 40% of references to be inaccurate. In practical terms this means that between 1 in 2 and 1 in 4 of all references that a librarian might be asked to check are likely to be wrong. Is our librarian's assertion that the situation is getting worse borne out by the evidence? One way to establish this would be to plot all the studies on a graph showing percentage errors versus date of study (in preference to date of publication). However this might well be confounded by interdisciplinary differences. Nevertheless, there are four studies, three carried out in anaesthesiaw1−2,w23 journals and one in the Journal of Hand Surgeryw16 that attempt to look at cross-sectional time trends. The journals Canadian Journal of Anaesthesiaw2 and Journal of Hand Surgeryw16 show a decrease in the number of errors while Anaesthesiaw1 and Acta Anaesthesiologica Scandinavicaw23 show an increase. However, of these studies, the Canadian Journal of Anaesthesiaw2 study is actually an intervention study where, following an identified problem with citation quality, the editors of the Journal requested any contributors whose papers were accepted for publication to verify the accuracy of reference citation by including a photocopy of the first page of each reference. Given, therefore, that one of the four studies describes a strategy to improve reference quality it seems that there is a slight but probably non-significant trend for citation quality to get worse. However, this needs to be investigated through empirical research. Similarly, further investigation is required of differences between specialities. For example, how do nursing journals perform compared with medical, surgical or radiology journals and why do such differences exist? A comparatively smaller number of articles address our second question ‘Which elements of a reference are most likely to be incorrect?’. However, it is possible that this is constrained by the comparatively limited availability of access to full-text journals during writing of this column. Whereas overall percentages are likely to be reported (or at least calculable) from a journal abstract, more detailed coverage of which of the six primary bibliographic elements are found to be incorrect is usually only identifiable from the main body of the text. Nevertheless, we do have at least nine studies that individually report total numbers of errors for three or more of the bibliographic elements (Table 2). What we find is a clear, and even dramatic, predominance of errors in Authors’ names. While the frequency of errors in Article Titles is also high, we should bear in mind that these may include anything up to a sentence or more which can be prone to spelling, typographical or punctuation errors. Errors in Year and Volume are typically less common than those connected with Page Numbers. An interesting observation, although not explored in either of the above tables, is that the numbers of fatal flaws (i.e. where the reference is completely irretrievable) are very few indeed. Our final question relates to ‘Can I develop a reference checking protocol to make the process more efficient and effective?’. An interesting finding is that the approach that most librarians will utilize instinctively, that is the author search, is the least likely to be successful. Statistically, the Journal Title and Year in combination are less likely to be incorrect than the Author. A slight confounder here is, of course, that reported author errors may not necessarily involve the first-cited author but typically cumulate errors for any author. Nevertheless, a strategy that involves browsing journal contents for a particular year (for journal titles with a small number of issues) or which combines Journal Title, Year and a comparatively rare (but properly spelt) word from the Article Title might well prove an effective alternative to an Author search. Such a strategy would thus circumvent the multiple possibilities of error in an author's name involving misspelt surnames and incorrect, transposed or omitted initials. Time and logistic constraints have meant that it is impossible to do justice in a regular quarterly Using Research column to an important topic that likely merits its own full-blown systematic review. An associated topic appearing in some, but not all, of the retrieved references is that of quotation accuracy, that is whether authors use citations for the purpose that is justified by their original context and meaning. Notwithstanding the acknowledged methodological limitations of our rapid review approach, it is hoped that this issue's column has illustrated the feasibility of transforming a question from library practice into an evidence-based topic. The fact that the topic chosen is one that surfaced the previous month on the lis-medical discussion list illustrates that there is not necessarily a perennial mismatch between questions that we want answered and the evidence from empirical research. Interestingly, the small number of errors that prevent references from being located has led many journals to conclude that significant expenditure of time and money by journal staff members in checking references is probably not justified.3 The International Committee of Medical Journal Editors (http://www.icmje.org/) clearly attributes responsibility for the accuracy to authors but the evidence of high rates of reference error suggests that this is not sufficient to ensure accurate references. One possibility is for editors to sample references from each paper scheduled for publication. Once an error is found, the paper would then be returned with an instruction to check all citations again.3 This would be accompanied by a sanction of delayed publication. Such a process would incidentally lead to a revival in the role of the librarian as citation checker—a key figure in the publish or perish process! Perhaps more realistically, the creation of authoritative bibliographic databases with unique reference identifiers will lead to an eventual standardization of reference elements. In the meantime, the above column should become required reading for all health librarians which assumes, of course, that it will get cited correctly! A list of included studies relating to this paper is available by using the following URL: http://www.blackwellpublishing.com/products/journals/suppmat/HIR/HIR538/HIR538sm.htm Appendix S1. Appendix1. Please note: The publisher is not responsible for the content or functionality of any supporting information supplied by the authors. Any queries (other than missing content) should be directed to the corresponding author for the article.

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.