Developing and Applying an Information Literacy Rubric to Student Annotated Bibliographies
Why this work is in the frame
A frame that forgets how it found something cannot be audited. These are the routes that admitted this work.
Bibliographic record
Abstract
Objective – This study demonstrates one method of developing and applying rubrics to student writing in order to gather evidence of how students utilize information literacy skills in the context of an authentic assessment activity. The process of creating a rubric, training scorers to use the rubric, collecting annotated bibliographies, applying the rubric to student work, and the results of the rubric assessment are described. Implications for information literacy instruction are also discussed.
 
 Methods – The focus of this study was the English 102 (ENG 102) course, a required research-based writing course that partners the instructors with the university librarians for information literacy instruction. The author developed an information literacy rubric to assess student evaluation of information resources in the ENG 102 annotated bibliography assignment and trained three other librarians how to apply the rubric to student work. The rubric assessed the extent to which students critically applied the evaluative criteria Currency, Relevance, Accuracy, Authority, and Purpose to the information sources in their annotations. At the end of the semester, the author collected up to three de-identified annotated bibliographies from each of the 58 sections of ENG 102. The rubric was applied to up to five annotations in each bibliography, resulting in a total examination of 773 annotations (some sections turned in fewer than 3 samples, and some bibliographies had fewer than 5 annotations).
 Results – The results of the study showed that students struggle with critically evaluating information resources, a finding that supports the existing information literacy assessment literature. The overwhelming majority of annotations consisted of summative information with little evidence that students used any evaluative criteria when they selected an information source. Of the five criteria examined, Relevance to the student’s research topic and Authority were the most commonly used methods of resource evaluation, while Currency, Accuracy, and Purpose were criteria least-often used. The low average scores on the rubric assessment indicate that students are not adequately learning how to apply this set of information literacy skills.
 
 Conclusions – The library instruction sessions for ENG 102 need to move beyond the skills of choosing and narrowing a topic, selecting keywords, and searching in a library database. Students also need more targeted instruction on higher-order skills, particularly how to critically evaluate and question the sources they find. The results of this assessment are being used to refocus the learning outcomes of ENG 102 library sessions so that instruction can better meet student needs. The results are also being used to make the case for further collaboration between ENG 102 and the university library.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.001 | 0.001 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.003 | 0.006 |
| Science and technology studies | 0.001 | 0.000 |
| Scholarly communication | 0.007 | 0.896 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it