Assessment as a Learning Project: Online Surveys with Immediate Formative Feedback
Bibliographic record
Abstract
PurposeThis paper proposes a practical assessment model that focuses on students' performance-related variables, for which information literacy (IL) is designed and delivered.High satisfaction rates about library instruction do not always correlate with user's performance level on evaluating information online and sourcing quality information; the findings from the case study at Cape Breton University (CBU), Nova Scotia, Canada, provides a potential solution to the current challenge in the assessment process. Design/Methodology/ApproachThe author launched a research project called Assessment as Learning Project: Online Surveys with Immediate Formative Feedback at CBU by designing and developing two online surveys for students who are taking courses in the School of Arts and Social Sciences.A 30-minute pre-survey (FluidSurveys & Moodle) includes seventeen exercise questions and consists of eighty-eight pieces of feedback in total.Fifteen questions are new, with the exception of two questions adapted from The Effectiveness of a University's Single-session Information Literacy Instruction (Hsieh & Holden, 2010).The pre-survey allows students to gauge their ability to assess credibility, accuracy, authority, and/or currency.Students can also learn how to develop their IL skills by reading the feedback that pops up immediately after students choose their answer.The pre-survey was delivered before a major assignment during the Winter semester.A 20-minute post-survey, including seven new exercise questions, was delivered at the end of the semester.The post-survey was designed to remind students of what they learned from the pre-survey.Many exercise questions include a quotation from a source.The sources vary and include comments on a historical figure, biography that includes historical accounts on non-Western cultures, data that pertains to emotionally disturbing experiences, and news articles that can commonly be shared through social media (e.g., http://yumetsub.site11.com/quiz_sample.html).This paper includes an analysis on the data from the surveys, for example, students' progress on their IL skills. FindingsThe results from the surveys indicate three potential benefits.First, given a reflective learning opportunity, students can recognize the strengths and weaknesses of their information literacy skills and learn how to improve their skills.Second, students can learn what kind of questions they can ask librarians because feedback indicates what kind of questions librarians can answer.Third, instructors/faculty can save the time to repeatedly teach students not to use certain types of sources in their assignments since the surveys explain and bring awareness to this for the students. Practical implications/ValueIf students are overconfident in their abilities to search, find and evaluate information, they risk underestimating the challenge of finding and using quality information online; thus helping them foster critical self-reflection is one way to mitigate the problem.An example is demonstrated by the assessment surveys that include various types of sources and provide guidance of how to consult a person, tool, or platform as a source of information.The assessment was tested in various disciplines in the School of Arts and Social Sciences.The results indicate a number of potentials to use and develop the surveys: for example, they can be incorporated into a curriculum assignment.
Stored with the screening record, where it is evidence for the labels above.
How this classification was reachedexpand
The three-model screen
all 5,600 screened works →1 of 3 models called this metaresearch. This work is contested: it sits on the field's empirical boundary, and whether it counts depends on which model you asked. It is one of the 51 works in the disagreement dossier.
Academic library study of information literacy instruction and assessment: how students find and evaluate sources; LIS work on information behaviour, though the object is student learning rather than researchers.
It evaluates students' information-literacy learning rather than researchers' information behavior.
Student information-literacy pedagogy/assessment at a university, not information behaviour of researchers.
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.077 | 0.117 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.002 | 0.002 |
| Science and technology studies | 0.001 | 0.001 |
| Scholarly communication | 0.003 | 0.003 |
| Open science | 0.002 | 0.003 |
| Research integrity | 0.001 | 0.001 |
| Insufficient payload (model declined to judge) | 0.005 | 0.003 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".