Debunked: Data Literacy For Adults Evaluation Report
Bibliographic record
Abstract
ADAPT is a world-leading Science Foundation Ireland Research Centre for AI-Driven Digital Content Technology. ADAPT has a dedicated team for Education and Public Engagement (EPE), which aims to inspire the Irish public to learn about emerging technologies that enhance engagement in our digital world and to have a voice on the future of this vital area of research. Data is at the heart of ADAPT’s activities, and data literacy is paramount for citizens to critically engage with emerging technologies. The increasing pervasiveness of digital content and technology in our everyday lives means that young people and adults need to have the skills to think critically about data and make informed decisions, simply in order to thrive in our always-connected world. In 2021 - 2022, ADAPT devised and delivered a series of one-off workshops for adults to promote awareness of the importance of the topic and an opportunity to improve their data literacy skills in an interactive, social space. Originally titled DALIDA, the series was launched publicly with the more memorable name ‘Debunked’. Data literacy is a broad term, encompassing media and social media literacy, as well as numerical literacy. Due to COVID-19 restrictions in 2021, Debunked ran as an online workshop series, rather than the in-person programme initially conceived. It was led by ADAPT researcher Dr Christophe Debruyne (who moved to University of Liége during the project) and the ADAPT EPE Team led by Laura Grehan and Project Manager Anne Kearns, with facilitation support from 23 other ADAPT researchers. Debunked also involved collaborators from Trinity College Dublin including Dr Ciarán O’Neill (Ussher Associate Professor in Nineteenth-Century History and former TCD Community Liaison Officer) and Ms. Mary Colclough (Community & Enterprise Engagement Manager). The primary aim of Debunkedwas to help people navigate misinformation, disinformation and malinformation online by improving their data literacy skills through these workshops. This report presents a formative evaluation of the inaugural Debunked series. Data was collected through a pre- and post-survey of workshop attendees, as well as semi-structured interviews with participants, programme team and collaborators. The results indicate that despite operational challenges encountered due to the move online as a result of Covid-19 public health restrictions, the ADAPT team were able to capitalise on strong workshop content developed in consultation with the public. Workshops made excellent use of narrative and storytelling that covered Irish history and memes, as well as print and online media, graphs and statistics. The resulting responses from participants covered a range of emotions, highlighting the strongly affective nature of practical and personal reflection on data literacy
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.022 | 0.042 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.002 | 0.004 |
| Bibliometrics | 0.001 | 0.001 |
| Science and technology studies | 0.002 | 0.001 |
| Scholarly communication | 0.003 | 0.002 |
| Open science | 0.001 | 0.003 |
| Research integrity | 0.001 | 0.003 |
| Insufficient payload (model declined to judge) | 0.018 | 0.005 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".