The news at 404: Archiving and accessing online news content
Bibliographic record
Abstract
As news outlets focus their resources on producing more digital content — in some cases ceasing print production entirely — the questions of if and how online news is archived become ever more pressing. While methods of delivering news content online have developed extensively over the past decade, strategies for preserving this content remain unclear, leaving content vulnerable to erasure. Moreover, the study of these archiving efforts remains largely at the fringe of academic research. Digital news cannot be archived in the same ways as its paper counterpart. The content is archived primarily through news media websites and news aggregation sites such as Factiva, LexisNexis, and ProQuest's Canadian Newsstream, who offer subscription-based access to archived digital content. This study aims to contribute to the understanding of the processes, policies, and perhaps politics, of news archiving in the digital era. This project documents the rates at which online Canadian national newspapers archive digital articles. In addition to monitoring archiving and permanent deletion trends, this project presents the variation in rates at which national news articles are archived by secondary archiving services Canadian Newsstream, Factiva, and LexisNexis. For this project, a sample of 688 online articles was collected over a constructed week from the Globe and Mail, National Post, and CBC News websites. Of the 688 total, 210 stories were from CBC, 240 were from the Globe, and 238 were from the Post. A quantitative content analysis was conducted on the sample to identify potential trends in how and why some articles are excluded from media outlets’ archives and secondary archives. At the end of a five year observation period, 584 of the original 688 articles were still available through the original sites. 55 of the original 688 stories were permanently deleted from the news sites. The study finds significant article loss by Canadian Newsstream, Factiva, and LexisNexis archives with rates of missing articles being three to five times higher for these secondary sites than the news media sites. Several factors impact the archiving rates for articles in the samples. They include: the parceling of licensed content; the use of video content on news websites; and the reliance on wire stories which are not archived at the same rate as content generated in-house. At its root, this project seeks to raise questions about long-term access to information. As the news media transitions further into the digital realm, the ability of individuals to access content becomes less certain. This potentially impacts community memory; the ability of individuals to access their history through media; and reduces the capacity of researchers to conduct news media-based historical analyses. Threats to future access to Canadian digital news media are threats to myriad forms of research that rely on news articles for historical information.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.007 | 0.032 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.007 | 0.013 |
| Science and technology studies | 0.010 | 0.005 |
| Scholarly communication | 0.013 | 0.008 |
| Open science | 0.001 | 0.003 |
| Research integrity | 0.001 | 0.001 |
| Insufficient payload (model declined to judge) | 0.006 | 0.002 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".