MétaCan
Menu
Back to cohort
Record W2092746927 · doi:10.1080/13645579.2011.638221

Exploring the validity of electronic newspaper databases

2011· article· en· W2092746927 on OpenAlex

Why this work is in the frame

A frame that forgets how it found something cannot be audited. These are the routes that admitted this work.

aboutThe title or abstract carries a Canadian signal from the geographic lexicon.
no affNo Canadian affiliation: this work is invisible to an affiliation-only frame.
No Canadian affiliation. An affiliation-only frame, the usual design, would never have seen this work. It is one of the works that make the case for inverting the frame.

Bibliographic record

VenueInternational Journal of Social Research Methodology · 2011
Typearticle
Languageen
FieldSocial Sciences
TopicMedia Studies and Communication
Canadian institutionsnot available
Fundersnot available
KeywordsNewspaperLexisContent analysisDatabaseAdvertisingPolitical scienceComputer scienceSociologyBusinessSocial science

Abstract

fetched live from OpenAlex

Abstract Do electronic newspaper databases contain all of the stories that appear in the print edition? And does this depend on the database used? To explore these questions, we collected print copies of newspapers from cities across the USA and Canada. We compared coverage of two topics in these newspapers with the coverage obtained from keyword searches in three electronic newspaper databases. We conclude that the stories obtained through electronic searches are consistent across databases but can vary from the print source. Importantly, national and international coverage is more likely to be missing than local or statewide/provincial coverage. Keywords: content analysisvalidityelectronic databasesnewspapers Notes 1. ProQuest began distributing electronic material on CD-ROM in the 1980s and moved to electronic distribution in 1996. Lexis-Nexis began providing desktop access to select newspapers using personal computers in 1979; web-based distribution began in 1994 and catered to legal professionals. It is unclear when NewsBank began distributing newspaper content electronically. 2. These dates were chosen for convenience but seemed fairly unremarkable in terms of news content. The big national story in most newspapers was Tea Party protests across the USA. The big international story was piracy off the coast of Somalia. 3. This does raise the issue of whether one's results are consistent over time. For another project, we repeated these searches on various days and found very little variation at all depending on the search day. Indeed, we got the exact same number of hits when we repeated the search on 18 May as on 11 May. 4. The Boston Herald does not have a 'metro' section per se, as it is a tabloid. We therefore searched for articles from the first and second pages of the newspapers. At the time we conducted this analysis, the Des Moines Register was no longer available in any of the three databases. 5. To be consistent with our electronic searchers, the coder did not note mentions of 'Mr. Obama,' 'President Obama,' or other variations on his name in the print copies of the newspapers. 6. Each of the electronic databases allows one to search wire service stories separately, which may be helpful in determining whether a newspaper uploads wire service content to the database. 7. Interview was conducted via email on 21 October 2009. 8. Graber relies upon data from The State of the News Media 2004 report, produced by the Project for Excellence in Journalism. 9. Quite simply, since approaches to uploading content vary from newspaper to newspaper and from database to database, we mean that the researcher would need to sit down and scan all of the results retrieved to see whether any letters to the editor, editorials, photo captions, or wire service stories (i.e. those types of content often excluded from an electronic database) appear or not. 10. Google. News (publishers) help. 'Restricted Content.' Retrieved 16 August 2011, from http://www.google.com/support/news_pub/bin/answer.py?hl=en&answer=68331.

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.

Direct model labels (unvalidated)

Per-model category and study-design labels from the labeling rounds. They are machine output, unvalidated, and the disagreement between models ships as data. No study design here is MEDLINE-validated yet.

Model armCategoriesStudy designConfidence
gemmaMetaresearch
Domain: Methods · Genre: Empirical
About the Canadian research system: no · About a Canadian topic: no
Observationalhigh
gptno category
Domain: not available · Genre: Empirical
About the Canadian research system: no · About a Canadian topic: no
Observationalhigh
models splitAgreement compares identical category sets and study designs across arms.

Full frame distilled prediction

Teacher imitation

Not calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.

metaresearch head score (Codex)0.021
metaresearch head score (Gemma)0.011
Version: codex-gemma-dda1882f352aValidation status: machine_predicted_unvalidated
Candidate categoriesMetaresearch
Consensus categoriesnone
DomainCandidate signal: none · Consensus signal: none
Study designCandidate signal: Theoretical or conceptual · Consensus signal: none
GenreCandidate signal: Empirical · Consensus signal: Empirical
Teacher disagreement score0.712
Threshold uncertainty score0.997

Codex and Gemma teacher scores by category

CategoryCodexGemma
Metaresearch0.0210.011
Meta-epidemiology (narrow)0.0000.000
Meta-epidemiology (broad)0.0000.000
Bibliometrics0.0000.000
Science and technology studies0.0000.001
Scholarly communication0.0000.000
Open science0.0010.000
Research integrity0.0000.001
Insufficient payload (model declined to judge)0.0000.000

Machine scores (provisional)

The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.

Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.

Opus teacher head0.935
GPT teacher head0.628
Teacher spread0.307 · how far apart the two teachers sit on this one work
Validation statusscore_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it