The Quality of Canadian and U.S. Government Health Documents Remains Unchallenged Until Better Research Can Be Undertaken. A review of: Lambert, Frank. “Assessing the Authoritativeness of Canadian and American Health Documents: A Comparative Analysis Using Informetric Methodologies.” Government Information Quarterly 22.2 (2005): 277‐96.
Bibliographic record
Abstract
Objective ‐ To assess by means of citation analysis whether the public trust afforded health documents published by the Canadian and U.S. governments is appropriate, and to ascertain whether differences in the respective health care systems influence how publications are produced. Design – Comparative study. Setting – The Canadian Depository Service Program (DSP) and the U.S. Department of Health and Human Services (DHHS) websites. Subjects – One hundred sixty‐six electronic documents sourced from the DSP website, and 284 electronic documents sourced from the DHHS website. Methods – Subjects were randomly selected from repositories offering the most comprehensive collections. Documents with evidence of references to other works used in preparation were separated from thos ewithout such characteristics. Data variables were collected from documents with evidence of references. Statistical analysis of the data was undertaken. Main results – Of the respective samples, 89(53%) from the DSP and 109 (38.4%) from the DHHS contained references. Personal authors were identified in 46 (51.7%) and 63(58%) of the respective subsets. Handbooks and guidebooks accounted for the largest portion of the DSP subset (29; 32.6%) and government periodicals were the largest constituent of the DHHS subset (41; 37.6%). Scholarly journals were the most common reference type for both the DSP (44%) and the DHHS (58.5%) subsets. The number of references per document was widely dispersed for both subsets; the DSP mean was approximately 64 (SD=114.68) and the DHHS was 73.71 (SD=168.85). Kruskal‐Wallis subset analysis of median number of references by document type found differences generalizable to the entire DSP (p<0.01) and DHHS (p<0.01) populations. Health Canada Reports, handbooks, andg uidebooks contained significantly more references than periodical articles or factsheets. Certain DHSS documents, classified as “other,” contained more references than periodical articles. Canadian documents were more likely to contain references than U.S. documents (p<0.01). Comparison of documents to determine whether one country employs more rigorous citation practices did not produce statistically significant results. U.S. Federal Government documents are more likely to be referenced in other U.S. government health documents, compared to Canadian publications (p<0.01). The presence of references in documents from either country significantly affected likelihood of being cited by web authors(p<0.01). Conclusion – Significant differences in reference use frequencies between DSP and DHHS documents challenges Foskett’s stance that documents of value contain references (Foskett). Use of peer‐reviewed scholarly journals for both DSP and DHHS publications was reassuring, suggesting a fairly rigorous publication standard. Reliance of DHHS publications upon federal government documents remains unclear. Referencing of DSP documents, irrespective of reference usage suggests a level of trust towards Canadian government health publications. Web authors appear more comfortable citing referenced DHHS documents. Further study could involve the examination of reference frequency byj ournal compared against journal impact factors.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.056 | 0.270 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.002 | 0.001 |
| Bibliometrics | 0.048 | 0.092 |
| Science and technology studies | 0.012 | 0.012 |
| Scholarly communication | 0.029 | 0.012 |
| Open science | 0.003 | 0.004 |
| Research integrity | 0.002 | 0.002 |
| Insufficient payload (model declined to judge) | 0.015 | 0.003 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".