Advancing bibliometric assessment of research productivity: an analysis of US Departments of Family Medicine
Bibliographic record
Abstract
INTRODUCTION Measurement of family medicine research productivity has lacked the replicable methodology needed to document progress. AIM In this study, we compared three methods: (1) faculty-to-publications; (2) publications-to-faculty; and (3) department-reported publications. METHODS In this cross-sectional analysis, publications in peer-reviewed, indexed journals for faculty in 13 US family medicine departments in 2015 were assessed. In the faculty-to-publications method, department websites to identify faculty and Web of Science to identify publications were used. For the publications-to-faculty method, PubMed's author affiliation field were used to identify publications, which were linked to faculty members. In the department-reported method, chairs provided lists of faculty and their publications. For each method, descriptive statistics to compare faculty and publication counts were calculated. RESULTS Overall, 750 faculty members with 1052 unique publications, using all three methods combined as the reference standard, were identified. The department-reported method revealed 878 publications (84%), compared to 616 (59%) for the faculty-to-publications method and 412 (39%) for the publication-to-faculty method. Across all departments, 32% of faculty had any publications, and the mean number of publications per faculty was 1.4 (mean of 4.4 per faculty among those who had published). Assistant Professors, Associate Professors, Professors and Chairs accounted for 92% of all publications. DISCUSSION Online searches capture a fraction of publications, but also capture publications missed through self-report. The ideal methodology includes all three. Tracking publications is important for quantifying the return on our discipline's research investment.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.029 | 0.143 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.002 |
| Bibliometrics | 0.041 | 0.074 |
| Science and technology studies | 0.001 | 0.001 |
| Scholarly communication | 0.004 | 0.003 |
| Open science | 0.001 | 0.003 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.002 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".