Trends in the Use of Promotional Language (Hype) in Abstracts of Successful National Institutes of Health Grant Applications, 1985-2020
Bibliographic record
Abstract
Importance: The integrity of the grant application process is important to the success of the entire research enterprise. However, little information is available concerning the prevalence and evolution of subjective or promotional language ("hype") that has the potential to undermine objectivity in the writing and evaluation of grant applications. Objective: To assess changes over time in the use of hype in abstracts of National Institutes of Health (NIH) grant applications. Design, Setting, and Participants: This cross-sectional study assessed the prevalence of promotional adjectives in abstracts in the NIH archive from 1985 to 2020. Main Outcomes and Measures: From all abstracts in the NIH RePORTER (Research Portfolio Online Reporting Tools: Expenditures and Results) archive, adjectives were automatically extracted, and their frequencies in the most recent year (2020) were assessed relative to the start year (1985). Adjectives that shifted significantly in frequency and that carried a promotional sense (ie, hype) were retained, and patterns of change were assessed by plotting yearly frequencies (1985-2020). By grouping the adjectives based on shared semantic properties, broad meanings commonly expressed by hype were identified. Absolute change was measured as the difference in normalized frequency between 1985 and 2020. Relative change was measured as the percentage change in normalized frequency in 2020 relative to 1985, or the first year of occurrence. Results: In total, 901 717 abstracts were analyzed and 139 adjective forms were identified as hype. Among these 139 adjective forms, 130 hype adjectives increased in frequency by 7690 words per million (wpm) (mean [SD] relative increase, 1378% [3132%]), while 9 hype adjectives decreased in frequency by 686 wpm (mean [SD] relative decrease, 44% [18%]). The largest absolute increases were for the terms novel (1054 wpm), critical (555 wpm), and key (461 wpm), while the largest relative increases were for the terms sustainable (25 157%), actionable (16 114%), and scalable (13 029%). Hype most often serves to promote the significance, novelty, scale, and rigor of a project; the utility of the expected outcomes; the qualities of the investigators and research environment; and the gravity of the problem; as well as conveying the personal attitudes of the applicants. Conclusions and Relevance: Levels of hype in successful NIH grant applications have increased over time from 1985 to 2020. The findings in this study should serve to sensitize applicants, reviewers, and funding agencies to the increasing prevalence of subjective, promotional language in funding applications.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.006 | 0.002 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.001 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".