YouTube or You Lose: Grand Challenges Canada Explores Whether Scientists Are Ready for Web-Based Grant Competitions
Bibliographic record
Abstract
It is not hard to trace the influence of technology on the way we read the literature or give scientific presentations. Not so long ago, chemists used hard copies of Chemical Abstracts to find papers and sticks of chalk to deliver talks. Only over the past decade have computer presentations become the norm. In contrast, the way that grants are evaluated has remained relatively unchanged: scientists submit written proposals that are then evaluated by committees of scientists in the field. Might this process soon change as well? The not-for-profit organization Grand Challenges Canada (GCC) recently sponsored a competition in which researchers presented audacious ideas to attack problems related to global \nhealth (Figure 1). In its search for bold ideas from scientists, the GCC organization tested a bold idea as well: each proposal had to be accompanied by a 2-min-long video for public consumption on the Internet. Web users were encouraged not only to view these video summaries but to participate in the evaluation of the proposals by means of clicking on a “thumbs up” button (similar to the “like” buttons found on YouTube and Facebook). The votes from the public video were used by GCC to evaluate each applicant’s ability to “engage the public and increase awareness in \nthe grand challenges facing global health today”.^1 The competition collected over 180,000 votes and over 100,000 unique online visits from 156 countries in a mere 4 weeks—staggering statistics for scientific videos. While each applicant also submitted a written version of the proposal, which was privately evaluated by “standard” peer-review, the public video feature was one of the first direct implementations of Web 2.0 technology (user-interactive \nsites and applications) to evaluate scientific proposals. The competition raises an important question: to what extent, if any, should Web 2.0 technology or other direct evaluation by the public be used to determine the outcome of scientific grant proposals?
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.006 | 0.018 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.002 | 0.003 |
| Science and technology studies | 0.013 | 0.003 |
| Scholarly communication | 0.014 | 0.006 |
| Open science | 0.001 | 0.004 |
| Research integrity | 0.003 | 0.003 |
| Insufficient payload (model declined to judge) | 0.045 | 0.008 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".