YouTube or You Lose: Grand Challenges Canada Explores Whether Scientists Are Ready for Web-Based Grant Competitions
Bibliographic record
Abstract
It is not hard to trace the influence of technology on the way we read the literature or give scientific presentations. Not so long ago, chemists used hard copies of Chemical Abstracts to find papers and sticks of chalk to deliver talks. Only over the past decade have computer presentations become the norm. In contrast, the way that grants are evaluated has remained relatively unchanged: scientists submit written proposals that are then evaluated by committees of scientists in the field. Might this process soon change as well? The not-for-profit organization Grand Challenges Canada (GCC) recently sponsored a competition in which researchers presented audacious ideas to attack problems related to global \nhealth (Figure 1). In its search for bold ideas from scientists, the GCC organization tested a bold idea as well: each proposal had to be accompanied by a 2-min-long video for public consumption on the Internet. Web users were encouraged not only to view these video summaries but to participate in the evaluation of the proposals by means of clicking on a “thumbs up” button (similar to the “like” buttons found on YouTube and Facebook). The votes from the public video were used by GCC to evaluate each applicant’s ability to “engage the public and increase awareness in \nthe grand challenges facing global health today”.^1 The competition collected over 180,000 votes and over 100,000 unique online visits from 156 countries in a mere 4 weeks—staggering statistics for scientific videos. While each applicant also submitted a written version of the proposal, which was privately evaluated by “standard” peer-review, the public video feature was one of the first direct implementations of Web 2.0 technology (user-interactive \nsites and applications) to evaluate scientific proposals. The competition raises an important question: to what extent, if any, should Web 2.0 technology or other direct evaluation by the public be used to determine the outcome of scientific grant proposals?
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.000 | 0.011 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".