Demystifying the NIH Proposal Review Process
Why this work is in the frame
A frame that forgets how it found something cannot be audited. These are the routes that admitted this work.
Bibliographic record
Abstract
Introduction Victoria J. Molfese, Ph.D., holds the Ashland/ Nystrand Chair in Early Childhood Education in the University of Louisville's Center for Research in Early Childhood. Prior to this appointment, Dr. Molfese served as Director of the Office of Research Development and Administration at Southern Illinois University for 13 years and was a member of the Psychology faculty. She was elected President of the Society of Research Administrators International in 1998 and has held SRA's Distinguished Faculty designation since 2002. Dr. Molfese's research focus is on how children learn and factors that influence learning, such as children's home environments and family background characteristics; how schools, teachers, and curriculum influence learning in preschoolers; and how to assess evidence of learning in children from infancy through early elementary grades. Her research has been supported by grants from private and government agencies, including the March of Dimes, the National Institutes of Health (NIH), and the U.S. Department of Education's Institute of Education Science (IES). She has served as a peer reviewer on several study sections, including review panels for the IES, NIH, NIMH (National Institute of Mental Health), the International Dyslexia Association, the National Foundation/March of Dimes, Networks of Centers of Excellence of Canada, the Ontario Mental Health Foundation, and the U.S. Department of Agriculture. In this article, Dr. Molfese answers a series of questions developed by the co-authors to reveal the interpersonal dynamics of an NIH study section and the nitty-gritty details of how an NIH proposal is reviewed. Dr. Molfese's voice of experience as an active researcher and distinguished research administrator provides a candid insider's view of this often mysterious process. Question: You have served as a principal investigator (PI) as well as a research administrator (RA). How do these two roles differ? Answer: PIs are in charge of conducting a research project over which they usually have quite a bit of control--after all, they designed the project and now have a chance to conduct it using grant funds. While all projects involve unexpected events--nothing ever turns out as perfectly as we think it will--most projects tend to involve components that the PI has done before (possibly on a smaller scale) and, therefore, most of the components are familiar. An RA depends on others--PIs or prospective PIs - to set job duties in motion. Because there appear to be endless variations (or variants!) of PIs, RAs encounter projects that often are completely unfamiliar to them. Even projects that could be familiar have PIs who put their personal spin on the project, which tends to make the familiar once again unfamiliar. RA and PIs have to learn to work together with their different motivations. The RA wants to get the proposal submitted to the agency with all compliance issues resolved in plenty of time to make the deadline, while the PI wants to continually rewrite the proposal until the last possible minute to get it perfect before the submission deadline, with no worries about compliance issues. Question: When did you begin reviewing proposals for the NIH? Answer: I began reviewing in 1994, at the suggestion of a friend. I had been asking questions about how people became reviewers and learned that a person can ask NIH to consider them as a possible reviewer. So, I sent my vita and a letter of interest to my program officer, and he contacted me to be part of a standing study section dealing with projects related to childhood development. Question: How did you get selected to serve as a peer reviewer for NIH? Answer: After the initial time I requested to be a reviewer, I found that I was asked by other agencies and other branches of the same agency to be a reviewer for their proposals. Clearly, there is a mechanism by which reviewers are shared by program officers and scientific review administrators (the people who lead the proposal review sessions). …
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.056 | 0.011 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.001 |
| Science and technology studies | 0.002 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.001 | 0.000 |
| Research integrity | 0.000 | 0.002 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it