Concerns That 10,000 Faculty Nationally and Internationally Have About Research Impact: Isn’t It All Academic Research Impact (ARI) If It Comes From The Academy?
Bibliographic record
Abstract
The Paradigmatic Shift Towards Research Impact in the AcademyIn the last four decades there has been increased emphasis for faculty to show and effectively expand the impact of their Academic Research Work*. This trend has occurred nationally, internationally, and is expected to persist through recent national and international events. There are now summits and conferences such as the National Alliance for Broader Impacts Summit (NABI) now called Advancing Research in Society (ARIS) in the United States (US) and International Impact for Science, Humanities, and Social Science Conferences. Many of these international conferences that focus on pushing forward the impact agenda are facilitated by The Network for Advancing & Evaluating the Societal Impact of Science (AESIS).There is rapid proliferation of both new businesses and independent organizations that focus on helping others manage and maximize the impact of their research. These businesses and organizations range from assessment to communications and scholarship, to training individuals on how to extend the reach of their research work. Some examples of these are: Knowledge Translation Australia by Tamika Heiden who also facilitates the largest online impact summit; Jenny Ames Consulting Ltd in the United Kingdom (UK) by Jenny Ames; Institute for Knowledge Mobilization and Peter Norman Levesque Consulting in Canada by Peter Levesque; and Broader Impacts Productions, LLC in the United States (US) by Kirsten Sanford.Simultaneously, impact blogs and blogging have increased in number over the last ten years. These impact blogs are also increasingly gaining support and recognition in the Academy. For example, one of these is the London School of Economic and Political Science (LSE) Impact Blog. The LSE Impact blog is based in the LSE Communications division and is financially supported by the HEIF5 program ran by LSE Knowledge Exchange.Evidence of this paradigmatic shift can also be seen by the number of societal benefitting-like terms, names and phrases now being used around the world. Many of these terms, names, and phrases have been contextualized for maximizing the impact of academic research. These include but are not limited to phrases, terms, and concepts such as: Capacity Building in Africa; Equity in Development in India; Broader Impacts, Broader Implications, Collective Impact, and Relevant or Ultimate Outcomes in the US; the Engagement and Impact Assessment (EI) Framework and Knowledge Exchange in Australia; Knowledge Mobilization in Canada; Valorization in the Netherlands; Harmonious Development in South America; Economic & Social Development and Influence in China; and the Research Excellence Framework (REF) in the European Union. Almost every country, roughly eighty-two percent (82%), uses a societal-benefit name, term, phrase, or concept that indicates ARI is important. This is accompanied by the growing number of professionals and positions to address and actionize these concepts, names, and phrases in academic institutions, agencies, organizations, and governance. For example, there is Susan Renoe, Director of NABI and ARIS who started The Connector formally called the Broader Impacts Network (BIN) in the US; David Phillips who leads an award-winning Knowledge Mobilization Unit in Canada; Julie Bayley who is the Director of Impact Development and Mark Reed who is a professor and transdisciplinary researcher specializing in environmental governance and research impact in peatlands and agri-food systems both in the UK; and Emma Johnston who initiated a Science for Impact Center in Australia focusing on Knowledge Exchange.In addition, some ranking organizations have started to include Overall Impact on Society (OIS) metrics to rank universities and colleges. This includes how university’s and college’s research are benefiting society. For example, in 2019 “THE WORLD University Rankings” facilitated by Times Higher Education, provided their first ever rankings specifically focused on University’s and College’s Impact on Society. These impact rankings are based on the United Nations (UN) seventeen (17) Sustainable Development Goals (SDGs) provided in Figure 1.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.076 | 0.237 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.004 | 0.008 |
| Science and technology studies | 0.012 | 0.015 |
| Scholarly communication | 0.022 | 0.026 |
| Open science | 0.003 | 0.012 |
| Research integrity | 0.015 | 0.015 |
| Insufficient payload (model declined to judge) | 0.025 | 0.007 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".