← Back to cohort
Proceedings of the 4th World Conference on Research Integrity
Susan Patricia O’Brien, Danny Chan, Fks Leung, Eun Jung Ko, Jin Sun Kwak, TaeHwan Gwon, Ji Min Lee, Min‐Ho Lee, Helga Nolte, Michael Gommel, Gerlinde Sponholz, Yordanka Krastev, Yamini Sandiran, Julia Connell, Nicky Solomon, Urša Opara Krašovec, Renata Šribar, Brian C. Martinson, Carol R. Thrush, C. K. Gunsalus, Ivan Oransky, Adam Marcus, Chris Graf, Verity Warne, Edward Wates, Sue Joshua, Miguel Roig, Michael D. Mumford, Mylène Deschênes, Catherine Olivier, Raphaëlle Dupras-Leduc, Zoë Hammatt, Raju Tamot, Robin Parker, Cynthia S. Ricard, Loc Nguyen-Khoa, Sandra L. Titus, Karsten Klint Jensen, Simon Godecharle, Benoît Nemery, Kris Dierickx, Mickey Gjerris, Maud Marion Laird Eriksen, Jeppe Berggren Hoej, Nicholas H. Steneck, Camille Nebeker, Michael Kalichman, Elizabeth Mejia Booen, Blanca Azucena Pacheco, Rebeca Espinosa Giacinto, Sheila Castaneda, Ding Li, Chen Qiong, Guoli Zhu, Zhonghe Sun, Badaruudin Abbasi, Parthasarathi Ganguly, Barna Ganguly, Noémie Aubert Bonn, Danièle Bourcier, J. Bordé, Michèle Leduc, Chien Chou, Sophia Jui-An Pan, Louis Tiefenauer, Daniel P. Barr, Paul Taylor, Lyn Horn, Deborah C. Poff, Martina Mavrinac, Gordana Brumini, Mladen Petrovečki, Christiane Coelho Santos, Sónia Vasconcelos, Rachael Lammey, Chris Hartgerink, Marcel A. L. M. van Assen, Jelte M. Wicherts, Hanne Silje Hauge, Aaron S. Manka, Raffael Iturrizaga, Nicole Foeger, A. Roxana Lescano, Claudio F. Lanata, Gissella Vasquez, Leguia Mariana, Marita Silva, Mathew Kasper, Claudia Montero, Daniel G. Bausch, Andrés G. Lescano, F Blom, L.M. Bouter, L.O.K. Lategan, Gustavo Fitas Manaia, Wayne T. McCormack, William L. Allen, Shane Connelly, Joshua S. Crites, Jeffrey A. Engler, Victoria Freedman, Cynthia Garvan, Paul Haidet, Joel W. Hockensmith, W. D. McElroy, Erik Sander, Rebecca L. Volpe, Michael F. Verderame, Snežana Krstić, Louise Handy, Debra Schaller-Demers, I. Lerouge, Gerard Cielen, Liliane Schoofs, Ana Marušić, Flaminio Squazzoni, David L. Vaux, Khalid Alwazzan, Ibrahim A. Alorainy, Nolan O’Brien, Suzanne Guérin, Philip Dodd, Frank Wells, Catherine Blewett, Fredric M. Litto, Erika Löfstrôm, Katrien Maes, Tony Peatfield, Olivier Boehme, Cinzia Caporale, Daniele Fanelli, John M. Carfora, Eric G. Strauss, William S. Lynn, Dieter De Bruyn, Bracke Nele, Katrien De Gelder, Stefanie Van der Burght, José Roberto Lapa e Silva, Helene Ingierd, Mark S. Frankel, Daniel R. Vasgird, Stephanie J. Bird, Mark Yarborough, Harold R. Garner, Lisa Campo‐Engelstein, Zubin Master, Elise Smith, David B. Resnik, Bryn Williams–Jones, Mario Malički, Ana Utrobičić, John P. A. Ioannidis, Gerben ter Riet, Tom Walley, Jenny T. van der Steen, Martha M. Sorenson, Francisco Prosdocimi, Hatisaburo Masuda, Edson H. Watanabe, José Carlos Pinto, Marisa Palácios, Jacqueline Leta, Adalberto Vieyra, André Pinto, Maurício Ferreira Santana, Rosemary Sadami Arai Shinkai, Boris Yudin, Étienne Vergès, Anne-Sophie Brun-Wauthier, Géraldine Vial, Torkild Vinther, V. Bähr, Dena Plemmons, Rebecca Lynn Davies, Katrina Laube, Cynthia Scheopner, John C. Galland, Hervé Maisonneuve, Évelyne Decullier, Brian Noble, Anja Gilis, David J. Gallacher, Tom Lavrijssen, Malwitz David, Malini Dasgupta, Hans Mols, Tony Savard, Eric Mah, Elizabeth Wager, Sabine Kleinert, Véronique Kiermer, Andrew L. Hufton, Melanie Clyne, Renan Moritz Varnier Rodrigues de Almeida, Aldo José Fontes-Pereira, Fernanda Catelani, Karina de Albuquerque Rocha, Elizabeth Iorns, W. C. Gunn, Lucienne Letellìer, Cornelia Malherbe, Takehito Kamata, Vidar Enebakk, Lynn Penrod, Thomas Nørgaard, Charlotte Elverdam, Yves Fassin, Cynthia Fekken, Jamal A. Adam, Melissa S. Anderson, Brett Buttliere, Trish Groves, Daniel Shanahan, Iveta Simera, Shona Kirtley, Eleana Villanueva, Caroline Struthers, Angela MacCarthy, Douglas G. Altman, Elena Grebenshchikova, Bronwyn Greene, Ted Rohr, Stephanie Harriman, Krista Varantola, Ursa Opara, Torkild Vinther, Jon Rokne, Gianluca Setti, G. Gordon MacPherson, Ellen Hazelkorn, Carthage Smith, Robert H. McLaughlin, Tatiana Duque Martins, Tetsuya Tanimoto, Ragnvald Kalleberg, Tajammul Hussein, Ping Sun, Ovid J. L. Tzeng, Susan Zimmerman, Elizabeth Heitman, Maria Andrea Rocio del Pilar Contre Nieto, Eric A. Fong, Al Wilhite, Dèsirée Motta-Roth, Joeri K. Tijdink, Yvo M. Smulders, Logan M. Steele, Logan L. Watts, James F. Johnson, Lee Williams, Sufia N. Amini, Lotty Hooft, Halil Kilicoglu, Janneke van Seters, H.J.P. Eijsackers, Fons Voragen, Akke van der Zijpp, F.W.A. Brom, Badaruddin Abbasi, Naif Nasser Almasoud, Adri Labuschagne, Johanne Severinsen, Espen Engh, Daham Ismail Alani, H. J. Kamaruzaman Jusoff, Anita M. Gordon, Helen C. Harton, Limbanazo Matandika, Aline Carolina de Oliveira Machado Prata, Mark W. Neff, Rodrigo Costas, Vincent Larivière, Douglas Leonardo Gomes Filho, Diego Oliveira Guedes, Niklas Keller, Markus A. Feufel, Nikolas Offenhauser, Maria Kowalczuk, Elizabeth Moylan, Katie Speanburg, Mariusz Lubomirski, David Malwitz, Anja Gillis, Maria Betânia de Freitas Marques, Laressa Lima Amâncio, Raphaela Dias Fernandes, Oliveira Patrocínio, Cláudia Maria Correia Borges Rech, Adriana Nascimento Sousa, Leila Posenato García, Jigisha Patel, Farida Lada, Giuseppe Di Capua, Silvia Peppoloni, James M. DuBois, John T. Chibnall, Jillon Van der Wall, Raymond C. Tait, Jacob Holen, Å. Johannessen, Torunn Ellefsen, Laura Beranzoli, Paulo S.L. Beirão
Bibliographic record
VenueResearch Integrity and Peer Review · 2016
Typearticle
Languageen
FieldMedicine
TopicArtificial Intelligence in Healthcare and Education
Canadian institutionsGovernment of CanadaDalhousie UniversityTreasury Board of Canada SecretariatUniversity of CalgaryMcGill UniversityUniversity of AlbertaUniversité de MontréalBrandon UniversityQueen's UniversityFonds de recherche du QuébecFonds de Recherche du Québec - Santé
FundersNational Research Foundation of KoreaUniversity of New South WalesNational Research Foundation
KeywordsResearch integrityPolitical scienceEngineering ethicsComputer scienceEngineering
Abstract
on Research Integrity was held in
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Direct model labels (unvalidated)
Per-model category and study-design labels from the labeling rounds. They are machine output, unvalidated, and the disagreement between models ships as data. No study design here is MEDLINE-validated yet.
| Model arm | Categories | Study design | Confidence |
|---|---|---|---|
| gemma | Research integrity Domain: not available · Genre: Other About the Canadian research system: no · About a Canadian topic: no | Not applicable | low |
| gpt | Research integrity Domain: not available · Genre: Other About the Canadian research system: no · About a Canadian topic: no | Not applicable | low |
models agreeAgreement compares identical category sets and study designs across arms.
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
metaresearch head score (Codex)0.197
metaresearch head score (Gemma)0.227
Version: metacan-v3-hybrid-931329e0061cValidation status: machine_predicted_unvalidated
Candidate categoriesMetaresearch, Research integrity
Consensus categoriesMetaresearch
DomainCandidate signal: Methods · Consensus signal: none
Study designCandidate signal: Not applicable · Consensus signal: Not applicable
GenreCandidate signal: Other · Consensus signal: none
Teacher disagreement score0.982
Threshold uncertainty score0.990
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.197 | 0.227 |
| Meta-epidemiology (narrow) | 0.002 | 0.002 |
| Meta-epidemiology (broad) | 0.007 | 0.004 |
| Bibliometrics | 0.010 | 0.005 |
| Science and technology studies | 0.011 | 0.012 |
| Scholarly communication | 0.047 | 0.019 |
| Open science | 0.005 | 0.017 |
| Research integrity | 0.018 | 0.027 |
| Insufficient payload (model declined to judge) | 0.077 | 0.034 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
Opus teacher head0.821
GPT teacher head0.615
Teacher spread0.206 · how far apart the two teachers sit on this one work
Validation status
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedLabeled directly by 2 models reading the full record.
Study designNot applicable
Domainnot available
GenreOther
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".
Quick stats
Citations12
Published2016
Admission routes1
Has abstractyes