MétaCan
Menu
Back to cohort
Record W4413161191 · doi:10.24908/ijesjp.v12i1.19535

Reflections on the 2024 ESJP review process: Sam

2025· article· en· W4413161191 on OpenAlexfundvenueno aff
Samantha Hoang

Bibliographic record

VenueInternational Journal of Engineering Social Justice and Peace · 2025
Typearticle
Languageen
FieldSocial Sciences
TopicDelphi Technique in Research
Canadian institutionsnot available
FundersQueen's UniversityDanmarks Tekniske UniversitetNational Science Foundation
KeywordsProcess (computing)Computer scienceProgramming language

Abstract

fetched live from OpenAlex

My name is Sam Hoang, and I am an Assistant Professor at Seattle University. ESJP 2024 was my first experience with a conference of this particular format. When learning about this conference from my colleague, Yen, I was curious about the invitation to submit artifacts that do not take the form of a traditional research paper. We were interested in writing a paper reflecting on our journeys to becoming faculty as women of color. Citing external sources when we were writing about our experiences did not make sense since our experiences are unique and not something that should require citation. Citations are something that I would expect to include in submissions to any other conference venue so the explicit invitation to submit nontraditional artifacts was very freeing in this aspect and made it a lot easier to think that our idea for a paper reflecting on our journey through academia would be an appropriate paper format and topic for the conference. It really freed up me and my coauthors to write without feeling like we were required to cite other sources to corroborate our experiences. With regards to the review process, we did not actually get many reviews in person since the discussion table included other authors that all reviewed each other. Because there were three papers to discuss in a short period of time, we did not have a chance to discuss our paper at all. Additionally, our reviewer was also one of the organizers and was busy with other tasks to keep the conference running during the discussion period. Therefore, our reviews were shared with us mainly via email. While I like the idea of doing a peer review session during the conference, for our paper specifically, we did not have a great experience with regards to the review session held during the conference. One major advantage to this reviewing structure is that we got to know our reviewer before seeing their review and vice versa. It made the review process feel warmer than I experienced with other conference and journal submissions. Getting to know our reviewer prior to receiving their review helped our reviewer understand our reasonings behind the chosen format of our submission. Additionally, presenting our paper before receiving our reviews also created an opportunity to present out work to the reviewer making it a much richer experience by adding onto what is written in the manuscript. If this format is pursued in the future, it would be helpful if there was more time to discuss the papers during the conference. Depending on the number of submissions, it would be helpful to have more presentation sessions and only have 1-2 papers to discuss in each discussion period so that there is more time for reviewers and authors to talk about their papers. It would also be helpful if there were other times or spaces designated in the schedule to give authors and reviewers more time to chat about their papers outside of the presentation and discussion session. With regards to the conference itself, I really enjoyed having various art materials available at the venue. Arts are not usually treated as relevant to engineering so it was awesome to see the arts be a major component of this conference. While I did not know what to do with the art materials at first, I was inspired by seeing other people using the materials. Once I started using the materials, I found it helpful for managing my fidgeting. I generally have a hard time sitting still so interacting with the art materials was a great way for me to channel my excess energy while still listening to the presented. (communication by email on 2025-04-22)

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.

How this classification was reachedexpand

Full frame distilled prediction

Teacher imitation

Not calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.

metaresearch head score (Codex)0.001
metaresearch head score (Gemma)0.002
Version: codex-gemma-dda1882f352aValidation status: machine_predicted_unvalidated
Candidate categoriesnone
Consensus categoriesnone
DomainCandidate signal: none · Consensus signal: none
Study designCandidate signal: Not applicable · Consensus signal: none
GenreCandidate signal: Empirical · Consensus signal: none
Teacher disagreement score0.918
Threshold uncertainty score0.275

Codex and Gemma teacher scores by category

CategoryCodexGemma
Metaresearch0.0010.002
Meta-epidemiology (narrow)0.0000.000
Meta-epidemiology (broad)0.0000.000
Bibliometrics0.0000.000
Science and technology studies0.0000.000
Scholarly communication0.0000.000
Open science0.0000.000
Research integrity0.0000.000
Insufficient payload (model declined to judge)0.0000.000

Machine scores (provisional)

The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.

Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.

Opus teacher head0.095
GPT teacher head0.513
Teacher spread0.418 · how far apart the two teachers sit on this one work
Validation statusscore_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it

Classification

machine, unvalidated

Machine predicted; a candidate call from one teacher head, not a consensus.

The models applied no category: nothing in the taxonomy fit this work.
Study designNot applicable
Domainnot available
GenreEmpirical

How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".

Quick stats

Citations0
Published2025
Admission routes2
Has abstractyes

Explore more

Same venueInternational Journal of Engineering Social Justice and PeaceSame topicDelphi Technique in ResearchFrench-language works237,207