{"id":"W4414954382","doi":"10.2172/2997408","title":"Summer Internship Report: ARA2 Benchmarking","year":2025,"lang":"en","type":"report","venue":"","topic":"Human Resource Development and Performance Evaluation","field":"Psychology","cited_by":0,"is_retracted":false,"has_abstract":false,"ca_institutions":"University of Waterloo","funders":"Lawrence Livermore National Laboratory; U.S. Department of Energy","keywords":"","routes":{"ca_aff":true,"ca_fund":false,"ca_venue":false,"about_ca":false,"invisible_to_affiliation_only":false},"retraction":null,"screen":null,"direct_labels":[],"prediction":{"model_version":"codex-gemma-dda1882f352a","candidate_categories":["metaepi_narrow","insufficient_payload"],"consensus_categories":["insufficient_payload"],"category_scores_codex":[0.002598618,0.0003796066,0.00050612,0.0005650534,0.0001233181,0.00008165907,0.0003340973,0.0006489766,0.06611208],"category_scores_gemma":[0.0001628463,0.0003419626,0.0002196934,0.000224672,0.00004387407,0.00005175573,0.0001324937,0.0007132072,0.001407911],"about_ca_system_candidate":false,"about_ca_system_consensus":false,"about_ca_system_score_codex":0.0003366848,"about_ca_system_score_gemma":0.0008589838,"about_ca_topic_candidate":false,"about_ca_topic_consensus":false,"about_ca_topic_score_codex":0.0004873403,"about_ca_topic_score_gemma":0.0003070669,"domain_scores_codex":[0.9967853,0.0001106772,0.0009976719,0.0007223518,0.000945071,0.0004388635],"domain_scores_gemma":[0.9980542,0.0001263129,0.000544705,0.0007943157,0.0004069697,0.00007346911],"domain_codex":null,"domain_gemma":null,"domain_candidate":null,"domain_consensus":null,"study_design_codex":"not_applicable","study_design_gemma":"not_applicable","study_design_scores_codex":[0.00003021457,0.00004090695,0.04870749,0.0001917446,0.0003424525,0.0003423946,0.0009991274,0.000001083382,0.000001238753,0.0002635747,0.8412255,0.1078543],"study_design_scores_gemma":[0.0002117926,0.00003378088,0.06494432,0.0004310892,0.0001016209,0.0001933886,0.0001779868,0.00001640059,0.000006689475,0.00007269364,0.9334181,0.0003921175],"study_design_candidate":"not_applicable","study_design_consensus":"not_applicable","genre_codex":"other","genre_gemma":"other","genre_scores_codex":[0.01627416,0.0007741033,0.0003102629,0.0001237178,0.01141643,0.0004055302,0.000006508684,0.0001626926,0.9705266],"genre_scores_gemma":[0.06631715,0.0001473535,0.0001939499,0.0002457719,0.002201134,0.0001487782,0.00169203,0.00004166294,0.9290122],"genre_candidate":"other","genre_consensus":"other","teacher_disagreement_score":0.1074622,"threshold_uncertainty_score":0.9999033,"prediction_status":"machine_predicted_unvalidated"},"machine_scores":{"provisional":true,"baseline":true,"maturity_gate_passed":false,"score_opus":0.1733975394905388,"score_gpt":0.445259909225525,"score_spread":0.2718623697349862,"validation_status":"score_only:v0-immature-baseline","note":"Baseline scores from an immature model (maturity gate not passed). Scores rank; they never assert a category."}}