{"id":"W4389615068","doi":"10.1148/radiol.230492","title":"Measuring Interrater Reliability","year":2023,"lang":"en","type":"editorial","venue":"Radiology","topic":"Reliability and Agreement in Measurement","field":"Decision Sciences","cited_by":30,"is_retracted":false,"has_abstract":false,"ca_institutions":"Centre Hospitalier de l’Université de Montréal","funders":"","keywords":"Medicine; Inter-rater reliability; Neuroradiology; Radiology; Scholarship; Medical physics; Nuclear medicine; Psychiatry; Psychology; Rating scale; Neurology","routes":{"ca_aff":true,"ca_fund":false,"ca_venue":false,"about_ca":false,"invisible_to_affiliation_only":false},"retraction":null,"screen":null,"direct_labels":[],"prediction":{"model_version":"codex-gemma-dda1882f352a","candidate_categories":["metaresearch","metaepi_narrow","research_integrity","insufficient_payload"],"consensus_categories":["insufficient_payload"],"category_scores_codex":[0.0225433,0.0004387348,0.001180351,0.0003927121,0.0001912676,0.0002265375,0.002592145,0.001401569,0.001012177],"category_scores_gemma":[0.0976086,0.0003025352,0.0004576108,0.0005908238,0.0003603881,0.0001486782,0.0005751962,0.001445565,0.007560997],"about_ca_system_candidate":false,"about_ca_system_consensus":false,"about_ca_system_score_codex":0.0003943268,"about_ca_system_score_gemma":0.0003669044,"about_ca_topic_candidate":false,"about_ca_topic_consensus":false,"about_ca_topic_score_codex":0.00008531289,"about_ca_topic_score_gemma":0.0001236169,"domain_scores_codex":[0.9901486,0.001824359,0.001747055,0.001767105,0.003782735,0.0007301161],"domain_scores_gemma":[0.9795576,0.0160081,0.0005871495,0.002391986,0.001259911,0.0001952012],"domain_codex":null,"domain_gemma":null,"domain_candidate":null,"domain_consensus":null,"study_design_codex":"not_applicable","study_design_gemma":"not_applicable","study_design_scores_codex":[0.00005552904,0.00003692952,0.0005217216,0.00002897347,0.0000439926,0.00001657558,0.0001462683,0.0001453672,0.00005063353,0.00002635012,0.9964296,0.002498055],"study_design_scores_gemma":[0.0002511038,0.0001531491,0.0005039124,0.00005252559,0.00002648708,0.000001612914,0.00007285012,0.0000852401,0.00002908245,0.01964505,0.9788671,0.0003118577],"study_design_candidate":"not_applicable","study_design_consensus":"not_applicable","genre_codex":"editorial","genre_gemma":"editorial","genre_scores_codex":[0.002092551,0.0001686886,0.000436076,0.001064206,0.9923473,0.0003320181,0.00009618537,0.0001793913,0.00328356],"genre_scores_gemma":[0.003797459,0.000171279,0.0002181236,0.000088211,0.9808455,0.0001066224,0.0001129743,0.00005984951,0.0146],"genre_candidate":"editorial","genre_consensus":"editorial","teacher_disagreement_score":0.07506529,"threshold_uncertainty_score":0.9999427,"prediction_status":"machine_predicted_unvalidated"},"machine_scores":{"provisional":true,"baseline":true,"maturity_gate_passed":false,"score_opus":0.2162369507691027,"score_gpt":0.3877990363725101,"score_spread":0.1715620856034074,"validation_status":"score_only:v0-immature-baseline","note":"Baseline scores from an immature model (maturity gate not passed). Scores rank; they never assert a category."}}