{"id":"W4411548986","doi":"10.2139/ssrn.5311193","title":"Benchmarking Clinical Prediction Problems on Observational Databases","year":2025,"lang":"en","type":"preprint","venue":"SSRN Electronic Journal","topic":"Machine Learning in Healthcare","field":"Computer Science","cited_by":0,"is_retracted":false,"has_abstract":false,"ca_institutions":"University of Calgary","funders":"","keywords":"Benchmarking; Observational study; Database; Computer science; Data mining; Data science; Statistics; Business; Mathematics","routes":{"ca_aff":true,"ca_fund":false,"ca_venue":false,"about_ca":false,"invisible_to_affiliation_only":false},"retraction":null,"screen":null,"direct_labels":[],"prediction":{"model_version":"codex-gemma-dda1882f352a","candidate_categories":["metaepi_narrow","research_integrity"],"consensus_categories":[],"category_scores_codex":[0.007622092,0.0003805158,0.0004697155,0.0003608983,0.0004749454,0.0002752233,0.002082324,0.0003298069,0.00001944617],"category_scores_gemma":[0.0007234594,0.0003615037,0.0003181538,0.0003400925,0.00004789006,0.0003405462,0.001328302,0.01503788,0.00001804562],"about_ca_system_candidate":true,"about_ca_system_consensus":false,"about_ca_system_score_codex":0.001665829,"about_ca_system_score_gemma":0.01385744,"about_ca_topic_candidate":false,"about_ca_topic_consensus":false,"about_ca_topic_score_codex":0.0002104593,"about_ca_topic_score_gemma":0.0004389426,"domain_scores_codex":[0.9938182,0.0009211724,0.001225619,0.001015059,0.0009202823,0.002099675],"domain_scores_gemma":[0.9968926,0.0006569262,0.0008752283,0.00106715,0.0003466867,0.0001614019],"domain_codex":null,"domain_gemma":null,"domain_candidate":null,"domain_consensus":null,"study_design_codex":"theoretical_or_conceptual","study_design_gemma":"theoretical_or_conceptual","study_design_scores_codex":[0.00003290642,0.00018374,0.1067114,0.0001784533,0.0002716353,0.000007913185,0.0001316539,0.03149638,9.593806e-7,0.6752465,0.001539128,0.1841993],"study_design_scores_gemma":[0.000920173,0.00123906,0.0713783,0.002110902,0.00008028118,0.0003060719,0.00004911047,0.3463861,0.000003007779,0.5493317,0.02745723,0.0007380245],"study_design_candidate":"theoretical_or_conceptual","study_design_consensus":"theoretical_or_conceptual","genre_codex":"methods","genre_gemma":"empirical","genre_scores_codex":[0.01399311,0.003033737,0.959631,0.01068467,0.008857599,0.0007601956,0.00008320341,0.0004213792,0.002535153],"genre_scores_gemma":[0.817638,0.04201063,0.1032038,0.006222703,0.01951083,0.0003271911,0.001524051,0.0001614695,0.009401291],"genre_candidate":"methods","genre_consensus":null,"teacher_disagreement_score":0.8564272,"threshold_uncertainty_score":0.9998837,"prediction_status":"machine_predicted_unvalidated"},"machine_scores":{"provisional":true,"baseline":true,"maturity_gate_passed":false,"score_opus":0.1050535358908832,"score_gpt":0.3880925681081164,"score_spread":0.2830390322172333,"validation_status":"score_only:v0-immature-baseline","note":"Baseline scores from an immature model (maturity gate not passed). Scores rank; they never assert a category."}}