{"id":"W4414594223","doi":"10.2196/80987","title":"Data Contamination in AI Evaluation","year":2025,"lang":"en","type":"article","venue":"JMIR Medical Informatics","topic":"Adversarial Robustness in Machine Learning","field":"Computer Science","cited_by":0,"is_retracted":false,"has_abstract":false,"ca_institutions":"","funders":"","keywords":"","routes":{"ca_aff":false,"ca_fund":false,"ca_venue":true,"about_ca":false,"invisible_to_affiliation_only":true},"retraction":null,"screen":null,"direct_labels":[],"prediction":{"model_version":"codex-gemma-dda1882f352a","candidate_categories":[],"consensus_categories":[],"category_scores_codex":[0.002551484,0.00007729317,0.0001233013,0.0002083499,0.00005309602,0.00008700712,0.00164092,0.0001217629,0.00005933045],"category_scores_gemma":[0.001580543,0.0000704343,0.00001309451,0.0006120706,0.00004217734,0.001524561,0.0009656011,0.0004143335,0.00003632496],"about_ca_system_candidate":false,"about_ca_system_consensus":false,"about_ca_system_score_codex":0.0001056845,"about_ca_system_score_gemma":0.0004619248,"about_ca_topic_candidate":false,"about_ca_topic_consensus":false,"about_ca_topic_score_codex":0.000009130326,"about_ca_topic_score_gemma":0.00002878449,"domain_scores_codex":[0.9981723,0.0001133043,0.0004838973,0.0001190979,0.0009473833,0.0001639773],"domain_scores_gemma":[0.9988301,0.0002030459,0.0001043723,0.0007048707,0.00009968446,0.00005792352],"domain_codex":null,"domain_gemma":null,"domain_candidate":null,"domain_consensus":null,"study_design_codex":"design_other","study_design_gemma":"simulation_or_modeling","study_design_scores_codex":[0.000004747775,0.00006959034,0.002420124,0.00009573135,0.0000119228,0.000005048216,0.003495222,0.002268393,0.00000111113,0.1051271,0.009104468,0.8773966],"study_design_scores_gemma":[0.0006887772,0.00000945404,0.003973533,0.0001034613,0.000003812707,0.000002090332,0.0002117267,0.9825289,0.00000456948,0.002451849,0.009955228,0.00006657853],"study_design_candidate":"simulation_or_modeling","study_design_consensus":null,"genre_codex":"methods","genre_gemma":"empirical","genre_scores_codex":[0.005691218,0.00002025794,0.9788496,0.003577179,0.0004656134,0.0003013509,0.000001282739,0.00008423742,0.01100921],"genre_scores_gemma":[0.9564112,0.00001095403,0.03764475,0.005520486,0.00007047933,0.00005832288,0.0001344135,0.000004437023,0.0001449743],"genre_candidate":"methods","genre_consensus":null,"teacher_disagreement_score":0.9802606,"threshold_uncertainty_score":0.3049265,"prediction_status":"machine_predicted_unvalidated"},"machine_scores":{"provisional":true,"baseline":true,"maturity_gate_passed":false,"score_opus":0.03185246388751473,"score_gpt":0.386149169344468,"score_spread":0.3542967054569533,"validation_status":"score_only:v0-immature-baseline","note":"Baseline scores from an immature model (maturity gate not passed). Scores rank; they never assert a category."}}