{"id":"W4414594223","doi":"10.2196/80987","title":"Data Contamination in AI Evaluation","year":2025,"lang":"en","type":"article","venue":"JMIR Medical Informatics","topic":"Adversarial Robustness in Machine Learning","field":"Computer Science","cited_by":0,"is_retracted":false,"has_abstract":false,"ca_institutions":"","funders":"","keywords":"","routes":{"ca_aff":false,"ca_fund":false,"ca_venue":true,"about_ca":false,"invisible_to_affiliation_only":true},"retraction":null,"screen":null,"direct_labels":[],"prediction":{"model_version":"metacan-v3-hybrid-931329e0061c","candidate_categories":["metaresearch"],"consensus_categories":[],"category_scores_codex":[0.04391285,0.001749477,0.001592329,0.003961559,0.001365595,0.005068874,0.002590418,0.003012935,0.005901248],"category_scores_gemma":[0.1806772,0.000814388,0.001130367,0.002349905,0.003023041,0.004821936,0.005054281,0.003424484,0.00222526],"about_ca_system_candidate":false,"about_ca_system_consensus":false,"about_ca_system_score_codex":0.002114778,"about_ca_system_score_gemma":0.00215456,"about_ca_topic_candidate":false,"about_ca_topic_consensus":false,"about_ca_topic_score_codex":0.002738719,"about_ca_topic_score_gemma":0.002148628,"domain_scores_codex":[0.9397628,0.04234691,0.002520728,0.003082825,0.01107769,0.001209102],"domain_scores_gemma":[0.8628801,0.104991,0.003721617,0.01641748,0.01051677,0.001473096],"domain_codex":null,"domain_gemma":"methods","domain_candidate":"methods","domain_consensus":null,"study_design_codex":"design_other","study_design_gemma":"simulation_or_modeling","study_design_scores_codex":[0.00378895,0.0007302117,0.01832098,0.001568632,0.001134674,0.0004581426,0.0008043682,0.1900976,0.01201185,0.07771174,0.04045291,0.6529199],"study_design_scores_gemma":[0.000195054,0.0007084123,0.00503645,0.0003218426,0.0001999769,0.0005468762,0.0002364198,0.8491575,0.02823319,0.09229342,0.02296691,0.0001040426],"study_design_candidate":"simulation_or_modeling","study_design_consensus":null,"genre_codex":"methods","genre_gemma":"empirical","genre_scores_codex":[0.06165273,0.006631005,0.8959002,0.003452167,0.0008664051,0.0009488928,0.001743411,0.005523034,0.02328207],"genre_scores_gemma":[0.7630352,0.0008459163,0.2242026,0.001456809,0.000428703,0.0006075245,0.003080278,0.00107812,0.005264929],"genre_candidate":"empirical","genre_consensus":null,"teacher_disagreement_score":0.9560872,"threshold_uncertainty_score":0.2322361,"prediction_status":"machine_predicted_unvalidated"},"machine_scores":{"provisional":true,"baseline":true,"maturity_gate_passed":false,"score_opus":0.03185246388751473,"score_gpt":0.386149169344468,"score_spread":0.3542967054569533,"validation_status":"score_only:v0-immature-baseline","note":"Baseline scores from an immature model (maturity gate not passed). Scores rank; they never assert a category."}}