{"id":"W4403922859","doi":"10.1145/3686215.3688382","title":"Detecting when Users Disagree with Generated Captions","year":2024,"lang":"en","type":"article","venue":"","topic":"Video Analysis and Summarization","field":"Computer Science","cited_by":3,"is_retracted":false,"has_abstract":true,"ca_institutions":"University of British Columbia","funders":"Universitas Brawijaya","keywords":"Computer science; Natural language processing","routes":{"ca_aff":true,"ca_fund":false,"ca_venue":false,"about_ca":false,"invisible_to_affiliation_only":false},"retraction":null,"screen":null,"direct_labels":[],"prediction":{"model_version":"metacan-v3-hybrid-931329e0061c","candidate_categories":[],"consensus_categories":[],"category_scores_codex":[0.006511693,0.001007421,0.0007895171,0.001393545,0.0007231134,0.002272404,0.0009393091,0.001271804,0.001823478],"category_scores_gemma":[0.07866663,0.0003250954,0.0002900353,0.0005329392,0.0004857249,0.002069005,0.001408842,0.0009369668,0.001196416],"about_ca_system_candidate":false,"about_ca_system_consensus":false,"about_ca_system_score_codex":0.0005227051,"about_ca_system_score_gemma":0.0003606772,"about_ca_topic_candidate":false,"about_ca_topic_consensus":false,"about_ca_topic_score_codex":0.001355571,"about_ca_topic_score_gemma":0.001592453,"domain_scores_codex":[0.9925584,0.003617935,0.0006828252,0.001106298,0.001752207,0.0002823477],"domain_scores_gemma":[0.9325913,0.04479806,0.007012784,0.004087179,0.01072204,0.0007886303],"domain_codex":null,"domain_gemma":null,"domain_candidate":null,"domain_consensus":null,"study_design_codex":"observational","study_design_gemma":"bench_or_experimental","study_design_scores_codex":[0.007126282,0.0007085336,0.3416592,0.001830379,0.000541429,0.001442254,0.05437861,0.006549968,0.2321106,0.002549555,0.01055214,0.3405509],"study_design_scores_gemma":[0.0002289104,0.003301055,0.5032448,0.0004251559,0.0003973222,0.002201759,0.02881685,0.3304507,0.1077845,0.005125631,0.01734326,0.0006800195],"study_design_candidate":"bench_or_experimental","study_design_consensus":null,"genre_codex":"empirical","genre_gemma":"empirical","genre_scores_codex":[0.9670343,0.0002380387,0.02663681,0.0001966472,0.0000733037,0.0003301762,0.0004397804,0.001275282,0.003775601],"genre_scores_gemma":[0.986387,0.0000636923,0.01189181,0.0001273323,0.00002842881,0.0001554215,0.000464861,0.00008696137,0.000794409],"genre_candidate":"empirical","genre_consensus":"empirical","teacher_disagreement_score":0.006511693,"threshold_uncertainty_score":0.03443754,"prediction_status":"machine_predicted_unvalidated"},"machine_scores":{"provisional":true,"baseline":true,"maturity_gate_passed":false,"score_opus":0.01576681960141888,"score_gpt":0.2208626449127426,"score_spread":0.2050958253113237,"validation_status":"score_only:v0-immature-baseline","note":"Baseline scores from an immature model (maturity gate not passed). Scores rank; they never assert a category."}}