{"id":"W2130998770","doi":"10.1145/1596517.1596518","title":"Extrinsic summarization evaluation","year":2009,"lang":"en","type":"article","venue":"ACM Transactions on Speech and Language Processing","topic":"Topic Modeling","field":"Computer Science","cited_by":30,"is_retracted":false,"has_abstract":true,"ca_institutions":"University of British Columbia","funders":"Sixth Framework Programme","keywords":"Automatic summarization; Computer science; Task (project management); Information retrieval; Audit; Human–computer interaction; Natural language processing; Artificial intelligence","routes":{"ca_aff":true,"ca_fund":false,"ca_venue":false,"about_ca":false,"invisible_to_affiliation_only":false},"retraction":null,"screen":null,"direct_labels":[],"prediction":{"model_version":"metacan-v3-hybrid-931329e0061c","candidate_categories":[],"consensus_categories":[],"category_scores_codex":[0.01229399,0.002190424,0.001283905,0.003015913,0.0008337756,0.00304517,0.001674084,0.00145449,0.01009868],"category_scores_gemma":[0.06587509,0.0002529407,0.0008075855,0.001603739,0.0005683884,0.003067926,0.001836114,0.00110242,0.00411756],"about_ca_system_candidate":false,"about_ca_system_consensus":false,"about_ca_system_score_codex":0.0008043082,"about_ca_system_score_gemma":0.0008347191,"about_ca_topic_candidate":false,"about_ca_topic_consensus":false,"about_ca_topic_score_codex":0.0008024478,"about_ca_topic_score_gemma":0.001114742,"domain_scores_codex":[0.9779556,0.01101388,0.002280524,0.001879535,0.006419891,0.000450543],"domain_scores_gemma":[0.9409888,0.0279407,0.003325114,0.005820352,0.02104724,0.0008779025],"domain_codex":null,"domain_gemma":null,"domain_candidate":null,"domain_consensus":null,"study_design_codex":"design_other","study_design_gemma":"simulation_or_modeling","study_design_scores_codex":[0.003788558,0.0008779663,0.009817999,0.003564328,0.000605443,0.0004358901,0.0009977736,0.01473072,0.06241746,0.003241143,0.03960078,0.859922],"study_design_scores_gemma":[0.001037001,0.009488849,0.0650547,0.0007347289,0.001635269,0.003004554,0.002959047,0.4277896,0.289561,0.009641589,0.1883651,0.0007285698],"study_design_candidate":"simulation_or_modeling","study_design_consensus":null,"genre_codex":"methods","genre_gemma":"empirical","genre_scores_codex":[0.2800617,0.005843942,0.6303888,0.001201964,0.001438467,0.003749424,0.0135761,0.02590509,0.03783436],"genre_scores_gemma":[0.6657206,0.001349329,0.2719752,0.0006516043,0.000780337,0.002088147,0.03493344,0.002982893,0.01951849],"genre_candidate":"empirical","genre_consensus":null,"teacher_disagreement_score":0.01229399,"threshold_uncertainty_score":0.06501764,"prediction_status":"machine_predicted_unvalidated"},"machine_scores":{"provisional":true,"baseline":true,"maturity_gate_passed":false,"score_opus":0.02421893031646663,"score_gpt":0.2929102533355549,"score_spread":0.2686913230190883,"validation_status":"score_only:v0-immature-baseline","note":"Baseline scores from an immature model (maturity gate not passed). Scores rank; they never assert a category."}}