{"id":"W3119636502","doi":"10.18653/v1/2021.acl-long.569","title":"UnNatural Language Inference","year":2021,"lang":"en","type":"preprint","venue":"","topic":"Topic Modeling","field":"Computer Science","cited_by":5,"is_retracted":false,"has_abstract":true,"ca_institutions":"Mila - Quebec Artificial Intelligence Institute; McGill University","funders":"McGill University","keywords":"Computer science; Inference; Natural language processing; Artificial intelligence; Mandarin Chinese; Syntax; Suite; Transformer; Language model; Word order; Linguistics; History","routes":{"ca_aff":true,"ca_fund":true,"ca_venue":false,"about_ca":false,"invisible_to_affiliation_only":false},"retraction":null,"screen":null,"direct_labels":[],"prediction":{"model_version":"metacan-v3-hybrid-931329e0061c","candidate_categories":[],"consensus_categories":[],"category_scores_codex":[0.004165142,0.001061807,0.0009860144,0.002078088,0.001850444,0.003668946,0.00200812,0.001482111,0.01600713],"category_scores_gemma":[0.01572675,0.0008523124,0.001916351,0.001619886,0.002883592,0.009123254,0.004651865,0.004371066,0.005349788],"about_ca_system_candidate":false,"about_ca_system_consensus":false,"about_ca_system_score_codex":0.001132016,"about_ca_system_score_gemma":0.001080958,"about_ca_topic_candidate":false,"about_ca_topic_consensus":false,"about_ca_topic_score_codex":0.001720434,"about_ca_topic_score_gemma":0.003069995,"domain_scores_codex":[0.9963182,0.001819338,0.0001741217,0.0008417296,0.0007164472,0.0001302039],"domain_scores_gemma":[0.9910657,0.005423999,0.0001659257,0.002505131,0.0006893514,0.0001499094],"domain_codex":null,"domain_gemma":null,"domain_candidate":null,"domain_consensus":null,"study_design_codex":"theoretical_or_conceptual","study_design_gemma":"simulation_or_modeling","study_design_scores_codex":[0.00007742675,0.00006004648,0.0005406354,0.0002889053,0.0001036276,0.0003104418,0.0005840177,0.003652404,0.001211758,0.8254378,0.06027249,0.1074604],"study_design_scores_gemma":[0.00001356112,0.000007209851,0.000115797,0.00006134121,0.00002578497,0.0001898323,0.00006844634,0.02186718,0.001228919,0.9148623,0.06154409,0.00001557571],"study_design_candidate":"simulation_or_modeling","study_design_consensus":null,"genre_codex":"methods","genre_gemma":"methods","genre_scores_codex":[0.005240675,0.003847305,0.9336472,0.00708654,0.001474729,0.00006510004,0.001190549,0.003420903,0.04402695],"genre_scores_gemma":[0.344752,0.00492878,0.5878454,0.004297058,0.00314662,0.0003371086,0.006944004,0.003733509,0.04401546],"genre_candidate":"methods","genre_consensus":"methods","teacher_disagreement_score":0.01600713,"threshold_uncertainty_score":0.05354917,"prediction_status":"machine_predicted_unvalidated"},"machine_scores":{"provisional":true,"baseline":true,"maturity_gate_passed":false,"score_opus":0.02933943172557949,"score_gpt":0.2968036872332471,"score_spread":0.2674642555076676,"validation_status":"score_only:v0-immature-baseline","note":"Baseline scores from an immature model (maturity gate not passed). Scores rank; they never assert a category."}}