{"id":"W342589544","doi":"","title":"York University at TREC 2012: CrowdSourcing Track.","year":2012,"lang":"en","type":"article","venue":"Text REtrieval Conference","topic":"Information Retrieval and Search Behavior","field":"Computer Science","cited_by":2,"is_retracted":false,"has_abstract":true,"ca_institutions":"York University","funders":"","keywords":"Crowdsourcing; Computer science; Quality (philosophy); Work (physics); Crowdsourcing software development; Data science; Control (management); Information retrieval; World Wide Web; Artificial intelligence; Engineering","routes":{"ca_aff":true,"ca_fund":false,"ca_venue":false,"about_ca":false,"invisible_to_affiliation_only":false},"retraction":null,"screen":null,"direct_labels":[],"prediction":{"model_version":"codex-gemma-dda1882f352a","candidate_categories":["insufficient_payload"],"consensus_categories":["insufficient_payload"],"category_scores_codex":[0.0006179572,0.000189224,0.000201965,0.0001596894,0.0003983033,0.0001651671,0.001004534,0.0001306756,0.0009599446],"category_scores_gemma":[0.0000739869,0.0001783555,0.0001029241,0.0007532671,0.0001111754,0.002084857,0.0004398398,0.0002845997,0.001684688],"about_ca_system_candidate":false,"about_ca_system_consensus":false,"about_ca_system_score_codex":0.0002244628,"about_ca_system_score_gemma":0.0001582114,"about_ca_topic_candidate":false,"about_ca_topic_consensus":false,"about_ca_topic_score_codex":0.00003806946,"about_ca_topic_score_gemma":0.000007871033,"domain_scores_codex":[0.9981668,0.000113912,0.0002393154,0.0002623849,0.0005579417,0.0006596216],"domain_scores_gemma":[0.9986211,0.0001265841,0.0001302604,0.0005256203,0.0002013741,0.0003950311],"domain_codex":null,"domain_gemma":null,"domain_candidate":null,"domain_consensus":null,"study_design_codex":"theoretical_or_conceptual","study_design_gemma":"not_applicable","study_design_scores_codex":[0.0009014531,0.0007410526,0.04982572,0.0001821166,0.0001121748,0.0001033574,0.03441041,0.00003645128,0.02777197,0.658189,0.02264289,0.2050834],"study_design_scores_gemma":[0.003251893,0.000587743,0.266388,0.0001383569,0.00008795928,0.0002364665,0.001345063,0.02074996,0.08391462,0.0006480517,0.6203349,0.002316959],"study_design_candidate":"theoretical_or_conceptual","study_design_consensus":null,"genre_codex":"empirical","genre_gemma":"empirical","genre_scores_codex":[0.7711383,0.0003044213,0.1706701,0.0008446156,0.0009436655,0.0004260537,0.0000252081,0.0006400004,0.0550077],"genre_scores_gemma":[0.9802427,0.00002341204,0.002389904,0.0001436409,0.00007623508,5.352064e-7,0.000007601637,0.000007060865,0.01710885],"genre_candidate":"empirical","genre_consensus":"empirical","teacher_disagreement_score":0.6575409,"threshold_uncertainty_score":0.9999533,"prediction_status":"machine_predicted_unvalidated"},"machine_scores":{"provisional":true,"baseline":true,"maturity_gate_passed":false,"score_opus":0.04766109456374461,"score_gpt":0.2510287572091137,"score_spread":0.2033676626453691,"validation_status":"score_only:v0-immature-baseline","note":"Baseline scores from an immature model (maturity gate not passed). Scores rank; they never assert a category."}}