{"id":"W4400484309","doi":"10.1145/3663529.3664462","title":"Evaluating Social Bias in Code Generation Models","year":2024,"lang":"en","type":"article","venue":"","topic":"Hate Speech and Cyberbullying Detection","field":"Computer Science","cited_by":0,"is_retracted":false,"has_abstract":true,"ca_institutions":"Concordia University","funders":"","keywords":"Computer science; Code (set theory); Programming language","routes":{"ca_aff":true,"ca_fund":false,"ca_venue":false,"about_ca":false,"invisible_to_affiliation_only":false},"retraction":null,"screen":null,"direct_labels":[],"prediction":{"model_version":"metacan-v3-hybrid-931329e0061c","candidate_categories":[],"consensus_categories":[],"category_scores_codex":[0.03507867,0.0008440667,0.000550145,0.002148066,0.0009429353,0.002227237,0.00110306,0.001565449,0.001168174],"category_scores_gemma":[0.2238344,0.0002673515,0.0004604821,0.00101986,0.002772922,0.002717946,0.002546498,0.001412997,0.0002656],"about_ca_system_candidate":false,"about_ca_system_consensus":false,"about_ca_system_score_codex":0.002171913,"about_ca_system_score_gemma":0.0020455,"about_ca_topic_candidate":false,"about_ca_topic_consensus":false,"about_ca_topic_score_codex":0.002235545,"about_ca_topic_score_gemma":0.002837181,"domain_scores_codex":[0.9593697,0.0280411,0.00161399,0.003115945,0.007121146,0.0007380332],"domain_scores_gemma":[0.6920452,0.2545606,0.01823885,0.01841564,0.01440797,0.002331729],"domain_codex":null,"domain_gemma":null,"domain_candidate":null,"domain_consensus":null,"study_design_codex":"observational","study_design_gemma":"simulation_or_modeling","study_design_scores_codex":[0.002046063,0.0008763727,0.434239,0.0006976678,0.000598366,0.0002953309,0.004694758,0.2491724,0.009351566,0.03908396,0.005359564,0.2535849],"study_design_scores_gemma":[0.0001687996,0.0008469377,0.03418125,0.0001586327,0.0000907346,0.0002068038,0.0009872079,0.8992864,0.01132179,0.0490642,0.003613869,0.00007338011],"study_design_candidate":"simulation_or_modeling","study_design_consensus":null,"genre_codex":"empirical","genre_gemma":"empirical","genre_scores_codex":[0.8849606,0.0006072649,0.1068844,0.0009530662,0.00007877008,0.0003272688,0.0004340856,0.0005076143,0.005246968],"genre_scores_gemma":[0.9766381,0.00005376149,0.02206602,0.0001196489,0.00002888407,0.0001673874,0.0005112636,0.0000757696,0.0003392773],"genre_candidate":"empirical","genre_consensus":"empirical","teacher_disagreement_score":0.03507867,"threshold_uncertainty_score":0.185516,"prediction_status":"machine_predicted_unvalidated"},"machine_scores":{"provisional":true,"baseline":true,"maturity_gate_passed":false,"score_opus":0.2575162152373642,"score_gpt":0.3771989944879567,"score_spread":0.1196827792505925,"validation_status":"score_only:v0-immature-baseline","note":"Baseline scores from an immature model (maturity gate not passed). Scores rank; they never assert a category."}}