{"id":"W4399860921","doi":"10.1145/3660650.3660664","title":"An Initial Exploration of Code Diagram Query Effectiveness","year":2024,"lang":"en","type":"article","venue":"","topic":"Teaching and Learning Programming","field":"Computer Science","cited_by":1,"is_retracted":false,"has_abstract":true,"ca_institutions":"University of Manitoba","funders":"","keywords":"Computer science; Programming language; Notional amount; Visualization; Program comprehension; Diagrammatic reasoning; Python (programming language); Software engineering; Artificial intelligence; Software; Software system","routes":{"ca_aff":true,"ca_fund":false,"ca_venue":false,"about_ca":false,"invisible_to_affiliation_only":false},"retraction":null,"screen":null,"direct_labels":[],"prediction":{"model_version":"codex-gemma-dda1882f352a","candidate_categories":[],"consensus_categories":[],"category_scores_codex":[0.000641999,0.00005467925,0.0000706318,0.00007627856,0.00004583508,0.000196869,0.0002118636,0.00003053622,0.000003451799],"category_scores_gemma":[0.0000329738,0.00004535864,0.00003172158,0.0001978382,0.00001774067,0.001000286,0.00003499058,0.0001200088,0.00001703556],"about_ca_system_candidate":false,"about_ca_system_consensus":false,"about_ca_system_score_codex":0.00001236731,"about_ca_system_score_gemma":0.00003512656,"about_ca_topic_candidate":false,"about_ca_topic_consensus":false,"about_ca_topic_score_codex":0.000104453,"about_ca_topic_score_gemma":0.00001214587,"domain_scores_codex":[0.9992651,0.0002431846,0.00009013858,0.0001932311,0.0001131858,0.00009511762],"domain_scores_gemma":[0.9995756,0.000158444,0.00001614514,0.0001951489,0.00002193467,0.00003274209],"domain_codex":null,"domain_gemma":null,"domain_candidate":null,"domain_consensus":null,"study_design_codex":"design_other","study_design_gemma":"simulation_or_modeling","study_design_scores_codex":[0.000002657137,0.00003881817,0.0002224031,0.00004866512,0.000006927716,0.000008003546,0.002344454,0.000691798,0.0006922773,0.09793517,0.00001666131,0.8979922],"study_design_scores_gemma":[0.0006140199,0.001966141,0.01463939,0.0006389818,0.0000314093,0.00003843611,0.0005565505,0.8706343,0.04845738,0.01303364,0.04859744,0.0007923279],"study_design_candidate":"design_other","study_design_consensus":null,"genre_codex":"methods","genre_gemma":"empirical","genre_scores_codex":[0.05907856,0.00006591134,0.9393811,0.0001093272,0.0003811865,0.00005792313,3.127877e-7,0.0005903509,0.000335364],"genre_scores_gemma":[0.9721574,0.000001302522,0.02771578,0.00001381853,0.00005971116,0.00001230334,0.000002890664,0.000005392838,0.00003143517],"genre_candidate":"empirical","genre_consensus":null,"teacher_disagreement_score":0.9130788,"threshold_uncertainty_score":0.1898412,"prediction_status":"machine_predicted_unvalidated"},"machine_scores":{"provisional":true,"baseline":true,"maturity_gate_passed":false,"score_opus":0.04202554863068285,"score_gpt":0.3533535432281289,"score_spread":0.311327994597446,"validation_status":"score_only:v0-immature-baseline","note":"Baseline scores from an immature model (maturity gate not passed). Scores rank; they never assert a category."}}