{"id":"W4389494812","doi":"10.2139/ssrn.4650826","title":"A Comprehensive Framework for Evaluating Software Engineering Technologies","year":2023,"lang":"en","type":"article","venue":"SSRN Electronic Journal","topic":"Data Stream Mining Techniques","field":"Computer Science","cited_by":3,"is_retracted":false,"has_abstract":false,"ca_institutions":"Wycliffe College","funders":"","keywords":"Software engineering; Computer science; Systems engineering; Engineering","routes":{"ca_aff":true,"ca_fund":false,"ca_venue":false,"about_ca":false,"invisible_to_affiliation_only":false},"retraction":null,"screen":null,"direct_labels":[],"prediction":{"model_version":"codex-gemma-dda1882f352a","candidate_categories":[],"consensus_categories":[],"category_scores_codex":[0.001237341,0.0001655373,0.0001834743,0.0003192998,0.0002253613,0.0001816264,0.001477781,0.000122811,7.558884e-7],"category_scores_gemma":[0.00179034,0.0001571948,0.000096916,0.0007478116,0.00002556506,0.000371419,0.0004072491,0.001344405,0.00002089282],"about_ca_system_candidate":false,"about_ca_system_consensus":false,"about_ca_system_score_codex":0.0003722811,"about_ca_system_score_gemma":0.0006425951,"about_ca_topic_candidate":false,"about_ca_topic_consensus":false,"about_ca_topic_score_codex":0.000003128931,"about_ca_topic_score_gemma":0.000002909372,"domain_scores_codex":[0.9971685,0.00002657053,0.0002370303,0.0003004272,0.0002778076,0.001989709],"domain_scores_gemma":[0.9986243,0.0005898306,0.0001351692,0.0004715898,0.0001446127,0.00003447243],"domain_codex":null,"domain_gemma":null,"domain_candidate":null,"domain_consensus":null,"study_design_codex":"theoretical_or_conceptual","study_design_gemma":"theoretical_or_conceptual","study_design_scores_codex":[0.000006100453,0.00001150023,0.000183349,0.00001905452,0.0000783479,0.000006319365,0.000153875,0.0002846025,0.0005624778,0.6787409,0.0005671377,0.3193863],"study_design_scores_gemma":[0.000197147,0.0005029621,0.0001320857,0.0001091205,0.00001065784,0.0002412044,0.0005806105,0.03111843,0.001173038,0.9637335,0.001965543,0.0002356813],"study_design_candidate":"theoretical_or_conceptual","study_design_consensus":"theoretical_or_conceptual","genre_codex":"methods","genre_gemma":"methods","genre_scores_codex":[0.01563729,0.0007453265,0.9788492,0.0009112991,0.0002149253,0.0001951319,0.000004653887,0.003436918,0.000005296765],"genre_scores_gemma":[0.1408033,0.0007087006,0.8581786,0.00004455886,0.00009014381,0.00008072826,0.000006526033,0.00003226368,0.00005513231],"genre_candidate":"methods","genre_consensus":"methods","teacher_disagreement_score":0.3191506,"threshold_uncertainty_score":0.6410216,"prediction_status":"machine_predicted_unvalidated"},"machine_scores":{"provisional":true,"baseline":true,"maturity_gate_passed":false,"score_opus":0.03318069035774149,"score_gpt":0.3221317097187211,"score_spread":0.2889510193609797,"validation_status":"score_only:v0-immature-baseline","note":"Baseline scores from an immature model (maturity gate not passed). Scores rank; they never assert a category."}}