Static Analyses in Python Programming Courses
Why this work is in the frame
A frame that forgets how it found something cannot be audited. These are the routes that admitted this work.
Bibliographic record
Abstract
Students learning to program often rely on feedback from the compiler and from instructor-provided test cases to help them identify errors in their code. This feedback focuses on functional correctness, and the output, which is often phrased in technical language, may be difficult to for novices to understand or effectively use. Static analyses may be effective as a complementary aid, as they can highlight common errors that may be potential sources of problems. In this paper, we introduce PyTA, a wrapper for pylint that provides custom checks for common novice errors as well as improved messages to help students fix the errors that are found. We report on our experience integrating PyTA into an existing online system used to deliver programming exercises to CS1 students and evaluate it by comparing exercise submissions collected from the integrated system to previously collected data. This analysis demonstrates that, for students who chose to read the PyTA output, we observed a decrease in time to solve errors, occurrences of repeated errors, and submissions to complete a programming problem. This suggests that PyTA, and static analyses in general, may help students identify functional issues in their code not highlighted by compiler feedback and that static analysis output may help students more quickly identify debug their code.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.000 | 0.000 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it