Evaluation of Submission Limits and Regression Penalties to Improve Student Behavior with Automatic Assessment Systems
Notice bibliographique
Résumé
Objectives . Automatic assessment systems are widely used to provide rapid feedback for students and reduce grading time. Despite the benefits of increased efficiency and improved pedagogical outcomes, an ongoing challenge is mitigating poor student behaviors when interacting with automatic assessment systems including numerous submissions, trial-and-error, and relying on marking feedback for problem solving. These behaviors negatively affect student learning as well as have significant impact on system resources. This research quantitatively examines how utilizing submission policies such as limiting the number of submissions and applying regression penalties can reduce negative student behaviors. The hypothesis is that both submission policies will have a significant impact on student behavior and reduce both the number of submissions and regressions in student performance. The research questions evaluate the impact on student behavior, determine which submission policy is the most effective, and what submission policy is preferred by students. Participants . The study involved two course sections in two different semesters consisting of a total of 224 students at the University of British Columbia, a research-intensive university. The students were evaluated using an automated assessment system in a large third year database course. Study Methods . The two course sections used an automated assessment system for constructing database design diagrams for assignments and exams. The first section had no limits on the number of submissions for both assignments and exams. The second section had limits for the exams but no limits on assignments. On the midterm, participants were randomly assigned to have either a restriction on the total number of submissions or unlimited submissions but with regression penalties if a graded answer was lower than a previous submission. On the final exam, students were given the option of selecting their submission policy. Student academic performance and submission profiles were compared between the course sections and the different submission policies. Findings. Unrestricted use of automatic grading systems results in high occurrence of undesirable student behavior including trial-and-error guessing and reduced time between submissions without sufficient independent thought. Both submission policies of limiting maximum submissions and utilizing regression penalties significantly reduce these behaviors by up to 85%. Overall, students prefer maximum submission limits, and demonstrate improved behavior and educational outcomes. Conclusions . Automated assessment systems when used for larger problems related to design and programming have benefits when deployed with submission restrictions (maximum attempts or regression penalty) for both improved student learning behaviors and to reduce the computational costs for the system. This is especially important for summative assessment but reasonable limits for formative assessments are also valuable.
Récupéré en direct depuis OpenAlex et désinversé. Les résumés ne sont pas conservés dans cette base de données : les index inversés représentent 8,6 Go des 9,3 Go de texte de la base, et le serveur dispose de 13 Go libres.
Comment cette classification a été obtenuedéplier
Prédiction machine sur la base complète
Imitation des enseignantsNi prévalence calibrée, ni vérité terrain. Validation humaine à venir. Le volet Gemma est une étiquette directe du modèle pour chaque travail de la base, lue sur la notice réduite au titre. Le volet Codex est un classifieur appris des 10 348 étiquettes directes de Codex et calibré sur les taux pondérés de l'échantillon; les champs sans appui suffisant ne portent aucun appel Codex. Le mode candidate est l'union des deux volets; le consensus est leur intersection. Ces sorties portent le statut machine_predicted_unvalidated et ne sont pas des étiquettes humaines.
Scores du classifieur distillé par catégorie (deux têtes)
| Catégorie | Codex | Gemma |
|---|---|---|
| Métarecherche | 0,016 | 0,108 |
| Méta-épidémiologie (sens strict) | 0,001 | 0,000 |
| Méta-épidémiologie (sens large) | 0,001 | 0,000 |
| Bibliométrie | 0,001 | 0,001 |
| Études des sciences et des technologies | 0,000 | 0,001 |
| Communication savante | 0,002 | 0,001 |
| Science ouverte | 0,002 | 0,001 |
| Intégrité de la recherche | 0,001 | 0,001 |
| Charge utile insuffisante (le modèle a refusé de juger) | 0,001 | 0,000 |
Scores machine (provisoires)
Les deux têtes enseignantes du modèle étudiant, lues sur ce travail. Un score ordonne la base pour la relecture; il n'affirme jamais une catégorie, et le statut de validation accompagne chaque rangée tel quel.
Scores de référence d'un modèle non mature (critères de maturité non atteints, 7 itérations). Un score ordonne; il n'affirme jamais une catégorie.
score_only:v0-immature-baseline · tel quel depuis la passe de notation : score_only signifie que le nombre peut ordonner les travaux, et qu'aucune étiquette de catégorie n'en découleClassification
machine, non validéePrédiction automatique; un appel candidat d’une seule source (Gemma direct ou Codex distillé), pas un consensus.
Le détail, modèle par modèle et score par score, se trouve en fin de page sous « Comment cette classification a été obtenue ».