Impact of student problem creation on self-reported confidence in mechanics
Bibliographic record
Abstract
Abstract (We intend to follow up with a full paper). Learning in engineering science courses typically involves solving textbook-style questions (given a problem statement, with one correct answer). As part of a project developing practice problems for students, we anecdotally noted that creation of problems deepended content understanding for the student problem developers. Problem creation could be an effective part of the engineering science teaching and learning toolbox. There is evidence of learning gains with student-created problems in immunology (Shakurnia 2018), general pathology and pathophysiology (Herroro 2019). Within the field of engineering, there are mixed results, with a study in the field of electrical engineering showing no effect on student learning (Algarni, 2021), and a study in manufacturing engineering showing significant improvement in learning (Brink 2004). No studies have examined the effectiveness of student problem creation in engineering mechanics, however. We seek to determine whether creating their own practice problems improves students' self-reported understanding of dynamics, and is viewed as an effective studying strategy. In this cohort study, we will use primarily quantitative methods to assess self-reported confidence and understanding of mechanics topics related to problem-creation activities within two populations. For the first population, a group of 135 students from a second-year dynamics course will be sent a pre-survey assessing their understanding of topics in dynamics, whether or not they create their own practice problems to aid with studying, and why they do or do not create practice problems. Students will be asked to develop their own practice textbook-style problem with a full solution as an optional bonus assignment. A second post-survey will ask students (both those who submitted problems and those who didn't) to repeat a self-evaluation of their understanding, and ask whether or not they plan to incorporate problem creation into their regular studying habits (and why or why not). Finally, a third separate survey will be sent out to 13 current and previous members of our open mechanics homework problem project (an ongoing 2-year project where students create ~50-100 problems each per work term), to evaluate whether creating their own problems improved their understanding of dynamics and/or statics, and whether they have since implemented problem creation into their studying. From the results from our pre and post surveys, we will measure any changes in the students' self-reported confidence in the concepts behind their question before and after they have completed it. We will also measure any change in whether the students would consider creating problems as part of their regular study practices. For the OER Mechanics survey, we will assess if students observed a change in self-reported confidence in the topics they created homework problems in. The results of this study can help inform whether student creation of problems could be used as an effective learning tool in engineering mechanics courses.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.001 | 0.000 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.001 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.001 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".