Iterative Drawing Reveals Diversity and Change in Student Thinking About Evolution
Bibliographic record
Abstract
One theoretical framework for learning is that knowledge exists in pieces. Understanding emerges by developing a coherent and stable integration of multiple ideas. When learning gains are evaluated iteratively, over time, it is clear that the mental constructs representing key ideas are dynamic. Here we describe an assessment framework that enables estimation of the dynamic nature of mental constructs as students make gains towards coherency of knowledge and understanding. The framework emphasizes the value of iterative assessment combined with multivariate methods borrowed from ecology for revealing and following gains in student thinking. We applied our framework for monitoring and describing student gains in their abilities to visualize and describe the process of evolution. Our approach was observational. We evaluated 276 drawings and accompanying text-based descriptions of evolution generated by 102 students by implementing the same open-ended assessment question four times in two sections of an upper division evolutionary biology course during spring 2021. Based on a binary rubric of 10 key ideas, students showed evidence of gains and losses of key ideas over time, and their learning trajectories were diverse and dynamic. Our findings revealed students take a multitude of pathways to concept mastery and that they struggled to succinctly construct and communicate comprehensive evolutionary models. Based on our study, we recommend using iterative free-response assessment with an explicit rubric and multivariate non-metric dimensional scale data visualization for revealing student thinking and guiding data-driven revision of curriculum, teaching strategies, and assessment for achieving greater coherency and stability of knowledge.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.001 | 0.000 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.001 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".