Measuring Students' Knowledge Mastery Patterns in Energy Using Cognitive Diagnostic Models
Bibliographic record
Abstract
Cognitive diagnostic models can uncover students’ mastery of multiple fine-grained skill attributes or problem-solving processes. A number of studies have applied cognitive diagnostic models to detect students’ knowledge mastery in mathematics and language testing. However, few studies focus on cognitive diagnostic assessment in K-12 science education, and no studies on the energy topic specifically. This study applied cognitive diagnostic models to Trends in International Mathematics and Science Study (TIMSS) science achievement data to assess students’ knowledge mastery in energy. Three TIMSS participating jurisdictions, i.e., Australia, Hong Kong, and Ontario were compared. A Q matrix (i.e., an item attribute alignment table) was proposed based on existing literature about learning progressions of energy in the physical science domain, and the TIMSS assessment framework. The Q matrix was validated through expert review and real data analysis. Then, one of the cognitive diagnostic models, i.e., the deterministic inputs, noisy and-gate (DINA) model was applied to each jurisdiction’s data. Results suggested that the hypothesized learning progression was consistent with Australian and Ontario students’ but not Hong Kong students’ observed progression in understanding the energy concept. According to overall attribute mastery probabilities and the latent class pattern, most students failed to explain simple electrical systems. Students also performed poorly in recognizing that heating an object can increase its temperature, and that hot objects can heat up cold objects. Identifying sources of energy was found to be easiest to be mastered. I discuss several potential curriculum-related issues that may affect students’ mastery patterns in different jurisdictions.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.004 | 0.016 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.001 |
| Bibliometrics | 0.003 | 0.002 |
| Science and technology studies | 0.000 | 0.001 |
| Scholarly communication | 0.002 | 0.001 |
| Open science | 0.001 | 0.001 |
| Research integrity | 0.000 | 0.001 |
| Insufficient payload (model declined to judge) | 0.002 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".