Using Exploratory and Confirmatory Methods to Identify the Cognitive Dimensions In a Large-Scale Science Assessment
Bibliographic record
Abstract
Studies of test dimensionality indicate that many large-scale science assessments measure multiple dimensions. These findings have reinforced the perspective that science achievement is an inherently dynamic process and that there is benefit in reporting subscores in science. A limitation with some of these studies is that they fail to indicate how the dimensions found to underlie science assessments relate to psychological theories of scientific reasoning. A convincing argument for the dynamic character of scientific reasoning and the need to report subscores should include how the dimensions relate to psychological theories of scientific reasoning. Otherwise, the broader, psychological character of student science performance will not be informed. The first objective of this article was to identify the dimensional structure of a new large-scale science assessment using nonparametric and parametric techniques, thus attempting to replicate findings from previous studies. The second objective was to determine whether a content-based or psychologically-based framework could be used to identify, define, and explain the dimensions found to underlie this new large-scale science assessment. The results of the current study indicate that a psychological theory of scientific reasoning could be used to describe the multiple dimensions underlying at least one large-scale science assessment.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.139 | 0.314 |
| Meta-epidemiology (narrow) | 0.002 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.002 |
| Bibliometrics | 0.012 | 0.008 |
| Science and technology studies | 0.002 | 0.003 |
| Scholarly communication | 0.005 | 0.004 |
| Open science | 0.002 | 0.003 |
| Research integrity | 0.001 | 0.002 |
| Insufficient payload (model declined to judge) | 0.001 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".