Using Exploratory and Confirmatory Methods to Identify the Cognitive Dimensions In a Large-Scale Science Assessment
Why this work is in the frame
A frame that forgets how it found something cannot be audited. These are the routes that admitted this work.
Bibliographic record
Abstract
Studies of test dimensionality indicate that many large-scale science assessments measure multiple dimensions. These findings have reinforced the perspective that science achievement is an inherently dynamic process and that there is benefit in reporting subscores in science. A limitation with some of these studies is that they fail to indicate how the dimensions found to underlie science assessments relate to psychological theories of scientific reasoning. A convincing argument for the dynamic character of scientific reasoning and the need to report subscores should include how the dimensions relate to psychological theories of scientific reasoning. Otherwise, the broader, psychological character of student science performance will not be informed. The first objective of this article was to identify the dimensional structure of a new large-scale science assessment using nonparametric and parametric techniques, thus attempting to replicate findings from previous studies. The second objective was to determine whether a content-based or psychologically-based framework could be used to identify, define, and explain the dimensions found to underlie this new large-scale science assessment. The results of the current study indicate that a psychological theory of scientific reasoning could be used to describe the multiple dimensions underlying at least one large-scale science assessment.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.008 | 0.002 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.001 | 0.001 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.001 |
| Open science | 0.001 | 0.001 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it