The ITEM Ontology: A Tool to Elucidate the Anatomy of Psychometric Indicators
Why this work is in the frame
A frame that forgets how it found something cannot be audited. These are the routes that admitted this work.
Bibliographic record
Abstract
Article Title: “The ITEM Ontology: A Tool to Elucidate the Anatomy of Psychometric Indicators” Authors: Kai R. Larsen, University of Colorado Boulder Roland M. Mueller, Berlin School of Economics and Law Dario Bonaretti, NEOMA Business School Diana Fischer-Preßler, University of Applied Sciences Frankfurt James (Jim) Burleson, California Polytechnic State University Nimisha Singh, Bennett University, India Jeffrey Parsons, Memorial University of Newfoundland Jean-Charles Pillet, Toulouse Business School Lan Sang, University of Colorado Boulder Zhu (Drew) Zhang, University of Rhode Island Problem definition: For decades, scientists have treated the survey item as the atomic unit of psychological measurement, an indivisible entity to be validated primarily through statistics. But what if we have been looking at the wrong level of analysis? Despite sophisticated modeling techniques, even well-validated instruments often contain items that are vague, double-barreled, or semantically misaligned with their intended constructs. These flaws remain undetected because current methods ignore the semantic anatomy of items. There is no shared language for decomposing indicators into the elements that carry meaning. Relevance: We show that psychometric measurement cannot be understood at the item or construct level and, for the first time, demonstrate that problems such as double-barreled items, restriction of range, and double negation can now be operationally defined and automatically detected. We argue that without tools like the ITEM Ontology, psychological measurement will never be precise enough to match the precision required for tasks such as medication optimization, leadership evaluation, technology impact assessment, and political polling. Theoretical foundations: This paper advances a new theory of survey indicator design by proposing that indicators can—and should—be deconstructed into meaningful components such as objects, actions, attributes, qualifiers, and response formats. This component-level analysis reveals structural and semantic flaws that traditional psychometric techniques often miss. Although grounded in linguistic and cognitive theory, our framework also draws on principles from established ontologies, particularly DOLCE, to ensure conceptual rigor and consistency. By making the internal anatomy of indicators explicit, the approach offers a foundational shift in how we understand, evaluate, and improve survey-based measurement. Methodology: We used design science methodologies to develop, refine, and test the ontology. Results: In addition to the ITEM ontology, we also provide a website ( http://www.itemontology.org/ ) that enables researchers to code indicators within the ontology, along with the ITEMIZER tool, which highlights potential wording issues and construct validity concerns associated with coded indicators. Contributions: This paper enables researchers and survey writers to understand the underlying components and their relationships within and across survey indicators. With the ITEM ontology, not only can we enhance the quality of future surveys, but we also provide a language to discuss how indicators are written and how they can be improved. We demonstrate the value of the ITEM Ontology by presenting two use cases for evaluating item quality and validating content. Keywords: ontology, psychometrics, indicator evaluation, validity, scale development Problem definition: What is your research problem/question in simple terms? Relevance: Why is this problem relevant for research and/or practice? Theoretical foundations: What theory (theories) underlie this work (if applicable)? Methodology: What is the research approach used (if applicable)? Anything innovative? Results: What are the key findings that readers may find interesting or revelatory? Contributions: What is novel/impactful about the knowledge offered by this paper for scholars, practitioners, and policymakers? Short quote, video, or interview (optional): Send us anything additional that you would like to include in the social media post.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.006 | 0.002 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.002 | 0.010 |
| Science and technology studies | 0.001 | 0.000 |
| Scholarly communication | 0.001 | 0.001 |
| Open science | 0.002 | 0.000 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it