Why this work is in the frame
A frame that forgets how it found something cannot be audited. These are the routes that admitted this work.
Bibliographic record
Abstract
In this workshop, we will demonstrate four research tools: 1) card sorting, 2) upward laddering, 3) think-aloud, and 4) a search visualizer. Card sorting can be useful as an exploratory knowledge modelling method to gain insight into people’s understanding of their surrounding world. Derived from Kelly’s Personal Construct Theory (PCT) (Kelly, 1955, Herd, 2001), card sorting can help to explore peoples’ constructs and their perceptions of how the constructs relate to each other; in other words, card sorting can help to elucidate mental frameworks. Although there are many kinds of card sorting, we will guide participants through a hands-on exercise using single-criterion card sorting (using both digital and in person techniques). Participants will learn about text-based, image-based, and object-based options, when to use card sorting, how to collect data, and how to analyse card sorting data through visual-numeric ‘heat maps’ (co-occurrence matrices). The card sorting exercise will lead into upward laddering, a technique that is often used to complement card sorting. While card sorting provides evidence of how constructs are related, upward laddering allows exploration into goals and values (Rugg & Gerrard, 2023). Workshop participants will have an opportunity to access a simple, robust, newly created online tool for upward laddering. Also used alongside card sorting is the think-aloud method which involves both observation to see how people perform a task and think-aloud to hear what people are thinking and noticing while they perform the task. This inexpensive, easy-to-use method allows researchers to tap into reasons and tacit knowledge. The fourth tool we will demonstrate is a newly launched search visualiser (SV). Using keywords, this search tool allows researchers to comb through specific databases and/or to access Google results. Rather than simply return a list of links to articles, the SV returns a visual depiction of the keywords within each text. Each key word is represented as a coloured square. A user can hover their mouse pointer over a given square to see the phrase within which the word appears. Using this tool, researchers can get a better, visual sense of whether the article is likely to offer useful content. There is now a version with audio for visually impaired users as well as a version that can explore synonyms to support textual-literary analysis.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.001 | 0.000 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.001 | 0.000 |
| Research integrity | 0.000 | 0.001 |
| Insufficient payload (model declined to judge) | 0.004 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it