Navigating Knowledge
Bibliographic record
Abstract
In this workshop, we will demonstrate four research tools: 1) card sorting, 2) upward laddering, 3) think-aloud, and 4) a search visualizer. Card sorting can be useful as an exploratory knowledge modelling method to gain insight into people’s understanding of their surrounding world. Derived from Kelly’s Personal Construct Theory (PCT) (Kelly, 1955, Herd, 2001), card sorting can help to explore peoples’ constructs and their perceptions of how the constructs relate to each other; in other words, card sorting can help to elucidate mental frameworks. Although there are many kinds of card sorting, we will guide participants through a hands-on exercise using single-criterion card sorting (using both digital and in person techniques). Participants will learn about text-based, image-based, and object-based options, when to use card sorting, how to collect data, and how to analyse card sorting data through visual-numeric ‘heat maps’ (co-occurrence matrices). The card sorting exercise will lead into upward laddering, a technique that is often used to complement card sorting. While card sorting provides evidence of how constructs are related, upward laddering allows exploration into goals and values (Rugg & Gerrard, 2023). Workshop participants will have an opportunity to access a simple, robust, newly created online tool for upward laddering. Also used alongside card sorting is the think-aloud method which involves both observation to see how people perform a task and think-aloud to hear what people are thinking and noticing while they perform the task. This inexpensive, easy-to-use method allows researchers to tap into reasons and tacit knowledge. The fourth tool we will demonstrate is a newly launched search visualiser (SV). Using keywords, this search tool allows researchers to comb through specific databases and/or to access Google results. Rather than simply return a list of links to articles, the SV returns a visual depiction of the keywords within each text. Each key word is represented as a coloured square. A user can hover their mouse pointer over a given square to see the phrase within which the word appears. Using this tool, researchers can get a better, visual sense of whether the article is likely to offer useful content. There is now a version with audio for visually impaired users as well as a version that can explore synonyms to support textual-literary analysis.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.003 | 0.012 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.003 | 0.004 |
| Science and technology studies | 0.004 | 0.003 |
| Scholarly communication | 0.011 | 0.024 |
| Open science | 0.003 | 0.018 |
| Research integrity | 0.003 | 0.002 |
| Insufficient payload (model declined to judge) | 0.040 | 0.010 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".