Interpretive collaborative review: enabling multi-perspectival dialogues to generate collaborative assignments of relevance to information resources in a dedicated problem domain
Bibliographic record
Abstract
Interpretive Collaborative Review (ICR) is a process designed to assemble electronically accessible research papers and other forms of information into collaboratively interpreted guides to information artefacts relevant to particular problems. The purpose of ICR is to enable collective understanding of a selected problem area that can be developed and represented by evaluating (reviewing) selected artefacts through a collaborative deliberation process. ICR has been conceptually formalized as an online environment enabling collaborative evaluation of relevancy relationships articulated in the triad of: 1) specific problems (topic), 2) diverse stakeholders and reviewer perspectives (context), and 3) particular settings where the problem matters (task). We define relevance as a cognitive recognition of proximal meaning relationships among the triad nodes of topic, task, and context. Three necessary dimensions of relevance relationships are proposed: 1) precedence, 2) validity, and 3) maturity. Based on experience with other forms of collaborative knowledge construction such as structured dialogue and cooperative learning, we conceptualized the ICR process as encompassing three phases: 1) discovery, promoting initial interpretations and definition, 2) deliberation, promoting emerging understanding and acceptance of degrees of interpretation within the group and 3) dissemination, promoting summation, validation, and distribution or publication of conclusions. The ICR method starts by recruiting a community of reviewers with necessarily diverse perspectives who agree to collaborate in identifying and evaluating information artefacts that can inform knowledge construction centered on a problem of common interest. A discovery phase allows reviewers to declare perspectives that are further delimited and explored collaboratively through the use of group dialogue around challenge questions. This is followed by a deliberative phase that facilitates collaborative dialogue aimed at developing a shared understanding of available information artefacts and their significance and of how those sources are relevant to the problem context. A final dissemination phase involves recording and publishing the knowledge synthesis and innovation that emerged from this collaborative dialogical process to affect knowledge transfer. Alignment of perspectives is promoted through collaborative generation of an aggregated report that describes the perceived relevancy relationships for each knowledge artefact evaluated in the review collection. While useful by itself, this report also serves as the raw material for a new form of scholarly publication, the 3D-Review, where relevancy relationships are used to guide suggested actions that could be taken with respect to advancing knowledge of the problem and options for addressing it. Both reports and reviews are indexable and electronically accessible, allowing other communities or individuals to find, retrieve, and act upon the new knowledge associated with the reports and reviews. This process of rigorous and purposeful deliberation enabled through online support of honest dialogue has the potential to develop into a new form of scholarly activity that should be useful in integrative scholarship.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.064 | 0.134 |
| Meta-epidemiology (narrow) | 0.002 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.002 |
| Bibliometrics | 0.006 | 0.003 |
| Science and technology studies | 0.005 | 0.009 |
| Scholarly communication | 0.016 | 0.013 |
| Open science | 0.005 | 0.021 |
| Research integrity | 0.004 | 0.003 |
| Insufficient payload (model declined to judge) | 0.008 | 0.002 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".