THE DESIGN OF A COLLABORATION RUBRIC FROM ITS VALUE SYSTEM
Bibliographic record
Abstract
In this paper the authors explore the use of rubrics for the evaluation of collaborations and its agents in both academic and practical settings. Rubrics are subjective scoring guides used for the quick evaluation of the characteristics of a concept based on a range of criteria. A comparative analysis of these rubrics suggest that collaborations and the collaborators are inconsistently evaluated based on the current design of these metrics. This inconsistency is captured through the choice of characteristics pertaining to collaboration, the use of these characteristics across rubrics designed for evaluating collaboration, the criteria pertaining to each characteristic, and the distribution of this range of criteria. This inconsistency misinforms the collaborators, misdirects the collaboration, distracts from appreciating the possibilities of collaboration through its involvement and transference of its lessons. It is suggested that the source of this error extends from an inconsistency in the understanding of collaboration and of the behaviours and attitudes expected of the collaborator. It is further suggested that value systems underlie one's attitudes toward collaboration, including which collaborative behaviours are viewed favourably, and a better understanding of the underlying values will help address the above-noted inconsistencies. An alternate rubric design will be proposed which will reflect a short list of favourable behaviours and attitudes pertaining to the value system of collaboration. The aim is to at least capture the context in which only a collaboration can exist, or at least, its opponents cannot. As opposed to the current designs for collaboration rubrics, it is believed that this alternative design will enable the evaluator to capture the presence of a collaboration and the strength of its agents. In addition, it will provide an appropriate direction and guidance in areas pertaining to collaboration, its related technologies, and this provides the opportunity for further exploration of collaborative activities, its benefits, and challenges.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.030 | 0.071 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.007 | 0.004 |
| Science and technology studies | 0.003 | 0.003 |
| Scholarly communication | 0.006 | 0.008 |
| Open science | 0.003 | 0.005 |
| Research integrity | 0.002 | 0.003 |
| Insufficient payload (model declined to judge) | 0.008 | 0.004 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".