Enhancing the scoping study methodology: a large, inter-professional team’s experience with Arksey and O’Malley’s framework
Bibliographic record
Abstract
BACKGROUND: Scoping studies are increasingly common for broadly searching the literature on a specific topic, yet researchers lack an agreed-upon definition of and framework for the methodology. In 2005, Arksey and O'Malley offered a methodological framework for conducting scoping studies. In their subsequent work, Levac et al. responded to Arksey and O'Malley's call for advances to their framework. Our paper builds on this collective work to further enhance the methodology. DISCUSSION: This paper begins with a background on what constitutes a scoping study, followed by a discussion about four primary subjects: (1) the types of questions for which Arksey and O'Malley's framework is most appropriate, (2) a contribution to the discussion aimed at enhancing the six steps of Arskey and O'Malley's framework, (3) the strengths and challenges of our experience working with Arksey and O'Malley's framework as a large, inter-professional team, and (4) lessons learned. Our goal in this paper is to add to the discussion encouraged by Arksey and O'Malley to further enhance this methodology. SUMMARY: Performing a scoping study using Arksey and O'Malley's framework was a valuable process for our research team even if how it was useful was unexpected. Based on our experience, we recommend researchers be aware of their expectations for how Arksey and O'Malley's framework might be useful in relation to their research question, and remain flexible to clarify concepts and to revise the research question as the team becomes familiar with the literature. Questions portraying comparisons such as between interventions, programs, or approaches seem to be the most suitable to scoping studies. We also suggest assessing the quality of studies and conducting a trial of the method before fully embarking on the charting process in order to ensure consistency. The benefits of engaging a large, inter-professional team such as ours throughout every stage of Arksey and O'Malley's framework far exceed the challenges and we recommend researchers consider the value of such a team. The strengths include breadth and depth of knowledge each team member brings to the study and time efficiencies. In our experience, the most significant challenges presented to our team were those related to consensus and resource limitations. Effective communication is key to the success of a large group. We propose that by clarifying the framework, the purposes of scoping studies are attainable and the definition is enriched.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.536 | 0.555 |
| Meta-epidemiology (narrow) | 0.002 | 0.003 |
| Meta-epidemiology (broad) | 0.003 | 0.003 |
| Bibliometrics | 0.012 | 0.013 |
| Science and technology studies | 0.019 | 0.031 |
| Scholarly communication | 0.030 | 0.032 |
| Open science | 0.010 | 0.042 |
| Research integrity | 0.014 | 0.028 |
| Insufficient payload (model declined to judge) | 0.004 | 0.002 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; the direct Gemma label and the distilled Codex classifier agree on what is shown here.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".