MétaCan
Menu
Back to cohort
Record W4306250992 · doi:10.4300/jgme-d-22-00621.1

Steps for Conducting a Scoping Review

2022· review· en· W4306250992 on OpenAlexaff
Susanne Mak, Aliki Thomas

Bibliographic record

VenueJournal of Graduate Medical Education · 2022
Typereview
Languageen
FieldHealth Professions
TopicHealth Sciences Research and Education
Canadian institutionsMcGill University
Fundersnot available
KeywordsData scienceComputer scienceMEDLINEMedical educationMedicinePolitical science

Abstract

fetched live from OpenAlex

A scoping review is a type of knowledge synthesis that uses a systematic and iterative approach to identify and synthesize an existing or emerging body of literature on a given topic.1 While there are several reasons for conducting a scoping review, the main reasons are to map the extent, range, and nature of the literature, as well as to determine possible gaps in the literature on a topic.1-3 Scoping reviews are not limited to peer-reviewed literature.3,4Before conducting the review, it is important to consider the composition of the research team: scoping reviews are not conducted by a single individual. The team should include someone with content expertise and an individual with experience conducting scoping reviews.1,3,5 Adding a librarian who can assist with building the search strategy is also extremely helpful.1,3 Thoughtful planning of the team membership will result in the right knowledge, skills, and expertise to successfully complete the review and ensure that the findings make a notable contribution to the field.An overview of the steps involved in conducting scoping reviews is provided below.Creating the research question is a vital first step.1,3-5 A question that is too broad increases the number of papers for consideration, which may affect the feasibility of the review.5 A question that is too narrow may compromise the breadth and depth of the review. Therefore, a preliminary search of the literature may be helpful in determining: (1) the breadth of your question; (2) whether a scoping review on the topic has already been conducted; and (3) if there is sufficient literature to warrant a scoping review. Consulting with a librarian can help in deciding if a scoping review is the appropriate review method.1,3 In particular, a librarian may confirm that there is insufficient literature or that there is too much, which will necessitate a more targeted research question.Early consultation with a librarian should occur to build the search strategy—keywords, Medical Subject Headings, databases—and further refine the strategy based on the papers found. For example, you may find too many irrelevant papers. In this case you may need to review your search strategy to identify the terms which introduce too much “noise.”You will also need to define the inclusion and exclusion criteria.1,3-5 Discussions with your team are important to ensure diverse perspectives and that the inclusion criteria are aligned with the research question.5,6Tools such as Covidence and Rayyan can be helpful in organizing papers and making the screening process more efficient (Box). Once you have collected the citations from the search, you can import these from reference management software (eg, EndNote) into Rayyan. After selecting papers for inclusion, the citations of the included papers can be exported to reference management software for the next stage of the review. Other helpful features of management software can include the identification of duplicates, proportion of an abstract that resembles another, and documentation of reasons for inclusion or exclusion. Both Covidence and Rayyan allow for blinding the results of team members' reviews to each other.Having additional reviewers will accelerate the pace of the review but will require calibration between reviewers.1,3,5 A calibration exercise consists of selecting 5% to 10% of the papers for independent screening by each reviewer.1 If a high level of agreement among reviewers is not achieved (eg, lower than 90%),7,8 the reviewers should discuss their points of disagreement and review (and possibly revise) the inclusion criteria.1 Another 10% of the papers are then selected for a second calibration exercise to test the modified inclusion criteria. If having 2 reviewers for each paper is not feasible, one reviewer can conduct an independent review, with a second reviewer verifying a portion of the papers, with the goal of 90% or better agreement.The actual screening of papers should consist of reading not only the title of the paper, but the abstract as well. If an abstract is not available, a full-text review of the paper is required. Screening papers by title alone is insufficient, as the contents of a paper are not always well reflected in the title.The team develops the data extraction form collaboratively. Although the extraction categories vary depending on the research question and review purpose, common categories are: author, year, geographical location, study population, main results, study limitations, and future directions.4,5 More specific categories will be needed to capture the data for a given research question.The extraction form will need to be pilot tested for further refinements and undergo a calibration exercise as well.1,3,5 This entails a dyad of reviewers independently extracting data from a small number of papers (eg, 5-10), and meeting afterward to discuss any discrepancies, with further refinement of the form if a high level of agreement between reviewers is not obtained.Once the data have been extracted from all papers, numerical and thematic analyses are conducted.5 The findings from the numerical analysis can be presented in a table or chart to showcase the most salient aspects of the review. Readers should be able to see alignment of findings with objectives for conducting the review.1,3 Thematic analysis9 consists of examining excerpts of text and asking how this text relates to the research question, as well as creating a code (label) that best reflects that text. A list of tentative codes (a codebook) is created and modified iteratively as the team engages in data analysis. Once codes are developed, a review of the codes and how they relate to each other can help to identify patterns among them, which leads to the creation of categories (collections of similar data in one place)10 and themes (patterns across the dataset).9Reflexivity is essential throughout the review process but especially during thematic analysis, with use of memos, to capture the thoughts that arise from examining and interpreting the data. Once the codes are generated, the research team will further refine them through discussion.6 The team should discuss not only the clarity of the operational definitions of the codes, but also how the codes are named and how they may relate to each other. As the codes are grouped together, the team will develop themes.5Reasons for stakeholder consultation may be to obtain input on the research question and sources of information, and to provide insights on a topic. Other purposes may include obtaining feedback to help shed light on the review findings and pinpoint gaps not explored in the literature. While a stakeholder consultation has been named as the final step of a review, it can be incorporated throughout the review stages and can occur through focus groups, individual interviews, or surveys.1,5A scoping review is useful to map the literature on evolving or emerging topics and to identify gaps. It may be a step before undertaking research or conducting another type of review, such as a systematic review. Before conducting a scoping review, it is important to consider how the research team will implement each step and who will be involved at each stage, while being mindful that the methodological approach provides teams with the opportunity to move back to earlier stages as the review evolves.

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.

How this classification was reachedexpand

Full frame machine prediction

Teacher imitation

Not calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.

metaresearch head score (Codex)0.357
metaresearch head score (Gemma)0.530
Version: metacan-v3-hybrid-931329e0061cValidation status: machine_predicted_unvalidated
Candidate categoriesMetaresearch
Consensus categoriesMetaresearch
DomainCandidate signal: Methods · Consensus signal: none
Study designCandidate signal: Not applicable · Consensus signal: none
GenreCandidate signal: Review · Consensus signal: none
Teacher disagreement score0.643
Threshold uncertainty score0.793

Distilled classifier scores by category (both heads)

CategoryCodexGemma
Metaresearch0.3570.530
Meta-epidemiology (narrow)0.0050.006
Meta-epidemiology (broad)0.0060.012
Bibliometrics0.0250.015
Science and technology studies0.0110.006
Scholarly communication0.0240.019
Open science0.0090.019
Research integrity0.0200.024
Insufficient payload (model declined to judge)0.0790.048

Machine scores (provisional)

The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.

Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.

Opus teacher head0.820
GPT teacher head0.708
Teacher spread0.112 · how far apart the two teachers sit on this one work
Validation statusscore_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it

Classification

machine, unvalidated

Machine predicted; the direct Gemma label and the distilled Codex classifier agree on what is shown here.

Study designNot applicable
DomainMethods
GenreReview

How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".

Quick stats

Citations587
Published2022
Admission routes1
Has abstractyes

Explore more

Same venueJournal of Graduate Medical EducationSame topicHealth Sciences Research and EducationFrench-language works237,207