Methods for data extraction and data transformation in convergent integrated mixed methods systematic reviews
Bibliographic record
Abstract
OBJECTIVE: The objective of this guidance paper is to describe data transformation involving qualitization, including when and how to undertake this process, and to clarify how it aligns with data extraction in order to expand on the current guidance for JBI convergent integrated mixed methods systematic reviews (MMSRs). INTRODUCTION: The convergent integrated approach to MMSRs involves combining extracted data from both quantitative studies (including the quantitative components of mixed methods studies) and qualitative studies (including the qualitative components of mixed methods studies). This process requires data transformation, which can occur either by converting qualitative data into quantitative data (ie, quantitizing ) or converting quantitative data into qualitative data (ie, qualitizing ). Data transformation involving qualitization is poorly understood in the context of MMSRs, and there is confusion regarding how to undertake this process, with much of the literature specific to primary mixed methods studies. There is a need to expand current guidance and provide more practical advice to reviewers on how to undertake this process. METHODS: The JBI MMSR Methodology Group took a multipronged approach to update its guidance. First, a structured search of the literature was conducted to determine what is known about data transformation, followed by analysis of a sample of systematic reviews that claimed to use the JBI convergent integrated approach to MMSRs. Approaches were summarized and used to inform the development of draft guidance. This guidance was iteratively revised following a series of online meetings, as well as presented to evidence synthesis experts at an international conference. Finally, the guidance was submitted to the JBI International Scientific Committee for discussion, feedback, and ratification. RESULTS: There is uncertainty in the literature regarding the process of data transformation within the context of MMSRs, with ill-defined approaches provided and variation in practice. In JBI convergent integrated MMSRs, it is recommended that data extraction from quantitative studies (or mixed method studies reporting quantitative findings) stays as close as possible to the data reported in the primary studies. Where data are absent or insufficient to meet the needs of the MMSR, systematic reviewers may need to construct the narrative representation using relevant data from the primary studies. Following data extraction, the process of qualitization occurs where extracted data (both quantitative and qualitative) are assembled, and reviewers are required to conduct detailed examination across data to identify likenesses and create categories based on similarities in meaning. CONCLUSION: To our knowledge, this is the most comprehensive guidance currently available for data extraction and qualitization for MMSRs. However, it is important to acknowledge the inherent variability in MMSRs and our methodology may need tailoring for certain situations. Further work will focus on examining how certainty and confidence in findings can be assessed within the framework of MMSRs.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.576 | 0.752 |
| Meta-epidemiology (narrow) | 0.007 | 0.007 |
| Meta-epidemiology (broad) | 0.012 | 0.021 |
| Bibliometrics | 0.036 | 0.041 |
| Science and technology studies | 0.005 | 0.009 |
| Scholarly communication | 0.016 | 0.012 |
| Open science | 0.011 | 0.015 |
| Research integrity | 0.009 | 0.012 |
| Insufficient payload (model declined to judge) | 0.051 | 0.017 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; the direct Gemma label and the distilled Codex classifier agree on what is shown here.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".