A Fully Integrated Systematic Review of Mixed Methods Design-Based Research
Bibliographic record
Abstract
Design-based research (DBR) is an educational research methodology that is commonly used in the fields of education, instructional technology, and learning sciences. When conducting DBR, researchers collaborate with practitioners (e.g., educators) and other stakeholders (e.g., parents, community members), often including the learners themselves, for the purpose of developing and evaluating innovative solutions to real-world problems within specific contexts, with a primary focus on improving practice and generating practical knowledge. DBR is particularly suited to mixed methods research. However, it is not clear the extent to which mixed methods research approaches are used in DBR studies, as opposed to monomethod research approaches that involve the sole use of qualitative research approaches or the sole use of quantitative research approaches. Therefore, in this study, what we refer to as a fully integrated systematic review of Scopus-indexed works from January 1, 1960 to May 31, 2022 was conducted to determine the prevalence of mixed methods DBR (MM-DBR) studies. This review yielded only 68 published works wherein the author explicitly declared their study as representing some form of a MMDBR study, with the majority of these MM-DBR studies being published within the last decade. Most notably, for all but 4 of these 68 studies, the level of integration occurred at the low end of the integration continuum, being characterized by mixed methods research designs wherein integration only occurred at the interpretation stage of the DBR process. More than two thirds of the authors (29.2%) neither explicitly specified nor described adequately their mixed methods research design. More than one half (i.e., 56.9%) of the MM-DBR studies were not grounded within the mixed methods research literature to any degree at all. Most notably, for all but four studies (i.e., 5.88%), the level of integration occurred at the low end of the integration continuum wherein integration only occurred at the interpretation stage of the MM-DBR process, representing only partial integration of the quantitative and qualitative research components/phases/cycles. As such, we call for more DBR researchers not only to consider using mixed methods research approaches but also to consider using full(er) integration approaches, as we move further into the fifth Industrial Revolution and beyond.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.119 | 0.293 |
| Meta-epidemiology (narrow) | 0.003 | 0.002 |
| Meta-epidemiology (broad) | 0.009 | 0.010 |
| Bibliometrics | 0.045 | 0.034 |
| Science and technology studies | 0.003 | 0.003 |
| Scholarly communication | 0.009 | 0.009 |
| Open science | 0.005 | 0.007 |
| Research integrity | 0.005 | 0.002 |
| Insufficient payload (model declined to judge) | 0.009 | 0.002 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".