Improving the usefulness of a tool for appraising the quality of qualitative, quantitative and mixed methods studies, the <scp>Mixed Methods Appraisal Tool</scp> (<scp>MMAT</scp>)
Why is this work in the frame?
A frame that forgets how it found something cannot be audited. These are the routes that admitted this work.
Full frame distilled prediction
Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
- Candidate categories
- Metaresearch, Science and technology studies
- Consensus categories
- Metaresearch
- Domain
- Candidate signal: MethodsConsensus signal: none
- Study design
- Candidate signal: QualitativeConsensus signal: Qualitative
- Genre
- Candidate signal: EmpiricalConsensus signal: none
- Teacher disagreement score
- 0.705
- Threshold uncertainty score
- 1.000
- Validation status
machine_predicted_unvalidated·codex-gemma-dda1882f352a
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.557 | 0.916 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.000 |
| Bibliometrics | 0.000 | 0.001 |
| Science and technology studies | 0.001 | 0.002 |
| Scholarly communication | 0.000 | 0.001 |
| Open science | 0.001 | 0.000 |
| Research integrity | 0.000 | 0.002 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
- Teacher spread
- 0.064 · how far apart the two teachers sit on this one work
- Validation status
score_only:v0-immature-baseline· verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it
Abstract
RATIONALE, AIMS, AND OBJECTIVES: Systematic reviews combining qualitative, quantitative, and/or mixed methods studies are increasingly popular because of their potential for addressing complex interventions and phenomena, specifically for assessing and improving clinical practice. A major challenge encountered with this type of review is the appraisal of the quality of individual studies given the heterogeneity of the study designs. The Mixed Methods Appraisal Tool (MMAT) was developed to help overcome this challenge. The aim of this study was to explore the usefulness of the MMAT by seeking the views and experiences of researchers who have used it. METHODS: We conducted a qualitative descriptive study using semistructured interviews with MMAT users. A purposeful sample was drawn from the researchers who had previously contacted the developer of the MMAT, and those who have published a systematic review for which they had used the MMAT. All interviews were transcribed verbatim and analyzed by 2 coders using thematic analysis. RESULTS: Twenty participants from 8 countries were interviewed. Thirteen themes were identified and grouped into the 2 dimensions of usefulness, ie, utility and usability. The themes related to utility concerned the coverage, completeness, flexibility, and other utilities of the tool. Those regarding usability were related to the learnability, efficiency, satisfaction, and errors that could be made due to difficulties understanding or selecting the items to appraise. CONCLUSIONS: On the basis of the results of this study, we make several recommendations for improving the MMAT. This will contribute to greater usefulness of the MMAT.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
The record
- Venue
- Journal of Evaluation in Clinical Practice
- Topic
- Health Policy Implementation Science
- Field
- Health Professions
- Canadian institutions
- McGill University
- Funders
- Canadian Institutes of Health Research
- Keywords
- UsabilityFlexibility (engineering)Psychological interventionThematic analysisQuality (philosophy)Qualitative researchSample (material)PsychologyQualitative propertyComputer scienceMedicineNursingHuman–computer interactionSociologyChemistryMathematicsStatistics
- Has abstract in OpenAlex
- yes