Controlling for Placebo Effects in Computerized Cognitive Training Studies With Healthy Older Adults From 2016-2018: Systematic Review
Why this work is in the frame
A frame that forgets how it found something cannot be audited. These are the routes that admitted this work.
Bibliographic record
Abstract
BACKGROUND: Computerized cognitive training has been proposed as a potential solution to age-related cognitive decline. However, published findings from evaluation studies of cognitive training games, including metastudies and systematic reviews, provide evidence both for and against transferability from trained tasks to untrained cognitive ability. There continues to be no consensus on this issue from the scientific community. Some researchers have proposed that the number of results supporting the efficacy of cognitive training may be inflated due to placebo effects. It has been suggested that placebo effects need to be better controlled by using an active control and measuring participant expectations for improvement in outcome measures. OBJECTIVE: This review examined placebo control methodology for recent evaluation studies of computerized cognitive training programs with older adult subjects, specifically looking for the use of an active control and measurement of expectations. METHODS: Data were extracted from PubMed. Evaluation studies of computerized cognitive training with older adult subjects (age ≥50 years) published between 2016 and 2018 were included. Methods sections of studies were searched for (1) control type (active or passive) and subtype (active: active-ingredient or similar-form; passive: no-contact or passive-task); (2) if expectations were measured, how were they measured, and whether they were used in analysis; and (3) whether researchers acknowledged a lack of active control and lack of expectation measurement as limitations (where appropriate). RESULTS: Of the 19 eligible studies, 4 (21%) measured expectations, and 9 (47%) included an active control condition, all of which were of the similar-form type. The majority of the studies (10/19, 53%) used only a passive control. Of the 9 studies that found results supporting the efficacy of cognitive training, 5 were for far transfer effects. Regarding the limitations, due to practical considerations, the search was limited to one source (PubMed) and to search results only. The search terms may have been too restrictive. Recruitment methods were not analyzed, although this aspect of research may play a critical role in systematically forming groups with different expectations for improvement. The population was limited to healthy older adults, while evaluation studies include other populations and cognitive training types, which may exhibit better or worse placebo control than the studies examined in this review. CONCLUSIONS: Poor placebo control was present in 47% (9/19) of the reviewed studies; however, the studies still published results supporting the effectiveness of cognitive training programs. Of these positive results, 5 were for far transfer effects, which form the basis for broad claims by cognitive training game makers about the scientific validity of their product. For a minimum level of placebo control, future evaluation studies should use a similar-form active control and administer a questionnaire to participants at the end of the training period about their own perceptions of improvement. Researchers are encouraged to think of more methods for the valid measure of expectations at other time points in the training.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.001 | 0.002 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.009 | 0.001 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.001 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it