Assessing the quality of the OPQ’s Guidelines for the Evaluation of Dyslexia in Children using the Appraisal of Guidelines for Research and Evaluation (AGREE) II: A brief report.
Bibliographic record
Abstract
Children with dyslexia have difficulties processing language in the areas of reading, writing, and spelling (Lyon, Shaywitz, & Shaywitz, 2003). The Diagnostic and Statistical Manual of Mental Disorders (5th ed.; American Psychiatric Association, 2013) classifies dyslexia as a specific learning disability and considers it to be a type of neurodevelopmental disorder. The Diagnostic and Statistical Manual of Mental Disorders, fifth edition, reports that the prevalence of dyslexia in school-age children is between 3% and 5%, whereas other studies found the prevalence to be between 5% and 15% (Daniel et al., 2006). It is estimated that 80-90% of learning difficulties present themselves in the school system (Leach, Scarborough, & Rescorla, 2003; Lerner, 1989; Lyon et al., 2001). Psychologists in both school and clinical settings may benefit from high-quality professional practice guidelines because they are expected to provide scientifically based recommendations to guide service delivery.The Quebec Order of Psychologists (OPQ), a prolific producer of practice guidelines, published the Guidelines for the evaluation of dyslexia in children (Lignes directrices pour l'evaluation de la dyslexie chez les enfants) in 2014. The production of this guideline began in 2009 with the participation of multiple experts. This 48-page document provides background information on dyslexia and describes so-called best practices for the evaluation of the condition, including the administration of tests. The guideline's stated objectives are to do the following: (a) to encourage rigorous methods that reflect the most up-to-date research; (b) to propose a standardized methodology based on diagnostic consensus; and (c) to identify the specific needs of children affected by dyslexia for support and accommodation (Ordre des psychologues du Quebec [OPQ], 2014).Although such objectives are both important and laudable, the availability of practice guidelines alone does not automatically lead to improved practices or health outcomes (Bergman, 1999; Cabana et al., 1999; Sackett, Rosenberg, Gray, Haynes, & Richardson, 1996; Ward & Grieco, 1996). The potential benefit of a guideline is only as good as the quality of the guideline itself (Burgers, Cluzeau, Hanna, Hunt, & Grol, 2003; Gordon & Cooper, 2010), and appropriate methodologies and rigorous strategies in the guideline development process are important for the successful dissemination and implementation of the clinical recommendations contained in the guideline (Alonso-Coello et al., 2010; Ansari & Rashidian, 2012; Blozik et al., 2012; Cahill & Heyland, 2010; Davis & Taylor-Vaisey, 1997; Grol, 2001; Norris, Holmer, Ogden, & Burda, 2011; Steinert, Richter, & Bergk, 2010). Unfortunately, in recent years, there has been a large volume of guidelines published in which the methodologies used to design the guidelines were inadequately described within the guideline or in related documents made available to professionals and service users (e.g., Rosenfeld, Shiffman, & Robertson, 2013; Stamoulos, Reyes, Trepanier, & Drapeau, 2014). For example, a description of the methods used to gather evidence and formulate recommendations, and whether the guideline was externally reviewed by experts prior to its publication, are considered minimal standards for the proLyane duction of quality guidelines (Brouwers et al., 2010; Burgers, Cluzeau, Hanna, Hunt, & Grol, 2003).The quality of practice guidelines has indeed been closely scrutinized over the past 15 years, especially in the United States and Europe, with reports of high variability in quality, many of which were cited as poor (Graham, Beardall, Carter, Tetroe, & Davies, 2003; Ruszczynski, Horvath, Dziechciarz, & Szajewska, 2016; Shaneyfelt, Mayo-Smith, & Rothwangl, 1999; Stamoulos, Reyes, Trepanier, & Drapeau, 2014). These studies raise concerns about the quality of the guidelines that are available to practitioners, about the value of the advice and recommendations they contain, and ultimately about the quality of the services received by patients and clients. …
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.129 | 0.229 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.004 | 0.005 |
| Bibliometrics | 0.014 | 0.008 |
| Science and technology studies | 0.002 | 0.002 |
| Scholarly communication | 0.004 | 0.003 |
| Open science | 0.003 | 0.005 |
| Research integrity | 0.002 | 0.003 |
| Insufficient payload (model declined to judge) | 0.003 | 0.002 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".