Adaptation of protocol M1-Alpha for bedside assessment of language
Bibliographic record
Abstract
The objective of this study was to adapt the M1-Alpha and to search for content validity evidence. The process of adaptation of the instrument was divided into four steps. Step 1: Three language and speech therapists, two of whom were experts on aphasiology and one, a master's student in human communication disorders, analyzed the instruments M1-Alpha (only available for research and for some clinicians), the Montreal-Toulouse Language Assessment Battery for Aphasia-BR (MTL-BR) and MTL-BR version B (not marketed), to check for possible needs for modifications and select the stimuli to compose the instrument being adapted. Step 2: Twenty-eight non-expert judges analyzed the representativeness of 65 drawings. When necessary, the judges could make suggestions for improvements. Step 3: Five judges, who are experts on assessment of aphasia, determined whether the pictorial stimuli were adequate while four other judges, experts on linguistics and phonological acquisition, determined if the words, pseudowords and sentences were appropriate. As in Step 2, the expert judges were instructed to suggest modifications when necessary Step 4: This step was divided into pilot study 1 and pilot study 2, both with healthy individuals. After modifications suggested by the judges, the first draft of the instrument was designed and then applied to the sample of pilot study 1, composed of seven individuals. Based on pilot study 1, punctuation errors and inadequacies of some stimuli could be identified. After the adjustments required, the latest version of the protocol M1-Alpha was designed and then called MTL-BR Brief. The instrument was applied to 63 individuals from pilot study 2. Analysis of data from steps 1 and 4 was descriptive. For step 2, index of agreement among judges was determined using simple percentage agreement. The results of the expert judges were analyzed by calculating the content validity ratio while the first-order agreement coefficient (AC1 statistic) of Gwet was used to check for the correlation between expert judges. After analysis of the instruments (Step 1), 65 figures were designed and redesigned and then analyzed by non-expert raters (Step 2), and 60 of them showed more than 80% agreement. After adjustments, they were analyzed by the expert judges (Step 3); all 23 words, pseudo words and phrases were considered to be appropriate, while 22 out of the 66 pictorial stimuli required changes. Subsequently, they were analyzed by the same expert judges, and there was high agreement AC1= 0,98 [IC=0,96-1]. On the basis of the pilot studies (Step 4), some errors were found and then corrected to produce the latest version of the instrument. In conclusion, satisfactory content validity could be achieved in the adaptation of the M1-Alpha test, originating the MTL-BR Brief Battery.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.062 | 0.097 |
| Meta-epidemiology (narrow) | 0.001 | 0.002 |
| Meta-epidemiology (broad) | 0.002 | 0.002 |
| Bibliometrics | 0.003 | 0.002 |
| Science and technology studies | 0.002 | 0.001 |
| Scholarly communication | 0.002 | 0.002 |
| Open science | 0.002 | 0.003 |
| Research integrity | 0.002 | 0.004 |
| Insufficient payload (model declined to judge) | 0.022 | 0.016 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".