MétaCan
Menu
Back to cohort
Record W276684465

The One-Legged High Jumper and the Perils of Prediction: Predicting Success for Students Based on Their Background Is More Accurate in the Aggregate Than in Individual Situations, Where It Should Never Be Applied

2012· article· en· W276684465 on OpenAlex

Why this work is in the frame

A frame that forgets how it found something cannot be audited. These are the routes that admitted this work.

aboutThe title or abstract carries a Canadian signal from the geographic lexicon.
no affNo Canadian affiliation: this work is invisible to an affiliation-only frame.
No Canadian affiliation. An affiliation-only frame, the usual design, would never have seen this work. It is one of the works that make the case for inverting the frame.

Bibliographic record

VenuePhi Delta Kappan · 2012
Typearticle
Languageen
FieldPsychology
TopicReading and Literacy Development
Canadian institutionsnot available
Fundersnot available
KeywordsSurprisePsychologyReading (process)Graduation (instrument)Mathematics educationPovertyLiteracySocial psychologyPedagogyPolitical scienceMathematics
DOInot available

Abstract

fetched live from OpenAlex

Pretty much everyone in education knows student outcomes are deeply affected by family background--that poverty, parental education, family interaction, and ethnicity are strongly correlated with how well students do and with whether they graduate from high school or go on to postsecondary education. We also know that students who fall behind often have a hard time catching up. That knowledge has helped design more effective school programs that can reduce achievement gaps. Yet, while it is important and powerful, knowledge about these correlations has a danger connected with it as well. That danger is the belief that we can predict an individual student's future based on his or her past. Lots of evidence shows that belief is not only wrong but can lead to lower expectations and self-fulfilling prophecies. Here's the problem. Educators will often say that they can look at children at age 8, or 10, or 12, and know which ones will be successful. In the aggregate, for large groups, such predictions have quite a high degree of validity, but they're much less useful when it comes to knowing the futures of individual students who will often surprise us--and themselves. Many studies have found that a large number of students defy negative expectations based on their backgrounds. A Canadian study (OECD, 2010) showed that nearly 40% of 15-year-olds with very low reading skills were in postsecondary education six years later, and another showed that students with poor literacy skills at age 15 made the largest gains in literacy by age 24 (OECD, 2012). A U.S. longitudinal study of 4,000 students (Hernandez, 2011) showed that poor reading at 3rd grade did predict failure to graduate from high school, but nonetheless more than 75% of poor readers in 3rd grade did eventually graduate, including about 70% of struggling readers from poor families. A thorough review of studies of high school graduation found that even complex prediction models with many variables did not have enough accuracy to make them useful in working with individual students (Gleason & Dynarski, 2002). In every study of this kind, a significant number of students who seem to have everything against them end up having good results. And if you don't trust the statistics, just think of examples in your own life and circle. All of us know people who overcame difficult life circumstances. When I speak to groups of educators, I often ask if there is someone in the audience who, at age 15 or 16, would have been regarded by her or his teachers as highly unlikely to succeed. I have yet to meet a group of educators where there was not someone who told me that in high school he or she was regarded by the school as having few prospects but who nonetheless went on to college and is now a successful teacher or principal. One of my favorite real-life examples of defying the odds is Arnie Boldt. Boldt lost a leg in a farm accident when he was three years old, yet became a world-class high jumper (you can find videos of him on YouTube) whose best jumps were well over 6 feet! And now, quite a bit later in life--he became an educator, by the way--he is a world-class paralympic cyclist! Looking at photos and videos of his accomplishments, one can't help but wonder who looked at a kid with one leg and saw a star high jumper? More to the point, how many other students are not getting the chance to develop their talents because we see the missing leg and not the talented person? Boldt himself is entirely modest about his amazing accomplishments. But, in the volunteer work he continues to do with young people who have lost limbs, he notes that the problem is usually not the kids' willingness to try, it's the fear and caution of the adults that gets in the way of their progress. In telling these stories I'm not suggesting that background factors don't matter; a huge amount of evidence tells us that they do. …

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.

Full frame distilled prediction

Teacher imitation

Not calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.

metaresearch head score (Codex)0.003
metaresearch head score (Gemma)0.000
Version: codex-gemma-dda1882f352aValidation status: machine_predicted_unvalidated
Candidate categoriesnone
Consensus categoriesnone
DomainCandidate signal: none · Consensus signal: none
Study designCandidate signal: Observational · Consensus signal: Observational
GenreCandidate signal: Empirical · Consensus signal: Empirical
Teacher disagreement score0.252
Threshold uncertainty score0.538

Codex and Gemma teacher scores by category

CategoryCodexGemma
Metaresearch0.0030.000
Meta-epidemiology (narrow)0.0000.000
Meta-epidemiology (broad)0.0000.000
Bibliometrics0.0000.000
Science and technology studies0.0000.000
Scholarly communication0.0000.000
Open science0.0010.000
Research integrity0.0000.000
Insufficient payload (model declined to judge)0.0000.000

Machine scores (provisional)

The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.

Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.

Opus teacher head0.088
GPT teacher head0.370
Teacher spread0.283 · how far apart the two teachers sit on this one work
Validation statusscore_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it