Establishing the Replicability and Generalizability of Multi-Study Longitudinal Research
Bibliographic record
Abstract
Abstract Replication and cross-validation of research findings across independent longitudinal studies is essential for a cumulative science. However, the interplay between harmonization, replication, and generalizability of results across interdisciplinary longitudinal studies can present remarkable challenges. Careful interpretation of multistudy results must include consideration of the age, birth cohort, health, and education of individuals in the sample, measurements, the number and spacing of assessments, and rates of response and attrition. Placed in a broader historical (or future) context, we must consider the representativeness of population sampling, historical period, and analytic method in understanding the replicability and generalizability of findings. In a multistudy context, harmonization can be considered at levels of research question, statistical models, and measurements and can minimize some sources of cross-study variability. I will discuss the challenges and benefits of harmonization and the coordinated analysis approach used by the IALSA research network to achieve results from multi-study integrative research.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.758 | 0.843 |
| Meta-epidemiology (narrow) | 0.002 | 0.002 |
| Meta-epidemiology (broad) | 0.004 | 0.007 |
| Bibliometrics | 0.009 | 0.006 |
| Science and technology studies | 0.005 | 0.011 |
| Scholarly communication | 0.010 | 0.008 |
| Open science | 0.006 | 0.009 |
| Research integrity | 0.004 | 0.005 |
| Insufficient payload (model declined to judge) | 0.003 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; the direct Gemma label and the distilled Codex classifier agree on what is shown here.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".