Not a walk in the park: the ECVAM whole embryo culture model challenged with pharmaceuticals and attempted improvements with random forest design
Why this work is in the frame
A frame that forgets how it found something cannot be audited. These are the routes that admitted this work.
Bibliographic record
Abstract
BACKGROUND: The European Committee for the Validation of Alternative Methods (ECVAM) supported the development of a linear discriminant embryotoxicity prediction model founded on rat whole embryo culture (Piersma et al. (2004). Altern Lab Anim 32:275–307). Our goals were to (1) assess the accuracy of this model with pharmaceuticals, and (2) to use the data to develop a more accurate prediction model. METHODS: Sixty-one chemicals of known in vivo activity were tested. They were part of the ECVAM validation set (N513), commercially available pharmaceuticals (N531), and Pfizer chemicals that did not reach the market, but for which developmental toxicity data were available (N517). They were tested according to the ECVAM procedures. Fifty-seven of these chemicals were used for Random Forest modeling to develop an alternate model with the goal of using surrogate endpoints for simplified assessments and to improve the predictivity of the model. RESULTS: Using part of the ECVAM chemical test set, the ECVAM prediction model was 77% accurate. This approximated what was reported in the validation study (80%; Piersma et al. (2004). Altern Lab Anim 32:275–307). However, when confronted with novel chemicals, the accuracy of the linear discriminant model dropped to 56%. In an attempt to improve this performance, we used a Random Forest model that provided rankings and confidence estimates. Although the model used simpler endpoints, its performance was no better than the ECVAM linear discriminant model. CONCLUSIONS: This study confirms previous concerns about the applicability of the ECVAM prediction model to a more diverse chemical set, and underscores the challenges associated with developing embryotoxicity prediction models.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.003 | 0.000 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.001 | 0.002 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.001 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it