Olympus, enhanced: benchmarking mixed-parameter and multi-objective optimization in chemistry and materials science
Why this work is in the frame
A frame that forgets how it found something cannot be audited. These are the routes that admitted this work.
Bibliographic record
Abstract
Experiment planning algorithms are a required component of autonomous platforms for scientific discovery. Selecting a suitable optimization algorithm for a novel application is an important yet difficult choice a researcher has to make based on past empirical performance on similar tasks. To facilitate the evaluation of various algorithms on chemistry and materials science optimization tasks, we previously introduced OLYMPUS (Mach. Learn.: Sci. Technol. 2, 035021, 2021), a Python package providing a consistent and easy-to-use interface to numerous optimization algorithms and benchmark datasets. While the original package was limited to continuous parameters and single objectives, in this work we expand OLYMPUS' capabilities to include mixed (continuous, discrete, and categorical) parameter types and multiple objectives. Several experiment planning algorithms already contained in OLYMPUS are extended to handle categorical and discrete parameter types, and five additional planners are implemented (23 in total). We also provide 23 additional benchmark datasets taken from the chemistry and materials science literature (33 in total), covering a wide range of research areas, from chemical reaction optimization to materials manufacturing. Finally, the visualization capabilities of OLYMPUS are enhanced to allow for easy inspection of the results, and the core functionality of the package is embedded in a Streamlit web application for code-free usage. We demonstrate how OLYMPUS enables researchers to rapidly benchmark different optimization strategies and gain insight into their behavior by focusing on two case studies: the optimization of a Suzuki-Miyaura cross-coupling reaction with categorical reaction conditions, and the multi-objective optimization of redox-active materials. The updated OLYMPUS package provides practitioners with a large suite of tools to efficiently benchmark and analyze experiment planning algorithms on mixed-parameter and multi-objective optimization tasks.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.003 | 0.002 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.001 | 0.000 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.000 | 0.001 |
| Scholarly communication | 0.001 | 0.000 |
| Open science | 0.001 | 0.003 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it