Reusing Garbage Data for Efficient Workflow Computation
Why this work is in the frame
A frame that forgets how it found something cannot be audited. These are the routes that admitted this work.
Bibliographic record
Abstract
High-performance computing (HPC) systems, including Clusters, Grids and the most recent Clouds, have emerged as attractive platforms to tackle various applications. One significant type of applications in the HPC systems is workflow computation, which has been applied in various scientific and engineering domains. The workflow computation frequently produces intermediate result files, which become garbage after being used and are usually cleaned up without making any contribution to future computation. In this paper, we argue that such garbage data could be useful in the future computation and should not be immediately cleaned up. This is because workflow computation usually contains multiple instances that may share some common data products produced in the past. This sharing scheme provides opportunities to reuse the historical data to speed-up subsequent computation and simplify re-computation due to faulty or crashed runs. To this end, we propose a novel approach, referred to as garbage data manager (GDM), for the workflow computation in HPC systems. The GDM organizes and manages the garbage data for batch schedulers to enhance the performance of subsequent computation. The essence of the GDM is to record the history of computation by constructing a dataflow graph on per instance (run) basis and set up inheritance relationships between the different instances of the same workflow, called run-tree, to achieve the data reuse. Our simulation results demonstrate that exploiting the garbage data is an effective way of improving the workflow computation.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.011 | 0.001 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.001 |
| Science and technology studies | 0.001 | 0.000 |
| Scholarly communication | 0.004 | 0.000 |
| Open science | 0.004 | 0.002 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it