Bibliographic record
Abstract
HyperQueue HyperQueue is a runtime for ergonomic and efficient execution of task graphs and workflows on HPC clusters. It scales to millions of tasks and hundreds of nodes, supports complex resource requirements and heterogeneous cluters, achieves high node utilization and can automatically allocate PBS/Slurm jobs on behalf of the user. Version 0.13.0 changelog New features Resource management Almost complete rewrite of resource management. CPU and other resources were unified: the most visible change is that you can define "cpus" and other resource; and other resources can now be defined in groups (NUMA-like resources). Many improvements in scheduler: Improved schedules for multi-resource requests; better behavior on non-heterogeneous clusters; better interaction between resources and priorities. Automatic allocation #467 You can now pause (and resume) autoalloc queues using hq alloc pause and hq alloc resume. Paused queues will not submit new allocations into the selected job manager. They can be later resumed. When an autoalloc queue hits too many submission or worker execution errors, it will now be paused instead of removed. Tasks HQ allows to limit how many times a task may be in a running state while worker is lost (such a task may be a potential source of worker's crash). If the limit is reached, the task is marked as failed. The limit can be configured by --crash-limit in submit. Groups of workers are introduced. A multi-node task is now started only on workers from the same group. By default, workers are grouped by PBS/Slurm allocations, but it can be configured manually. Changes Resource management --cpus=no-ht is now changed to a flag --no-hyper-threading. Explicit list definition of a resource was changed from --resource xxx=list(1,2,3) to --resource xxx=[1,2,3]. (this is the result of unification of CPUs with other resources). Python API: Attribute generic in ResourceRequest is renamed to resources Tasks #461 When a task is cancelled, times out or its worker is killed, HyperQueue now tries to make sure that both the tasks and any processes that it has spawned will be also terminated. #480 You can now select multiple tasks in hq task info. Artifact summary: hq-v0.13.0-*: Main HyperQueue build containing the hq binary. Download this archive to use HyperQueue from the command line. hyperqueue-0.13.0-*: Wheel containing the hyperqueue package with HyperQueue Python bindings.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.005 | 0.011 |
| Meta-epidemiology (narrow) | 0.004 | 0.004 |
| Meta-epidemiology (broad) | 0.002 | 0.003 |
| Bibliometrics | 0.002 | 0.001 |
| Science and technology studies | 0.001 | 0.001 |
| Scholarly communication | 0.006 | 0.005 |
| Open science | 0.009 | 0.007 |
| Research integrity | 0.003 | 0.007 |
| Insufficient payload (model declined to judge) | 0.224 | 0.242 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".