Bibliographic record
Abstract
Verification Witnesses This file describes the contents of an archive of the 10th Competition on Software Verification (SV-COMP 2021). https://sv-comp.sosy-lab.org/2021/ The competition was run by Dirk Beyer, LMU Munich, Germany. More information is available in the following article: Dirk Beyer. Software Verification: 10th Comparative Evaluation (SV-COMP 2021). In Proceedings of the 27th International Conference on Tools and Algorithms for the Construction and Analysis of Systems (TACAS 2021, Luxembourg, March 27 - April 1), 2021. Springer. Copyright (C) Dirk Beyer https://www.sosy-lab.org/people/beyer/ SPDX-License-Identifier: CC-BY-4.0 https://spdx.org/licenses/CC-BY-4.0.html Contents LICENSE.txt: specifies the license README.txt: this file witnessFileByHash/: This directory contains verification witnesses. Each verification witness in this directory is stored in a file whose name is the SHA2 256-bit hash of its contents followed by the filename extension .graphml. The format of each verification witness is described on the format web page: https://github.com/sosy-lab/sv-witnesses/ A verification witness contains also metadata in order to relate it to the verification task for which it was produced. witnessInfoByHash/: This directory contains for each verification witness in directory witnessFileByHash/ a record in JSON format (also using the SHA2 256-bit hash of the witness as filename, with .json as filename extension) that contains the meta data. witnessListByProgramHashJSON/: For convenient access to all verification witnesses for a certain program, this directory represents a function that maps each program (via its SHA2256-bit hash) to a set of verification witnesses (JSON records for verification witnesses as described above) that the verification tools have produced for that program. For each program for which verification witnesses exist, the directory contains a JSON file (using the SHA2 256-bit hash of the program as filename, with .json as filename extension) that contains all JSON records for verification witnesses for that program. The data structure is described in the following article: Dirk Beyer. A Data Set of Program Invariants and Error Paths. In Proceedings of the 2019 IEEE/ACM 16th International Conference on Mining Software Repositories (MSR 2019, Montreal, Canada, May 26-27), pages 111-115, 2019. IEEE. https://doi.org/10.1109/MSR.2019.00026 Other Archives Overview over archives from SV-COMP 2021 that are available at Zenodo: https://doi.org/10.5281/zenodo.4459196 Witness store (containing the generated verification witnesses) https://doi.org/10.5281/zenodo.4458215 Results (XML result files, log files, file mappings, HTML tables) https://doi.org/10.5281/zenodo.4459126 Verification tasks, version svcomp21 https://doi.org/10.5281/zenodo.4317433 BenchExec, version 3.6 All benchmarks were executed for SV-COMP 2021 https://sv-comp.sosy-lab.org/2021/ by Dirk Beyer, LMU Munich, based on the following components: https://gitlab.com/sosy-lab/sv-comp/archives-2021 svcomp21-0-g08c7a98 https://gitlab.com/sosy-lab/software/sv-benchmarks svcomp21-0-g4cc6b6d96a https://gitlab.com/sosy-lab/software/benchexec 3.6-0-gb278ebbb https://gitlab.com/sosy-lab/benchmarking/competition-scripts svcomp21-0-g8339740 https://gitlab.com/sosy-lab/sv-comp/bench-defs svcomp21-0-ga57fe48 Contact Feel free to contact me in case of questions: https://www.sosy-lab.org/people/beyer/
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.014 | 0.050 |
| Meta-epidemiology (narrow) | 0.003 | 0.002 |
| Meta-epidemiology (broad) | 0.001 | 0.002 |
| Bibliometrics | 0.003 | 0.002 |
| Science and technology studies | 0.002 | 0.001 |
| Scholarly communication | 0.006 | 0.010 |
| Open science | 0.003 | 0.010 |
| Research integrity | 0.002 | 0.004 |
| Insufficient payload (model declined to judge) | 0.197 | 0.119 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".