Searching for Bugs Using Probabilistic Suspect Implications
Why this work is in the frame
A frame that forgets how it found something cannot be audited. These are the routes that admitted this work.
Bibliographic record
Abstract
Due to the excessive cost associated with manual RTL design debugging, automated tools are often employed to identify a set of suspect bug locations. To further accelerate the process, one observes that the anytime behavior of these tools allows partial results to be analyzed before the suspect search is complete. Thus, it is preferable for the tool to maximize the number of suspects that are found in the early stages of its search. Toward this end, this article proposes a new SAT-based debugging algorithm which predicts where solutions are most likely to be found and prioritize examining these locations. Two techniques are proposed to predict solution locations by learning from historical debug data. The first technique does so using belief propagation on a probabilistic graph, while the second trains a neural network to classify candidate suspects as solutions or nonsolutions. Intensive empirical evaluation demonstrates that these techniques can predict suspect sets with accuracies of 81% and 87%, respectively, but the second method requires more training data and careful hyperparameter tuning in order to do so. Furthermore, when guided by these suspect prediction models, the proposed debugging algorithm finds an average of 83% more suspects within a given amount of time.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.001 | 0.000 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.001 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.001 | 0.000 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it