How the process of discovering cyberattacks biases our understanding of cybersecurity
Bibliographic record
Abstract
Abstract Social scientists do not directly study cyberattacks; they draw inferences from attack reports that are public and visible. Like human rights violations or war casualties, there are missing cyberattacks that researchers have not observed. The existing approach is to either ignore missing data and assume they do not exist or argue that reported attacks accurately represent the missing events. This article is the first to detail the steps between attack, discovery and public report to identify sources of bias in cyber data. Visibility bias presents significant inferential challenges for cybersecurity – some attacks are easy to observe or claimed by attackers, while others take a long time to surface or are carried out by actors seeking to hide their actions. The article argues that missing attacks in public reporting likely share features of reported attacks that take the longest to surface. It builds on datasets of cyberattacks by or against Five Eyes (an intelligence alliance composed of Australia, Canada, New Zealand, the United Kingdom and the United States) governments and adds new data on when attacks occurred, when the media first reported them, and the characteristics of attackers and techniques. Leveraging survival models, it demonstrates how the delay between attack and disclosure depends on both the attacker’s identity (state or non-state) and the technical characteristics of the attack (whether it targets information confidentiality, integrity, or availability). The article argues that missing cybersecurity events are least likely to be carried out by non-state actors or target information availability. Our understanding of ‘persistent engagement,’ relative capabilities, ‘intelligence contests’ and cyber coercion rely on accurately measuring restraint. This article’s findings cast significant doubt on whether researchers have accurately measured and observed restraint, and informs how others should consider external validity. This article has implications for our understanding of data bias, empirical cybersecurity research and secrecy in international relations.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.064 | 0.279 |
| Meta-epidemiology (narrow) | 0.000 | 0.001 |
| Meta-epidemiology (broad) | 0.001 | 0.001 |
| Bibliometrics | 0.004 | 0.003 |
| Science and technology studies | 0.002 | 0.012 |
| Scholarly communication | 0.012 | 0.020 |
| Open science | 0.002 | 0.006 |
| Research integrity | 0.003 | 0.006 |
| Insufficient payload (model declined to judge) | 0.006 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".