Human Factors in Interactive Machine Learning: A Cybersecurity Case Study
Bibliographic record
Abstract
Cybersecurity is emerging as a major issue for many organizations and countries. Machine learning has been used to recognize threats, but it is difficult to predict future threats based on past events, since malicious attackers are constantly finding ways to circumvent defences and the algorithms that they rely on. Interactive Machine learning (iML) has been developed as a way to combine human and algorithmic expertise in a variety of domains and we are currently applying it to cybersecurity. In this application of iML, implicit knowledge about human behaviour, and about the changing nature of threats, can supplement the explicit knowledge encoded in algorithms to create more effective defences against cyber-attacks. In this paper we present the example problem of data exfiltration where insiders, or outsiders masquerading as insiders, who copy and transfer data maliciously, against the interests of an organization. We will review human factors issues associated with the development of iML solutions for data exfiltration. We also present a case study involving development of an iML solution for a large financial services company. In this case study we review work carried out on developing visualization dashboards and discussing prospects for further iML integration. Our goal in writing this paper is to motivate future researchers to consider the role of the human more fully in ML, not only in the data exfiltration and cybersecurity domain but also in a range of other applications where human expertise is important and needs to combine with ML prediction to solve challenging problems.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.000 | 0.000 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.001 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.000 | 0.001 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".