PredictDeep: Security Analytics as a Service for Anomaly Detection and Prediction
Why this work is in the frame
A frame that forgets how it found something cannot be audited. These are the routes that admitted this work.
Bibliographic record
Abstract
As businesses embrace digitization, the Internet of Everything (IoE) begins to take shape and the Cloud continues to empower new innovations for big data -at the heart, Cloud analytic applications gain increasing momentum. Such applications have remarkable benefits for big data processing, making it easy, fast, scalable, and cost-effective; albeit, they pose many security risks. Security breaches causing anomalous activities due to malicious, vulnerable, or misconfigured analytic applications are considered the top security risks to big “sensitive” data. The risk is further expanded from the coupling of data analytics with the Cloud. Towards maintaining secure and trustworthy applications, effective anomaly detection and prediction become crucial tasks to be offered by Cloud providers. This paper presents, PredictDeep, a novel security analytics framework for anomaly detection and prediction. The proposed framework leverages log data collected from monitoring systems with graph analytics and deep learning techniques to add intelligence for detecting and predicting known and unknown patterns of security anomalies. It represents the collected data and transforms them into a graph model. The graph model captures the analytical activities as well as their interrelation. In this sense, such a model provides informed insight of the monitored application, understanding its behavior, and revealing anomalous patterns. Different from existing traditional rule-based machine learning and statistics-based approaches, our solution takes the benefits of incorporating not only available node attributes but also graph structure and context information to extract rich features that boost the anomaly classification and prediction. We leverage graph embeddings to represent the nodes and relationships in the graph model as feature vectors to learn and predict anomalies in an inductive way utilizing recent advanced deep graph neural network techniques. This design augments our solution with robustness and computational efficiency. Extensive experiments are conducted over an open-source Hadoop log dataset. The evaluation results demonstrate that PredictDeep is a viable solution and very effective.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.000 | 0.000 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.001 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from it