Application of Machine Learning in Program and Malware Analysis

Program and malware analysis tools are important for computer security. Recently, machine learning has been applied to help with program and malware analysis. In this thesis, we investigate a few areas in which the application of machine learning can be used to improve program analysis. We find several cases where understanding the problem domain of program analysis can be more important than achieving raw machine learning capabilities on its own. The first application is symbolic analysis. We use path features instead of constraint features in related work to train models that predict whether paths are possible in symbolic analysis. These statistical path features can be applied without constraint collection. With these features, simple machine learning models are enough to save time, and their predictions can generalize across software applications. Also, we examine the sometimes complex interactions between factors such as model performance, design of symbolic analysis tool, distribution of path analysis time, and the algorithm that prioritizes and selects paths for symbolic analysis. We find that leveraging domain-specific knowledge about analyzed paths can achieve better time savings than improving model performance. Another application is malware classification. While machine learning models can detect malware, these models need to be frequently retrained to ensure they are up to date. Data drift detectors select drifting software samples into the datasets for retraining models, so model performance does not decrease significantly upon changes in software data. However, how malware samples can evade these data drift detectors during malware classification has not been studied. We examine how evasion and poisoning attacks work against data drift detectors and malware classifiers. We find how unique properties from the underlying models of the data drift detectors cause some attacks that work against malware classification to fail against data drift detection. With improper attack methods, a more capable attacker can even decrease the chance of attack success against data drift detection. Finally, we also propose a poisoning attack that properly works against one such data drift detector for retraining malware classifiers.

Récupéré en direct depuis OpenAlex et désinversé. Les résumés ne sont pas conservés dans cette base de données : les index inversés représentent 8,6 Go des 9,3 Go de texte de la base, et le serveur dispose de 13 Go libres.