EvoCrawl: Exploring Web Application Code and State using Evolutionary Search
Notice bibliographique
Résumé
As more critical services move onto the web, it has become increasingly important to detect and address vulnerabilities in web applications.These vulnerabilities only occur under specific conditions: when 1) the vulnerable code is executed and 2) the web application is in the required state.If the application is not in the required state, then even if the vulnerable code is executed, the vulnerability may not be triggered.Previous work naively explores the application state by filling every field and triggering every JavaScript event before submitting HTML forms.However, this simplistic approach can fail to satisfy constraints between the web page elements, as well as input format constraints.To address this, we present EvoCrawl, a web crawler that uses evolutionary search to efficiently find different sequences of web interactions.EvoCrawl finds sequences that can successfully submit inputs to web applications and thus explore more code and server-side states than previous approaches.To assess the benefits of EvoCrawl, we evaluate it against three stateof-the-art vulnerability scanners on ten web applications.We find that EvoCrawl achieves better code coverage due to its ability to execute code that can only be executed when the application is in a particular state.On average, EvoCrawl achieves a 59% increase in code coverage and successfully submits HTML forms 5 more frequently than the next best tool.By integrating IDOR and XSS vulnerability scanners, we used EvoCrawl to find eight zero-day IDOR and XSS vulnerabilities in WordPress, HotCRP, Kanboard, ImpressCMS, and GitLab. I. INTRODUCTIONSince 2017, broken access control and XSS code injection have been consistently ranked among the most prevalent vulnerabilities in OWASP Top 10.As stated in OWASP's 2021 report [1], 94% of tested applications exhibited some form of broken access control or injection vulnerabilities, underscoring the need for developers to safeguard their web applications against these defects.There are two main approaches to detecting such vulnerabilities: static analysis and dynamic analysis. Static analysis tools [2], [3], [4],[5] require the application's source code, which limits their applicability to applications written in other programming languages.Conversely, dynamic analysis can be agnostic to the programming language, but can only detect vulnerabilities if they occur during the tool's application exploration.Therefore, dynamic analysis focuses on exploring as much code as possible.Web vulnerability scanners are often paired with a web crawler, which attempts to maximize the code coverage of an application by scanning as many pages as possible.However, simply crawling pages is not sufficient to maximize code coverage.This is because some code in a web application is associated with functionality that can only be triggered when the application is in a specific server-side state.For instance, in GitLab, a web-based revision control system, functions that enable users to manipulate code repositories require a repository to be created first.As a result, a vulnerability scanner will be unable to test any of that code if it is unable to interact with GitLab and create new repositories.Therefore, to achieve good code coverage, and thus find more vulnerabilities, a web crawler must explore both web pages and application states.This importance has not been lost in previous approaches to web application vulnerability detection.For example, Black-Widow [6] and Enemy of the State [7] incorporate HTML forms into their navigation graph, and submitting these HTML forms enables them to explore different server-side states.In addition, to handle AJAX-enabled dynamic web pages, BlackWidow also adds JavaScript events to its navigation graph.Triggering JavaScript events can enable additional fields and elements on a web page, allowing BlackWidow to submit more data and explore more server-side states.However, to correctly submit data to a web application that will modify the server-side state, a web crawler must satisfy both ordering and formatting constraints on interactions with HTML elements and trigger JavaScript events in the right order.For example, to submit data via a form, the web application may impose an ordering constraint that requires the crawler to first enter data into text fields, or select the correct options from dropdown boxes or radio buttons, before hitting the submit button.Similarly, a web application may impose formatting constraints such that fields that require a date or an e-mail must have well-formed inputs.Finding a sequence that meets ordering constraints requires a search over all possible sequences of interactions with HTML elements and JavaScript events (which we collectively refer to as web elements), which grows exponentially with the number of such elements and events.BlackWidow and Enemy of the State naively avoid searching this large space by filling in
Récupéré en direct depuis OpenAlex et désinversé. Les résumés ne sont pas conservés dans cette base de données : les index inversés représentent 8,6 Go des 9,3 Go de texte de la base, et le serveur dispose de 13 Go libres.
Comment cette classification a été obtenuedéplier
Prédiction distillée sur la base complète
Imitation des enseignantsNi prévalence calibrée, ni vérité terrain. Validation humaine à venir. Apprise à partir de 10 348 étiquettes directes de Codex et de 10 348 étiquettes directes de Gemma. Le mode candidate est l'union des têtes enseignantes seuillées; le consensus est leur intersection. Ces sorties portent le statut machine_predicted_unvalidated et ne sont ni des étiquettes humaines ni des étiquettes directes de modèles de pointe.
Scores Codex et Gemma par catégorie
| Catégorie | Codex | Gemma |
|---|---|---|
| Métarecherche | 0,000 | 0,000 |
| Méta-épidémiologie (sens strict) | 0,000 | 0,000 |
| Méta-épidémiologie (sens large) | 0,000 | 0,000 |
| Bibliométrie | 0,000 | 0,001 |
| Études des sciences et des technologies | 0,000 | 0,000 |
| Communication savante | 0,000 | 0,000 |
| Science ouverte | 0,000 | 0,000 |
| Intégrité de la recherche | 0,000 | 0,000 |
| Charge utile insuffisante (le modèle a refusé de juger) | 0,000 | 0,000 |
Scores machine (provisoires)
Les deux têtes enseignantes du modèle étudiant, lues sur ce travail. Un score ordonne la base pour la relecture; il n'affirme jamais une catégorie, et le statut de validation accompagne chaque rangée tel quel.
Scores de référence d'un modèle non mature (critères de maturité non atteints, 7 itérations). Un score ordonne; il n'affirme jamais une catégorie.
score_only:v0-immature-baseline · tel quel depuis la passe de notation : score_only signifie que le nombre peut ordonner les travaux, et qu'aucune étiquette de catégorie n'en découleClassification
machine, non validéePrédiction automatique; un appel candidat d’une seule tête enseignante, pas un consensus.
Le détail, modèle par modèle et score par score, se trouve en fin de page sous « Comment cette classification a été obtenue ».