Understanding open source software peer review: review processes, parameters and statistical models, and underlying behaviours and mechanisms
Notice bibliographique
Résumé
Peer review is seen as an important quality assurance mechanism in both industrial development and the open source software (OSS) community. The techniques for performing inspections have been well studied in industry; in OSS development, peer review practices are less well understood. In contrast to industry, where reviews are typically assigned to specific individuals, in OSS, changes are broadcast to hundreds of potentially interested stakeholders. What is surprising is that this approach works very well, despite concerns that reviews may be ignored, or that discussions will deadlock because too many uninformed stakeholders are involved. In this work we use a multi-case study methodology to develop a theory of OSS peer review. There are three research stages. In the first stage, we examine the policies of 25 OSS projects to understand the review processes used on successful OSS projects. We also select six projects for further analysis: Apache, Subversion, Linux, FreeBSD, KDE, and Gnome. In the second stage, using archival records from the six projects, we construct a series of metrics that produces measures similar to those used in traditional inspection experiments. We measure the frequency of review, the size and complexity of the contribution under review, the level of participation during review, the experience and expertise of the individuals involved in the review, the review interval, and number of issues discussed during review. We create statistical models of the review efficiency, review interval, and effectiveness, the issues discussed during review, to determine which measures have the largest impact on review efficacy. In the third stage, we use grounded theory to analyze 500 instances of peer review and interview ten core developers across the six projects. This approach allows us to understand why developers decide to perform reviews, what happens when reviews are ignored, how developers interact during a review, what happens when too many stakeholders are involved during review, and the effect of project size on the review techniques. Our findings provide insights into the simple, community-wide mechanisms and behaviours that developers use to effectively manage large quantities of reviews and other development discussions. The primary contribution of this work is a theory of OSS peer review. We find that OSS reviews can be described as (1) early, frequent reviews (2) of small, independent, complete contributions (3) that, despite being asynchronously broadcast to a large group of stakeholders, are reviewed by a small group of self-selected experts (4) resulting in an efficient and effective peer review technique.
Récupéré en direct depuis OpenAlex et désinversé. Les résumés ne sont pas conservés dans cette base de données : les index inversés représentent 8,6 Go des 9,3 Go de texte de la base, et le serveur dispose de 13 Go libres.
Comment cette classification a été obtenuedéplier
Prédiction distillée sur la base complète
Imitation des enseignantsNi prévalence calibrée, ni vérité terrain. Validation humaine à venir. Apprise à partir de 10 348 étiquettes directes de Codex et de 10 348 étiquettes directes de Gemma. Le mode candidate est l'union des têtes enseignantes seuillées; le consensus est leur intersection. Ces sorties portent le statut machine_predicted_unvalidated et ne sont ni des étiquettes humaines ni des étiquettes directes de modèles de pointe.
Scores Codex et Gemma par catégorie
| Catégorie | Codex | Gemma |
|---|---|---|
| Métarecherche | 0,001 | 0,002 |
| Méta-épidémiologie (sens strict) | 0,000 | 0,000 |
| Méta-épidémiologie (sens large) | 0,001 | 0,000 |
| Bibliométrie | 0,000 | 0,000 |
| Études des sciences et des technologies | 0,000 | 0,000 |
| Communication savante | 0,001 | 0,001 |
| Science ouverte | 0,001 | 0,001 |
| Intégrité de la recherche | 0,000 | 0,001 |
| Charge utile insuffisante (le modèle a refusé de juger) | 0,000 | 0,000 |
Scores machine (provisoires)
Les deux têtes enseignantes du modèle étudiant, lues sur ce travail. Un score ordonne la base pour la relecture; il n'affirme jamais une catégorie, et le statut de validation accompagne chaque rangée tel quel.
Scores de référence d'un modèle non mature (critères de maturité non atteints, 7 itérations). Un score ordonne; il n'affirme jamais une catégorie.
score_only:v0-immature-baseline · tel quel depuis la passe de notation : score_only signifie que le nombre peut ordonner les travaux, et qu'aucune étiquette de catégorie n'en découleClassification
machine, non validéePrédiction automatique; un appel candidat d’une seule tête enseignante, pas un consensus.
Le détail, modèle par modèle et score par score, se trouve en fin de page sous « Comment cette classification a été obtenue ».