A systematic literature review of software engineering research on Jupyter notebook
Bibliographic record
Abstract
• This research provides the first comprehensive systematic literature review on software engineering research specifically targeting Jupyter notebooks, identifying 199 primary studies published up to September 2025 and categorizing them into 11 core software engineering topics. • This research reveals that a large portion of the studies have been published outside traditional software engineering venues, with Human-Computer Interaction conferences like ACM Conference on Human Factors in Computing Systems (CHI) being the top publishing venues, highlighting the interdisciplinary nature of Jupyter Notebook research. • This research identifies a reusability gap in existing research, showing that only 82 out of 199 studies offer usable replication packages, and most are hosted on GitHub instead of permanent repositories, which violates open science best practices. • This research identifies that notebook-specific solutions for software engineering issues such as testing, refactoring, and documentation are relatively underexplored. Future directions include resolving duplicated execution numbers, refactoring inter-notebook clones, and generating grouped documentation for coherent-code cells are future directions derived from our study. • This research proposes the integration of modern AI-based solutions into Jupyter notebooks to support various software engineering topics, including code search and code generation. Additionally, future research should leverage advanced AI techniques (e.g., large language models), to improve conversational AI-powered assistants for automated code generation by multi-step workflow automation in data science notebooks. • Although the paper exceeds the recommended length due to the inclusion of detailed tables, figures, and categorized analyses (covering 11 topics and 21 subtopics), we believe that this extended content is essential for clearly and completely reporting our findings. As the first systematic literature review in this domain, we have carefully structured the paper to ensure readability. We believe the length is justified by the value and breadth of this paper’s contributions. Context : Jupyter Notebook has emerged as a versatile tool that transforms how researchers, developers, and data scientists conduct and communicate their work. As the adoption of Jupyter notebooks continues to rise, so does the interest from the software engineering research community in improving the software engineering practices for Jupyter notebooks. Objective : The purpose of this study is to analyze trends, gaps, and methodologies used in software engineering research on Jupyter notebooks. Method : We selected 199 relevant publications up to September 2025, following established systematic literature review guidelines. We explored publication trends, categorized them based on software engineering topics, and reported findings based on those topics. Results : The most popular venues for publishing software engineering research on Jupyter notebooks are related to human-computer interaction instead of traditional software engineering venues. Researchers have addressed a wide range of software engineering topics on notebooks, such as code reuse, readability, and execution environment. Although reusability is one of the research topics for Jupyter notebooks, only 82 of the 199 studies can be reused based on their provided URLs. Additionally, most replication packages are not hosted on permanent repositories for long-term availability and adherence to open science principles. Conclusion : Solutions specific to notebooks for software engineering issues, including testing, refactoring, and documentation, are underexplored. Future research opportunities exist in automatic testing frameworks, refactoring clones between notebooks, and generating group documentation for coherent code cells.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.021 | 0.123 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.004 | 0.005 |
| Bibliometrics | 0.029 | 0.026 |
| Science and technology studies | 0.001 | 0.002 |
| Scholarly communication | 0.004 | 0.007 |
| Open science | 0.003 | 0.003 |
| Research integrity | 0.002 | 0.002 |
| Insufficient payload (model declined to judge) | 0.010 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".