Bibliographic record
Abstract
Please use "Tree View" to browse the files efficiently. This dataset contains replication artifacts, data, and analysis artifacts for the study "Refactoring strategies used by developers during software evolution tasks". The study was presented in the paper "A Study of Refactorings During Software Change Tasks". The purpose of the study was to learn about how developers approach software change tasks that contain refactoring steps. Developers were recruited and presented with three software change tasks that contain steps that are amenable to automation with refactoring tools. The tasks were based on commits mined from open-source repositories. The developers solved the tasks while following a think aloud protocol. Interleaved with the tasks and just after all the tasks were finished we conducted interviews with the developer. We recorded their vocalizing, the interviews, their screens, and the resulting source code. We then transcribed the audio and annotated the transcripts with timestamps, and invocations of refactoring tools, if any. Article abstract: Developers frequently undertake software change tasks that could be partially or fully automated by refactoring tools. As has been reported by others, all too often, these refactoring steps are instead performed manually by developers. These missed opportunities are referred to as occasions of disuse of refactoring tools. We perform an observational study in which 17 developers with professional experience attempt to solve three change tasks with steps amenable to the use of refactoring tools. We found that the strategies developers use to approach these tasks shape their workflow, which in turn shapes the opportunities for refactoring tool use. We report on a number of findings about developer strategies, demonstrating the difficulty of aligning the kind of refactoring steps that emerge during a change task based on the strategy with the tools available. We also report on findings about refactoring tools, such as the difficulties developers face in controlling the scope of application of the tools. Our findings can help inform the designers of refactoring tools. Data description: This dataset contains the transcript data that was collected during the experiments. It also contains the screen recordings of the participants' screen that was collected during the experiment. It is also contains the source code that the participants' edits resulted in. It also contains the results of coding the transcripts and the videos with invocations of refactoring tools (also visible on the screen recording) and mentions of refactoring tools. It also contains the workflows that we created for each participant which contains a summary of all refactoring tool invocations (also visible in transcripts) and their overarching strategy. Participants worked in IntelliJ (Java IDE, integrated development environment). Their screen and audio was recorded with QuickTime Player. The source code was in Java 1.8. The code had a git repository. The project was built with Maven. The Maven build file contains maven-compiler-plugin, org.hamcrest.hamcrest-core, org.junit.junit5-api, org.junit.junit4-engine. The audio was transcribed with the Atom Editor. The CSV-files were created with Google Sheets.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.001 | 0.008 |
| Meta-epidemiology (narrow) | 0.002 | 0.001 |
| Meta-epidemiology (broad) | 0.002 | 0.001 |
| Bibliometrics | 0.004 | 0.009 |
| Science and technology studies | 0.001 | 0.000 |
| Scholarly communication | 0.004 | 0.002 |
| Open science | 0.003 | 0.003 |
| Research integrity | 0.003 | 0.002 |
| Insufficient payload (model declined to judge) | 0.304 | 0.379 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".