Investigating How Engineers Use Computer-Aided Design Versus Pen-and-Paper for Conceptual Design – A Methodology Review and Mixed-Method Pilot Study
Bibliographic record
Abstract
Abstract As digital-assisted technology becomes more ubiquitous, engineering designers and educators worry that high reliance on digital tools will reduce engineers’ innate creative abilities or hinder the creative design processes. In the past two decades, research has investigated human-computer interaction and developed theories and design methods to improve computer-aided engineering design. However, whether and how computer-aided design tools affect the creative conceptual design process remains unclear. In this methodology paper, we identify key challenges in comparing digital and traditional tools and present an experimental design to address them. We integrated validated psychological assessments, such as the Torrance Tests of Creative Thinking (TTCT), with advanced neurophysiological techniques, including electroencephalography (EEG) and eye-tracking, to provide a multi-modal understanding of the design process. We also adapted the IKEA Hacking Task for both CAD and pen-and-paper conditions, to assess design creativity with ecological validity. By sharing our methodology, we aim to provide a blueprint for future studies investigating human-tool interactions in engineering design. Our approach not only addresses key limitations in current literature but also offers practical insights for designing experiments that balance rigor with real-world applicability. This work contributes to the mechanical engineering design community by advancing the methodological foundations for studying creativity and cognitive processes in engineering design.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.062 | 0.077 |
| Meta-epidemiology (narrow) | 0.001 | 0.001 |
| Meta-epidemiology (broad) | 0.002 | 0.003 |
| Bibliometrics | 0.005 | 0.004 |
| Science and technology studies | 0.001 | 0.001 |
| Scholarly communication | 0.003 | 0.002 |
| Open science | 0.002 | 0.002 |
| Research integrity | 0.001 | 0.001 |
| Insufficient payload (model declined to judge) | 0.004 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".