Bibliographic record
Abstract
A significant part of professional communication development in engineering is the ability to learn and understand technical vocabulary. Mastering such vocabulary is often a desired learning outcome of engineering education. In promoting this goal, this research investigates the development of a tool that creates wordlists of characteristic discipline-specific vocabulary for a given course. These wordlists explicitly highlight requisite vocabulary learning and, when used as a teaching aid, can promote greater accessibility in the learning environment. Literature, including work in higher education, diversity and language learning, suggest that designing accessible learning environments can increase the quality of instruction and learning for all students. Studying the student/instructor interface using the framework of Universal Instructional Design identified vocabulary learning as an invisible barrier in engineering education. A preliminary investigation of this barrier suggested that students have difficulty assessing their understanding of technical vocabulary. Subsequently, computing word frequency on engineering course material was investigated as an approach for characterizing this barrier. However, it was concluded that a more nuanced method was necessary.This research program was built on previous work in the fields of linguistics and computer science, and lead to the design of an algorithm. The developed algorithm is based on a statistical technique called, Term Frequency-Inverse Document Frequency. Comparator sets of documents are used to hierarchically identify characteristic terms on a target document, such as course materials from a previous term of study. The approach draws on a standardized artifact of the engineering learning environment as its dataset; a repository of 2254 engineering final exams from the University of Toronto, to process the target material. After producing wordlists for ten courses, with the goal of highlighting characteristic discipline-specific terms, the effectiveness of the approach was evaluated by comparing the computed results to the judgment of subject-matter experts. The overall data show a good correlation between the program and the subject-matter experts. The results indicated a balance between accuracy and feasibility, and suggested that this approach could mimic subject-matter expertise to create a list discipline-specific vocabulary from course materials.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.003 | 0.022 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.003 | 0.004 |
| Science and technology studies | 0.001 | 0.002 |
| Scholarly communication | 0.006 | 0.004 |
| Open science | 0.000 | 0.002 |
| Research integrity | 0.000 | 0.001 |
| Insufficient payload (model declined to judge) | 0.003 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".