DEVELOPING A FRAMEWORK TO EVALUATE INDIVIDUAL LEARNING IN ENGINEERING DESIGN PROBLEMS – PART 2: ASSESSMENT OF INDIVIDUAL LEARNING IN TEAM ENVIRONMENTS
Bibliographic record
Abstract
Engineers often model their teaching based on how they expect students to function once they graduate, and as a result they typically have a strong affinity for Problem-Based Learning (PBL) and Team-Based Learning (TBL) approaches. In many cases, this works quite well and has been proven to increase student engagement and performance of design teams. At the same time, as more of the curriculum relies on group projects, there is a simple dilemma that is created: how do you ensure that the learning outcomes a team demonstrates are accurate reflections of what each individual in the team is learning? This rarely poses a challenge in engineering practice, as engineers are expected to specialize, but in an outcome-based accreditation, this can become a serious issue if there isn’t a careful consideration of this in the structure of TBL courses. This paper explores the application of an evaluative framework to a course with strong PBL and TBL components that is set up to ensure that students are not only exposed to all learning outcomes as they tackle a project, but are individually evaluated on their ability to show competence in these outcomes. The prime methodology of this framework is an evaluative tool called an "open-ended design exam" that uses a 1:1 mapping of team and individual learning, with scaffolding in the course frameworks to support this. This paper presents application of this approach to two courses developed at the University of Manitoba, outcomes and responses to the course layout, and suggestions for extensions to other courses or programs.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.002 | 0.001 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.001 | 0.001 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.001 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".