USING AN ONLINE MARKING SYSTEM FOR A LARGE CLASS ENVIRONMENT
Bibliographic record
Abstract
The use of online team marking has the potential to both simplify and expedite the process of marking exams, papers, and other artifacts. An online team marking tool (Crowdmark) has been piloted at UBC in Mechanical Engineering (125 student midterm) and two common first year introduction to engineering courses (840 student final exam, and 730 student midterm and final exam).Crowdmark, the particular software tool used, printed a unique QR code on each page of each exam and then exams were written by students in a conventional pencil-andpaper fashion. After the exam, papers were digitized and uploaded to the Crowdmark system. Following a brief training and orientation session, all marking took place by teaching assistants through the Crowdmark interface. Overall grader preference was positive, with the majority of graders expressing a strong preference for the Crowdmark system over conventional paper-based grading. In MECH 223, extensive historical data for marking time was available, and a significant reduction in marking time per exam (30%) was observed. This time savings included time saved handling papers and entering grades. Additional benefits were also observed through the use of this system: grades and histograms are available per question in real-time; time and grader tracking data is available; exam regrading is simplified; and there is a digital record of each exam for archiving purposes as well as to prevent issues of students altering papers prior to requesting regrading. Special safeguards had to be put in place due to freedom of information and privacy protection (FOIPP) requirements in British Columbia. We haveobserved a slightly lower cost per graded page with Crowdmark ($0.426/page) compared to a conventional exam ($0.439/page), but this includes outsourcing printing and scanning to an industrial-scale printing company. We consider this essentially cost-neutral, but like Crowdmarkfor all of the other benefits it offers.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.000 | 0.000 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.000 | 0.000 |
| Research integrity | 0.000 | 0.000 |
| Insufficient payload (model declined to judge) | 0.000 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".