Association of Online Learning Behavior and Learning Outcomes for Medical Students: Large-Scale Usage Data Analysis
Bibliographic record
Abstract
Background Digital learning environments have become very common in the training of medical professionals, and students often use such platforms for exam preparation. Multiple choice questions (MCQs) are a common format in medical exams and are used by students to prepare for said exams. Objective We aimed to examine whether particular learning activities contributed more strongly than others to users’ exam performance. Methods We analyzed data from users of an online platform that provides learning materials for medical students in preparation for their final exams. We analyzed whether the number of learning cards viewed and the number of MCQs taken were positively related to learning outcomes. We also examined whether viewing learning cards or answering MCQs was more effective. Finally, we tested whether taking individual notes predicted learning outcomes, and whether taking notes had an effect after controlling for the effects of learning cards and MCQs. Our analyses from the online platform Amboss are based on user activity data, which supplied the number of learning cards studied and test questions answered. We also included the number of notes from each of those 23,633 users who had studied at least 200 learning cards and had answered at least 1000 test exam questions in the 180 days before their state exam. The activity data for this analysis was collected retrospectively, using Amboss archival usage data from April 2014 to April 2017. Learning outcomes were measured using the final state exam scores that were calculated by using the answers voluntarily entered by the participants. Results We found correlations between the number of cards studied (r=.22; P<.001) and the number of test questions that had been answered (r=.23; P<.001) with the percentage of correct answers in the learners’ medical exams. The number of test questions answered still yielded a significant effect, even after controlling for the number of learning cards studied using a hierarchical regression analysis (β=.14; P<.001; ΔR2=.017; P<.001). We found a negative interaction between the number of learning cards and MCQs, indicating that users with high scores for learning cards and MCQs had the highest exam scores. Those 8040 participants who had taken at least one note had a higher percentage of correct answers (80.94%; SD=7.44) than those who had not taken any notes (78.73%; SD=7.80; t23631=20.95; P<.001). In a stepwise regression, the number of notes the participants had taken predicted the percentage of correct answers over and above the effect of the number of learning cards studied and of the number of test questions entered in step one (β=.06; P<.001; ΔR2=.004; P<.001). Conclusions These results show that online learning platforms are particularly helpful whenever learners engage in active elaboration in learning material, such as by answering MCQs or taking notes.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame distilled prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. Learned from the 10,348 direct Codex labels and 10,348 direct Gemma labels. Candidate is the union of thresholded teacher heads; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels or direct frontier model labels.
Codex and Gemma teacher scores by category
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.009 | 0.016 |
| Meta-epidemiology (narrow) | 0.000 | 0.000 |
| Meta-epidemiology (broad) | 0.000 | 0.000 |
| Bibliometrics | 0.000 | 0.000 |
| Science and technology studies | 0.000 | 0.000 |
| Scholarly communication | 0.000 | 0.000 |
| Open science | 0.001 | 0.000 |
| Research integrity | 0.001 | 0.001 |
| Insufficient payload (model declined to judge) | 0.002 | 0.000 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one teacher head, not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".