The role of feedback in the design of learning activities
Bibliographic record
Abstract
Learning Analytics plays an increasingly important role in informing educational institutions about their performance, and in supplying them with data on which they can guide their future policy. In this paper we analyse the challenges involved in obtaining useful data about learning activities, and in responding appropriately to them. The paper describes a case study carried out at the Open Learning Division of Thompson Rivers University which sought to lay the groundwork for an enhancement of instructional design practice by identifying the factors which are responsible for the success or failure of learning activities. The responsibility for development of learning activities lies principally with the Instructional Design team. The five members of this team were interviewed, and their perspectives were supplemented by interviews with eight lecturers, academic managers, and those responsible for faculty development. The 13 interviews were transcribed, and Qualitative Data Analysis techniques applied to draw out the principal themes. This process identified factors determining the success of learning activities, and requests for feedback, which will feed into the collection of data from students as they take their courses. On examination of the data valuable information was found which went beyond the original scope of the inquiry. This concerned, first, the methods, problems and workarounds in the instructional design group when defining activities; and second, the organisational, technical and policy constraints on the design group, and their consequences. The perceived flows of feedback within the learning activity process were analysed, from the perspective of the instructional designers, and an explanation for the barriers encountered is proposed in terms of variety management. The instructional design team is required to use its experience to resolve issues which are too complex for formal analysis, and the principal problems are identified. It is proposed that (a) documents should be developed to represent agreed practice in dealing with these problems so as to reduce cognitive load on the instructional designers; and (b) that the collection of data on learning activities should be focused on confirming the accuracy of the suppositions and mechanisms implied in this practice.
Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.
How this classification was reachedexpand
Full frame machine prediction
Teacher imitationNot calibrated prevalence, not ground truth. Human validation pending. The Gemma side is a direct model label for every work in the frame, read from the title-only record. The Codex side is a classifier learned from the 10,348 direct Codex labels and calibrated to design-weighted sample rates; fields without enough sample support carry no Codex call. Candidate is the union of the two sides; consensus is their intersection. These outputs are machine_predicted_unvalidated and are not human labels.
Distilled classifier scores by category (both heads)
| Category | Codex | Gemma |
|---|---|---|
| Metaresearch | 0.119 | 0.325 |
| Meta-epidemiology (narrow) | 0.002 | 0.002 |
| Meta-epidemiology (broad) | 0.002 | 0.001 |
| Bibliometrics | 0.006 | 0.003 |
| Science and technology studies | 0.004 | 0.008 |
| Scholarly communication | 0.017 | 0.019 |
| Open science | 0.005 | 0.005 |
| Research integrity | 0.003 | 0.003 |
| Insufficient payload (model declined to judge) | 0.003 | 0.001 |
Machine scores (provisional)
The two teacher heads of the student model, read on this work. A score orders the frame for review; it never asserts a category, and the validation status ships verbatim with every row.
Baseline scores from an immature model (maturity gate not passed, 7 training rounds). Scores rank; they never assert a category.
score_only:v0-immature-baseline · verbatim from the scoring run: score_only means the number may rank works, and no category label ships from itClassification
machine, unvalidatedMachine predicted; a candidate call from one source (direct Gemma or distilled Codex), not a consensus.
How this classification was reached, model by model and score by score, is at the end of the page under "How this classification was reached".