Identifying Misconceptions using Structural Assessment of Knowledge
Notice bibliographique
Résumé
Identifying Misconceptions using Structural Assessment of Knowledge David L. Trumpower (david.trumpower@uottawa.ca) Harold Sharara (hshar044@uottawa.ca) University of Ottawa, Faculty of Education, 145 Jean-Jacques-Lussier Street Ottawa, ON K1N 6N5 Canada Keywords: knowledge organization; diagnostic assessment; Pathfinder; problem solving. Introduction Domain expertise requires not only an abundance of knowledge, but also well organized knowledge. Knowledge organization has been measured using a technique known as structural assessment of knowledge (Goldsmith, Johnson, & Acton, 1991) in which ratings of concept relatedness are transformed via scaling algorithm into a network representation. The quality of a network is determined by some quantitative measure of its overall similarity to a referent network. Although overall similarity has been shown to be a valid measure of domain knowledge (e.g., Goldsmith, et al., 1991), it is not particularly diagnostic in nature. This can be realized by noting that two networks may have the same overall similarity to a referent network although they might differ with respect to the specific links that they share with that referent. In this study, we assess the absence of specific links in structural knowledge representations (rather than an overall similarity measure) in an attempt to diagnose misconceptions, as indicated by performance on different problem types, in a computer programming domain. Pointer and the concepts Position, Increment, and Assign were said to possess the ‘Pointer schema’. PFnets that contained links between the concept Go-To and the concepts Step and If-Then were said to possess the ‘Go-To schema’ (see Figure 1). Of the 35 participants, 8 possessed the Pointer schema, whereas 12 possessed the Go-To schema. Only 3 participants possessed both schemas. Participants who possessed the Pointer schema solved more type P problems successfully than those who did not possess the Pointer schema, t(24.64)=2.81, p=.01. However, there was no difference in the number of other type problems solved by those who did and did not possess the Pointer schema, p>.05. Likewise, participants who possessed the Go-To schema solved more type G problems successfully than those who did not possess the Go-To schema, t(32.58)=2.69, p=.01, but there was no difference in the number of other type problems solved by those who did and did not possess the Go-To schema, p>.05. These results demonstrate that the absence of specific links in PFnets can be used to diagnose different misconceptions. This more fine grained assessment of structural knowledge representations may be useful for formative evaluations and focusing of instruction. Method Thirty-five undergraduate psychology students who had no prior computer programming experience were allowed 15 minutes to study a simple, custom designed computer programming language. The language was modeled after Pascal but was limited in scope, consisting of just 12 key concepts. Participants then rated the relatedness of all pairwise combinations of the 12 concepts on a 5-point scale (1=Not at all related, 5=Very related). Finally, participants attempted to solve a set of computer programming problems. Five of the problems required knowledge of the relationships among the concepts Position, Pointer, Assign, and Increment for successful solution (type P problems), whereas three problems required knowledge of the relationships among the concepts If-Then, Go-To, and Step (type G problems). Position Pointer List Increment Letter Ordered Switch Assign Instruction Step Go-To If-Then Figure 1. Expert’s PFnet with the Pointer schema and the Go-To schema highlighted in italics and bold, respectively. References Results & Discussion Participants’ relatedness ratings were transformed into network representations (PFnets) using the Pathfinder scaling algorithm (Schvaneveldt, 1990). PFnets were then analyzed for the presence of specific subsets of links, or schemas. PFnets that contained links between the concept Goldsmith, T. E., Johnson, P. J., & Acton, W. H. (1991). Assessing structural knowledge. Journal of Educational Psychology, 83, 88-96. Schvaneveldt, R. W. (1990). Pathfinder associative networks: Studies in knowledge organization. Norwood, NJ: Ablex.
Récupéré en direct depuis OpenAlex et désinversé. Les résumés ne sont pas conservés dans cette base de données : les index inversés représentent 8,6 Go des 9,3 Go de texte de la base, et le serveur dispose de 13 Go libres.
Comment cette classification a été obtenuedéplier
Prédiction distillée sur la base complète
Imitation des enseignantsNi prévalence calibrée, ni vérité terrain. Validation humaine à venir. Apprise à partir de 10 348 étiquettes directes de Codex et de 10 348 étiquettes directes de Gemma. Le mode candidate est l'union des têtes enseignantes seuillées; le consensus est leur intersection. Ces sorties portent le statut machine_predicted_unvalidated et ne sont ni des étiquettes humaines ni des étiquettes directes de modèles de pointe.
Scores Codex et Gemma par catégorie
| Catégorie | Codex | Gemma |
|---|---|---|
| Métarecherche | 0,000 | 0,000 |
| Méta-épidémiologie (sens strict) | 0,000 | 0,000 |
| Méta-épidémiologie (sens large) | 0,000 | 0,000 |
| Bibliométrie | 0,000 | 0,000 |
| Études des sciences et des technologies | 0,000 | 0,000 |
| Communication savante | 0,000 | 0,001 |
| Science ouverte | 0,000 | 0,000 |
| Intégrité de la recherche | 0,000 | 0,000 |
| Charge utile insuffisante (le modèle a refusé de juger) | 0,002 | 0,000 |
Scores machine (provisoires)
Les deux têtes enseignantes du modèle étudiant, lues sur ce travail. Un score ordonne la base pour la relecture; il n'affirme jamais une catégorie, et le statut de validation accompagne chaque rangée tel quel.
Scores de référence d'un modèle non mature (critères de maturité non atteints, 7 itérations). Un score ordonne; il n'affirme jamais une catégorie.
score_only:v0-immature-baseline · tel quel depuis la passe de notation : score_only signifie que le nombre peut ordonner les travaux, et qu'aucune étiquette de catégorie n'en découleClassification
machine, non validéePrédiction automatique; un appel candidat d’une seule tête enseignante, pas un consensus.
Le détail, modèle par modèle et score par score, se trouve en fin de page sous « Comment cette classification a été obtenue ».