A Framework for Evaluating Semantic Knowledge in Problem-Solving- Based Intelligent Tutoring Systems

Abstract We describe a framework for building intelligent tutoring systems that offer an advanced evaluation of learners' semantic knowledge. The knowledge model makes a pedagogical distinction between contextual and general semantic knowledge. General knowledge is defined as the knowledge that is valid in every situation of a curriculum, and that a learner should possess. In opposition, contextual knowledge is the knowledge obtained from the interpretation of a situation. Because the model connects the description of general knowledge to the description of procedural knowledge through “semantic knowledge retrieval”, the evaluation of general knowledge is not only achieved through direct questions, but also indirectly through observation of problem-solving exercises. Introduction To build e-learning systems that can offer highly-tailored assistance, a well-known approach is to model the cognitive processes of learners according to cognitive theories. The most famous examples of this type of tutoring systems are the Cognitive Tutors by Anderson et al. (1995). They are based on the assumption that the mind can be simulated best by a symbolic production rules system (Anderson 1993). Recently, this idea has been embedded in a development kit named CTAT (Aleven et al 2006). The cognitive Tutors internally describe an exercise with a main goal and a set of applicable rules. Each action done by a learner is seen as the application of a rule that execute an action and can create sub-goals. A rule can be marked correct or erroneous, and be annotated with different tutoring resources. Though these systems obtain great success, they are focused on teaching procedural knowledge (rules) in the context of problem-solving exercises. Anderson et al. (1995) makes this clear: “we have placed the emphasis on the procedural (…) because our view is that the acquisition of the declarative knowledge is relatively problem-free. (…) Declarative knowledge can be acquired by simply being told and our tutors always apply in a context where student receive such declarative instruction external to the tutors. (…) Production rules (…) are skills that are only acquired by doing.”. We claim that this view is limited in two ways. First, it supposes that the declarative knowledge can be taught in an explicit way effectively by human tutors. But, this is not always the case. For some domains the declarative knowledge is best learned by doing. As it will be illustrated here, such domains are the tasks that involve spatial representations. Complex spatial representations are viewed by many researchers as being encoded as semantic knowledge (for example, Tversky 1993). In the remainder of this paper, we adopt the term “semantic knowledge” instead of “declarative knowledge” to designate the declarative knowledge that is not associated with the memory of events (Tulving 1972). Second, Cognitive Tutors cannot evaluate semantic knowledge. They assume that learners will acquire the semantic knowledge before doing the problem-solving exercises, or that it will be available during the exercises and that the learners will know when to use it. The problem is that if a learner possess erroneous semantic knowledge or don't know when to use the semantic knowledge, the Cognitive Tutors will wrongly understand the mistakes made by the learner in terms of procedural errors, possibly triggering inappropriate tutoring behavior. On the other hand, evaluation of a learner's semantic knowledge in tutoring systems is generally achieved by asking direct questions about that knowledge such as multiple-choice test (for example, Morales & Aguera 2002). Another approach is the automatic scoring of concepts maps that a learner draws by comparing them with an expert map (Taricani & Clariana 2006). A concept map is basically a graph where each node is a concept or concept instance and each link represents a relationship. Our hypothesis is that a more accurate evaluation of semantic and procedural knowledge can be achieved by making explicit the semantic knowledge that a learner should learn, and evaluate it not only with questions but also with procedural knowledge in problem-solving tasks. This is in accordance with educational researchers that emphasize the importance of understanding how the semantic and procedural knowledge are expressed together

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.