Modeling Embodied Cognition in a Complex Real-Time Task

Modeling Embodied Cognition in a Complex Real-Time Task Michael J. Schoelles (mschoell@gmu.edu) Wayne D. Gray (gray@gmu.edu) Human Factors & Applied Cognition George Mason University Fairfax, VA 22030 USA The interaction between perception and cognition is an important component of human performance in complex dynamic tasks. In time critical situations we propose that subjects develop microstategies (Gray, Schoelles, & Fu, 1999) that manipulate these interactions to improve performance. In this paper, we report on our effort to model these interactions. The model in its current state performs a complex dynamic decision making task in a scaled world simulation of a radar operator (Argus Prime). The ultimate goal of the model is to predict changes in performance as the cognitive and perceptual workload of the task changes. The task in the Argus Prime experimental environment requires a mix of perceptual and cognitive actions. The task involves four subtasks. For target selection, the user attends to icons on the screen (perception), decides to process an icon (cognition), and selects it (motor). In information retrieval the user reads the raw data values for this object (perception). Score calculation entails mapping raw data to target score (cognition), mapping score to threat value (cognition), selecting a threat value (perception and motor), and entering the decision (motor). Finally, feedback processing consists of perceiving feedback (perception) and processing the feedback (cognition). As this brief task analysis illustrates, each subtask combines cognitive, perceptual, and motor operators. Less apparent from this overview is when the actions can proceed in parallel and when they constrain each other. The cognitive architecture on which the model is built is ACT-R/PM. The ACT-R/PM architecture combines ACT- R’s theory of cognition (Anderson & Lebiere, 1998) with modal theories of visual attention (Anderson, Matessa, & Lebiere, 1997) and motor movement (Kieras & Meyer, 1997). ACT-R/PM explicitly specifies timing information for all three processes as well as parallelism between them. The software architecture facilitates extensions beyond the modal theory of visual attention and motor movements. Our current efforts are taking advantage of this architectural feature to match the modeling effort with the issues raised by the analytic and empirical research in the Argus effort. In particular, we are working on three extensions, one for eye movements, tracking objects, and perceptual support for working memory. Eye Movements. For the analysis of the eye tracking data shows we have incorporated Eye Movements and Movements of Attention extension (EMMA) (Salvucci, 2000) into the model. EMMA provides multiple eye movements per attention shift and provides encoding time for objects based on frequency of attending to the same object and the object’s distance or eccentricity from the current point-of-gaze Tracking Objects. We are currently incorporating into the target selection task a theory of multiple object tracking. Sears and Pylyshyn (in press) have applied the FINST model to multiple object tracking. This theory hypothesizes a stimulus driven mechanism that individuates objects in the environment by pointing to them; that is, assigning an index. The indexing precedes object identification and the index remains bound to the object even if characteristics of the object change. In particular, if the location of the object changes continuously then the index can still be used to point to the object. Attention can be directed to the object with the index as its argument. The dynamic environment of Argus Prime seems well suited to modeling this theory as a possible mechanism used by subjects in the target selection phase. Perceptual Support for Working Memory. ACT-R/PM provides for both external and internal sources of activation for memory retrieval. Currently the amount of external source activation is a free parameter. Our current efforts are involved with quantifying how the level of external source activation varies with task conditions and what microstategies subjects develop to optimize retrievals by controlling the mix of internal and external source activation. Acknowledgements The work reported was supported by a grant from the Air Force Office of Scientific Research AFOSR#F49620-97-1- References Anderson, J. R., & Lebiere, C. (Eds.). (1998). Atomic components of thought. Hillsdale, NJ: Erlbaum. Anderson, J. R., Matessa, M., & Lebiere, C. (1997). ACT-R: A theory of higher-level cognition and its relation to visual attention. Human-Computer Interaction, 12(4), Gray, W. D., Schoelles, M. J., & Fu, W.-t. (1999). Modeling microstrategies in a continuous dynamic task. Manuscript submitted for publication. Kieras, D. E., & Meyer, D. E. (1997). An overview of the EPIC architecture for cognition and performance with application to human-computer interaction. Human- Computer Interaction, 12(4), 391-438. Sears, C. R., & Pylyshyn, Z. W. (in press). Multiple object tracking and attentional processing. Canadian Journal of Experimental Psychology. Salvucci, D. D. (2000). A model of eye movements and visual attention. In Proceedings of the International Conference on Cognitive Modeling (pp. 252-259). Veenendal, The Netherlands: Universal Press.

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.