The Visual Interface

John R. Anderson
Mike Matessa
Christian Lebiere


Theories of higher-level cognition typically ignore lower-level processes such as visual attention and perception. They simply assume that lower-level processes deliver some relatively high-level description of the stimulus situation upon which the higher-level processes operate. This certainly is an accurate characterization of our past work on the ACT-R theory (e.g., Anderson, 1993). The typical task that ACT-R has been applied to is one in which the subject must process some visual array - the array may contain a sentence to be recognized, a puzzle to be solved, or a computer program being written. We have always assumed that some processed representation of this visual array is placed into working memory in some highly encoded form and we modeled processing given that representation.

The strategy of focusing on higher-level processes might seem eminently reasonable for a theory of higher-level cognition. However, the strategy creates two stresses for the plausibility of the resulting models. One stress is that by assuming a processed representation of the input the theorists are granting themselves unanalyzed degrees of freedom in terms of choice of representation. It is not always clear whether the success of the model depends on the theory of the higher-level processes or the choice of the processed representation. The other stress is that the theorist may be ignoring significant problems in access to that information which may be contributing to dependent variables such as accuracy and latency. For instance, the visual input may contain more information than can be held in a single attentional fixation, and shifts of attention (with or without accompanying eye movements) may become a significant but ignored part of the processing. For these reasons we were encouraged to join the growing number of efforts (e.g., Kieras & Meyer, 1994; Wiesmeyer, 1992) to embed a theory of visual processing within a higher-level theory of cognition.

The choice to focus on vision was largely strategic - reflecting the fact that most of the tasks that ACT-R has modeled involved input from the visual modality. To be more exact, most tasks have involved processing input from a computer screen and so we have developed a theory of the processing of a computer screen. However, while we started with a theory of the visual interface, we have more recently become concerned with a more general theory of the coordination of perception, action, and cognition. This more general system will be described in the next chapter on ACT-R/PM. Here we will describe ACT-R's visual interface and the evidence for its connection to the cognitive system. The visual interface is the most developed part of ACT-R/PM and greatly influenced its overall design.


Sperling Task

Subitizing Task

Visual Search Task

Word Superiority Task

Menu Selection Task