Models of Attention
John G Taylor, PhD
Models of attention have a venerable history going back to Aristotle, who considered attention as a narrowing of the senses.
More recently, numerous experiments have been performed to discover brain regions involved in various aspects of attention. Using global functional brain imaging techniques (PET and fMRI) various experiments have shown that moving the focus of attention is achieved by a different brain network from that involved in processing the input being attended to [1, 2].
The regions exercising control of attention movement are in parietal and prefrontal sites, while attended sites are in primary and secondary cortices in the various senses (and also in motor cortex for response). These locations have been supported by study of deficits in the speed of attention movement, as shown from studies by Posner and colleagues . Attention modulates activity in the input sites, as shown both globally from fMRI data in attention paradigms  and by analysis of single cells in monkey early visual cortex .
Attention achieves its effects on earlier cortical sites by feedback, which changes the classical receptive fields of cells in anaesthetised monkeys as compared to awake animals . Detailed timing analyses in humans, using EEG and fMRI methods, support the existence of attention-controlled feedback , as well as the general model of control arising from superior parietal sites in the fast dorsal steam to gate slower object representations in the ventral stream 
Attention control has been found to arise by two mechanisms, one by bottom-up signals from the occurrence of unexpected and strong inputs (such as a brief flash of light), the other by top-down control from some required goal (such as by the face of a friend being searched for in a crowd). It had been thought that bottom-up signals normally achieved attention capture; it is now appreciated that top-down control is usually in charge. Involuntary attention capture by distracting inputs occurs only if they have a property that a person is using to find a target . Thus there is a single control network deciding between the importance of desired (top-down) and unexpected (bottom-up) sites for attention.
The lack of attention capture has been carefully investigated as has the phenomenon of inattentional blindness, in which apparently important and unexpected events just do not draw our attention to them [10, 11]. The two sorts of attention, termed exogenous for bottom-up and endogenous for top-down, have been found to possess quite different times for onset and decay: exogenous attention is rapid, and reaches its maximum effect about 100-200 msecs after cue onset in humans, and then falling away as rapidly. On the other hand endogenous attention is slower, rising gradually to a maximum only at about 300-400 msecs after cueing occurs.
Attention can be divided between two modalities, such as vision and audition,
but the degree of coupling of the control over attention in different modalities is still controversial 
Theoretical models of attention have been produced which try to keep up with the rapid pace of experimental advance described above. These models are of two sorts. One is psychological/ functional, and leads to insights into information flow but not to quantitative comparisons with data. The other uses neural networks, and provides detailed simulations of psychological paradigms by interacting neurons in modules that may or may not be part of the psychological models. These latter models can therefore be more stringently tested. There are now numerous neural network models of attention; only outlines of a selection of them can be considered.
The first psychological model  dissociates the overall control of attention into: alert ® interrupt ® localize ® disengage ® move ® engage ® inhibit. Evidence has been brought forward to support such a dissociation, with various modules, including the pulvinar nucleus in thalamus, as running the separate stages. A second approach  is based on selection of an area in visual space, using both inhibitory and excitatory mechanisms in a separate module to identify objects, and the movement of attention is then achieved by sliding down the gradient of an excitatory hill in another competitive network. A third model also uses biased competition to move attention to objects . These and related models use a form of competition in a higher-level module to guide movement of attention on a lower order one.
Neural network simulations of attention tasks implement this general idea. Simulation of visual search times for targets in the presence of distracters has been performed using an input module (representing early visual cortex), a higher order module where a competition determines where attention shall be focused (as in parietal lobe), and an object-coding module to represent objects learnt in the past (as in temporal cortex). A linear increase in search time with number of distracters has been observed in such simulations [16, 17]. Biasing the competition for attention by a frontal template has also been studied in a model with explicit frontal sites . Finally both a salience-based approach and a synchronized oscillator method have been developed which cause object segmentation in cluttered scenes followed by attention orienting [19, 20].
An explicit engineering control framework has been introduced which fuses these approaches . It uses a plant site (identified as early cortex and temporal lobe), an inverse control module (identified as in parietal lobe), a rules module (in prefrontal cortex) and an observer or forward model (with components in both parietal and prefrontal lobes).This model leads to close agreement of the dependence of response speed-up achieved by attention to a target as the cue-to-target time interval varies from 0 to 1.5 seconds. The calculated rise and fall of the exogenous attention shift benefit and the slower but steady rise of the endogenous shift benefit possess the features observed in humans mentioned earlier .Detailed contributions of some of the control components are still being assessed.
. Hopfinger JB et al (2000) The neural mechanisms of top-down attentional control. Nat Neurosci 3:284-291.
. Kastner S & Ungerleider LG (2000) Mechanisms of Visual Attention in the Human Cortex. Ann Rev Neurosci 23:315-341.
. Posner M & Petersen SE (1990) The Attention System of the Human Brain. Ann Rev Neurosci 13:25-42.
. Friston KJ et al (1995) Characterizing Modulatory Interactions Between Areas V1 and V2 in Human Cortex: A New Treatment of Functional MRI Data. Hum Brain Map 2:211-234.
. Reynolds JH et al (1999) Competitive Mechanisms Subserve Attention in Macaque Areas V2 and V4. J Neursoci 19:1736-1753.
. Lamme VF & Roelfsema (2000) The distinct modes of vision offered by feed-forward and recurrent processing. Trends Neurosci 23:571-579.
 Martinetz A et al (2001) Putting spatial attention on the map. Vis Res 41:1437-1457.
 Vidyasagar TR (1999) A neuronal model of attentional spotlight: parietal guiding the temporal. Brain Res Revs 30:66-76.
. Pashler H (2001) Attention and Performance. Ann Rev Psych 52:629-51.
. Mack A & Rock I (1998) Inattentional Blindness: Perception without Attention. Ch 3, pp 55-76 in visual attention, ed RD Wright. Cambridge MA: MIT Press..
. Simms DJ (2000) Attentional capture and inattentional blindness. Trends Cognit Sci 4:147-155.
. Spence C et al (2000) Cross-modal selective attention: On the difficulty of ignoring sounds at the locus of visual attention. Proc Psychophys. 62:410-424.
 Posner MI et al (1987) Isolating attentional systems: a cognitive-anatomical analysis. Psychobiology 15:107-121.
 LaBerge D & Brown V (1989) Theory of Attentional Operations in Shape Identification. Psych Rev 96:101-124.
 Desimone R & Duncan J (1995) Neural Mechanics of Selective Visual Attention. Ann Rev Neurosci 18:193-222.
 Mozer MC & Sitton M (1999) Computational modeling of spatial attention. Ch 9, pp 341-393 in Attention. ed H Pashler. New York: Taylor & Francis.
 Deco G & Zihl J (1998) A Neuronal Model of Binding and Selective Attention for Visual Search. Pp 262-271 in Cognitive Neuroscience, Heinke D, Humphreys GW & Olson A (eds). London: Springer.
 Jackson SR et al (1994) Networks of Anatomical Areas Controlling Visuospatial Attention. Neural Networks 7:925-944.
 Lee DK, Itti L, Koch C, Braun J. (1999) Attention activates winner-take-all competition among visual filters. Nature Neurosci 2(4):375-81.
 Fellenz W (1994) A sequential model for attentive object selection. In Proc 39th IWK Conf, Ilmenau.
 Taylor JG (2001) Attention as a Neural Control System. pp 272-276 in Proc Int Joint Conf on Neural Networks (IJCNN’01), IEEE Cat #01CH37222C, ISBN# 0-07803-7046-5.
 Taylor JG & Rogers M (2001) A control model of the movement of attention.
Neural Networks 15: 309-326.
John G Taylor
Department of Mathematics, King’s College, Strand, London WC2R2LS, UK