Searching Images and Visual Attention | UNL Center for Avian Cognition

Our view of the world is an apparently unbiased, photographic image of our surroundings, but that is an illusion. What we think we see is actually an elaborate reconstruction that is continuously modified to suit changing cognitive needs. In Ian Fleming’s short story, The Hildebrand Rarity, James Bond is given the task of collecting one of the world’s rarest fish. It has only been seen once before, so all he has to go on is a verbal description of a pink, striped fish with sharp, spiny fins. As Bond swims along the reef, he muses to himself, “When you are looking for one particular species under water, ... you have to keep your brain and your eyes focused for that one individual pattern. The riot of colour and movement and the endless variety of light and shadows fight your concentration all the time” (Fleming 1960).

What Fleming calls "concentration" is visual attention, the process of "taking possession by the mind," as William James described it, of "one out of several simultaneously possible objects or trains of thought (James, 1890)." If you are searching for a specific item in a natural environment, a focused, attentional search is vastly more efficient than an undirected scan that sequentially examines each distinctive patch of color or texture. That is especially true if the environment, like Bond’s coral reef, is a confusing mosaic of contrasting colors and textures. By focusing your visual search, you actually change your perception of the surroundings to emphasize the things you are searching for. Attention pulls together your cognitive resources and brings them to bear on the demands of the task. But the benefits of visual attention come at a cost: a focused search limits the range of things that can readily be found. Attention is like a spotlight: it helps you to see better where the light of enhanced cognition is shining, but it reduces your ability to see elsewhere. You are more likely to find what you are looking for, but you may utterly fail to find something other than what you expect.

Jakob von Uexküll described the role of attention in visual search in terms of what he called a searching image, a cognitive representation of the sought-for stimulus. In his best-known anecdote, von Uexküll was accustomed to having water at his meals supplied from an earthenware pitcher. One day, the pitcher had been broken, so his server replaced it with a glass carafe. When von Uexküll came to the table, he looked in vain for the pitcher. He asked for water and was told that it was in its usual place. Only then, he claimed, did the scattered reflections from the glass carafe “fly together” and form themselves into a coherent image. Von Uexküll asserted that the searching image -- that is, the expected visual features of the pitcher -- "overrode" the perceptual input and prevented him from recognizing the carafe (von Uexküll 1934). The fact that attention has this dark side, obliterating perception of even presumably obvious alternative stimuli, is well understood. It has been referred to as "inattentional blindness" (Rock et al. 1992).

The connection between von Uexküll's insight and the cognitive processes used by foraging predators was first proposed by Luuk Tinbergen (1960). While studying the prey items that insect-eating birds feed to their young, Tinbergen tracked the abundance of many different species of woodland caterpillars and recorded the sequence in which great tits brought each type of caterpillar back to their nest boxes. He found that the birds tended to feed in runs, taking the same type of caterpillars one after another for a time and then switching to a different type. The birds appeared to be searching for only one kind of prey at any given moment, suggesting that recent experience with a particular food type increased the chance of a similar, subsequent discovery. Tinbergen inferred that the birds were hunting using searching images, filtering out alternative stimuli and limiting their search to the visual features of one prey type at a time. In essence, he was proposing that the selection bias was a result of visual attention (Dawkins 1971, Bond 1983, Bond & Kamil 1999, Kamil & Bond 2006).

To operate in the dynamic universe of Tinbergen's birds, however, attentional search has to involve more than just a single visual image. The same species of caterpillar can occur in many different sizes and can be sighted from different angles in different locations at different distances under different illumination conditions. The representation that matches even a single prey type must improve detection along many stimulus dimensions simultaneously, so it must be a multi-layered structure, a template of the sought-for target that, through experience, has come to incorporate all of the essential features that affect the probability of detection. Searching images are the result of activating the template, bringing it to the cognitive foreground and installing it as the current attentional filter (Langley 1996, Bravo & Farid 2009, 2012).

There is room for only one searching image in short-term memory at a time: when one image is replaced by another, the original disappears (Gorea & Sagi 2000, Zhang et al. 2011). To avoid the costs of inattentional blindness in a rapidly changing environment, visual predators need to be prepared to switch from one searching image to another on short notice. How the bird decides to make a switch is not clear, but the replacement process seems to get easier over time (Bond 1983, Langley et al. 1995) as if the template was becoming vaguer and more generalized. The persistence of a searching image may simply depend on how frequently it has been rewarded. If a bird initially found a new caterpillar every ten seconds, but then searched in vain for perhaps two minutes, it might become more open to looking for other prey types, making use of a broader and less specialized template. Tinbergen noticed that there was often a delay of several days between the first hatch of a particular kind of caterpillar in the spring and the time when it began appearing in great tit nests. He inferred that the birds initially ignored new prey types because they were not expecting to see them; a new, distinctive prey would readily be overlooked. But when the old familiar prey type began to become rarer, the tits would eventually start to search more broadly. And when they eventually encountered this novel kind of caterpillar, they would immediately assemble a corresponding template and re-focus their search on the new target (Tinbergen 1960, Bond 1983, Bond & Kamil 1999, Bond & Kamil 1998).

Additional discussion of visual attention can be found in Wolfe 1998, Kamil & Bond 2006, and Bond 2007, as well as in our 2013 book, Concealing Coloration in Animals.