Cognitive Psychology and Its Implications

4

Mental Imagery

Try answering these two questions:

• How many windows are in your house? • How many nouns are in the American Pledge of Allegiance?

Most people who answer these questions have the same experience. For the first

question they imagine themselves walking around their house and counting windows.

For the second question, if they do not actually say the Pledge of Alliance out loud,

they imagine themselves saying the Pledge of Allegiance. In both cases they are creating

mental images of what they would have perceived had they actually walked around

the house or said the Pledge of Allegiance.

Use of visual imagery is particularly important. As a result of our primate heritage,

a large portion of our brain functions to process visual information. Therefore, we use

these brain structures as much as we can, even in the absence of a visual signal from

the outside world, by creating mental images in our heads. Some of humankind’s most

creative acts involve visual imagery. For instance, Einstein claimed he discovered the

theory of relativity by imagining himself traveling beside a beam of light.

A major debate in this field of research has been the degree to which the processes

behind visual imagery are the same as the perceptual and attentional processes that we

considered in the previous two chapters. Some researchers (e.g., Pylyshyn, 1973, in an

article sarcastically titled “What the mind’s eye tells the mind’s brain”) have argued that

the perceptual experience that we have while doing an activity such as picturing the

windows in our house is an epiphenomenon; that is, it is a mental experience that does

not have any functional role in information processing. The philosopher Daniel Dennett

(1969) also argued that mental images are epiphenomenal—that is, that the perceptual

components of mental images are not really functional in any way:

Consider the Tiger and his Stripes. I can dream, imagine or see a striped tiger, but

must the tiger I experience have a particular number of stripes? If seeing or imagining

is having a mental image, then the image of the tiger must—obeying the rules of

images in general—reveal a definite number of stripes showing, and one should be

able to pin this down with such questions as “more than ten?”, “less than twenty?”

(p. 136)

Anderson7e_Chapter_04.qxd 8/20/09 9:42 AM Page 92

Verbal Imagery versus Visual Imagery | 93

Dennett’s argument is that if we are actually seeing a tiger in a mental image, we

should be able to count its stripes just like we could if we actually saw a tiger.

Because we cannot count the stripes in a mental image of a tiger, we are not having

a real perceptual experience. This argument is not considered decisive, but it does

illustrate the discomfort some people have with the claim that mental images are

actually perceptual in character. Cognitive Psychology and Its Implications

This chapter will review some of the experimental evidence showing the ways that

mental imagery does play a role in information processing. We will define mental

imagery broadly as the processing of perceptual-like information in the absence of an

external source for the perceptual information. We will consider the following questions: • How do we process the information in a mental image? • How is imaginal processing related to perceptual processing? • What brain areas are involved in mental imagery? • How do we develop mental images of our environment and use these

to navigate through the environment?

•Verbal Imagery versus Visual Imagery

There is increasing evidence from cognitive neuroscience that several different

brain regions are involved in imagery. This evidence has come both from studies

of patients suffering damage to various brain regions and from studies of the

brain activation of normal individuals as they engage in various imagery tasks.

In one of the early studies of brain activation patterns during imagery, Roland

and Friberg (1985) identified many of the brain regions that have been investigated

in subsequent research. They had participants either mentally rehearse a

word jingle or mentally rehearse finding their way around streets in their neighborhoods.

The investigators measured changes in blood flow in various parts of

the cortex. Figure 4.1 illustrates the principal areas they identified.When participants

engaged in the verbal jingle task, there was activation in the prefrontal cortex

near Broca’s area and in the parietal-temporal region of the posterior cortex

R

R

R

R

J

J

FIGURE 4.1 Results from

Roland and Friberg’s (1985)

study of brain activation

patterns during mental imagery.

Regions of the left cortex

showed increased blood flow

when participants imagined

a verbal jingle (J) or a spatial

route (R).

Brain Structures

Anderson7e_Chapter_04.qxd 8/20/09 9:42 AM Page 93

near Wernicke’s area. As discussed in Chapter 1, patients with damage to these

regions show deficits in language processing. When participants engaged in the

visual task, there was activation in the parietal cortex, occipital cortex, and temporal

cortex. All these areas are involved in visual perception and attention, as

we saw in Chapters 2 and 3.When people process imagery of language or visual

information, some of the same areas are active as when they process actual

speech or visual information. Cognitive Psychology and Its Implications

An experiment by Santa (1977) demonstrated the functional consequence

of representing information in a visual image versus representing it in a verbal

image. The two conditions of Santa’s experiment are shown in Figure 4.2. In

the geometric condition (Figure 4.2a), participants studied an array of three

geometric objects, arranged with one object centered below the other two.

This array had a facelike property—without much effort, we can see eyes and a

mouth. After participants studied the array, it was removed, and they had to

hold the information in their minds. They were presented with one of several

different test arrays. The participants’ task was to verify that the test array contained

the same elements as the study array, although not necessarily in the same

94 | Mental Imagery

Study

array

arrays

Test

Test

arrays

Study

array

Identical,

same configuration

Same elements,

linear configuration

Different elements,

same configuration

Different elements,

linear configuration

Triangle Circle

Square

Triangle Circle

Square

Triangle Circle Square

Triangle Circle

Arrow

Triangle Circle Arrow

Identical,

same configuration

Same word,

linear configuration

Different words,

same configuration

Different words,

linear configuration

(a) Geometric condition

(b) Verbal condition

FIGURE 4.2 The procedure followed in Santa’s (1977) experiment demonstrating that visual

and verbal information is represented differently in mental images. Participants studied an initial

array of objects or words and then had to decide whether a test array contained the same

elements. Geometric shapes were used in (a), words for the shapes in (b).

Anderson7e_Chapter_04.qxd 8/20/09 9:42 AM Page 94

spatial configuration. Thus, participants should

have responded positively to the first two test

arrays and negatively to the last two. Santa was

interested in the contrast between the two positive

test arrays. The first was identical to the

study array (same-configuration condition). In

the second array, the elements were displayed

in a line (linear-configuration condition). Santa

predicted that participants would make a positive

identification more quickly in the first case,

where the configuration was identical—because,

he hypothesized, the mental image for the study

stimulus would preserve spatial information. The

results for the geometric condition are shown in

Figure 4.3. As you can see, Santa’s predictions were confirmed. Participants were

faster in their judgments when the geometric test array preserved the configuration

information in the study array. Cognitive Psychology and Its Implications

The results from the geometric condition are more impressive when contrasted

with the results from the verbal condition, illustrated in Figure 4.2b.

Here, participants studied words arranged exactly as the objects in the geometric

condition were arranged. Because it involved words, however, the study stimulus

did not suggest a face or have any pictorial properties. Santa speculated that participants

would read the array left to right and top down and encode a verbal

image with the information. So, given the study array, participants would encode

it as “triangle, circle, square.” After they studied the initial array, one of the test

arrays was presented. Participants had to judge whether the words were identical.

All the test stimuli involved words, but otherwise they presented the same

possibilities as the test stimuli in the geometric condition. The two positive stimuli

exemplify the same-configuration condition and the linear-configuration

condition. Note that the order of words in the linear array was the same as it

was in the study stimulus. Santa predicted that, unlike the geometric condition,

because participants had encoded the words into a linearly ordered verbal image,

they would be fastest when the test array was linear. As Figure 4.3 illustrates,

his predictions were again confirmed. Cognitive Psychology and Its Implications

Different parts of the brain are involved in verbal and visual imagery,

and they represent and process information differently.

•Visual Imagery

Most of the research on mental imagery has involved visual imagery, and this

will be the principal focus of this chapter. One function of mental imagery is to

anticipate how objects will look from different perspectives. People often have

the impression that they rotate objects mentally to achieve perspective. Roger

Shepard and his colleagues have been involved in a long series of experiments

Visual Imagery | 95

Geometric

Verbal

Reaction time (s)

1.25

1.15

Same

configuration

Linear

configuration

FIGURE 4.3 Results from

Santa’s (1977) experiment. The

data confirmed two of Santa’s

hypotheses: (1) In the geometric

condition, participants would

make a positive identification

more quickly when the configuration

was identical than when

it was linear, because the visual

image of the study stimulus

would preserve spatial information.

(2) In the verbal condition,

participants would make a

positive identification more

quickly when the configuration

was linear than when it was

identical, because participants

had encoded the words from

the study array linearly, in

accordance with normal reading

order in English.

Anderson7e_Chapter_04.qxd 8/20/09 9:42 AM Page 95

on mental rotation. Their research was among the first to study the functional

properties of mental images, and it has been very influential. It is interesting to

note that this research was inspired by a dream (Shepard, 1967): Shepard awoke

one day and remembered having visualized a 3-D structure turning in space.

He convinced Jackie Metzler, a first-year graduate student at Stanford, to study

mental rotation, and the rest is history.

Their first experiment was reported in the journal Science (Shepard &

Metzler, 1971). Participants were presented with pairs of 2-D representations

of 3-D objects, like those in Figure 4.4. Their task was to determine whether the

objects were identical except for orientation. The two objects in Figure 4.4a

are identical, as are the two objects in Figure 4.4b, but in both cases the pairs

are presented at different orientations. Participants reported that to match the

two shapes, they rotated one of the objects in each pair mentally until it was

congruent with the other object. There is no way to rotate one of the objects in

Figure 4.4c so that it is identical with the other.

The graphs in Figure 4.5 show the times required for participants to decide

that the members of pairs were identical. The reaction times are plotted as a

function of the angular disparity between the two objects presented. The angular

disparity is the amount one object would have to be rotated to match the other

object in orientation. Note that the relationship is linear—for every increment

in amount of rotation, there is an equal increment in reaction time. Reaction

time is plotted for two different kinds of rotation. One is for 2-D rotations

(Figure 4.4a), which can be performed in the picture plane (i.e., by rotating the

page); the other is for depth rotations (Figure 4.4b), which require the participant

to rotate the object into the page. Note that the two functions are very

similar. Processing an object in depth (in three dimensions) does not appear

to have taken longer than processing an object in the picture plane. Hence,

participants must have been operating on 3-D representations of the objects in

both the picture-plane and depth conditions. Cognitive Psychology and Its Implications

These data might seem to indicate that participants rotated the object in a

3-D space within their heads. The greater the angle of disparity between the two

objects, the longer participants took to complete the rotation. Though the

participants were obviously not actually rotating a real object in their heads,

the mental process appears to be analogous to physical rotation.

96 | Mental Imagery

(a) (b) (c)

FIGURE 4.4 Stimuli in the Shepard and Metzler (1971) study on mental rotation. (a) The

objects differ by an 80° rotation in the picture plane (two dimensions). (b) The objects differ

by an 80° rotation in depth (three dimensions). (c) The objects cannot be rotated into

congruence. (From Metzler & Shepard, 1974. Reprinted by permission of the publisher. © 1974 by Erlbaum.)

Anderson7e_Chapter_04.qxd 8/20/09 9:42 AM Page 96

There has been a great deal of subsequent research examining the mental

rotation of all sorts of different objects. The typical finding is that the time

required to complete a rotation does vary with the angle of disparity. In recent

years, there have been a number of brain-imaging studies that looked at what

regions are active during mental rotation. Consistently, the parietal region

(roughly the region labeled R at the upper back of the brain in Figure 4.1) has

been activated across a range of tasks. This finding corresponds with the results

we reviewed in Chapter 3 showing that the parietal region is important in spatial

attention. Some tasks involve activation of other areas. For instance, Kosslyn,

DiGirolamo, Thompson, and Alpert (1998) found that imagining the rotation of

one’s hand produced activation in themotor cortex. Cognitive Psychology and Its Implications

Neural recordings of monkeys have provided some evidence about neural

representation during mental rotation involving hand movement. Georgopoulos,

Lurito, Petrides, Schwartz, and Massey (1989) had monkeys perform a task in

which they moved a handle at a specific angle in response to a given stimulus. In

the base condition, monkeys just moved the handle to the position of the stimulus.

Georgopoulos et al. found cells that fired for particular positions. So, for

instance, there were cells that fired most strongly when the monkey was moving to

the 9 o’clock position and other cells that responded most stronglywhen the monkey

moved to the 12 o’clock position. In the rotation condition, the monkeys had

to move the handle to a position rotated some number of degrees from the stimulus.

For instance, if the monkeys had to move the handle 90° counterclockwise and

the stimulus appeared at the 12 o’clock position, they would have to move the

handle to 9 o’clock. If the stimulus appeared at the 6 o’clock position, they would

have to move to 3 o’clock. The greater the angle, the longer it took the monkeys

Visual Imagery | 97

Angle of rotation (degrees)

(a) (b)

0 40 80 120 160

0

1

2

4

3

5

Mean reaction (s)

0

1

2

4

3

5

Mean reaction (s)

0 40 80 120 160

FIGURE 4.5 Results of the Shepard and Metzler (1971) study on mental rotation. The mean

time required to determine that two objects have the same 3-D shape is plotted as a function

of the angular difference in their portrayed orientations. (a) Plot for pairs differing by a rotation

in the picture plane (two dimensions). (b) Plot for pairs differing by a rotation in depth (three

dimensions). (From Metzler & Shepard, 1974. Reprinted by permission of the publisher. © 1974 by Erlbaum.)

Anderson7e_Chapter_04.qxd 8/20/09 9:42 AM Page 97

to initiate the movement, suggesting that this task involved a mental rotation

process to achieve the transformation. In this rotation condition, Georgopoulos

et al. found that various cells fired at different times during the transformation.At

the beginning of a transformation trial, when the stimulus was presented, the cells

that fired most were associated with a move in the direction of the stimulus.By the

end of a transformation trial, when the monkey actually moved the handle, maximumactivity

occurred in cells associated with the movement. Between the beginning

and the end of the trial, cells representing intermediate directions were most

active. These results suggest that mental rotation involves gradual shifts of firing

from cells that encode the initial stimulus to cells that encode the transformed

stimulus or, in this case, the transformed response.

When people must transform the orientation of a mental image to make

a comparison, they rotate its representation through the intermediate

positions until it achieves the desired orientation. Cognitive Psychology and Its Implications

Image Scanning

Something else we often do with mental images is to scan them looking for some

critical information. For instance, when people are asked how many windows

there are in their houses (the task described at the beginning of this chapter),

many report mentally going through the house visually as they count the

windows. Researchers have been interested in the degree to which people are

actually scanning perceptual representations in such tasks, as opposed to just

retrieving abstract information. For instance, are we really “seeing” each window

in the room or are we just remembering how many windows are in the room?

Brooks (1968) performed an important series of experiments on the scanning

of visual images. He had participants scan imagined diagrams such as the

one shown in Figure 4.6. For example, the participant was to scan around an

imagined block F from a prescribed starting point and in a prescribed direction,

categorizing each corner of the block as a point in the top or bottom (assigned a

yes response) or as a point in between (assigned a no response). In the example

(beginning with the starting corner), the correct sequence of responses is yes, yes,

yes, no, no, no, no, no, no, yes. For a nonvisual contrast task, Brooks also gave

participants sentences such as “A bird in the hand is not in the bush.” Participants

had to scan the sentence while holding it in memory, deciding whether

each word was a noun or not. A second experimental variable was how participants

made their responses. Participants responded in one of three ways:

(1) said yes or no; (2) tapped with the left hand for yes and with the right hand

for no; or (3) pointed to successive Y’s or N’s on a sheet of paper such as the

one shown in Figure 4.7. The two variables of stimulus material (diagram or

sentence) and output mode were crossed to yield six conditions.

Table 4.1 gives the results of Brooks’s experiment in terms of the mean

time spent in classifying the sentences or diagrams in each output condition.

The important result for our purposes is that participants took much longer

for diagrams in the pointing condition than in any other condition, but this

was not the case when participants were working with sentences. Apparently,

98 | Mental Imagery

FIGURE 4.6 An example of a

simple block diagram that Brooks

used to study the scanning of

mental images. The asterisk and

arrow show the starting point

and the direction for scanning

the image. (From Brooks, 1968.

Reprinted by permission of the publisher.

© 1968 by the Canadian Psychological

Association.)

Anderson7e_Chapter_04.qxd 8/21/09 6:58 PM Page 98

scanning a physical visual array conflicted with scanning a mental array.

This result strongly reinforces the conclusion that when people are scanning

a mental array, they are scanning a representation that is analogous

to a physical visual array. Requiring the person simultaneously to engage

in a conflicting scanning action on an external physical visual array disrupts

the mental scan.

One might think that Brooks’s result was due to the conflict between

engaging in a visual pointing task and scanning a visual image.

Subsequent research makes it clear, however, that the interference is not a

result of the visual character of the task per se. Rather, the problem is spatial

and not specifically visual; it arises from the conflicting directions in

which participants had to scan the physical visual array and the mental

image. For instance, in another experiment, Brooks found evidence of

similar interference when participants had their eyes closed and indicated

yes or no by scanning an array of raised Y’s and N’s with their fingers. In

this case, the actual stimuli were tactile, not visual. Thus, the conflict is

spatial, not specifically visual. Cognitive Psychology and Its Implications

Baddeley and Lieberman (reported in Baddeley, 1976) performed an

experiment that further supports the view that the nature of the interference

in the Brooks task is spatial rather than visual. Participants were

required to perform two tasks simultaneously. All participants performed

the Brooks letter-image task. However, participants in one group simultaneously

monitored a series of stimuli of two possible brightnesses and had

to press a key whenever the brighter stimulus appeared. This task involved

the processing of visual but not spatial information. Participants in the

other condition were blindfolded and seated in

front of a swinging pendulum. The pendulum

emitted a tone and contained a photocell. Participants

were instructed to try to keep the beam of

a flashlight on the swinging pendulum. Whenever

they were on target, the photocell caused the

tone to change frequency, thus providing auditory

feedback. This test involved the processing

of spatial but not visual information. The spatial

auditory tracking task produced far greater

impairment in the image scanning task than did

the brightness judgment task. This result also

indicates that the nature of the impairment in

the Brooks task was spatial, not visual. Cognitive Psychology and Its Implications

People suffer interference in scanning a mental image if they have to

simultaneously process a conflicting perceptual structure.