8 Imagery

Introduction

Mental imagery (varieties of which are sometimes colloquially refered to as "visualizing," "seeing in the mind's eye," "hearing in the head," "imagining the feel of," etc.) is quasi-perceptual experience; it resembles perceptual experience, but occurs in the absence of the appropriate external stimuli. It is also generally understood to bear intentionality (i.e., mental images are always images of something or other), and thereby to function as a form of mental representation. Traditionally, visual mental imagery, the most discussed variety, was thought to be caused by the presence of picture-like representations (mental images) in the mind, soul, or brain, but this is no longer universally accepted.

Very often, imagery experiences are understood by their subjects as echoes, copies, or reconstructions of actual perceptual experiences from their past; at other times they may seem to anticipate possible, often desired or feared, future experiences. Thus imagery has often been believed to play a very large, even pivotal, role in both memory (Yates, 1966; Paivio, 1986) and motivation (McMahon, 1973). It is also commonly believed to be centrally involved in visuo-spatial reasoning and inventive or creative thought. Indeed, according to a long dominant philosophical tradition, it plays a crucial role in all thought processes, and provides the semantic grounding for language. However, in the 20th century vigorous objections were raised against this tradition, and it is now widely repudiated.

The Imagery Debate

What is it about?

Visual imagery is a flow of thoughts you can see, hear, feel, smell, or taste. Visual imagery is a window on your inner world, a way of viewing your own ideas, feelings, and interpretations. But it is more than a mere window ---why---It is a means of transformation and liberation from distortions in this realm that may unconsciously direct your life and shape your health.

Imagination, in this sense, is not sufficiently valued in our culture.

Without imagination, humanity would be long extinct.

Visual imagery is probably best known for its direct effects on physiology. Through imagery, you can stimulate changes in many body functions usually considered inaccessible to conscious influence.

How is it?

The original imagery debate is concerned with the question how cognitive mechanisms in the brain function when imagining pictures. First attempts at explaining these processes simply dealt with how during real visual stimuli the light (photons) hits the retina where the picture is decomposed and
reassembled again in the brain. Similar processes occur in the brain in the absence of visual stimuli during the act of imagery. Pictures are produced in our mind without an actual visaul input. Modern cognitive psychologists rather deny the production of pictures in the brain because then, there has to be something (Homunculus) that continuously looks at the pictures and interprets them. Because of the lack of reasonable explanations a behaviourist theory arose that opposed the view that pictures are actually projected into the brain.

Today's imagery debate is mainly influenced by two opposing theories: (1) Zenon Pylyshyn’s propositional theory and (2) Stephen Kosslyn’s depictive representation theory of imagery processing. Pylyshyn idea is that information is stored in the brain in a propositional manner. The sentences “the sun is shining” and “it’s the case that the sun is shining” have the same proposition, namely “shining (sun)” which is stored in a Meta language (all propositions are of the form predicate(arguments)). Contradicting Kosslyn states that there has to be some kind of spatial image representation. His image-scanning experiments discovered that we actually create a mental picture of scenes while trying to solve small cognitive tasks. Kosslyn argued that the responsible mechanisms involve a spatial representation which is similar to the way we conceive things by actually perceiving them. Other advocates of the depictive representation of scenes in our mind, Shepard and Metzler, developed the mental rotation task. Two objects are presented to a participant in different angles and his job is to decide whether the objects are identical or not. The result shows that the reaction times increases linearly with the rotation angle of the objects. This phenomenon can’t be explained by a propositional model, but instead implies that participants mentally rotate the objects in order to match the objects to one another. This process is called mental chronometry.

The actual difference between imagery and perception occurs in their distinct processing behaviour. Perception is a bottom-up process that originates with an image on the retina, whereas imagery is a top-down mechanism which originates when activity is generated in higher visual centers without an actual stimulus. Another distinction can be made by saying that perception occurs automatically and remains relatively stable, whereas imagery needs effort and is fragile.

Biological reasoning of debate

Partially imagery is represented by certain neurons in our medial temporal lobe which might respond to one image, but not to another (category-specific neurons). Lesion techniques have advanced the research on the representation of imagery in our brain. For example, the size of our mental images decreases when our primary visual cortex is damaged. There are also such phenomena as unilateral neglect where the patient simply neglects half of his visual field. This occurs both when perceiving and when imagining an object or a scene. The deficit, also called hemi-neglect, is usually due to a lesion in the right hemisphere (most likely the superior temporal gyrus).

Spatial Representation

Abstract

This target article reviews evidence for the functional equivalence of spatial representation of observed enviroment and environments described in discourse. It is argued that people possess a spatial representation system that constructs mental spatial models on the basis of perceptual and linguistic information. Evidence for a distinct spatial system is reviewed.

Introduction

1.1 Space can be understood through perception and language, but are the mental representations of space the same in both cases? Evidence for this position comes from a number of areas, including mental imagery , such representations appear to be equivalent in form and operation to representations of observed environments.

1.2 A number of empirical effects observed in spatial learning studies can be obtained when subjects do not study a map or physical route but instead read a description of an environment.

1.3 People's spatial representations of descriptions can be seen to interact with perceptual spatial systems.

1.4 Further evidence that spatial descriptions are represented in a spatial format comes from the study of mental models. People generally represent texts in mental models rather than by retaining the linguistic structure of the text

1.5 Although most research on mental models has focussed on text comprehension, researchers generally believe that mental models are perceptually based .Indeed, people have been found to use spatial frameworks like those created for texts to retrieve spatial information about observed scenes (Bryant, 1991). Thus, people create the same sorts of spatial memory representations whether they read about an environment or see it themselves.

What is it?

People create the same sorts of cognitive maps and mental spatial models from verbal descriptions and direct observations. This suggests that people have a distinct spatial representation system that creates spatial models from disparate sources of input and is independent of memory systems for other domains of knowledge. The primary role of the SRS is to organize spatial information in a general form that can be accessed by either perceptual or linguistic mechanisms. The SRS provides the coordinate frameworks in which to locate objects, thus creating a model of a perceived or described environment. The advantage of a coordinate representation is that it is directly analogous to the structure of real space and captures all possible relations between objects encoded in the coordinate space. These frameworks also reflect differences in the salience of objects and locations in accord with properties of the environment and the ways in which people interact with it . Thus, the SRS creates representations that are models of the physical and functional aspects of the environment.

How is spatial knowledge encoded?

What, then, can be said about the primary components of cognitive spatial representation? Certainly, the distinction between the external world and our internal view of it is key, and it is helpful to explore the relationship between the two further from a process-oriented perspective.

The classical approach assumes a complex intern al representation in the mind that is constructed through a series of specific perceived stimuli, and that these stimuli generate specific internal
responses. Research dealing specifically with geographic-scale space has worked from the perspective that the macro-scale physical environment is extremely complex and essentially beyond the control of the individual. This research, such as that of Lynch and of Golledge and his colleagues, has shown that there is a complex of behavioral responses generated from correspondingly complex external stimuli, which are themselves interrelated. Moreover, the results of this research offers a view of our geographic knowledge as a highly interrelated external/internal system. Using landmarks encountered within the external landscape as navigational cues is the clearest example of this interrelationship.

The rationale is as follows: We gain information about our external environment from different kinds of perceptual experience; by navigating through and interacting directly with geographic space as well as by reading maps, through language, photographs and other communication media. With all of these different types of experience, we encounter elements within the external world that act at symbols. These symbols, whether a landmark within the real landscape, a word or phrase, a line on a map, or a building in a photograph, trigger our internal knowledge representation and generate appropriate responses. In other words, elements that we encounter within our environment act as knowledge stores external to ourselves.

Each external symbol has meaning that is acquired through the sum of the individual perceiver's previous experience. That meaning is imparted by both the specific cultural context of that individual and by the specific meaning intended by the generator of that symbol. Of course, there are many elements within the natural environment not "generated" by anyone, but that nevertheless are imparted with very powerful meaning by cultures (e.g., the sun, moon and stars). Manmade elements within the environment, including elements such as buildings, are often specifically designed to act as symbols as at least part of their function. The sheer size of downtown office buildings, the pillars of a bank facade and church spires pointing skyward are designed to evoke an impression of power, stability or holiness, respectively.

These external symbols are themselves interrelated, and specific groupings of symbols may constitute self-contained external models of geographic space. Maps and landscape photographs are certainly clear examples of this. Elements of differing form (e.g., maps and text) can also be interrelated. These various external models of geographic space correspond to external memory.

From the perspective just described, the sum total of any individual's knowledge is contained in a multiplicity of internal and external representations that function as a single, interactive whole. The representation as a whole can therefore be characterized as a synergistic, self-organizing and highly dynamic network.

Propositional Representation

Theory

The theory of Propositional Representation was founded by Dr. Zenon Pylyshyn who invented it in 1973. He described it as an epiphenomenon which accompanies the process of imagery, but is not part of it. Mental images do not show us how the mind works exactly. They only show us that something is happening. Just like the display of a compact disc player. There are flashing lights that display that something happens. We are also able to conclude what happens, but the display does not show us how the processes inside the compact disc player work. Even if the display would be broken, the complact disc player would still continue to play music.

Representation

The basic idea of Propositional Representation is that relationships between objects are representated by symbols and not by spatial mental images of the scene. For example: A bottle under a table would be represented by a formula made of symbols like 'UNDER(BOTTLE,TABLE)' and not by an image which shows a bottle under a table.

Complex objects

According to the theory of spatial representation, complex objects would appear as a mental image of an object, for example a ship. This mental image would consist of all properties which are remembered. Maybe it would look like this:

Even those complex objects can be generated and described by propositional representation. A complex object like a ship would consist of a structure of nodes which represent the objects properties and the relationship of this properties. A propositional representation of a ship may look like this:

Proofs for propositional representation

As we have seen, even complex objects can be represented propositionally by a symbolic structure of nodes. This indicates that people would need a short time to "travel" mentally from one node to an adjacent node and would need much more time to "travel" from one node to another one if they would have to pass many nodes on the way.

Imagery and Perception

Size and the Visual Field

If an object is observed from different distances, it is harder to perceive details if the object is far away because the objects fills only a small part of the visual field. Kosslyn made an experiment in 1973 in which he wanted to find out if this is also true for mental images. He told participants to imagine objects which are far away and objects which are near. After asking the participants about details, he supposed that details can be observed better if the object is near and fills the visual field. He also told the participants to imagine animals with different sizes near by another. For example an elephant and a rabbit. The elephant filled much more of the visual field than the rabbit and it turned out that the participants were able to answer questions about the elephant more rapidly than about the rabbit. After that the participants had to imagine the small animal besides an even smaller animal, like a fly. This time, the rabbit filled the bigger part of the visual field and again, questions about the bigger animal were answered faster. The result of Kosslyns experiments is that people can observe more details of an object if it fills a bigger part of their visual field. This provides evidence that mental images are represented spatial.

Current state of imagery debate

It seems hard to decide, which position has the stronger arguments, since both give good explanations. The development of the recent years shows that the use of modern techniques of neuroscience, like magnetic resonance imaging, was stronger involved in experiments on visual imagery. But the results are not as clear as one could expect: The regions in the brain which are activated during the processing of mental images are the same which are also important for normal visual processing(especially the visual cortex). On the other hand, there are so-called double dissociations between visual images and visual processing, that means that both can be distorted independently from each other. The experimentators concluded that there are different mechanisms involved. Recapitulating, it can be said that there is no definite answer on the debate up until now, but this is also due to the different formulations of the two sides, for example, the propositions of Pylyshyn can probably not be found by applying methods of neuroscience.

Imagery and memory

The loci of imagery effects in several domains are clarified by separating issues related to the storage of information in memory and its use following retrieval. Empirical findings from studies of memory for word and sentence lists, language comprehension and memory, and symbolic comparisons are discussed. These consistently indicate a functional role for imagery in human cognition but provide no data necessitating the storage of perceptual information related to verbal materials in an analog form. Instead, concreteness effects in memory appear to result from differential processing of relational (shared) and item-specific (distinctive) information for high- and low-imagery materials. The available evidence suggests that verbal and imaginal processing systems may operate in conjunction with a more generic semantic memory, the form of which is not an issue here, yielding apparently contradictory findings in support of both dual-code and common-code theories.

References

Ashwin Ram Kenneth Moorman (1999) Understanding Language Understanding - chapter 5

Bertram F. Malle Louis J. Moses Dare A. Baldwin (2001) Intentions and Intentionality - chapter 9

Emmanuel Dupoux . Language, Brain, and Cognitive Development - chapter7

E.Bruce Goldstein, Cognitive Psychology, Connecting Mind, Research, and Everyday Experience (2005) - ISBN: 0-534-57732-6.

John H. Holland, Keith J. Holyoak, Richard E. Nisbett, Paul R. Thagard (1986) Induction Johnson-Laird, P. N. (1983). Mental models: Towards a cognitive science of language, inference, and consciousness. Cambridge, MA: Harvard University Press.Anderson, J. R. (1978). "Arguments Concerning Representations for Mental Imagery." Psychological Review 85(4): 249-276.

Bryant, D. J., B. Tversky, et al. (1992). "Internal and External Spatial Frameworks for Representing Described Scenes." Jornal of Memory and Language 31: 74-98.

Downs, R. (1985). The Representation of Space: Its Development in Children and in Cartography. The Development of Spatial Cognition. R. Cohen. Hillsdale, NJ, Lawrence Erlbaum Associates: 323-344.

Franklin, N. (1992). "Spatial Representation for Described Environments." Geoforum 23(2): 165-174.

Garling, T., A. Book, et al. (1984). "Cognitive Mapping of Large-Scale Environments." Environment and Planning 16(1): 3-34.

Hayward, W. G. and M. J. Tarr (1995). "Spatial Language and Spatial Representation." Cognition 55: 39-84.

Ioerger, T. R. (1994). "The Manipulation of Images to Handle Indeterminacy in Spatial Reasoning." Cognitive Science 18: 551-593.

Kuipers, B. (1978). "Modeling Spatial Knowledge." Cognitive Science 2: 129-153.

Montello, D. R. (1992). The Geometry of Environmental Knowledge. International Conference GIS - From Space to Territory: Theories and Methods of Spatio-Temporal Reasoning. A. U. Frank, I. Campari and U. Formentini. Pisa, Italy, Springer-Verlag.

Portugali, J., Ed. (1996). The Construction of Cognitive Maps. The GeoJournal Library. Dordrecht, Kluwer Academic Publishers.

Taylor, H. and B. Tversky (1992). "Descriptions and Depictions of Environments." Memory & Cognition 20(5): 483-496.* Clahsen, Harald: Lexical Entries and Rules of Language: A Multidisciplinary Study of German Inflection.

Further Reading

• Cherney, Leora (2001): Right Hemisphere Brain Damage

• Grodzinsky, Yosef (2000): The neurology of syntax: Language use without Broca's area.

• Müller, H.M. & Kutas, M. (1996). What's in a name? Electrophysiological differences between spoken nouns, proper names and one's own name.NeuroReport 8:221-225.

• Müller, H. M., King, J. W. & Kutas, M. (1997). Event-related potentials elicited by spoken relative clausesCognitive Brain Research 4:193-203.

Links - german

• University of Bielefeld:

• Müller, H. M., Weiss, S. & Rickheit, G. (1997). Experimentelle Neurolinguistik: Der Zusammenhang von Sprache und GehirnIn: Bielefelder Linguistik (Hrsg.) Aisthesis-Verlag, pp. 125-128.

• Müller, H.M. & Kutas, M. (1997). Die Verarbeitung von Eigennamen und Gattungsbezeichnungen: Eine elektrophysiologische Studie. In: G. Rickheit (Hrsg.). Studien zur Klinischen Linguistik - Methoden, Modelle, Intervention. Opladen: Westdeutscher Verlag, pp. 147-169.

Wikibooks | 79

Chapter 8

• Müller, H.M., King, J.W. & Kutas, M. (1998). Elektrophysiologische Analyse der Verarbeitung natürlichsprachlicher Sätze mit unterschiedlicher Belastung des Arbeitsgedächtnisses. Klinische Neurophysiologie

Links

Cognitive Psychology Osnabrück

Dr. Rolf A. Zwaan's Homepage with many Papers