AT FIRST GLANCE, it seems trivial to say that I can see, but not hear, the shape of objects. If I say, “I see a piano,” you will readily assume that I am seeing the piano's shape as a visual image. But if I say, “I hear a piano,” you will probably assume that I am hearing the sound that the piano makes, rather than hearing its shape as an auditory image.
But is it true that objects have no auditory shape comparable to their visual shape? What would it be to 'hear shapes?' Why does that idea seem so odd? Why do the phrases “seeing a piano” and “hearing a piano” have such radically different meanings? And what can we learn about auditory perception from considering these questions? Let's begin by making a distinction between (1) the shape or structure of energy per se and (2) the shape or structure of objects perceived through energy. For audition, let's call these two modes of perception hearing sound (i.e.,
“I hear the sound the piano is producing”') and hearing shape (i.e., 'I hear the shape of the piano').
These two modes of auditory perception have corresponding modes of visual perception. Hearing sound is analogous to seeing spatial and temporal patterns of light energy (seeing light). And hearing shape is analogous to seeing objects by way of light energy interacting with them (seeing shape).
Thus, in the former sense we might say, “I see the light the candle is producing,” and in the latter sense, "I see the shape of the candle."
ABOUT THE AUTHOR: Dr. Rod Rees teaches psychology at Western Washington University, Bellingham, Washington.
Figure 1 shows these four perceptions in a 2-by-2 matrix. From the sym metrical nature of the matrix, it would seem that we have a very straightforward pair of analogies: (1) seeing light is analogous to hearing sound, and (2) seeing shape is analogous to hearing shape. But it's not quite that simple.
There seems to be some fundamental difference between auditory and visual perception that belies the nice symmetry in Fig. 1-for some reason, “seeing a piano” has a totally different meaning than “hearing a piano.” Let's now examine these four perceptual possibilities in greater detail- what do we mean, more precisely, when we say, 'I see light” or 'I hear sound?'
I See the Light
Even the simple statement 'I see light”' can have several different meanings. It might refer to certain diffuse or refractive lighting conditions such as haze, beams of light, film colors, or rainbows.
Or it might refer to a source of light such as a candle's flame, a glowing tungsten filament, or a phosphorescent chunk of radium.
Are we really seeing light energy per se in these examples, or are we seeing objects? Not an easy question, is it? Haze, for example, is produced by random reflections of light off minute air borne particles, so that what we are really seeing are objects, albeit a vast multitude of very tiny ones. What about rainbows? Surely they are not “things” (not in the sense that we can find the end of one!), but then again it is the presence of airborne water droplets that creates the refractive conditions necessary to separate the spectral colors that we call a “rainbow.” And with the glowing tungsten filament, we are aware not only of the sensation of brightness but also quite distinctly of the shape of the wire filament that glows so brightly.
Yet another meaning of “I see light' refers to the conditions under which objects become visually apparent. We have no difficulty distinguishing a lighted room from a dark room, but what is the difference-what do we mean when we say we see a lighted room? What we really mean is that we can see the shape of the room and the objects in it, not that we can see the energy that lights the room. In general, when we say, 'I see light,’ we are really referring to the things, the shapes, the objects that the light has revealed by reflection, by re fraction, by interference, or by location as a source. We don't really “see the light,” or at least not very often or very convincingly. We don't see light; we see with light.
The Sound of No Hands Clapping
What does it mean to 'hear a piano”- to hear sounds? Do we hear strings vibrating, speaker cones oscillating, air molecules pounding against our tympanic membranes, staccato electromechanical pulses racing through our auditory nerves? I might imagine any of these objects as the source of a sound, but the source is not the sound. If I try to imagine vibrating strings, for example, I get a visual image of vibrating strings-a visual shape, not an auditory shape.
Light seems to have little perceptual quality of its own, separate from the objects that it reveals, but sound seems to be readily separated from its objects, to have an independent reality. Perceptually, sound is much more “solid” than light. Light seems to be embedded in its objects-stuck, so to speak, on the surface of things-so that it doesn't appear on its own. But sound seems to float free of its objects, almost as if it were a palpable entity that has been “released”' from them. Sound seems almost to be an object in its own right, to exist as a thing, almost but not quite touchable. It is tantalizingly “there,”“' just on the edge of being a solid, three dimensional entity. Sound is much less ephemeral than light.
Unlike light, we don't hear with sound as much as we hear sound per se. When I 'hear a piano” I might see the visual shape of a piano in my imagination, but what I hear is not the auditory shape of the piano-it is the sound the piano is producing.
The Sound of Two Hands Not Clapping
The asymmetry in the 2-by-2 matrix should now be clear. A more accurate rendering is shown in Fig. 2. Seeing shape is more perceptually salient than seeing light, whereas hearing sound is more salient than hearing shape.
As a further asymmetry, we can “hear shape” much more readily than we can “see light.” In fact, you've probably already thought of several examples of hearing shape. Indeed, sonic echoes reveal many things about the shape of things (and you don't have to be a bat or a dolphin to hear them).
Perhaps the most striking example is in hearing the shape of rooms. This can be demonstrated under controlled lab oratory conditions by listening to recordings produced in a variety of rooms of different size, contour, and texture.
Binaural recordings played back over headphones are especially good at revealing the auditory shape of rooms, but well-miked stereo recordings played back over phase-coherent loudspeakers also can function in the same way. The shape of a room is encoded within the ambient sound field in both the time and frequency domains (see References:
Heyser, 1976a, 1976b), and our cars and brain have the capability of decoding that information. Literally, not merely figuratively, we can hear the shape of a room. The Taj Mahal does not 'look' the same as the Seattle King Dome (fortunately for the Taj Mahal), nor does it “'listen” the same (unfortunately for the King Dome).
Not only can we hear the shape of a room, but we can also hear objects within a room. (Remember, I'm not referring to the sound an object produces, but to the auditory shape of the object.) Let's imagine that you're listening to your favorite organ recording of Bach's Toccata and Fugue in D Minor (BWM 565) over your rather beamy set of loudspeakers (eyes closed, of course) when someone tiptoes be tween you and the speaker(s). Instant sonic shadow! He didn't make any noise, yet you heard his ghostly presence.
Sonically, what your ear and brain detected was a change in your loud speaker's frequency spectrum (a high frequency rolloff) and altered phase relations. But the phenomenon that you perceived was of a large shadowy object moving into the space between you and Bach's Toccata. You heard the shape of that object, or at least its presence in the room, although it was not a very detailed or specific shape in the way it would have been if your eyes had been open.
Let me collect these various observations about sight and sound to summarize the asymmetrical matrix in Fig. 2. Four distinct perceptual qualities arise:
1) in vision, we are exceptionally aware of the visual shape of objects as solid three-dimensional entities com posed of numerous surface qualities; whereas,
2) light energy per se provides, at best, an ephemeral and fleeting sense of awareness, and then only by virtue of interaction with objects;
3) in audition, we are sometimes aware of the auditory shape of an object as a space with size, contour, and texture, or as a shadowy presence with in a space; whereas,
4) sound energy per se provides a striking, nearly palpable sense of aware ness, almost as an object in its own right and distinct from its source.
Why the Differences? What is the source of these differences between visual and auditory perception? Possibly they are caused by some basic difference between the nature of light energy and the nature of sound energy, but I don't think so. Remember, these perceptual differences are not absolute. It's more an emphasis of one quality of awareness over another- more a bias toward perceiving auditory events in one way and visual events in another. It's not that we can't perceive visual energy or auditory shapes. It's just that normally we don't. So rather than looking for basic differences be tween light energy and sound energy, we might be better off looking for the natural conditions that could lead to a bias for one quality of awareness over another.
FIGURE 1; FIGURE 2: The four perceptual possibilities, emphasizing their complex asymmetrical saliency.
The Long and the Short of It
Consider our constant and critical need to be aware of small objects and minute textural details. From that point of view, the wavelengths of ambient energy sources become important in how perception evolved. Shorter wavelengths have greater potential for enabling us to resolve small objects and fine textural details. Short wavelengths interact with small things but long wavelengths don't.
Much of what we need to know about objects is too small to be resolved by the relatively long waves that constitute sound energy, but is exquisitely revealed by interaction with much shorter light waves. If we had to rely exclusively on long sound waves, our world would appear devoid of most of the small objects and fine textural de tail that we now perceive through vision. So our perception has become biased toward “seeing” objects and textures through their interactions with short-wavelength ambient light energy instead of through long-wavelength ambient sound energy. Thus, we perceive objects and textures as auditory shapes only when their scale is large.
Production Of and Interaction With
A second natural biasing condition has to do with production of energy in contrast to interaction with energy. Throughout our evolutionary history, there have been relatively few sources that have produced significant and useful amounts of light energy. The primary ones have been sun and fires. Most objects are not sources of light, but instead interact with the ambient light coming from these sources. It seems natural, then, that we use ambient light energy to tell us about the vast number of objects with which it interacts, more so than about the very few sources that produce it.
On the other hand, most objects on the human scale are capable of producing audible sound energy if sufficiently disturbed. Most objects are resonant sources of audible sound. Moreover, the sonic qualities of an object's resonance tell us much about the nature of the object, as well as about the event that caused it to resonate.
Perceptually, a sound seems to 'stand for' or signify the object and event that produced it. It's as if a sound takes on some of the solid reality of the object and event it signifies, with the result that we perceive a signifying sound as a sonic event in and of itself.
In most natural circumstances, from the loudness of a sound we can know roughly the size of the object that produced it and the magnitude of its causal event. Moreover, from the pattern of resonant frequencies that an object produces, we can know much about its composition (e.g., wood or brass), its structure (e.g., solid or hollow), its location, and other physical qualities. The nature of auditory perception is such that a sound is perceived as having the same qualities as those of the object that produced it.
For example, if someone hits a hollow wooden box, we would describe the resulting sound as 'hollow and wooden.’ We commonly give to the sound the physical attributes of the object that produced it. On the other hand, if someone illuminates that same hollow wooden box, we would be very unlikely to say that the light was “hollow and wooden.” In fact, we would not likely describe the light at all, but instead would say that the box was “hollow and wooden.”
Summing Up
Initially, I asked why the phrases “I see a piano” and “I hear a piano” have such radically different meanings, and what we might learn about auditory perception from exploring this question. What arose was the asymmetrical matrix of Fig. 2, illustrating that in vision we are biased toward perceiving shape while in audition we are biased toward perceiving sound. To account for this asymmetry, I noted several natural facts and their resulting generalizations:
1) Ambient light energy works well for revealing the multitude of small objects and fine textural detail in our natural environment by interacting with the objects and textures. Hence, in vision we see shapes.
2) There are relatively few sources of light energy that call for our attention.
Hence, in vision we don't see light.
3) Ambient sound energy does not work well for revealing small objects and fine textural details. Hence, in audition we hear only relatively large objects.
4) Almost all objects and events on the human scale can be significant sources of sound energy. Hence, in audition sounds signify the objects and events that produced them, and we perceive signifying sounds as events in themselves.
5) Sounds encode the physical qualities of the objects and events that produce them. Hence, in audition we perceive in sounds the same qualities as those of the objects and events that produced them.
I hope you will compare these generalizations about auditory perception with facts from your own sonic experiences. Surely, there must be many auditory phenomena that violate these general statements. To uncover the phenomena of perception, look care fully not only at what you hear but also at how you verbalize what you hear.
With your feedback over the next few months (TAA, Attn: “What I Hear”), perhaps we can begin to sketch out a more detailed and sophisticated answer to that age-old question, 'Did you hear something?”
REFERENCES
1. Heyser, Richard C., “Perspectives in Audio Analysis: Changing the Frame of Reference,” Part 1, Journal of the Audio Engineering Society, 24, 8, 660-667. (1976a)
2. Heyser, Richard C., “Perspectives in Audio Analysis: Changing the Frame of Reference,” Part II, Journal of the Audio Engineering Society, 24, 9, 742-751. (1976b)
FURTHER READING
Bech, Sgren, and O. Juhl Pederson, eds. Proceedings of a Symposium on Perception of Reproduced Sound, Stougaard Jensen, Copenhagen, 1987. Hardbound, 165 pp.
++++++++++++++++
Also see: