The Conflicted Character of Picture Perception

It is often said that picture perception has a dual character, which makes looking at pictures fundamentally different from looking at things like trees, chairs, and people. This duality can be explained in a variety of ways, but it is most commonly explained in terms of competing perceptions of space.1 That is, it is frequently asserted that the perception of pictures (at least those depicting objects in space) involves two distinct impressions of space—the two dimensional picture surface and the three dimensional depicted scene—that are somehow in conflict, or inconsistent, with one another. For instance, R. L. Gregory contends that, visually, pictures have “an extraordinary double reality: flat objects seen as flat, and at the same time as quite different three dimensional objects in a different space.”2 Similarly, James J. Gibson claims that “a picture is both a surface in its own right and a display of information about something else. The viewer cannot help but see both, yet this is a paradox, for the two kinds of awareness are discrepant.”3 In much of the psychological literature on the subject, this perceptual conflict is generally understood as a function of “conflicting cues.” When we look at a picture, so the story goes, there are cues that indicate that we are looking at a flat surface, and also cues that indicate that we are looking at a three dimensional scene. These cues suggest two mutually incompatible impressions of the object we are looking at (the picture), which our visual system is able to resolve in one way or another. The end result is that we perceive both the two dimensional surface of the picture and a three dimensional “pictorial space” simultaneously, despite the inherent contradiction between the two. In what follows, I will argue that the view that picture perception is dualistic, in the sense that it involves the reconciliation or accommodation of two competing impressions of space, is mistaken. First, the notion that there is a contradiction in perceiving an object as flat while perceiving “pictorial space” ignores the fact that the two impressions of space at issue are fundamentally distinct. In order for there to be a contradiction, a picture would have to be flat and not flat in the same respect, or in the same sense. However, the two dimensional picture surface is perceived as belonging to “environmental” or “real space,” while the three dimensional scene is perceived in terms of an “imaginary” or “pictorial space”—and therefore there is simply no contradiction in the simultaneous awareness of each. Second, the contention that picture perception involves confronting conflicting depth cues is based on an unwarranted and problematic assumption. The notion that we are presented with conflicting cues when we look at a picture presupposes that the same cues always communicate the same information, regardless of the context in which we encounter them. Yet, the nature of picture perception itself provides evidence that the information that a given cue communicates depends, at least to some extent, on the context, or its combination with other cues. Given these difficulties with the popular view that there is an inherent conflict in picture perception, then, we ought to reconsider the connection between perceiving a picture's surface and perceiving “pictorial space.” Although there seems to be an important relationship between these two perceptual activities, this relationship should not be characterized as one of conflict. R. L. Gregory has provided perhaps the most extreme statement of the view that there is an inherent conflict in picture perception. According to Gregory, “pictures are paradoxes” in that looking at a picture involves the simultaneous awareness of a surface and a three dimensional scene: “no object can be in two places at the same time; no object can lie in both two and three dimensional space. Yet pictures are both visibly flat and three dimensional … Pictures are impossible.”4 For Gregory, the paradox that pictures pose to the visual system is not unlike other more obvious visual paradoxes: for instance, the inconsistent spatial features of M. C. Escher's work, or the “impossible figures” that psychologists have created in order to conduct experiments. In fact, Gregory's description of the effects of Schuster's three pronged ambiguous figure echoes his comments regarding the paradoxical nature of pictures in general: “Is there a middle prong? If so, does it follow the same position in depth as the outer two, or does it dip down below them? It seems, in a curious way, to do both at the same time. But that is impossible for one object, or one part of an object cannot exist in two places at the same time.”5 All pictures of three dimensional scenes, then, are allegedly paradoxical or impossible in the very same sense that Schuster's three pronged figure is paradoxical. In both cases, the visual system is presented with an object that possesses two mutually inconsistent spatial properties. Pictures of impossible figures are simply a special case of the paradox that all pictures represent—in such instances we are faced with an additional perceptual paradox. The ordinary pictures that we commonly encounter in our day to day lives may be more frequent and cause us less confusion, but they are no less paradoxical. Gregory's contention that pictures present the visual system with inconsistent impressions of space continues to have currency in current literature on picture perception. For instance, Reinhard Niederée and Dieter Heyer argue that the spatial duality of pictures represents a fundamental challenge to popular accounts of visual perception.6 The problem, as they see it, is that these popular accounts require that for any given stimulus, the visual system must produce “a single consistent scene description.”7 However, they argue that when we look at a picture, we perceive both a two dimensional surface and a three dimensional scene at the same time. Consequently, no single consistent scene description is possible, “for, how could a flat opaque surface on a wall and a transparent opening in that wall with a scene being visible through it coexist at the same location?”8 Similarly, Rainer Mausfeld suggests that the dual character of pictures consists in the fact that they “can generate an in depth spatial impression of the scene depicted while at the same time appearing as flat two dimensional surfaces hanging on a wall.”9 Mausfeld, like Niederée and Heyer, accepts that there is a perceptual conflict here, in the sense that “we seem to have two mutually incompatible spatial representations at the same time.”10 These authors are unlike Gregory in that they reject the notion that the perceptual conflict inherent in picture perception makes pictures somehow unique or peculiar, and they disagree as to how the visual system handles this perceptual conflict. However, there is fundamental agreement among these authors that perceiving a picture's surface and perceiving pictorial space constitutes a contradiction, and that the fact that we are able to do both simultaneously requires some specific explanation. This notion that looking at a picture involves mutually incompatible impressions of space may seem intuitively plausible. After all, it is quite true that “no object can lie in both two and three dimensional space,” as Gregory says. However, when one examines this claim closely, there appears to be an equivocation with respect to the meaning of “space.” In order for there to be a contradiction inherent in picture perception, we would have to see pictures as both two dimensional and three dimensional in the same sense. However, it seems clear that when we say that a picture looks two dimensional, and also looks three dimensional, we are talking about two very different kinds of space. The obvious difference between looking out a window and looking “into” a picture is that the space in the first case looks like the sort we could move around in, whereas the space in the second case does not. When we look at a picture we may be able to make judgments about whether certain actions could hypothetically be performed within the depicted scene; however, we obviously do not think we could perform any such actions within the pictorial space.11 The fact that pictorial space is distinct from real or environmental space and is perceived as such is not something that those who find a contradiction in picture perception would deny. For example, Niederée and Heyer point out that the “planar aspect” of a picture “is experienced as belonging to ‘real space,’” while the “spatial aspect” is not.12 Similarly, Mausfeld comments that while a line drawing of a cube would be seen as a three dimensional object, “it would hardly fool us into trying to grasp and rotate the cube.”13 Although these writers recognize the distinction between real and pictorial space, they do not recognize the full significance of the distinction; once it is granted that real and pictorial space are distinct, the supposed contradiction inherent in picture perception disappears. There is simply no contradiction or conflict in an object appearing to take up two dimensions in real space while also appearing to take up three dimensions in a pictorial or imaginary space. And, of course, there is not a single “object” being perceived as having two distinct spatial characteristics—strictly speaking, an artist does not “make the picture plane appear largely three dimensional,” as Gregory suggests.14 Even if we were to say that certain “segments” of a picture's surface appear to be related to one another both two and three dimensionally, there is still no contradiction so long as the relevant segments are perceived as related two dimensionally in real space, and three dimensionally in pictorial space (which is obviously the case for most pictures, since if the segments making up the depicted objects were perceived to be related three dimensionally in real space, we would experience a full blown illusion of depth and might be tempted to grasp a drawing of a cube). The apparent contradiction between perceiving a picture's surface and perceiving the three dimensional scene it depicts, then, is not a contradiction at all because the surface and the depicted scene do not occupy the same sort of space. That is, the picture is both two dimensional and not two dimensional, but not in the same sense. Thus, we have a ready answer to Niederée and Heyer's question regarding how a flat opaque surface and transparent opening through which a three dimensional scene is visible can coexist at the same location: the two are not perceived as occupying the same location. That the surface and the scene are not perceived as occupying the same space is precisely why looking at pictures generally does not produce the kind of confusion that we experience when looking at pictures of impossible figures (a point that Gregory largely ignores). When we look at Schuster's three pronged figure, for example, we are confused by the fact that the middle prong seems to exist simultaneously in two locations of pictorial space because such a state of affairs represents a paradox. But the fact that we can see the depiction of the three pronged figure as three dimensional while simultaneously perceiving the picture's surface does not confuse us because the figure is not perceived as having three dimensions in real space. Yet, it might still be objected that, in fact, we have direct experience of the incompatibility of our perception of the surface and the depiction. Mausfeld, for instance, suggests that part of the reason for claiming that we “have two mutually incompatible spatial representations” when we look at a picture is that these representations “are available internally and we can, without any effort, switch back and forth between them.”15 This contention that the conflict between a picture's surface and the scene it depicts is expressed through phenomenal “switches” between the perceptions of each is not unique to Mausfeld. In fact, E. H. Gombrich believed that the conflict between perceiving a picture's surface and perceiving what it depicts is so great that he insisted it is impossible to perceive both simultaneously. Referring to a picture of a battle horse, Gombrich claims it is impossible “to ‘see’ both the plane surface and the battle horse at the same time … To understand the battle horse is for a moment to disregard the plane surface. We cannot have it both ways.”16 Nor must one accept Gombrich's rather extreme view in order to recognize a fundamental rivalry between our perception of a picture's surface and what it depicts. For instance, Patrick Maynard suggests that we should not deny the importance of the 2D/3D rivalry present in much pictorial depiction, since observers are able to shift their attention between the features of a depicted scene and the features of the medium, and because artists commonly exploit 2D/3D perceptual conflicts.17 However, while it is certainly true that when we look at a picture we can focus our attention almost exclusively on either the surface or the depicted scene, and that an artist has techniques by which to draw our attention to one or the other, these facts are simply not evidence of an inherent conflict between the perception of the two dimensional surface and the three dimensional scene. When we listen to an actor recite Shakespeare, we can focus our attention almost exclusively on either the rhythm or the meaning of Shakespeare's dialogue. Moreover, the actor has techniques by which to draw our attention to either the rhythm or the meaning as he or she sees fit. Similarly, when we listen to a jazz quartet we can focus our attention on any one member of the group as we please, excluding the others—and of course the band has techniques for drawing our attention to one member or another, during his or her solo. Clearly, there is no inherent contradiction or rivalry between perceiving the “rhythm” and “meaning” of dialogue, nor between hearing a saxophone and a piano; these activities are not “mutually incompatible,” but when we experience both simultaneously they can act as rivals for our attention. Consequently, we should not conclude from the fact that we can “switch” between perceiving a picture's surface and perceiving the three dimensional scene it depicts that somehow these perceptual activities are inherently incompatible. Many researchers in the field of picture perception would not fully accept Gregory's contention that pictures are paradoxes; however, much of the literature on the topic nonetheless assumes that seeing a three dimensional scene “in” a two dimensional surface involves a fundamental perceptual conflict. Most commonly, this conflict is explained in terms of conflicting depth cues. James E. Cutting and Manfredo Massironi, for example, contend that the spatial duality of pictures “is carried by ‘conflicting cues’…which are not particularly common in the real world.”18 This concept of conflicting cues assumes a specific picture of how we perceive spatial relations, where the visual system constructs its “percepts” based on a variety of different sources of information. For instance, Cutting and Peter M. Vishton identify nine sources of information regarding spatial layout: occlusion, relative size, relative density, height in the visual field, aerial perspective, motion perspective, convergence, accommodation, and binocular disparities.19 These various cues provide different kinds of information: for example, occlusion provides only ordinal information, not information about the amount of depth, whereas binocular disparities may provide an absolute metric for distance.20 Also, many of the cues vary with respect to their effectiveness at different distances: for example, occlusion has the same effectiveness at all distances, whereas the informativeness of binocular disparities decreases dramatically with distance.21 According to this view, the perception of spatial relations emerges from a complex and somewhat elastic process of combining the information communicated by different cues, depending on the weight of the cues at different distances. The most important point to consider with respect to the subject at hand is that sometimes these cues can communicate contradictory information. Under controlled laboratory conditions, researchers often intentionally create such conflicts in order to observe which cues have the most influence over the visual system.22 Such research has had a significant impact on the basic assumptions of those working in the field of picture perception. The popular account of picture perception is that when we look at a picture, “monocular” depth cues such as occlusion, relative size, relative density, height in the visual field, and aerial perspective tell us that we are looking at a three dimensional object or scene. However, at the same time, the information we get from cues such as accommodation and convergence, binocular disparities, and motion perspective tell us that we are looking at a two dimensional surface. Although there are disagreements with regard to how the visual system handles this contradictory information, perhaps the most common view is that “human vision involuntarily strikes some kind of compromise between the flatness of the picture surface and the relief due to monocular cues.”23 Such a claim is the common explanation for the often repeated that pictorial depth is when a picture is with a single from a and when Niederée and Heyer contend that when faced with the conflicting cues a picture our visual system both a two dimensional surface and a three dimensional scene which coexist and are somehow into a such however, there is fundamental agreement regarding the fact that looking at a picture involves the perception of conflicting cues that the visual system must in one way or another. like the that pictures are perceived as mutually incompatible spatial the notion that pictures present the visual system with conflicting cues is intuitively However, there is an problematic at we can that pictures present us with conflicting depth cues only so long as we that the same cues always communicate the same information, regardless of the context in which they are To such a view, we would have to that our visual system a single meaning to visual cue and the significance of any such cue any However, a position that the visual system the to to different or would be to say the It is common sense that we the same visual depending on our about our and there is simply no reason to deny that the same is true of our perception of To the that something is a with a certain the way it looks to us in fact, the way we look at When we a or at a we are to see pictures, and it is only to that such influence our of the visual cues we find in such Most if we that the same cues are always understood by the visual system as the same information, we will not be able to account for the perception of pictorial space at To for example, that when we look at a picture the visual system a compromise between the monocular cues a three dimensional scene and the information a flat surface, does not why we perceive the three dimensional scene in pictorial as to environmental space. compromise in such a case would be to the difference between the two of cues and perceive the depicted scene as occupying a real space in which a could move The very fact that we perceive pictorial space, then, that the same cues do not communicate the same information in We simply would not perceive pictorial space if the monocular cues in a picture not communicate different information to the visual system when perceived in combination with cues the flatness of the picture surface. the most explanation of picture perception is that when binocular disparities, accommodation, and tell us that we are looking at a flat surface, monocular cues such as occlusion and relative provide information regarding the spatial relations of depicted objects in pictorial space. If this is it that whether these monocular cues provide information about depicted objects in pictorial space, or objects in real space, on the information provided by other visual cues present in a given That is, we ought to conclude from the nature of picture perception that certain depth cues provide different information to the visual depending on the context in which they are According to this then, picture perception does not a conflict between depth cues at When we look at a picture there are not two of cues competing information about a single there is a certain combination of depth cues that makes it for us to perceive depicted objects related three dimensionally in pictorial space. Moreover, not only do these suggest that picture perception is not inherently they suggest that seeing depth in pictures is not a particularly special perceptual When we look at a picture, a certain combination of cues a certain the impression of a flat surface in real space depicting a three dimensional scene in pictorial is a process not fundamentally different from looking at things like trees, chairs, and people. In other the perception of pictorial space is in much the same way that the perception of real space is Given that the monocular depth cues provide information about objects in pictorial space, it is rather to see where a conflict between cues could be In fact, this with the very concept of “conflicting has not by For example, J. and J. about the the observers are always simultaneously of both the space that the as a object and the pictorial space that as a cue conflict is perhaps an as it suggests that different cues are in conflict with respect to the of a single Yet, and seem to the of in order to account for the fact that pictorial depth is when those cues the picture surface are This of depth is, of course, important and should not be The of this that we should not claim that our visual system simply ignores the surface of a picture when we make about spatial relations between depicted as Cutting seems to However, can we say that monocular the perception of pictorial depth by conflicting information: if we accept that there is a conflict between cues in picture perception, we might be able to why space is when we look at a picture ordinary conditions, but we will our to how the perception of pictorial space is in the first Thus, is not an in the context of picture perception it is a and Given the the most that can be said is that our perception of a picture's surface has some influence on our perception of the three dimensional scene or object that the picture depicts. In other all we is that different of depth cues produce different However, there is no reason to that the perception of a picture's surface the perception of a depicted scene by the visual system with contradictory fact, there are to that there is no inherent conflict in picture perception at

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.