Gesture In Linguistics Research Paper

1. What Is Gesture?

Though there is controversy as to how to classify nonverbal behaviors (Efron 1941, Ekman and Friesen 1969, McNeill 1992), the central cases of nonverbal behaviors that have been designated by the term ‘gesture’ can be captured by the following definition. Gesture is nonpractical, nonlinguistic discrete body movement that arises from the intention to express something, and that encodes some aspects of what is intended to be expressed and/or how the accompanying speech should be interpreted.

Gestures are discrete spurts of body movement, which occur in the background of a period of nongesturing. This is similar to verbal utterances occurring in the background of silence. The sporadic nature of gestures contrasts with the inherently continuous nature of other nonverbal semiotic codes such as posture, interpersonal spacing, and gaze.

Gesture is nonpractical in the sense that gestural body movement is not involved in achieving nonexpressive practical goals. For example, the way one holds a cigarette may be used intentionally to express a certain attitude or social affiliation, but it is not considered to be gesture because the body movement serves a practical purpose for smoking.

Gestures arise from the intention to express something. Gesture can be the intended code for transmission of the information. Raising a hand when taking a vote in a meeting is such a case. However, gesture can arise in the situation where gesture seems not to be the code intended for the recipient. For example, it is not uncommon to produce direction-indicating gestures as one gives directions on the telephone. In such cases, the gestural body movement is still caused by the intention to express the instruction for how to get to the destination.

In this respect, gesture contrasts with self-touch, such as rubbing the chin or fiddling with clothing, in which there is no apparent intention to express. Self-touch also appears in certain noncommunicative settings, for example, while solving a crossword puzzle alone. The presence of expressive intention also distinguishes gesture from bodily symptoms for certain emotional and physiological states, such as blushing. However, some of the affect-displays, such as frowning, can be produced under voluntary control. They can be intentionally produced to express certain affect. This type of behavior blurs the boundary between what is a gesture and what is not.

The issue of intentionality has raised both conceptual and methodological concerns in defining gesture. Many authors maintain that intentionality is an essential ingredient of gesture (Ekman and Friesen 1969, Kendon 1981). However, difficult questions arise once one tries to tackle the issue squarely. Does intentionality have binary states (presence vs. absence) or is it a matter of degree? How can one tell what is intended to be expressed? How can one tell whether a given act is intentional, or how can one assess the degree of intentionality? Kendon (1981) suggested that a solution to the problem of identifying intentionality may be to focus on whether communicational partners treat each other’s acts as intentional or unintentional. However, this does not solve the problem for gestures that cannot be seen by the communicational partners, as in the case of gesturing during telephone conversation.

Going back to further definitional features of gesture, gesture encodes some aspects of what is intended to be expressed and/or how the accompanying speech should be interpreted. In other words, gestural body movement itself is the carrier of the information to be communicated. In this respect, gesture differs from ‘placing,’ in which the body moves an object to a meaningful position (Clark in press). For example, putting merchandise on the cashier’s counter signals the intention to purchase the merchandise. In placing, the body movement brings about a meaningful spatial configuration, but in gesture, the body movement itself carries the meaning.

2. In What Way Is Gesture Nonlinguistic?

One of the defining features of gesture is its nonlinguistic nature. Gesture is distinct from sign language, which is language that transmits codes in the visuo-motor modality. It bears different semiotic properties from language (Kendon 1993, McNeill 1992). Gestures vary as to the degree to which they are semiotically different from language.

2.1 Most Language-Like Gestures—Emblems

Gestures that are the most language-like are ‘emblems’ (Efron 1941, Ekman and Friesen 1969) such as the Euro–American OK sign with a ring formed by the thumb and the index finger. Emblems share some properties with words in a language. Emblems have standards of form. Their form-meaning relationship shows a relatively high degree of arbitrariness. Standards of form and form-meaning pairing are shared by a community of users (Calbris 1990, Morris et al. 1979).

Despite the above similarities to words, emblems also exhibit important differences. First of all, emblems are extremely limited in their potential for sequential combination, that is, combinatorial rules that operate over multiple emblems, namely, something equivalent to syntax, are impoverished.

Furthermore, Sparhawk (1978) showed that Persian emblems are not built up on the basis of a unified system of contrasting formational features, namely, something equivalent to phonology. It was found that the palm facing up vs. down is a formational feature that contrasts many Persian emblems. However, for the emblems with a certain class of meaning, namely beckoning emblems (e.g., the ‘come here’ emblem), this feature is not contrastive, and thus the palm can be facing either up or down. This is because the feature is considered to be irrelevant for the iconic make-up of the emblems.

The conclusion that emblems are not governed by unified ‘phonology’ has an important consequence for the ‘Kinesics’ approach to the analysis of semiotic body movements (Birdwhistell 1970), which is inspired by structural linguistics. In this approach, the existence of a system of ‘kinemes,’ the gestural equivalent of phonemes, is a fundamental assumption. However, even in the most language-like class of gestures, ‘kinemes’ do not constitute a unified system.

In summary, emblems differ from words in a language in that emblems are not fully unified by formal principles that relate emblems with one another. In other words, formal systems that are equivalent to syntax and phonology organize emblems only to a very limited extent.

2.2 Least Language-Like Gestures—Gesticulations

Gestures that are least language-like are called ‘gesticulations’ (Kendon 1993) or simply ‘gestures’ (McNeill 1992), which spontaneously accompany speech, and usually share semantic and/or discourse-pragmatic contents with the concurrent speech.

Gesticulation consists of two subtypes: ‘beats’ and ‘representational gestures.’ Beats are small rhythmic bidirectional movements, in which hands typically move up and down or flip out and back. They occur when the concurrent speech should be interpreted in a context different from the ordinary flow of discourse (e.g., speech is being repaired, or speech is expressing the beginning of a new discourse unit) (McNeill 1992). Representational gestures express location, direction, action, movement of objects, and shape of object, by means of iconicity (i.e., resemblance between the gestural shape and the referent) and/or spatial indexicality (i.e., the spatial contiguity between gesture and the referent). Representational gestures can be metaphorical in the sense that they can spatialize abstract ideas (Calbris 1990, McNeill 1992).

Gesticulations share fewer semiotic properties with language than emblems. The form of gesticulations is less constrained by conventionalized standards. The iconic representation in a gesticulation is largely idiosyncratic moment-by-moment creation. Thus, gesticulations are flexibly shaped in accordance with the variation in the spatial or actional characteristics of the referent. A sequence of gesticulations can comprise a larger meaning unit only by virtue of iconicity, as a multigesture depiction of a ‘picture’ of the referent(s).

3. In What Ways Does Gesture Ha E An Essential Connection To Language?

In the previous section, the difference between gesture and language as semiotic systems was highlighted. Gesture, however, has essential connections to interactional, conceptual, and psychological aspects of language.

3.1 Gesture In Face-To-Face Conversation

Gestures contribute to face-to-face conversation in various ways. First, gesture can express some aspects of the content to be conveyed, which may or may not be expressed in the concurrent speech, and gesturally expressed information is taken up by the interlocutor (Kendon 1994). The interlocutor’s attention to gesture is also evidenced by the fact that gestures elicit verbal acknowledgement like ‘uh huh,’ and nonverbal acknowledgement like nodding from the interlocutor (Goodwin and Goodwin 1986, Streeck 1994).

This leads to the second way in which gestures contribute to conversation. Gesture is a part of complex cues as to when nonspeaking interlocutors can jump in and take the speaking turn (Schegloff 1984, Goodwin and Goodwin 1986, Streeck and Hartge 1992). One of the relevant features of gestures is that they are initiated slightly prior to the portion of speech that expresses the related content, and their temporal extent typically overlaps with the co-expressive speech (Schegloff 1984, Morrel-Samuels and Krauss 1992, McNeill 1992). The temporal precedence of gesture projects that the speaker is going to deliver speech further, and this can preempt the interlocutor’s attempt to take the turn.

The third way in which gestures contribute to conversation is to mark discourse-pragmatic status of the concurrent speech. Kendon (1995) reports Southern Italian emblems that indicate discourse pragmatic status of the concurrent speech. More specifically, they mark the topic-comment structure and the illocutionary force (e.g., demanding information, making a plea for understanding) of an utterance.

3.2 Gesture And Deixis

Gesture plays a crucial role in spatial deixis, in which the attention of the interlocutor is directed to the direction along which the referent (or the physical clue to the referent) can be found.

Gesture also plays a crucial role in ‘abstract deixis,’ in which a pointing gesture creates a referent in seemingly empty space in front of the speaker (McNeill et al. 1993). The speaker can point repeatedly to a locus in the empty space, as the same entity is referred to in the concurrent speech. The referent that such gestures spatialize can be something that could exist in space (e.g., a person who is not present), or something inherently nonspatial (e.g., time, plot-line of a story). Thus, the role of gesture in deixis is not only a passive one, in which a gesture is oriented toward a preexisting target, but also an active and creative one, in which the act of gesturing materializes conceptualization of the speaker as a spatial entity in the extralinguistic context.

3.3 Gesture And Speech Production

Gesture and speech are produced in close temporal and informational coordination. As noted in Sect. 3.1, a gesture typically is initiated prior to the co-expressive portion of speech, and the gesture and the coexpressive speech temporally overlap. In addition, the production of gesture has a systematic relationship to intonational units. A gesture rarely crosses tone unit boundaries, and typically, there is one gesture per tone unit (Kendon 1980, McClave 1991). These findings suggest that there is a mental process that coordinates speech and gesture production (Kendon 1980, MorrelSamuels and Krauss 1992, McNeill 1992).

3.3.1 Gesture And Ideophones. The exact meeting point of gesture and speech production processes is a highly debated issue (see McNeill 2000). One phenomenon that sheds some light on this issue is the fact that linguistic entities that have imagistic and emotive meaning are highly associated with iconic representational gesture. One such entity is the class of words called ‘ideophones’ (also referred to as ‘mimetics’). Ideophones are sound-symbolic words, for which native speakers report vivid imagery and emotion as their meaning. They include onomatopoetic words such as animal cries, but their referential domains go far beyond sound emitting entities (e.g., color, smell, wetness, density, emotion, and manner of motion).

Japanese is one of the languages in which ideophones comprise a large word class and they are commonly used. It was found that when Japanese speakers used an ideophone in narrative, it was almost always accompanied by a co-expressive iconic gesture (Kita 1997). This suggests that iconic gestures and ideophones are produced from common underlying mental representation that is imagistic in nature. In other words, iconic gestures are produced from imagistic representation that is evoked in the speaker’s mind.

3.3.2 Gesture And Psycholinguistics. Gestures that spontaneously accompany speech can be used as a

‘window’ into speech production processes, through which one can gain insights into the speaker’s on-line conceptualization (McNeill 1992). In such a line of inquiry, representational gestures have been in the focus. Since the shape of representational gestures is not completely determined by convention, there are ample degrees of freedom left for idiosyncratic expression of the speaker’s thought.

Representational gestures are created on-line by the speaker, and are typically co-expressive with the concurrent speech in the sense that they express the same object, action, event, or spatial relation as the speech, albeit from a different perspective and with a different mode of expression. As discussed above, the form of a representational gesture and its meaning are mediated by iconicity and spatial indexicality. Furthermore, ideophones, whose semantics is inherently imagistic, are almost always produced with a coexpressive iconic gesture. In short, representational gesture is imagistic, idiosyncratic, and non-linguistic; in contrast, speech is mainly propositional, and based on a conventionalized system of signs.

It has been argued that the underlying mental representation for a gesture-speech complex is produced by a single mental process involved in the verbalization of thought (Kendon 1980, McNeill 1992). McNeill stresses the significance of the semiotic duality of a gesture-speech complex. He argues that when the speaker tries to verbalize thought, he or she is engaged in two kinds of thinking: imagistic thinking and linguistic thinking. When the speaker finds a connection between idiosyncratic imagery and socially-constituted linguistic signs, a mental seed from which a gesture-speech complex develops is formed. In this process, imagery is grounded in a socially shared linguistic system, and words are connected to a specific image in the speaker’s mind. According to McNeill, the formation of a unified representation that integrates idiosyncratic and social sign systems is a crucial initial step in the verbalization of thought.

It has been suggested that gestures are not only the reflection of the speaker’s thought, but they also facilitate verbalization of thought. There is controversy as to the exact nature of the facilitation (see McNeill 2000). Some argue that representational gestures facilitate retrieval of lexical items from the mental lexicon (Morrel-Samuels and Krauss 1992), but others argue that conceptual planning for speaking is facilitated (Alibali et al. 2000).

4. Conclusion

Gesture is a semiotic system distinct from language. At the same time, it is intimately linked to various aspects of linguistic performance. Gesture is an integral part of the organization of face-to-face conversation. Pointing gesture is an essential part of referential acts by means of spatial deixis. The production of intonation and ideophones are tightly coupled with the production of a gesture that is coexpressive. The speaker’s imagistic thinking during verbalization of thought is revealed by gestures that spontaneously accompany speech. Because of the interactional, referential, and psychological unity of gesture and language, the understanding of language is not complete unless gesture is taken into account.

Bibliography:

1. Alibali M W, Kita S, Young A 2000 Gesture and the process of speech production: We think therefore we gesture. Language and Cognitive Processes 15: 593–613
2. Birdwhistell R L 1970 Kinesics and Context: Essays on Body Motion Communication. University of Pennsylvania Press, Philadelphia, PA
3. Calbris G 1990 The Semiotics of French Gestures. Indiana University Press, Bloomington, IN
4. Clark H in press Pointing and placing. In: Kita S (ed.) Pointing: Where Language, Cognition, and Culture Meet. L. Erlbaum Associates, Mahwah, NJ
5. Efron D 1941 Gesture and Environment. King’s Crown Press, New York
6. Ekman P, Friesen W V 1969 The repertoire of nonverbal behavioral categories: Origins, usage and coding. Semiotica 1: 49–98
7. Goodwin M H, Goodwin C 1986 Gesture and coparticipation in the activity of searching for a word. Semiotica 62(1 2): 51–75
8. Kendon A 1980 Gesticulation and speech: Two aspects of the process of utterance. In: Key M R (ed.) The Relation between Verbal and Nonverbal Communication. Mouton, The Hague
9. Kendon A 1981 Introduction: Current issues in the study of ‘nonverbal communication.’ In: Kendon A, Sebeek T A, Umiker-Sebeok J (eds.) Nonverbal Communication, Interaction, and Gesture: Selections from Semiotica. Mouton, The Hague
10. Kendon A 1993 Human gesture. In: Gibson K R, Ingold T (eds.) Tools, Language and Cognition in Human Evolution. Cambridge University Press, Cambridge, UK
11. Kendon A 1994 Do gestures communicate? A review. Research in Language and Social Interaction 27: 175–200
12. Kendon A 1995 Gestures as illocutionary and discourse structure markers in Southern Italian conversation. Journal of Pragmatics 23: 247–79
13. Kita S 1997 Two-dimensional semantic analysis of Japanese mimetics. Linguistics 35: 379– 415
14. McClave E 1991 Intonation and Gesture. Unpublished dissertation, Georgetown University
15. McNeill D 1992 Hand and Mind: What Gestures Reveal About Thought. University of Chicago Press, Chicago
16. McNeill D (ed.) 2000 Language and Gesture. Cambridge University Press, Cambridge, UK
17. McNeill D, Cassell J, Levy E T 1993 Abstract deixis. Semiotica 95(1 2): 5–9
18. Morrel-Samuels P, Krauss R M 1992 Word familiarity predicts temporal asynchrony of hand gestures and speech. Journal of Experimental Psychology: Learning, Memory, and Cognition 18: 615–22
19. Morris D, Collett P, Marsh P, O’Shaughnessy M 1979 Gestures, Their Origins and Distribution. Stein and Day, New York
20. Schegloff E A 1984 On some gestures’ relation to talk. In: Atkinson J M, Heritage J (eds.) Structures of Social Action: Studies in Conversation Analysis. Cambridge University Press, Cambridge, UK
21. Sparhawk C M 1978 Contrastive-identificational features of Persian gesture. Semiotica 24: 49–86
22. Streeck J 1994 Gesture as communication II: The audience as coauthor. Research on Language and Social Interaction 27: 239–67
23. Streeck J, Hartge U 1992 Previews: Gestures at the transition place. In: Auer P, di Luzio A (eds.) The Contextualization of Language. J. Benjamins, Amsterdam