Metaphor processing is supramodal semantic processing: The role of the bilateral lateral temporal regions in multimodal communication.

Metaphor processing is supramodal semantic processing: The role of the bilateral lateral temporal regions in multimodal communication.
Brain Lang. 2020 Feb 29;205:104772
Authors: Joue G, Boven L, Willmes K, Evola V, Demenescu LR, Hassemer J, Mittelberg I, Mathiak K, Schneider F, Habel U
Abstract
This paper presents an fMRI study on healthy adult understanding of metaphors in multimodal communication. We investigated metaphors expressed either only in coverbal gestures ("monomodal metaphors") or in speech with accompanying gestures ("multimodal metaphors"). Monomodal metaphoric gestures convey metaphoric information not expressed in the accompanying speech (e.g. saying the non-metaphoric utterance, "She felt bad" while dropping down the hand with palm facing up; here, the gesture alone indicates metaphoricity), whereas coverbal gestures in multimodal metaphors indicate metaphoricity redundant to the speech (e.g. saying the metaphoric utterance, "Her spirits fell" while dropping the hand with palm facing up). In other words, in monomodal metaphors, gestures add information not spoken, whereas the gestures in multimodal metaphors can be redundant to the spoken content. Understanding and integrating the information in each modality, here spoken and visual, is important in multimodal communication, but most prior studies have only considered multimodal metaphors where the gesture is redundant to what is spoken. Our participants watched au...

Source: Brain and Language - February 28, 2020 Category: Neurology Authors: Joue G, Boven L, Willmes K, Evola V, Demenescu LR, Hassemer J, Mittelberg I, Mathiak K, Schneider F, Habel U Tags: Brain Lang Source Type: research