Exploiting the speech-gesture link to capture fine-grained prosodic prominence impressions and listening strategies

Publication date: September 2019Source: Journal of Phonetics, Volume 76Author(s): Petra Wagner, Aleksandra Ćwiek, Barbara SamlowskiAbstractIn this paper, we explore the possibility to gather perceptual impressions of prosodic prominence by exploiting the strong prosody-gesture link, i.e., by having listeners transform a perceptual impression into a motor movement, namely drumming, for two domains of prominence: word-level and syllable-level. A feasibility study reveals that such a procedure is indeed easily and speedily mastered by naïve listeners, but more difficult for word-level prominences. We furthermore examine whether “drummed” annotations are comparable to those gathered with more established annotation protocols based on cumulative naïve impressions and fine-grained expert ratings. These comparisons reveal high correspondences across all prominence annotation protocols, thus corroborating the general usefulness of the gestural approach. The analyses also reveal that all annotation protocols are strongly driven by structural linguistic considerations. We then use Random Forest Models to investigate the relative impact of signal and structural cues to prominence annotations. We find that expert ratings of prosodic prominence are guided comparatively more by structural concerns than those of naïve annotators, that word-level annotations are influenced more by structural linguistic cues than syllable-level ones, and that “drummed” annotations are driven least...
Source: Journal of Phonetics - Category: Speech-Language Pathology Source Type: research