Article
On speech-hand synchrony during conversations in a virtual underground station (en)
* Presenting author
Abstract:
Lip movements or body movements provide additional visual information to speech in face-to-face communication in noisy environments. People increase gesture-speech synchrony in cognitively demanding tasks and here we test hypothesis whether it can be a compensation strategy playing an important role for successful communication. The present study investigates whether talkers increase speech and hand gesture synchrony in the presence of noise in a conversation setting by analyzing the coherence of the wavelet cross-spectrum. Speech and body motion were tracked during either a dyadic or a triadic conversation in a virtual underground station created by the real-time Simulated Open Field Environment. The four groups of three participants and two groups of two participants had a free conversation and an additional noise was optionally present at either 72 dB SPL (moderate) or 82 dB SPL (high). The preliminary analysis suggested increased coherence of hand movements and speech in the high noise level condition relative to no noise, however, the data collection is in progress. Although the increased speech-hand synchrony could be a side product of the effortful situation it could also serve to transmit the information more effectively to the listener, thus providing additional cues for better speech intelligibility.