Cross-modal Association between Auditory and Visuospatial Information in Mandarin Tone Perception in Noise by Native and Non-native Perceivers
dc.contributor.author | Hannah, Beverly | |
dc.contributor.author | Wang, Yue | |
dc.contributor.author | Jongman, Allard | |
dc.contributor.author | Sereno, Joan A. | |
dc.contributor.author | Cao, Jiguo | |
dc.contributor.author | Nie, Yunlong | |
dc.contributor.editor | Singh, Leher | |
dc.date.accessioned | 2018-11-12T22:23:51Z | |
dc.date.available | 2018-11-12T22:23:51Z | |
dc.date.issued | 2017-12-04 | |
dc.identifier.citation | Hannah B, Wang Y, Jongman A, Sereno JA, Cao J and Nie Y (2017) Cross-Modal Association between Auditory and Visuospatial Information in Mandarin Tone Perception in Noise by Native and Non-native Perceivers.Front. Psychol. 8:2051.doi: 10.3389/fpsyg.2017.02051 | en_US |
dc.identifier.uri | http://hdl.handle.net/1808/27307 | |
dc.description.abstract | Speech perception involves multiple input modalities. Research has indicated that perceivers establish cross-modal associations between auditory and visuospatial events to aid perception. Such intermodal relations can be particularly beneficial for speech development and learning, where infants and non-native perceivers need additional resources to acquire and process new sounds. This study examines how facial articulatory cues and co-speech hand gestures mimicking pitch contours in space affect non-native Mandarin tone perception. Native English as well as Mandarin perceivers identified tones embedded in noise with either congruent or incongruent Auditory-Facial (AF) and Auditory-FacialGestural (AFG) inputs. Native Mandarin results showed the expected ceiling-level performance in the congruent AF and AFG conditions. In the incongruent conditions, while AF identification was primarily auditory-based, AFG identification was partially based on gestures, demonstrating the use of gestures as valid cues in tone identification. The English perceivers’ performance was poor in the congruent AF condition, but improved significantly in AFG. While the incongruent AF identification showed some reliance on facial information, incongruent AFG identification relied more on gestural than auditory-facial information. These results indicate positive effects of facial and especially gestural input on non-native tone perception, suggesting that cross-modal (visuospatial) resources can be recruited to aid auditory perception when phonetic demands are high. The current findings may inform patterns of tone acquisition and development, suggesting how multi-modal speech enhancement principles may be applied to facilitate speech learning. | en_US |
dc.publisher | Frontiers Media | en_US |
dc.rights | Copyright © 2017 Hannah, Wang, Jongman, Sereno, Cao and Nie. This is an open access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms. | en_US |
dc.rights.uri | https://creativecommons.org/licenses/by/4.0/ | en_US |
dc.subject | Cross-modal association | en_US |
dc.subject | Gesture | en_US |
dc.subject | Audio-visual | en_US |
dc.subject | Tone perception | en_US |
dc.subject | Mandarin | en_US |
dc.subject | English | en_US |
dc.title | Cross-modal Association between Auditory and Visuospatial Information in Mandarin Tone Perception in Noise by Native and Non-native Perceivers | en_US |
dc.type | Article | en_US |
kusw.kuauthor | Jongman, Allard | |
kusw.kuauthor | Sereno, Joan A. | |
kusw.kudepartment | Linguistics | en_US |
kusw.oanotes | Per SHERPA/RoMEO 11/12/2018: Author's Pre-print: green tick author can archive pre-print (ie pre-refereeing) Author's Post-print: green tick author can archive post-print (ie final draft post-refereeing) Publisher's Version/PDF: green tick author can archive publisher's version/PDF General Conditions: On open access repositories Authors retain copyright Creative Commons Attribution License Published source must be acknowledged with citation First publication by Frontiers Media must be acknowledged Publisher's version/PDF may be used Articles are placed in PubMed Central immediately on behalf of authors. | en_US |
dc.identifier.doi | 10.3389/fpsyg.2017.02051 | en_US |
kusw.oaversion | Scholarly/refereed, publisher version | en_US |
kusw.oapolicy | This item meets KU Open Access policy criteria. | en_US |
kusw.proid | 147674511360 | en_US |
dc.rights.accessrights | openAccess | en_US |
Files in this item
This item appears in the following Collection(s)
Except where otherwise noted, this item's license is described as: Copyright © 2017 Hannah, Wang, Jongman, Sereno, Cao and Nie. This is an open access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.