Linguistics Scholarly Workshttps://hdl.handle.net/1808/7452024-03-28T16:12:48Z2024-03-28T16:12:48ZPerception and production of Mandarin-Accented English: The effect of degree of Accentedness on the Interlanguage Speech Intelligibility Benefit for Listeners (ISIB-L) and Talkers (ISIB-T)Fishero, SheyenneSereno, Joan A.Jongman, Allardhttps://hdl.handle.net/1808/347002023-08-10T06:06:47Z2023-06-17T00:00:00ZPerception and production of Mandarin-Accented English: The effect of degree of Accentedness on the Interlanguage Speech Intelligibility Benefit for Listeners (ISIB-L) and Talkers (ISIB-T)
Fishero, Sheyenne; Sereno, Joan A.; Jongman, Allard
Previous research on the Interlanguage Speech Intelligibility Benefit (ISIB) indicates nonnative listeners may have an advantage at understanding nonnative speech of talkers with the same first language (L1) due to shared interlanguage knowledge. The present study offers a comprehensive analysis of various factors that may modulate this advantage, including the proficiency of both the listeners and the talkers, the mapping of phonemes between the L1 and second language (L2), and the acoustic properties of the phones. Accuracy scores on a lexical decision task were used to investigate both native English listeners’ and native Mandarin learners’ of English perception of native English and Mandarin-accented English speech. Results show clear ISIB-L and ISIB-T effects and demonstrate the dynamic nature of ISIB effects, with both being modulated by speaker and listener proficiency. More striking ISIB effects typically occur at the most extreme ends of accentedness. Additionally, an advantage for common-phoneme over unique-phoneme words in nonnative speech was observed. While nonnative productions of common-phoneme words are more accurate than those of unique-phoneme words, for the most accented productions, nonnative listeners are faster to respond to these unique, often mispronounced, productions.
The nonnative listener advantage at perceiving nonnative speech depends on various factors, including listener proficiency, speaker proficiency, phoneme characteristics, and the acoustics of specific speech tokens.
2023-06-17T00:00:00ZPlain-to-clear speech video conversion for enhanced intelligibilitySachdeva, ShubamRuan, HaoyaoHamarneh, GhassanBehne, Dawn M.Jongman, AllardSereno, Joan A.Wang, Yuehttps://hdl.handle.net/1808/340782023-04-11T06:07:05Z2023-01-28T00:00:00ZPlain-to-clear speech video conversion for enhanced intelligibility
Sachdeva, Shubam; Ruan, Haoyao; Hamarneh, Ghassan; Behne, Dawn M.; Jongman, Allard; Sereno, Joan A.; Wang, Yue
Clearly articulated speech, relative to plain-style speech, has been shown to improve intelligibility. We examine if visible speech cues in video only can be systematically modified to enhance clear-speech visual features and improve intelligibility. We extract clear-speech visual features of English words varying in vowels produced by multiple male and female talkers. Via a frame-by-frame image-warping based video generation method with a controllable parameter (displacement factor), we apply the extracted clear-speech visual features to videos of plain speech to synthesize clear speech videos. We evaluate the generated videos using a robust, state of the art AI Lip Reader as well as human intelligibility testing. The contributions of this study are: (1) we successfully extract relevant visual cues for video modifications across speech styles, and have achieved enhanced intelligibility for AI; (2) this work suggests that universal talker-independent clear-speech features may be utilized to modify any talker’s visual speech style; (3) we introduce “displacement factor” as a way of systematically scaling the magnitude of displacement modifications between speech styles; and (4) the high definition generated videos make them ideal candidates for human-centric intelligibility and perceptual training studies.
2023-01-28T00:00:00ZObituary: Wendy Herd (1973-2020)Jongman, AllardSereno, Joan A.https://hdl.handle.net/1808/340612023-03-21T06:07:39Z2020-01-01T00:00:00ZObituary: Wendy Herd (1973-2020)
Jongman, Allard; Sereno, Joan A.
2020-01-01T00:00:00ZThe Effect of Lexicality, Frequency, and Markedness on Mandarin Tonal CategorizationYang, Tzu-HsuanJin, Shao-JieLu, Yu-Anhttps://hdl.handle.net/1808/336342022-10-28T08:01:47Z2022-07-22T00:00:00ZThe Effect of Lexicality, Frequency, and Markedness on Mandarin Tonal Categorization
Yang, Tzu-Hsuan; Jin, Shao-Jie; Lu, Yu-An
While the Ganong lexicality effect has been observed for phonemic and tonal categorization, the effects of frequency and markedness are less clear, especially in terms of tonal categorization. In this study, we use Mandarin Chinese to investigate the effects of lexicality, tone frequency and markedness. We examined Mandarin speakers’ tonal categorization of tokens on all possible tonal continua with one end being a word and the other being a tonotactic gap (i.e., an unattested syllable-tone combination). The results of a forced-choice identification experiment showed a general bias against the gap endpoints, with the noted exception of continua involving T4 (X51), the most frequent lexical tone. Specifically, when T4 served as the gap endpoint, no obvious bias against it was observed regardless of its lexical status. Moreover, on the T3–T4 continua, there was an apparent bias against T3 (X214), the tone with the most complex contour, again, regardless of lexicality, suggesting a strong markedness effect. Taken together, the results of this study show the individual effects of lexicality, tone frequency and markedness, as well as their interactions, which contribute to our understanding of tonal categorization in relation to lexical statistics (tone frequency) and phonology (markedness).
2022-07-22T00:00:00Z