KUKU

KU ScholarWorks

  • myKU
  • Email
  • Enroll & Pay
  • KU Directory
    • Login
    View Item 
    •   KU ScholarWorks
    • Speech-Language-Hearing: Sciences and Disorders
    • Speech-Language-Hearing Scholarly Works
    • View Item
    •   KU ScholarWorks
    • Speech-Language-Hearing: Sciences and Disorders
    • Speech-Language-Hearing Scholarly Works
    • View Item
    JavaScript is disabled for your browser. Some features of this site may not work without it.

    A Noninvasive Brain-Computer Interface for Real-Time Speech Synthesis: The Importance of Multimodal Feedback.

    Thumbnail
    View/Open
    File available after 04/01/2019 (1.032Mb)
    Issue Date
    2018-04
    Author
    Brumberg, Jonathan S.
    Pitt, Kevin M.
    Burnison, Jeremy Dean
    Publisher
    Institute of Electrical and Electronics Engineers
    Type
    Article
    Article Version
    Scholarly/refereed, author accepted manuscript
    Rights
    © 2018 IEEE
    Metadata
    Show full item record
    Abstract
    We conducted a study of a motor imagery brain-computer interface (BCI) using electroencephalography to continuously control a formant frequency speech synthesizer with instantaneous auditory and visual feedback. Over a three-session training period, sixteen participants learned to control the BCI for production of three vowel sounds (/ textipa i/ [heed], / textipa A/ [hot], and / textipa u/ [who'd]) and were split into three groups: those receiving unimodal auditory feedback of synthesized speech, those receiving unimodal visual feedback of formant frequencies, and those receiving multimodal, audio-visual (AV) feedback. Audio feedback was provided by a formant frequency artificial speech synthesizer, and visual feedback was given as a 2-D cursor on a graphical representation of the plane defined by the first two formant frequencies. We found that combined AV feedback led to the greatest performance in terms of percent accuracy, distance to target, and movement time to target compared with either unimodal feedback of auditory or visual information. These results indicate that performance is enhanced when multimodal feedback is meaningful for the BCI task goals, rather than as a generic biofeedback signal of BCI progress.
    URI
    http://hdl.handle.net/1808/26516
    DOI
    https://doi.org/10.1109/TNSRE.2018.2808425
    Collections
    • Speech-Language-Hearing Scholarly Works [81]
    Citation
    Brumberg, J. S., Pitt, K. M., & Burnison, J. D. (2018). A non-invasive brain-computer interface for real-time speech synthesis: The Importance of Multimodal Feedback. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 26(4), 874-881. DOI:10.1109/TNSRE.2018.2808425

    Items in KU ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.


    We want to hear from you! Please share your stories about how Open Access to this item benefits YOU.


    Contact KU ScholarWorks
    785-864-8983
    KU Libraries
    1425 Jayhawk Blvd
    Lawrence, KS 66045
    785-864-8983

    KU Libraries
    1425 Jayhawk Blvd
    Lawrence, KS 66045
    Image Credits
     

     

    Browse

    All of KU ScholarWorksCommunities & CollectionsThis Collection

    My Account

    LoginRegister

    Statistics

    View Usage Statistics

    Contact KU ScholarWorks
    785-864-8983
    KU Libraries
    1425 Jayhawk Blvd
    Lawrence, KS 66045
    785-864-8983

    KU Libraries
    1425 Jayhawk Blvd
    Lawrence, KS 66045
    Image Credits
     

     

    The University of Kansas
      Contact KU ScholarWorks
    Lawrence, KS | Maps
     
    • Academics
    • Admission
    • Alumni
    • Athletics
    • Campuses
    • Giving
    • Jobs

    The University of Kansas prohibits discrimination on the basis of race, color, ethnicity, religion, sex, national origin, age, ancestry, disability, status as a veteran, sexual orientation, marital status, parental status, gender identity, gender expression and genetic information in the University’s programs and activities. The following person has been designated to handle inquiries regarding the non-discrimination policies: Director of the Office of Institutional Opportunity and Access, IOA@ku.edu, 1246 W. Campus Road, Room 153A, Lawrence, KS, 66045, (785)864-6414, 711 TTY.

     Contact KU
    Lawrence, KS | Maps