Decoding speech from brain activity

Technology that translates brain activity into speech could be transformative for patients deprived of it. Studies have reported impressive results when decoding brain activity in response to performed or perceived speech. However, to serve the needs of the speech impaired, one should aim for non-vocalized speech, also called imagined or inner speech. Despite intensive efforts, this remains an elusive challenge. Several observations can be made which we believe prevent progress. We propose a new multiway decoder that accounts for the nonlinear relation between brain recordings and speech features, that is fast to respond, and that facilitates the transfer of model knowledge across speech modes and subjects. We will then use the decoder, prior developed based on model transfer, to provide real-time audible feedback to the subject when attempting imagined speech, which we hypothesize to be crucial for the subject to master imagined speech production. When successful, this project can contribute to future developments aimed at improving the quality of life of speech-impaired patients.