Can a brain scan show what you've heard?

A brain scan that allows you to see what sound a person has heard. Researchers from Maastricht University have recently achieved a world first by reconstructing heard sound based on a person’s brain activity. 'A better understanding of how the brain registers and filters sounds will also enable us to filter out background noise in hearing aids in the future', says research leader Elia Formisano. The results were recently published in the leading scientific journal PNAS.

Reconstruction of heard sound

The researchers from Maastricht University researched how we listen in our everyday lives, by measuring, with the help of MRI scans, how our brain responds to a wide range of sounds, including speech, music and environmental noise. With the help of mathematical models and machine learning, the researchers were able for the first time to use responses in the brain to determine what exactly a trial subject had heard. 'This successful reconstruction of sounds in the brain demonstrates that we can simulate actual brain activity effectively using applied mathematical models', says Formisano. 'And, as such, that we can use these models to predict brain function. A next step is therefore to see whether we can expand this research to include a combination of sound and image.'

Filter out background noise

Formisano’s research team found that sound frequencies that indicate speech or voices can be reconstructed more accurately than other sound components. 'This indicates that even general mechanisms for processing sound in the human brain are optimised for a highly detailed analysis of the sounds that are most relevant to human behaviour, such as speech and voices', explains Formisano. 'Maybe this is evolution at play once again, in that it is more important for people to be able to distinguish between speech and voices than between barking dogs.' The Maastricht-based researchers’ discovery is therefore relevant for manufacturers of hearing aids, for example. 'We can use these methods in new studies in order to specifically research how the brain filters sounds. A better understanding of how the brain registers and filters sounds will then also enable us to filter out background noise in hearing aids, for example. That would make a huge difference to a great many people.'

Multidisciplinary approach

The research team was a collaboration between scientists from the Maastricht Brain Imaging Center (MBIC) and the Maastricht Centre for Systems Biology (MaCSBio). 'Both MBIC and MaCSBio have scientists with expertise in a wide variety of fields: biologists, neuroscientists, mathematicians and physicists are all represented', says researcher Michelle Moerel. 'This multidisciplinary approach not only gives us access to the latest computational technologies, it also means that we have a strong biological basis for asking the right research questions and interpreting the results broadly. This may lead to new insights that might not be achieved within the confines of an individual research field.”