Research Seminar with the Faculty of Psychology and Neuroscience

Schedule

LECTURE 1: 12:00 - 12:30

Speaker: Tim Dick (DACS)

Subject: "Disentangling Intermediate Representations in Sound-to-Event DNNs using Invertible Flow Models".

Abstract: 
Neural representations derived from fMRI responses to natural sounds within non-primary auditory cortical regions mirror those found in the intermediate layers of deep neural networks (DNNs) trained for sound recognition. However, the underlying characteristics of these representations remain elusive. In this study, we investigate the nature of these intermediate representations employing a disentangling invertible flow model. We recorded a novel dataset of natural sounds, designed to probe the hypothesis that sound-to-event DNNs encode distinct basic sound generation mechanisms (human actions) and source properties (object materials) independently within their intermediate layers. To simulate brain responses to these natural sounds, we utilized the layer-by-layer activation of a convolutional DNN (Yamnet), pre-trained to categorize sound spectrograms into semantic categories. Crucially, through systematic manipulations of the obtained latent representations using the disentangling invertible flow model, we demonstrate predictable effects in the DNN's output. This in silico demonstration offers a promising avenue for subsequent neuroscientific in vivo experimentation.

LECTURE 2: 12:30 - 13:00

Speaker: Maria de Araújo Vitória 

Subject: "Aud2Sem: How the brain transforms acoustic information of natural sounds into semantical representations".

Abstract: 
Deriving meaning from sounds is crucial for comprehending our environment and we do it effortlessly. When we hear a sound, the cochlea converts sound vibrations into neural signals that travel to the primary auditory and non-primary auditory cortex areas. From there, the information is transferred to the ventro-lateral prefrontal cortex and other areas. Throughout this network, acoustic and semantic processing occurs, yet the mechanisms by which acoustic information is transformed into meaningful semantic representations remain incompletely understood. By optimizing natural stimuli selection and incorporating AI-based models of auditory processing in the analyses of behavioral and neural responses to natural sounds , we aim to gain deeper insights into the acoustic-to-semantic transformations that underlie sound recognition.