UM Data Science Research Seminar with FASoS/FSE-TAIM

Schedule

LECTURE 1 (12:00 -12:20) - Q&A (12:20 - 12:30)

Speaker: Aashutosh Ganesh

Subject: "Memories in Our Mind's Eye: A Systematic Review of Video Memorability”

Abstract
Video memorability is the study of intrinsic properties that make certain videos more memorable than others. The complex, multimodal nature of video has led to the development of numerous datasets, computational models for memorability prediction, and downstream applications, including advertising. Despite substantial progress across these directions, a comprehensive synthesis of the literature remains lacking. This work conducts a systematic survey of video memorability research across three dimensions: (1) datasets (2) predictive models, (3) applications. We categorise contributed datasets according to variations in annotation protocols, the types of videos collected, and the availability of auxiliary annotations. We further classify feature representations using a taxonomy adapted from prior literature and provide an overview of modelling strategies, including state-of-the-art performance on each dataset. In addition, we survey current application domains that leverage memorability prediction. Our analysis identifies several critical gaps, including the need for large-scale long-term memorability datasets with robust annotations, the integration of explainable AI methods to better characterise memorability, and the exploration of emerging applications in advertising, film, and clinical applications.
 

LECTURE 2 (12:30 -12:50) - Q&A (12:50 - 13:00)

Speaker: Daniella Pauly Jensen

Subject: "Whose 'Mind's Eye' and Whose Memories? A Feminist STS Perspective on Video Memorability and AI for Media"

Abstract
Following Aashutosh’s review of video memorability research, my presentation shifts focus to the critical social dimensions embedded within AI for media. In this presentation, I use a feminist Science and Technology Studies (STS) lens to bridge technical innovation with societal insight. While the integration of memorability objectives into video-level generative models may be technically innovative, such a lens prompts questions about whose “mind's eye” and whose “memories” are being computationally modelled. I will show how choices in memorability modelling pipelines (e.g., clip sourcing, audio inclusion, annotation protocols, memorability scores) can encode assumptions about whose memories are modelled and whose perspectives are rendered visible. Examining the foundations of "memorability" metrics and the construction of datasets, allows us to begin to reveal how seemingly objective technical systems are shaped by situated human practices, potentially embedding biases related to culture, identity, and experience.

For this presentation, I also draw on ethnographic research within the data science department of Radio Television Luxembourg (RTL) and the Trustworthy AI for Media (TAIM) Lab that Aashutosh and I are a part of. I will introduce an exploratory, working framework of how I have seen “diversity” and “bias” be framed in computer science work, and map some of these to the video memorability work. I invite the audience to reflect on whether these categories resonate with their own experiences. With my presentation, I am to show how a feminist STS lens can offer a complementary perspective to enrich system design by offering valuable insights for designing more equitable and robust systems that move beyond a singular, universal "mind's eye" to embrace the rich plurality of human memory and experience in media.