The human behaviour of a computer scientist
Bulat Khaertdinov is the first winner of the Faculty of Science and Engineering’s Dissertation Prize. At the Department of Advanced Computer Sciences, he trains artificial intelligence to recognise and respond to human behaviour. Read on and familiarise yourself with the work of a computer scientist.
For Bulat, like many of us, a day in the office is often a day in front of a computer screen. But there is a big difference in the nature of his work; while most of us simply use software like word processors, Bulat teaches computers how to recognise human behaviour and then take some intelligent action. “Working with computers is fun and quite interesting, especially if you like it as much as I do.”
As a student, Bulat’s interest went out to physics. “When I studied Physics and Information Security, I quickly realised that I did not like the physics part much. Luckily, I got the chance to focus my bachelor’s thesis on artificial intelligence. It is super fascinating to teach computers to make their own decisions based on raw sensory data. Similar to what we humans do, although they are still not as good as us.”
FSE Dissertation Prize
In 2024 the Faculty of Science and Engineering for the first time awarded the annual Dissertation Prize. Out of all the excellent PhD students, three nominees were chosen to compete for the win (in random order):
Balazs Erdos, MaCSBIO
‘Quantifying Metabolic Health from High-Dimensional Post-Meal Dynamics’
Rocio Arreguin Campos, Sensor Engineering
Bulat Khaertdinov, DACS
‘Feature Representation Learning for Human Activity Recognition’
The jury was very impressed by the high quality of all three dissertations. Bulat's thesis stood out because of the high quality of the writing style and the clear communication style. His thesis was very well put together. In addition, three publications have emerged from the master’s thesis he supervised.
Fascination
After finishing his studies, Bulat took up a position as a PhD student at Maastricht University. At the Department of Advanced Computer Sciences, he started working within the research theme Human Behaviour Understanding. “As a PhD student, I focused on analysing the data that humans generate, which we capture through various sensors. For example, we can measure heart rate, detect sweat on the palms, and track movements from devices like smartphones, smartwatches, virtual reality headsets, or their controllers."
Bulat’s PhD research showed how this data can be analysed; now, as a postdoc, he is using his knowledge to detect human emotions. “We use artificial intelligence to personalise the virtual reality training based upon the users’ emotional state. For instance, if a user is frustrated during a training session, the system can lower the difficulty or even step back to a previous level. If the user appears bored, we can increase the challenge.”
It is super fascinating to teach computers to make their own decisions based on raw sensory data. Similar to what we humans do, although they are still not as good as us.
Bulat KhaertdinovContext
Bulat makes detecting human behaviour and emotions sound easy; just a few sensors, and it is fixed. But it is not! Your smartphone can probably detect when you walk and calculate your speed and walking distance. “This is likely a conventional (non-AI) algorithm that reacts to the movement of your device or smartphone”, Bulat explains. For instance, if you are cycling with a phone in your pocket, your legs move in a comparable way; nevertheless, the algorithm cannot figure out that you are on a bike. The movements are similar, but the context is different. Making AI understand the influence of context is one of the hardest challenges AI researchers in human behaviour analysis face.
Let’s take a look at the VR training again. If the AI detects frustration, it adjusts the training difficulty. But was it indeed the training that caused the frustration? Maybe the person was annoyed by a noise in the room? “We can try to contextualise decision-making of our models as much as possible through additional sources of information, or modalities, like the noise, temperature, and humidity in the room, to help the AI make more accurate decisions", Bulat says.
Emotions
Being a computer scientist is not just about sitting in front of a computer screen. Of course, Bulat convenes with his colleagues. Occasionally, they gather the data they need to train the artificial intelligence systems. Late last year, they experimented with human volunteers. The persons played VR games while the researchers measured their data and later asked the volunteers how they felt during the training. Frustrated, engaged, or bored?
Without a doubt, during his research, Bulat will have felt bored, engaged and frustrated; after all, he is human. But in the end, it all was worth it. "I was truly surprised when I was nominated for the FSE Dissertation Prize, and I felt incredibly honoured when I won it," he says.
