Success Story: Multimodal Prediction of Alexithymia from Physiological and Audio Signals

NCC Presenting the Success Story

The Cyprus NCC is led and coordinated by the Computation-based Science and Technology Research Center (CaSToRC), at the Cyprus Institute. CaSToRC pioneers the development of computational and large-scale data methodologies to advance scientific and technological disciplines. In parallel, it supports user communities in academia, industry and government in Cyprus to use HPC, advanced mathematical modelling, simulation, data science and HPDA for computational research and innovation.

Scientific Partners Involved:

The University of Cyprus (UCY) is a young and rapidly expanding university and a leading research organization in Cyprus. It has successfully implemented hundreds of research projects, receiving funding nationally as well as from >100 EU programmes, 66 MSCA, H2020, COST and Teaming. UCY has substantial experience in managing and supporting research programmes, with numerous scientific publications and presentations in international conferences reflecting UCY research record.

Technical/scientific Challenge:

Alexithymia is a trait that reflects a person’s difficulty in recognizing and expressing their emotions, which has been associated with various forms of mental illness. Identifying alexithymia can have therapeutic, preventive, and diagnostic benefits. However, there has been limited research on proposing predictive models for alexithymia, and literature on multimodal approaches is almost non-existent. Using machine learning tools can greatly enhance both.

Scientific Impact:

The resulting models are lightweight and can potentially be used in embedded devices for personalised monitoring.

At the same time, insights stemming from the experiments can help speed up the data collection experiments carried out in the Department of Psychology, by focusing on phases of the experiment that appear to be highly discriminative for Alexithymia – reducing the effort required to collect the data.

Finally, features were extracted from two novel datasets collected at the UCY, and the team is making steps towards making these data publicly available to speed-up research in this domain.

Solution:

Leveraging the expertise of UCY on this topic, and the wealth of multimodal data captured at the UCY laboratory (physiological signals, such as heart rate, skin conductance, facial electromyograms, and audio signals), the team developed fast and efficient techniques based on a set of discriminative temporal features, that capture spectral information in a localized manner (e.g., via wavelet transformations). In this way, simple Machine Learning classifiers achieve up to 95.7% F1-score – even when using data from only one of the 12 phases of the data collection experiment.

Two datasets of physiological signals are made available on request by following the provided link: https://github.com/CaSToRC-CyI/ACII2023

Benefits:

• Utilising simple, efficient machine learning algorithms with high F1-scores, optimised for low computational power, particularly for embedded devices.
• Enhancing research accessibility by providing a readily available alexithymia database (physiological signals) upon request, benefiting mental health research and clinical applications.
• Facilitating the design of more efficient experiments with reduced burden on subjects, by identifying experimental conditions that yield higher accuracy in condition detection, potentially minimising time and data collection sessions.