New sensor technology, developed by Mentech Innovation and awarded the CZ healthcare prize helps to identify misunderstood behavior.
Bo Sanne is a spontaneous girl with the intellectual level of a 2-year-old. Bo Sanne lives at Severinus in Veldhoven and receives support there 24 hours a day.
Sometimes Bo Sanne seems to get angry out of nowhere, she can injure herself and her surroundings. If her care environment could cope with signals of stress build-up earlier, outbursts of anger could in many cases be prevented. This can be done, for example, by responding more adequately and earlier to her request for help, by offering her a buddy or by removing her from a stressful situation.
Many people with an intellectual disability are often insufficiently able to express their emotions, pain or stress build-up and therefore often experience unnecessary moments of stress. Because their behavior is not well understood, health care providers cannot always provide the required attention and meaningful care. It is important to better understand difficult-to-understand behavior for self-directedness of this target group, for giving meaningful care to increase the quality of life and happiness of this vulnerable target group. In addition to fewer escalations, better understanding and care also leads to fewer dropouts among healthcare providers.
Since 2016, Mentech Innovation has been investigating in close collaboration with Severinus the use of smart sensor technology to interpret emotions, pain and tension in people with ununderstood behavior. Based on this research, Mentech Innovation has developed the HUME, an emotional artificial intelligence system that can measure stress and in the future emotions such as sadness, joy and pain in people with a servere intellectual disability or dementia. The system consists of a sensor module and a software platform with models for stress and emotion recognition. The HUME can, for example, be used as an early detection system for signaling stress build-up.
The HUME® measures, among other things, activity, temperature, heart rate and skin conductance. With the help of advanced models and behavioral observations, relationships are established between stress and physiology. This stress indication is currently being validated in a pilot study in which behavioral experts provide insight into the measured physiological parameters by means of video recordings. The results of these analyzes are made transparent through an app so that family and supervisors can “read” and learn to recognize stress. In a later stadium, this system can also be used preventively to prevent escalations by, for example, a traffic light function on the smartphone or smartwatch of the attendant.
Bo Sanne has been participating in the pilot for more than three-quarters of a year. Together with the care environment (remedial educationalists, care coordinators) a number of situations / moments were selected in which stress build-up was indicated on the one hand via video observations and signaling plans and on the other hand the physiology of Bo Sanne was measured with the HUME.
A typical result can be seen in the first graph, where the time-dependent heartbeat while watching a DVD of “Sleeping Beauty” . The presence of stress is determined via video observations and behavioral analyzes and is then indicated via a color coding (orange is stress, blue is relaxed, dark blue is physical stress) in the same graph. The graph clearly shows the relationship between increased heart rate (arousal) and stress build-up (orange markers) while watching exciting moments (such as the appearance of the witch).
Offering proximity leads to a much quieter pattern with much less severe stress build-up, see the second graph.
Mentech Innovation develops smart models (based on machine learning) for predicting behavior. These models are based on reference situations in which the physiological HUME measurements have been trained with test persons who are well able to indicate their emotional state. The application of such a trained model to Bo Sanne’s physiological measurements already leads to 60% predictability in the occurrence of stress build-up and arousal, see the third graph.
Further refinement of these models as well as the addition of additional parameters (such as sound recognition and environmental influences) are needed to further increase predictability. In addition, Mentech Innovation develops models and methodologies to make a distinction between positive and negative emotions (valence). The stress models will be further developed in the coming months, eventually to be able to introduce a validated HUME system in 2020.
Through the participation of Bo Sanne in the pilot study, we collect relevant data to validate the HUME. In addition, the intensive observations and analitics provide guidance with new insights into the causes of stress build-up and possible interventions to reduce or even prevent this build-up of stress.