During the pandemic, the world has become even more remote, and people are finding themselves at a distance from each other. A reason for this can be that a family member is studying abroad or someone is in a lockdown due to COVID in the last few years. The contact with family members or friends will most likely be less. An example could also be a time difference. When someone is studying in a country far from their home country, there could be a significant time difference which can make it impossible to contact each other during each other’s day. The Eindhoven based company I-IKIGAI is developing a device that can share a person’s mood with another person.
Project description
The Feel device predicts the mood based on sound, shares it with an family member or friend and visualizes it. The device can record audio, process audio (machine learning), share data and visualize the mood. These parts are divided under software, electrical and mechanical. The software part contains a server client connection using MQTT, The electronics focusses into the electronics necessary for the functioning of the Feel device (recording, processing and visualizing). The mechanical part consists of the enclosure which houses all the electronics
and provides cooling.
Project results
Our prototype Feel device is able to predict emotions from recorded audio fragments, using a Machine Learning (ML) model created with Edge Impulse, which is able to predict from 4 emotions (angry, happy, sad and neutral).
These emotions are converted to moods by taking the average emotion over a time period. The model has an overall validation accuracy of almost 76% with an accuracy of: Angry 92%, Sad 60.6%, Neutral 87.2%, and Happy 27.3% respectively for each emotion.
The neutral emotion plays an important role as most of the sound recordings captured day to day will fall in the neutral category. The accuracy of 87.2% for neutral also includes background sounds (living room and kitchen sounds) as these will have a big impact on predictions.
Video
Acknowledgement
The student project team would like to thank the company I-IKIGAI for their extensive support during the project.