Human-Robot Interaction
An interactive storytelling robot for early constructive childhood intervention.
In this project, we expound on the idea of leveraging the potential of storytelling through an interactive toy i.e. transitional object as a means of intentful intervention to help children understand and cope better with stressors in developmental ages.
Timeline
2020
Role
Interaction Designer, Computer Vision Developer
Background
Early life adversity is a major risk factor for the development of psychological and behavioural problems in adult life. Traumatic experiences in childhood are linked to higher rates of depression, anxiety disorders and a range of mental health issues. Additionally, stories form the basis of understanding in children and help develop empathy and cultivate imaginative and divergent thinking in them.
Research
Early life adversity is a major risk factor for the development of psychological and behavioural problems later in life. Higher rates of depression, suicidal tendencies, anxiety disorders, post-traumatic stress disorder, and aggressive behaviour have been reported in adults who experienced childhood maltreatment.
Traumatic childhood events also contribute to increased drug use and dependence. Initiation of drug-taking behaviour begins at a much younger age in those who’ve experienced childhood trauma. Exposure to stressful events in childhood can increase the impact of stressful events throughout life. Add divorce or unemployment to childhood trauma, and someone can be more likely to develop psychological disorders or addiction.
Through literature review and brainstorming, we tried to understand the various stressors (causes) that trigger different mental health issues.
Solution
Juno is a companion toy that encourages children to share their thoughts and stories. Juno understands the child’s mood and any erratic behaviors by analysing verbal and non verbal conversations using complex computer vision and machine learning models. Juno categorises the adversities faced by the child and pulls in a relevant story from a large collection to narrate to the child. The child feels motivated to fight his adversities, boosting self esteem and confidence. A companion app for parents keeps them well informed about the child’s wellbeing and suggests action items based on the child’s happiness indicator. The companion app suggests early intervention in case any critical adversities faced by the child.
Detecting Emotions
To create a low fidelity prototype of the robot we needed to detect the emotional status of the user. To capture the emotional status unobtrusively we decided to use computer vision and voice tone analysis. Existing machine learning models were leveraged through APIs for the same. We used a Javascript-based Facial Recognition system for identifying emotions from visual cues. IBM Watson Text to Speech and Tone analyser was used to identify emotions from speech.
Technologies Used : NodeJS, IBM Watson, FaceApiJS.
