HEAR ME: AI-Based Smart Personal Assistant for People with Hearing and Speech Disabilities

For deaf and non-vocal people, connecting with the world can be deeply frustrating. But it doesn't have to be. HEAR ME is an app that uses AI technology and mindful interaction design to provide 360-degree support for the deaf and non-vocal community. It not only bridges their communication gap, but offers 24/7 mood tracking and AI avatar-based personal assistance as well as emotional support.

Not every person understands or can use sign language, and not every video/audio content on the internet is paired with sign language and subtitles. Hence, the world is not "designed" for deaf and non-vocal people. Because of these communication gaps, deaf and non-vocal people experience countless frustrations every day, and their potentials are largely untapped. HEAR ME aims to transform that by leveraging AI technology and mindful design to provide holistic support for the deaf and non-vocal community, enabling them to better connect with the world, and live an easier, happier and more fulfilling life.

Its real-time AI avatar interpreter as well as local human interpreter features can help user translate between spoken and sign languages for a variety of scenarios ranging from face-to-face conversations, to video calls, movie/TV, etc. On top of that, it performs 24/7 mood tracking and avatar-based personal assistance service to enhance user's mental well-being.



In the nutshell, the app HEAR ME consists of 5 main features.  1, Real-time sign-language interpretation by AI avatar. 2, Explore human sign-language interpreters and make appointments. 3, Deaf and non-vocal community and social feature. 4, Heart-Rate-Variability based mood tracking through wearable devices and mood analysis. 5, AI-avatar personal assistant. Users can access these features on their mobile app, tablet, smart watch, and even XR headset.   Its intelligent AI avatar interpretation features will auto-activate when it detects conversation in the surroundings. Paired with a smart watch, HEAR ME tracks the emotional state of the user and provides proactive personal assistant support when the user is experiencing emotional stress.