2024

HEAR ME: AI-Based Smart Personal Assistant for People with Hearing and Speech Disabilities

Entrant Company

HMW Design Studio

Category

Innovation in Design - Social Impact Design

Client's Name

Self-Initiated

Country / Region

United States

For deaf and non-vocal people, connecting with the world can be deeply frustrating. But it doesn't have to be. HEAR ME is an app that uses AI technology and mindful interaction design to provide 360-degree support for the deaf and non-vocal community. It not only bridges their communication gap, but offers 24/7 mood tracking and AI avatar-based personal assistance as well as emotional support.

Not every person understands or can use sign language, and not every video/audio content on the internet is paired with sign language and subtitles. Hence, the world is not "designed" for deaf and non-vocal people. Because of these communication gaps, deaf and non-vocal people experience countless frustrations every day, and their potentials are largely untapped. HEAR ME aims to transform that by leveraging AI technology and mindful design to provide holistic support for the deaf and non-vocal community, enabling them to better connect with the world, and live an easier, happier and more fulfilling life.

Its real-time AI avatar interpreter as well as local human interpreter features can help user translate between spoken and sign languages for a variety of scenarios ranging from face-to-face conversations, to video calls, movie/TV, etc. On top of that, it performs 24/7 mood tracking and avatar-based personal assistance service to enhance user's mental well-being.



In the nutshell, the app HEAR ME consists of 5 main features.  1, Real-time sign-language interpretation by AI avatar. 2, Explore human sign-language interpreters and make appointments. 3, Deaf and non-vocal community and social feature. 4, Heart-Rate-Variability based mood tracking through wearable devices and mood analysis. 5, AI-avatar personal assistant. Users can access these features on their mobile app, tablet, smart watch, and even XR headset.   Its intelligent AI avatar interpretation features will auto-activate when it detects conversation in the surroundings. Paired with a smart watch, HEAR ME tracks the emotional state of the user and provides proactive personal assistant support when the user is experiencing emotional stress.

Credits

Design Lead
Xiaobi Pan
Design Lead
Junru Xu
 
2024
BYDZYNE.AI

Entrant Company

ByDzyne, INC.

Category

Innovation in Technology - Artificial Intelligence (AI)

Country / Region

United States

 
2024
A distributed recommendation system to generate insights and optimize oil production

Entrant Company

Abhay Dutt Paroha, SLB

Category

Innovation in Technology - Cloud Technology

Country / Region

United States

 
2024
EZVIZ RS20 Pro Robot Vacuum & Mop Combo

Entrant Company

EZVIZ

Category

Innovation in Technology - Smart Technology

Country / Region

China

 
2024
Adia Robotics

Entrant Company

Adia Robotics

Category

Innovation in Technology - Best Autonomous Technology Innovation

Country / Region

United States