2024

HEAR ME: AI-Based Smart Personal Assistant for People with Hearing and Speech Disabilities

Entrant Company

HMW Design Studio

Category

Innovation in Design - Social Impact Design

Client's Name

Self-Initiated

Country / Region

United States

For deaf and non-vocal people, connecting with the world can be deeply frustrating. But it doesn't have to be. HEAR ME is an app that uses AI technology and mindful interaction design to provide 360-degree support for the deaf and non-vocal community. It not only bridges their communication gap, but offers 24/7 mood tracking and AI avatar-based personal assistance as well as emotional support.

Not every person understands or can use sign language, and not every video/audio content on the internet is paired with sign language and subtitles. Hence, the world is not "designed" for deaf and non-vocal people. Because of these communication gaps, deaf and non-vocal people experience countless frustrations every day, and their potentials are largely untapped. HEAR ME aims to transform that by leveraging AI technology and mindful design to provide holistic support for the deaf and non-vocal community, enabling them to better connect with the world, and live an easier, happier and more fulfilling life.

Its real-time AI avatar interpreter as well as local human interpreter features can help user translate between spoken and sign languages for a variety of scenarios ranging from face-to-face conversations, to video calls, movie/TV, etc. On top of that, it performs 24/7 mood tracking and avatar-based personal assistance service to enhance user's mental well-being.



In the nutshell, the app HEAR ME consists of 5 main features.  1, Real-time sign-language interpretation by AI avatar. 2, Explore human sign-language interpreters and make appointments. 3, Deaf and non-vocal community and social feature. 4, Heart-Rate-Variability based mood tracking through wearable devices and mood analysis. 5, AI-avatar personal assistant. Users can access these features on their mobile app, tablet, smart watch, and even XR headset.   Its intelligent AI avatar interpretation features will auto-activate when it detects conversation in the surroundings. Paired with a smart watch, HEAR ME tracks the emotional state of the user and provides proactive personal assistant support when the user is experiencing emotional stress.

Credits

Design Lead
Xiaobi Pan
Design Lead
Junru Xu
 
2024
2024 Yehliu Night Tours

Entrant Company

Modern Design Management Consultant Co.,Ltd.|BEAMY CO., LTD.

Category

Innovation in Campaign - Event Campaign

Country / Region

Taiwan

 
2024
Genesis - The New Dawn

Entrant Company

Lola Yiting Zhang

Category

Innovation in Design - Futuristic Design

Country / Region

United States

 
2024
Excellence in Payment Solutions

Entrant Company

Kumar Shanmugasamy

Category

Innovation in Technology - Banking Technology (New)

Country / Region

United States

 
2024
Enhancing Sales Forecasting Accuracy in CRM/ERP using AI

Entrant Company

Arun Gupta

Category

Innovation in Technology - Best Emerging Technology Innovation

Country / Region

United States