07

07

/

/

07

07

CONSUMER TECHNOLOGY

CONSUMER TECHNOLOGY

·

·

2021

2021

A Siri That Understands Everyone

A Siri That Understands Everyone

Redesigning Siri’s voice recognition experience to better support diverse accents and multilingual users through inclusive research and personalized settings.

Redesigning Siri’s voice recognition experience to better support diverse accents and multilingual users through inclusive research and personalized settings.

Redesigning Siri’s voice recognition experience to better support diverse accents and multilingual users through inclusive research and personalized settings.

Inclusive Design

Inclusive Design

Voice Interfaces

Voice Interfaces

CLIENT

CLIENT

Independent Project (intended for Apple)

Independent Project (intended for Apple)

MY ROLE

MY ROLE

UX Designer & Researcher

UX Designer & Researcher

TIMELINE

TIMELINE

Jan - Mar 2021

Jan - Mar 2021

·

·

10 Weeks

10 Weeks

Overview

Apple Siri’s speech recognition has presented significant racial and gender bias. This is a redesign of Siri’s settings and feature to improve speech recognition accuracy and performance. Our team’s goal was to re-program Siri to better understand the diverse voices of each of their users.

The Challenge

Siri misinterprets words spoken by users whose first language is not English.

Apple Siri uses speech recognition to allow users to perform tasks on their iPhone, such as speech-to-text for texting, playing music, or answering questions from the web. Their current dataset does not recognize speech spoken by users whose first language is not English or users who don’t identify as a white American. They are forced to eeh-nuhn-see-eyt each one of her words in order for Siri to understand.

Claudia Lopez Lloreda Siri user from Puerto Rico

“Clow-dia,” I say once. Twice. A third time. Defeated, I say the Americanized version of my name: “Claw-dee-ah.” Finally, Siri recognizes it.

My Role

As a UX Designer and Researcher, I focused on understanding speech recognition barriers and translating user findings into design opportunities.

My contributions included:

Research

Evaluating the Current Experience

One of the main pain points we discovered was that Siri was limited to understanding one language at a time . The user is prompted to select one language and one voice for Siri to respond in. If a user speaks regularly in two languages or has a strong accent in one, there is no option to help Siri better recognize their voice.


Reviewing Existing Research

To narrow the scope of the project, our team cross referenced existing research. This revealed that Siri has a 92% accuracy rate when it comes to understanding white American males, a reflection of their first dataset being military personnel. With over 60% of military personnel being white American males, Apple unknowingly developed their program to have a large disparity in voices recognized.


User Testing

Our team wanted to understand the experience of Siri users and their experience with speech recognition. I conducted 10 user interviews with 5 individuals who spoke at least 2 different languages, and 5 individuals who spoke only English and grew up in America. We made sure each of them had an iPhone with Siri installed and asked them to complete 5 tasks on their device with Siri.



Design Process

Exploring personalization as a design opportunity

Our research revealed that Siri's existing settings offered limited control over how the system interpreted different speech patterns.

We explored two complementary approaches:

  1. User-Controlled Personalization

We developed an initial concept called Accent Assumption, allowing users to specify more detailed accent and language preferences within Siri's settings.

Our first idea was to re-design the settings of Siri by implementing a feature titled “Accent Assumption”. This would allow for people to choose from a longer and more specific list of accents and languages. Interface aside, our second idea was to include more diverse datasets and training . We proposed that Siri increase their datasets and training programs to include more racial minorities, ethnic identities, and accents. These changes are outlined in the following prototype.

  1. More Representative Speech Data

We also proposed expanding the diversity of speech recognition training data to better represent different accents and linguistic backgrounds.

Through low-fidelity explorations, we considered how these capabilities could fit into Siri's existing settings without introducing unnecessary complexity.

Our design direction focused on making voice recognition more adaptable while preserving a familiar user experience.

Low-fidelity Siri settings exploration, second sketch

The Solution

A More Personalized Siri Experience

Our final prototype introduced additional accent recognition preferences within Siri’s existing settings.

The concept aimed to give users greater control over how Siri interpreted their speech, particularly for those whose pronunciation differed from standardized language expectations.

Alongside the interface redesign, we recommended expanding the representation of accents and languages in speech recognition training data.

The proposed solution addressed two levels of the experience: user-facing personalization and underlying recognition accuracy.

While the prototype demonstrated the interface concept, improvements to recognition accuracy would require technical implementation and further validation.

Impact & Results

Turning Research into Inclusive Design Opportunities

This project resulted in a research-informed redesign concept addressing limitations in Siri's speech recognition experience.

As an independent concept, the redesign was not implemented or tested for measurable improvements in speech recognition accuracy.

The project's primary outcome was translating observed usability barriers into a tangible product design direction.

Reflection

Inclusive design starts with questioning assumptions.

This project reinforced that accessibility goes beyond interface design. The systems powering an experience can introduce barriers that aren't immediately visible.

Looking back, I would approach the problem with a more rigorous research framework, including a larger and more representative participant sample, measurable recognition error rates, and usability testing of the proposed solution.

I would also question whether asking users to categorize their own accents is the best approach, or whether more adaptive recognition could provide a more seamless and inclusive experience.

My biggest takeaway: Good product design isn't about making everyone interact with technology the same way. It's about creating systems flexible enough to accommodate how people naturally behave.