Apple Siri’s speech recognition has presented significant racial and gender bias. This is a redesign of Siri's settings and feature to improve speech recognition accuracy and performance. Our team's goal was to re-program Siri to better understand the diverse voices of each of their users.
10 Weeks (Jan - Mar 2021)
UX Designer and Researcher
Zerabruk Abraham
Beneyam Mesele
Natalie Roy
Apple Siri uses speech recognition to allow users to perform tasks on their iPhone, such as speech-to-text for texting, playing music, or answering questions from the web. Their current dataset does not recognize speech spoken by users whose first language is not English or users who don’t identify as a white American. They are forced to eeh-nuhn-see-eyt each one of her words in order for Siri to understand.
“Clow-dia,” I say once. Twice.
A third time.
Defeated, I say the Americanized version of my name:
“Claw-dee-ah.”
Finally, Siri recognizes it.
One of the main pain points we discovered was that Siri was limited to understanding one language at a time. The user is prompted to select one language and one voice for Siri to respond in. If a user speaks regularly in two languages or has a strong accent in one, there is no option to help Siri better recognize their voice.
To narrow the scope of the project, our team cross referenced existing research. This revealed that Siri has a 92% accuracy rate when it comes to understanding white American males, a reflection of their first dataset being military personnel. With over 60% of military personnel being white American males, Apple unknowingly developed their program to have a large disparity in voices recognized.
Our team wanted to understand the experience of Siri users and their experience with speech recognition. I conducted 10 user interviews with 5 individuals who spoke at least 2 different languages, and 5 individuals who spoke only English and grew up in America. We made sure each of them had an iPhone with Siri installed and asked them to complete 5 tasks on their device with Siri.
Our first idea was to re-design the settings of Siri by implementing a feature titled “Accent Assumption”. This would allow for people to choose from a longer and more specific list of accents and languages.
Interface aside, our second idea was to include more diverse datasets and training. We proposed that Siri increase their datasets and training programs to include more racial minorities, ethnic identities, and accents.
These changes are outlined in the following prototype.
Understand who the target user and demographic is before designing. This will help the team recognize what racial bias, gender bias, or disabilities they should account for.
Conduct user research to identify what the needs of the users are.
Collaborate with everyone. Ideation processes with engineers, management, QAs, and etc. can give new insight and diverse backgrounds.
.png){kind=spacer}