After a lifetime living with the effects of the progressive muscle wasting condition muscular dystrophy I would describe my speech as non-standard because I don’t have the breath to speak long sentences, and when I am tired some words are occasionally slightly slurred.
I also need to use a ventilator to help me breathe parts of each day, and wearing the accompanying nasal mask also affects my speech.
It’s all meant that using on-device dictation on Apple and Android phones, as well as Dragon and Apple Voice Control dictation apps, which I rely on because I cannot take to the keyboard to type, can be frustrating when it comes to accuracy. I’m constantly having to correct and clear up annoying little errors with speech recognition in emails and WhatsApp messages etc, which is incredibly frustrating, and knocks productivity.
Perhaps I’m being too harsh on myself because in general I have a fairly standard middle-class British accent, which is bordering on a BBC news reader’s voice. It’s just my delivery can be a bit slow sometimes due to respiratory failure.
I have felt for a long time that major tech companies aren’t putting enough investment into the quality of recognition when it comes to their voice services. If you have struggled to dictate a message on a smartphone, or get a Amazon Echo device or a Apple HomePod to understand you, you will know exactly what I mean.
Dictation accuracy is complicated by the fact that there are so many factors at play when trying to achieve good levels of accuracy, including environment acoustics, the type of microphone you use, the quality of the dictation app, range of accents and dialects, and how tired you are because everyone’s speech changes throughout the day.
I don’t normally struggle to be understood by people but I do struggle sometimes to get voice dictation apps on phones and computers to understand me so the text that I dictate is as accurate as possible.
Against this background, and with all these factors in mind, I was excited when just before Christmas my friend Jen sent me a link to a news story about a new Google initiative called Project Relate that has just launched in the UK, which aims to help people who have conditions that makes their speech difficult to interpret.
Google has long been on the forefront of pioneering voice technology that uses machine learning to enable users to communicate with their devices in a more personalised manner.
Project Relate seeks to improve the way people communicate with their own technology and aims to make it accessible for everyone who might want to use it—regardless of age or ability.
This blog post fills in some of the details about what Project Relate is exactly and how it works, my experiences of using it, as well as how you can get involved if you’re interested.
Google’s Project Relate: What is it?
Project Relate is an effort to bridge the communication gap between people and technology. It seeks to create a more natural and intuitive way for users to interact with their devices, making it easier for anyone—regardless of age or ability—to access and use digital technologies. By leveraging machine learning techniques, Project Relate seeks to enable users to communicate with their devices in a more personalised manner, allowing them to get the most out of their experiences.
The Project Relate app is a mobile application that can be trained to understand your specific speech, if you provide it with examples of you talking. Project Relate needs at least 500 phrases to be trained; most people need 1-2 hours, which can be spread over multiple sessions, to complete their 500 phrases. You need an Android phone (e.g. Pixel, Samsung, OnePlus) running Android OS 8 or later to be able to use the app.
In the news article I read Google said it was seeking English-speaking testers in Australia, Canada, India, New Zealand, and the United States to try out the app and provide feedback to help improve it.
I jumped at the chance to sign up for Project Relate a few days before Christmas. As an iPhone owner and long time Apple fan I even went over to the dark side and bought a Google Pixel 7 Pro phone so I could try out the app as it is only available on Android phones.
I had to fill out an interest form on the official Project Relate website, and within a few days the company got back to me with a link to download the Project Relate app from the Google Play Store to my Pixel phone.
With the help of testers like me, Google says it hopes to build a future in which individuals with disabilities can more easily communicate and be understood.
What’s it like to use the Project Relate app?
Over the past few weeks I have had the opportunity to try out Project Relate as an early tester and I have to say, I am really impressed.
The app is incredibly easy to use and the initial set up process, which involves recording a set of 500 phrases, took me a couple of hours. The app then uses these recordings to automatically learn and adapt to your unique speech patterns. In my case this is coping with slightly slower speech delivery and a softer voice, combined with how my speech changes when I am using my ventilator nasal pillow mask, and the noise of air gushing up my nose.
After recording my initial 500 words and phrases it took about five days for the app to process my speech whereupon I received an alert to say Google now better understands my way of speaking and I was good to go.
Three main features
The Project Relate app has three main functions:
The Listen function transcribes your speech to text in real-time, allowing you to copy-paste text into other apps or let others read what you want to tell them.
The Repeat function allows you to restate what you have said using a clear, synthesised voice. This feature is incredibly helpful in face-to-face conversations or when giving voice commands to home assistant devices.
The Assistant function allows you to speak directly to Google Assistant from within the app, making it easy to complete tasks such as turning on lights or playing music.
Google has said it worked closely with individuals with non-standard speech to develop Project Relate. I can see the effort put into the app, and it truly makes a difference in my ability to communicate and be understood using voice dictation.
If you have ever seen Google’s impressive demo of voice dictation on Pixel phones with the Tensor chip then Project Relate recognition accuracy is like that on steroids!
Project relate keyboard
The feature I use the most is the Project Relate keyboard. This is really useful and has significantly improved my dictation in Gmail, Whatsapp and Facebook Messenger, and indeed anywhere you can dictate text on an Android device like my Pixel phone.
The Project Relate keyboard has helped me produce the most accurate and fastest dictation I have ever been able to produce on a mobile device. It is, quite simply, breathtakingly accurate!
Custom words
A huge shortcoming with other voice dictation apps like Apple Voice Control is the inability to understand names and places not in its dictionary. Project Relate lets you record custom words such as names and places that are important to you. This is incredibly useful and means if you have friends who have foreign names you can record their names and when you come to dictate them the Project Relate app has no problem understanding you.
Beta research app release
Project Relate is a beta research app release so there are some shortcomings, which would be great if Google can address in future updates:
a) the command “new paragraph” doesn’t work
b,) you cannot dictate emoji as you can in Gboard keyboard for example
c) in messaging apps and Gmail there is no “send” command
d) there is no “delete” commandi or “clear all” command
e) annoyingly, for some reason, the app inserts a space between a word and a full stop
After the first 500 words you can go on to record a total of 5640 words and phrases, which should improve accuracy as you use the app. I have recorded 2900 words and phrases since I first started using the app.
However, it is not clear to me how often further analysis of your speech happens after the initial 500words. This isn’t made clear in the app or accompanying information. When you have to spend so much time recording phrases it would incentivise you to go further if the app updated you every 500 or a 1000 words that your speech processing has been updated.
I am in the UK and my phone language is UK English. Whilst I am able to use the Project Relate app it would be great if it was compatible with UK English and mirror my phone language settings.
I use the Google Assistant the most handsfree and Project Relate doesn’t make clear that you can only get better recognition for voice commands you give the Google assistant if you use it within the Project Relate app, which is not as intuitive and convenient as using the Assistant handsfree. It would be great in the future if Google can enable Project Relate and the Assistant to work handsfree without the need to open the Project Relate app.
Well worth a try
If you have a condition that makes your speech difficult to understand, and you struggle with voice technology, I highly recommend giving Project Relate a try. You can express interest by filling out Google’s interest form at g.co/ProjectRelate
Project Relate has the potential to be significant for disabled people and I am confident as Google refines the app it will change the way you communicate with technology devices for the better.
For some people accurate voice dictation can make or break careers, success in education, and keeping in touch with friends and loved ones. Without the ability to communicate easily online disabled people are in danger of getting left behind in the digital world we increasingly occupy and rely on.
Google’s Project Relate app should go a long way towards levelling the playing field for disabled people online.
