Google has a history of building accessibility into its products. Now, with the launch of Android 10, they’ve created the first operating system in the world to provide live captions to all video content.

 

Introducing Android 10 Live Caption

Android 10 (formerly Android ‘Q’) launched in May earlier this year, via the Android beta program. The official launch came in September, to much applaud from inclusive design champions. While new updates like dark mode and ‘Focus mode’ (which allows you to silence apps that you find distracting) are cool, none are more impressive than their Live Caption feature.

Live captions can now be automated in almost any app, and with any video media, that’s playing. That means Youtube, Instagram, Snapchat, podcasts and video chat apps. It even works with videos and audio that you record yourself.

For users who are deaf or hard of hearing, this feature is a real game-changer.

Live captioning works on the local device, without the need for a data connection. It allows captions to automate in airplane mode, on the tube, and in areas with little to no reception.

“It’s important that our products, that’s Google’s products and Android specifically, work for everybody. When we sit back and look at the fact that there are 2.5 billion devices on the Android platform, it’s imperative that we leverage the innovation that we can bring to users to solve real-world needs.”

Brian Kemler, Product Manager, Android Accessibility.

 

How does Android 10’s Live Captioning work?

Android 10 uses artificial intelligence to process audio into captions, with surprising accuracy. The text transcription displays in a black box with a white font that can be moved about the screen. Even if your phone is on mute or the volume is turned down, the audio is still picked up.

GIF shows woman talking on Facetime on an android phone and captions appearing that transcribe her speech

 

Live captions benefit everyone, not just deaf users

There are 466 million people in the world with a disabling hearing loss. And that figure is going to increase with an ageing global population. By 2050, the World Health Organization estimates that figure will be more than 900 million people.

Screenshots showing Android's live captioning working on a video call and with audio voice notes.

These developments to Android’s OS will change the way deaf and hard of hearing users interact with digital media. But it’s not only hard of hearing users that will gain from this latest update.

People with autism can struggle to decipher speech with lots of background noise. Closed captions can allow autistic people to enjoy videos when the volume is down or muted.

People with ADHD suggest that captioning helps them to focus on video content, and saves them having to repeatedly rewind.

And countless studies suggest that closed captions benefit just about everyone, regardless of disability.

Captions improve literacy rates in children and adults learning to read. They benefit those learning a second language and help hearing adults. Studies have even found that captions improve information retention in students.

In many ways, captions reflect Universal Design principles at their core, i.e. they help everybody.

 

Live captions make business sense

Think about the number of videos that are consumed daily by the billions of people using the internet.

A large number will be uploaded without closed captions enabled, or published to a platform that doesn’t enable captions (Instagram, Snapchat). If the video contains speech, that video is inaccessible for the deaf community.

Video captioning gives people with hearing disabilities the keys access to your content. By not enabling captions on their videos, platforms make a conscious decision to exclude this large community. In 2019, 87% of businesses use video as a marketing tool. But Instagram, for example, does not offer native captioning on videos or their Live Stories.

“Building for everyone means ensuring that everyone can access our products. We believe technology can help us be more inclusive, and AI is providing us with new tools to dramatically improve the experience for people with disabilities.”

Sundar Pichai, Google CEO

A huge 85% of Facebook video is watched without sound on. From people watching videos in public spaces, at work, on the train. If your video contains any dialogue, captions are the only way to communicate the speaker’s message.

As businesses increase their dependency on video as a marketing tool, one way to make sure your content is seen by more people is to make it accessible. The more people your marketing message reaches, the likelihood of sales increases.

In case you need any more convincing of the marketing power of captioned video, here are a few statistics:

Captioning opens a world up to all types of users. It gives businesses the chance to reach people who were unable to consume their video content before. It allows this often ignored group to take part in the conversation.

Android has stepped up where other mainstream platforms and apps have let the disabled community down. It’s a great step in the right direction.