How cognitive computing can help the blind ‘see’
Picture for a moment you are blind and speaking to someone who tells you they can use new technology to help you see. This was the experience for Telstra’s Kelly Schulz in interviewing Japanese leading accessibility researcher and IBM Fellow Dr Chieko Asakawa. But it’s perhaps Dr Asakawa’s innovation journey that is even more inspiring than her current project.
On the timeline of innovation, the modern paperback sits back with the evolution of the printing press – closer to the 1880s than the 1980s. But imagine for a moment that you weren’t able to casually pick up and enjoy new releases: Stephen King’s brand new thriller, Misery, or Tom Clancy’s action packed The Hunt For Red October, or even Miss Piggy’s Guide To Life. This was the reality of the 1980s world for Dr Chieko Asakawa, living completely blind.
Dr Chieko Asakawa wasn’t always blind. She was born with full sight, but after a swimming pool accident at age 11, began losing her sight, and by 14 she was fully blind.
While compact discs were first hitting stores and the Sony Walkman became the ‘must have’ accessory, people unable to read the printed word were excluded from easy access to the most basic books and printed information. The only way to create braille books was manually on a braille machine, which had changed little in the 100+ years since its development.
“My innovation journey began when I joined IBM back in the 1980s,” Dr Asakawa recalls.
“It made me believe that one day, I may be able to make one of my dreams come true through technology innovation. In those days if you made a typo in braille, you’d have to do the typing all over again. I wanted to change that. But how?”
And then, a thought: What if braille was digitised?
She developed a braille word-processor allowing books to be typed on a standard keyboard, edited, and printed in braille – improving accuracy and speed to market, and ultimately spawning library networks of braille reading material in Japan.
By the 1990s and having been assuredly bitten by the innovation bug, Dr Asakawa’s innate curiosity for new technology found her pondering the potential of browsing the internet. Inspired by the information resources available and the aspiration for vision-impaired people to access the world digitally, she set about building a synthesised voice browser. Winning much praise, the browser enabled the independence of people who are blind or vision impaired to access information previously unavailable to them. As they say, knowledge is power, but access to education opportunities, social interactions, and economic participation can be life enabling for people with a disability.
And it’s those fundamental life-altering ambitions that led Dr Asakawa to build on her success and delve into cognitive computing.
There is relative safety for someone who is blind when exploring the world while sitting at their computer keyboard, but navigation in “real life” can be significantly more challenging. We live in a visual world full of information, signage, social cues, and everyday details that are missing for someone without sight.
Dr Asakawa’s dream is that cognitive computing will overcome the lack of visual information by giving audio details on timely and relevant information – such as where the bananas are located in the market or if my friend Jenny is smiling or looks sad.
“Cognitive assistant technologies can supplement missing or weakened functions of humans. For me, cognitive assistant technologies will augment my missing ability to see by whispering information that I need with just the right timing,” she explains.
“I will no longer need to wait for someone to tell me who I am about to say hello to. Instead, I can initiate the conversation to become more proactive and also get a sense of other people’s facial expressions during a meeting.”
Still in development, the technology needed to bring Dr Asakawa’s dream to life is forging ahead rapidly. Sensors, localisation, video and speech recognition technologies are all elements that need to come together accurately and almost instantly to create the overall augmented reality experience.
Field testing in February 2017 saw 220 Bluetooth beacons situated throughout a shopping precinct of the shopping park buildings connected with underground passages in Tokyo. Dr Asakawa says that even with a map and a good sense of direction it’s not an easy place to navigate, let alone for someone with a vision impairment. The test combined multiple data sources including maps and shopping centre information and achieved a 1 to 2 metre location accuracy. While a largely successful test, much more research is needed to bring about accessibility to the disparate spectrum of environments a person encounters on a daily basis.
And it’s not a job Dr Asakawa can do alone.
“There is nothing more powerful than people getting together to overcome challenges that we are facing today,” she says.
“We must believe that we can make the impossible possible through collaboration. If more people think that creating an accessible environment is possible by adopting and enhancing technologies it would certainly make a huge difference for Australians with disabilities.”
It would be easy to think that much of Dr Asakawa’s work remedies technologies and a world that have been created without a sense of inclusion and accessibility, which in some sense is true. It is however worth noting that without Alexander Graham Bell’s desire to help his mother and wife who were profoundly deaf, we wouldn’t have the telephone. And the development of the typewriter was advanced in no small part by Agostino Fantoni’s support for his sister who was blind and needed an independent way to write letters.
Identifying and addressing the needs of people is and has always been the nexus to innovation. Fostering accessibility and inclusion of people with a disability through technology is an opportunity that businesses and communities cannot afford to neglect.
For over three decades Dr Asakawa has been at the forefront of unlocking the power of technology to break down the barriers people with disabilities face every day. She is inspiring in every sense of the word’s meaning. And now, through cognitive computing, she is on the cusp of bringing ‘vision’ to people like me.
Our vision for an inclusive future
The digital world provides an unprecedented potential to positively impact the lives of people with disability. In March Telstra launched our first Accessibility & Inclusion Plan, shifting our focus from disability to accessibility and creating an inclusive society where the barriers to equal participation are removed. We know that the digital world presents new opportunities to bridge accessibility gaps and we want to harness that, by pushing the boundaries of technology to shape solutions for the future, drawing on universal design and partnering with others to innovate. Find out more here.
Sign up today for more daily articles and videos from Telstra