People with physical disabilities in Hong Kong can now further integrate into society using tech inventions specially developed for their benefit. Oasis Hu reports from Hong Kong.
Sunny Cheung Sai-ho (right) introduces CP2Joy alongside his father. Cheung, who was born with severe cerebral palsy, developed the system that employs joystick and voice recognition to help people with severe physical disabilities type and use a computer. (PHOTO PROVIDED TO CHINA DAILY)
Text messaging is universally used today, especially by millennials who embrace the use of technology. But it could be a difficult chore for some, like Sunny Cheung Sai-ho, who was born with severe cerebral palsy and is able to move only one of his fingers.
It used to take him half an hour to type just five letters on the computer. That’s now in the past as he can do it in one minute with the aid of CP2Joy (cerebral palsy to joy) — the first human computer device developed for severely disabled people in Hong Kong.
Cheung, 39, had difficulty using the keyboard or the mouse due to his muscle atrophy and severe tremors. “His hands were shaking all the time, damaging keyboards as he tried to type,” recalls Cheung’s father.
For Cheung, many gadgets that are widely used to aid the disabled haven’t been helpful either. His uncontrollable eyeball movements prevent him from using eye-tracking and gaze-interaction devices. Voice recognition software is also out of the question as he can make only a few monotones. Staring at the computer screen is irksome too.
CP2Joy — a system Cheung created with both software and hardware — is tailor-made for people with severe physical disabilities. Instead of the traditional keyboard and mouse, CP2Joy uses a joystick, which Cheung finds easy to control by using his palm instead of his fingers.
He drew inspiration for creating the device from his wheelchair, which is equipped with a lever to control and operate it. By sending out commands with single syllables that sound like “A, E, O”, Cheung can trigger the shortcut preset to his computer via voice recognition software to control the computer. Although typing long paragraphs can still be a chore, he has managed to usher in a new world by connecting himself to a larger planet — the World Wide Web.
Before the advent of CP2Joy, Cheung had relied solely on his parents and his social worker to help him communicate with the outside world, including doing interviews and giving talks in public, as they were among only a few people who could understand him. Now, he has created his own mate that helps him to navigate the challenges in life.
Raymond Tang Wai-man (with glasses), who is physically disabled and works as a waiter in a restaurant in Mong Kok, serves a customer remotely from his home in Sham Shui Po by manoeuvring a robot in the restaurant. (OASIS HU / CHINA DAILY)
Using CP2Joy, Cheung can type three words in less than three minutes. Using the system has helped him overcome some of the bullying and discrimination he was previously subjected to due to his physical disability.
CP2Joy, plus day-and-night study of computer skills and programming techniques, have helped him earn a postgraduate degree in advanced information systems from Hong Kong Baptist University. “He doesn’t even let himself take a day off,” Cheung’s father says.
Besides moonlighting as a programmer, he currently owns a company, providing tailor-made computer services for the disabled. He is also a columnist for a local newspaper. “The computer has changed my destiny. I hope technology can help more people with disabilities to receive education, integrate into society and improve their lives,” Cheung wrote after learning there are some 200,000 people in Hong Kong who are like him and may need such aid.
Cheung launched CP2Joy in 2018 after 18 months of research. He set up a social enterprise a year later to commercialize the software. His company is the first in Hong Kong to customize the software and provide technical support for people with special needs, as well as for special schools and nongovernmental organizations for the disabled.
By August, more than 60 people with severe disabilities had tried out CP2Joy. Two special schools have purchased the system and three others have expressed interest in acquiring it.
According to the Hong Kong Special Administrative Region government, 534,200 people, or 7.1 percent of the local population, suffer from one or more types of disability, such as mental problems or difficulties in seeing or hearing.
A pair of WeVoice Glasses, for demonstration. The glasses are developed by InnoSpire Technology to facilitate visually-impaired people using text recognition, photo description and video-call functions on its matching app WeVoice Plus. (OASIS HU / CHINA DAILY)
Fiction into reality
Help from technological advances comes in various forms. At a restaurant called “Dignity Kitchen” in Mong Kok, Kowloon, Raymond Tang Wai-man, a 31-year-old waiter, who is physically disabled and cannot walk due to cerebral palsy, doesn’t have to show up physically to serve customers. He presents himself by deploying a one-meter-tall robot that rolls around the eatery, greeting and ushering in customers to make their purchases. Tang merely sits at his home in Sham Shui Po, remotely controlling the robot in the restaurant — a common scene in science fiction dramas.
In real life, the advent of remote telecommunication has brought hope and jobs to people with cerebral palsy or those who use wheelchairs. Tang had been jobless for more than two years until he found out that “remote waiters” could be employed. After familiarizing himself with the operations at “Dignity Kitchen”, he began working from home by deploying the robot in greeting customers, leading them to different vendors to pick up their food, and introducing the restaurant to them.
Tang works six hours each week and is paid HK$37.5 ($4.8) per hour. It has been his only source of income during the COVID-19 pandemic. He also has the chance to communicate with various people without having to leave home. Children can also play hide-and-seek with the robot while some foreign customers would like to know Tang’s story.
“My communication skills have improved a lot. What’s more, I could feel a sense of freedom by directing the robot to walk. I never had such feelings before,” says Tang.
Currently, about 43,400 people in Hong Kong, or 0.6 percent of the city’s population, have severe difficulties in seeing. Among them, about 4,200 are blind.
The telecommunications industry has provided some options to help the disabled use smartphones, such as “TalkBack” on Android and “VoiceOver” on iOS. They enable users to make voice calls using a broadband internet connection and can give audible descriptions of contents appearing on the screen.
But, elsewhere, it’s still an unknown world to the disabled. However, the answer may be found in an app called “WeVoice Plus”, developed by Kelvin Siu Kai-wing — a former research engineer who founded InnoSpire Technology in 2018.
Staff of Beyond Vision International examine tactile pictures produced using the company’s Tactile-Audio Interaction System. The system transforms flat images into uneven, three-dimensional and tactile pictures in which the shapes and contours featured in the images are highlighted. (PHOTO PROVIDED TO CHINA DAILY)
The device has three basic functions — providing text recognition, photo description and real-time video calls, helping visually-impaired people to see the world. Users can direct their phone’s camera at the texts or photos, and the text recognition function will activate the artificial intelligence to recognize and read the contents for them. According to Siu, privacy can also be protected, with other people unable to read bank statements or personal letters appearing on the screen.
When artificial intelligence fails to read the texts or photos due to complexity, users can use a “photo description” function, in which registered volunteer users of the app can read the pictures posted by the visually impaired. Everyone can register as a volunteer and offer his or her help at any time, says Siu. If all these functions fail, users can also make a real-time video call so that volunteers can offer direct help.
InnoSpire Technology is also developing intelligent glasses called “WeVoice Glasses” that can help users to operate the text recognition, photo description and video-call functions easily. For instance, it might be inconvenient for visually-impaired people to use the app on a rainy day while holding an umbrella or a cane, making it difficult for them to grasp the phone. Users can then put on the “WeVoice Glasses” and press the button on them to take photos, or call volunteers via Bluetooth.
The product is still being tested and will be available next year. Siu plans to price it at about HK$5,000, or just one-tenth the price of “smart glasses” in the market.
Siu has organized more than 30 workshops and 10 seminars at universities, enterprises and nongovernmental organizations so far to recruit volunteers and teach visually-impaired people how to use the app. The number of users has gone up from 100 since its launch in 2020 to more than 5,000 over the past year, with enlisted volunteers exceeding 200.
Siu says he will continue to upgrade the app’s functions and market the device on the Chinese mainland, Taiwan and worldwide, enabling more disabled people to benefit from the technological creation.
Students with seeing difficulties feel a tactile version of Mona Lisa generated by the system. (PHOTO PROVIDED TO CHINA DAILY)
Seeing without sight
When Malaysian Rico Chan was pursuing a master’s degree in visual arts at Hong Kong Polytechnic University 16 years ago, he saw a photography exhibition where all the photos on display were taken by blind people. The photos were so stunning that they made him realize “sight is not the only way to see the world, and that is why humans have five senses”.
The exhibition inspired Chan to conduct research into the facilities available to assist the visually impaired. He was disappointed to discover that while, for most people, more than 80 percent of information is transmitted visually, facilities to help visually-impaired people receive visual information are scarce.
He noted that while most art galleries or museums do have barrier-free access for visually-impaired people, leading them to the information desks or restrooms, they don’t provide the tools to enable them to appreciate and understand the exhibits inside the galleries, exhibition halls or showrooms.
To provide the visually impaired with facilities to “see” the world, Chan founded Beyond Vision Projects — a social enterprise that aims to improve the quality of life for those who are visually defective, by offering them access to visual information. The organization came into being in 2016 and was developed into an NGO, Beyond Vision International, last year.
It took Chan about six years to develop and improve on the Tactile-Audio Interaction System — a multisensory interactive platform for the visually impaired to “see” images through the synchronization of touch and sound. The system can transform flat images into uneven, three-dimensional and tactile pictures, where the shapes and frameworks of the images’ contents are highlighted. Visually-impaired people can thus get the main messages expressed in the images by touching them.
Chan also added audio to the system to explain the uneven paintings. Buttons are installed at the bottom of each painting. When visually-impaired people press the button, they can hear a pre-recorded, thorough explanation of the images, including details of the subject, the author and the date and time the message was recorded. They can “see” the painting by touching it and listening to the recorded message at the same time.
The Tactile Audio Interaction System can be used in various scenarios. It can turn artworks in museums into versions that can be enjoyed by the blind, or used to make teaching materials in special schools touchable. The technology can also be applied in daily life, turning images, such as brochures, posters and maps, into touchable versions.
Using the system, Chan has held 45 art exhibitions so far, drawing more than 31,800 visitors, and has organized 70 workshops with more than 1,000 participants and featuring 850 tactile paintings and textbooks.
“I’m doing all these things not only to solve some practical problems, but also to make people with disabilities feel they are part of us. We don’t forget them. We care about them. We want them to be included. This is the greatest meaning and real value of technological progress,” says Chan.
When new technology is created, it brings new possibilities to mankind. For the disabled, it’s even more deeply felt. Cheung and others like him have come a long way, and are grateful for the progress they have made in their lives.
Contact the writer at oasishu@chinadailyhk.com