Researchers from the University of Sydney are developing a 3D printed sensor bracelet that could allow individuals living with hand impairment to easily use computers and play video games that require handheld controllers.
People with cerebral palsy and motor neurone disease struggle with hand movement, preventing them from gaming or using computers. However, this new bracelet allows users to engage in such activities using only their fingers.
The bracelet is made from a form of resin and tracks the tiny wrist movements that occur when someone moves their fingers. The bracelet user's wrist movements are sent to the computer they are using wirelessly where, using machine learning, the movements are interpreted, classified and adapted. The interpreted information can then be used to engage with a device, whether that engagement is controlling the computer interface, playing a game or using a smart device to communicate.
The bracelet and project development was done by a team of engineers from the University of Sydney's School of Computer Science. The team was led by Stephen Lin, an undergraduate honours student at Sydney University, under the supervision of Senior Lecturer Anusha Withana, PhD.
"It accurately detects subtle finger movements through vibrations in the carpal tunnel–an area of the wrist that contains the tendons which control the hand. These subtle movements are then transmitted from the sensors to a program using Bluetooth, which interprets an individual’s movement patterns using machine learning.
"It then communicates this with the game. It does so almost instantaneously, allowing the user to play games that would otherwise require the use of a handheld controller.”
"Currently, these signals are transmitted to a computer-based program, but we hope to progress this to a free phone app so it can be easily downloadable,” said Stephen Lin, leader of the project's engineering team.
The research behind the new sensor technology began in 2020 when Withana received a fellowship to study the wearable applications of new sensor fabrication techniques. The team's current research delves into further applications of the sensor device in the assistance of people with cerebral palsy and is funded by the Cerebral palsy (CP) Alliance and the Neurodisability Assist Trust.
CP Alliance Chair of Cerebral Palsy Research at Sydney University, Prof. Nadia Badawi, said that the new sensor bracelet's accessibility is very important since cerebral palsy is a common disability.
"Cerebral palsy is the most common physical disability in childhood globally, meaning it is vital that these tech advancements are accessible, customisable and as widely available as possible,” said Badawi.
“We know that assistive technology holds the key to a brighter future for many children with cerebral palsy and similar disabilities, with the potential to transform communication, mobility and participation in society."
Cerebral palsy can significantly affect the muscles of a person; 50 percent of people living with the disability find verbal communication difficult or impossible, whilst two-thirds struggle with the movement of one or both of their arms.
“This invaluable project can have a real impact in assisting children with cerebral palsy to play, learn and express themselves," said Badawi.
"At Cerebral Palsy Alliance, we’re proud to support innovative projects such as this through our grants program, which has committed more than $59 million in funding to leading researchers in 38 countries around the world.”
The sensor bracelets were designed using computational fabrication methods, meaning that the device's components can be printed off an inexpensive commercial 3D printer. To improve the accessibility of the new tech, the research team has also developed an easy to use tool that allows users to customise their devices. The team intends to circulate this tool globally, releasing it as open-source software.
“Accessibility shouldn't come at a huge cost. Our mission is to provide an affordable, easy-to-use solution to assist people around the world who are living with disability," said Lin.
"We want this technology to be available to anyone who needs it, which is why we plan to release it publicly without IP.”
“Access to assistive devices is essential for many people with disabilities. However, access to such devices has been a major issue in low-to-middle income countries,” he said.
Head of Rehabilitation Services at MJF Charitable Foundation, Gopi Kitnasamy, said that the sensor bracelet is helping people with disabilities participate in activities they might not be able to do or find difficult. Kitnasamy said that he hopes the new device can help his son.
“My 23-year-old son lives with cerebral palsy and uses a wheelchair. He loves watching car and bike races and playing video games, with a keen interest in car racing and shooting games," he said.
“As he has a lot of dyskinetic (involuntary) movements, he finds it difficult to use joysticks or controllers, so he ends up just watching as we play for him."
Kitnasamy said that although his son can select the cars, drivers, and tracks in the game, he is not able to play, which is very frustrating for him and his family.
“We have long dreamt of finding a way for him to play the game himself fully. We have tried various controllers with sensors, but these devices are very expensive, and they didn’t help in the way he wanted."
“I was so thrilled and excited to learn about the researchers’ sensor bracelet, which appears promising in terms of functionality and affordability," he said.
"We hope the research progresses, not only to give people like my son access to video games but also to improve rehabilitative services and to make the impossible possible."