First-of-its-kind robotic glove provides hope to piano players who have suffered a stroke

[July 3, 2023: Staff Writer, The Brighter Side of News]

Stroke survivors and those who've experienced neurotrauma frequently struggle with a profound loss of strength and coordination in their upper limbs, transforming routine daily tasks into significant challenges.

Over the years, numerous attempts have been made to address these issues through the deployment of robotic aids designed to boost their physical capabilities. Yet, the rigidity inherent in these mechanical devices has often proved problematic, particularly when handling complex tasks such as playing a musical instrument. This problematic backdrop, however, is set to change with the groundbreaking introduction of a first-of-its-kind robotic glove.

In an effort to breathe new life into the piano playing dreams of stroke survivors, researchers at Florida Atlantic University’s (FAU) College of Engineering and Computer Science have devised an innovative robotic hand exoskeleton.

Engineered to support rehabilitation and relearning of motor skills, the glove employs a blend of artificial intelligence (AI), soft actuators, and flexible tactile sensors, thereby offering a unique capacity to discern correct and incorrect musical renditions of the same song. This is the first instance where these capabilities have been integrated into a single hand exoskeleton.

Related Stories

"Restoring the ability to perform complex and highly skilled movements such as playing the piano involves relearning specific movements or skills," stated Erik Engeberg, Ph.D., senior author of the research and a professor in FAU's Department of Ocean and Mechanical Engineering. He is also a member of the FAU Center for Complex Systems and Brain Sciences and the FAU Stiles-Nicholson Brain Institute. "Our robotic glove, crafted from soft, flexible materials and fitted with sensors, offers gentle support and assistance, enabling individuals to relearn and regain their motor abilities."

The pioneering research team integrated specialized sensor arrays into each fingertip of the exoskeleton, which has ushered in a level of precision and guidance unmatched by previous exoskeletons. By continually monitoring and adjusting to the user's movements, the robotic glove facilitates the recovery of the intricate finger movements indispensable to piano playing. The device also provides immediate feedback, significantly easing the grasping of correct movement techniques for the user.

The researchers sought to demonstrate the glove's capabilities by programming it to differentiate between correct and incorrect renditions of "Mary Had a Little Lamb". They introduced variations in the performance by generating a pool of twelve different types of errors that could occur at the beginning or end of a note, or due to timing errors. Ten different song variations were tested, each comprising three groups of three variations, plus the correct song played flawlessly.

Machine learning algorithms such as Random Forest (RF), K-Nearest Neighbor (KNN), and Artificial Neural Network (ANN) were trained with the data harvested from the tactile sensors embedded in the glove's fingertips. The purpose was to classify the song variations when the robotic glove was either used independently or worn by a person. The performance of these algorithms was then compared to establish the efficacy of classifying correct and incorrect song renditions with and without the human subject.

Published in the prestigious journal, Frontiers in Robotics and AI, the results of the study were immensely encouraging. The ANN algorithm exhibited the highest classification accuracy, 97.13 percent when used with the human subject and 94.60 percent without the human subject. It could identify out-of-time key presses and accurately determine the percentage error of a certain song. These findings underscore the potential of the smart robotic glove as a rehabilitative tool for disabled individuals attempting to relearn dexterous tasks like playing musical instruments.

The innovative robotic glove was designed with the use of 3D printed polyvinyl acid stents and hydrogel casting, a process that allowed the integration of five actuators into a single wearable device. The end product is a glove that snugly fits the user's hand. The unique fabrication process used, combined with the potential for customization through 3D scanning technology or CT scans, offers unprecedented adaptability to the unique anatomy of individual patients.

Engeberg pointed out that the simplicity of their design, with all actuators and sensors combined into a single molding process, is a key advantage. He further noted that although the study's application was for playing a song, the approach could be applied to a multitude of daily life tasks. The device could potentially facilitate intricate rehabilitation programs tailor-made for each patient.

The data harvested from the robotic glove could be leveraged by clinicians to formulate personalized action plans. They can identify patient weaknesses manifested as sections of the song consistently played incorrectly, and then discern which motor functions require further improvement. As patients progress, the rehabilitation team could prescribe more challenging songs, using a game-like progression system to provide a customizable path to improvement.

The Dean of the FAU College of Engineering and Computer Science, Stella Batalama, Ph.D., lauded the technology, "The technology developed by professor Engeberg and the research team is truly a gamechanger for individuals with neuromuscular disorders and reduced limb functionality." She noted that despite other soft robotic actuators having been used to play the piano, their robotic glove is the only one that has demonstrated the ability to ‘feel’ the difference between correct and incorrect versions of the same song.

Alongside Engeberg, the study’s co-authors include Maohua Lin, a Ph.D. student and first author; graduate students Rudy Paul and Darryl Dieujuste, and recent graduate Moaed Abd, Ph.D., all from the FAU College of Engineering and Computer Science. Other co-authors are James Jones from Boise State University and Harvey Chim, M.D., a professor in the Division of Plastic and Reconstructive Surgery at the University of Florida.

The study was generously supported by the National Institute of Biomedical Imaging and Bioengineering of the National Institutes of Health (NIH), the National Institute of Aging of the NIH, and the National Science Foundation. Additional funding was provided by a seed grant from the FAU College of Engineering and Computer Science and the FAU Institute for Sensing and Embedded Network Systems Engineering (I-SENSE).

The innovative glove marks a pivotal moment in assistive technology, and the research findings represent a significant leap forward in the quest for personalized rehabilitative solutions. The device holds immense promise for empowering stroke survivors and individuals with neurotrauma to reclaim their autonomy and engage in complex tasks, thereby enhancing their quality of life. As research continues, the potential applications for this glove seem almost limitless.

Note: Materials provided above by The Brighter Side of News. Content may be edited for style and length.

Like these kind of feel good stories? Get the Brighter Side of News' newsletter.