Spray-on smart skin uses AI to rapidly understand hand movements
A new type of smart skin developed at Stanford University could potentially revolutionize the way we interact with technology in the future.
[Dec. 31, 2022: Staff Writer, The Brighter Side of News]
The stretchable, biocompatible material is sprayed onto the back of the hand like suntan spray and includes a tiny electrical network that is able to sense the skin's movements and gestures. (CREDIT: Nature Electronics)
A new type of smart skin developed at Stanford University could potentially revolutionize the way we interact with technology in the future.
The stretchable, biocompatible material is sprayed onto the back of the hand like suntan spray and includes a tiny electrical network that is able to sense the skin's movements and gestures through the use of artificial intelligence (AI). This technology has the potential to be used in a variety of fields, including gaming, sports, telemedicine, and robotics.
Current methods of sensing hand movements and gestures, such as wristbands and wearable gloves, have been actively explored, but these devices often require multiple sensors to track each joint of the finger, making them bulky and difficult to use on a daily basis. In contrast, the new smart skin technology is leaner in form and more adaptable to any user, even with limited data. It is also able to learn faster due to its leaner software approach.
The smart skin is made from a sprayable electrically sensitive mesh network embedded in polyurethane, the same material used to make skateboard wheels and to protect hardwood floors.
Related Stories:
The mesh is made up of millions of nanowires coated in silver and gold that form dynamic electrical pathways when in contact with each other. This mesh is biocompatible, breathable, and can stay on the skin unless rubbed with soap and water. It is also able to conform closely to the wrinkles and folds of each individual finger that wears it. A lightweight Bluetooth module can then be attached to the mesh, allowing it to wirelessly transfer the signal changes.
"As the fingers bend and twist, the nanowires in the mesh get squeezed together and stretched apart, changing the electrical conductivity of the mesh. These changes can be measured and analyzed to tell us precisely how a hand or a finger or a joint is moving," explained Zhenan Bao, a K.K. Lee Professor of Chemical Engineering and senior author of the study. This precision could be particularly useful in virtual reality applications, allowing for more realistic experiences by conveying finely detailed motions.
One of the key advantages of this technology is that it can be sprayed directly onto the skin, eliminating the need for a substrate and unwanted motion artifacts. This allows the use of a single trace of conductive mesh to generate multi-joint information of the fingers.
Two-handed QWERTY keyboard typing recognition with nanomesh printed on both hands and real-time recognition of interacting objects. (Credit: Kyun Kyu “Richard” Kim, Bao Group, Stanford University)
The spray-on nature of the device also allows it to conform to any size or shaped hand, and there is potential for it to be adapted to the face in order to capture subtle emotional cues. This could lead to new approaches in computer animation or avatar-led virtual meetings with more realistic facial expressions and hand gestures.
Through the use of machine learning, computers are able to monitor the changing patterns in conductivity and map them to specific physical tasks and gestures. For example, the algorithm can learn to recognize the motion of typing an "X" on a keyboard based on the changing patterns in electrical conductivity. Once the algorithm is trained, the physical keyboard is no longer necessary. The same principles can be used to recognize sign language or to recognize objects by tracing their exterior surfaces.
"We brought the aspects of human learning that rapidly adapt to tasks with only a handful of trials known as 'meta-learning.' This allows the device to rapidly recognize arbitrary new hand tasks and users with a few quick trials," said Kyun Kyu "Richard" Kim, a post-doctoral scholar in Bao's lab, and first author of the study.
"Moreover, it's a surprisingly simple approach to this complex challenge that means we can achieve faster computational processing time with less data because our nanomesh captures subtle details in its signals," Kim added.
Demonstration of fast adaptation to various daily tasks. (CREDIT: Nature Electronics)
Unlike existing technologies, which are computationally intensive and require large amounts of data to be laboriously labeled and processed, the smart skin technology is able to learn quickly and efficiently with only a small amount of data. This makes it more practical for daily use and opens up a wide range of potential applications.
One potential application of the smart skin technology is in telemedicine, allowing doctors to remotely assess a patient's hand movements and gestures in order to diagnose conditions or monitor the progress of rehabilitation. It could also be used in sports training, allowing coaches to analyze the movements of athletes in order to improve performance.
In the gaming industry, the technology could be used to create more immersive and realistic virtual reality experiences. And in robotics, the technology could be used to create more.
More information: Kyun Kyu Kim et al, A substrate-less nanomesh receptor with meta-learning for rapid hand task recognition, Nature Electronics (2022).
Note: Materials provided above by The Brighter Side of News. Content may be edited for style and length.
Like these kind of feel good stories? Get the Brighter Side of News' newsletter.
