AI-powered robot learned to perform surgery by watching doctors work

A robot has achieved a medical milestone by mastering surgical skills from watching videos of experienced surgeons. This breakthrough could revolutionize robotic surgery by moving closer to fully autonomous systems capable of performing complex procedures without human intervention.

The innovation, driven by Johns Hopkins University and Stanford University researchers, leverages imitation learning to train robots on surgical techniques. Unlike traditional methods that require painstakingly programming each movement, imitation learning enables robots to learn by observing.

“It’s really magical to have this model. We feed it camera input, and it predicts the robotic movements needed for surgery,” said senior author Axel Krieger. “We believe this marks a significant step forward toward a new frontier in medical robotics.”

At the core of the project is the da Vinci Surgical System, a widely used but often imprecise robotic tool. Nearly 7,000 da Vinci robots operate globally, and more than 50,000 surgeons are trained to use them, creating an extensive archive of surgical videos.

The researchers tapped into this resource, using hundreds of videos from wrist cameras mounted on da Vinci robot arms. These recordings, typically archived after post-operative analysis, served as the robot’s educational material.

The team developed a model that combines imitation learning with advanced machine learning techniques, including the architecture behind ChatGPT. However, while ChatGPT processes words and text, this model operates in the language of robotics—kinematics. It translates robotic movements into mathematical data, enabling the system to perform tasks with precision.

Training the robot focused on three essential surgical skills: manipulating a needle, lifting body tissue, and suturing. In each task, the robot matched the performance of human surgeons. What sets this model apart is its ability to learn relative movements rather than absolute actions, overcoming the inherent inaccuracies of the da Vinci system.

Lead author Ji Woong “Brian” Kim highlighted the model’s efficiency. “All we need is image input, and this AI system finds the right action,” he explained. “Even with a few hundred demos, the model can learn procedures and adapt to new environments it hasn’t encountered.”

One of the model’s standout capabilities is its adaptability. If a robot drops a needle during a procedure, it can autonomously retrieve it and continue without further instruction. “Here, the model is so good at learning things we haven’t taught it,” Krieger said. “This isn’t something I taught it to do.”

This advancement addresses a significant limitation in robotic surgery: the painstakingly slow process of hand-coding robots for specific tasks. Previously, programming a robot to perform a simple task like suturing for a single type of surgery could take years. With imitation learning, a robot can learn a new surgical procedure in just a few days.

“It’s very limiting,” Krieger remarked about traditional methods. “What’s new here is that we only need to collect imitation learning of different procedures, and we can train a robot to learn them quickly. It allows us to accelerate the goal of autonomy while reducing medical errors and achieving more accurate surgery.”

The implications of this technology extend beyond improving efficiency. Autonomous surgical robots could address the global shortage of skilled surgeons, particularly in underserved areas. They also promise greater consistency and accuracy, potentially reducing complications and improving patient outcomes.

Looking ahead, the researchers aim to expand the model’s capabilities to train robots for complete surgeries, not just individual tasks. This marks a significant leap toward the vision of autonomous robotic surgery, where machines could operate independently, guided by the experience encoded in thousands of surgical videos.

The team’s findings are being showcased at the Conference on Robot Learning in Munich, a premier event for robotics and machine learning. Their work demonstrates the potential of combining cutting-edge machine learning with real-world applications in medicine.

This breakthrough not only underscores the transformative potential of artificial intelligence in healthcare but also sets the stage for a future where surgical robots could operate autonomously, offering precision and efficiency unmatched by human hands.

Note: Materials provided above by The Brighter Side of News. Content may be edited for style and length.

Like these kind of feel good stories? Get The Brighter Side of News' newsletter.