Aug 2021     Issue 17
Research
Relational Graph Learning on Visual and Kinematics Embeddings for Accurate Gesture Recognition in Robotic Surgery
Prof.Qi Dou and Prof. Pheng Ann Heng, Department of Computer Science and Engineering
Prof. Yunhui Liu, Department of Mechanical and Automation Engineering

An engineering team working on AI for surgical robotics CUHK T Stone Robotics Institute has won the Best Paper Award in Medical Robotics at the IEEE International Conference on Robotics and Automation 2021 (ICRA 2021). The award-winning project was led by Prof. Qi Dou, Assistant Professor and Prof. Pheng Ann Heng, Professor from the Department of Computer Science and Engineering, together with Prof. Yunhui Liu, Choh-Ming Li Professor of Mechanical and Automation Engineering.

ICRA is the most important and leading annual international conference in the field of robotics and automation, and it took place in Xi’an, China between 30 May and 5 June this year. The winning project by CUHK entitled “Relational Graph Learning on Visual and Kinematics Embeddings for Accurate Gesture Recognition in Robotic Surgery” is an AI system which can accurately recognise the robotic surgical gesture, which is important and fundamental to cognitive assistance and image-guided automation in robotic surgery. The team has applied a relational graph learning technique to integrate complementary information inherent in visual and kinematic multi-modal data recorded from the surgical robots. It can help to enhance the effectiveness of the “eyes” and “hands” of the surgical robot to understand the complex procedure. This is the first attempt in the area of robotics and automation to explore advanced AI techniques for correlating vision and kinematics, which opens a new door to significantly improving intelligent perception for medical robotics.

This project is jointly validated with the Laboratory for Computational Sensing and Robotics at Johns Hopkins University (JHU), through international collaboration in the Multi-scale Medical Robotics Centre, CUHK. The developed AI system has been extensively validated on two sets of medical robotics platforms of da Vinci Research Kit (dVRK) systems established at CUHK and JHU. The AI model presents outstanding and stable performance on both platforms, demonstrating a promising generalisability and independence of the system, which is crucial for data-driven AI techniques to be widely incorporated into different medical robots.

The project showcases the academic research mission and excellence of the research team in artificial intelligence for medical robotics and surgical data science, which will be a core fuel for benefiting patients who are required to undergo surgery with the next generation of medical robots in the modern operating room.

This article was orginally published on CUHK Communications and Public Office website.



Past Issue      
Contact Us
Subscribe    Email to friend    Unsubscribe
Copyright © 2024.
All Rights Reserved. The Chinese University of Hong Kong.