loading page

Automatic Tracking of Surgical Instruments with A  Continuum Laparoscope Using Data-driven Control in Robotic Surgery 
  • +2
  • Xiaowen Kong,
  • Hangjie Mo,
  • Erbao Dong,
  • Yunhui Liu,
  • Dong Sun
Xiaowen Kong

Corresponding Author:[email protected]

Author Profile
Hangjie Mo
Erbao Dong
Yunhui Liu
Dong Sun


In existing surgery process, surgeons need to manually adjust the laparoscopes to provide a better field of view during operation, which may distract surgeons and slow down the surgery process. This paper presents a data-driven control method that uses a continuum laparoscope to adjust the field of view by tracking the surgical instruments. A Koopman-based system identification method is firstly applied to linearize the nonlinear system. Shifted Chebyshev polynomials are used to construct observation functions that transfer low-dimension observations to high-dimension ones. The Koopman operator is approximated using a finite-dimensional estimation method. An optimal controller is further developed according to the trained linear model. Furthermore, a learning-based pose estimation framework is designed to detect keypoints on surgical instruments and provide visual feedback for adjusting the laparoscope. Compared with other detection methods, the proposed scheme achieves a higher detection precision and provides more optional keypoints for tracking. Simulation and experiments validate the feasibility of the proposed control method. Experiment results show that the proposed method can automatically adjust the field of continuum laparoscope through tracking surgical instruments in a timely manner and the number of surgical tools is not limited.
06 Jul 2022Submitted to AISY Interactive Papers
06 Jul 2022Published in AISY Interactive Papers
23 Dec 2022Published in Advanced Intelligent Systems on pages 2200188. 10.1002/aisy.202200188