Introduction: Providing accurate intraoperative surgical guidance with imaging feedback for liver surgeries is still challenging. This abstract introduces a novel approach integrating a Simultaneous Localization and Mapping (SLAM) algorithm for real-time 3D liver surface reconstruction from stereo laparoscopy videos with an ultrasound volume reconstruction algorithm to provide real-time intraoperative guidance. To integrate visualization of both the liver surface and internal vascular anatomy, we use electromagnetic sensors (EM) attached to both the laparoscope and the ultrasound probe (Figure a).
Methods: There are three aspects of our algorithms, which are introduced as follows:
1) The SLAM algorithm utilized in this study can reconstruct and create a larger 3D model of the exposed liver surface from stereo laparoscopic videos (Figure c-h).
2) An algorithm has been developed to reconstruct ultrasound volumes from conventional 2D ultrasound images combined with EM position data, enhancing the 3D visualization of the internal liver anatomy (Figure b, i).
3) By attaching two EM sensors on the stereo laparoscope and the ultrasound probe respectively, we obtain the spatial alignment of the 3D surface model and ultrasound volume, creating a comprehensive representation of the surgical field (Figure i). Our registration algorithms include (1) hand-eye calibration of the laparoscope and the attached EM sensor, motion trajectories-based registration to transform the surface model to the EM space; and (2) ultrasound calibration between the ultrasound probe and the attached EM sensor to transform the ultrasound volume to the EM space (Figure b).
Results: Under an IRB-approved protocol, we evaluated the approach on seven patients. The visualization of the 3D liver surface model, along with the registered ultrasound volume, is shown in Figure i. Two quantitative metrics are reported. The first compares the SLAM-based laparoscope motion trajectories and the EM tracking results, which reflects the accuracy of the liver surface reconstruction. The average error is 5.8 mm. The second is the ultrasound calibration error, which maps the ultrasound images to the EM (in turn, laparoscope) coordinates. The average error is 1.5 mm.
Conclusion: The integration of SLAM-based 3D surface reconstruction and ultrasound volume reconstruction, complemented by EM technology, marks a significant advancement in the development of an intraoperative navigation system for laparoscopic surgery while compensating for tissue deformation. Preliminary results suggest that the proposed intraoperative navigation system could benefit surgeons in accurately localizing anatomical structures in real time.
Figure. (a) EM-tracked stereo laparoscope and ultrasound probe, (b) ultrasound calibration procedure to map the ultrasound image to the EM coordinates, (c-h) step-by-step reconstruction of the liver surface from the stereo laparoscope using SLAM, blue points show the estimated laparoscope position, (i) 3D visualization of the navigation system showing the combined liver surface generated from the stereo laparoscope and liver vasculature generated from the ultrasound images
