Toward long-term and accurate augmented-reality for monocular endoscopic videos

Gustavo A. Puerto-Souza, Jeffrey A Cadeddu, Gian Luca Mariottini

Research output: Contribution to journalArticle

26 Scopus citations

Abstract

By overlaying preoperative radiological 3-D models onto the intraoperative laparoscopic video, augmented-reality (AR) displays promise to increase surgeons' visual awareness of high-risk surgical targets (e.g., the location of a tumor). Existing AR surgical systems lack in robustness and accuracy because of the many challenges in endoscopic imagery, such as frequent changes in illumination, rapid camera motions, prolonged organ occlusions, and tissue deformations. The frequent occurrence of these events can cause the loss of image (anchor) points, and thus, the loss of the AR display after a few frames. In this paper, we present the design of a new AR system that represents a first step toward long term and accurate augmented surgical display for monocular (calibrated and uncalibrated) endoscopic videos. Our system uses correspondence-search methods, and a new weighted sliding-window registration approach, to automatically and accurately recover the overlay by predicting the image locations of a high number of anchor points that were lost after a sudden image change. The effectiveness of the proposed system in maintaining a long term (over 2 min) and accurate (less than 1 mm) augmentation has been documented over a set of real partial-nephrectomy laparascopic videos.

Original languageEnglish (US)
Article number2323999
Pages (from-to)2609-2620
Number of pages12
JournalIEEE Transactions on Biomedical Engineering
Volume61
Issue number10
DOIs
StatePublished - Oct 1 2014

    Fingerprint

Keywords

  • Augmented reality (AR)
  • endoscopic vision
  • feature tracking

ASJC Scopus subject areas

  • Biomedical Engineering

Cite this