End-to-End Real-time Catheter Segmentation with Optical Flow-Guided Warping during Endovascular Intervention

Anh Nguyen; Dennis  Kundrat; Giulio  Dagnino; Wenqiang  Chi; Mohamed E. M. K.  Abdelaziz; Yao  Guo; YingLiang Ma; Trevor M. Y. Kwok; Celia  Riga; Guang-Zhong Yang

doi:10.1109/ICRA40945.2020.9197307

End-to-End Real-time Catheter Segmentation with Optical Flow-Guided Warping during Endovascular Intervention

Anh Nguyen
, Dennis Kundrat
, Giulio Dagnino
, Wenqiang Chi
, Mohamed E. M. K. Abdelaziz
, Yao Guo
, YingLiang Ma
, Trevor M. Y. Kwok
, Celia Riga
, Guang-Zhong Yang

Research output: Chapter in Book/Report/Conference proceeding › Conference proceeding › peer-review

29 Citations (Scopus)

262 Downloads (Pure)

Abstract

Accurate real-time catheter segmentation is an important pre-requisite for robot-assisted endovascular intervention. Most of the existing learning-based methods for catheter segmentation and tracking are only trained on smallscale datasets or synthetic data due to the difficulties of ground-truth annotation. Furthermore, the temporal continuity in intraoperative imaging sequences is not fully utilised. In
this paper, we present FW-Net, an end-to-end and real-time deep learning framework for endovascular intervention. The proposed FW-Net has three modules: a segmentation network with encoder-decoder architecture, a flow network to extract optical flow information, and a novel flow-guided warping function to learn the frame-to-frame temporal continuity. We show that by effectively learning temporal continuity, the network can successfully segment and track the catheters in real-time sequences using only raw ground-truth for training. Detailed validation results confirm that our FW-Net outperforms stateof-
the-art techniques while achieving real-time performance.

Original language	English
Title of host publication	2020 IEEE International Conference on Robotics and Automation, ICRA 2020
Publisher	IEEE
Pages	9967 - 9973
Number of pages	7
ISBN (Electronic)	978-1-7281-7395-5
DOIs	https://doi.org/10.1109/ICRA40945.2020.9197307
Publication status	Published - 15 Sept 2020
Event	International Conference on Robotics and Automation - Virtual Conference, Paris, France Duration: 31 May 2020 → 4 Jun 2020 https://www.icra2020.org/

Publication series

Name	Proceedings - IEEE International Conference on Robotics and Automation
ISSN (Print)	1050-4729

Conference

Conference	International Conference on Robotics and Automation
Abbreviated title	ICRA 2020
Country/Territory	France
City	Paris
Period	31/05/20 → 4/06/20
Internet address	https://www.icra2020.org/

Funding

Funders	Funder number
Wellcome Trust
Engineering and Physical Sciences Research Council	EP/N024877/1

ASJC Scopus subject areas

Software
Control and Systems Engineering
Artificial Intelligence
Electrical and Electronic Engineering

Access to Document

10.1109/ICRA40945.2020.9197307

Post-PrintAccepted author manuscript, 696 KBLicence: Unspecified

Cite this

Nguyen, A., Kundrat, D., Dagnino, G., Chi, W., Abdelaziz, M. E. M. K., Guo, Y., Ma, Y., Kwok, T. M. Y., Riga, C., & Yang, G.-Z. (2020). End-to-End Real-time Catheter Segmentation with Optical Flow-Guided Warping during Endovascular Intervention. In 2020 IEEE International Conference on Robotics and Automation, ICRA 2020 (pp. 9967 - 9973). (Proceedings - IEEE International Conference on Robotics and Automation). IEEE. https://doi.org/10.1109/ICRA40945.2020.9197307

End-to-End Real-time Catheter Segmentation with Optical Flow-Guided Warping during Endovascular Intervention. / Nguyen, Anh; Kundrat, Dennis ; Dagnino, Giulio et al.
2020 IEEE International Conference on Robotics and Automation, ICRA 2020. IEEE, 2020. p. 9967 - 9973 (Proceedings - IEEE International Conference on Robotics and Automation).

Research output: Chapter in Book/Report/Conference proceeding › Conference proceeding › peer-review

Nguyen, A, Kundrat, D, Dagnino, G, Chi, W, Abdelaziz, MEMK, Guo, Y, Ma, Y, Kwok, TMY, Riga, C & Yang, G-Z 2020, End-to-End Real-time Catheter Segmentation with Optical Flow-Guided Warping during Endovascular Intervention. in 2020 IEEE International Conference on Robotics and Automation, ICRA 2020. Proceedings - IEEE International Conference on Robotics and Automation, IEEE, pp. 9967 - 9973, International Conference on Robotics and Automation, Paris, France, 31/05/20. https://doi.org/10.1109/ICRA40945.2020.9197307

Nguyen A, Kundrat D, Dagnino G, Chi W, Abdelaziz MEMK, Guo Y et al. End-to-End Real-time Catheter Segmentation with Optical Flow-Guided Warping during Endovascular Intervention. In 2020 IEEE International Conference on Robotics and Automation, ICRA 2020. IEEE. 2020. p. 9967 - 9973. (Proceedings - IEEE International Conference on Robotics and Automation). doi: 10.1109/ICRA40945.2020.9197307

@inproceedings{e341b653de924e7fb47962a3057218e9,

title = "End-to-End Real-time Catheter Segmentation with Optical Flow-Guided Warping during Endovascular Intervention",

abstract = "Accurate real-time catheter segmentation is an important pre-requisite for robot-assisted endovascular intervention. Most of the existing learning-based methods for catheter segmentation and tracking are only trained on smallscale datasets or synthetic data due to the difficulties of ground-truth annotation. Furthermore, the temporal continuity in intraoperative imaging sequences is not fully utilised. In this paper, we present FW-Net, an end-to-end and real-time deep learning framework for endovascular intervention. The proposed FW-Net has three modules: a segmentation network with encoder-decoder architecture, a flow network to extract optical flow information, and a novel flow-guided warping function to learn the frame-to-frame temporal continuity. We show that by effectively learning temporal continuity, the network can successfully segment and track the catheters in real-time sequences using only raw ground-truth for training. Detailed validation results confirm that our FW-Net outperforms stateof- the-art techniques while achieving real-time performance.",

author = "Anh Nguyen and Dennis Kundrat and Giulio Dagnino and Wenqiang Chi and Abdelaziz, \{Mohamed E. M. K.\} and Yao Guo and YingLiang Ma and Kwok, \{Trevor M. Y.\} and Celia Riga and Guang-Zhong Yang",

year = "2020",

month = sep,

day = "15",

doi = "10.1109/ICRA40945.2020.9197307",

language = "English",

series = "Proceedings - IEEE International Conference on Robotics and Automation",

publisher = "IEEE",

pages = "9967 -- 9973",

booktitle = "2020 IEEE International Conference on Robotics and Automation, ICRA 2020",

address = "United States",

note = "International Conference on Robotics and Automation, ICRA 2020 ; Conference date: 31-05-2020 Through 04-06-2020",

url = "https://www.icra2020.org/",

}

TY - GEN

T1 - End-to-End Real-time Catheter Segmentation with Optical Flow-Guided Warping during Endovascular Intervention

AU - Nguyen, Anh

AU - Kundrat, Dennis

AU - Dagnino, Giulio

AU - Chi, Wenqiang

AU - Abdelaziz, Mohamed E. M. K.

AU - Guo, Yao

AU - Ma, YingLiang

AU - Kwok, Trevor M. Y.

AU - Riga, Celia

AU - Yang, Guang-Zhong

PY - 2020/9/15

Y1 - 2020/9/15

N2 - Accurate real-time catheter segmentation is an important pre-requisite for robot-assisted endovascular intervention. Most of the existing learning-based methods for catheter segmentation and tracking are only trained on smallscale datasets or synthetic data due to the difficulties of ground-truth annotation. Furthermore, the temporal continuity in intraoperative imaging sequences is not fully utilised. In this paper, we present FW-Net, an end-to-end and real-time deep learning framework for endovascular intervention. The proposed FW-Net has three modules: a segmentation network with encoder-decoder architecture, a flow network to extract optical flow information, and a novel flow-guided warping function to learn the frame-to-frame temporal continuity. We show that by effectively learning temporal continuity, the network can successfully segment and track the catheters in real-time sequences using only raw ground-truth for training. Detailed validation results confirm that our FW-Net outperforms stateof- the-art techniques while achieving real-time performance.

AB - Accurate real-time catheter segmentation is an important pre-requisite for robot-assisted endovascular intervention. Most of the existing learning-based methods for catheter segmentation and tracking are only trained on smallscale datasets or synthetic data due to the difficulties of ground-truth annotation. Furthermore, the temporal continuity in intraoperative imaging sequences is not fully utilised. In this paper, we present FW-Net, an end-to-end and real-time deep learning framework for endovascular intervention. The proposed FW-Net has three modules: a segmentation network with encoder-decoder architecture, a flow network to extract optical flow information, and a novel flow-guided warping function to learn the frame-to-frame temporal continuity. We show that by effectively learning temporal continuity, the network can successfully segment and track the catheters in real-time sequences using only raw ground-truth for training. Detailed validation results confirm that our FW-Net outperforms stateof- the-art techniques while achieving real-time performance.

UR - https://www.scopus.com/pages/publications/85092731403

U2 - 10.1109/ICRA40945.2020.9197307

DO - 10.1109/ICRA40945.2020.9197307

M3 - Conference proceeding

T3 - Proceedings - IEEE International Conference on Robotics and Automation

SP - 9967

EP - 9973

BT - 2020 IEEE International Conference on Robotics and Automation, ICRA 2020

PB - IEEE

T2 - International Conference on Robotics and Automation

Y2 - 31 May 2020 through 4 June 2020

ER -

End-to-End Real-time Catheter Segmentation with Optical Flow-Guided Warping during Endovascular Intervention

Abstract

Publication series

Conference

Funding

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this