Computer Vision Analysis of Surgical Footage

Rotation advisor: Yin Li, PhD.

In collaboration with: Stephen K. Sauer, MD.

Motivation

• Cataracts are the leading cause of blindness in the world, according to the World Health Organization.

• Cataracts Surgeries are the most common surgical interventions performed in the world (~19M interventions / year).

• Cataract cases are estimated to increase 78% by 2050.

• Ophthalmology residents spend a large portion of their training in learning cataract surgery.

• A key challenge in the training is to develop a systematic and objective assessment of the surgical competency of the residents.

Results

• The trained Mask R-CNN model achieves outstanding performance detecting pupils and twelve different surgical tools.
• We are able to monitor important metrics such as pupil deviation and pupil magnification in real-time.

Surgery Clip	Real-Time Monitoring

Dataset

• CaDIS
• 4738 images extracted from 25 videos with corresponding semantic annotation.
• Training: 3582, Validation: 542, Testing: 614.

Method

• Mask R-CNN
• Detectron2