SurgiAR: Augmented Reality for Surgical Training

"SurgiAR" is an augmented reality application to improve surgical instrument handling skills for medical trainees. Developed using Python, it incorporates Computer Vision and YOLOv5 object detection to provide a dynamic training experience. The application includes a Tkinter-based user interface and integrates YOLOv5 with OpenCV for accurate tool tracking, showcasing a practical blend of problem-solving, machine learning, and software design.

Why SurgiAR?

With the numerous risks associated with practicing instrument handling skills during live patient operations, SurgiAR was developed to offer a safer alternative for surgical training.

Instruments

The instruments used for the training are:

• Micro-scissors, and
• Micro-forceps.

Training Tasks

The main training task available in the application is the Path tracking task. It allows the user to move an object along a path on the computer screen using specific surgical instruments.

Software Tools

• Python
• YOLOv5
• Tkinter
• LabelImg

App Images

Annotation of microsurgical tools with LabelImg

Error dialog alerting user of no records found

App window displaying some performance results of a user

App window displaying a path (line) that has just been drawn by the user and would be used subsequently for training

App window displaying performance metrics employed in this version of the application

Still image of app window showing live training with a microsurgical instrument on a path drawn by the trainee