Image guidance in surgical and non-surgical invasive procedures has been utilized in various forms over recent years.  This approach facilitates increasingly complex procedures, despite smaller and smaller entry points in the body surface. Most of these procedures involve the operator performing three-dimensional manipulations of the target anatomy using 3 orthogonal two-dimensional (2D) images, 2D live video, and/or a 2D rendering of a 3D image dataset.

 We propose a new approach to the fusion of real-time 3D digital video of the surgical field and multi-slice image data of the same anatomy.  In this approach, the operator sees the surgical field in 3D.  Superimposed upon the real-time image of the anatomy is a translucent 3D representation of the target anatomical feature such as a tumor or the cerebral ventricle derived from MRI, CT, or PET.

 Registration of the multi-slice data with the video data is  required.  In our approach, registration is accomplished using machine-vision surface-analysis of the multi-slice data matched with surface analysis from image pairs derived from the stereo cameras trained on the surgical field.   No fiducial markers are required.  Large and rapid movements of the stereo imager (endoscope, microscope, etc.) and the surgical field are accommodated in this method.