Deep-learning-assisted volume visualization

visualization-semi-automatic

Deep-learning-assisted Volume Visualization, by Cheng, Cardone, Jain, et al., IEEE TVCG (2018), published ahead of print.

Abstract:

Designing volume visualizations showing various structures of interest is critical to the exploratory analysis of volumetric data. The last few years have witnessed dramatic advances in the use of convolutional neural networks for identification of objects in large image collections. Whereas such machine learning methods have shown superior performance in a number of applications, their direct use in volume visualization has not yet been explored. In this paper, we present a deep-learning-assisted volume visualization to depict complex structures, which are otherwise challenging for conventional approaches. A significant challenge in designing volume visualizations based on the high-dimensional deep features lies in efficiently handling the immense amount of information that deep-learning methods provide. In this paper, we present a new technique that uses spectral methods to facilitate user interactions with high-dimensional features. We also present a new deep-learning-assisted technique for hierarchically exploring a volumetric dataset. We have validated our approach on two electron microscopy volumes and one magnetic resonance imaging dataset.

Computer-assisted individual drill guide template for minimally invasive lumbar pedicle screw placement trajectory

CIDGT-drill-guide

Feasibility and accuracy of computer-assisted individual drill guide template for minimally invasive lumbar pedicle screw placement trajectory, Wang, Hongwei et al. Injury (2018) published ahead of print.

Abstract

Objective
To discuss the feasibility and accuracy of a specific computer-assisted individual drill guide template (CIDGT) for minimally invasive lumbar pedicle screw placement trajectory (MI-LPT) through a bovine cadaveric experimental study.

Design
A 3-D reconstruction model, including lumbar vertebras (L1-L5), was generated, and the optimal MI-LPTs were determined. A drill guide template with a surface made of the antitemplate of the vertebral surface, including the spinous process and the entry point vertebral surface, was created by reverse engineering and rapid prototyping techniques. Then, MI-LPTs were determined by the drill guide templates, and the trajectories made by K-wires were observed by postoperative CT scan.

Setting
General Hospital of Shenyang Military Area Command of Chinese PLA.

Results
In total, 150 K-wires for MI-LPTs were successfully inserted into L1-L5. The required mean time and fluoroscopy times between fixation of the template to the spinous process, entry point vertebral surface, and insertion of the K-wires for minimally invasive lumbar pedicle screw placement trajectories into each vertebra were 79.4 ± 15.0 seconds and 2.1 ± 0.8 times. There were no significant differences between the preoperative plan and postoperative assessment in the distance from the puncture to the midline and inclination angles according to the different levels (P > 0.05, respectively). The mean deviation between the preoperative plan and postoperative assessment in the distance from the puncture to the midline and inclination angles were 0.8 ± 0.5 mm and 0.9 ± 0.5°, respectively.

Conclusions
The potential use of the novel CIDGT, which was based on the unique morphology of the lumbar vertebra to place minimally invasive lumbar pedicle screws, is promising and could prevent too much radiation exposure intraoperatively.

spine-3d-printed-mis
The computer vertebra biomodel and drill template were both manufactured in acrylate resin, using the stereolithography rapid prototyping technique

Robustness of whole spine 3D reconstruction using 2D biplanar X-ray images

2d-3d-spine-3

Robustness of Whole Spine Reconstruction using Anterior-Posterior and Lateral Planar X-ray Images, by Kim, K., Jargalsuren, S., Khuyagbaatar, B. et al. Int. J. Precis. Eng. Manuf. (2018) 19: 281.

Abstract:

The X-ray-based reconstruction methods using anterior-posterior (AP) and lateral (LAT) images assumes that the angle between the AP and LAT images is perpendicular. However, it is difficult to maintain the perfect perpendicular angle between the AP and LAT images when taking those two images sequentially in real situations. In this study, the robustness of a three-dimensional (3D) whole spine reconstruction method using AP and LAT planar X-ray images was analyzed by investigating cases in which the AP and LAT images were not taken perpendicularly. 3D models of the patient-specific spine from five subjects were reconstructed using AP and LAT X-Ray images and the 3D template models of C1 to L5 vertebrae based on B-spline free-form deformation (FFD) technology. The shape error, projected area error, and relative error in length were quantified by comparing the reconstructed model (FFD model) to the reference model (CT model). The results indicated that the reconstruction method might be considered robust in case that there is a small angular error, such as 5°, between the AP and LAT images. This study suggested that simple technical indications to obtain the perpendicular AP and LAT images in real situations can improve accuracy of 3D spine reconstruction.

2d-3d-spine-2
Reconstruction process of a patient-specific 3D model

Apple’s smart AR/VR glasses: Optical system now patent-pending

apple-AR-glass

Patently Apple reports on a new optical system for head-mounted displays related to virtual reality.

The patent application number is PCT/US2017/044247.

Abstract:

A head-mounted display may include a display system and an optical system in a housing. The display system may have a pixel array that produces light associated with images. The display system may also have a linear polarizer through which light from the pixel array passes and a quarter wave plate through which the light passes after passing through the quarter wave plate. The optical system may be a catadioptric optical system having one or more lens elements. The lens elements may include a plano-convex lens and a plano-concave lens. A partially reflective mirror may be formed on a convex surface of the plano-convex lens. A reflective polarizer may be formed on the planar surface of the plano-convex lens or the concave surface of the plano-concave lens. An additional quarter wave plate may be located between the reflective polarizer and the partially reflective mirror.

Unsupervised learning model for deformable medical image registration

Preprint at arXiv An Unsupervised Learning Model for Deformable Medical Image Registration, by Balakrishnan, Zhao, Sabuncu, Guttag, and Dalca.

Abstract:

We present an efficient learning-based algorithm for deformable, pairwise 3D medical image registration. Current registration methods optimize an energy function independently for each pair of images, which can be time-consuming for large data. We define registration as a parametric function, and optimize its parameters given a set of images from a collection of interest. Given a new pair of scans, we can quickly compute a registration field by directly evaluating the function using the learned parameters. We model this function using a CNN, and use a spatial transform layer to reconstruct one image from another while imposing smoothness constraints on the registration field. The proposed method does not require supervised information such as ground truth registration fields or anatomical landmarks. We demonstrate registration accuracy comparable to state-of-the-art 3D image registration, while operating orders of magnitude faster in practice. Our method promises to significantly speed up medical image analysis and processing pipelines, while facilitating novel directions in learning-based registration and its applications. Our code is available at this https URL

Robotic drill guide positioning in spine surgery using known-component 3D–2D image registration

Robotic drill guide positioning using known-component 3D–2D image registration, by Yi, Ramchandran, Siewerdsen, and Uneri, J. of Medical Imaging (2018), 5(2):021212.

Abstract:

A method for x-ray image-guided robotic instrument positioning is reported and evaluated in preclinical studies of spinal pedicle screw placement with the aim of improving delivery of transpedicle K-wires and screws. The known-component (KC) registration algorithm was used to register the three-dimensional patient CT and drill guide surface model to intraoperative two-dimensional radiographs. Resulting transformations, combined with offline hand–eye calibration, drive the robotically held drill guide to target trajectories defined in the preoperative CT. The method was assessed in comparison with a more conventional tracker-based approach, and robustness to clinically realistic errors was tested in phantom and cadaver. Deviations from planned trajectories were analyzed in terms of target registration error (TRE) at the tooltip (mm) and approach angle (deg). In phantom studies, the KC approach resulted in TRE = 1.51 ± 0.51 mm and 1.01 deg ± 0.92 deg, comparable with accuracy in tracker-based approach. In cadaver studies with realistic anatomical deformation, the KC approach yielded TRE = 2.31 ± 1.05 mm and 0.66 deg ± 0.62 deg, with statistically significant improvement versus tracker (TRE = 6.09 ± 1.22 mm and 1.06 deg ± 0.90 deg). Robustness to deformation is attributed to relatively local rigidity of anatomy in radiographic views. X-ray guidance offered accurate robotic positioning and could fit naturally within clinical workflow of fluoroscopically guided procedures.

Mid-term Survivorship and patient satisfaction of robotic-arm assisted medial unicompartmental knee arthroplasty

Mid-term Survivorship and Patient Satisfaction of Robotic-Arm Assisted Medial Unicompartmental Knee Arthroplasty: A Multicenter Study by Kleeblad et al., Arthroplasty (2018), in press, accepted manuscript.

Abstract
Background
Studies have showed improved accuracy of lower leg alignment, precise component position, and soft-tissue balance with robotic-assisted unicompartmental knee arthroplasty (UKA). No studies, however, have assessed the effect on mid-term survivorship. Therefore, the purpose of this prospective multicenter study was to determine mid-tem survivorship, modes of failure, and satisfaction of robotic-assisted medial UKA.

Methods
473 consecutive patients (528 knees) underwent robotic-arm assisted medial UKA surgery at four separate institutions between March 2009 and December 2011. All patients received a fixed-bearing metal-backed onlay tibial component. Each patient was contacted at minimum five-year follow-up and asked a series of questions to determine survival and satisfaction. Kaplan-Meier method was used to determine survivorship.

Results
Data was collected for 384 patients (432 knees) with mean follow-up of 5.7 years (5.0 – 7.7). The follow-up rate was 81.2%. In total, 13 revisions were performed, of which 11 knees were converted to TKA and in two cases one UKA component was revised, resulting in 97% survivorship. The mean time to revision was 2.27 years. The most common failure mode was aseptic loosening (7/13). Fourteen reoperations were reported. Of all unrevised patients, 91% was either very satisfied or satisfied with their knee function.

Conclusion
Robotic-arm assisted medial UKA showed high survivorship and satisfaction at mid-term follow-up in this prospective multicenter study. However, in spite of the robotic technique, early fixation failure remains the primary cause for revision with cemented implants. Comparative studies are necessary to confirm these findings and compare to conventional implanted UKA and TKA.

New system of computer-assisted navigation leading to reduction in operating time in uncemented total hip replacement

A new system of computer-assisted navigation leading to reduction in operating time in uncemented total hip replacement in a matched population by Chaudhry, Ismail, and Davis, European Journal of Orthopaedic Surgery & Traumatology (2018).

Abstract:

Computer-assisted navigation techniques are used to optimise component placement and alignment in total hip replacement. It has developed in the last 10 years but despite its advantages only 0.3% of all total hip replacements in England and Wales are done using computer navigation. One of the reasons for this is that computer-assisted technology increases operative time. A new method of pelvic registration has been developed without the need to register the anterior pelvic plane (BrainLab hip 6.0) which has shown to improve the accuracy of THR. The purpose of this study was to find out if the new method reduces the operating time. This was a retrospective analysis of comparing operating time in computer navigated primary uncemented total hip replacement using two methods of registration. Group 1 included 128 cases that were performed using BrainLab versions 2.1-5.1. This version relied on the acquisition of the anterior pelvic plane for registration. Group 2 included 128 cases that were performed using the newest navigation software, BrainLab hip 6.0 (registration possible with the patient in the lateral decubitus position). The operating time was 65.79 (40–98) minutes using the old method of registration and was 50.87 (33–74) minutes using the new method of registration. This difference was statistically significant. The body mass index (BMI) was comparable in both groups. The study supports the use of new method of registration in improving the operating time in computer navigated primary uncemented total hip replacements.

Current role of computer navigation in total knee arthroplasty (review)

caos-tka

Current Role of Computer Navigation in Total Knee Arthroplasty, by Christopher W. Jones and Seth A. Jerabek, AAHKS Symposium (Accepted manuscript, in press).

Abstract:

Background
Computer-assisted surgical (CAS) navigation has been developed with the aim of improving the accuracy and precision of total knee arthroplasty (TKA) component positioning and therefore overall limb alignment. The historical goal of knee arthroplasty has been to restore the mechanical alignment of the lower limb by aligning the femoral and tibial components perpendicular to the mechanical axis of the femur and tibia. Despite over four decades of TKA component development and nearly two decades of interest in CAS, the fundamental question remains; does the alignment goal and/or the method of achieving that goal affect the outcome of the TKA in terms of patient reported outcome measures and/or overall survivorship? The quest for reliable and reproducible achievement of the intra-operative alignment goal has been the primary motivator for the introduction, development and refinement of CAS navigation. Numerous proprietary systems now exist and rapid technological advancements in computer processing power are stimulating further development of robotic surgical systems. Three categories of CAS can be defined; image-based large console navigation; imageless large-console navigation and more recently, accelerometer based hand-held navigation systems have been developed.

Conclusion
A review of the current literature demonstrates that there are enough well-designed studies to conclude that both large-console CAS and handheld navigation systems improve the accuracy and precision of component alignment in TKA. However, missing from the evidence base, other than the subgroup analysis provided by the AOANJRR, are any conclusive demonstrations of a clinical superiority in terms of improved patient reported outcome measures and/or decreased cumulative revision rates in the long term. Few authors would argue that accuracy of alignment is a goal to ignore, therefore in the absence of clinical evidence, many of the arguments against the use of large console CAS navigation centre on the prohibitive cost of the systems. The utilization of low-cost, handheld CAS navigation systems may therefore bridge this important gap and over time, further clinical evidence may emerge

Combining intraoperative ultrasound brain shift correction and augmented reality visualizations

brain-AR

Combining intraoperative ultrasound brain shift correction and augmented reality visualizations: a pilot study of eight cases by Gerard et al., J. of Medical Imaging, 5(2), 021210 (2018).

NOTE: You can read or download the paper at ResearchGate.

Abstract:

We present our work investigating the feasibility of combining intraoperative ultrasound for brain shift correction and augmented reality (AR) visualization for intraoperative interpretation of patient-specific models in image-guided neurosurgery (IGNS) of brain tumors. We combine two imaging technologies for image-guided brain tumor neurosurgery. Throughout surgical interventions, AR was used to assess different surgical strategies using three-dimensional (3-D) patient-specific models of the patient’s cortex, vasculature, and lesion. Ultrasound imaging was acquired intraoperatively, and preoperative images and models were registered to the intraoperative data. The quality and reliability of the AR views were evaluated with both qualitative and quantitative metrics. A pilot study of eight patients demonstrates the feasible combination of these two technologies and their complementary features. In each case, the AR visualizations enabled the surgeon to accurately visualize the anatomy and pathology of interest for an extended period of the intervention. Inaccuracies associated with misregistration, brain shift, and AR were improved in all cases. These results demonstrate the potential of combining ultrasound-based registration with AR to become a useful tool for neurosurgeons to improve intraoperative patient-specific planning by improving the understanding of complex 3-D medical imaging data and prolonging the reliable use of IGNS.

brain-AR-flowchart
Flowchart of the intraoperative workflow and how surgical tasks are related to IGNS tasks. A-Patient-to-image registration. After the patient’s head is immobilized a tracking reference is attached to the clamp and 8 facial landmarks are chosen that correspond to identical landmarks on the preoperative images to create a mapping between the two spaces. B Augmented reality visualization on the skull is being qualitatively assessed by comparing the tumor contour as defined by the preoperative guidance images and the overlay of the augmented image. C– A series of US images are acquired once the craniotomy has been performed on the dura and then reconstructed and registered with the preoperative MRI images using the gradient orientation alignment algorithm. D-Augmented reality visualization on the cortex showing the location of the tumor (green) and a vessel of interest (blue). E– The AR accuracy is quantitatively evaluated by having the surgeon choose an identifiable landmark on the physical patient, recording the coordinates, and then choosing the corresponding landmark on the augmented image, recording the coordinates and measuring the two-dimensional distance between the coordinates.