Together, a new laser-based imaging system and artificial intelligence (AI) algorithms could revolutionize brain tumor imaging and diagnosis, says Daniel A. Orringer, MD, a neurosurgeon at NYU Langone’s Brain and Spine Tumor Center and pioneer of the new technology. Ultimately the system could have far-reaching impact well beyond brain tumors based on success at Perlmutter Cancer Center in imaging lung, breast, prostate and kidney cancers.
Dr. Orringer and colleagues developed the system using a breakthrough in optics that allows for imaging of fresh tissue without sectioning or staining, called stimulated Raman histology (SRH).
“The impact on patient care is immediate.”
Daniel A. Orringer, MD
SRH utilizes the intrinsic properties of lipids, proteins, and nucleic acids to generate image contrast, showcasing diagnostic microscopic features and histologic findings poorly visualized with conventional hematoxylin and eosin (H&E)-stained images, while eliminating the artifacts inherent in frozen or smear tissue preparations.
“Our research shows that SRH images reveal microscopic architectural features comparable to conventional H&E images,” Orringer says.
Incorporating AI Algorithms
Dr. Orringer and colleagues hypothesized that AI algorithms, in combination with SRH, could augment the ability of pathologists to interpret histologic images.
They developed a parallel workflow that combines SRH with deep convolutional neural networks (CNNs) to predict diagnosis at the bedside in a near real-time, automated manner. Deep CNNs extract image features for classification. Human-level accuracy for image classification tasks has been achieved through deep learning in many specialties of medicine, including pathology.
Astonishing Results
Remarkably, the diagnostic accuracy of the new AI-based system (94.6 percent) was comparable with conventional pathologist-based interpretation (93.9 percent).
“These findings illustrate how intraoperative cancer diagnosis can be streamlined, creating a complementary pathway for tissue diagnosis that is independent of the traditional pathology laboratory. The impact on patient care is immediate. We are able to care for brain tumor patients in a manner that is safer and more accurate than ever before,” Dr. Orringer says.
The implementation of this new system is one of many recent efforts by NYU Langone to integrate AI-based technology in clinical practice to improve cancer diagnostics.
Leveraging Optics and Artificial Intelligence (AI)
Daniel A. Orringer, MD, leads a team exploring advanced optics in the OR. PHOTO: NYU Langone Staff
A new technology that combines AI with advanced optical imaging provides accurate, near real-time intraoperative diagnosis of brain tumors.
Daniel A. Orringer, MD, a neurosurgeon, expects the system to transform the conventional workflow for intraoperative histology, streamlining microscopic imaging. It may also make microscopic and genetic data more accessible to surgical teams.
Laser-Based Imaging Technology
PHOTO: NYU Langone Staff
During surgery, data are gathered using stimulated Raman histology (SRH), a breakthrough in optics that provides rapid, label-free, submicrometer-resolution images of unprocessed biologic tissues.
Advances in fiber-laser technology enable SRH images to be used in concert with intraoperative MRI and fluorescence-guided surgery to provide precision guidance.
Deep Convolutional Neural Networks
PHOTO: NYU Langone Staff
Using AI algorithms, deep convolutional neural networks (CNNs) apply trainable feature extractors, which provide a learned and optimized hierarchy of image features for classification. Dr. Orringer and his team have developed a way to use CNNs to facilitate brain tumor diagnosis during surgery. Working in parallel, SRH imaging and deep CNNs predict diagnosis at the bedside.
Future Applications
Dr. Orringer and Eric K. Oermann, MD, apply the AI technology. PHOTO: NYU Langone Staff
AI algorithms currently under development may hold promise in predicting prognosis and treatment response based on a tumor’s genetic classification. Already, the researchers are applying AI to facilitate interpretation of lung biopsies, anticipating this technology could detect the genetic alterations most amenable to precision therapies.
In addition, clinical trials for brain lesions increasingly incorporate an intraoperative component. Patients could potentially receive an investigational drug during surgery, Dr. Orringer says, driven by advances in AI-based diagnosis.
