Medical Imaging Sonification

Synopsis

The goal of this research project is to utilize sonification to examine the vast datasets yielded by current imaging techniques. The project aims to supplement current diagnosis techniques and reduce inter-observer variability in diagnoses and improve diagnostic capabilities. Currently, this research project focuses on PET scans of the human brain and the diagnosis of moderate to severe Alzheimer’s disease.

The goal of this research project is to utilize sonification to examine the vast datasets yielded by current imaging techniques. The project aims to supplement current diagnosis techniques and reduce inter-observer variability in diagnoses and improve diagnostic capabilities. Currently, this research project focuses on PET scans of the human brain and the diagnosis of moderate to severe Alzheimer’s disease.

Details

A broad category of diagnostic testing, which has revolutionized diagnosis and management of disease, is medical imaging. A variety of techniques based on x-rays, mechanical vibration, fluorescence, rotation of atoms and radioactive decay have been developed and produce multi-dimensional and time-varying arrays of spatial information. Current methodologies used to “perceive” and interpret medical image data are largely based on the human visual system, and in some cases, enhanced by the use of simple graphs or tables depicting numeric data. Such visual and traditional quantitative analysis methods have led to great advancements in nuclear medicine, radiology and other fields.

Despite these innovations, many limitations remain with respect to the medical community’s ability to perceive and analyze the vast amounts of data now being generated by CT scanners, PET scanners, MRI machines and newer combined devices (including PET/CT and PET/MRI). Diagnostic accuracy is higher than ever, but clinicians still are unable to detect certain conditions when the information provided by visual analysis or basic quantization does not uncover perceptible differences between disease and health.

There are two possible causes for the existing limitations in the diagnosis of disease using medical image data. One possibility is simply that the techniques in use do not provide enough information to allow absolute differentiation between disease and health, no matter how sophisticated our methods of analysis might be or might become. Another possibility is that the information needed to diagnose disease is “hiding” within these images and is not perceptible because the means by which we have chosen to examine the data are insufficient for detection of complex patterns associated with disease. This second possibility suggests that the medical community needs to find new ways to process and understand the data that is being acquired by advanced scanners and other testing equipment.

One potential alternate method for interrogating medical image data is by means of translation of information in to a format that can be processed by an alternate pathway: the human auditory system. Already regarded by experts as superior to vision in the domains of frequency and time, the human ear and associated auditory cortices present a compelling alternative system for perception of medical scan data. Independent of considerations of resolution, the complex neurological pathways of hearing offer a new perspective for understanding spatial, frequency and temporal data. Fortunately, a nascent field exists to allow for such a line of inquiry: sonification.

Using a model of molecular brain imaging whereby detection of small molecules is accomplished through the use of radioactive decay of injected “tracers” (PET imaging), we propose that patterns, when sonified, will emerge from the data and show information that was not previously detected by visual and pre-existing computer analysis techniques. These patterns, when heard and processed by the human brain, might one day allow the medical community to detect diseases that are presently invisible by currently existing methodologies.

Funding

This research is supported by the NYU-HHC Clinical and Translational Science Institute (CTSI).

Soniscan

Screenshot of the SoniScan sonification tool

SoniScan is a sonification tool developed in the Matlab technical computing software aimed at, but not limited to, the sonification of medical imaging data. SoniScan provides a graphical interface through which a user can load, manipulate, and sonify Digital Imaging and Communications in Medicine (DICOM) data – a standard format for viewing and distributing medical imaging data. The program was constructed with a modular approach in mind at both a macro and micro level, in order to allow maximum flexibility for exploring sonification methods that are conducive to brain data display.

Figure illustrating workflow of SoniScan

The Control module reads in and prepares the DICOM data to be sonified. This module has five main functionalities: data read, data selection and zoom, sonification duration, and data adjustment.

The sonification of the data stored in a given 2-dimensional matrix can be done by sonifying one data voxel at a time, by row, by column, or by sonifying the full matrix simultaneously. The Sonification Path module allows the user to define the exact path through which to scan the selected data. SoniScan currently contains three preset paths (see figure below): from left to right, from top to bottom, or all data simultaneously. In addition to these three “conventional” paths, SoniScan allows sonification along a split path. The selected data frame is split along a specified line and sonified as two halves.

Scan paths of SoniScan