Aim of the Project

IRMA (Image Retrieval in Medical Applications) is a cooperative project of the Department of Diagnostic Radiology, the Department of Medical Informatics, Division of Medical Image Processing and the Chair of Computer Science VI at the Aachen University of Technology (RWTH Aachen). Aim of the project is the development and implementation of high-level methods for content-based image retrieval with prototypical application to medico-diagnostic tasks on a radiologic image archive. We want to perform semantic and formalized queries on the medical image database which includes intra- and interindividual variance and diseases. Example tasks are the staging of a patient's therapy or the retrieval of images with similar diagnostic findings in large electronic archives. Formal content-based queries also take into account the technical conditions of the examination and the image acquisition modalities. The system ought to classify and register radiologic images in a general way without restriction to a certain diagnostic problem or question. Methods of pattern recognition and structural analysis are used to describe the image content in a feature based, formal and generalized way. The formalized and normalized description of the images is then used as a mean to compare images in the archive which allows a fast and reliable retrieval.  In addition to the queries on an existing electronic archive, the automatic classification and indexing allows a simple insertion of conventional radiographs into the system. This is possible without interaction and therefore costly editing of diagnostic findings is avoided by the IRMA-approach. Authorized extraction of date and name information from secondary digitized x-ray films complements the DICOM import. 

Assignment of Responsibilities between the Cooperation Partners.

Each partner has already implemented particular solutions and will develop new ones, which are currently integrated into the framework of IRMA.

• At the Institute of Medical Informatics, the registration and evaluation of geometric content information is developed. Multiscale image segmentation methods are developed and integrated for further evaluation. Furthermore, the IRMA-distributed development platform and database is implemented and maintained here.
• At the Department of Diagnostic Radiology, the classification by supporting texture analysis as well as model- and knowledge-based segmentation of radiographs is performed. Radiologists using the system evaluate retrieval results to improve the reliability of queries and the usability of the graphical user interface (GUI). They also compile sets of training data for large scale evaluation.
• At the Chair of Computer Science VI, algorithms for cluster analysis and image classification are improved and newly developed. Here, the focus is drawn to the statistical approach, which enables discriminant analysis on large feature spaces and large sets of training data. Consequently, the extraction and automatic selection of features belong to this task. It is needed for a flexible and broad concept to information extraction in medical images.

Approach and Methodology

The IRMA project aims at goals in two research fields:

• Automated classification of radiographs based on global features with respect to imaging modality, direction, body region examined and biological system under investigation.
• Identification of image features that are relevant for medical diagnosis. These features are derived from a-priori classified and registered images.

The resulting system must retrieve images similar to a query image with respect to a selected set of features. These features can, for example, be based on the visual similarity of certain image structures. During this project phase, the image data consists of radiographs, while later phases deal with medical images from arbitrary modalities. 

Classification of secondary digitized radiographs

The classification step aims to determine, for example, the examined body part and the imaging parameters used. A coarse classification is done by textural analysis of the image data. Observations from clinical routine made by the Department of Diagnostic Radiology showed the applicability of this approach. Each category has to be further subdivided via methods developed and implemented at the Chair of Computer Science VI. This step is supported by a segmentation of each image by means of a shape analysis. 

All images are registered using a prototype for the respective image category. The approach is designed to be applicable to any imaging modalities (e.g. CT, MRI). 

Extraction of medico-diagnostic image features

Through the selection of relevant medico-diagnostic features locally derived from the registered image data, a content-based database query can be specified. At present, the database contains primary and secondary digitized radiographs, which have been classified by radiologists. By mapping the medical diagnosis to each respective image, features can be extracted that describe and discriminate relevant image content. This includes the identification of diagnostically relevant regions of interest (ROI). By local image analysis, a hierarchical blob representation is obtained describing the image structure. For each prototype, the physician marks a ROI, which is applied for the registered image currently examined. Thus, relevant data can be identified and extracted from the current image. In the following step, the data is passed to a statistical classifier. The classification utilizes experience gained from 1D-data analysis in the field of automated speech recognition. 

System and development environment

IRMA was implemented as a development environment, which allows the distributed storage of different resource types. All resources are administered by using a relational database system. Feature extraction algorithms can be executed within a network cluster. Resources are images, extracted features, as well as the feature extraction algorithms. All resources require the option to be distributed among all project partners. One central aspect of the development environment is a platform-independent framework, which enables the application of feature extraction algorithms to image data. For each image, the database stores information about the image's physical location within the network cluster (i.e. name, file server). Each partner has full access to all images stored within the system. Images are transfered on-demand via the LAN/WAN without user interaction. Extracted features are stored within the database and they are available for evaluation during queries, such as query-by-examples (QBE). All steps required for QBE are organized by the system framework, which initiates the execution of the extraction algorithm for all images. In the same fashion, feature extraction algorithms are automatically distributed. Each feature extraction algorithm uses the same standard programming interface. However, the feature extraction process can be extremely time consuming. Therefore, the system environment allows to install a background process on each host of the network cluster, which polls a job list within the database. Thus, the exectution of feature extraction tasks is balanced throughout the cluster. The database also contains information about each feature extraction algorithm. If a partner requires a new algorithm, the development environment automatically transfers the source files and installs the algorithm. Especially this automated method transfer greatly improves the collaboration between all partners.