Invention for Multi-modal medical image processing

Invented by Peter Kecskemethy, Tobias Rijken, Kheiron Medical Technologies Ltd

The market for multi-modal medical image processing is experiencing significant growth as healthcare providers and researchers increasingly rely on advanced imaging techniques to diagnose and treat various medical conditions. Multi-modal medical image processing involves the integration and analysis of data from multiple imaging modalities, such as magnetic resonance imaging (MRI), computed tomography (CT), positron emission tomography (PET), and ultrasound, to provide a comprehensive view of a patient’s anatomy and physiology. One of the key drivers of the market for multi-modal medical image processing is the growing prevalence of chronic diseases and the need for accurate and early diagnosis. Conditions such as cancer, cardiovascular diseases, and neurological disorders often require the use of multiple imaging modalities to accurately assess the extent of the disease and plan appropriate treatment strategies. Multi-modal medical image processing enables healthcare professionals to combine and analyze data from different imaging modalities, improving diagnostic accuracy and facilitating personalized treatment plans. Another factor contributing to the market growth is the increasing adoption of artificial intelligence (AI) and machine learning (ML) algorithms in medical imaging. AI and ML algorithms can analyze large volumes of imaging data and extract meaningful insights, aiding in the detection and characterization of diseases. Multi-modal medical image processing algorithms can integrate data from different imaging modalities and leverage AI and ML techniques to improve image quality, automate image analysis, and assist in the identification of abnormalities. The market for multi-modal medical image processing is also benefiting from advancements in imaging technology. The development of hybrid imaging systems, such as PET-CT and PET-MRI, has enabled simultaneous acquisition of data from multiple modalities, reducing the need for separate imaging sessions and improving patient comfort. These hybrid systems generate large amounts of complex data, which can be effectively processed and analyzed using multi-modal medical image processing techniques. Furthermore, the increasing demand for personalized medicine and precision healthcare is driving the adoption of multi-modal medical image processing. By combining data from different imaging modalities, healthcare providers can gain a comprehensive understanding of a patient’s condition, allowing for tailored treatment plans and improved patient outcomes. Multi-modal medical image processing also enables the monitoring of treatment response and disease progression over time, facilitating timely adjustments to the treatment strategy. However, there are challenges that need to be addressed for the widespread adoption of multi-modal medical image processing. Integration and interoperability of different imaging modalities and data formats remain a significant hurdle. Standardization efforts are underway to ensure seamless data exchange and compatibility between different imaging systems and software platforms. In conclusion, the market for multi-modal medical image processing is witnessing substantial growth due to the increasing prevalence of chronic diseases, the adoption of AI and ML algorithms, advancements in imaging technology, and the demand for personalized medicine. As healthcare providers and researchers continue to recognize the value of integrating and analyzing data from multiple imaging modalities, the market for multi-modal medical image processing is expected to expand further, revolutionizing the field of medical imaging and improving patient care.

The Kheiron Medical Technologies Ltd invention works as follows

The present invention is concerned with the identification of areas of interest in medical imaging. The present invention is more specifically a method of identifying regions of interests in medical imaging based upon encoding or classification methods that have been trained using multiple types of medical image data.

Background for Multi-modal medical image processing

Computer-aided diagnostics (also known as CAD) are a set of tools that have been used by medical professionals for over 40 years to help interpret medical images created using techniques like magnetic resonance imaging, ultrasound imaging, and x-rays.

Medical imaging techniques produce large amounts of image data that are typically processed manually by medical professionals, such as radiologists, and then analysed or evaluated in relation to the diagnoses being made. “Typical diagnoses relate, for instance, to cancer detection and screening, as in mammography.

Computer-aided diagnostics can be used in conjunction with medical imaging techniques to process image data in order to highlight regions that are conspicuous in each image and may indicate a region of the image that is of interest to a number of medical professionals. For example, a radiologist.

Computer-aided diagnostics are almost never substituted for the opinions of medical professionals, as they have not reached a level that medical professionals find acceptable.

Further, certain medical image data such as x ray image data is not always optimal to reach a diagnosis due to the lack of clarity or resolution in the images obtained. As such, further data or tests need to be performed by the medical professional or other medical professionals to confirm any preliminary diagnosis on the basis of the immediately-available image data.

Aspects or embodiments seek to provide an algorithm for training an encoder, and/or a classifier, based on multimodal inputs. This allows the classification of regions of interest within medical images using a single source of input data.

The method comprises the following steps: receiving unlabelled data and encoding it using a trained encoder. Using a joint representation module, determining the joint representation. And generating labelled information for the input by using the joint presentation as input to a classifier.

The method can be used to automatically determine a joint representation using a number of pre-trained components. The method determines a joint image automatically using pre-trained components.

Optionally the encoder module, the joint representation and the classifier can be trained using input training data that include a variety of modalities. During test time (or run-time), a single input modality may be received by the method. To improve the accuracy in identifying regions that are of interest when analyzing medical or clinical data the pre-trained component is trained using multiple input modalities. These can be used to determine a joint representation even if only one input was received.

Optionally one or more input data modalities are provided.” In some cases, inputs can include more than one medical image.

The input data can be a combination of: a mammography, an X-ray, a computerised Tomography (CT) scan, a magnetic imaging (MRI) scan; histology; mammography data; a genetic sequence and/or ultrasound data.

The joint representation module can be trained by using the outputs of the encoders.

The joint representation module can receive the encoded data in the form of three-dimensional tensors. The joint representation can be in the form a vector.

Optimally generating labelled input data also includes generating an indicator of one or multiple regions of interest within the unlabelled input data.

The number of modalities for unlabelled data can be one less than the number modalities used for input training data to train the encoder module, joint representation module, and classifier. This will ensure that the components are always provided with an additional data modality to aid input data.

The second aspect of the invention is a method for training a classifier to handle medical or clinical data. It involves: receiving unlabelled data in a prelabelled data set; encoding unlabelled data to create a joint representation, performing classification on the joint to produce labelled information; comparing the prelabelled data to the labelled information and generating comparison data; adjusting an adaptable algorithm to respond to the comparison; and finally repeating these steps until the comparison has reached a threshold that indicates no more adjustments are needed to

The encoders can be connected or paired to perform the step of encoding unlabeled input data.

By training a classifier with at least two data sources, a trained encoder pair and joint representation for classification of medical or clinical data can be created, for example, to identify areas of interest.

Optionally the input data may include data from multiple sources. Two different input data formats are possible. The unlabelled data can be in the form one or more images of medical conditions. The unlabelled data can be in the form a number of medical images. The plurality of images can be related. The input data can include one or more: a CT scan, an X-ray, a computerised Tomography (CT) scanner, an MRI scan and/or an ultrasonic scan. The medical image can be in the form a DICOM.

This data can be used to train a classifier and encoders. The training of a classifier, encoder and joint representation is more robust if the medical images are related. For example, if all of them relate to the same type of cancerous growth.

Optionally the step of encoding unlabelled input data for a plurality modalities in order to form a combined representation is performed separately by each modality.

Click here to view the patent on Google Patents.