A team of researchers from Carnegie Mellon University, in collaboration with the Cleveland Clinic’s Cardiovascular Innovation Research Centre, has developed an artificial intelligence (AI) system capable of interpreting some of the most complex heart scans in medicine, cardiac magnetic resonance imaging (MRI), without the need for manually labelled training data.
The novel system, called CMR-CLIP, is designed to interpret cardiac MRI scans by connecting moving images of the heart with corresponding clinical radiology reports. In testing, it significantly outperformed general-purpose AI models, in some cases by more than 35%. The system also showed strong potential for improving cardiac imaging analysis, case retrieval and clinical decision support.
“This work demonstrates that domain-specific foundation models can significantly outperform general-purpose AI systems in specialised clinical applications,” said Ding Zhao, associate professor in Carnegie Mellon University’s Department of Mechanical Engineering and co-principal investigator on the study. “By designing models that reflect the structure and complexity of cardiac MRI data, rather than adapting generic image models, we can unlock new levels of performance and clinical utility.”
David Chen, PhD, of the Cleveland Clinic, a co-principal investigator on the project, emphasised the clinical implications of the work. “Cardiac MRI interpretation is highly specialised and time-intensive. Systems like CMR-CLIP have the potential to support clinicians through automated screening and interpretation support, particularly in settings where expert readers are limited. Such reader assistant tools are critical to improving patient access to this powerful diagnostic technology.”
Cardiac MRI is widely regarded as the gold standard for evaluating heart structure, function and tissue health. A single scan can provide a comprehensive view of the heart, including pumping performance, muscle damage, blood flow and structural abnormalities. However, each study can contain hundreds to thousands of images across multiple views and time points. Even for trained specialists, interpreting a single exam can take 40 minutes or more. Because the technology is expensive and concentrated in major medical centres, there is a limited supply of experts available to meet growing clinical demand.
This combination of complexity and limited data has also made cardiac MRI one of the most challenging domains for AI. Most machine learning systems rely on large, carefully labelled datasets, but in cardiac imaging, expert annotations are scarce, time-consuming to produce and costly to scale.
To overcome this barrier, the research team leveraged a resource already embedded in routine clinical workflows: radiology reports. Every cardiac MRI exam is paired with a written summary in which clinicians document key findings in an “impression” section. Instead of relying on manual labels, the team trained CMR-CLIP to align MRI image sequences with these natural language clinical summaries, enabling the model to learn directly from how physicians describe and interpret scans in practice.
Rather than treating cardiac MRI as a collection of static images, CMR-CLIP represents each study as a video of the beating heart. The model processes multiple standard views of the heart alongside time-resolved sequences that capture motion and tissue behaviour. This lets the model capture both structure and movement, much like a cardiologist does when reviewing a scan.
Trained on more than 13,000 de-identified real patient studies from the Cleveland Clinic, the system learned from over a million images and hundreds of thousands of motion sequences collected over more than a decade. When tested, CMR-CLIP was able to identify cardiac conditions in a “zero-shot” setting, meaning it had never been directly trained on those specific labels, simply by matching images to descriptive prompts like “enlarged left ventricle.”
Even more striking, with just a single example of a condition, CMR-CLIP could often match the performance of other systems that required dozens of labelled cases. In more specialised diagnostic tasks, the model reached near-clinical levels of performance, including accuracy rates as high as 99% for certain heart conditions. It also demonstrated the ability to search through large databases of scans using natural language, retrieving similar cases in a way that could one day help clinicians quickly compare patients with rare or complex presentations.
A key test of whether the system was truly learning meaningful representations came when it was evaluated outside the institution where it was trained. The model still performed strongly on two entirely separate datasets (one collected in France, one in Cleveland Clinic Florida), suggesting it could generalise beyond a single hospital system.
“This work highlights a new direction for medical AI by showing how large-scale clinical data can be used to train models without requiring time-consuming manual labelling,” said Deborah Kwon, M.D., Director of Cardiac MRI at Cleveland Clinic, clinical lead and co-author of this study. “This technology has the potential to not only improve efficiency but also quality of reporting to support more consistent and clinically meaningful interpretations, as well as serve as an important teaching tool in a highly specialised and complex imaging field.”
Looking ahead, the research team plans to extend the model to additional cardiac imaging sequences, including perfusion imaging, T2-weighted imaging and parametric mapping, as well as explore applications in automated report generation and interactive clinical decision support systems in resource-limited applications. This research was published in Nature Communications.
More related news:
IFF Secures Heart Health Approval for Soy Protein in ANZ Markets
NEL Achieves the Milestone of 100 Canine Open Heart Surgeries
Hello Heart Introduces Pill Box to Boost Medication Adherence
