Technical Evaluation of Automated Left Ventricular Quantification in Cardiac Magnetic Resonance Imaging: Comparison with Manual Reference Measurements

A retrospective observational design was employed for this study; the study included 178 cardiovascular patients who underwent cardiac magnetic resonance (CMR) imaging between January 2025 and January 2026 at King Abdullah Medical City Specialist Hospital, a single tertiary referral center. Cases were retrospectively retrieved from the cardiology department database and the Picture Archiving and Communication System (PACS).⁵ Patients with complete CMR imaging datasets and available automated left ventricular (LV) measurements were included in the analysis.

Data collection was performed retrospectively and included CMR image datasets and measurements obtained using the Siemens syngo.via post-processing system and Inline CMR analysis. Manual measurements obtained by radiologists and cardiologists using standard volumetric contouring techniques served as reference measurements. During the study period, no changes were made to the clinical imaging protocols, patient management pathways or acquisition procedures. Given that all data acquired were retrospective, there was no effect on image acquisition or clinical decision making. This retrospective observational study was approved by the Institutional Review Board (IRB) / Research Ethics Committee of King Abdullah Medical City Specialist Hospital (approval no. 25-1408, May 2025). Because the study utilized fully anonymized, historically archived data retrieved retrospectively from the Picture Archiving and Communication System (PACS) between January 2025 and January 2026, the requirement for active informed patient consent was waived. Patient confidentiality and data privacy were strictly maintained throughout the study in accordance with institutional guidelines.

The five left ventricular variables considered in this study were Ejection Fraction (EF), End-Diastolic Volume (EDV), End-Systolic Volume (ESV), Stroke Volume (SV), and Myocardial Mass. Left Ventricular Ejection Fraction (LVEF) was calculated using the volumetric equation:

LVEF (%) = [(LVEDV − LVESV) / LVEDV] × 100 (1)

LVEDV represents the left ventricular end-diastolic volume and LVESV represents the left ventricular end-systolic volume.

Alternatively, LVEF represents the proportion of blood ejected from the left ventricle during a single cardiac cycle and is calculated using Stroke Volume (SV) as follows.⁶

LVEF (%) = [LVSV / LVEDV] × 100 (2)

where LVSV = LVEDV − LVESV is the left ventricular stroke volume, i.e., the amount of blood that is ejected per heartbeat; and the mass of the left ventricular myocardium, which is derived from the volumetric analysis, is known as Myocardial Mass (g).⁷

Descriptive statistics were calculated for the five LV parameters obtained using the three measurement methods. Agreement was assessed using the intraclass correlation coefficient (ICC) and Bland–Altman analysis; while measurement error was evaluated using root mean square error (RMSE) and mean absolute percentage error (MAPE). These analyses were performed for both automated versus manual and Inline versus manual comparisons across all five LV parameters.

For Bland–Altman analysis, signed differences between paired measurements were calculated to determine mean bias and limits of agreement. Absolute and percentage differences were calculated for the corresponding error analyses. RMSE was calculated from the squared differences between paired measurements, while MAPE was used to express measurement error as a percentage of the manual reference measurements. Statistical analysis was performed using IBM SPSS Statistics (version 24; IBM Corp., Armonk, NY, USA).

Additional analyses, including normality testing, Friedman and Wilcoxon signed-rank tests, were considered secondary or exploratory analyses and are presented in the Supplementary Material.

The sample comprised 130 males (73.0%) and 48 females (27.0%). Three cardiac magnetic resonance (CMR) procedure types were represented: Cardiac Function with Contrast (CFC, n = 142, 79.8%), Cardiac MRI with Contrast (CMw/C, n = 26, 14.6%), and Cardiac MRI without Contrast (CMw/oC, n = 10, 5.6%). Table 1 presents the sample characteristics.

Table 2 presents the summary statistics of the five cardiac parameters; across all cardiac parameters, values derived from Automated cardiac MRI analysis were closer to the Manual reference standard. In contrast, Inline CMR measurements showed greater deviation from the Manual reference standard (Figure 1, Figure 2, Figure 3, Figure 4, Figure 5).

Figure 1. Mean Ejection Fraction: Inline vs Automated vs Manual

Download figureNote. Mean Ejection Fraction are shown for each of Inline, Automated and Manual measurements.

Figure 2. Mean End-Diastolic Volume: Inline vs Automated vs manual.

Download figureNote. Mean End-Diastolic Volume scores are shown for Inline, Automated, Manual measurements

Figure 3. Mean End-Systolic Volume: Inline vs Automated vs Manual

Download figureNote. Mean End-Systolic Volume scores are shown for Inline, Automated, Manual measurements

Figure 4. Mean stroke Volume: Inline vs Automated vs Manual

Download figureNote. Mean Stroke Volume scores are shown for Inline, Automated, Manual measurements

Figure 5. Mean Myocardial Mass: Inline vs Automated vs Manual

Download figureNote. Mean Myocardial Mass are shown for Inline, Automated, Manual measurements

The Root Mean Square Error (RMSE) and Mean Absolute Percentage Error (MAPE) values are displayed for both comparison pairs in Table 3. These measures were employed to measure the agreement between methods. Specifically, RMSE places greater weight on larger individual deviations and is expressed in the original measurement units, making it sensitive to extreme errors in cardiac parameters. In contrast, MAPE expresses the average error relative to the Manual reference values in percentage terms, which allows for standardized comparison across parameters measured in different units (Figure 6).

Figure 6. MAPE by Parameter: Automated vs Manual

Download figureNote. Mean MAPE score are shown for Automated, and Manual measurements comparisons

The Automated vs Manual comparison was the primary analysis. In this comparison, RMSE values were generally low across EF (1.57%), EDV (2.41 ml), SV (6.90 ml), and Mass (6.21 g). ESV showed a higher RMSE of 31.26 ml. MAPE values further confirmed consistent close agreement between both methods: EF 2.37%, EDV 0.72%, ESV 3.21%, SV 2.65%, and Mass 1.86%. All five MAPE values remained below 3.5%, including ESV (3.21%); however, ESV showed a higher maximum individual percentage error (89.26%) than the other parameters, reflecting the influence of isolated outlier cases rather than poor average agreement. Overall, these results indicate that the Automated post-processing measurements achieved high accuracy relative to Manual reference values.

This comparison yielded poorer accuracy metrics compared to the Automated vs Manual analysis across all five cardiac parameters. The RMSE for EF was 8.49%, which is more than five times higher than the Automated equivalent, with a MAPE of 19.41%, indicating that, on average, the raw Inline EF value deviated from the Manual reference by nearly one fifth of the reference value. The volumetric parameters demonstrated the largest discrepancies, with EDV showing an RMSE of 122.48 ml (MAPE = 11.30%) and ESV showing an RMSE of 124.70 ml (MAPE = 19.03%). Stroke volume and myocardial mass also exhibited substantially higher errors compared to the Automated approach, with MAPE values of 17.90% and 13.20% respectively.

The maximum percentage errors observed for Inline vs Manual were extreme, reaching 382.43% for SV and 346.99% for Mass in individual cases, indicating that the raw scanner output can, in a minority of cases, deviate drastically from the clinician-derived reference. These findings indicate that Inline measurements may require additional verification or post-processing before being used as a substitute for manual reference measurements (Figure 7).

Figure 7. MAPE by Parameter: Inline vs Manual

Download figureNote. Mean MAPE score are shown for Inline, and Manual measurements comparisons

Bland-Altman analysis was conducted for both comparison pairs across all five cardiac parameters. For each parameter, the mean bias (signed difference: Method minus Manual) and 95% Limits of Agreement (LoA = bias ± 1.96 SD) were derived from the signed difference descriptive statistics. Bland-Altman scatter plots were generated for all ten comparisons; reference lines were added to each plot representing the mean bias and upper and lower LoA. Results are summarized in Table 4.

The Automated method demonstrated small, clinically negligible biases relative to the Manual reference across four of five parameters. EF bias was −0.83% (95% LoA: −3.45 to 1.80%), indicating that Automated measurements were very slightly lower than Manual on average, with 95% of the differences expected to lie between −3.45 and +1.80 percentage points. EDV bias was +0.74 ml (LoA: −3.77 to 5.25 ml), and Mass bias was −1.03 g (LoA: −13.02 to 10.97 g) both narrow and centred near zero. SV showed a small negative bias (−1.45 ml, LoA: −14.71 to 11.80 ml), indicating that Automated SV values were on average slightly lower than Manual. ESV produced a larger SD (31.32 ml) and correspondingly wider LoA (−62.81 to 59.96 ml), driven by the small number of extreme outlier cases. The median absolute ESV difference (0.72 ml) confirms that agreement was close in most cases, and the wide limits reflect outlier influence rather than systematic disagreement across the sample.

The Inline vs manual comparison revealed systematic and clinically significant biases across all five parameters. EF showed a positive bias of +4.22% (LoA: −10.26 to 18.70%), confirming the systematic inflation of Inline EF relative to Manual. This is clinically meaningful: a patient with a true EF of 42% (borderline reduced) could receive an Inline EF reading as high as 60%, potentially obscuring the need for clinical intervention. Additionally, volume parameters showed particularly wide limits of agreement, reflecting both systematic bias and extreme variability. EDV bias was −21.75 ml (LoA: −258.66 to 215.15 ml) and ESV bias was −26.37 ml (LoA: −265.94 to 213.20 ml). These extraordinarily wide limits indicate that the Inline scanner output is not interchangeable with Manual measurements for volume assessment in this population. SV bias was +1.98 ml (LoA: −30.15 to 34.11 ml), appearing modest in absolute terms but with limits spanning approximately 64 ml, clinically unacceptable for individual patient assessment. Mass bias was −6.21 g (LoA: −63.18 to 50.77 g), again with wide limits reflecting substantial individual variability.

Overall, the two comparisons revealed that Automated measurements were the closest to the standard reference (Manual) across nearly all cases, with biases that are small in absolute terms and clinically acceptable for EF, EDV, and Mass. On the other hand, inline measurements varied widely with significant biases.

ICC results are presented in Table 5 with 95% confidence intervals and interpreted using the benchmarks of Koo and Li:⁸ ICC > .90 = Excellent; .75–.90 = Good; .50–.75 = Moderate; < .50 = Poor. All ICC values were statistically significant (all p < .001).

The Automated method achieved excellent reliability relative to the Manual reference standard across all five cardiac parameters. The ICC for EF was .996 (95% CI: .988–.998), for EDV was 1.000 (95% CI: 1.000–1.000), for ESV was .985 (95% CI: .980–.989), for SV was .957 (95% CI: .941–.968), and for Mass was .992 (95% CI: .990–.994). All five values exceeded the excellent threshold of .90, with EF, EDV, and Mass approaching the theoretical maximum of 1.0. This finding is consistent with the Bland-Altman analysis and the accuracy metrics that found the Automated measurements to be near the manual measurements.

The Inline comparison produced substantially lower ICC values across all five parameters, compared to the Automated process. EF achieved good reliability (ICC = .874, 95% CI: .739–.929), though with a notably wide confidence interval indicating greater uncertainty. Mass (.823, 95% CI: .766–.867) falls within the ‘Good’ range. SV (.733, 95% CI: .657–.794), EDV (.673, 95% CI: .582–.746) and ESV (.619, 95% CI: .516–.704) fall within the ‘Moderate’ range, indicating that Inline volumetric measurements share a meaningful portion of variance with Manual values but are not sufficiently reliable for interchangeable clinical use.

The contrast between the two comparisons is striking: every automated parameter exceeded the Excellent threshold, whereas three Inline parameters (SV, EDV and ESV) fell within the Moderate range. This finding, combined with the Bland-Altman results, provides strong and consistent evidence that Automated post-processed measurements are substantially more reliable than Inline scanner output.

The Automated post-processing pipeline in this study demonstrated excellent agreement with Manual reference standards across all evaluated cardiac parameters (EF, EDV, ESV, SV, and myocardial mass). The ICC values observed ranged from 0.957 to 1.000, indicating excellent agreement; while MAPE remained below 3.5% for all variables. Bland–Altman analysis further confirmed minimal systematic bias, with EF (-0.83%), EDV (0.74 mL), SV (-1.45 mL), and myocardial mass (-1.03 g) showing clinically negligible deviations from Manual measurements.

The high agreement observed in the present study is consistent with previous evaluations of automated CMR segmentation, which have reported performance approaching expert manual measurements.⁹ The high ICC values (>0.95) observed in this study are likewise consistent with Böttcher et al.,¹⁰ who noted that clinically meaningful differences occurred in approximately 18% of cases when compared with expert-corrected manual analysis, although overall agreement remained excellent.

Inline scanner-derived measurements demonstrated systematic and clinically relevant deviations from Manual reference standards, in contrast to the Automated pipeline. EF was consistently overestimated (+4.22%), while EDV and ESV were underestimated, producing a distorted stroke volume profile. Furthermore, Inline measurements showed lower agreement than automated measurements, with ICCs ranging from 0.619 to 0.874; EF and myocardial mass showed good agreement, whereas EDV, ESV, and SV fell within the moderate ICC range. Bland–Altman limits of agreement were wide, reaching up to ±250 mL in extreme volumetric cases, highlighting substantial variability at the individual patient level. These findings align with the UK Biobank InlineVF evaluation,¹¹ which reported similar systematic patterns of bias, including LV mass overestimation (~29.9 g) and moderate agreement for EF (ICC ≈ 0.586).

In a similar large-scale clinical evaluation, Böttcher et al.¹⁰ noted that fully automated inline-style analyses required expert correction in 18% of cases despite high ICC values, suggesting that Inline systems may produce acceptable aggregate performance but unreliable individual-level measurements.

A clear and consistent hierarchy in performance was observed at comparison between Inline, Automated, and Manual methods across all five cardiac parameters. The Automated method demonstrated excellent agreement with Manual reference measurements across all five evaluated parameters; while Inline measurements demonstrated only moderate-to-good agreement with Manual reference values (good for EF and myocardial mass; moderate for SV, EDV and ESV). Systematic bias was evident across all parameters, particularly EF overestimation and EDV/ESV underestimation. This distinction is supported by ICC analysis and reinforced by Bland–Altman bias patterns and percentage error metrics. Importantly, while Inline methods may show acceptable correlation at the population level, their wide variability and directional bias significantly reduce reliability for individual clinical decision-making.

Overall, automated post-processing showed closer agreement with manual reference measurements than Inline measurements across the evaluated LV parameters.

The findings of this study have direct implications for clinical cardiac MRI workflows. The findings support the feasibility of incorporating automated post-processing into routine LV quantification workflows, with expert review when segmentation or measurement reliability is uncertain. This supports the growing body of literature advocating for AI-assisted cardiac imaging workflows, where automated segmentation serves as the primary quantification method with selective human oversight.

In contrast, the greater disagreement observed with Inline measurements suggests that these measurements may warrant confirmation when used for clinical decision-making. The systematic bias and wider variability suggest that Inline measurements should be interpreted cautiously during clinical decision-making. Despite advances in automation, Manual review remains essential in selected cases, particularly where image quality is suboptimal or anatomical abnormalities are present.

Several limitations could be addressed in future studies. First, the Manual measurements, while treated as the reference standard, are subject to inter- and intra-observer variability; a single clinician measurement per patient was used without inter-rater reliability assessment. Second, the Automated method was evaluated as a whole pipeline; the contribution of individual algorithmic steps to any residual errors was not examined.