Introduction

Most cardiac magnetic resonance (CMR) imaging sequences use ECG-triggering to restrain acquisition at a quiescent phase of the cardiac cycle (i.e. at end systole or mid-diastole) to minimize cardiac motion, and ensure data consistency across heartbeats¹. The trigger time and length of acquisition window are adjusted for each subject. This process is performed by visual inspection of all CINE images² and thus operator dependent, time consuming and requires an operator with experience. In this study, we sought to investigate the feasibility of a fully automated method to detect the quiescent cardiac periods of the cardiac cycle from cine images using an integrated convolutional neural network (CNN) and long short-term memory (CNN-LSTM) network.

Methods

Data description and acquisition
CINE images from 178 consecutive patients (including both congenital and non-congenital heart diseases patients) referred for clinical CMR examinations were retrospectively collected (104 male, aged 18-70 years). All imaging was performed on a 1.5T scanner (MAGNETOM Aera; Siemens Healthcare) equipped with a 32-channel cardiac coil receiver array. CINE images were acquired for each patient in the four-chamber orientation within a breathhold with the following parameters (TR/TE: 2.3/1.16 ms, flip angle: 53°, FOV: 300×250mm², slice thickness: 6mm and in-plane resolution: 1.2×2.1mm², number of cardiac phases: 25 to 30.
Data labelling and pre-processing
The CINE image labelling was done by an experienced cardiologist by visual assessment of the quiescent phases (start and end of both end-systolic and mid-diastolic rest periods).
Proposed CNN-LSTM model architecture
This model integrated CNN and LSTM deep learning networks, taking advantage of the local coherence and the long-term dependency of the CINE images. The proposed CNN-LSTM architecture is illustrated in Figure 1. The input of the model was intensity-normalized. The network consisted of 4 convolutional layers. All the convolutional layers had the same stride of 1 padding in the temporal and spatial domains. Each convolution layer was followed by a batch normalization layer coupled with a 2D maximum pooling layer of matrix size 2×2 respectively. The features extracted by the CNN were fed to 3 LSTM layers consisting of 20 filters. Only the output of the last LSTM layers were fed into the first of the 4 fully connected layers with number of units: 100, 100, 100 and 4, respectively, with each of these layers coupled with a linear unit (ReLU) activation function. The final output of the model was given by the last dense layer and corresponds to the frame indices of the start and end of end-systolic and mid-diastolic rest periods.
Training the CNN-LSTM model
In this study, 80% of the patients were used for training the CNN-LSTM network and 20% were used for testing the model. During training, overfitting was also monitored by using 20% of the training CINE image data as cross-validation data. Throughout training, mini-batches of two 2D + cardiac phase were used.
Implementation details
The deep learning model used the Keras framework with TensorFlow backend. All the code was implemented on a Linux (Ubuntu 16.04) desktop computer with an Intel (R) Xenon (R) Silver 4114 CPU and NVIDIA Titan V GPU.

Results

The proposed CNN-LSTM model outperformed a single CNN model and a single LSTM model as it had the lowest mean absolute error (MAE) for all the quiescent cardiac phases (see Table 1). The performance of the proposed CNN-LSTM model is illustrated in one case in Figure 2 where excellent agreement was observed between the two techniques for both systolic and diastolic rest periods. Bland-Altman analysis revealed that the predicted and visually defined rest periods were in good agreement for the test dataset as shown in Figure 3. The agreement was higher for the systolic rest period than the diastolic rest period.

Discussion

The CNN-LSTM model provides a fully automated and user-independent method for the detection of the quiescent cardiac periods from cine cardiac images, which has potential to facilitate CMR scanning. The relatively better results for the end-systolic phase of the cardiac cycle may be due to the motion of the heart at this stage of the cardiac cycle being stronger and, as a result, the evaluation of the edge of the quiescent period is less subjective. In preparing the data, one clinician performed the data labelling, and this could be improved by having the data labelled by multiple experts to improve the robustness of this approach. In future, this approach could be implemented online to ease the workflow of CMR examination.

Conclusions

A fully automated approach for real-time identification of the cardiac cycle quiescent periods based on cine images has been successfully developed using a novel hybrid CNN-LSTM model. The predicted quiescent periods were in good agreement with those obtained by visual assessment.

Acknowledgements

This work was supported by the EPSRC grant (EP/R010935/1) the Wellcome EPSRC Centre for Medical Engineering at Kings College London (WT 203148/Z/16/Z) and by the National Institute for Health Research (NIHR) Biomedical Research Centre based at Guy’s and St Thomas’ NHS Foundation Trust and King’s College London. The views expressed are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health.