Introduction

Magnetic resonance imaging (MRI) is used in clinical practice to diagnose structural fetal disorders in pregnancy but is limited by fetal motion [1]. Meanwhile, fetal motion is an important measure of neurological function, informing postnatal outcomes [2]. However, current clinical MR and ultrasound assessments of fetal motion are qualitative, and do not quantify the detailed 3D motion of all body parts simultaneously [3]. Better assessment of fetal development and monitoring body malformations require novel tools to automatically and efficiently estimate fetal poses and whole-body motion throughout pregnancy. Here we propose an automatic pipeline for fetal pose and motion estimation of fetal MRI.

Methods

Fig. 1 shows the overview of the proposed pipeline.
Data acquisition: Multi-slice EPI data (matrix size$$$=120\times120\times80$$$, resolution$$$=3\text{mm}\times3\text{mm}\times3\text{mm}$$$, TR$$$=5-8\text{s}$$$, interleaved, scan duration between 10 and 30 minutes) were acquired in pregnant women with singletons of gestational age 25 to 35 weeks, resulting in a dataset with 77 series and 19,816 volumes.
Image super-resolution: To improve temporal resolution, we split each stack in interleaved acquisition into two stacks, resulting in lower resolution along the z-axis. We then preprocessed the data using a super-resolution network [4] to recover the details along the z-axis.
Pose estimation: Next, we used a 3D-Unet [5] to detect fifteen fetal keypoints [6] (ankles, knees, hips, bladder, shoulders, elbows, wrists and eyes) from MR volumes, which defined the fetal pose in each frame. To guide the network to learn structural features from images, we adopted an auxiliary task of predicting midpoints of fetal limbs (Fig. 2A) during training.
Anomalous pose correction: The pose estimation network provided a coarse estimate of fetal pose in each time frame, which was sensitive to the artifacts and motion blurring in the image. Therefore, we proposed a method of anomalous pose detection and correction in our pipeline. We consider the following structural priors:
1) For correct poses, the predicted midpoints and the midpoints of predicted keypoints should be close to each other (Fig. 2B). We defined the midpoint error as the distance between them.
2) The length of fetal limbs/bones should be constant during the scan (Fig. 2C). We defined the bone length error as the relative error between the predicted length and the average length in the series.
A fetal pose is considered to be anomalous if midpoint error is greater than $$$3\text{mm}$$$ or bone length error is greater than $$$30%$$$. We corrected the anomalous poses by replacing them with the average of adjacent normal poses.
We used percentage of correct keypoints (PCK) and percentage of correct poses (PCP) to evaluate the accuracy of pose estimation, where PCK($$$s$$$) is the percentage of keypoints with error less than $$$s$$$ and PCP($$$s$$$) is the percentage of poses where all keypoints have error less than $$$s$$$.
Motion estimation: Before estimating fetal motion, poses are transformed to fetal coordinates as in [7] to remove maternal motion. To quantify fetal motion, we estimated velocities of keypoints and angular velocities of joints (knees and elbows). Moreover, the accuracy of motion detection was computed for evaluation, where a threshold of $$$3\text{mm/s}$$$ was used to distinguish between motion and idleness.
We split the dataset into the training set (49 subjects) and test set (28 subjects) and followed the training strategies in [4] and [6] for the super-resolution network and pose estimation network respectively.

Results

We trained pose estimation networks with and without image super-resolution. Fig. 3A shows examples of original and super-resolved data, and Fig 3B shows the PCK on the test set of different models trained on $$$100\%,50\%,25\%$$$, and $$$12.5\%$$$ of training data respectively.
We compared the performance of three different methods for fetal pose and motion estimation, 1) the proposed pipeline, 2) the proposed pipeline without pose correction, 3) a baseline method using a UNet-based pose estimation network [6]. The quantitative metrics of different models are illustrated in Fig. 4. Fig. 5 shows examples of fetal pose time series and curves of fetal motion estimated by the models with and without pose correction.

Discussion

Fig. 3A shows that the super-resolution network improves image quality by recovering details loss due to interleaved acquisition. As shown in Fig. 3B, these recovered details are beneficial to pose estimation. Fig. 4 shows that our pipeline achieves higher accuracy in fetal pose and motion estimation. From Fig. 5, we can see that the model without pose correction sometimes confuses the right wrist and ankle, while the correction stage identifies incorrect keypoints with structural priors. Also, the curves of the motion of the right wrist and ankle show that the model without correction may lead to incorrect motion estimation and detect false motion when the fetus is quiescent.

Conclusion

In this work, we propose an automatic pipeline for fetal pose estimation and motion analysis using deep learning. The super-resolution network can restore image details that improve accuracy of fetal pose estimation. Additionally, the task of midpoint prediction provides a structural prior for detecting anomalous poses. Results show that the proposed pipeline outperforms other fetal pose estimation methods and can serve as a promising tool for the future study of fetal motion tracking and analysis. Also, estimation of fetal limb lengths may provide useful biometric information.

Acknowledgements

This research was supported by NIH U01HD087211, NIH R01EB017337 and NIH NIBIB NAC P41EB015902.