Shanshan Wang1, Zhenghang Su1,2, Leslie Ying3, Xi Peng1, and Dong Liang1
1Shenzhen Institutes of Advanced Technologies, Shenzhen, China, People's Republic of, 2School of Information Technologies, Guangdong University of Technology, Guangzhou, China, People's Republic of, 3Department of Biomedical Engineering and Department of Electrical Engineering, The State University of New York, Buffalo, NY, United States
Synopsis
This paper proposes a deep learning based approach for accelerating MR
imaging. With the utilization of a large number of existing high-quality MR
images, we train an off-line convolutional neural network (CNN) to identify the
mapping relationship between MR images obtained from zero-filled and
fully-sampled k-space data. Then the trained CNN is employed to predict an
image from undersampled data, which is used as the reference in solving an online
constrained imaging problem. Results on in vivo datasets show that the proposed
approach is capable of restoring fine details and presents great potential for
efficient and effective MR imaging.INTRODUCTION
Signal processing based MR image reconstruction from reduced samples
have played an essential role in accelerating MR scans in recent years [1,2]. A
foundation for the success of these methods is the utilization of prior
information on MR images, such as sparsity [3], low-rank [4], manifold fitting [5] and generalized series [6].
Nevertheless, despite all the achievements obtained by the existing methods, the
exploited priors are still quite limited to the knowledge about the target
image or very few reference images. Based on the observation that the anatomic
structure of the same organ/tissue between different people are quite similar,
we try to learn an off-line prior model to aid online fast imaging by taking
advantage of the enormous images acquired each day. Specifically an off-line
convolutional neural network is trained to describe the end-to-end mapping
between zero-filled and fully-sampled MR images. The network is not only
capable of restoring the details and fine structures of the MR images, but is
also compatible with online constrained reconstruction model for efficient and
effective MR imaging.
THEORY AND METHOD
The proposed method consists of two main
parts: the off-line training and online imaging. In the off-line settings,
consider T pre-acquired MR images $$$u_t$$$ reconstructed from fully-sampled data, we
design and train an L-layer convolutional neural network $$$\left\{
\begin{array}{l}
C_0=x \\
C_l=\sigma(W_l*C_{l-1}+b_l), l\in{1,2,...,L-1} \\
C_L=\sigma(W_L*C_{L-1}+b_L) \end{array}
\right.$$$ by minimizing $$$\mathop{\rm
argmin}_{{ \Theta}} \left\{\frac{1}{{\rm 2T}}\sum^T_{t=1}\|C(z_t;\Theta)-u_t\|_2^2\right\}
$$$ where $$$z_t$$$ is the zero-filled MR image
generated as the direct inverse of the undersampled data and $$$C$$$ means the
end-to-end mapping function with hidden parameters $$$\Theta=\{(W_1,b_1),...,(W_l,b_l),...,(W_L,b_L)\}$$$.
Once we learned the hidden parameters $$$\hat{\Theta}$$$ from the pre-acquired
datasets, we can reconstruct MR images by considering the online constrained
optimization problem $$$\mathop{\rm argmin}_{{u}}
\left\{\|C(F^Hf;\hat{\Theta})-u\|_2^2+\lambda \|f-PFu\|_2^2\right\}$$$, where $$$f$$$
means the undersampled data, $$$P$$$ is the undersampling diagonal mask, $$$F$$$ denotes
the full Fourier encoding matrix normalized as $$$F^HF=I$$$ and $$$H$$$ represents the
Hermitian transpose operation. As we can see, the first term in the cost
function promotes the similarity between the network prediction and the target
image, and the second term enforces the data fidelity in k-space. As a simple
least squares problem, we can give an analytical solution $$$Fu(k_x,k_y)=\left\{
\begin{array}{ll}
S(k_x,k_y) &, ~{\rm if}~
(k_x,k_y)\notin \Omega \\
\frac{S(k_x,k_y)+\lambda
S_0(k_x,k_y)}{1+\lambda} &, ~{\rm if}~ (k_x,k_y)\in \Omega\\\end{array}
\right.$$$, where $$$S_0(k_x,k_y)= FF_M^Hf$$$, $$$S(k_x,k_y)=FC(F^Hf;
\hat{\Theta})$$$; and $$$\Omega$$$ means the sampled locations.
EXPERIMENT
The training data
consists of over 500 fully sampled MR brain images we collected from a 3T
scanner (SIEMENS MAGNETOM Trio). Informed consent was obtained from the imaging
subject in compliance with the Institutional Review Board policy. Undersampled
measurements were retrospectively obtained using the 2D Poisson disk sampling
mask. To increase the robustness of the proposed approach, we further generate
more samples by separating the image pairs into 33×33 sub-image pairs, among
which 90% are used for updating the network dataset and the rest 10% for
validating the training process. We use three layers of convolution for the
network. The first layer consists of 64 filters with the size of
9×9,
while the second layer has 32 filters of size 5×5
and the last layer is one filter with size 5×5
.
The filter weights of each layers are initialized by random values from a
Gaussian distribution with zero mean and standard deviation 0.001. The bias is
all initialized as 0. The training takes about three days, on a workstation
equipped with 128G memory and a processor of 16 cores (Intel Xeon (R) CPU
E5-2680 V3 @2.5GHz).We evaluated the proposed approach on a fully sampled
transversal brain dataset which was acquired on a 3T scanner (SIEMENS MAGNETOM
Trio) with a 12-channel head coil by T2-weighted turbo spin-echo (TSE) sequence
(TE=91.0ms, TR=5000ms, FOV=20×20 cm, matrix=256×270, slice thickness=3mm) and a
sagittal brain image which was acquired on a GE 3T scanner (GE Healthcare,
Waukesha, WI) with a 32-channel head coil by 3D T1-weighted spoiled gradient
echo sequence (TE=minimum full, TR=7.5ms, FOV=24
24cm, matrix=256×256,
slice thickness=1.7mm). Undersampled measurements were retrospectively obtained
using the 2D Poisson disk sampling mask.
RESULTS AND DISCUSSION
The first row of Fig. 1 presents the test results of the proposed method
on the transversal brain at the acceleration of 5. It can be observed that the
zero-filled images are very blurry with some details lost. After being put
through the network, some fine structures and textures are restored and the
noise is reduced. Further combining with the online constrained imaging model,
we can reconstruct an image quite close to the reference one. For a close-up
look, the zoom-in results have also been presented for the transversal brain.
We also have presented the sagittal brain reconstruction at the acceleration of
3. According to Fig. 1k, we can see the difference image is noise-like and
consists of only the contour information. There are no obvious details and
structures lost. It demonstrates that the proposed network is capable of
restoring the details and fine structures which are discarded in the
zero-filled MR image. Furthermore, although the off-line training takes roughly
three days, under the same GPU configurations, it takes far less than 1 second
for each online MR reconstruction case.
CONCLUSION
We
propose to design and train an off-line convolutional neural network to aid
online fast MR imaging. Experimental results on two in vivo datasets have shown
that the network is capable of restoring fine structures and details while
removing noise, and have demonstrated great potential for efficient and
effective MR imaging.
Acknowledgements
Grant
support : China NSFC 61471350, the Natural Science Foundation of Guangdong
2015A020214019,
2015A030310314, the Basic
Research Program of Shenzhen JCYJ20140610152828678, JCYJ20140610151856736 and
the youth innovation project of SIAT under Y4G0071001 and US NIH R21EB020861
for Ying.References
[1] Caballero J et al. ISMRM, p1560, 2014. [2] Ravishankar S et al., IEEE TMI 30(5):1028-1041, 2011. [3] Liu Q et al. IEEE TIP,
22(12):4652-4663, 2013. [4] Haldar J et
al. ISBI, 857-860, 2011. [5] Shi C et
al. ISBI, 901–904, 2015. [6] Liang Z et
al. IEEE TMI, 22(8): 1026–1030, 2003.