4199
MRzero –- Automated invention of MRI sequences using supervised learning1Max Planck Institute for Biological Cybernetics, Tübingen, Germany, 2Max Planck Institute for Intelligent Systems, Tübingen, Germany, 3Eberhard Karls University Tübingen, Tübingen, Germany, 4University Clinic Erlangen, Erlangen, Germany
Synopsis
We propose a framework — MRzero — that allows automatic invention of MR sequences. At the core of the framework is a differentiable forward process allowing to simulate image measurement and reconstruction. The sequence parameters are variables of optimization. As a cost function we use mean squared error distance to a certain given target contrast of interest. To avoid overfitting we propose a method that generates synthetic data that is used for training. In the experiments, we demonstrate the ability of the method to learn RF flip angles and spatial encoding from scratch given a target obtained with GRE sequence.
INTRODUCTION
In the context of MR in medicine, generating contrast between tissues is of central importance. A direct relation of the MR image contrast and the actual MR sequence with its many free parameters raises a question if both image and contrast generation can be performed in a completely automatic manner. Multiple recent research studies addressed this task by optimization of RF pulses and image acquisition parameters1,2,3,4,5. The principal idea different from the previous paradigm would be to use a certain desired image contrast as a target, and then automatically create the MR sequence and signal-to-image reconstruction routine that is able to generate this contrast using optimization techniques.METHODS
In this work we address the problem of automated sequence design and optimization. The approach that we propose allows optimization over spatial encoding gradients (2D), excitation flip angles and their phases, and timing of different events within a sequence affecting the relaxation weighting. The optimization is carried out in an MR scanner simulation environment (implemented in Torch8 ) mirroring the acquisition at a real MR scanner. The forward process is differentiable in all parameters, and supports an efficient analytic derivative-driven non-linear optimization. Figure 1 outlines the general method pipeline and problem definition. The pipeline consists of two modules: acquisition and reconstruction. The input to the pipeline are the spin system characterization variables and the output is the image. Details on the MRzero implementation are provided in (6).To avoid overfitting, we use a synthetic dataset of target/input pairs, and try to learn a scanner function that minimizes the combined error over all training pairs. Each sample from such database is a block of voxels of non-zero PD at varying spatial locations (see Figure 3). Additionally, we vary T1/T2/B0 values of voxels within each block. We then make a forward pass subject to a target sequence of choice to produce target images that we want to learn to output with our scanner function. The measurements were performed on a PRISMA 3T scanner (Siemens Healthineers, Erlangen Germany) using a 2 and a 24 channel Head Coil.
RESULTS
In the first example, the target is a Cartesian gradient echo image with constant flip angle of 5 degree, as visible in column 6 of Figure 2. A sequence with 96 acquisitions is initialized with all gradients and flip angles set to zero. An additional SAR penalty was added to the cost function. During the training, the image error with respect to the target decays from 1000% to 10% (Figure 2f) by inventing suitable x- and y-gradients, as well as non-zero RF amplitudes and phases. The obtained sequence is non-Cartesian and shows a complex RF pattern. Despite this, there is a close match with the target image in the simulation (Figure 2c) and in the real measurement of the corresponding phantom (Figure 2d).Interestingly, although we optimized our sequence w.r.t. to a phantom image we did not observe image artifacts when measuring in vivo (Figure 2e). Still, we are aware that depending on type of the sequence being learned we can overfit when training just on a single sample (an example is given in (6)). To make the method more general and reduce the dependency on comprehensive large databases of PD/T1/T2 values we have used a sparse synthetic dataset approach (see Figure 3). A database of input and target blocks at different positions and with different MR properties (1024 samples) was generated and used in the training phase. At every optimizer step a minibatch containing a single sample randomly drawn from the database was used to compute the loss and update the parameters. We show just a single sample from the database as a target; in fact, there is a unique target for each of the 1024 examples. Generalization of the learned sequence to both phantom and in vivo measurement could be validated (Figure 3de).
DISCUSSION
Optimization of MR sequences is a well-studied problem and the novelty of this work is to establish a very general framework for optimization, but solely using the target contrast of interest directly as an optimization target. In more formal language, sequence generation is formulated as a supervised learning problem, where the term supervised learning refers to the process of learning a function that maps an input to an output, based on a database of input-output pairs. These pairs are given by (i) the signals of voxels with certain MR properties, generated and encoded using the end-to-end simulation, and (ii) the corresponding target voxel signal. In this way MRzero explores a novel way of generating MR sequence and reconstruction, which has limitless possibilities e.g. quantification, segmentation, as well as contrasts of other imaging modalities.CONCLUSION
We have developed a fully automated MRI sequence generator based on the Bloch equation simulations and supervised learning. We were able to achieve a close agreement between simulated results and real measurements. The approach was verified in several experiments in which the functionality of the method was demonstrated. The differentiable Bloch simulation of many samples is computationally demanding, but manageable, and the flexibility and generality of the approach opens many opportunities.Acknowledgements
We thank Maxim Zaitsev and his team for developing and sharing puseq toolbox7 . This research would not be possible without this versatile and powerful framework.
Funding:
- Deutsche Forschungsgemeinschaft DFG SCHE 658/12
- ERC Advanced Grant No 834940
- European Union’s Horizon 2020 research and innovation program 667510
- German Research Foundation DFG ZA 814/2-1
References
1. Roux PL, HinksRS. Stabilization of echo amplitudes in FSE sequences. Magnetic resonance in medicine 1993; 30(2):183–190.
2. Shinnar M, Eleff S, Subramanian H, Leigh JS. The synthesis of pulse sequences yielding arbitrary magnetization vectors. Magnetic resonance in medicine 1989;12(1):74–80.
3. Hargreaves BA, Nishimura DG, Conolly SM. Time-optimal multidimensional gradient waveform design for rapid imaging. Magnetic Resonance in Medicine: An Official Journal of the International Society for Magnetic Resonance in Medicine 2004;51(1):81–92.
4. Lustig M, Kim SJ, Pauly JM. A fast method for designing time-optimal gradient waveforms for arbitrary k-space trajectories. IEEE transactions on medical imaging 2008;27(6):866–873.
5. Zhu, AUTOmated pulse SEQuence generation (AUTOSEQ) and neural network decodingf or fast quantitative MR parameter measurement using continuous and simultaneous RF transmit and receive; 2019.
6. Loktyushin A, Herz K, Dang N, et al. MRzero -- Fully automated invention of MRI sequences using supervised learning. arXiv:2002.04265 [physics] 2020.
7. Layton KJ, Kroboth S, Jia F, Littin S, Yu H, Leupold J, et.al. Pulseq: A rapid and hardware-independent pulse sequence prototyping framework. Magnetic Resonance in Medicine 2017; 77(4): 1544–1552.
8. Paszke A, Gross S, Chintala S, Chanan G, Yang E, DeVito Z, et.al., Automatic differentiation in PyTorch; 2017.
Figures
