Edith Franziska Baader^{1}, Fabian Theißen^{1}, Nicola Pezzotti^{2,3}, and Volkmar Schulz^{1,4,5,6}

^{1}Department of Physics of Molecular Imaging Systems, Institute for Experimental Molecular Imaging, RWTH Aachen University, Aachen, Germany, ^{2}Philips Research, Eindhoven, Netherlands, ^{3}Department of Mathematics and Computer Science, Eindhoven University of Technology, Eindhoven, Netherlands, ^{4}Physics Institute III B, RWTH Aachen University, Aachen, Germany, ^{5}Fraunhofer Institute for Digital Medicine MEVIS, Bremen, Germany, ^{6}Hyperion Hybrid Imaging Systems GmbH, Aachen, Germany

One approach to accelerate MRI scans is to acquire fewer k-space samples. Commonly, the sampling pattern is selected before the scan, ignoring the sequential nature of the sampling process. A field of machine learning addressing sequential decision processes is reinforcement learning (RL). We present an approach for creating adaptive two-dimensional (2D) k-space trajectories using RL and the so-called action space shaping. The trained RL algorithm adapts to a variety of basic 2D shapes outperforming simple baseline trajectories. By shaping the action space of the RL agent we achieve better generalization and interpretability of the agent.

Previous research on adaptive sampling approaches focuses on choosing the next k-space line in Cartesian readout schemes

For the implementation of the environment, we use the toolkit Open AI Gym

The agent is trained for 50 million time steps with a set of 18 simulated k-spaces of size 45x45 using phantoms of basic geometric shapes generated by an adaptation of the geometric shapes generator proposed by El Korchi et al.

Furthermore, the constraints on the k-space trajectory can be adapted straightforwardly by changing the action mask. For example, to speed up the acquisition, valid k-space pixels might be at a certain radius from the current position and the trajectory is continued by a straight line to this pixel. This is possible thanks to our action space shaping approach, in which the probabilities are computed for all k-space pixels.

1. Sutton R S, Barto A G. Reinforcement learning: an introduction. The MIT Press, second edition; 2018

2. Bakker T, van Hoof H, Welling M. Experimental design for MRI by greedy policy search. arXiv:2010.16262. 2020.

3. Jin K H, Unser M, Yi K M. Self-supervised deep active accelerated MRI. arXiv:1901.04547. 2019.

4. Pineda L et al. Active MR k-space sampling with reinforcement learning. MICCAI 2020; pp 23-33.

5. Walker-Samuel S. Using deep reinforcement learning to actively, adaptively and autonomously control a simulated MRI. ISMRM 2019.

6. Zhang Z et al. Reducing uncertainty in undersampled MRI reconstruction with active acquisition. 2019 IEEE/CVF CVPR. 2019; pp 2049-2053.

7. Yin T et al. End-to-end sequential sampling and reconstruction for MR imaging. arXiv:2105.06460. 2021.

8. Kanervisto A, Scheller C, Hautamäki V. Action Space Shaping in Deep Reinforcement Learning. 2020 IEEE CoG. 2020; pp 479-486.

9. Tang C-Y, Liu C-H, Chen W-K, You S D. Implementing action mask in proximal policy optimization (PPO) algorithm. ICT Express. 2020; 6(3):200-203.

10. Brockman G et al. Openai gym. arXiv:1606.01540. 2016.

11. Mnih V et al. Asynchronous Methods for Deep Reinforcement Learning. arXiv:1602.01783. 2016.

12. Hill A et al. Stable Baselines. https://github.com/hill-a/stable-baselines; 2018.

13. El Korchi A, Ghanou Y. 2D geometric shapes dataset - for machine learning and pattern recognition. Data in Brief. 2020; 32:106090.

14. Tong G et al. Virtual scanner: MRI on a browser. Journal of Open Source Software. 2019; 4(43):1637.

DOI: https://doi.org/10.58530/2022/2453