3959

Consistency, ablation, and scalability studies of DeepRF
Dongmyung Shin1, Jiye Kim1, Juhyung Park1, and Jongho Lee1
1Electrical and Computer Engineering, Seoul National University, Seoul, Korea, Republic of

Synopsis

A recently developed RF pulse design method, DeepRF, is investigated using consistency, ablation, and scalability studies. The consistency of DeepRF designs is confirmed by repeating the same slice-selective inversion pulse design. The importance of the combination of two modules in DeepRF (i.e., RF generation and RF refinement) is verified through the ablation of each module. The scalability of DeepRF for a range of a design parameter is validated by designing several slice-selective inversion pulses with different time-bandwidth products.

Introduction

Deep reinforcement learning (DRL) has demonstrated that an artificial intelligence (AI) agent can master complex games (e.g., Go and StarCraft) by repetitively playing self-matches1,2. Using a similar approach, Shin et al. recently demonstrated that an AI agent can design several types of radiofrequency (RF) pulses3,4. In this work, we investigated the following questions: Does the design result change a lot for repetitions? Is each module, RF generation and RF refinement, critical for the final results? Can DeepRF provide a reasonable improvement over a range of design parameters?

Methods

[DeepRF] In DeepRF, RF pulses of a desired profile are initially generated from a recurrent neural network (RNN) agent using DRL (RF generation module, Fig 1a) and then subsequently refined using a neural network-like computational graph5, referred to as Bloch graph (RF refinement module, Fig 1b). In our setting, 76,800 episodes (1 episode = 1 RF) were performed for one execution of DRL. Since the results of DRL are well-known to be sensitive to initialization6, 50 DRL executions with random initializations were performed to reduce the variance of the DRL results. Therefore, a total of 3,840,000 (= 76,8000 × 50) RF pulses are generated using the RF generation module. Then, the top 256 RF pulses were selected and further optimized in the RF refinement module. For each RF, 10,000 iterations of the refinement were performed and the best pulse (e.g., RF with minimum specific absorption rate (SAR)) was chosen as the final design.
[Consistency study] To check the consistency of the design results of DeepRF, we performed three repetitions of DeepRF designs and compared these results. The design objective of DeepRF was to match the simulated profile of an SLR inversion pulse7 (time-bandwidth product (TBW) = 4.3, duration = 5.12 ms, passband/stopband ripples = 1%) while penalizing SAR.
[Ablation study] To test the importance of each module in DeepRF (Fig. 1), we ablated each module and performed three repetitions of the above inversion pulse design using the RF generation module or RF refinement module. The initial input pulses for the RF refinement module were RF pulses of random shapes.
[Scalability study] To verify the scalability of DeepRF for a range of a design parameter, four SLR inversion pulses with different TBWs (4.3, 5.6, 6.8, and 8.1) were designed (duration = 12.8 ms, passband/stopband ripples = 1%), and the corresponding DeepRF pulses of the matched profiles were designed with the SAR regularization.

Results

The results of three repetitions of the DeepRF designs are shown in Fig. 2. The pulse shapes of the DeepRF pulses (1st and 2nd columns in Fig. 2) show similarity with each other and the SARs of the DeepRF pulses are the same for all the repetitions (17.5 (mG)2sec) and smaller than that of SLR pulse (19.6 (mG)2sec). The slice profiles (3rd and 4th columns in Fig. 2) from the DeepRF and SLR pulses perfectly match. The ripple constraints (= 1%) are satisfied in all cases (5th column in Fig. 2).
The DeepRF design results with the ablation of each module are shown in Fig. 3 and Fig. 4. When the RF refinement module was ablated (Fig. 3), the pulse shapes from the ablation (1st and 2nd columns in Fig. 3) are inconsistent, and the slice profiles from the ablation and SLR were different (3rd and 4th columns in Fig. 3). The SARs of the ablation results are higher than the original DeepRF results (22.8 or 23.8 or 22.7 vs. 17.5 (mG)2sec). When the RF generation module is ablated (Fig. 4), although the slice profiles from the ablation and SLR were almost the same (3rd and 4th columns in Fig. 4), the pulse shapes from the ablation (1st and 2nd columns in Fig. 4) are inconsistent. The SARs of the ablation results are higher than the original DeepRF results (18.1 or 18.0 vs. 17.5 (mG)2sec), demonstrating the necessity of combining the RF generation and RF refinement modules.
Finally, the DeepRF design results with the different TBWs are shown in Fig. 5. For all the designs, the pulse shapes of the DeepRF pulses are clearly different from those of the SLR pulses (1st and 2nd columns in Fig. 5). The slice profiles from the DeepRF and SLR pulses are almost identical (3rd and 4th columns in Fig. 5). The SARs of the DeepRF pulses are smaller than those of the SLR pulses in all TBWs (1st column in Fig. 5).

Discussion and Conclusions

In summary, our study demonstrates that the DeepRF designs are fairly consistent for repetitions, and the DeepRF designs with the ablation of the RF generation module or RF refinement module were inferior to the original design. Additionally, the DeepRF designs were successfully performed over a range of the design parameter, TBW. Those results suggest that DeepRF is a reliable and scalable RF pulse design method.

Acknowledgements

This work was supported in part by the National Research Foundation of Korea under Grant NRF-2018R1A2B3008445 and in part by the Samsung Research Funding and Incubation Center of Samsung Electronics under Project SRFC-IT1801-09.

References

[1] Silver D, Schrittwieser J, Simonyan K, et al. Mastering the game of go without human knowledge. Nature. 2017;550:354-359.

[2] Vinyals O, Babuschkin I, Czarnecki WM, et al. Grandmaster level in StarCraft II using multi-agent reinforcement learning. Nature. 2019;575:350-354.

[3] Shin D, Ji S, Lee D, et al. Deep Reinforcement Learning Designed Shinnar-Le Roux RF Pulse using Root-Flipping: DeepRFSLR. IEEE Trans Med Imag. 2020;39(12):4391-4400.

[4] Shin D and Lee J. DeepRF: Designing an RF pulse using a self-learning machine. 28th ISMRM Virtual Conference & Exhibition. August 08-14, 2020.

[5] Baydin AG, Pearlmutter BA, Radul AA, et al. Automatic differentiation in machine learning: a survey. J Mach Learn Res. 2017;18(1):5595-5637.

[6] Henderson P, Islam R, Bachman P, et al. Deep reinforcement learning that matters. 32nd AAAI Conference on Artificial Intelligence. February 2-7, 2018.

[7] Pauly J, Le Roux P, Nishimura D, et al. Parameter relations for the Shinnar-Le Roux selective excitation pulse design algorithm. IEEE Trans Med Imag. 1991;10(1):53-65.

Figures

An overview of DeepRF. (a) In the RF generation module, a series of RF values (i.e., actions) are generated from the RNN agent to shape an RF envelope (Nth RF), and the virtual MRI simulates a slice profile. Then, a value of the objective function (e.g., a difference between simulated and desired profiles) is calculated from which the agent changes its behavior to generate a next RF pulse ((N+1)th RF). (b) In the RF refinement module, an RF pulse is refined (Mth RF to (M+1)th RF) with respect to the objective function using RF value changes (∆RF) calculated from the Bloch graph.

The results of three repetitions of DeepRF to check the consistency of the design results. The design objective of DeepRF was to match the simulated profile of an SLR inversion pulse while penalizing SAR. The pulse shapes of the DeepRF pulses (1st and 2nd columns) show similarity with each other, and the SARs of the DeepRF pulses are smaller than those of the SLR pulses (17.5 vs. 19.6 (mG)2sec). The slice profiles (3rd and 4th columns) from the DeepRF and SLR pulses perfectly match. The ripple constraints (= 1%) are satisfied in all cases (5th column).

The results of three repetitions of DeepRF with the ablation of the RF refinement module. The pulse shapes from the ablation (1st and 2nd columns) are inconsistent, and the slice profiles from the ablation and SLR were different (3rd and 4th columns). The SARs of the ablation results are higher than the original DeepRF results (22.8 or 23.8 or 22.7 vs. 17.5 (mG)2sec)

The results of three repetitions of DeepRF with the ablation of the RF generation module. As initial input pulses for the RF refinement module, RF pulses of random shapes were used. Although the slice profiles from the ablation and SLR were almost the same (3rd and 4th columns), the pulse shapes from the ablation (1st and 2nd columns) are inconsistent. The SARs of the ablation results are higher than the original DeepRF results (18.1 or 18.0 vs. 17.5 (mG)2sec).

The results of the DeepRF and SLR pulse designs with different TBWs (4.3, 5.6, 6.8, and 8.1). For all the designs, the pulse shapes of the DeepRF pulses are clearly different from those of the SLR pulses (1st and 2nd columns). The slice profiles from the DeepRF and SLR pulses are almost identical (3rd and 4th columns). The SARs of the DeepRF pulses are smaller than those of the SLR pulses in all TBWs (1st column).

Proc. Intl. Soc. Mag. Reson. Med. 29 (2021)
3959