1991

A Complexity Guidance based Dynamic Activation Network for Phase Image Processing
Zeyu Liao1 and Lijun Bao1
1Department of Electronic Science, Xiamen University, Xiamen, China

Synopsis

Keywords: Data Processing, Machine Learning/Artificial Intelligence

Motivation: Existing phase processing methods often require users to trade off between time and precision. Therefore, a phase processing network that can dynamically activate different parts based on input samples is of great research value.

Goal(s): We hope that the proposed network can adaptively determine whether to begin with VOI extraction, i.e., removing the brain skull, and provide different solutions for samples of different complexity.

Approach: We combine dynamic neural network and deformable convolution in the network design to realize dynamic activation and verify it on MRI phase data.

Results: Our dynamic activation based network (DANet) implements adaptive phase processing and achieve competitive performance.

Impact: Our methodological framework can be applied across various field related to phase signal processing, such as Optical Interferometry (OI), Magnetic Resonance Imaging (MRI), Fringe Projection Profilometry (FPP), and Interferometric Synthetic Aperture Radar (InSAR).

INTRODUCTION

In many imaging or measurement techniques, the estimation of true phase is an important yet challenging problem. Due to the use of four-quadrant arctangent function for signal values in phase images, the obtained raw phase values are confined to the range of (-π, π]. The mathematical relationship for phase unwrapping is as: $$φ_{real}= φ_{wrap}+k*2π,k∈Z$$φreal represents the real phase, while φwrap represents the wrapped phase. k denotes wrap counts, which quantifies the number of times the phase has wrapped. Optimization based phase unwrapping methods can be divided into regional growth methods1 and laplacian methods2, the former has long processing time, and the latter has obvious shortcomings in performance such as low estimations. Deep learning-based phase unwrapping methods3,4 often accelerate processing speed. However, increase of parameters will be necessary when higher performance is required.

METHOD

Figure 1 provides a visual representation of the overall framework, while details of module in DANet are shown in Figure 2. Inspired by dynamic neural networks5 and deformable convolution6, the proposed DANet is designed to fulfill two tasks, i.e. VOI extraction and phase signal unwrapping. It consists of following components:
Judge: A lightweight module that determines whether to activate the mask stage contains a transformer encoder block.
Mask Stage: This stage is dedicated to VOI extraction, i.e., removing the brain skull.
Partition Stage: The role of this stage is to partition wrap complexity by predicting the wrap count k and generate a heatmap C.
Dynamic Gate: A gate operation with no parameters, determines whether the current sample skips restore stage based on the heatmap C.
Restore Stage: At this stage, the phase image is finally restored. And the heatmap C can control the shape of deformable convolution.
In essence, when input data do not require VOI extraction and the phase problem is not overly complex, only the partition stage is activated for phase unwrapping. In the most complex case, all three stages are activated. This dynamic adaptability of our framework allows to flexibly handle various tasks and achieve adaptive performance. We define N to represent the number of distinct classes. In the judge module and mask stage, N is set to 2, indicating a binary classification. In the partition stage, N is set to 21, suggesting a multi-class classification with 21 distinct levels of wrap complexity in heatmap C, i.e., corresponding to wrap counts of -9 to 11.

RESULT

Figure 3 shows the complete data flow of DANet. If the input data needsVOI extraction, the judge module will output signal 1 to activate the mask stage; otherwise, the output signal 0 lets input data directly enter the partition stage. At the partition stage, a heatmap C is generated to distinguish the relevant areas of different wrapping counts. C has two key roles: firstly, it can represent the complexity and enable dynamic gate to the activation of restore stage; on the other hand, C is employed to create a set of one-hot maps and be sent to the restore stage to accelerate the convergence of deformable convolution. In figure 3, the loss curves of each part are also shown.
Figure 4 is the result of mask stage. We can find that the addition of deformable convolution enhances model ability to extract mask and the model can correct some errors in labels. In addition, the appropriate threshold selection is also important for mask stage. Figure 5 shows a comparison with other phase processing methods of Laplacian2, DLPU7, and PU-M-Net8. The results demonstrate that our proposed DANet has better performance. Furthermore, we computed average quantitative metrics on the entire test set in Table 1, which include the Root Mean Square Error (RMSE), the Mean Absolute Percentage Error (MAPE), the Peak Signal-to-Noise Ratio (PSNR), and the Structural Similarity Index (SSIM).

CONCLUSION

We propose a complexity guidance based dynamic activation network (DANet) for phase image processing. The network introduces a lightweight module and a data complexity heatmap C to achieve dynamic activation. At the same time, we embed deformable convolutions at appropriate locations in the network, and use the complexity heatmap C to speed up shape convergence. Our DANet demonstrates excellent performance with respects to dynamic adaptability and precision.

Acknowledgements

This work was supported in part by the National Natural Science Foundation of China under Grant 62071405.

References

[1] Karsa A, Shmueli K, et al. SEGUE: A speedy region-growing algorithm for unwrapping estimated phase. IEEE Transactions on Medical Imaging. 2018;38(6):1347-1357.
[2] Schofield M, Zhu Y. Fast phase unwrapping algorithm for interferometric applications. Optics letters. 2003;28(14):1194-1196.
[3] Zhu W, Beroza G. PhaseNet: A deep-neural-network-based seismic arrival-time picking method. Geophysical Journal International. 2019;216(1):261-273.
[4] Spoorthi G, Gorthi R, Gorthi S. PhaseNet 2.0: Phase unwrapping of noisy data based on deep learning approach. IEEE transactions on image processing. 2020;29:4862-4872.
[5] Han Y, Huang G, Song S, et al. Dynamic neural networks: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2021;44(11):7436-7456.
[6] Dai J, Qi H, Xiong Y, et al. Deformable convolutional networks. Proceedings of the IEEE international conference on computer vision. 2017;764-773.
[7] Wang K, Li Y, Kemao Q, et al. One-step robust deep learning phase unwrapping. Optics express. 2019; 27(10):15100-15115.
[8] Xu M, Tang C, Shen Y, et al. PU-M-Net for phase unwrapping with speckle reduction and structure protection in ESPI. Optics and Lasers in Engineering. 2022;151:106824.

Figures

Fig.1. The proposed DANet framework. All three stages are composed of residual blocks, skip connections, and two or three times of convolution downsampling. The judge module consists of a single-layer ViT encoder, and the dynamic gate is a parameter-free operation. The mask stage and restore stage of the network utilize deformable convolutions to enhance shape capturing ability, but the blocks used in these two stages are different. The latter leverages the complexity heatmap generated by the partition stage to accelerate the convergence of deformable convolutions.

Fig.2. The detail of the proposed DANet are as follows: (a) DC3, representing 3x3x3 deformable convolution. (b) Conv3, representing standard 3x3x3 convolution. (c) The deformable convolution module takes two input sources: the current feature map data and the offset generation information source P. (d) The judge module comprises a ViT encoder layer, a Global Adaptive Pooling (GAP) layer and a linear layer. (e) P is no longer the same as the data but is instead a multi-channel complexity heatmap.

Fig.3. The complete data flow and loss curves of DANet. If the input data needs to be VOI extracted, the judge module will output signal 1 to activate the mask stage; otherwise, the output signal 0 makes the input data directly enter the partition stage. At the partition stage, a heatmap C is generated for dynamically gating the activation of the final restore stage and accelerate the convergence of deformable convolution in restore stage.

Fig.4. Illustration of the results from the mask stage in DANet. In the figure, mag represents the magnitude image, label signifies the ground truth, pred denotes the network's output and the bottom row represents the difference map. Threshold settings of 0.8 and 0.3 made a noticeable distinction in results. The red box area shows that the network combined with deformable convolution fixes part of the label error.

Fig.5. Comparative experiments. We assess the performance of various phase unwrapping methods, including traditional optimization techniques like Laplacian phase unwrapping, as well as deep learning approaches such as DLPU and PU-M-Net. The top row displays, the original phase data, the results produced by each method, and the data labels. The bottom line shows the 3D visualization of: (a) wrapped phase (b) complexity heatmap (c) real phase.

Proc. Intl. Soc. Mag. Reson. Med. 32 (2024)
1991
DOI: https://doi.org/10.58530/2024/1991