Introduction

While convolutional neural networks (CNNs) have been used extensively to solve the inverse problem to convert the magnetic field to magnetic susceptibility in quantitative susceptibility mapping (QSM) ^1,2, limitations regarding their applicability in clinical settings still remain. So far, current QSM network models do not leverage known a-priori information. Known aspects like the voxel aspect ratio (VAR) or the orientation of the field-of-view (FOV) with respect to B₀ are generally not considered. Recently, incorporation of side information in network models was proposed in crowd counting with cameras³. Motivated by this approach, we demonstrate for the first time the use of adaptive convolution (AC) to incorporate a-priori information into deep learning models for QSM dipole inversion.

Theory

AC identifies changes in the input associated with the presented side information and correlates these with network parameters. When the convolutional filter weights are considered as points on a low-dimensional manifold in the high-dimensional filter weight space, the weights move on the manifold (Fig. 1). Hence, the convolution filter weights change adaptively as a function of the side information and, thus, the network is expected to learn the relationship between the susceptibility map and B-field distribution more easily. AC-layers are built from feed-forward networks to model the filter manifold (Fig. 2). The filter manifold network (FMN) computes convolutional filter weights and bias of the adaptive convolution layer. Overall, by giving side information to the AC layer, the most suitable filter weights are chosen to compute output feature maps.

Methods

A U-Net⁴ with Octave Convolution⁵ was trained to predict the 3D susceptibility map from the same-sized magnetic field distribution (Fig. 3). The network architecture of all models was five layers deep with 16 initial channels. Different placements of the AC-layer in the U-Net (Fig. 3), at the first block (First) and the first encoding block (Enc1), were tested. 1000 synthetic non-anatomical susceptibility maps (320x320x320) compromising randomly distributed shapes with susceptibilities from a Gaussian distribution (µ=0, σ=0.25) served as training data. The z-component of the voxels-size was drawn from the uniform distribution [1, 6] while the x- and y-component remained constant at one. Different FOV-orientations were simulated by randomly rotating B₀=[0,0,1]^T around the x-, y- and z-axis. The x- and y-values were drawn from the Gaussian distribution (µ=0, σ=11) and the z-values from (µ=0, σ=15). Variations in voxel-size and B₀ occurred with probability 0.8, ensuring standard parameters in the training data. The B-field distribution was obtained via fast forward convolution in k-space⁶. 64 patches of dimension 64x64x64 were randomly extracted from these datasets at each iteration for network training. All models were trained with the Adam optimizer⁷ for 170 epochs and learning rate of 0.001. The mean-squared-error served as loss metric and training was performed on an NVIDIA QUADRO RTX 6000. Validation was performed after each epoch on 100 validation datasets. The network models were evaluated on synthetic data corresponding to the training dataset to increase comparability of results and assessment.

Results

NRMSE, SSIM and visual assessment metrics (Fig. 4) show that placing the AC-layer at the first encoding block (Enc1) leads to susceptibility maps with higher similarity to the ground truth. Substantial differences to the ground truth are visible for the susceptibility map computed by the conventional model (Fig. 4). The computed susceptibility map lacks on sharp edges, well-resolved shapes, appears generally blurred and a Moiré pattern is present. The metrics of the adaptive model (Enc1) show its superiority over the conventional model.

Discussion

The parameters for the FMN were chosen based on recommendations for crowd counting³. It was assumed that a similar number of layers and parameters are required to obtain FMNs that can comprehensively learn the relationship between the side information and the associated changes in the image -the filter manifold- and are consequently able to adapt the kernel parameters. The AC-layer must be placed at a level in the network where changes in the image associated with the side information are most pronounced. The results show that this is the case by placing the AC in the first encoding block (First) as indicated by sharp edges in the activation maps (Fig. 3B). Consequently, the FMN can more easily correlate the side information with changes in the feature maps. The conventional model without AC cannot compute valuable susceptibility maps from arbitrary-orientation magnetic field distributions sampled with anisotropic voxels and would require considerably more training epochs. This implies that less data and training time is required for training AC network models. Remaining differences between reconstructed and ground truth susceptibilities can be reduced with more training epochs. Standard QSM dipole inversion algorithms make use of additional parameters to find optimal solutions. Similarly, we believe that providing a-priori information acts as constraint for parameter optimization, guiding the network towards valid solutions. In future studies, we will investigate different ways to add side information to the network, optimize the FMN and integrate AC in different network architectures.

Conclusion

Side information can be incorporated in AC network models to solve the ill-posed QSM dipole inversion. Network models including AC perform superior on datasets with arbitrary orientations and anisotropic voxel-size and offer the possibility to reduce training time.