X-ray computed tomography (CT) plays an important role in diverse clinical applications; however, radiation exposure to patients is of great concern. Low-dose CT (LDCT) is a major approach to resolve the radiation issue.

Conventional analytical CT image reconstruction methods such as filtered back-projection (FBP) [1] often provide unsatisfactory results in LDCT. The model-based image reconstruction (MBIR) methods enable better reconstruction qualities in low-dose scans, by incorporating the CT imaging physics, the measurement noise statistics, and appropriate prior models of the unknown object [2, 3, 4]. Applying the learned regularizers to MBIR has taken researchers one step closer to enabling LDCT in clinics [5, 6, 7, 8, 9]. Although MBIR methods using learned regularizers gave promising results in LDCT, their algorithmic convergence speed is not yet fully optimized [8].

Researchers also have explored image denoising deep neural network (NN) methods that remove noise in coarsely reconstructed LDCT image via good mapping capabilities of deep NNs. Examples include the residual encoder-decoder convolutional neural network (RED-CNN) [10], and the wavelet residual network (WavResNet) that learns a mapping of contourlet coefficients between low- and standard-dose image pairs [11]. The image denoising performance of such deep NNs largely depends on the quality of reference training images [12] and the number of training pairs [13]. In CT imaging, however, due to risks of high radiation exposure to patients, collecting many high-quality reference images is challenging in practice. When training samples are limited, NNs with high model complexity, e.g., deep NNs, can suffer from high overfitting risks [12, 14, 13]. The iterative neural network (INN) approach can moderate the overfitting issue by using both image refining (or denoising) NNs with low to moderate complexity, and MBIR optimization in an iterative fashion. Recently, [15] constructed the BCD-Net architecture by generalizing a block coordinate descent (BCD) algorithm that solves the MBIR problem using convolutional regularizers. Incorporating refined images via layer-wise refining NNs to MBIR modules, BCD-Net showed better generalization capability and reconstruction quality than a recent (noniterative) denoising deep NN, FBPConvNet [16], and a state-of-the-art INN method, ADMM-Net [17, 18], respectively [14]. A practical limitation of applying BCD-Net (and ADMM-Net) to LDCT is that the MBIR module at each layer needs multiple inner iterations, leading to substantial increase of the total image reconstruction time. To resolve this issue, [19] proposed a fast INN architecture, Momentum-Net, that is constructed by generalizing a block proximal extrapolated gradient MBIR algorithm using convolutional regularizers. Different from BCD-Net, Momentum-Net introduces additional extrapolation modules and uses practical closed-form MBIR solutions, which can accelerate the LDCT image reconstruction process.

This paper applies Momentum-Net to LDCT image reconstruction with appropriate model selections. At each Momentum-Net layer, we use the majorized penalized weighted least-square (PWLS) MBIR cost and a four-layer residual convolutional neural network (CNN), so-called SimpleCNN [20]. Our experiments show that the proposed MomentumNet architecture significantly improves LDCT image reconstruction accuracy over a state-of-the-art noniterative image denoising deep NN, WavResNet [11]. In addition, we investigate the performance of applying Real Spectral Normalization (RSN) [20] to image refining NN learning; however, this normalized refining NN learning does not improve the reconstruction accuracy of Momentum-Net.

The objective of CT image reconstruction is to reconstruct the unknown linear attenuation map of an object from the post-log sinogram . In the Momentum-Net architecture,each layer consists of three core modules: image refining, extrapolation, and MBIR. The abstract MomentumNet architecture is shown in Fig. 1. We then describe each module in detail.

2.1. Image refining module

We design the image refining module to provide a high-quality prior estimate for the subsequent MBIR module. In different Momentum-Net layers, we learn a different four-layer residual CNN, SimpleCNN [20]. The denoised image at the lth layer is obtained by the lth layer SimpleCNN denoiser defined as follows:

where denotes learned residual mapping at the lth layer and represents its parameter set.

Next, we investigate the effect of learning nonexpansive via the real spectral normalization (RSN) technique [20], on INN performance. We refer to this learned residual denoiser with RSN as SimpleCNN-RSN. As nonexpansive residual mapping does not guarantee nonexpansive denoiser in (1), we remove the skip connection from denoiser (1), and investigate the effect of RSNbased learning on INN performance. We refer to this learned denoiser with RSN as Dn-RSN.

In training the aforementioned NNs at each MomentumNet layer, the parameters of the current layer NN are initialized with those trained in the previous layer, except for the first layer where we adopt the random initialization. For training loss function, we use the mean-square-error (MSE) between denoised images and reference images, i.e., , where is a high-quality ref- erence image.

Fig. 1: Architecture of Momentum-Net at the lth layer [19], for represents the reconstructed image at the lth Momentum-Net layer. The initial inputs .

We apply the -relaxation strategy to image refining modules [19]:

The -relaxation strategy mixes the information of a reconstructed and denoised image, improving the MBIR performance, when the measurement noise is moderate or the imaging system is moderately ill-posed. We observed that improves LDCT image reconstruction accuracy than , where is a machine epsilon. This correspond to the sparseview CT experiment results in [19].

2.2. Extrapolation module

Many optimization literatures have shown that using the momentum of previous updates can accelerate the algorithm convergence by smoothing “zig-zagging” trajectories [21, 7]. In particular, the momentum terms, , can amplify the changes in subsequent Momentum-Net layers [19]. The extrapolation modules give extrapolated points using the momentum terms:

with and . We update the momentum coefficients as follows [19]:

where . This choice gave significant acceleration in solving several block optimization problems; see [7] and references therein.

2.3. MBIR module

We apply the PWLS method [22] to the MBIR modules of Momentum-Net. At the lth Momentum-Net layer, we consider the following MBIR problem:

where is the system matrix and W is a diagonal weighting matrix whose diagonal elements are for , with the variance of the electronic noise [23, 24]. The regularization parameter balances the data-fidelity term (the first term in (5)) and the prior estimate from the image refining module (the second term in (5)).

The MBIR module of Momentum-Net solves a majorized version of MBIR problem (5) at the extrapolated point , and the reconstructed image is obtained as

where is the majorization matrix for that is given in (5) [7, 8], the operator sets negative elements as zeros, and 1 is a vector with ones.

We trained three Momentum-Nets with different combinations of image refining NN architectures and learning methods, i.e., SimpleCNN [20], SimpleCNN-RSN, and DnRSN, as described in Sec. 2.1. We compared the performance of the trained Momentum-Nets with a state-of-the-art LDCT denoising method, WavResNet [11], and examined the effect of learning nonexpansive mappings with RSN [20] on Momentum-Net image reconstruction accuracy. We evaluated the results in terms of visual image qualities and the RMSE (root mean-square-errors) metric1.

3.1. Experimental setup

A. Imaging system and data: We evaluated the proposed method with the NIH AAPM-Mayo Clinic Low Dose CT Grand Challenge data [25]. The data includes standard-dose scans of ten patient examinations. We used the standard-dose CT images with 3 mm slice thickness as reference images, and simulated the low-dose post-log sinogram y using the Poisson-Gaussian model [24, 26]:

for , with the number of incident photons per ray, , and the variance of electronic noise, 25 [23, 26]. We generated the sinograms with 2D fan-beam CT geometry using 736 ‘detectors or rays’ 1152 ‘regularly space projection views or angles’, and no-scatter monoenergetic source. The reconstructed or denoised images are of size at a resolution of 0.69 mm 0.69 mm.

We used FBP images as (initial) inputs to MomentumNets and WavResNet. In Momentum-Net training, we used 120 training pairs from six out of ten patients: each pair consists of reconstructed low-dose images and standard-dose reference images. In training the noniterative deep NN, WavResNet, we used all the 1466 low-dose FBP and reference image

Fig. 2: RMSE minimization behavior of Momentum-Net with SimpleCNN: (Left) RMSEs of three examples, (Right) Mean RMSE of 12 test images.

Table 1: Mean and standard deviation (STD) of RMSE (HU) of 12 reconstructed test images with different methods.

pairs from the same six training patients, in order to reduce overfitting risks. We tested all these methods with another 12 low-dose scans collected from the remaining four patients.

B. Momentum-Net parameters and training: We set in the Momentum-Net extrapolation modules. In MBIR modules, we used the adaptive regularization parameter selection scheme based on the spectral radius [19] to provide a distinct value for each sample. We chose 119 for the desired factor .

In training the three Momentum-Nets, we decreased the learning rate from by a ratio of 0.9 every ten epochs, and used Adam [27] to optimize network parameters. We performed image-based training with the mini-batch size of 5 and 100 epochs in each Momentum-Net layer. The three Momentum-Nets used the same dimensions of convolutional kernels as [20], i.e., the first layer uses 64 filters of size , the second and third layer uses 64 filters of size , and the last layer uses one filter of size to reconstruct the output.

3.2. Proposed Momentum-Net with SimpleCNN

Fig. 2 shows that the proposed Momentum-Net with SimpleCNN decreases RMSE dramatically in the first 30 layers, and tends to converge in 50 layers. The Momentum-Net reduces the mean RMSE value by 4.5 HU and gives smaller standard deviations in RMSE, compared to WavResNet, as reported in Table 1. This implies that the proposed MomentumNet with SimpleCNN can improve both the accuracy and stability of low-dose CT image reconstruction than a state-of-the-art image denoising deep NN, WavResNet. The proposed Momentum-Net with SimpleCNN better removes noise and streak artifacts than WavResNet. It also provides clearer reconstructions of some details; see, in Fig. 3, the boundaries

Fig. 3: Three examples (from top to bottom) of the reconstructed testing images using Momentum-Net with SimpleCNN (the second column), with SimpleCNN-RSN (the third column), and with Dn-RSN (the fourth column). The compared WavResNet denoised images are shown in the first column, and the reference images are in the fifth column. See their FBP images in Fig. 4.

Fig. 4: FBP images of test examples.

shown in the zoomed region at the top-right corner in the first example, the arrow pointed structures in zoomed areas of the second example, and the arrow pointed tissues in the left zoomed region in the third example.

3.3. Momentum-Nets involving RSN-based training

We show the reconstructed examples by Momentum-Net with SimpleCNN-RSN and Dn-RSN in the third and fourth columns of Fig. 3 respectively. Comparing the first three and the last columns in Fig. 3, we observe that Momentum-Net with SimpleCNN-RSN provides generally noisier reconstructions than WavResNet and Momentum-Net with SimpleCNN. However, Momentum-Net with SimpleCNN-RSN sometimes can provide clearer details than WavResNet. For example, in the right zoomed box of the second example, Momentum-Net with SimpleCNN-RSN shows better reconstruction quality for the arrow pointed structures than WavResNet, and in the left zoomed box in the third row, the former gives clearer small tissues marked by red arrows than the latter. Table 1 reports that Momentum-Net with SimpleCNN-RSN is approximately 2.9 RMSE (HU) higher than WavResNet, while it has smaller standard deviations. This implies that Momentum-Net with SimpleCNN-RSN is more stable than WavResNet, although it may not provide better image qualities. Momentum-Net with Dn-RSN, however, provides the worst visual and numerical results among the compared four methods in this paper.

We proposed an accurate and stable Momentum-Net architecture for LDCT image reconstruction. By applying a four-layer residual CNN to image refining modules, we achieved significant reconstruction improvements in both numerical and visual results, compared with a state-of-the-art noniterative LDCT denoising method WavResNet [11]. We additionally investigated how does learning nonexpansive mappings with the spectral normalization technique RSN affect Momentum-Net performance. Applying RSN to learning either residual mapping or image denoising NNs does not improve the reconstruction performance of Momentum-Net in low-dose CT. In the future, we will investigate other training techniques that imposes some mathematical conditions to image refining NNs. We are also interested in developing methods to further accelerate the Momentum-Net convergence in LDCT image reconstruction.

The authors thank Dr. Cynthia McCollough, the Mayo Clinic, the American Association of Physicists in Medicine, and the National Institute of Biomedical Imaging and Bioengineering for providing the Mayo Clinic data.

[1] L. A . Feldkamp, L. C. Davis, and J. W. Kress, “Practical cone- beam algorithm,” J. Opt. Soc. Amer. A, Opt. Image Sci., vol. 1, no. 6, pp. 612–619, 1984.

[2] J. A. Fessler, “Statistical image reconstruction methods for transmission tomography,” in Handbook of Medical Imaging, Volume 2. Medical Image Processing and Analysis, M. Sonka and J. Michael Fitzpatrick, Eds., pp. 1–70. SPIE, Bellingham, 2000.

[3] I. A. Elbakri and J. A. Fessler, “Statistical image reconstruction for polyenergetic X-ray computed tomography,” IEEE Trans. Med. Imag., vol. 21, no. 2, pp. 89–99, 2002.

[4] J. Nuyts, B. De Man, J. A. Fessler, W. Zbijewski, and F. J. Beekman, “Modelling the physics in the iterative reconstruction for transmission computed tomography,” Phys. Med. Biol., vol. 58, no. 12, pp. R63, 2013.

[5] Q. Xu, H. Yu, X. Mou, L. Zhang, J. Hsieh, and G. Wang, “Low- dose X-ray CT reconstruction via dictionary learning,” IEEE Trans. Med. Imag., vol. 31, no. 9, pp. 1682–97, Sept. 2012.

[6] X. Zheng, S. Ravishankar, Y. Long, and J. A. Fessler, “PWLS- ULTRA: An efficient clustering and learning-based approach for low-dose 3D CT image reconstruction,” IEEE Trans. Med. Imag., vol. 37, no. 6, pp. 1498–1510, 2018.

[7] I. Y. Chun and J. A. Fessler, “Convolutional dictionary learn- ing: Acceleration and convergence,” IEEE Trans. Image Process., vol. 27, no. 4, pp. 1697–1712, Apr. 2018.

[8] I. Y. Chun and J. A. Fessler, “Convolutional analysis operator learning: Acceleration and convergence,” IEEE Trans. Image Process., vol. 29, no. 1, pp. 2108–2122, 2020.

[9] I. Y. Chun, D. Hong, B. Adcock, and J. A. Fessler, “Convolutional analysis operator learning: Dependence on training data,” IEEE Signal Process. Lett., vol. 26, no. 8, pp. 1137– 1141, 2019.

[10] H. Chen, Y. Zhang, M. K. Kalra, F. Lin, Y. Chen, P. Liao, J. Zhou, and G. Wang, “Low-dose CT with a residual encoder-decoder convolutional neural network,” IEEE Trans. Med. Imag., vol. 36, no. 12, pp. 2524–2535, 2017.

[11] E. Kang, W. Chang, J. Yoo, and J. C. Ye, “Deep convolutional framelet denoising for low-dose CT via wavelet residual network,” IEEE Trans. Med. Imag., vol. 37, no. 6, pp. 1358–1369, 2018.

[12] X. Zheng, I. Y. Chun, Z. Li, Y. Long, and J. A. Fessler, “Sparse- view X-ray CT reconstruction using prior with learned transform,” submitted, Feb. 2019.

[13] H. Lim, I. Y. Chun, Y. K. Dewaraja, and J. A. Fessler, “Im- proved low-count quantitative PET reconstruction with an iterative neural network,” submitted, May 2019.

[14] I. Y. Chun, X. Zheng, Y. Long, and J. A. Fessler, “BCD-Net for low- dose CT reconstruction: Acceleration, convergence, and generalization,” in Proc. Med. Image Computing and Computer Assist. Interven., Shenzhen, China, Oct. 2019.

[15] I. Y. Chun and J. A Fessler, “Deep BCD-net using identical encoding-decoding CNN structures for iterative image recovery,” in Proc. IEEE IVMSP Workshop, Zagori, Greece, Jun. 2018, pp. 1–5.

[16] K. H. Jin, M. T. McCann, E. Froustey, and M. Unser, “Deep convolutional neural network for inverse problems in imaging,” IEEE Trans. Image Process., vol. 26, no. 9, pp. 4509–4522, 2017.

[17] Y. Yang, J. Sun, H. Li, and Z. Xu, “Deep ADMM-Net for compressive sensing MRI,” in Proc. NIPS 29, Long Beach, CA, Dec. 2016, pp. 10–18.

[18] S. H Chan, X. Wang, and O. A Elgendy, “Plug-and-play ADMM for image restoration: Fixed-point convergence and applications,” IEEE Trans. Comput. Imag., vol. 3, no. 1, pp. 84–98, Nov. 2017.

[19] I. Y. Chun, Z. Huang, H. Lim, and J. A. Fessler, “Momentum- Net: Fast and convergent iterative neural network for inverse problems,” IEEE Trans. Pattern Anal. Mach. Intell., 2020.

[20] E. Ryu, J. Liu, S. Wang, X. Chen, Z. Wang, and W. Yin, “Plug- and-play methods provably converge with properly trained denoisers,” in International Conference on Machine Learning (ICML), 2019, pp. 5546–5557.

[21] Y. Nesterov, “Gradient methods for minimizing composite functions,” Math. Program., vol. 140, no. 1, pp. 125–161, 2013.

[22] J. A. Fessler, “Penalized weighted least-squares image reconstruction for positron emission tomography,” IEEE Trans. Med. Imag., vol. 13, no. 2, pp. 290–300, Jun. 1994.

[23] L. Fu, T. C. Lee, S. M. Kim, A. M. Alessio, P. E. Kinahan, Z. Q. Chang, K. Sauer, M. K. Kalra, and B. De Man, “Comparison between pre-log and post-log statistical models in ultra-low-dose CT reconstruction,” IEEE Trans. Med. Imag., vol. 36, no. 3, pp. 707–720, 2017.

[24] I. Y. Chun and T. Talavage, “Efficient compressed sensing sta- tistical X-ray CT reconstruction from fewer measurements,” in Proc. Intl. Mtg. on Fully 3D Image Recon. in Rad. and Nuc. Med, Lake Tahoe, CA, Jun. 2013, pp. 30–33.

[25] C. McCollough, “TU-FG-207A-04: Overview of the low dose CT grand challenge.,” Med. Phys., vol. 43, no. 2, pp. 3759–60, 2016.

[26] S. Ye, S. Ravishankar, Y. Long, and J. A. Fessler, “SPULTRA: Low-dose CT image reconstruction with joint statistical and learned image models,” IEEE Trans. on Med. Imag., 2019.

[27] D. P. Kingma and J. L. Ba, “Adam: A method for stochastic optimization,” in Proc. ICLR 2015, San Diego, CA, May 2015, pp. 1–15.