Many DL networks have been proposed for noise and artifact reduction. Convolutional neural networks (CNN), which has a small receptive field, is widely used for noise reduction because Gaussian noise is incoherent and position-independent. However, only the networks with a larger receptive field are often adopted to improve artifacts because most of the artifacts are distributed globally. In this section, we provide an overview of deep learning architecture
commonly used for noise and artifact reduction.
Single-scale CNN (SCNN), which is widely used in the various fields shown in Fig. 1, is efficient in improving noisy images. SCNN consists of more than one convolution layer without a down- or up-sampling structure, such as pooling and stride. In most cases, residual components such as noise or artifact images are used as the output because it is well-known that the manifold of the residual components has a simpler structure compared to that of the clean image components
, resulting in faster conversion and better generalization of the network. Denoising convolutional neural networks (DnCNN) is also a thriving network used for image denoising through a process of learning residual components of images, which is noise. Besides these, batch normalization is used to improve training time for the network. ResNet is commonly used to achieve a complex and deeper structure of the network to overcome the vanishing gradient problem. ResNet is also used for artifact reduction because it has a large receptive field that covers whole input images owing to a large number of layers
Autoencoders can extract clean features from noisy images, similar to the principal component analysis procedure, which is used to extract signifi-cant information from high-dimensional datasets Given that Gondara first developed an autoencoderbased denoising filter for X-ray images
, many studies have been proposed for CT, MRI, and others, recently
. Autoencoder, which has a different mechanism compared to SCNN, compresses the entire image information into a low-dimensional structure to effectively learn underlying manifold. Unfortunately, autoencoders may lose important information such as edge or fine structure, if the input image is not a redundant representation.
U-net, often used for image segmentation, has a large receptive field by utilizing multi-scale featuresThis network has asymmetric up- and down-sampling structures with a skip connection, which concatenates each layer, as shown in Fig. 2. The skip connection contributes to capturing localized features such as the fine structure of images. For the artifact reduction, U-net is the most popular network because the large receptive field is generally required to extract the artifact that is commonly observed globally. The study by Lee et al. implied that U-net gave a better de-aliasing performance as compared to SCNN
although a relatively large number of trainable parameters is required for U-net.
Generative adversarial networks (GAN) approach is a promising learning technique for removing noise
Figure 1: Example of a single-scale CNN structure consisting of multiple convolution layers. Each convolution layer has a set of filters, which is followed by activation function such as ReLU, to extract features of input images. The residual learning approach is usually adopted for noise and artifact reduction. Noise or artifact reduced images are calculated by subtracting the output of the network from input images.
and artifacts. GAN consists of two separate networks: generator and discriminator. Generator is the network that creates the output images, noiseor artifact-free images, whereas discriminator acts as a classifier to determine whether the generated output is real or fake. Feedback from the discriminator contributes to updating the network of the generator to create convincing fake images to fool the discriminator. U-net is widely used for the generator.
Many applications have been proposed using the above successful DL networks for noise and artifact reduction. Although most parts of them are still research stage, promising results were demonstrated with a limited size of datasets. In this section, we introduce state-of-art applications shown in Tab. 1.
Some DL networks focus on denoising for brain MR imaging. Bermudez et al. proposed an autoencoder with skip connections for T1-weighted (T1w) imaging of the brain. The developed network, trained with 528 T1w images, significantly improved the image quality based on PSNR analysis. DnCNN is also widely used for denoising of brain MR images
. Denoising using DL is used not only for the improvement of image quality but also for reducing scan time. Currently, MR images with higher resolutions are required to achieve better diagnostic performance. However, improvement of patient experience, particularly scan time, in the MR scanner is also an urgent priority. To overcome these conflict-ing problems, many DL-based denoising approaches
have been developed. Kawamura et al. proposed the DnCNN-based approach for denoising multi-shot diffusion-weighted imaging for the brain. The study showed that highly swift DWI could be achieved by reducing the number of excitations (NEX), as shown in Fig. 3. Kidoh et al. proposed the SCNN network, which can be used for multiple sequences such as T1w, T2w, FLAIR, and MPRAGE. The proposed network utilizes a discrete cosine transform (DCT) layer to separate zero and high-frequency components. The separated high-frequency components go through 22 convolution layers, whereas the zerofrequency component is connected to the last layer. This structure may contribute to robust denoising performance because it doesnt use contrast information of images. Imaging can be achieved at a pace of more than twofold by using the proposed method.
Moreover, Arterial spin labeling (ASL) perfusion imaging, which generally suffers from low SNR, with DL denoising is also a promising application. Xie et al. developed the DL denoising approach for ASL using a wide interface network (WIN)
. The WIN is a residual learning CNN with a large size of convolution kernel and filters. Quantitative analysis based on SNR suggested that the proposed method can reduce the scan time for ASL by reducing NEX. Kim et al. proposed a network consisting of two separate pathways for extracting multi-scale features
The pathway with convolution kernel, batch normalization, and ReLU is used for the local structure of images. Meanwhile, the other pathway uses a dilated pathway to increase the receptive field in order to extract large-scale information. Evaluation by a radiologist implied that the ASL images with low NEX
Figure 2: Overview of U-net, which has up- and down-sampling structure - the same as autoencoder - used for noise and artifact reduction. Skip connections by copying features from earlier layers into later layers are used to preserve high-frequency information, which is often lost in the down-sampling step. This network is commonly used for removing artifact reduction, and requires a larger receptive field compared to noise reduction.
were significantly improved by using this proposed CNN method.
Further, Streak artifact reduction is one of the most successful applications using DL. There are many studies for reducing streak artifact for computed tomography (CT) images because streak artifact on CT is a very common undesirable image degradation
. Recently, radial sampling for abdominal MR imaging is getting attention because of its robustness against respiratory induced motion
. In the case of radial sampling, streak artifact can be observed in MR images as well as CT. Han et al. developed U-net to remove streak artifacts for the brain and abdominal MR images; a network adopted residual learning with pre-training using CT and MRI datasets. This study indicated that the DL-based method could be superior to conventional compressed sensing algorithms.
Several studies of retrospective and blinded motion artifact correction techniques for the brainc-spine
, upper abdomen
, and cardiac
imaging have been proposed. Pawar et al. proposed the motion artifact reduction technique using the U-net for MPRAGE images of the brain. Evaluation based on naturalness image quality evaluator (NIQE)
, which is reference-free image quality metrics, revealed the proposed method improved the motion artifact. Tamada et al. also proposed a motion artifact reduction for DCE-MRI of the liver using DnCNN-based network with multi-channel input and output. A training dataset was generated by simulating the respiratory motion. The results
indicated the DL approach possessed the ability to remove artifacts, as show in Fig. 4. It is a challenge to acquire artifact-free images under the influence of cardiac motion. Oksuz et al. proposed the reconstruction technique for cardiac imaging by combining the 3D CNN and the recurrent convolutional neural network (RCNN)used for 2D spatiotemporal imaging. The 3D CNN is adopted to detect the corrupted k-space lines by the motion whereas RCNN removes the motion artifact coming from the corrupted lines.
Besides, some applications are used for removing aliasing artifact alternatives to CS reconstruction. CS reconstruction, which is a fast MRI technique using random undersampling and redundancy of MR images, generally requires long computation time. DL approaches can contribute to the reduction of computation cost for the undersampled datasets. A de-aliasing method using the U-net for undersampled k-space was proposed to achieve efficient reconstruction by Hyun et al.
. Yang et al. proposed a GAN-based reconstruction method for undersampled images
. Recently, a model-based DL network, which has unrolled architecture, was proposed to reduce trainable parameters of the network
The DL-based approach is also useful in solving other complicated problems such as removing Gibbs, and banding artifact
et al. proposed the CNN-based Gibbs artifact, which can be an alternative to conventional k-space domain filtering methods
. The results indicated that the proposed method removed artifacts without addi-
Figure 3: Specimen images of denoising brain DWI using DnCNN, the Noise2Noise approach. This network includes 17 convolution layers with batch normalization and ReLU. Multi-shot DWI images (a) acquired with NEX of 1 were denoised using the network. PSNR analysis between the denoised and high-NEX images indicated that the network successfully reduced the noise component.
tional blurring, which is often observed in traditional methods. Kwon proposed metal artifact reduction using the U-net and bipolar readout imaging to improve blurring and distortion caused by a strong off-resonance effect. Seo et al. developed the CNN to correct metal artifact of MR images acquired using slice encoding for metal artifact correction
In this paper, we summarized DL networks and its applications used for noise and artifact reduction. Typical networks including SCNN, U-net, autoencoder, GAN architectures, which have characteristics of structure different from each other, were explained. Further, we reviewed applications using these networks. The studies reviewed here demonstrate the excellent performance of noise and artifact reduction for MRI. These results revealed that DL-base approaches can be used to remove complicated artifacts such as aliasing, streaking, and so on, which are still challenging problems using conventional methods.
Although many promising techniques have been proposed, challenges of removing noise and artifact using DL exist. To begin with, it is challenging to prepare training datasets for some cases. For noise reduction, images with a large number of NEX are often used as clean datasets. However, motions, such as respiratory, cardiac, and CSF pulsation, during a scan can lead to blurring and undesirable image degradation. Noise2Noise approach could be one of the practical solutions for noise reduction
less, datasets generated by simulation are generally used for artifact reduction. In this case, the discrepancy between the simulated and acquired datasets leads to poor filtering performance.
Furthermore, it is difficult to conduct appropriate image quality evaluation for noise and artifact reduction. MSE, PSNR, and SSIM are widely used to evaluate the image quality, particularly for noise reduction networks. Conversely, there is difficulty in using these conventional metrics for artifact reduction because it is challenging to prepare ground truth datasets that have no artifact images. To evaluate the image quality without ground truth, reference-free metrics such as Blind/Referenceless Image Spatial Quality Evaluatorreasonable alternative to conventional metrics. However, clinical evaluation is essential for practical use because these metrics offer excellent performance for natural images. It is essential to pay attention to the contrast, texture, and fine structure after DL-based filtering because inappropriate training, such as the use of a small number of datasets and wrong noise or artifact modeling, could lead to a critical change of contexture of images.
In this study, we reviewed DL-based methods to reduce noise and artifact for MR images. Because of the recent advancement of deep learning technology, a wide variety of techniques are proposed to serve these purposes. However, even though DL-based methods
Figure 4: Sample depictions of aliasing reduction of the liver using multi-channel DnCNN, which have seven layers of convolution layers. Multi-contrast images obtained from DCE-MRI were used for the input of the network. MR images with respiratory artifact and blurring were significantly improved by via this network. The acquisition was implemented using the multi-phase 3D T1-weighted spoiled gradient-echo sequence with a dual-echo bipolar readout.
are getting popular in the MR community owing to its excellent performance and low computational cost, there are still issues to be solved for practical use. Nevertheless, some applications are overcoming these setbacks and are about to come onto the clinical site as products from MR vendors. The DL-based method will therefore be widely used in a range of MR applications shortly.
1. Bellon EM, Haacke EM, Coleman PE, Sacco DC, Steiger DA, Gangarosa RE. MR artifacts: a review. American Journal of Roentgenology 1986; 147:1271-1281.
2. Balafar M. Review of noise reducing algorithms for brain MRI images. methods 2012; 10:11.
3. Chandarana H, Feng L, Block TK, et al. Freebreathing contrast-enhanced multiphase MRI of the liver using a combination of compressed sensing, parallel imaging, and golden-angle radial sampling 2013; 48.
4. Chandarana H, Block KT, Winfeld MJ, et al. Free-breathing contrast-enhanced T1-weighted gradient-echo imaging with radial k-space sampling for paediatric abdominopelvic MRI 2014; 24:320-326.
5. Chandarana H, Block TK, Rosenkrantz AB, et al. Free-breathing radial 3D fat-suppressed T1-weighted gradient echo sequence: a viable alternative for contrast-enhanced liver imaging in patients unable to suspend respiration 2011; 46:648-653.
6. Feng L, Grimm R, Block KT, et al. Golde-
nangle radial sparse parallel MRI: combination of compressed sensing, parallel imaging, and goldenangle radial sampling for fast and flexible dynamic volumetric MRI 2014; 72:707-717.
7. Feng L, Axel L, Chandarana H, Block KT, Sodickson DK, Otazo RJMrim. XDGRASP: goldenangle radial MRI with reconstruction of extra motionstate dimensions using compressed sensing 2016; 75:775-788.
8. Chavhan GB, Babyn PS, Vasanawala SSJR. Abdominal MR imaging in children: motion compensation, sequence optimization, and protocol organization 2013; 33:703-719.
9. Zhang T, Chowdhury S, Lustig M, et al. Clinical performance of contrast enhanced abdominal pediatric MRI with fast combined parallel imaging compressed sensing reconstruction 2014; 40:13-25.
10. Cheng JY, Alley MT, Cunningham CH, Vasanawala SS, Pauly JM, Lustig MJMrim. Nonrigid motion correction in 3D using autofocusing withlocalized linear translations 2012; 68:1785-1797.
11. Lundervold AS, Lundervold A. An overview of deep learning in medical imaging focusing on MRI. Zeitschrift fr Medizinische Physik 2019; 29:102-127.
12. Kim J, Kwon Lee J, Mu Lee K. Accurate image super-resolution using very deep convolutional networks. Proceedings of the IEEE conference on computer vision and pattern recognition, 2016; 1646-1654.
13. Zhang K, Zuo W, Chen Y, Meng D, Zhang LJIToIP. Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising 2017; 26:3142-3155.
14. Lee D, Yoo J, Ye JC. Deep residual learning for compressed sensing MRI. Biomedical Imaging (ISBI 2017), 2017 IEEE 14th International Symposium on, 2017; 15-18.
15. Ghodrati V, Shao J, Bydder M, et al. MR image reconstruction using deep learning: evaluation of network structure and loss functions. Quantitative imaging in medicine and surgery 2019; 9:1516.
16. Vincent P, Larochelle H, Bengio Y, Manzagol P-A. Extracting and composing robust features with denoising autoencoders. Proceedings of the 25th international conference on Machine learning, 2008; 1096-1103.
17. Gondara L. Medical image denoising using convolutional denoising autoencoders. 2016 IEEE 16th International Conference on Data Mining Workshops (ICDMW), 2016; 241-246.
18. Nishio M, Nagashima C, Hirabayashi S, et al. Convolutional auto-encoder for image denoising of ultra-low-dose CT. Heliyon 2017; 3:e00393.
19. Bermudez C, Plassard AJ, Davis LT, Newton AT, Resnick SM, Landman BA. Learning implicit brain MRI manifolds with deep learning. Medical Imaging 2018: Image Processing, 2018; 105741L.
20. Ronneberger O, Fischer P, Brox T. U-net: Convolutional networks for biomedical image segmentation. International Conference on Medical image computing and computer-assisted intervention, 2015; 234-241.
21. Goodfellow I, Pouget-Abadie J, Mirza M, et al. Generative adversarial nets. Advances in neural information processing systems, 2014; 2672-2680.
22. Jiang D, Dou W, Vosters L, Xu X, Sun Y, Tan T. Denoising of 3D magnetic resonance images with multi-channel residual learning of convolutional neural network. Japanese journal of radiology 2018; 36:566-574.
23. Rao GS, Srinivas B. De-noising of MRI Brain Tumor image using Deep Convolutional Neural Network. Available at SSRN 3357284 2019.
24. Kidoh M, Shinoda K, Kitajima M, et al. Deep Learning Based Noise Reduction for Brain MR Imaging: Tests on Phantoms and Healthy Volunteers. Magnetic Resonance in Medical Sciences 2019:mp. 2019-0018.
25. Xie D, Bai L, Wang Z. Denoising arterial spin labeling cerebral blood flow images using deep learning. arXiv preprint arXiv:180109672 2018.
26. Kim KH, Choi SH, Park S-H. Improving arterial spin labeling by using deep learning. Radiology 2018; 287:658-666.
27. Han Y, Yoo J, Kim HH, Shin HJ, Sung K, Ye JC. Deep learning with domain adaptation for
accelerated projectionreconstruction MR. Magnetic resonance in medicine 2018; 80:1189-1205.
28. Gjesteby L, Yang Q, Xi Y, et al. Reducing metal streak artifacts in CT images via deep learning: Pilot results. The 14th international meeting on fully three-dimensional image reconstruction in radiology and nuclear medicine, 2017; 611-614.
29. Han YS, Yoo J, Ye JC. Deep residual learning for compressed sensing CT reconstruction via persistent homology analysis. arXiv preprint arXiv:161106391 2016.
30. Han Y, Ye JC. Framing U-Net via deep convolutional framelets: Application to sparse-view CT. Ieee T Med Imaging 2018; 37:1418-1429.
31. Pawar K, Chen Z, Shah NJ, Egan GF. Moconet: Motion correction in 3D MPRAGE images using a convolutional neural network approach. arXiv preprint arXiv:180710831 2018.
32. Duffy BA, Zhang W, Tang H, et al. Retrospective correction of motion artifact affected structural MRI images using deep learning of simulated motion 2018.
33. Johnson PM, Drangova M. Motion correction in MRI using deep learning. ISMRM Scientific Meeting & Exhibition, Honolulu, 2018; 4098.
34. Lee H, Ryu K, Nam Y, Lee J, Kim D-H. Reduction of respiratory motion artifact in c-spine imaging using deep learning: Is substitution of navigator possible? ISMRM Scientific Meeting & Exhibition, 2018; 2660.
35. Tamada D, Kromrey M-L, Ichikawa S, Onishi H, Motosugi U. Motion Artifact Reduction Using a Convolutional Neural Network for Dynamic Contrast Enhanced MR Imaging of the Liver. Magnetic Resonance in Medical Sciences 2020; 19:64-76.
36. Tamada D, Onishi H, Motosugi U. Motion Artifact Reduction in Abdominal MR Imaging using the U-NET Network. ICMRM and Scientific Meeting of KSMRM, Seoul, Korea, 2018; PP0311.
37. Jiang W, Liu Z, Lee K-H, et al. Respiratory motion correction in abdominal MRI using a densely connected U-Net with GAN-guided training. arXiv preprint arXiv:190609745 2019.
38. Oksuz I, Clough J, Ruijsink B, et al. Detection and Correction of Cardiac MRI Motion Artefacts During Reconstruction from k-space. International Conference on Medical Image Computing and Computer-Assisted Intervention, 2019; 695-703.
39. Mittal A, Soundararajan R, Bovik AC. Making a “completely blind” image quality analyzer. IEEE Signal Processing Letters 2012; 20:209-212.
40. Qin C, Schlemper J, Caballero J, Price AN, Hajnal JV, Rueckert D. Convolutional recurrent neu-
ral networks for dynamic MR image reconstruction. Ieee T Med Imaging 2018; 38:280-290.
41. Hyun CM, Kim HP, Lee SM, Lee S, Seo JKJPim, biology. Deep learning for undersampled MRI reconstruction 2018.
42. Yang G, Yu S, Dong H, et al. DAGAN: Deep de-aliasing generative adversarial networks for fast compressed sensing MRI reconstruction. Ieee T Med Imaging 2017; 37:1310-1321.
43. Aggarwal HK, Mani MP, Jacob M. MoDL: Model-based deep learning architecture for inverse problems. Ieee T Med Imaging 2018; 38:394-405.
44. Zhang Q, Ruan G, Yang W, Zhao K, Wu EX, Feng Y. Gibbs-Ringing Artifact Reduction in MRI via Machine Learning Using Convolutional Neural Network. ISMRM 2018, 2018; 0429.
45. Zhang Q, Ruan G, Yang W, et al. MRI Gibbsringing artifact reduction by means of machine learning using convolutional neural networks. Magnetic resonance in medicine 2019; 82:2133-2145.
46. Muckley MJ, Ades-Aron B, Papaioannou A, et al. Training a Neural Network for Gibbs and Noise Removal in Diffusion MRI. arXiv preprint arXiv:190504176 2019.
47. Zhao X, Zhang H, Zhou Y, Bian W, Zhang T, Zou X. Gibbs-ringing artifact suppression with knowledge transfer from natural images to MR images. Multimedia Tools and Applications 2019:1-23.
48. Kwon K, Kim D, Park H. A Learning-Based Metal Artifacts Correction Method for MRI Using Dual-Polarity Readout Gradients and Simulated Data. International Conference on Medical Image Computing and Computer-Assisted Intervention, 2018; 189-197.
49. Seo S, Do WJ, Luu HM, Kim KH, Choi SH, Park SH. Artificial neural network for Slice Encoding for Metal Artifact Correction (SEMAC) MRI. Magnetic Resonance in Medicine 2019.
50. Kim JW, Kwon K, Kim B, Park H. Attention Guided Metal Artifact Correction in MRI using Deep Neural Networks. arXiv preprint arXiv:191008705 2019.
51. Kim KH, Park S-H. Artificial neural network for suppression of banding artifacts in balanced steady-state free precession MRI. Magnetic resonance imaging 2017; 37:139-146.
52. Lehtinen J, Munkberg J, Hasselgren J, et al. Noise2noise: Learning image restoration without clean data. arXiv preprint arXiv:180304189 2018.
53. Mittal A, Moorthy AK, Bovik AC. Noreference image quality assessment in the spatial domain. IEEE Transactions on image processing 2012; 21:4695-4708.
Table 1: Overview of DL studies for noise and artifact reduction