Đang tải... (xem toàn văn)
Image denoising plays a crucial role as a preprocessing step in the analysis of medical images. Over the past three decades, various algorithms have been proposed, each exhibiting different denoising performances. More recently, deep learning-based models have demonstrated remarkable success, surpassing traditional methods. However, these advanced models face limitations, particularly in terms of demanding large training sample sizes and incurring high computational costs. In this paper, we address these challenges and propose a novel approach utilizing denoising autoencoders constructed with convolutional layers. Our method is distinctive in its ability to efficiently denoise medical images even when trained on a small sample size. We demonstrate the effectiveness of our approach by combining heterogeneous images, thereby boosting the effective sample size and subsequently enhancing denoising performance. One notable advantage of our method is its adaptability to the complexities of medical imaging. Even with the simplest network architectures, our denoising autoencoders exhibit an exceptional ability to reconstruct images, even in scenarios where corruption levels are so high that noise and signal become indistinguishable to the human eye. Through this research, we aim to provide a practical and effective solution for medical image denoising that overcomes the limitations associated with large training datasets and computational resource requirements commonly observed in deep learning-based models.
Trang 1UNIVERSITY OF INFORMATION TECHNOLOGY COMPUTER SCIENCE
PROJECT REPORT
CS406.O11.KHCL
PROJECT: Image Denoising using AutoEncoder
Full Name: Nguyễn Hà Anh Vũ ID: 21520531
Ho Chi Minh City, January 10, 2024
Trang 2Acknowledgment
First and foremost, I would like to express my deep gratitude to you, Professor, for your outstanding support and guidance throughout the Image Processing and Applications course (CS406.O11.KHCL) This has truly enriched my learning experience and made it more meaningful
You not only imparted the course material with clarity but also shared profound knowledge and practical experiences, allowing me to gain a deeper understanding of this field
Moreover, your facilitation of hands-on activities and application of knowledge through world projects has enabled me to develop practical skills and implement the learned concepts in a professional context
real-Once again, I want to extend my sincere appreciation to Professor Mai Tiến Dũng for your dedication and passion in teaching I look forward to continuing my learning journey under your guidance in the upcoming courses
Thank you sincerely Ho Chi Minh City, January 10, 2024
Trang 3Table of Contents
A Introduction 4
I.Overview 5
1.Introduction to Digital Image Processing 5
2.Differences between Image Processing and Computer Vision 6
3.The history of image processing 6
4.Abstraction 7
5.Motivation 7
6.Related work 8
II.Problem Identification 9
1.Input: Image with Noise 9
2.Output: Denoised Images 9
2)Challenges in Image Denoising: 11
3)Significance of Medical Image Denoising: 12
4)Impact on Diagnosis and Treatment: 12
1.Digital Dental Periapical X-Ray Database for Caries Screening 18
2.The mini-MIAS database of mammograms 19
II.Choosing Hyperparameters for Model Training 20
Trang 41.Choosing Epochs: 20
2.Choosing Batch Size: 20
3.Choosing Learning Rate: 21
4.Choosing Optimizer: 21
5.Choosing Loss Function: 21
Autoencoder Model Architecture: 21
Training Configuration: 22
III.Metrics 22
1.Peak Signal-to-Noise Ratio (PSNR): 22
2.Structural Similarity Index (SSIM): 23
3.Why Choose PSNR and SSIM: 24
4.Why not choose MSE: 24
Picture 2: Key Stages in Digital Image Processing 5
Picture 3: Differences between Image Processing & Computer Vision 6
Picture 4: (“Image denoising: Can plain neural networks compete with BM3D?.” (CVPR), 2012) 8
Picture 5: Examples for Input 9
Picture 6: Examples for Output 9
Picture 7: Noisy image 11
Picture 8: Denoising Medical Image 11
Picture 9: Basic autoencoder 13
Picture 10: stacked denoising autoencoder 14
Picture 11: Methodology 15
Picture 12:adding noise to data 16
Picture 13: "Digital Dental Periapical X-Ray Database for Caries Screening" 19
Picture 14: The mini-MIAS database of mammograms 20
Picture 15: Model Architecture 22
Picture 16: Why not choose MSE 24
Picture 17: Result adding 2 noise 25
Picture 18: Result on Dental (Not augmentation) 25
Picture 19: Result on MIAS (Not augmentation) 26
Picture 20: Result on Denal (Augmentation) 27
Picture 21: Result on MIAS (Augmentation) 27
Picture 22: Gadio.app 28
Picture 23: Demo 29
Trang 5A Introduction
I Overview
1 Introduction to Digital Image Processing
Image processing is a significant field in computer science and information technology, primarily focusing on the processing, transformation, and understanding of information from images and videos The main goal of image processing is to extract useful information from images and videos, making them easy to read, analyze, or use in various applications
In the realm of digital image processing, the field concentrates on two major tasks: • Improvement of pictorial information for human interpretation
• Processing of image data for storage, transmission, and representation for autonomous machine perception
These tasks involve applying methods and algorithms to enhance image quality, such as increasing resolution, improving contrast, noise reduction, and various other processing tasks to make image information clearer and more manageable Concurrently, image processing also involves the analysis and extraction of feature information from images, aiding computers in understanding the content of images and videos automatically
The continuum from image processing to computer vision can be broken up into low-, mid- and high-level processes
Picture 1: Levels of Image Processing
There are some Key Stages in Digital Image Processing:
Picture 2: Key Stages in Digital Image Processing
Enhancement, is a facet that focuses on minimizing or eliminating noise from an image
Trang 62 Differences between Image Processing and Computer Vision
Attribute Image Processing Computer Vision Definition
A field of computer science and
technology focusing on the processing, transformation, and understanding of information from images and videos
A branch of artificial intelligence aimed at enabling computers to understand and solve visual tasks similar to humans
Output Processed images (enhanced, filtered, etc.)
Information or decisions based on the recognition and understanding of visual content
Illustrative Example
- Improving image quality (resolution, contrast)
- Removing noise from images (Image Denoising)
- …
- Face recognition in images or videos
- Object classification in images (animals, objects)
- …
Picture 3: Differences between Image Processing & Computer Vision
3 The history of image processing
The history of image processing began with early efforts to understand and utilize images, and it has undergone significant advancements over the decades Below is a summary of the development history of image processing:
b) Preliminary Image Processing Era (1970s - 1980s):
The advent of digital computers opened up the possibility of digital image processing and the implementation of more complex algorithms
Methods like low-pass filtering and high-pass filtering were developed to enhance image quality
Trang 7c) Breakthroughs in Computer Vision (1980s - 1990s):
Computer vision began to emerge as an independent field, concentrating on image recognition and understanding
Algorithms such as Hough Transform, Edge Detection, and Segmentations appeared, paving the way for new applications
d) Advancements in Statistical Methods and Machine Learning (2000s - Present):
The prevalence of statistical methods and machine learning changed the landscape of image processing, with the emergence of deep learning models
Convolutional Neural Networks (CNNs) demonstrated outstanding performance in various computer vision tasks, from object recognition to medical image processing
e) Wide-Spread Applications and Future Prospects (Present - Ongoing):
Image processing has become a crucial component in various fields, including healthcare, autonomous vehicles, security, and many others
Ongoing research continues to focus on developing advanced image processing methods and implementing them in real-world applications
4 Abstraction
Image denoising plays a crucial role as a preprocessing step in the analysis of medical images Over the past three decades, various algorithms have been proposed, each exhibiting different denoising performances More recently, deep learning-based models have demonstrated remarkable success, surpassing traditional methods However, these advanced models face limitations, particularly in terms of demanding large training sample sizes and incurring high computational costs
In this paper, we address these challenges and propose a novel approach utilizing denoising autoencoders constructed with convolutional layers Our method is distinctive in its ability to efficiently denoise medical images even when trained on a small sample size We demonstrate the effectiveness of our approach by combining heterogeneous images, thereby boosting the effective sample size and subsequently enhancing denoising performance
One notable advantage of our method is its adaptability to the complexities of medical imaging Even with the simplest network architectures, our denoising autoencoders exhibit an exceptional ability to reconstruct images, even in scenarios where corruption levels are so high that noise and signal become indistinguishable to the human eye
Through this research, we aim to provide a practical and effective solution for medical image denoising that overcomes the limitations associated with large training datasets and computational resource requirements commonly observed in deep learning-based models
5 Motivation
In the realm of medical imaging, achieving precise diagnoses and conducting thorough analyses of medical images necessitate a high level of accuracy and detail Unfortunately, these images are frequently marred by noise originating from diverse factors Addressing this significant challenge has become imperative, and I am motivated to leverage the capabilities of deep learning to tackle this issue
Trang 8The inherent complexity and intricacy of medical images demand sophisticated solutions for effective denoising By harnessing the power of autoencoders, a type of neural network, I aim to develop a robust and efficient method for image denoising in medical imaging Autoencoders have demonstrated remarkable capabilities in learning complex patterns and representations, making them well-suited for handling the intricate nature of medical image data
My motivation stems from the realization that a successful image denoising approach can substantially enhance the accuracy of diagnostic procedures and contribute to more reliable medical analyses Through the utilization of autoencoders, I aspire to bring about advancements in the field, ultimately leading to improved image quality and facilitating more accurate medical assessments
6 Related work
The landscape of image denoising techniques has witnessed significant advancements, with various approaches demonstrating remarkable capabilities in addressing noise-related challenges Notably, BM3D has been regarded as state-of-the-art in image denoising, characterized by its well-engineered methodology However, Burger et al challenged this notion by showcasing that a simple multi-layer perceptron (MLP) can achieve denoising performance comparable to BM3D
Picture 4: (“Image denoising: Can plain neural networks compete with BM3D?.” (CVPR), 2012)
An addition to the image denoising is the introduction of denoising autoencoders Serving as fundamental components for deep networks, these autoencoders, as extended by Vincent et al., offer a novel approach to image denoising The concept involves stacking denoising autoencoders to construct deep networks, where the output of one denoising autoencoder is fed as input to the subsequent layer
Jain et al proposed image denoising using convolutional neural networks (CNNs), demonstrating that even with a small sample of training images, performance on par or superior to state-of-the-art methods based on wavelets and Markov random fields can be
Trang 9achieved Additionally, Xie et al leveraged stacked sparse autoencoders for both image denoising and inpainting, showcasing performance comparable to K-SVD
Agostenelli et al explored the application of adaptive multi-column deep neural networks for image denoising, constructed through a combination of stacked sparse autoencoders This innovative system demonstrated robustness across various noise types The collective findings from these related works highlight the versatility and effectiveness of different autoencoder-based approaches, including MLPs, convolutional neural networks, and stacked sparse autoencoders, in the domain of image denoising
II Problem Identification
1 Input: Image with Noise
The primary input of problem is a grayscale image that are affected by noise Noise can be introduced during the image acquisition process or due to other environmental factors
Picture 5: Examples for Input
2 Output: Denoised Images
-> The desired output is a denoised image where the unwanted noise has been effectively removed while preserving the essential features and details in the images
Picture 6: Examples for Output
3 Constraints:
a) Limited Training Data: Constraints on the size of the training dataset may limit the
number of available images for model training This can be a challenge, especially in medical data where images are scarce
b) Model Depth: Constraints on the number of layers in the autoencoder can be applied to
control the complexity of the model and mitigate the risk of overfitting
c) Model Stability: Ensuring that the model is not overly complex to avoid overfitting and
maintain stable performance on new data
d) Acceptable Training Time: Limiting the training time of the model can be an important
constraint, especially when computational resources are limited
e) Optimal Performance: Constraints on the model's performance, particularly achieving
high denoising performance on various types of noise and different lighting conditions
Trang 10f) Flexibility: The model needs to be flexible and adaptable to various types of medical
images and imaging conditions
g) Noise Tolerance: The model should have the ability to handle and denoise images in
noisy conditions without losing important information
h) Scalability: The model should be scalable to apply to a large volume of diverse medical
images without requiring extensive re-tuning
4 Requirments:
a) High Denoising Accuracy: The autoencoder should demonstrate high accuracy in
denoising images, effectively removing noise while preserving essential details
b) Adaptability to Various Image Types: The model should be able to adapt to different
types of medical images, considering variations in imaging modalities and structures
c) Efficient Handling of Limited Training Data: The autoencoder should be designed to
effectively learn from a limited dataset, considering potential constraints on the availability of labeled training images
d) Flexibility in Model Architecture: The architecture of the autoencoder should be
flexible, allowing adjustments to the number of layers and neurons to achieve optimal denoising performance
e) Robustness to Varied Lighting Conditions: The autoencoder should be robust to
changes in lighting conditions, providing consistent denoising performance across different levels of illumination
f) Interpretability of Results: The denoising results should be interpretable and should
not introduce artifacts or distortions that could mislead medical professionals during image analysis
g) Scalability: The model should be scalable to handle a growing dataset and potential
advancements in imaging technology without requiring significant reconfiguration
h) Compatibility with Existing Infrastructure: The integration of the autoencoder into
existing medical imaging systems or workflows should be seamless and compatible with established infrastructure
i) User-Friendly Interface for Training and Evaluation: If applicable, there should be
a user-friendly interface for training the autoencoder and evaluating its denoising performance, making it accessible to practitioners without extensive machine learning expertise.
Trang 11B PRELIMINARIES
I Noisy image
A " Noisy image " refers to an image that has been corrupted, containing unwanted or random components added from various sources Noise can appear in an image due to various factors such as poor lighting conditions, inaccurate sensor equipment, or imperfect data transmission processes
Picture 7: Noisy image
Where is the noisy image produced as a sum of original image x and some noise z
Images with noise often appear blurry and unclear, reducing the quality of the original image and posing challenges in analyzing and interpreting information within the image The goal of the Image Denoising problem is to utilize an Autoencoder model to eliminate or minimize these noisy components, reconstructing the original image with improved quality while retaining essential information
II Denoising
1) Overview:
Denoising image, the process of removing noise or unwanted artifacts from images, plays a crucial role in various scientific fields In particular, Medical Image Denoising is of paramount importance due to its direct impact on diagnostic accuracy and the overall reliability of medical imaging
Picture 8: Denoising Medical Image
2) Challenges in Image Denoising:
In scientific research, especially in disciplines such as astronomy, biology, and materials science, images captured through various instruments are often contaminated with noise This noise can obscure critical details and affect the accuracy of subsequent analyses Addressing these challenges requires advanced denoising techniques capable of preserving essential features while eliminating unwanted distortions
Trang 123) Significance of Medical Image Denoising:
In the medical field, where image quality directly influences diagnostic decisions, Medical Image Denoising is a critical step in enhancing the clarity and precision of medical imaging modalities such as X-rays, CT scans, MRIs, and ultrasound The importance of this process extends to various medical applications, including disease detection, treatment planning, and surgical guidance
4) Impact on Diagnosis and Treatment:
- Enhanced Visibility: Denoising ensures that medical images are free from artifacts,
allowing healthcare professionals to have a clearer view of anatomical structures and abnormalities
- Improved Diagnostic Accuracy: Clean and precise images contribute to accurate
diagnosis, reducing the likelihood of misinterpretations or missed diagnoses
- Optimized Treatment Planning: Medical Image Denoising aids in the planning of surgical
procedures, radiation therapy, and other interventions by providing clinicians with quality images for detailed analysis
high-5) Technological Advances:
Recent advancements in denoising techniques, particularly the application of deep learning algorithms such as convolutional neural networks (CNNs) and autoencoders, have significantly improved the efficacy of image denoising These methods can adaptively learn complex patterns in medical images, making them valuable tools in the pursuit of high-quality diagnostic imaging
III Autoencoder
An autoencoder is a type of neural network that tries to learn an approximation to identity function using backpropagation, given a set of unlabeled training inputs x(1), x(2), , x(n) , it uses z(i) = x(i)
An autoencoder first takes an input x ∈ [0,1]d and maps(encode) it to a hidden representation y ∈ [0, 1]d′ using deterministic mapping, such as
y = s(W x + b)
where s can be any non linear function Latent representation y is then mapped back(decode) into a reconstruction z, which is of same shape as x using similar mapping
z = s(W′y + b′)
Model parameters (W,W′,b,b′) are optimized to minimize reconstruction error prime symbol is not a matrix transpose Model parameters (W,W′,b,b′) are optimized
Trang 13to minimize recon- struction error, which can be assessed using different loss functions such as squared error or cross-entropy.
Picture 9: Basic autoencoder
Layer L1 is input which is encoded in layer L2 using latent representation and input is reconstructed at L3 Using number of hidden units lower than inputs forces autoencoder to learn a compressed approximation Mostly an autoencoder learns low dimensional representation very similar to Principal Component Analysis (PCA) Having hidden units larger than number of inputs can still discover useful insights by imposing certain sparsity constraints.
IV Denoising Autoencoder
Denoising autoencoder is a stochastic extension to classic autoencoder, that is we force the model to learn reconstruction of input given its noisy version A stochastic corruption process randomly sets some of the inputs to zero, forcing denoising autoencoder to predict missing(corrupted) values for randomly selected subsets of missing patterns
Denoising autoencoders can be stacked to create a deep network (stacked denoising autoencoder)
Trang 14Picture 10: stacked denoising autoencoder
Output from the layer below is fed to the current layer and training is done layer wise
V Convolutional Autoencoder
Convolutional autoencoders are based on standard autoencoder architecture with
convolutional encoding and decoding layers Compared to classic autoencoders,
convolutional autoencoders are better suited for image processing as they utilize full capability of convolutional neural networks to exploit image structure
In convolutional autoencoders, weights are shared among all input locations which helps preserve local spatiality Rep- resentation of ith feature map is given as
where bias is broadcasted to whole map, ∗ denotes convo- lution (2D) and s is an activation Single bias per latent map is used and reconstruction is obtained as
Trang 15where c is bias per input channel, H is group of latent feature maps, W ̃ is flip operation over both weight dimensions
Backpropogation is used for computation of gradient of the error function with respect to the parameters
C Methodology
Picture 11: Methodology
The goal of Methodology for Image Denoising using Autoencoderis to leverage Autoencoder architecture for image denoising Initially, a dataset of clean images is available To train the Autoencoder model, noise is introduced to these images, creating a set of noisy images The purpose is to enable the Autoencoder to learn the mapping from noisy images to clean images, facilitating the ability to denoise images effectively
I Data Preprocessing
1 Data Collection
Collect a dataset of clean images to serve as the foundation for training the Autoencoder
2 Resizing Images for Model Input
To ensure that both clean and noisy images; images from another dataset are appropriately sized for input into the Autoencoder model, we need to perform the image resizing process This process involves adjusting the resolution of the images to match the requirements of the model
3 Noise adding
Apply different types and levels of noise to the clean images to generate a diverse set of noisy images This step is crucial for training the Autoencoder to handle various noise patterns