Xử lý ảnh trong cơ điện tử machine vision chapter 6 image segmentation

TRƯỜNG ĐẠI HỌC BÁCH KHOA HÀ NỘI XỬ LÝ ẢNH TRONG CƠ ĐIỆN TỬ Machine Vision Giảng viên: TS Nguyễn Thành Hùng Đơn vị: Bộ môn Cơ điện tử, Viện Cơ khí Hà Nội, 2021 Chapter Image Segmentation Fundamentals Point, Line, and Edge Detection Thresholding Image Segmentation Using Deep Learning Rafael C Gonzalez, Richard E Woods, “Digital image processing,” Pearson (2018) Fundamentals ➢ Let R represent the entire spatial region occupied by an image We may view image segmentation as a process that partitions R into n subregions, R1, R2, …, Rn, such that: where Q(Rk) is a logical predicate defined over the points in set Rk and  is the null set Rafael C Gonzalez, Richard E Woods, “Digital image processing,” Pearson (2018) Fundamentals ➢ Two regions Ri and Rj are said to be adjacent if their union forms a connected set ➢ The regions are said to disjoint If the set formed by the union of two regions is not connected ➢ The fundamental problem in segmentation is to partition an image into regions that satisfy the preceding conditions ➢ Segmentation algorithms for monochrome images generally are based on one of two basic categories dealing with properties of intensity values: discontinuity and similarity ▪ Edge-based segmentation ▪ Region-base segmentation Rafael C Gonzalez, Richard E Woods, “Digital image processing,” Pearson (2018) Fundamentals (a) Image of a constant intensity region (b) Boundary based on intensity discontinuities (c) Result of segmentation (d) Image of a texture region (e) Result of intensity discontinuity computations (note the large number of small edges) (f) Result of segmentation based on region properties Rafael C Gonzalez, Richard E Woods, “Digital image processing,” Pearson (2018) Fundamentals ❖ Traditional Image Segmentation techniques Chapter Image Segmentation Fundamentals Point, Line, and Edge Detection Thresholding Image Segmentation Using Deep Learning Rafael C Gonzalez, Richard E Woods, “Digital image processing,” Pearson (2018) Point, Line, and Edge Detection ➢ Detecting sharp, local changes in intensity ➢ The three types of image characteristics: isolated points, lines, and edges ➢ Edge pixels: the intensity of an image changes abruptly ➢ Edges (or edge segments): sets of connected edge pixels ➢ Edge detectors: local image processing tools designed to detect edge pixels Rafael C Gonzalez, Richard E Woods, “Digital image processing,” Pearson (2018) Point, Line, and Edge Detection ➢ A line: ▪ a (typically) thin edge segment ▪ the intensity of the background on either side of the line is either much higher or much lower than the intensity of the line pixels ▪ “roof edges” ➢ Isolated point: a foreground (background) pixel surrounded by background (foreground) pixels Rafael C Gonzalez, Richard E Woods, “Digital image processing,” Pearson (2018) Point, Line, and Edge Detection ❖ Background ➢ an approximation to the first-order derivative at an arbitrary point x of a onedimensional function f(x) x = for the sample preceding x and x = -1 for the sample following x Rafael C Gonzalez, Richard E Woods, “Digital image processing,” Pearson (2018) 10 Image Segmentation Using Deep Learning ❖ Deep Learning-based methods ➢ Like most of the other applications, using a CNN for semantic segmentation is the obvious choice ➢ When using a CNN for semantic segmentation, the output is also an image rather than a fixed length vector A Beginner's guide to Deep Learning based Semantic Segmentation using Keras | Divam Gupta 69 Image Segmentation Using Deep Learning ❖ Convolutional neural networks for segmentation ➢ The architecture of the model contains several convolutional layers, non-linear activations, batch normalization, and pooling layers ➢ The initial layers learn the low-level concepts such as edges and colors ➢ The later level layers learn the higher level concepts such as different objects A Beginner's guide to Deep Learning based Semantic Segmentation using Keras | Divam Gupta 70 Image Segmentation Using Deep Learning ❖ Convolutional neural networks for segmentation Spatial tensor is downsampled and converted to a vector A Beginner's guide to Deep Learning based Semantic Segmentation using Keras | Divam Gupta 71 Image Segmentation Using Deep Learning ❖ Convolutional neural networks for segmentation Encoder-Decoder architecture A Beginner's guide to Deep Learning based Semantic Segmentation using Keras | Divam Gupta 72 Image Segmentation Using Deep Learning ❖ Skip connections Encoder-Decoder with skip connections A Beginner's guide to Deep Learning based Semantic Segmentation using Keras | Divam Gupta 73 Image Segmentation Using Deep Learning ❖ Transfer learning Transfer learning for Segmentation A Beginner's guide to Deep Learning based Semantic Segmentation using Keras | Divam Gupta 74 Image Segmentation Using Deep Learning ❖ Loss function ➢ Each pixel of the output of the network is compared with the corresponding pixel in the ground truth segmentation image We apply standard cross-entropy loss on each pixel In binary classification, where the number of classes M equals 2, cross-entropy can be calculated as: If M>2 (i.e multiclass classification), we calculate a separate loss for each class label per observation and sum the result A Beginner's guide to Deep Learning based Semantic Segmentation using Keras | Divam Gupta 75 Image Segmentation Using Deep Learning ❖ Implementation ➢ Dataset: The first step in training our segmentation model is to prepare the dataset ➢ Data augmentation: Example of image augmentation for segmentation A Beginner's guide to Deep Learning based Semantic Segmentation using Keras | Divam Gupta 76 Image Segmentation Using Deep Learning ❖ Implementation ➢ Building the model: define our segmentation model with skip connections ➢ Choosing the model: ▪ Choosing the base model: select an appropriate base network → ResNet, VGG16, MobileNet, Custom CNN, … ▪ Select the segmentation architecture: FCN, SegNet, UNet, PSPNet, … A Beginner's guide to Deep Learning based Semantic Segmentation using Keras | Divam Gupta 77 Image Segmentation Using Deep Learning ❖ Implementation Architecture of FCN32 A Beginner's guide to Deep Learning based Semantic Segmentation using Keras | Divam Gupta 78 Image Segmentation Using Deep Learning ❖ Implementation Architecture of SegNet A Beginner's guide to Deep Learning based Semantic Segmentation using Keras | Divam Gupta 79 Image Segmentation Using Deep Learning ❖ Implementation Architecture of UNet A Beginner's guide to Deep Learning based Semantic Segmentation using Keras | Divam Gupta 80 Image Segmentation Using Deep Learning ❖ Implementation Architecture of PSPNet A Beginner's guide to Deep Learning based Semantic Segmentation using Keras | Divam Gupta 81 Image Segmentation Using Deep Learning ❖ Implementation ➢ Choosing the input size: If there are a large number of objects in the image, the input size shall be larger The standard input size is somewhere from 200x200 to 600x600 ➢ Training: output are the model weights ➢ Testing: get predictions from a saved model A Beginner's guide to Deep Learning based Semantic Segmentation using Keras | Divam Gupta 82 Image Segmentation Using Deep Learning ❖ Implementation Image Segmentation results of DeepLabV3 on sample images Photo Credit: https://arxiv.org/pdf/2001.05566 pdf 83 ... “Digital image processing,” Pearson (2018) Fundamentals ❖ Traditional Image Segmentation techniques Chapter Image Segmentation Fundamentals Point, Line, and Edge Detection Thresholding Image Segmentation. .. original image Rafael C Gonzalez, Richard E Woods, “Digital image processing,” Pearson (2018) 50 Chapter Image Segmentation Fundamentals Point, Line, and Edge Detection Thresholding Image Segmentation. . .Chapter Image Segmentation Fundamentals Point, Line, and Edge Detection Thresholding Image Segmentation Using Deep Learning Rafael C Gonzalez, Richard E Woods, “Digital image processing,”