Trong quá trình đưa vào hoạt động và thử nghiệm, hệ thống vẫn có một số hạn chế:
1. Chưa tối ưu nhận diện được một vài kiểu vết xước.
2. Chưa nhận diện được các vết xước có kích thước nhỏ hơn 1mm. Từ đó, phương án giải quyết được đề xuất như sau:
✓ Rút ngắn độ dài băng tải sẽ làm tăng năng suất hệ thống.
✓ Sử dụng Camera độ phân giải cao giúp nhận diện các vết xước có kích thước nhỏ.
52
TÀI LIỆU THAM KHẢO
[1] Clyde F. Coombs, 2007. Printed circuits handbook. McGraw–Hill Professional. [2] X. Luo and H. Hu, 2020. Selected and refined local attention module for object
detection. electronics letters, vol. 56, no. 14, pp. 712-714.
[3] S. K. G. Manikonda and D. N. Gaonkar, 2020. Islanding detection method based on image classification technique using histogram of oriented gradient features, IET Generation, Transmission & Distribution, vol. 14, no. 14, pp. 2790-2799. [4] G. A. Tahir and C. K. Loo, 2020. An Open-Ended Continual Learning for Food
Recognition Using Class Incremental Extreme Learning Machines, IEEE Access, vol. 8, pp. 82328-82346.
[5] K. Huang, H. Yang, I. King and M. R. Lyu, 2008. Maxi–Min margin machine: Learning large margin classifiers locally and globally, IEEE Transactions on Neural Networks, vol. 19, no. 2, pp. 260-272.
[6] D. Martens, B. B. Baesens and T. Van Gestel, 2009. Decompositional rule extraction from support vector machines by active learning," IEEE Transactions on Knowledge and Data Engineering, vol. 21, no. 2, pp. 178-191.
[7] E. Pasolli, F. Melgani and Y. Bazi, 2011. Support vector machine active learning through significance space construction, IEEE Geoscience and Remote Sensing Letters, vol. 8, no. 3, pp. 431-435.
[8] G. Krummenacher, C. S. Ong, S. Koller, S. Kobayashi and J. M. Buhmann, 2018. Wheel defect detection with machine learning, IEEE Transactions on Intelligent Transportation Systems, vol. 19, no. 4, pp. 1176-1187.
[9] P. Kumar Shukla et al., 2020. Efficient prediction of drug–drug interaction using deep learning models, IET Systems Biology, vol. 14, no. 4, pp. 211-216.
[10] J. S. Shemona and A. K. Chellappan, 2020. Segmentation techniques for early cancer detection in red blood cells with deep learning-based classifier—a comparative approach, IET Image Processing, vol. 14, no. 9, pp. 1726-1732.
53
[11] A. Alamri, et al., 2020. An effective bio-signal-based driver behavior monitoring system using a generalized deep learning approach, IEEE Access, vol. 8, pp. 135037-135049.
[12] M. Zhu and Q. Chen, 2020. Big data image classification based on distributed deep representation learning model, IEEE Access, vol. 8, pp. 133890-133904. [13] J. Jiang, J. Cheng and D. Tao, 2012. Color biological features-based solder paste
defects detection and classification on printed circuit boards, IEEE Transactions on Components, Packaging and Manufacturing Technology, vol. 2, no. 9, pp. 1536-1544.
[14] Y. Hara, H. Doi, K. Karasaki and T. Iida, 1988. A system for PCB automated inspection using fluorescent light, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 10, no. 1, pp. 69-78.
[15] Reinhard Klette (2014). Concise Computer Vision. Springer. ISBN 978-1-4471- 6320-6.
[16] Pulli, Kari; Baksheev, Anatoly; Kornyakov, Kirill; Eruhimov, Victor (1 April 2012). "Realtime Computer Vision with OpenCV". Queue. 10 (4): 40:40–40:56. doi:10.1145/2181796.2206309.
[17] Luger, George; Stubblefield, William (2004). Artificial Intelligence: Structures and Strategies for Complex Problem Solving (5th ed.). Benjamin/Cummings. ISBN 978-0-8053-4780-7. Archived from the original on 26 July 2020. Retrieved 17 December 2019.
[18] Bengio, Yoshua; LeCun, Yann; Hinton, Geoffrey (2015). "Deep Learning". Nature. 521 (7553): 436–44, doi:10.1038/nature14539
[19] Hopfield, J. J. (1982). "Neural networks and physical systems with emergent collective computational abilities". Proc. Natl. Acad. Sci. U.S.A. 79 (8): 2554– 2558, doi:10.1073/pnas.79.8.2554
[20] Van-Truong Nguyen, Huy-Anh Bui; “A REAL-TIME DEFECT DETECTION IN PRINTED CIRCUIT BOARDS APPLYING DEEP LEARNING”;2021.
54
[21] K. He, X. Zhang, S. Ren and J. Sun, "Deep Residual Learning for Image Recognition," 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 770-778, doi: 10.1109/CVPR.2016.90.
[22] Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng-Yang Fu, Alexander C. Berg, “SSD: Single Shot MultiBox Detector”, https://arxiv.org/abs/1512.02325v5
[23] Liang-Chieh Chen, George Papandreou, IEEE, Iasonas Kokkinos, Kevin Murphy, and Alan L. Yuille, “DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs”, arXiv:1606.00915v2 [cs.CV] 12 May 2017
[24] Adrian Kaehler and Gary Bradski, “Learning OpenCV 3 Computer Vision in C++ with the OpenCV Library”, O’Reilly Media, Inc, 2017.
[25] Rublee, Ethan; Rabaud, Vincent; Konolige, Kurt; Bradski, Gary (2011). "ORB: an efficient alternative to SIFT or SURF"(PDF). IEEE International Conference on Computer Vision (ICCV).
[26] M.A. Fischler and R.C. Bolles. Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 24(6):381–395, 1981.
[27] Roshan Jameer Patan, Madhuparna Chakraborty, Sarungbam Sanahanbi Devi; “Blind color image de-convolution in YUV space”; 2016 International Conference on Communication and Signal Processing (ICCSP)
[28] Adrian Kaehler and Gary Bradski, “Learning OpenCV 3 Computer Vision in C++ with the OpenCV Library”, O’Reilly Media, Inc, 2017.
[29] ADIBHATLA, V.A., CHIH, H.C., HSU, C.C., CHENG, J., ABBOD, M.F., SHIEH, J.S., 2020, Defect Detection in Printed Circuit Boards Using You- Only-Look-Once Convolutional Neural Networks, Electronics, 9, 1-9.
[30] Y. LeCun, B. Boser, JS. Denker, D. Henderson, RE. Howard, W. Hubbard, LD Jackel, “Backpropagation Applied to Handwritten Zip Code Recognition,” Neural Computation, 1(4), 1989, p.p. 541-551
[31] Zhuo Lan, Yang Hong, Yuan Li, “An improved YOLOv3 method for PCB surface defect detection”, 2021 IEEE International Conference on Power Electronics, Computer Applications (ICPECA) | 978-1-7281-9004-4/20/$31.00 ©2021 IEEE | DOI: 10.1109/ICPECA51329.2021.9362675
55
PHỤ LỤC 1. Chương trình xử lí ảnh
2. Bộ bản vẽ kỹ thuật hệ thống phân loại bảng mạch in PCB
[1] Gối đỡ trục quay. [2] Gối đỡ trục điều chỉnh. [3] Gá cảm biến.
[4] Gá đỡ động cơ băng tải. [5] Gối đỡ trục chuyền động. [6] Gá đỡ động cơ tay máy 1. [7] Gá đỡ động cơ tay máy 2. [8] Hộp đèn trợ sáng.
[9] Tấm chặn PCB lỗi. [10] Hộp đèn trợ sáng. [11] Chi tiết A-A. [12] Chi tiết B-B. [14] Chi tiết C-C. [15] Bản vẽ khung. [16] Bản vẽ phân rã. [17] Bản vẽ lắp hệ thống cơ khí. 3. Bản vẽ hệ thống điều khiển
1. Chương trình xử lí ảnh • Xoay ảnh và trừ ảnh: #chương trình test import cv2 as cv import numpy as np def __rotate_image__(template_image,compare_image): gray_tem = cv.cvtColor(template_image,cv.COLOR_BGR2GRAY) gray_com = cv.cvtColor(compare_image,cv.COLOR_BGR2GRAY) orb_fe = cv.ORB_create() kp1,des1 = orb_fe.detectAndCompute(gray_com,None) kp2,des2 = orb_fe.detectAndCompute(gray_tem,None) bf = cv.BFMatcher(cv.NORM_HAMMING, crossCheck=True) matches = bf.match(des1, des2)
matches = sorted(matches, key = lambda x:x.distance) diem_chon = [] ma_trix = [] kp2_moi = [] kp1_moi = [] ma_trix_1 = [] thresh_area = 200 kernel = np.ones((7,7),np.uint8) for i in range(len(matches[:20])): diem_chon.append(matches[i])
matchedVis = cv.drawMatches(template_image, kp1, compare_image, kp2,matches[:20], None)
ptsA = np.zeros((len(matches[:50]), 2), dtype="float") ptsB = np.zeros((len(matches[:50]), 2), dtype="float") for (i, m) in enumerate(matches[:50]):
ptsA[i] = kp1[m.queryIdx].pt ptsB[i] = kp2[m.trainIdx].pt
(H, mask) = cv.findHomography(ptsA,ptsB ,method=cv.RANSAC) (h, w) = template_image.shape[:2]
#ảnh sau khi quay
aligned = cv.warpPerspective(compare_image, H, (w, h)) return aligned
#trừ ảnh
def __subtract_image__(aligned, template_image,area_thresh,solid_val,ratio_scale): error_box = [] template_image_hsv = cv.cvtColor(template_image,cv.COLOR_BGR2YUV) aligned_hsv = cv.cvtColor(aligned,cv.COLOR_BGR2YUV) kernel = np.ones((3,3),np.uint8) #trừ ảnh diff = cv.absdiff(aligned_hsv,template_image_hsv) gray = cv.cvtColor(diff,cv.COLOR_BGR2GRAY) ret,thres = cv.threshold(gray,25,255,cv.THRESH_BINARY)#17 opening = cv.morphologyEx(thres, cv.MORPH_OPEN, kernel) #show ảnh kết quả sau lọc nhiễu
cv.imshow("img",opening) #tìm thông số của các vùng lỗi.
output = cv.connectedComponentsWithStats(opening, 4, cv.CV_32S) (so_lop,lop,stat,centroid) = output h_,w_,_ = aligned.shape for i in range(0,so_lop): x = stat[i,cv.CC_STAT_LEFT] y = stat[i,cv.CC_STAT_TOP] w = stat[i,cv.CC_STAT_WIDTH] h = stat[i,cv.CC_STAT_HEIGHT] area = stat[i,cv.CC_STAT_AREA] scale_x = w/float(h)
scale_y = h/float(w) solid = area/float(w*h) if h<w:
if area>AreaValue and area<h_*w_*0.4 and scale_y>SolidValue and solid>0.4: cv.rectangle(aligned,(x,y),(x+w,y+h),(255,0,255),3) error_box.append((x,y,w,h)) img = aligned[y:y+h,x:x+w] rd = np.random.randint(0,200,1)[0] cv.imwrite("./ketqua/linhkien_{}.jpg".format(rd),img) elif h>w:
if area>AreaValue and area<h_*w_*0.4 and scale_x>SolidValue and solid>0.4:
cv.rectangle(aligned,(x,y),(x+w,y+h),(255,0,255),3) error_box.append((x,y,w,h))
return error_box,aligned,len(error_box) #hàm trả về các vùng lỗi(thông số cần thiết ) và số vùng lỗi SolidValue = 0.395 RatioValue = 0.169 AreaValue = 386 framegoc=cv.imread('unotest.jpg',1) sampleImg=cv.imread('./info/anh_mau/uno_dan.jpg',1) img_ref =framegoc[:,742:742+702] sampleImg = cv.rotate(sampleImg,cv.ROTATE_90_COUNTERCLOCKWISE) alignedImg=__rotate_image__(sampleImg,framegoc)
__subtract_image__(alignedImg, sampleImg, AreaValue, SolidValue, RatioValue) cv.imshow("abc",alignedImg)
cv.imshow("sample",sampleImg) cv.imshow("test",framegoc)
• Test model u2net:
from sklearn.model_selection import train_test_split import os import random import tensorflow as tf import cv2 import numpy as np import time
# Import thu vien segmentation_models
from segmentation_models.metrics import iou_score from segmentation_models import Unet
import segmentation_models as sm from keras.models import load_model sm.set_framework("tf.keras") sm.framework() BACKBONE = "resnet34" model= Unet(BACKBONE,encoder_weights="imagenet",classes=1,activation="sigmoid",inpu t_shape=(512,512,3),encoder_freeze=True) loss1 = sm.losses.categorical_focal_dice_loss model.load_weights("./weight26.hdf5") img="./anh_def/duoi8.jpg"
# Anh dau vao, ko phai mask image = cv2.imread(img)
image = cv2.resize(image, (512, 512)) xanh_thap = np.array([40,40,40]) xanh_tren=np.array([120,120,120])
thresh=cv2.inRange(image,xanh_thap,xanh_tren) # Dua qua model de predicted segmentation map timer=time.time()
mask_predict = model.predict(image[np.newaxis, :, :, :]) timer_last=time.time() deltime=timer_last-timer z = mask_predict[0,:,:,0]#[:, :, 0] arr=np.uint8(z*20) cv2.imwrite("mask.jpg",arr) ret,thres = cv2.threshold(arr,0,255,cv2.THRESH_BINARY) contours,hierachy = cv2.findContours(thres,cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_NONE) for contour in contours:
area = cv2.contourArea(contour) if(area > 80):
x, y, w, h = cv2.boundingRect(contour)
#frame = cv2.rectangle(image, (x, y), (x + w, y + h), (0, 255, 0), 2) frame1 = cv2.drawContours(image,contour, -1, (0,255,0), 3) cv2.imshow("predict1",frame1) print(type(image[0][0][1])) print(z.shape) print(type(z[0][0])) print(arr.shape) print(type(arr[0][0])) cv2.imshow("predict",thres) cv2.imshow("image",image) cv2.waitKey(0)