(Đồ án hcmute) design of an image acquisition system for the detection of occlusal dental problems

BỘ GIÁO DỤC VÀ ĐÀO TẠO TRƯỜNG ĐẠI HỌC SƯ PHẠM KỸ THUẬT THÀNH PHỐ HỒ CHÍ MINH ĐỒ ÁN TỐT NGHIỆP NGÀNH BIOMEDICAL ENGINEERING DESIGN OF AN IMAGE ACQUISITION SYSTEM FOR THE DETECTION OF OCCLUSAL DENTAL PROBLEMS GVHD: NGUYEN THANH HAI SVTH: NGUYỄN PHÚC BẢO SVTH: NGUYỄN LÊ GIA BÁCH SKL 0 7 Tp Hồ Chí Minh, tháng 08/2020 an HO CHI MINH CITY UNIVERSITY OF TECHNOLOGY AND EDUCATION FACULTY OF ELECTRICAL AND ELECTRONICS ENGINEERING DEPARTMENT OF ELECTRONICS - BIOMEDICAL ENGINEERING - GRADUATION THESIS BIOMEDICAL ENGINEERING PROJECT: DESIGN OF AN IMAGE ACQUISITION SYSTEM FOR THE DETECTION OF OCCLUSAL DENTAL PROBLEMS Advisor: Assoc Prof Dr NGUYEN THANH HAI Nguyễn Phúc Bảo Student ID: 16129006 Nguyễn Lê Gia Bách Student ID: 16129008 Ho Chi Minh City, August 2020 an HO CHI MINH CITY UNIVERSITY OF TECHNOLOGY AND EDUCATION FACULTY OF ELECTRICAL AND ELECTRONICS ENGINEERING DEPARTMENT OF ELECTRONICS - BIOMEDICAL ENGINEERING - GRADUATION THESIS BIOMEDICAL ENGINEERING PROJEC T'S NAME: DESIGN OF AN IMAGE ACQUISITION SYSTEM FOR THE DETECTION OF OCCLUSAL DENTAL PROBLEMS Advisor: Assoc Prof Dr NGUYEN THANH HAI Nguyễn Phúc Bảo Student ID: 16129006 Nguyễn Lê Gia Bách Student ID: 16129008 Ho Chi Minh City, August 2020 i an HCMC UNIVERSITY OF TECHNOLOGY AND EDUCATION THE SOCIALIST REPUBLIC OF VIETNAM Faculty of Electrical & Electronics Engineering Department of Electronics Industrial-Biomedical Engineering Independence – Freedom– Happiness o0o -Ho Chi Minh City, July 30, 2020 MISSION Student name: Major: Type of training: Year: Nguyễn Phúc Bảo Nguyễn Lê Gia Bách Biomedical Engineering Regular full time 2016 Student ID: 16129006 Student ID: 16129008 Major ID: D520212 Code: Class: 161290B I GRADUATION PROJECT: DESIGN OF ONE IMAGE ACQUISITION SYSTEM FOR DETECTION OF OCCLUSAL DENTAL PROBLEMS II MISSION Initial resources: - Raspberry Pi B plus - Camera (F) for Raspberry - Human teeth - Infrared lights Implementation: - Understanding the anatomy of human teeth - Understanding the basic concept of infrared light and how it impacts enamel - Design and fabricate a 3D model - Use Python programing language - Design PCB and fabricate a power supply board - Design a prototype version - Project evaluation III DATE OF DELIVERY: IV DATE OF COMPLETION: V ADVISOR NAME: ADVISOR 15/03/2020 03/08/2020 Assoc Prof Dr NGUYEN THANH HAI DEPARTMENT OF ELECTRONICS INDUSTRIALBIOMEDICAL ENGINEERING ii an HCMC UNIVERSITY OF TECHNOLOGY AND EDUCATION THE SOCIALIST REPUBLIC OF VIETNAM Faculty of Electrical & Electronics Engineering Independence – Freedom– Happiness Department of Electronics Industrial-Biomedical Engineering o0o Ho Chi Minh City, July 30, 2020 SCHEDULE Student name: Nguyễn Phúc Bảo Student ID: 16129006 Student name: Nguyễn Lê Gia Bách Student ID: 16129008 Class: 161290B Graduation project: Design of one image acquisition system for detection of occlusal dental problems Week Week 1st (March March 16) Task description Lecturer confirm Meet the advisor to listen to the request for doing a project, choose a project topic Week 2nd (March 17 March 24) Week 3rd (March 25 April 1) Week 4th & week 5th (April April 16) Week 6th & week 7th (April 17 May 1) Advisor reviews topic Write an abstract of the selected topic: what the topic purpose is, the design request, the limits of the topic Research into a theoretical basis of the project on many resources Buy components, design the prototype and conduct experiments Week 8th & week 9th Change the design and continue (May - May conducting experiments 17) Week 10th & week 11th (May 18 June 2) Make a power supply board and continue conducting experiments iii an Week 12th & week 13th Write a thesis report, and continue (June - June conducting experiments 17) Week 14th & week 15th (June 18 July 3) Week 16th & week 17th (July - July 18) Week 18th (July 19 July 25) Complete the model and continue conducting experiments Complete the thesis report, submit to the advisor for the last inspection Submit the final version of the thesis report and report the thesis Week 19th (July 26 August 1) Make a presentation ADVISOR (Sign and write your full name) iv an GUARANTEE We guarantee that this project is our research, under Assoc Prof Dr NGUYEN THANH HAI 's guidance The results published in this project are honest and are not replicated from any other work Students: Nguyễn Phúc Bảo Nguyễn Lê Gia Bách v an ACKNOWLEDGMENTS To complete this graduation project, the first words of our group would like to say sincere thanks to Assoc Prof Dr NGUYEN THANH HAI for his dedication that helps us a lot to accomplish our project In the process of doing this thesis, Dr Hai not only provides us useful knowledge but also trains us to have a severe attitude of scientific research, which is very necessary for the career path later The group would like to thank the teachers in the Department of Industrial Electronics - Biomedical Engineering for providing the best conditions for us to complete the topic Besides, we would like to thank all the lecturers who taught us foundation knowledge in the previous semesters, thanks to this background knowledge, and the group was able to complete the project Thank parents for your hard-working in raising us to school The more extended period spending on working, the more we understand the suffering of our parents Finally, we would like to express our sincere thanks to the people who have contributed and helped the group to implement this project successfully Sincerely thanks! Ho Chi Minh City, July 28, 2020 Students Nguyen Phuc Bao Nguyen Le Gia Bach vi an TABLE OF CONTENTS Cover i Mission ii Schedule iii Guarantee v Acknowledgments vi Table of contents vii List of figures ix List of tables x Abstract xi CHAPTER INTRODUCTION 1.1 Problem Statement 1.2 Objectives 1.3 Research Content .2 1.4 Limitations .2 1.5 Thesis Report Outline CHAPTER MATERIALS AND METHODS .4 2.1 Overview of Human Teeth 2.2 Dental Anatomy .5 2.3 Dental Problems 2.4 Radiographic Views 2.5 Near-Infrared Laser and Tooth Enamel 2.6 Image Processing Methods 11 2.6.1 Histogram Equalization 11 2.6.2 Otsu’s Method 12 2.7 Hardware .15 2.7.1 Infrared Camera 15 2.7.2 Infrared Light 16 2.7.3 Microcontroller 17 2.7.4 Power Supply 18 2.8 Software 21 CHAPTER DESIGN AND CALCULATION 22 vii an 3.1 Calculation of The Image Acquisition System 22 3.1.1 Block Diagram of An Acquisitive Device 22 3.1.2 The 5-Volt Power Supply 22 3.1.3 Case Designs for The Image Acquisition System 24 3.2 Devices Connection .26 CHAPTER ASSEMBLY OF THE IMAGE ACQUISITION SYSTEM .27 4.1 Power Supply 27 4.1.1 Assembly of The Power Supply 28 4.1.2 Inspection of The Power Supply 28 4.2 Enclosure of The Image Acquisition System 29 4.3 Image Acquisition System Software .30 CHAPTER RESULTS AND DISCUSSION .32 5.1 Results 32 5.1.2 The Power Supply 32 5.1.3 Result Images Acquisition 34 5.2 Instruction 38 5.2.1 Setup Instruction 38 5.2.2 Operating Instruction 38 5.3 Discussions 39 5.3.1 Advantages 39 5.3.2 Disadvantages .39 CHAPTER CONCLUSIONS AND FUTURE WORK 40 6.1 Conclusions 40 6.2 Future Works 40 REFERENCES 41 APPENDIX 43 viii an Chapter Results and discussion 5.1.3 Result Images Acquisition a) Image of the tooth under visible light Before emitting the beam of NIR lights through the tooth, it was noticed that the teeth were opalescent, reflecting the normal light, and the deep marks on the teeth could not be witnessed by human eyes Figure 5.2 illustrates the images of the tooth sample before processing Figure Image of the tooth under visible light b) Results after processed In the research process, to choose the proper wavelength for IR LEDs, we experienced several difficulties Try and error is the Method that we must use to find the ultimate infrared LEDs wavelength Somehow, it costs significantly (time and money) Figure 5.3a) was captured in a condition: 940nm IR LEDs, low light The IR rays went through the tooth theoretically, but this image cannot be processed (Image processing algorithm cannot process this picture) Figure 5.3b) was captured in condition: 850nm IR LEDs, low light Tooth details were witnessed easily, but the tooth decays were not emphasized The reason could be the fact that high-power LEDs impact scattering of IR rays DEPARTMENT OF ELECTRONICS - BIOMEDICAL ENGINEERING an 34 Chapter Results and discussion a) 940nm b) 850nm Figure The images with two different NIR wavelengths 940nm and 850nm Eventually, we decided to use 760 nm IR LEDs because it gives us better results that we are going to show in table 5.2 Several suspected caries extracted teeth were examined in the project To acquire the results that were shown in the table, an input image must follow seven steps: Step 1: Resize an input image to 180x180 Step 2: Get the red channel of the image Step 3: Use the Gaussian filter to reduce speckle noises and the median filter to reduce salt and pepper noise Step 4: Apply the Histogram Equalization method (mentioned in 2.6.1 section) to the image to adjust contrast Step 5: Find a threshold with Otsu’s Method The formula was mention in section 2.6.2 Step 6: Increase the contrast of the image with the 5.2 formula: 𝑔(𝑖, 𝑗) = 𝛼 𝑓 (𝑖, 𝑗) + 𝛽 (5.2) Where - i and j indicate that the pixel is located in the i-th row and j-th column - g(i,j) is a new value of a pixel - f(i,j) is a default value of a pixel The parameters α and β are sometimes referred to as the gain and bias parameters; these parameters are often said to control contrast and brightness, respectively If the parameter α is greater than zero, the contrast of the image is increased, vice versa Step 7: Use the threshold found with Otsu’s Method to mark the cavities Because pixel values of the cavity are very close to zero, we should subtract 60 from the threshold value to determine the exact decay traces, avoiding marking to the corners and edges of the teeth Pixels with a value lower than the threshold value will be assigned a value of 0; otherwise, the value will have remained the same DEPARTMENT OF ELECTRONICS - BIOMEDICAL ENGINEERING an 35 Chapter Results and discussion All those teeth are molar or pre-molar teeth The pictures were taken from the crown areas For each specific case, we will put our comments to show our prediction and evaluation from the student’s perspective Table 5.2 showed the results of the real teeth in volunteer mouth; the results of the sample teeth are shown in table 5.3 Table The comments of the real tooth Order After processed Description No sign of dental decay There is a significant black point on the crown area A volunteer revealed that his tooth was filled No sign of dental decay DEPARTMENT OF ELECTRONICS - BIOMEDICAL ENGINEERING an 36 Chapter Results and discussion Table The comments of the sample tooth Order After processed Description No abnormal sign, maybe it is a healthy tooth The lesion can be seen apparently We could not determine how deep it is The exact location of a suspected lesion is shown on the output data on the top view The exact location of fillings on the tooth shown on the output image DEPARTMENT OF ELECTRONICS - BIOMEDICAL ENGINEERING an 37 Chapter Results and discussion Table Accuracy of the system refers to a visual ability of normal human eyes ACCURACY OF THE SYSTEM 16 15 14 14 12 10 10 3 Molar Pre-Model Visible Canine Healty System Decay detection of molar reaches 93.3% (14/15) Decay detection of pre-molar reaches 90% (9/10) 5.2 INSTRUCTION 5.2.1 Setup instruction Step 1: Plug the male jack in the power outlet, turn on the switch, press the small button on the supply to make the power supply active, the green light of the power supply lights up Step 2: Plugin Raspberry Pi Model B power USB and power jack Step 3: Wait for raspberry booting (roughly 10 to 20 seconds) Step 4: Display Raspberry screen You can display a raspberry screen in two different methods Step 4a.1: On our computer, use any kind of IP Scanner to find what your Raspberry’s IP is Step 4a.2: Use the RealVNC program to connect to Raspberry Pi Model B Step 4b Use an HDMI cable to connect Raspberry board to a TV/Monitor Step 5: Open Get_image.py on a desktop on Raspberry 5.2.2 Operating Instruction - After powering up the device, launch the device Use a small spoon to push out the cheekbones, making it easy to insert the device into the teeth DEPARTMENT OF ELECTRONICS - BIOMEDICAL ENGINEERING an 38 Chapter Results and discussion - Put the device into the teeth so that the teeth are between the two lights; the device is facing up so that it is as perpendicular to the teeth as possible to obtain the best image - You can directly view the image via Raspberry or use the Space key to capture images Causing: The device operates stably at room temperature, avoiding use in hot environments 5.3 DISCUSSIONS 5.3.1 Advantages Overall, we have a significant improvement in the project model In the beginning, we just tended to experiment with the impact of IR on enamel, but we have made a usable device that can be put into a patient’s mouth The device can be moved easily in the mouth to capture the best results Additionally, we designed the probe that is separated from the Embedded computer; thus, users can take advantage of this design to hold the probe simply The scientific research, we got a basic concept of Infrared LED and how it influences enamel The different IR wavelengths influence enamel in different ways Although we not have a chance of using an IR LED with an ideal wavelength, which we found in theory, we still found a way to emphasize decay traces About image processing, we created a program removing some noise in a picture and clarify traces of dental problems Although it is not the best results that can be acquired, they are still better than a way we witness with human eyes 5.3.2 Disadvantages Instead of doing our best to accomplish the project, the project still has unexpected disadvantages The worst disadvantage of our project belongs to the image processing program In some cases, it cannot mark the dental problems properly About the power supply, the case of the power supply is quite big, and there is no cooling system for the power supply, which has to operate continuously DEPARTMENT OF ELECTRONICS - BIOMEDICAL ENGINEERING an 39 Chapter Conclusions and future works CHAPTER CONCLUSIONS AND FUTURE WORK 6.1 CONCLUSIONS To sum up, this device was made for research purposes After completing this project, we have learned a tremendous amount of knowledge that contains: Infrared technology, 3D printing technology, designing a power supply, We knew the importance of experimentation in scientific fields, especially in our major (biomedical engineering) Although many disadvantages, our device still satisfies our beginning purpose: examine the influence of Infrared to enamel In the future, we hope that our device will have improvements to become a popular tool that everyone can use daily 6.2 FUTURE WORKS With the development of science and technology, non-invasive equipment has become increasingly essential In the future, we are eager to keep developing this project and hope that it can meet the medical standard The device should be made of high-quality plastic to reduce weight and not make patients allergic Additionally, we can design our device, which looks like a pen, and use a battery to power it Maybe, we can have the patient’s sympathy because of its appearance DEPARTMENT OF ELECTRONICS - BIOMEDICAL ENGINEERING an 40 REFERENCES REFERENCES [1] “Human tooth”, https://en.wikipedia.org/, accessed on 26/02/2020 [2] “Teeth: Names, types, and functions”, https://medicalnewstoday.com/, accessed on 26/02/2020 [3] “The Roles and Structure of Teeth”, https://www.lions.co.jp, accessed on 26/02/2020 [4] “Tooth Plaque and Dental Tartar”, https://oralb.com, accessed on 26/02/2020 [5] SECTION ON ORAL, HEALTH; SECTION ON ORAL, HEALTH (December 2014), “Maintaining and improving the oral health of young children”, Pediatrics [6] Southam JC, Soames JV (1993), “2 Dental Caries”, Oral pathology (2nd ed.) Oxford: Oxford Univ [7] Smith B, Pickard HM, Kidd EA (1990), “1 Why restores teeth?”, Pickard's manual of operative dentistry (6th ed.), Oxford University Press [8] Daniel Fried, Richard E Glena, John D B Featherstone, and Wolf Seka, “Nature of light scattering in dental enamel and dentin at visible and nearinfrared wavelengths”, 1/3/1995, accessed on 27/02/2019 [9] Cynthia L Darling, Gigi D Huynh, Daniel Fried, “Light scattering properties of nature and artificially demineralized dental enamel at 1310 nm”, Journal of biomedical optic 11(3) (May/June 2006) [10] “Scattering”, https://en.wikipedia.org/, accessed on 03/03/2020 [11] Gail F Williamson, RDH, MS, “Intraoral Imaging: Basic Principles, Techniques and Error Correction”, https://dentalcare.com/, accessed on 03/03/2020 [12] “Xray”, https://www.facialart.com/, accessed on 03/03/2020 DEPARTMENT OF ELECTRONICS - BIOMEDICAL ENGINEERING an 41 REFERENCES [13] Dr George Ghidrai, “THE DENTAL RADIOGRAPHY OR DENTAL XRAY”, https://infodentis.com, (May 2019), accessed on 03/03/2020 [14] “RPi Camera (F) Raspberry Pi Camera Module, Supports Night Vision”, https://www.waveshare.com/, accessed on 19/05/2020 [15] Prof Nguyễn Thanh Hải, Dr Trần Quốc Cường, “Giáo trình xử lý ảnh y sinh”, NXB Đại học Quốc gia TP Hồ Chí Minh, 2016 [16] Dr Andrew Greensted, “Otsu Thresholding”, http://www.labbookpages.co.uk, accessed on 20/07/2020 [17] “Arduino”, http://en.wikipedia.org, accessed on 19/05/2020 [18] “Raspberry Pi gì? Các dịng Raspberry Pi phổ biến”, http://bkaii.com.vn, accessed on 22/10/2019 [19] “RPi Camera (F), Supports Night Vision, Adjustable-Focus”, https://www.waveshare.com, accessed on 19/05/2020 [20] “Power management Integrated circuit”, http://en.wikipedia.org, accessed on 19/05/2020 [21] Sam Sattel, “How Power Supplies Work”, https://www.autodesk.com/, accessed on 31/07/2020 [22] “Transformer”, https://en.wikipedia.org/, accessed on 31/07/2020 [23] “Bridge Rectifier”, https://www.physics-and-radio-electronics.com/, accessed on 31/07/2020 [24] “Tkinter”, https://en.wikipedia.org, 27/9/2019, accessed on 18/07/2020 [25] “RealVNC- Product”, http://realvnc.com/, accessed on 19/07/2020 DEPARTMENT OF ELECTRONICS - BIOMEDICAL ENGINEERING an 42 APPENDIX APPENDIX 7.1 CAMERA COMPARISON Table Comparison of Raspberry Camera Name Pixels Sensor Adjustable focus RPi Camera V2 Mega IMX219 RPi NoIR Camera V2 Mega IMX219 RPi NoIR Camera V2 Mega OV5647 RPi Camera (B) Mega OV5647 RPi Camera (D) Mega OV5647 RPi Camera (E) Mega OV5647 RPi Camera (F) Mega OV5647 √ RPi Camera (G) Mega OV5647 √ RPi IR-CUT Camera2 Mega OV5647 √ Infrared night vision √ √ √ √ √ DEPARTMENT OF ELECTRONICS - BIOMEDICAL ENGINEERING an 43 APPENDIX RPi Zero V1.3 Mega Camera3 OV5647 7.2 INFRARED LEDs COMPARISON Table Infrared LEDs Name Picture Wavelength Price Infrared Led Light 5W 850nm 850nm 44.000 VNĐ Infrared 3mm LED 940nm 1.000 VNĐ Infrared highpower LED 740nm 30.000 VNĐ 7.3 RASPBERRY COMPARISON Table The different between raspberry models Raspberry Pi B Raspberry Pi B+ Raspberry Pi B Raspberry Pi Model A+ Raspberry Pi Zero 1.5 GHz 1.4 GHz 1.2 GHz 1.4 GHz GHz 4 Price CPU Clock No of Core DEPARTMENT OF ELECTRONICS - BIOMEDICAL ENGINEERING an 44 APPENDIX GPU Video Core IV Video Core IV Video Core IV Video Core IV Video Core IV RAM 1GB , 2GB, 512MB 4GB DDR2 LPDDR4 1GB DDR2 512MB DDR2 512 MB Camera Yes Yes Yes Yes Yes GPIO Yes Yes Yes Yes Yes USB 2x USB3.0 + 2x USB2.0 4x USB2.0 4x USB2.0 + micro OTG 1xUSB 2.0 micro + micro OTG Power ratings 1.25A @5V 1.13A @5V 1.34A @5V 200mA 160mA 7.4 THE CODE OF MAIN PROGRAM ##import lib import cv2 import os #Get name name = input("Enter a sample name: ") # define the name of the directory to be deleted dir_path = "/home/pi/Desktop" path = dir_path+"/"+name print(path) ## create dir try: os.makedirs(path) except OSError: DEPARTMENT OF ELECTRONICS - BIOMEDICAL ENGINEERING an 45 APPENDIX print ("Creation of the directory %s failed" % path) else: print ("Successfully created the directory %s" % path) ##take photos cam = cv2.VideoCapture(0) cam.set(cv2.CAP_PROP_FPS,15) print("Press spacebar to take photo!",) print("Press Esc to Escape") img_counter = while True: ret, frame = cam.read() out = frame[100:280,220:400] #Convert RGB to Gray out_gray = cv2.cvtColor(out, cv2.COLOR_BGR2GRAY) # Loc nhieu bang bo loc GaussianBlur out_blur = cv2.GaussianBlur(out_gray, (3, 3), 0) #Histogram Equalization hist = cv2.equalizeHist(out_blur) #Increase contrast and brightness alpha = 1.1 beta = 20 new_out = np.zeros(hist.shape, hist.dtype) for y in range(out.shape[0]): new_out[y] = np.clip(alpha*hist[y] + beta, 0, 255) # Binary ret1, thresh = cv2.threshold(hist,0,255,cv2.THRESH_BINARY+cv2.THRESH_OTSU) ret2,thre = cv2.threshold(new_out,ret1-60,255,cv2.THRESH_TOZERO) #Show Image DEPARTMENT OF ELECTRONICS - BIOMEDICAL ENGINEERING an 46 APPENDIX cv2.imshow("Caries", thre) if not ret: break k = cv2.waitKey(1) if k == 27: # ESC pressed print("Escape hit, closing ") break elif k == 32: # SPACE pressed img_name = "{}_Original.png".format(img_counter) cv2.imwrite(path+"/"+img_name, out) img_name = "{}_Gray.png".format(img_counter) cv2.imwrite(path+"/"+img_name, out_gray) img_name = "{}_Caries.png".format(img_counter) cv2.imwrite(path+"/"+img_name, thre) print("Captured ",img_counter," Image(s)") img_counter += cam.release() cv2.destroyAllWindows() DEPARTMENT OF ELECTRONICS - BIOMEDICAL ENGINEERING an 47 S an K L 0 ... dimensions of the image - L = 2x, x is a bit of the image -