(Tiểu luận) basic programming for electronics final project report project automatic real time cutter based on vision

PHENIKAA UNIVERSITY FALCULTY OF ELECTRICAL ELECTRONIC ENGINEERING -o0o - BASIC PROGRAMMING FOR ELECTRONICS FINAL PROJECT REPORT Project: Automatic real-time cutter based on vision Lecturer: Dr.Minhhuy Le Vu Phuong Thuy Pham Thi Thu Hien Tran Thi Thanh Thuy School year 2021-2022 h Basic Programming for Electronics WORKLOAD DISTRIBUTION Full name Task Vu Phuong Thuy o Cutter tab: coding software and setup hardware o Write report o Make Powerpoint o Edit video o Present software of Cutter tab Pham Thi Thu Hien o Manage tab o Present hardware of Cutter tab Tran Thi Thanh Thuy o Manage tab o Present Manage tab Workload distribution h Basic Programming for Electronics - Final project TABLE OF CONTENTS SECTION 1: PREFACE SECTION 2: INTRODUCTION SECTION 3: PROJECTS DETAILS GENERAL INTRODUCTION 1.1 Some video links 1.2 Idea 1.3 Software interface 1.4 Hardware PROJECT DETAILS 2.1 Cutter 2.1.1 Coding Image Conversion from Color to Binary Image (Fig 6.1) Binary Image Conversion (Fig 6.2) 10 Fill Hole (Morphology) (Fig 6.3) 10 Removing Small Objects (Fig 6.4 ) 11 Particle Analysis (Figure 6.5 ) 11 Calculate real length of the object (Fig 5.2) 12 Arduino Code (Fig.8) .14 2.1.2 Design and Operation .14 Operation 16 2.2 Manage application .18 STEP Get information and list into queue(13.1) 19 STEP 2: Check receive and calculate total revenue(5.2) 22 STEP 3: Write data to file 25 SECTION 4: DISCUSSION .25 SECTION 5: REFERENCES 26 1|Pag e h Basic Programming for Electronics - Final project Section 1: PREFACE “First of all, we would like to thank Phenikaa University for introducing Basic Programming for Electronics into the curriculum This subject gives us much knowledge about control programming, software as well as hardware control The course also helps us improve many soft skills: presentation skills, report writing skills, teamwork skills, etc These are extremely useful knowledge for students and our future work In particular, we would like to express my deep gratitude to the subject lecturer - Dr Le Minh Huy for teaching and imparting valuable knowledge to us during the past study period, he helped us to expand our knowledge of control programming in the field that we are studying, prepared lessons and plans so that students can absorb the lesson most effectively even though this subject is highly rated as very hard We also want to say thank you to Mr Nguyen The Thao for his help from the beginning, Thuy’s father for making our machine from our sketch and Mr Tran Van Quang for his help too We have tried our best to complete this report However, because our knowledge is still limited and our ability to absorb reality is still confusing, the project certainly cannot avoid shortcomings and many gaps Hoping you can look at the strengths and ignore some of the weaknesses of this project Of course, we hope you will consider and comment to improve our project Thank you sincerely!" 2|Pag e h Basic Programming for Electronics - Final project Section 2: INTRODUCTION Automation is increasingly being used for industry and business, particularly since technological advances have increased their range However, it is just applied in big industries or businesses Small business households find it difficult to access technical development because they can not afford operational costs It becomes a big barrier to expanding the automation range The aim of this project was to develop an automatic real-time cutter based on their length and a small application to manage client information The based knowledge described here is LabVIEW Vision, basic knowledge, and Arduino control At the core of the system is the vision part It measures the length of objects measured by the vision system in real-time then controls hardware by Arduino code Besides, there is a manage client information app to make management easier The report begins with a detailed description of the project: idea, hardware and software, design and operation The discussion about results of laboratory and execution are then presented and analyzed Finally, the limitations of the project are discussed and there is some recommendation for future work 3|P a g e h Basic Programming for Electronics - Final project Section 3: PROJECTS DETAILS General introduction 1.1 Some video links Before starting, to understand our project easily, you can watch some our videos: Presentation: https://drive.google.com/file/d/1HTONvMHj1GfDYRcECacIZmYTXtZ1NK2a/view ?usp=sharing Manage tab: https://drive.google.com/file/d/1GMJDhjKJAhcTH_qEHt0zKo1CduQqKXB/view?usp=sharing Hardware: https://drive.google.com/file/d/1Grc_pc-mH39pU9ReUXQ1905ab9Vb0pt/view?usp=sharing Real-time experiment: https://drive.google.com/file/d/1GXsWh0iEkDTvTmJ69rAlAlU1XB3yDsLB/view?u sp=sharing 1.2 Idea In this project, we experiments cutting rice pipes (a kind of snack was made of rice) A member of our team finds it difficult to cut rice pipe with scissors and can not it for a long time Therefore, we think carefully about something that can make it automatic, easier Gradually, we build this idea then make it real After finishing, we continue on developing a small application to manage 1.3 Software interface There are two tabs: Cutter and Manage 4|P a g e h Basic Programming for Electronics - Final project Fig.1 Cutter tab Cutter tab (Fig.1) is used to control cut task Users will interact with green palette We need to choose camera, expected length, allowed error, camera distance we set up and focal length of the camera (some function we will explain later) Then in the pink palette, users can get the video recorded from camera, video after processing, number of particles and especially, we can know the real length of the object and control hardware to cut when it reaches the expected length 5|P a g e h Basic Programming for Electronics - Final project Fig.2 Manage tab Manage tab (Fig.2) is a small application to manage client information Users enter name, kilograms of rice, starting time (the time start to produce) This app will list their name in a queue, then give the amount of money they have to afford and the time they can receive their products In the ‘check receive’ palette, managers only need to enter the client’s name and kilograms of rice when they want to receive their product, then their name will be removed from the queue and list it into the ‘received’ list Finally, you can get the total revenue 6|P a g e h Basic Programming for Electronics - Final project 1.4 Hardware Fig.3 Hardware It is a quite simple machine having a camera(4.1) (we use phone camera here) to capture rice pipe video a, motor(4.3), and a knife(4.2) to cut Fig.4 Hardware’s parts 7|P a g e h Basic Programming for Electronics - Final project Project details 2.1 Cutter Let’s see the workflow of our project 2.1.1 Coding We use this part to measure length to control hardware Fig.5 Main block diagram Figure 5.1 capture video from camera to the next task Next, video will be put in to particle analysis processing to measure object in pixel then calculate the real length (5.2) Finally, it will sent requirement to hardware and cut object(5.3) We will introduce more about this code Let’s see the following flowchart illustrating particle analysis processing From this, we build particle_analysis_subvi 8|P a g e h Basic Programming for Electronics - Final project Fig.7 Difference between before and after processing Calculate real length of the object (Fig 5.2) Back to the main block diagram (Fig.5), after getting left and right coordinate then we multiply it by 0.2 (1 𝑝𝑖𝑥𝑒𝑙 ≈ 0.2𝑚𝑚), we use the following equation to calculate real distance to the left and right coordinate: 𝑊= 𝑃 ∗𝐷 𝐹 Where 𝑊 is the actual size of the object, 𝑃 is the size of the image of the object reflected on the camera, 𝐹 is the focal length of the camera and 𝐷 is distance from camera to object Finally, we subtract real distance to the right coordinate by the real distance to the left coordinate and we can get the real length of the object 12 | P a g e h Basic Programming for Electronics - Final project Fig 5.2* Calculate real length of the object (Fig 5.2) Fig 5.3 Control hardware with a LabVIEW condition by Arduino Fig 5.3 is code to control hardware with a LabVIEW condition by Arduino We make an error for the length that user can enter value for this property In LabVIEW, we just write the data into COM3 They are ‘a’ and ‘b’ and connect them with a boolean to control cases If real length is in this range ( 𝑒𝑥𝑝𝑒𝑐𝑡𝑒𝑑 𝑙𝑒𝑛𝑔𝑡ℎ ± 𝑒𝑟𝑟𝑜𝑟), it will be ‘a’ case, conversely, it will be ‘b’ case 13 | P a g e h Basic Programming for Electronics - Final project Arduino Code (Fig.8) In Arduino code, we code case, if in case ‘a’, the motor will turn on and it will turn off in case ‘b’ Fig.8 Arduino code 2.1.2 Design and Operation To experiment our project, we had the following components Component/Specification Quantity Power supply 12V Arduino Uno Mosfet Resistor DC Motor USB cable Table of components 14 | P a g e h Basic Programming for Electronics - Final project Fig.9 Some our electronic components The following figure is how we set up everything to experiment Fig.10 Experiment setup We set a camera parallel to the plane The camera can change the distance to the object The machine is put as the above figure 15 | P a g e h Basic Programming for Electronics - Final project The motor is put above the gate and connected with a knife in a point and another point is connected with a spring When the motor turns on, it makes the knife go down If the motor turns off, the spring will pull it up Operation After capturing the video, measuring the real length, requirement is sent to arduino to control cutter, we get the result 16 | P a g e h Basic Programming for Electronics - Final project Fig.11 Moment it cut As we can see, the expected length entered is 100 mm and error is 10 mm The machine cut when the real length is 100.639 mm It is not bad But the real length is just the app measured, how about the actual length? We measured it Let’s see the following figure Fig.12 Actual length As we can see, the actual length is approximately equal to the length that our app measured 17 | P a g e h Basic Programming for Electronics - Final project 2.2 Manage application To observe generally, you can see the following flowchart and main block diagram 1 3 Fig.13 Flowchart and Manage tab block diagram There are three steps in this part: Get information and list into queue, Check receive and calculate total revenue, Write data to file 18 | P a g e h Basic Programming for Electronics - Final project STEP Get information and list into queue(13.1) First, we create two arrays then create their local variables to easy to update new element to arrays Then we calculate amount of money by multiplying kilograms with unit price Next, we calculate the time that client can get their products To this, we use sum_subvi and cal_time_subvi Fig.14 sum_subvi Input of this subvi is an array and a numeric and its output is sum array In this case, the array is time array calculated by mutiply kilogram of rice by time per kilogram and the numeric is new time value updated after entering new value and multiplying it with time The subvi will get sum of the last element of array and new value updated For example, we have an time array [1, 3, 2, 4] so we have sum array is [1, 4, 6, 10], if we add new element to time array, the subvi will get the last element of sum array and add to is 10 + = 16 Then we get sum array [1, 4, 6, 10, 16] 19 | P a g e h Basic Programming for Electronics - Final project After getting sum array, we use cal_time_subvi to get time that client can receive their product Fig.15 cal_time_subvi Input of this subvi (Fig.15) is starting time (the time start producing) Entering three properties: hour, minute and second, then we convert it to string then concatenate them and convert the final string to timestamp Next, we get sum of sum array got from sum_subvi and the timestamp As the result, we get timestamp array representing the time that client can receive (Fig.16) Completing the first step, we will get a queue with information of client 20 | P a g e h Basic Programming for Electronics - Final project Fig.16 Block diagram and front panel of the first step 21 | P a g e h Basic Programming for Electronics - Final project STEP 2: Check receive and calculate total revenue(5.2) In this step, we will remove information of received client and list it to new list called ‘received’ We use delete_element_array_subvi to remove element Let we explain it now Fig.17 delete_element_array_subvi As you can see, to build this subvi (Fig.17) we use another subvi called get_index_to_delete_subvi (Fig.18) to get index of deleted element Fig.18 get_index_to_delete_subvi 22 | P a g e h Basic Programming for Electronics - Final project In Fig.18, we consider string first, there is a string array and a string representing names list and the name entered We use while loop to get in order element of array and compare it with the name entered whether they are the same or not Similarly, we compare numeric array representing kilogram of rice with value entered If two previous conditions are satisfied, the loop will be stopped and returns index (number of loops) Back to the delete_element_array_subvi (Fig.17), after getting the index, we use Delete From Array function to remove elements of all array we want to delete This subvi returns deleted array To remove information directly in the queue, we use local variable (Fig.19) as we mentioned before Then we create new array to store informations deleted(19.1) (use Insert To Array function) and plot it in waveform graph Finally, we get total revenue by using the previous sum_subvi After calculating sum array of remained cost, we get the last element of this array It is total revenue(19.2) Fig.19 Step block diagram 23 | P a g e h Basic Programming for Electronics - Final project Let’s observe the difference between before and after removing (Fig.13) When we enter name and kilogram, press ‘enter’ button, information will be sent to received palette Fig.13 Compare between before and after removing information 24 | P a g e h Basic Programming for Electronics - Final project STEP 3: Write data to file This step is very simple We use Write Delimited Spreadsheet function to write data to csv file Section 4: DISCUSSION About our cutter tab, it can measure quite good The length that it can measured is approximately the actual length as mentioned before But actually, it also have some limitations There are errors because of some external influences such as: error of measure instrument, the rice pipe is not straight, etc Therefore, the actual length also has error The way we set up laboratory instruments for example camera distance is also a reason why we have error This project start from basic knowledge about LabVIEW Vision, it can not good in all environment As we can see, our background is a dark plane, don’t have much noise Actually, in some instructions and documents, they use NI Vision Assistant but we can’t use it because of computer display resolution It is not enough to run Vision Assistant, so we have to make everything manually Therefore, to make it better, we need to research deeper to apply more advanced vision knowledge In the future, we will try to develop our project to apply in other environment and it also works good 25 | P a g e h Basic Programming for Electronics - Final project Section 5: REFERENCES [1] Practical Guide to Machine Vision Software - An Introduction with LabVIEW, Kye-Si Kwon and Steven Ready [2] Real Distance Measurement Using Object Detection of Artificial Intelligence, Jae Moon Lee*, Kitae Hwang, In Hwan Jung [3] Arduino-Based Embedded Systems, Rajesh Singh, Anita Gehlot, Bhupendra Singh, Sushabhan Choudhury [4] https://www.youtube.com/watch?v=poi0peoOCDM [5] https://www.youtube.com/watch?v=TukfO4CofPU&t=1359s [6] https://www.youtube.com/watch?v=BoOddqcUyME&t=198s 26 | P a g e h