Nghiên cứu giải pháp nâng cao tốc độ xử lý tín hiệu trong hệ thống thị giác nổi tt tiếng anh

MINISTRY OF EDUCATION AND TRAINING MINISTRY OF NATIONAL DEFENSE ACADEMY OF MILITARY SCIENCE AND TECHNOLOGY DOAN VAN TUAN RESEARCH ON THE APPROACH TO IMPROVE SIGNAL PROCESSING SPEED IN THE STEREO VISION SYSTEM Specialization: Electronic Engineering Code: 52 02 03 SUMMARY OF PhD THESIS IN ELECTRONIC ENGINEERING Ha noi, 2019 The thesis has been completed at: ACADEMY OF MILITARY SCIENCE AND TECHNOLOGY Scientific supervisors: Dr Ha Huu Huy Assoc Prof Dr Bui Trung Thanh Reviewer 1: Assoc Prof Dr Hoang Manh Thang Hanoi University of Science and Technology Reviewer 2: Assoc Prof Dr Le Nhat Thang Posts and Telecommunications Institute of Technology Reviewer 3: Dr Vu Le Ha Academy of Military Science and Technology The thesis was defended at the Doctoral Evaluating Council at Academy level held at Academy of Military Science and Technology at …… , date 2019 The thesis can be found at: - The library of Academy of Military Science and Technology - Vietnam National Library INTRODUCTION The necessity of the thesis Today, science and technology has developed strongly, especially since the industrial revolution 4.0 was initiated from Germany in 2013 One of the factors that dominated the industrial revolution 4.0 is that robots will gradually replace labors working in factories Therefore, the robot must process information in a three-dimensional environment (3D) through the vision system to orient, locate, identify and accurately locate the surrounding objects called stereo vision or 3D robot vision In addition, stereo vision is also applied in identification, regeneration, positioning, surgery, self-propelled vehicles, mapping and in art People want to create a robot vision system like human vision, the simplest vision system is to use stereo camera including two cameras combined with embedded processing system to replace two human eyes Stereo camera information is processed through algorithms based on processing devices such as CPU, DSP, GPU, FPGA and ASIC combined with implementation languages such as Matlab, OpenCV, CUDA Such a system is called a stereo vision system The major challenges for the wellknown stereo visio system are the stereo camera as the data source from the stereo camera image is increasing, the execution speed requires real-time response, high reliability and finite memory capacity To solve this problem, one of the most effective solutions is to develop processing algorithms, while the processing platforms have not yet developed according to human needs The objectives of the thesis Study specific approachs to improve belief propagation algorithms to speed-up of execution and reduce the amount of memory required when executing the disparity map of dense two-frame high resolution stereo camera in the stereo vision system application for 3D robot vision Research objectives and scope of the thesis -The thesis focuses on studying the solution to reduce the energy function of the belief propagation algorithms implements the disparity map of dense two-frame high resolution stereo camera image in the stereo vision system - Stereo camera images are taken from the test data set - Study and propose solutions to improve speed-up BP algorithm to advance the effectiveness of the disparity map implementation Research methodology of the thesis The thesis focuses on researching solutions to optimization energy function of the belief propagation algorithms implements the disparity map of dense two-frame high resolution stereo camera Analyze improvement belief propation algorithms and propose solutions to reduce the the energy function of the belief propagation algorithm and select appropriate processing platform to achieve the purpose of the thesis From mathematical analysis, parameterization of the parameters, the thesis uses simulation tools, data from the test data set to prove the correctness of the research results The scientific and practical contributions of the thesis The disparity map of stereo camera has a very important role in 3D Robot vision From the disparity map, combined with triangulation, the depth map and the distance from the camera to the object can be estimated This technique is widely applied in industry, robotics, surgery, selfpropelled vehicles, localization and mapping The thesis has proposed two solutions to reduce the cost function for belief propagation algorithm The first solution is to reduce the number of nodes in the Markov random field model through loops using the CTF method to level The second solution is to combine the local census transform algorithm and global belief propagation algorithm has improved the cost reduction of the initial start button when implementing the belief propagation algorithm's message update Research content and structure of the thesis The whole thesis consists of 137 pages presented in chapters, 40 drawings, 29 tables and 14 charts Chapter 1: OVERVIEW OF STEREO VISION AND SIGNAL PROCESSING IN THE STEREO VISION SYSTEM 1.1 Overview of stereo vision Stereo vision is a very important component in computer vision and has been researched and developed by many scientists in the last two decades [46] Stereo vision system is widely applied in many areas such as robots, self-propelled vehicles, medical, arts, entertainment and especially in industrial networks 4.0 [59] People want to create a robot vision system that works in a 3D environment similar to human vision called stereo vision system like Figure 1.1, when robots and humans work together, interactive [107] Image information Image processing Application Figure 1.1 Scheme block of stereo vision system 1.2 Model camera 1.3 Calibration methods The camera calibration method will determine the speed and reliability of the camera's internal and external parameters Currently there are a number of classic image calibration methods such as Hall [39], Salvi [37], Tsai [91] and Weng [76] based on the corresponding camera models Each model will have appropriate calibration methods and have different advantages and disadvantages 1.4 Rectification methods Rectification methods optimize finding homologous points in the stereo camera images and improve image processing reliability The rectification method is divided into two types In the first form, the rectification methods after calibration [9], [105] The second form, the rectification methods performed without calibration [26] 1.5 Stereo matching algorithms Over the past two decades, many matching algorithms have been proposed [46] The matching algorithm is classified according to the stereo camera image Matching algorithms for sparse two-frame high resolution stereo camera images such as SIFT [10], SURF [66] are often used for stereo vision systems that require high speed and memory capacity low requirements however not require high reliability, often applied to navigation systems, mapping or SLAM [36] and self-propelled vehicles Matching algorithms for dense two-frame high resolution stereo camera images as [7], [44] which are often used for stereo vision systems that require high reliability, often applied to industrial product inspection systems, 3D visual system of robot vision and in surgery or object reproduction, however, large computational complexity and high memory requirements Matching algorithms for dense two-frame high resolution stereo camera images have three main types: local algorithms [15], [101], global algorithms [48], [78] and semi-global algorithm [24], [90] 1.6 Hardware processing method in stereo vision - Method CPU - Method DSP - Method GPU - Method FPGA/ASIC 1.7 Evaluation hardware processing method in stereo vision From CPU → DSP → GPU → FPGA → ASIC, processing efficiency increases sequentially, while costs and power consumption decrease accordingly Stereo matching algorithms have more flexibility and short development cycles, while hardware performs a longer design cycle with less design flexibility because at the same time the algorithm must be considered optimally and collect hardware map From a practical point of view, the stereo vision processing hardware system needs to be more accessible to real-time stereo vision systems because it consumes low power and is cheaper 1.8 Research directions to improve the efficiency of the stereo vision system - Image segmentation or hierarchy optimization - Occlusion and consistency handling - Matching cost & energy optimization improvement - Cooperative optimization - Efficient memory arrangement method - Advanced VLSI design method 1.9 Conclusion of chapter Chapter presents an overview of the main components of the stereo vision system consisting of two main blocks of image information block and image processing block Each component has also been analyzed and given an assessment of its role in the system The image information block consists of two main components: stereo camera and camera calibration This block provides stereo camera parameters such as image size and depth disparity, internal parameters and parameters outside the stereo camera The parameters also affect the reliability of the system The image processing unit will determine the effectiveness of the system, including software and hardware Software is programming languages that perform processing algorithms including image rectification algorithms, matching algorithms In particular, the role of matching algorithms will primarily affect the efficiency of the system Hardware is the processing platform for implementing software solutions and it also plays a role in improving the efficiency of the stereo vision system In addition, the match choice between the processing platform and the matching algorithm also influences the performance of the stereo vision system The selected hardware is the GPU processing platform of Nvidia GXT 750Ti with 2GB, 460 memory and 128 bit bandwidth using CUDA 7.5 software and QT creator compiler combined with Intel core i7 CPU, RAM GB with Windows 8.1 operating system The GPU processing platform is selected because it supports parallel processing structures, has multiple processor cores, broadband and memory is increasingly being increased in accordance with the experimental program of the thesis Chapter 2: RESEARCH BELIEF PROPAGATION ALGORITHMS AND BUILD THE METHODS TO IMPROVE SIGNAL PROCESSING SPEED IN THE STEREO VISION SYSTEM 2.1 Markov random field Markov random field is a branch of probability theory Markov random field is used as a tool to processing image data modeling, combined with winner-take-all algorithms In addition, the Markov random field is used as a means of generating inference results on images The inferences related to basic image and frame structure will solve problems such as image reconstruction, image segmentation, stereo vision and creating object labeling The markov random field model usually has two forms: grid-like structure and part-based structure 2.2 Belief propagatin Belief propagation uses messages containing the disparity values of the corresponding points and moves between nodes according to iterative methods to perform inference on the graph model This method provides accurate inferences with part-like structure models and provides approximate reasoning for grid-like structure A belief propagation algorithm is used to identify maximum a posteriori (MAP) in markov random field models for stereo vision problems 2.3 Census transform Census transform is a non-parametric transformation algorithm, it does not depend on the light conditions of the image [86] The operating principle of census transform is to convert each pixel into a bit length bit string with local space architecture For each neighboring pixel except the center point will transform respectively into a bit in the sequence of N bits according to threshold if the value of intensity is close, the neighboring bit is greater than the central bit strength value corresponding to a bit equal to 1, then the bit is 2.4 Approachs to improve processing speed-up of belief propagation algorithm - Parallel calculations - Reduce computational complexity - Reduce the amount of memory required when performing - Minimum update messages - Optimize the way to access memory - Reuse memory - Improve reliability - Cooperative optimization - Select appropriate processing algorithms and handling platform The thesis proposes two solutions to improve the processing speed-up for the belief propagation algorithm is a cost reduct energy function and a combined optimization solution 2.5 Propose solutions to reduce cost functions 2.5.1 Proposed algorithm The model of proposed algorithm coarse to fine belief propagation (CFBP: proposed algorithm 1) is built based on the markov random field model in grid-like structure, node with neighborhood as Figure 2.16 Consider G = (E, V) where G is a graphical model, E is a set of nodes, V is a set of edges The node is the label that is assigned the value of the intencity disparity of the stereo corresponding point of the stereo camera, often called data value or data function The edge is the label assigned the disparity value of the two neighboring labels, often called cost smooth or smooth function Figure 2.16 Scheme of proposed algorithm model From the model proposed algorithm shows that the proposed algorithm used the coarse-to-fine (CTF) level method as shown in Figure 2.17 to reduce the number of nodes after loops Method CTF is used to deduce the reduction of the number of nodes by levels After executing CTF level l, the number of nodes on the current loop will decrease S = 2x2l times the number of nodes in the previous loop The cost value for doing 4node reasoning on a node is determined by the formula (2.36) The message in the proposed algorithm passing in a parallel scheme as shown in Figure 2.18 The initial start node selected is the node labeled (0, 0) with the initial '  message values set to be m0'  and m0,0 Figure 2.17 Structure scheme CTF level Figure 2.18 Scheme passing message The energy function at CTF level is given by E CTF ( x)    ( xi ) (2.36) i[1,4] The number of iterations performed CTF level is k 2' and is determined by the formula (2.37) Considering the stereo camera image resolution is m, n and k where m is the number of pixels in the row, n is the number of pixels in the column and k is the number of depth diaparity of the image k2'   log m (2.37) Number of loops k1' be done in every CTF level is defined as: k k1'  log2 m  + The energy function is given by (2.38) k1''  k Z (log m  1) '' (2.50) where Z '' is the coefficient of depth change Calculating the cost value for passing the message of the proposed algorithm is the same as that of the proposed algorithm except that the proposed algorithm must perform k '  k1'  k2' the loop while the proposed algorithm performs the k ''  k1''  k2'' the loop 2.6 Propose solutions to cooperative optimization 2.6.1 Proposed algorithm The model of proposed algorithm census transform belief propagation (CTBP: proposed algorithm 3) is built based on the markov random field model in grid-like structure, node with neighborhood as Figure 2.22 Consider G = (E, V) where G is a graphical model, E is a set of nodes, V is a set of edges The node is the label that is assigned the value of the intencity disparity of the stereo corresponding point of the stereo camera, often called data value or data function The edge is the label assigned the disparity value of the two neighboring labels, often called cost smooth or smooth function Let V1, V2, V3, V4 and E1, E2, E3, E4 respectively nodes and edges of part 1, part 2, part and part of the proposed algorithm model Figure 2.22 Scheme of proposed algorithm model From the proposed algorithm model show that, the start nodes for passing message to be labeled (0, 0) on belief propagation algorithm model m n has been replaced by node which is the labeled node  ,  on the  2 proposed algorithm model The process of finding the corresponding m n point for the node is labeled  ,  is done via census transform method  2 as formula (2.35) with window 3x3 The image is rectified as parts through the corresponding point determined by CT method Each part will be passing message according to BP algorithm with the pass scheme as m n shown in Figure 2.18, the start node is the node labeled  ,  All  2 parts of the model will be implemented simultaneously + Calculate the cost value The parts of the model are the same size, the method of calculating the cost value is the same + Calculate the cost value for part of model Calculating the cost value of passing the message of proposed algorithm is the same calculating the cost value of passing the message of the proposed algorithm 1, only different in the size of the propagation nodes The proposed algorithm has a start pass node that is labeled (0,0) while the proposed algorithm has a start pass node that is labeled m n  ,   2 2.6.2 Proposed algorithm Model of proposed algorithm census transform change space belief propagation (CTCSBP: proposed algorithm 4) is built based on the MRF model in grid-like structure, node with neighborhood the same proposed algorithm as Figure 2.24 However, there is another difference between these two models is that while proposed algorithm must perform the number of loops equal to the number of disparity of the image, proposed algorithm has the number of loops changed according to the ratio of Z '''' according to the formula (2.71 ) compared to the node disparity Let V1, V2, V3, V4 and E1, E2, E3, E4 respectively nodes and edges of part 1, part 2, part and part of the proposed algorithm model Calculating the cost value for passing the message of the proposed algorithm is the same as that of the proposed algorithm except that the proposed algorithm must perform k '''  k the loop while the proposed algorithm performs the k '''' the loop Figure 2.24 Scheme of proposed algorithm model k k ''''  '''' (2.71) Z 2.7 Evaluation methodology Two general approaches to this are to compute error statistics with respect to some ground truth data and to evaluate the synthetic images obtained by warping the reference or unseen images by the computed disparity map [29]: +RMSE (root-mean-squared-error) measured in disparity units between the computed disparity map dC(x,y) and the ground truth map dT(x,y): 1 2 R    dc ( x, y )  dT ( x, y )   N ( x, y )  + Percentage of bad matching pixels: B   ( d C ( x, y )  d T ( x , y )   d ) N ( x, y ) (2.82) (2.83) 2.8 Conclusion of chapter Chapter of the thesis presents the theoretical basis of the belief propagation algorithm is a markov random field including graph theory combined with probability theory Research and application of belief propagation algorithm to determine the disparity map of dense two-frame high resolution stereo camera image Analysis and evaluation of belief propagation algorithms improvements implemente disparity map of stereo cameras, there by giving directions to improve and improve signal processing speed of belief propagation algorithm application in stereo vision system From the analysis and evaluation of the algorithms implemented, the thesis has proposed two solutions to improve the signal processing speed of the belief propagation algorithm application for stereo visison is reduce cost function solution and cooperative optimization solution Both solutions have a model based on the markov random field in grid-like structure, node with neighborhood The reduce cost function solution is represented by two proposed algorithms that are coarse to fine belief propagation (CFBP: proposed algorithm 1) and coarse to fine change space belief propagation (CFCSBP: proposed algorithm 2) The cooperative optimization solution is a combination of local CT algorithms and global algorithms BP, represented by two proposed algorithms that are census transform belief propagation (CTBP: proposed algorithm 3) and census transform change space belief propagation (CTCSBP: proposed algorithm 4) Chapter EXPERIMENT AND EVALUATION OF THE RESULTS 3.1 Tools and experimental data test The experimental system as shown in Figure 3.1 with the PC configuration described in Table 3.1 and stereo camera image in the test data set [30] is described in Table 3.2 Figure 3.1 Experimental system Table 3.1 Describe the PC Desktop configuration Hardware Software CPU RAM Graphic card Operating Application system software Intel 8GB Geforce GTX750 Ti Window QT Creator 5.8 core RAM: 2GB 8.1 OpenCV 3.0 i7 Core: 460 64 bit Visual Studio 2013 BUS: 128 bit CUDA Table 3.2 Test data set Image Right Disparity Symbol Size Disparity Left image name image map true #1 Baby 620x555 300 #2 Aloe 641x555 270 #3 Cloth 626x555 290 #4 Flower 656x555 pots 251 #5 Bowling 665x555 240 #6 Book 695x555 200 3.2.Root mean squared error (RMSE) 3.3 Experiments and results 3.3.1 Belief propagation standard The processing speed of implementing the disparity map of belief propagation algorithm [78] in the system Figure 3.1 has the parameters described as Table 3.1 and test data Table 3.2 is shown in Table 3.3 Table 3.3 Processing speed of BP algorithm (ms) Image #1 #2 #3 #4 #5 #6 TT BP 439 457 442 473 478 494 3.3.2 Proposed algorithm Disparity map results of camera stereo image Table 3.2 when using the proposed algorithm is shown in Figure 3.3 (a) (b) (c) (d) (e) (f) Figure 3.3 Disparity map uses proposed algorithm 1: (a), (b), (c), (d), (e) ,(f) is the corresponding disparity map of the images #1, #2, #3, #4, #5,#6 The processing speed of implementing the disparity map of proposed CFBP algorithm in the system Figure 3.1 has the parameters described as Table 3.1 and test data Table 3.2 is shown in Table 3.6 Table 3.6 Processing speed of proposed CFBP algorithm (ms) Image #1 #2 #3 #4 #5 #6 TT CFBP 206 217 213 224 227 235 3.3.3 Proposed algorithm Disparity map results of camera stereo image Table 3.2 when using the proposed algorithm with the coefficient Z ''  , is shown in Figure 3.4 (a) (b) (c) (d) (e) (f) Figure 3.4 Disparity map uses proposed algorithm 2: (a), (b), (c), (d), (e), (f) is the corresponding disparity map of the images #1, #2, #3, #4, #5, #6 The processing speed of implementing the disparity map of proposed '' algorithm with the coefficient Z  in the system Figure 3.1 has the parameters described as Table 3.1 and test data Table 3.2 is shown in Table 3.9 Table 3.9 Processing speed of proposed algorithm (ms) #1 #2 #3 #4 #5 #6 TT CFCSBP 191 199 195 203 204 211 3.3.2 Proposed algorithm Disparity map results of camera stereo image Table 3.2 when using the proposed algorithm combined with CT algorithm with 3x3 window and length scan xCT  10 , is shown in Figure 3.5 Image (a) (b) (c) (d) (e) (f) Figure 3.5 Disparity map uses proposed algorithm 3: (a), (b), (c), (d), (e) , (f) is the corresponding disparity map of the images #1, #2, #3, #4, #5, #6 The processing speed of implementing the disparity map of proposed CTBP algorithm in the system Figure 3.1 has the parameters described as Table 3.1 and test data Table 3.2 is shown in Table 3.12 Table 3.12 Processing speed of proposed CTBP algorithm (ms) Image #1 #2 #3 #4 #5 #6 TT CTBP 182 185 182 187 188 191 3.3.2 Proposed algorithm Disparity map results of camera stereo image Table 3.2 when using the proposed algorithm with the coefficient Z ''''  , is shown in Figure 3.6 (a) (b) (c) (d) (e) (f) Figure 3.6 Disparity map uses proposed algorithm 4: (a), (b), (c), (d), (e), (f) is the corresponding disparity map of the images #1, #2, #3, #4, #5, #6 The processing speed of implementing the disparity map of proposed '''' algorithm with the coefficient Z  in the system Figure 3.1 has the parameters described as Table 3.1 and test data Table 3.2 is shown in Table 3.15 Table 3.15 Processing speed of proposed algorithm (ms) #1 #2 #3 #4 #5 #6 TT CTCSBP 83 84 83 89 92 94 3.4 Evaluation proposed algorithm 3.4.1 Evaluation proposed algorithm compare with BP algorithm The processing speed of the disparity map execution of the stereo camera depends on factors such as the execution algorithm (programming skills), the compiler, the configuration of the system and the input data Therefore, in order to evaluate the performance processing speed of the proposed algorithm and BP algorithm [78], the thesis has implemented these two algorithms on the same experimental system as Figure 3.1 with the parameters shown in Table 3.1 and test data set as shown in Table 3.2 The results of the processing speed comparison between the two algorithms are described in Table 3.18 and Diagram 4.1 Table 3.18 Compare the processing speed of proposed and BP algorithms (ms) Image BP Proposed Speedup #1 439 206 113,11% #2 457 217 110,60% #3 442 213 107,51% #4 473 224 111,16% #5 478 227 110,57% #6 494 235 110,21% Table 3.18 shows that, for test images of equal size and different disparitys, the execution time is almost unchanged when performing the same algorithm This shows that the processing speed of the disparity map implementation does not depend on the complexity and difference of the sample image, but depends on the resolution of the image In addition, Table 3.18 also shows that the processing speed of the proposed algorithm increased 113.11% compared to the standard BP algorithm when Image implementing image # in Table 3.2 To be intuitive, the thesis describes comparing the performance rates of the two algorithms through diagram 4.1 Diagram 4.1 Compare the processing speed of CFBP and BP algorithms (ms) 3.4.2 Evaluation proposed algorithm compare with BP algorithm In order to evaluate the processing speed of the proposed algorithm 2, thesis compares the processing speed of the proposed algorithm with the processing speed of BP algorithm implementation on the same experimental system as Figure 3.1 with the parameters shown in Table 3.1, the test data set as shown in Table 3.2 and the coefficient depth changes Z ''  The processing speed performance results between the proposed algorithm and the BP algorithm are described in Table 3.21 Table 3.21 shows that, for test images of equal size and different disparitys, the execution time is almost unchanged when performing the same algorithm This shows that the processing speed of the disparity map is not dependent on the complexity and the disparity of the sample image, but depends on the resolution of the image Table 3.21 Compare the processing speed of proposed and BP algorithms (ms) Image #1 #2 #3 #4 #5 #6 BP 439 457 442 473 478 494 Proposed 191 199 195 203 204 211 Speedup 129,84% 129,65% 126,67% 133,00% 134,31% 134,12% In addition, to be intuitive, the thesis describes comparisons, evaluation the processing speed of the proposed algorithm and BP algorithm through Diagram 4.4 Diagram 4.4 Compare the processing speed of CFCSBP and BP algorithms (ms) 3.4.3 Evaluation proposed algorithm compare with BP algorithm The processing speed of the disparity map execution of the stereo camera depends on factors such as execution algorithm, compiler, system configuration and input data Therefore, to evaluate the performance processing speed of the proposed algorithm and BP algorithm [78], the thesis has implemented these two algorithms on the same experimental system as Figure 3.1 with the parameters shown in Table 3.1 and test data set as shown in Table 3.2 The results of processing speed comparison between two algorithms are described in Table 3.24 Table 3.24 Compare the processing speed of proposed and BP algorithms (ms) Image BP Proposed Speedup #1 439 182 141,21% #2 457 185 147,03% #3 442 182 142,86% #4 473 187 152,94% #5 478 188 154,26% #6 494 191 158,64% Table 3.24 shows that, for test images of equal size and different disparitys, the execution time is almost unchanged when performing the same algorithm This shows that the processing speed at which the disparity map is not depend on the complexity and disparity of the image, but depends on the resolution of the image In addition, Table 3.24 also shows the processing speed of the proposed algorithm to increase 141.21% compared to the standard BP algorithm when implementing image # in Table 3.2 To be intuitive, the thesis describes comparing the performance rates of the two algorithms through diagram 4.7 Diagram 4.7 Compare the processing speed of CTBP and BP algorithms (ms) 3.4.4 Evaluation proposed algorithm compare with BP algorithm In order to evaluation the processing speed of the proposed algorithm 4, the thesis compares the processing speed of the proposed algorithm with the processing speed of BP algorithm implementation on the same experimental system as Figure 3.1 with the parameters shown in Table 3.1, test data set as shown in Table 3.2 and the coefficient depth changes Z ''''  The performance results between the proposed algorithm and BP algorithm are described in Table 3.26 Table 3.26 Compare the processing speed of proposed and BP algorithms (ms) Image #1 #2 #3 #4 #5 #6 BP 439 457 442 473 478 494 Proposed 83 84 83 89 92 94 Speedup 428,92% 444,05% 432,53% 431,46% 419,57% 425,53% To be intuitive, the thesis describes comparing the performance rates of the two algorithms through diagram 4.9 Diagram 4.9 Compare the processing speed of CTCSBP and BP algorithms (ms) 3.4.5 Overall comparison proposed algorithm To compare the proposed algorithms, the thesis employs the proposed algorithms on the same experimental system with the same compiler and input data Focusing on comparing performance processing speed, in addition, the thesis also considers factors such as memory capacity requirements and reliability Table 3.29 shows the processing speed at which the proposed algorithms are implemented when implemented on the same system as Figure 3.1 with the configuration as shown in Table 3.1 and input data as shown in Table 3.2 with the choice coefficient depth change Z ''  and coefficient depth change Z ''''  Table 3.29 Compare the processing speed of proposed algorithms (ms) Image BP Proposed Proposed Proposed Proposed #1 439 206 191 182 83 #2 457 217 199 185 84 #3 442 213 195 182 83 #4 473 224 203 187 89 #5 478 227 204 188 92 #6 494 235 211 191 94 Table 3.29 shows that the proposed algorithms have improved performance compared to standard BP algorithms In the proposed algorithms, the proposed algorithm with the coefficient depth change Z ''''  for the best processing speed To be intuitive, the thesis represents comparing the performance processing speed of proposed algorithms through diagram 4.12 Diagram 4.12 Compare the processing speed of proposed algorithms (ms) 3.5 Conclusion of chapter Chapter has experimented with proposed algorithms and belief propagation standard algorithms on the same platform with the same compiler and input test data set Three main factors to compare and evaluate algorithms are performance speed, required memory capacity and reliability The thesis has evaluated each proposed algorithm with belief propagation standard algorithm according to three factors: performance processing speed, required memory capacity and reliability However, in the above three factors, the thesis focuses on evaluating the processing speed of implementation is mainly Most of the proposed algorithms have improved the processing speed of execution, the amount of memory required but must pay for reliability In the proposed algorithms, the proposed algorithm has the most advantages, quickly improving the execution processing speed while the reliability is negligible CONCLUSION The research results of the thesis - The thesis systematized the components of 3D robot vision including component image information and image processing components Evaluate, classify and select components of stereo vision applications for 3D robot vision - State the theoretical basis of the approximate infering algorithm based on the markov random field model of the application to determine the disparity map of the stereo camera image Present solutions to improve the BP deductive algorithm and choose the appropriate processing platform to implement the BP algorithm - Modeling algorithms proposed to improve BP and select processing platforms to improve processing speed compared to BP algorithms Propose a model and develop a program to implement proposed algorithms, from which methods to evaluate the effectiveness of the proposed algorithm are proposed New contributions of the thesis - Proposing a solution to reduce the cost function with two algorithms, which is a coarse to fine belief propagation (CFBP) algorithm using coarse to fine method for reduce number of node after loops and coarse to fine change space belief propagation (CFCSBP) algorithm using the depth change of stereo camera image then improve signal processing speed in stereo vision system - Proposing a solution to cooperative optimization between global BP algorithm and local CT algorithm into two algorithms which are a census transform belief propagation (CTBP) algorithm change the original start node when the pasing message and census transform change space belief propagation (CTCSBP) algorithm using the depth change of stereo camera image then improve signal processing speed in stereo vision system Further research of the thesis Continuing to study and implement the proposed algorithms based on embedded system using real camera stereo data to map the depth of objects and distance from stereo camera to objects THE SCIENTIFIC PUBLICATIONS [1] Doan Van Tuan, Bui Trung Thanh, Ha Huu Huy (2015), Stereo vision system and evaluation of stereo matching algorithms, Journal of Science and Technology of Thai Nguyen University, No 137 July 2015, p 155 - 160 [2] Doan Van Tuan, Bui Trung Thanh, Ha Huu Huy (2017), Solution to improve the speed of the belief propagatin algorithm for stereo vision system, Journal of Science and Technology of Thai Nguyen University, No 162 February / 2017, page 65 - 71 [3] Doan Van Tuan, Bui Trung Thanh (2017), A solution to make disparity map of dense two-frame stereo camera image applications for 3D images and depth maps, Journal of Military Science and Technology Research, No 51 October 2017, pp 100 - 109 [4] Doan Van Tuan, Bui Trung Thanh (2017), Improving the quality of belief propagation algorithms to determine the disparity map application for robot vision, Journal of Military Science and Technology Research, No 52 December / 2017, pp 111 - 120 [5] Doan Van Tuan, Bui Trung Thanh, Ha Huu Huy (2018), Solutions to improve signal processing efficiency in Robot vision, Journal of Military Science and Technology Research, No 53 February 2018, page 19 - 27 ... AND TECHNOLOGY Scientific supervisors: Dr Ha Huu Huy Assoc Prof Dr Bui Trung Thanh Reviewer 1: Assoc Prof Dr Hoang Manh Thang Hanoi University of Science and Technology Reviewer 2: Assoc Prof Dr... Library INTRODUCTION The necessity of the thesis Today, science and technology has developed strongly, especially since the industrial revolution 4.0 was initiated from Germany in 2013 One of... and global belief propagation algorithm has improved the cost reduction of the initial start button when implementing the belief propagation algorithm's message update Research content and structure