University of New Orleans ScholarWorks@UNO University of New Orleans Theses and Dissertations Dissertations and Theses Summer 8-10-2016 Analysis of Optimization Methods in Multisteerable Filter Design Philip Zanco University of New Orleans, New Orleans, pzanco@uno.edu Follow this and additional works at: https://scholarworks.uno.edu/td Part of the Other Electrical and Computer Engineering Commons Recommended Citation Zanco, Philip, "Analysis of Optimization Methods in Multisteerable Filter Design" (2016) University of New Orleans Theses and Dissertations 2227 https://scholarworks.uno.edu/td/2227 This Thesis is protected by copyright and/or related rights It has been brought to you by ScholarWorks@UNO with permission from the rights-holder(s) You are free to use this Thesis in any way that is permitted by the copyright and related rights legislation that applies to your use For other uses you need to obtain permission from the rightsholder(s) directly, unless additional rights are indicated by a Creative Commons license in the record and/or on the work itself This Thesis has been accepted for inclusion in University of New Orleans Theses and Dissertations by an authorized administrator of ScholarWorks@UNO For more information, please contact scholarworks@uno.edu Analysis of Optimization Methods in Multisteerable Filter Design A Thesis Submitted to Graduate Faculty of the University of New Orleans in partial fulfillment of the requirements for the degree of Master of Science in Engineering Electrical By Philip Ronald Zanco B.S University of New Orleans, 2013 August, 2016 © 2016 Philip R Zanco All Rights Reserved ii To my parents, family, colleagues, and friends iii ACKNOWLEDGEMENT This thesis would not have been possible without the guidance and help of several individuals who in one way or another contributed and extended their valuable assistance in the preparation and completion of this study First and foremost, I would like to express my utmost gratitude to my advisor, Professor Dimitrios Charalampidis for his valuable guidance and selfless support during the years of my Masters studies His patience and unfailing encouragement have been the major contributing factors in the completion of my thesis research I am also extremely thankful to my parents, and sister for their never ending love and support Finally, I would like to thank all the professors and staff at the University of New Orleans who contributed in providing me with a quality education iv TABLE OF CONTENTS LIST OF TABLES ………………………………………………………………………… vii LIST OF FIGURES ……………………………………………………………………… viii ABSTRACT ………………………………………………………………………………… ix Chapter …………………………………………………………………………… INTRODUCTION ………………………………………………………………… 1.1 Historical Review of Steerable Filter Applications…………………………………… Chapter …………………………………………………………………………… THEORY OF MULTISTEERABLE MATCHED FILTERS…………………………… 2.1 Basic Concepts and Definitions ……………………………………………………… 2.2 Background Information and Past Approaches …………………………………… 2.3 Design of Single Steerable Filters……………………………………………………… 2.3.1 Matched Filtering and Steerable Filters…………………………………… 2.3.2 Single Steerable Filters …………………………………………………………… 2.4 Design of Multisteerable Filters……………………………………………………… 2.4.1 Creating a Checkerboard Filter from Two Edge Filters ………………………… 2.4.2 Properties of Multisteerable Filters ……………………………………………… 10 2.4.3 Synthesizing Multisteerable Filters from Polynomials…………………………… 11 2.4.4 Fourier Implementation of Steerable Filters ……………………………………… 12 2.5 Additional Techniques………………………………………………………………… 13 2.6 Similarity Function…………………………………………………………………… 14 Chapter …………………………………………………………………………… 17 EFFICIENT IMPLEMENTATION AND OPTIMIZATION………………………………………………………………………… 17 3.1 Overview ……………………………………………………………………………… 17 3.2 Modifications to Polynomial Method……………………………………………… 17 3.3 Storing Coefficients…………………………………………………………………… 18 3.4 Corner Detection……………………………………………………………………… 19 3.5 Optimization Methods……………………………………………………………… 20 3.5.1 Gradient Descent Optimization………………………………………………… 20 3.5.2 Particle Swarm Optimization……………………………………………………… 25 Chapter …………………………………………………………………………… 27 SIMULATION AND RESULTS………………………………………………………… 27 4.1 Synthesized Corners…………………………………………………………………… 27 4.1.1 Example 1………………………………………………………………………… 28 4.1.2 Example 2………………………………………………………………………… 30 4.2 Synthesized Corners with Noise Added……………………………………………… 32 4.2.1 Example – Speckle Noise……………………………………………………… 35 4.3 Corners in Real Images………………………………………………………………… 38 4.3.1 Example – Corners in Baboon Image…………………………………………… 39 4.3.2 Example – Corners in Stool Image……………………………………………… 41 4.3.3 Example – Corners in Cameraman Image……………………………………… 42 v Chapter …………………………………………………………………………… 43 CONCLUDING REMARKS AND FUTURE WORK ………………………………… 43 5.1 Concluding Remarks …………………………………………………………… 43 5.2 Future Work …………………………………………………………………… 45 BIBLIOGRAPHY ………………………………………………………………………… 46 VITA ……………………………………………………………………………………… 48 vi LIST OF FIGURES 2.1 2.2 2.3 Polar Separability of a Single Steerable Filter…………………………………… Creation of a Checkerboard Pattern from Two Edge Filters……………………… Synthesizing a Corner Template from Two Edge Templates……………………… 11 3.1 3.2 3.3 3.4 3.5 Examples for α=90°, β=0° (left) and α=0°, β=90 (right) ………………………… 18 Objective Function with Multiple Local Maxima………………………………… 22 Comparison of Feature with Template…………………………………………… 23 Decrementing α by 30° (left) and Incrementing α by 30° (right) ………………… 23 Objective Function with Indistinguishable Local Optimum……………………… 24 4.1 4.2 4.3 4.4 4.5 4.6 4.7 4.8 4.9 4.10 4.11 4.12 4.13 Comparison of Corner Orientation Estimation Optimization Methods – Ex 1… 28 Comparison of Iterations Required Between Methods – Ex 1…………………… 30 Comparison of Corner Orientation Estimation Optimization Methods – Ex 2… 30 Comparison of Iterations Required Between Methods – Ex 2…………………… 32 Graphical Comparison of Optimization Methods with Speckle Noise Present…… 35 Similarity Function of a Corner with Speckle Noise……………………………… 36 Similarity Function Zoomed in Around Convergence Point……………………… 37 Similarity Function of a Corner with No Noise Added……………………… …… 38 Images Used for Real Corners…………………………………………… 39 Graphical Comparison of Optimization Methods on Baboon Image……………… 40 Comparison of Iterations Required Between Methods……………………… …… 40 Graphical Comparison of Optimization Methods on Stool Image……………… … 41 Graphical Comparison of Optimization Methods on Cameraman Image……… … 42 vii LIST OF TABLES 2.1 Comparison of Spatial Domain and Fourier Domain Implementations……… …… 16 4.1 4.2 4.3 4.4 4.5 Comparison with Ideal Conditions for 100 Trials per Case……….….….….… … 28 Accuracy Comparison of Optimization Methods……… ….….….….….….….… 29 Gradient Descent Converges to Local Maximum……… ….….….….….….… … 31 Comparison with Noisy Conditions for 100 Trials per Case……… ….….….….… 33 Accuracy Comparison Under Noisy Conditions……… ….….….….….….….…… 35 viii ABSTRACT The purpose of this thesis is to study and investigate a practical and efficient implementation of corner orientation detection using multisteerable filters First, practical theory involved in applying multisteerable filters for corner orientation estimation is presented Methods to improve the efficiency with which multisteerable corner filters are applied to images are investigated and presented Prior research in this area presented an optimization equation for determining the best match of corner orientations in images; however, little research has been done on optimization techniques to solve this equation Optimization techniques to find the maximum response of a similarity function to determine how similar a corner feature is to a multioriented corner template are also explored and compared in this research Key Words: Steerable filters, feature detection, orientation estimation, corner detection ix While the gradient descent method did show a significant statistical increase in the average number of iterations required to converge when Salt & Pepper noise was added, this method did not show a significant increase when Speckle noise was added One thing to note about the gradient descent method is that it is set to stop running after 10,000 iterations have occurred to prevent the program from running indefinitely Some of the trial runs for the Salt & Pepper method failed to converge based on the stopping criteria before 10,000 iterations were performed, which significantly impacted the average convergence rate for this method For the case of Salt and Pepper noise, the particle swarm method again showed no significant change from the perfect conditions However, when Speckle noise was added to the corner, the particle swarm method did converge to a local maximum 7% of the time This may be due to the fact that the area around the global maximum with values close to the global maximum is smaller under these types of noise conditions Since this method is based on randomization, the probability of finding a point near the global maximum when the area surrounding the global maximum is smaller goes down However, the particle swarm method should be more robust to noise if more iterations are performed Clearly, the particle swarm method is more robust to noise in the presence of all types of noise that were tested for in these trials This may be due to the fact that there are more local maxima in the similarity function when noise is present, which could cause the gradient descent to converge to local maxima more often Since the particle swarm method is based on randomization, however, it is less prone to converging to a local maximum Since in the event Speckle noise is present in an image, both methods performed the worst, an example will be shown in the following section for this type of noise in order to get a better idea of why both methods are converging to local maxima more often 34 4.2.1 Example – Speckle Noise In this example, the values of α and β were randomly chosen to be 221.1638° and 7.2180° respectively Figure 4.5 shows the generated corner with speckle noise added, as well as the detected corner orientations for each method From this Figure, it is evident that the particle swarm method converged near the correct solution, but the gradient descent method did not Table 4.3 provides a more in depth analysis of the accuracy of the results for each method In this example, the particle swarm method converged in 463 iterations, while the gradient descent method converged in 3478 iterations While the particle swarm method still shows its advantage of being able to converge faster, in this example it also shows that it has a greater tendency to converge to a global maximum, especially when noise is present Figure 4.5: Graphical Comparison of Optimization Methods with Speckle Noise Present Table 4.5: Accuracy Comparison Under Noisy Conditions Alpha Error (α) Beta Error (β) Actual Corner 221.1638 7.218 - Brute Force 230.616 9.4522 3.794 3.424 Gradient Descent 136.174 84.9898 183.178 175.96 Particle Swarm 226.971 5.8072 4.429 2.789 From Table 4.5, it is evident that the gradient descent method converged to a true local maximum While the gradient descent method converged an orientation that is oriented in a similar direction as the actual corner, the overall error is quite large because it did not converge to the correct point A graph of the similarity function for this example is shown in Figure 4.6 35 Figure 4.6: Similarity Function of Corner with Speckle Noise Figure 4.6 shows the similarity function obtained from the brute force method The local maxima were obtained, and plotted as red crosses on the figure From the figure, we can see that there are many local maxima in this image Figure 4.7 shows the graph in Figure 4.6 zoomed in around the point where the gradient descent method converged While clearly there are larger maxima oriented in the -α direction, the point on which the gradient descent method converged is a region in the similarity function that is relatively flat For this reason, the gradient will never be large enough to cause a move toward convergence in the direction of the global maximum 36 Figure 4.7: Similarity Function Zoomed in Around Convergence Point For comparison, Figure 4.8 shows the cost function for an ideal corner with no noise for the same values of α and β Again, the local maxima are plotted on top of the similarity function as red crosses For the ideal corner, there are still local maxima, but they appear at small angle differences, which would cause the gradient descent method to reinitialize, but not converge at those local maxima Thus, it is apparently that noisy conditions may cause extraneous local maxima to appear in the similar function, which may cause problems for the convergence of the gradient descent method to the global maximum Real images are captured from imaging systems that are not perfect, and thus there is noise present in these images For this reason, an optimization method based on randomization such as particle swarm is expected to perform much better on real images 37 Figure 4.8: Similarity Function of Corner with No Noise Added 4.3 Corners in Real Images Next, it was desired to test each optimization method using corners from real images Figure 4.9 shows different images that were used to test these optimization methods Some of the images contain simple patterns where the corners should be easily apparently, while others are more complicated images from the real world In all of these cases, the images were first converted to grayscale images since this method only works on grayscale images Then the Harris corner detection algorithm was used to obtain the coordinates of corners in real images A window of size 61 by 61 around the corners was used as input to the orientation detection algorithm 38 Figure 4.9: Images Used for Real Corners 4.3.1 Example - Corners in Baboon Image In this example, corners were found in the image of the baboon Figure 4.10 shows the corner in the original image as detected by Harris corner detection, as well as the detected corner orientations for each method From this figure, it is evident that all of the optimization methods converged to the same solution on this corner from a real image We can also see from this Figure that the corner that was detected is quite different from the types of corners that were synthesized In this case, the orientation of the corner could just have been determined to be shifted 180° from what was detected This problem is caused because of what we consider to be a corner In reality, every pixel in an image could be considered to be a corner, but tradeoffs must be made in order to use corners as features in images Figure 4.11 shows a comparison of the number of iterations required for each method Again, particle swarm converges much faster than the gradient descent method 39 Figure 4.10: Graphical Comparison of Optimization Methods on Baboon Image Figure 4.11: Comparison of Iterations Required Between Methods 40 4.3.2 Example - Corners in Stool Image In this example, corners were found in the image of the stool Figure 4.12 shows the corner in the original image as detected by Harris corner detection, as well as the detected corner orientations for each method From this figure, it is evident that all of the optimization methods converged to the same solution on this corner from a real image The detected corners, however, closely resemble the corner in the original image We can see that the left edge of the corner is slightly skewed to the left as opposed to the original image, but this is most likely caused by the fact that the center of the corner is slightly off Figure 4.12: Graphical Comparison of Optimization Methods on Stool Image 41 4.3.3 Example – Corners in Cameraman Image In this example, corners were found in the image of the cameraman Figure 4.13 shows the corner in the original image as detected by Harris corner detection, as well as the detected corner orientations for each method From this figure, it is evident that all of the optimization methods converged to the same solution on this corner from a real image In this case, it would be very difficult for a human to determine that this point is a corner However, since there is a larger mass of white pixels on the left side of the image, the orientation estimator correctly determined that the corner was oriented in this direction Figure 4.13: Graphical Comparison of Optimization Methods on Cameraman Image 42 CHAPTER CONCLUDING REMARKS AND FUTURE WORK 1.1 5.1 Concluding Remarks The research presented in the thesis provides a practical method for estimating the orientation of corners in real images The journal paper by Muhlich, Friedrich, and Aach presented the theory of efficiently implementing multisteerable filters in the Fourier domain, but did not show examples of how a corner orientation estimator could be implemented [9] This research focuses on efficiently implementing a multisteerable filter for corner orientation detection In the conclusion of the work by Muhlich, Friedrich, and Aach, they mentioned that further research could be done on finding efficient optimization methods for finding the global maximum in the similarity function [9] This research aimed to fill in the missing information with regards to efficiently solving for the global maximum in the similarity function for corners To increase the speed of determining the orientation angle of a corner, several methods were proposed in this research The first method involving storing the coefficients in memory Since the coefficients are much smaller than an image, they can easily be stored in memory and loaded before attempting to determine the orientation of a corner Storing the coefficients in this way sped up the performance of the corner detection algorithm by around times However, it was noted that further improvements might by implementing the similarity function in parallel For particle swarm method, the calculation of the similarity function for each particle can be done in parallel, which should greatly increase the speed of this method However, the gradient descent 43 method cannot be implemented in parallel because only one point is tested at a time, and the gradient must first be calculated to determine the next point to test Several different optimization methods were tested in this research, which include Levenberg-Marquardt optimization, gradient descent optimization, and particle swarm optimization It was determined that the Levenberg-Marquardt optimization method was not well suited for optimizing the type of similarity function characteristic of corner features Muhlich, Friedrich, and Aach used Levenberg-Marquardt optimization to solve for the global optimum in the similarity function; however, none of their examples involved corners and their similarity functions were obtained with only one orientation angle [9] The gradient descent method performed well on synthesized ideal corners, but took many more iterations to converge than the particle swarm method Moreover, when noise was present, as is typical in real images, the gradient descent method had a much larger probability of converging to a local maximum It was shown that the presence of noise in an image can directly cause more local optimum to appear in the similarity function The particle swarm method, however, performed much better by not converging to a local maximum except in the case of speckle noise However, even when speckle noise was present in an image, the particle swarm method only converged to a local maximum 7% of the time, while the gradient descent method converged to a local maximum over 50% of the time Finally, both optimization methods were tested on corners from real images It was shown that all of the optimization methods converged to the global maximum for the examples shown in this thesis; however, the particle swarm method still proved to be superior to the gradient descent method in terms of number of iterations required to converge One important note from corners in real images is that they not always appear to be corners to a human being Moreover, it is often 44 difficult to distinguish by eye the orientation of a corner However, it is not very important the orientation of the angle be determined with extreme accuracy What is most important for most computer vision applications is that a method of determining the orientation of corners remains consistent under various homography transformations, and will not converge to local maximum For this reason, the particle swarm optimization method described in this thesis provides a good framework for comparing orientation angles of corners in images 5.2 Future Work For future work, other optimization methods could be tested, or modifications to the particle swarm method can be implemented to result in greater accuracy for determining the orientation angles of corners In the work by Muhlich, Friedrich, and Aach, they proposed windowing the coefficients with a window such as the Hamming window in order to reduce oscillations [9] However, oscillations are similar to noise, and particle swarm optimization method proved to be quite robust to noisy conditions, so a windowing function may not be necessary Furthermore, windowing the coefficients in this way reduces the sharpness of the edges that make up the corner, which makes it difficult for there to be extremely high precision in determining the orientation angle Further research could be done on implementing multisteerable filters for corners without the use of windowing function in order to increase the accuracy of the orientation estimator Lastly, further improvements could be made by exploring a parallel implementation of multisteerable filters using the graphics processing unit 45 BIBLIOGRAPHY [1] D Tamrakar and P Khanna, "Multispectral palmprint recognition using steerable filter," 2014 9th International Conference on Industrial and Information Systems (ICIIS), Gwalior, 2014, pp 1-5 [2] Qi Li, Isao Sato and Yutaka Murakami, "Steerable filter based multiscale registration method for JERS-1 SAR and ASTER images," 2007 IEEE International Geoscience and Remote Sensing Symposium, Barcelona, 2007, pp 381-384 [3] E Shang, J Li, X An and H He, "Lane Detection Using Steerable Filters and FPGA-based Implementation," Image and Graphics (ICIG), 2011 Sixth International Conference on, Hefei, Anhui, 2011, pp 908-913 [4] S Xu, Y Jia and X Zhang, "Coding Facial Expression with Oriented Steerable Filters," 2006 International Conference on Image Processing, Atlanta, GA, 2006, pp 2057-2060 [5] D Alexiadis, N Mitianoudis and T Stathaki, "Multidimensional steerable filters and 3D flow estimation," 2014 IEEE International Conference on Image Processing (ICIP), Paris, 2014, pp 2012-2016 [6] Zhou Weiping and Shu Huazhong, "Detection of Cerebral Vessels in MRA Using 3D Steerable Filters," 2005 IEEE Engineering in Medicine and Biology 27th Annual Conference, Shanghai, 2005, pp 3249-3252 [7] A Ozmen and E Tufan Akman, "Edge detection using steerable filters and CNN," Signal Processing Conference, 2002 11th European, Toulouse, 2002, pp 1-4 [8] S Anand and H Abirami, "Non-separable steerable filters for edge detection," Signal Processing, Communication, Computing and Networking Technologies (ICSCCN), 2011 International Conference on, Thuckafay, 2011, pp 665-669 [9] M Mühlich, D Friedrich and T Aach, "Design and Implementation of Multisteerable Matched Filters," in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol 34, no 2, pp 279-291, Feb 2012 [10] F David, J Ponce Computer Vision: A Modern Approach Boston: Pearson, 2012 [11] Z Ye, Y Pei and J Shi, "An Improved Algorithm for Harris Corner Detection," Image and Signal Processing, 2009 CISP '09 2nd International Congress on, Tianjin, 2009, pp 1-4 [12] H Jiang, T P Tian and S Sclaroff, "Scale and rotation invariant matching using linearly augmented trees," Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on, Providence, RI, 2011, pp 2473-2480 [13] M Jacob and M Unser, "Design of steerable filters for feature detection using canny-like criteria," in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol 26, no 8, pp 1007-1019, Aug 2004 46 [14] W T Freeman and E H Adelson, "The design and use of steerable filters," in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol 13, no 9, pp 891-906, Sep 1991 [15] J Bigun, G.H Granlund, “Optimal Orientation Detection of Linear Symmetry,” Proc IEEE First Int’l Conf Computer Vision, pp 433-438, June 1987 [16] J Bigun, G H Granlund and J Wiklund, "Multidimensional orientation estimation with applications to texture analysis and optical flow," in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol 13, no 8, pp 775-790, Aug 1991 [17] M Kass and A Witkin, “Analyzing Oriented Patterns,” Computer Vision, Graphics, and Image Processing, vol 37, pp 362-385, 1987 [18] S Di Zenzo, “A Note on the Gradient of a Multi-Image,” Computer Vision, Graphics, and Image Processing, vol 33, pp 116-125, 1986 [19] M Shizawa and K Mase, "Simultaneous multiple optical flow estimation," Pattern Recognition, 1990 Proceedings., 10th International Conference on, Atlantic City, NJ, 1990, pp 274-278 vol.1 [20] M Shizawa and K Maze, "Unified computational theory for motion transparency and motion boundaries based on eigenenergy analysis," Computer Vision and Pattern Recognition, 1991 Proceedings CVPR '91., IEEE Computer Society Conference on, Maui, HI, 1991, pp 289295 [21] T Iso and M Shizawa, “Detecting L-, T-, and X-Junctions from Low-Order Image Derivatives,” Proc Visual Comm And Image Processing, pp 1185-1197, Nov 1993 [22] T Aach, C Mota, I Stuke, M Muhlich and E Barth, "Analysis of Superimposed Oriented Patterns," in IEEE Transactions on Image Processing, vol 15, no 12, pp 3690-3700, Dec 2006 [23] M Mulich and T Aach, “A Theory for Multiple Orientation Estimation,” Proc European Conf Computer Vision, H Bischof and A Leonardis, eds., pp 69-82, 2006 [24] M Muhlich and T Aach, "Analysis of Multiple Orientations," in IEEE Transactions on Image Processing, vol 18, no 7, pp 1424-1437, July 2009 47 VITA The author was born in 1991 in New Orleans, Louisiana He completed his Bachelors of Science in Electrical Engineering from the University of New Orleans in 2013 He is expected to finish his Masters in Electrical Engineering from the University of New Orleans in August 2016 His areas of interests are computer vision and power system protection 48 .. .Analysis of Optimization Methods in Multisteerable Filter Design A Thesis Submitted to Graduate Faculty of the University of New Orleans in partial fulfillment of the requirements... The development of the design theory of multisteerable filters provided a significant increase in the possibility of efficiently determining the orientation of corner features in images Once these... relationship of the orientation angles of the individual single-oriented filters directly relates to the multiple orientations of the resultant multisteerable filter [9] By inserting single steerable filters