Speed limit signs detection and recognition using scan line technique

44 Dao Thi Phuong, Nguyen Quang Do, Tran Huy Thong SPEED LIMIT SIGNS DETECTION AND RECOGNITION USING SCAN LINE TECHNIQUE NHẬN DẠNG BIỂN BÁO HẠN CHẾ TỐC ĐỘ DÙNG KỸ THUẬT SCAN LINE Dao Thi Phuong, Nguye[.]

44 Dao Thi Phuong, Nguyen Quang Do, Tran Huy Thong SPEED LIMIT SIGNS DETECTION AND RECOGNITION USING SCAN-LINE TECHNIQUE NHẬN DẠNG BIỂN BÁO HẠN CHẾ TỐC ĐỘ DÙNG KỸ THUẬT SCAN-LINE Dao Thi Phuong, Nguyen Quang Do, Tran Huy Thong Hochiminh City International University; Email: dtphuong@hcmiu.edu.vn Abstract - Detection and recognition of traffic sign while driving is not a trivial task because the external environment is surfeited of road signs, billboards, adverts, shop fronts, etc… Currently, an automatic traffic sign detection and recognition system is under research because this system provides an additional level of driver assistance, leading to improved safety for passengers, road users and vehicles In this paper, the algorithm of Speed Limit Traffic Sign Detection and Recognition will be researched The system works in two layers: detection and recognition Sign detection stage finds speed limit sign by using color segmentation combined with shape detection and pixel connectivity; and speed limit sign recognition classifies the speed information inside the sign by using scan-line technique The proposed method was tested on Vietnamese traffic signs with successful detection and recognition rate of about 99% in case of static images and about 84.6% in case of the input being video streams Tóm tắt - Việc phát nhận dạng biển báo giao thông lái xe công việc dễ dàng tình trạng biển báo bị chen lẫn vô số biển quảng cáo, biển hiệu, bảng đèn,… hộ kinh doanh hai bên đường Gần đây, nhà khoa học thực nghiên cứu hệ thống nhận dạng biển báo giao thông tự động, giúp giảm thiểu áp lực cho lái xe, đưa cảnh báo kịp thời để tăng tính an tồn cho bên tham gia giao thông Trong báo này, tác giả tập trung nghiên cứu tìm giải thuật phát nhận dạng biển báo hạn chế tốc độ Key words - speed limit sign; color segmentation; shape detection; pixel connectivity; scan-line Từ khóa - biển báo hạn chế tốc độ; nhận dạng màu; nhận dạng hình dạng; liên kết điểm ảnh; scan-line Introduction The speed limit sign detection and recognition is one of the parts of the automatic traffic sign detection and recognition system The speed limit sign is used to guide the drivers to drive under the maximum speed [1] This helps reduce the number of traffic accidents caused by over-speed [1] An automatic speed limit sign recognition system may be helpful to make drivers aware of speed limit information which they have missed due to tiredness, distraction or lack of concentration [1] The traffic sign detection and recognition have been studied for a long time The first method is proposed and attempted to apply in real time system by Akatsuka and Imai [2] [2] The identification of the road signs is achieved by two main stages: detection, and recognition [4] In the detection phase, the image is pre-processed, enhanced, and segmented according to the sign properties such as color or shape The output is a segmented image containing potential regions which could be recognized as road signs The efficiency and speed of the detection are important factors which play a strong role in the whole process, because it reduces the search space and indicate only potential regions [4] In the recognition stage, each of the candidates is tested against a certain set of features (a pattern) to decide whether it is in the group of road signs or not, and then according to these features they are classified into different groups These features are chosen so as to emphasize the differences among the classes [4] Most of the reviewed approaches recognize speed limit traffic signs by using Artificial Neural Network [5][9] and Template Matching [10],[11] However, there are still several problems with these techniques The Artificial Neural Network method requires large amount of training to achieve good result before applying It also requires high processing time for a well-trained robust neural network For Template Matching, the performance is varying in different fonts, damage and rotation of speed performance is also poor [1] Figure Example of limit signs in Vietnam This paper will introduce the new method to recognize the number inside speed limit traffic signs, which is scanline technique [12] In this research, the database is image and video which contains Vietnamese speed limit sign This system will be interested in the object which is designed in a circular shape with a thick red rim, black number inside white background, and all numbers inside speed limit sign have the same height The distance from camera to speed limit sign is about 10 – 12 m Proposed Approach The main stages of this method can be described in Figure 1) Frame data: input video takes frames to process 2) Color segmentation: extract red color regions from input image 3) Shape detection: determine the candidate regions whether circle is or not 4) Sign classification: distinguish the speed limit THE UNIVERSITY OF DANANG, JOURNAL OF SCIENCE AND TECHNOLOGY, NO 6(79).2014, VOL sign with the other red circle traffic signs 5) Sign recognition: read the digits inside each speed limit sign Inputut Table The HSV value of red color Red color Hue (H) < H < 0.08 0.92 < H < Saturation (S) 0.25 < S < Value (V) 0.25 < V < Video data Image Frame 45 When a red traffic sign appears in front of red buildings, red cars, or in front of the other red objects…, the red rim of the traffic sign is melted into the background In order to overcome this problem, the image is inverted, the system will process on none red region inside of traffic sign Then, based on the area of traffic sign with input image, the system will locate that none red According to database, the traffic signs usually occupy an area about 0.3% - 5% of input image Color Segmentation Shape Detection Sign Classification Sign Recognition Outputut Figure The proposed approach Frame Data Videos will be separated to many frames The frame gets rate of system which is equal the multiple of frame rate of camera and maximum processing time In other words, in each array of many frames which the system receives from camera, it will be removed other frames and choice to process with the first frame 3.1 Color Segmentation The input image will be converted into HSV color space which has less variation to light condition than RGB color space, hence the effect of light to the speed limit sign can be reduced to an acceptable level; the fixed threshold value can be applied for segmentation The color transformation is done by the following equation: '  if B  G H ; H = ' otherwise  360 − H ; Max( R, G, B) − Min( R, G, B) S= Max( R, G, B) V= Max( R, G, B) 255 Figure Locate the candidate region 3.2 Shape Detection To detect circle object, the system will calculate the circularity of the objects Based on the properties of circle object, the circularity is computed as follows: circularity = perimeter (4)    area According to research of Thomas B.Moeslund [13] and database, the circularity value is about to 1.2 for the circle traffic sign 3.3 Speed Limit Sign Detection In order to distinguish the speed limit sign with the other red circle traffic signs, the content of non-red circle inside traffic sign is analyzed (1) (2) (3) Where:   0.5( R − G) + ( R − B)   H ' = cos−1    ( R − G) + ( R − B)(G − B)  H ′ = cos −1 { 0.5[(R−G)+(R−B)] } √(R−G)2 +(R−B)(G−B) From HSV image; the value pixels satisfying the red pixel value will be marked as white color, and all rest pixels are marked as black Figure Red circle traffic signs Firstly, the candidate circles inside the traffic sign are extracted, and convert it to gray-scale image Then, adaptive thresholding method is applied to obtain the binary image [14] Adaptive thresholding method was chosen because the surface of traffic signs are usually uneven illumination, the adaptive thresholding method solves this problem very well 46 Dao Thi Phuong, Nguyen Quang Do, Tran Huy Thong Figure Binarized images The binary image result will be labeled objects to identify speed limit sign If there are – labeled objects found with roughly the same height (10 – 100 km/h), the corresponding circle is considered as a speed limit sign achieve exactly result If the number of scan-lines is increased, the speed processing will be affected Pixels at the border between object and background which encountered by scan lines will be marked The number of marked points combine with their location is considered as “features” of each The system based on this feature to classify the digit Table The “features” for each digit #CP 6 6 6 #LLP 2 4 4 4 #RLP 2 4 4 #MP 2 4 4 #ULP 4 4 2 4 #LLP 2 4 4 Vertical axis: Figure Speed Limit Sign (10 – 100km/h) In case only object is found inside of traffic sign (5 km/h), the system will based on the height and width of object to determine the speed limit sign or not According to database, the height of character is about 0.4% – 0.55%, and the width is about 0.2% – 0.35% of traffic sign; the candidate is speed limit sign • # CP: No of points in middle line • # LLP: No of points in left line • # RLP: No of points in right line Horizontal axis: • # MP: No of points in middle line • # ULP: No of points in upper line • # LLP: No of points in lower line Results To evaluate the performance of detection and recognition algorithm, a sample database is set up as show in Table 3a and Table Table 3a Database- static images Figure Speed Limit Sign (5km/h) Database 3.4 Speed Limit Sign Recognition The scan-line algorithm scans character with three vertical scan lines and three horizontal scan lines The three vertical scan lines are: middle line, left line (between column of left boundary and middle line) and right line (between column of middle line and right boundary) The three horizontal scan lines are: middle line, upper line (between row of top boundary and middle line), and lower line (between row of middle line and bottom boundary) km/h 10 km/h 20 km/h 30 km/h 15 35 km/h 40 km/h 28 50 km/h 52 60 km/h 80 km/h Other signs Number of images 192 Table 3b Database - videos Figure Speed limit sign scanned by scan-lines The result will more precisely when using more lines, but lines in vertical and lines in horizontal are able to Database km/h Number of images THE UNIVERSITY OF DANANG, JOURNAL OF SCIENCE AND TECHNOLOGY, NO 6(79).2014, VOL 47 10 km/h 5 Conclusions 20 km/h 30 km/h 40 km/h 60 km/h Other signs In this paper, a scan-line based numerical digit recognition algorithm has been researched Experimental results indicate that the proposed system of this research achieved speed limit sign recognition with high overall accuracy in both images and videos The main advantages of this method are: simple to implement, able to run in real time, good resistance to variation of font, and not require any pre-processing or training Database consists of 300 static images covering different speed limit signs (5, 10, 20, 30, 35, 40, 50, 60, 80 km/h); 26 videos contain different speed limit signs (5, 10, 20, 30, 40, 60 km/h) and other regulatory traffic signs with resolution of 640x480 They are captured and recorded in different environments as urban, suburban…, at different times of day (morning, afternoon) and under different light conditions (under sunlight, non-sunlight…) The results of the proposal approach are shown as follows: 4.1 Color Segmentation Results Applying the color segmentation method on the database signs has proved that the segmentation works well for correctly colored signs Note that this stage only extracts red traffic sign because this paper focuses on speed limit sign which has surrounded by red outer Table Result of color segmentation Segmented False segmented % of segmented Images 300 100% Videos 26 100% Speed limit sign detection results The speed limit sign detection on database signs has also proved to be successful Table Result of speed limit detection Detected False detected % of detected Images 299 99.67% Videos 26 100% 4.2 Speed limit sign recognition results Finally, the recognition of traffic signs is tested All the database signs are fully segmented and well detected These results are shown in the following table: References [1] Wen Liu, Real–time Speed Limit Sign Recognition System A Driving Assistance System, University of Dublin, Trinity College, pp 1, pp 27, August 2011 [2] H Akatsuka and S Imai, Road signpost recognition system, Proc of SAE vehicle highway infra-structure: safety comptatilbility, pages 189–196, 1987 [3] Lukas Sekanina, Jim Torresen, Detection of Norwegian speed limit sign, Proc of the 16th European Simulation Multiconference, Delft, NL, SCS, 2002, p 337-340, ISBN 90-77039-07-4 [4] Hasan Fleyeh, Mark Dougherty, Road Sign Detection and Recognition, Advanced OR and AI Methods in Transportation, pp 645-653 [5] K.A.Ishak, et al., A Speed limit Sign Recognition System Using Artificial Neural Network, Research and Development, 2006 SCOReD 2006, 4th Student Conference on, 2006, pp 127-131 [6] C Hsin-Han, et al., Road speed sign recognition using edge-voting principle and learning vector quantization network, Computer Symposium (ICS), 2010 International, 2010, pp 246-251 [7] A Broggi, et al., Real Time Road Signs Recognition, Intelligent Vehicles Symposium, 2007 IEEE, 2007, pp 981-986 [8] M.L.Eichner and T.P.Breckon, Integrated speed limit detection and recognition from real-time video, Intelligent Vehicles Symposium, 2008 IEEE, 2008, pp 626-631 [9] J.Torresen, et al., Efficient recognition of speed limit signs, Intelligent Transportation Systems, 2004 Proceedings The 7th International IEEE Conference on, 2004, pp 652-656 [10] J Miura, et al., An active vision system for real-time traffic sign recognition, Intelligent Transportation Systems, 2000 Proceedings 2000 IEEE, 2000, pp 52-57 [11] W.Yongping, et al., A Method of Fast and Robust for Traffic Sign Recognition, in Image and Graphics, 2009 ICIG '09 Fifth International Conference on 2009, pp 891-895 [12] W.C.Xu Han Wei, A New Algorithm for Numeral Recognition, Surveying and Mapping of Geology and Mineral Resources, vol 2, p 31, 2002 [13] Thomas B Moeslund, Image and Video Processing, 2nd Edition, 2009 [14] Rafeal C.Gonzalez, Richard E.Woods, Digital Image Processing, 2nd Edition, pp 600 Table Result of recognition stage Recognized False recognized % of recognized Images 107 99% Videos 22 84.6% (The Board of Editors received the paper on 31/03/2014, its review was completed on 21/04/2014) ... 3a and Table Table 3a Database- static images Figure Speed Limit Sign (5km/h) Database 3.4 Speed Limit Sign Recognition The scan- line algorithm scans character with three vertical scan lines and. .. three horizontal scan lines The three vertical scan lines are: middle line, left line (between column of left boundary and middle line) and right line (between column of middle line and right boundary)... Other signs Number of images 192 Table 3b Database - videos Figure Speed limit sign scanned by scan- lines The result will more precisely when using more lines, but lines in vertical and lines