This paper presents the improved multiple-source Dijkstra algorithm as an efficient solution to the multiple objects, interactive image segmentation problem, which can be applied for precomputing process in many tasks of image processing. Given an input image, we can build a sparse undirected graph of pixels and pick some pixels with predefined labels as sources.
Research IMPROVED DIJKSTRA'S ALGORITHM ON THE IMAGE SEGMENTATION PROBLEM Nguyen Nam Phuc1*, Le Tien Hung2, Nguyen Quoc Trung3, Ha Huu Huy4 Abstract: This paper presents the improved multiple-source Dijkstra algorithm as an efficient solution to the multiple objects, interactive image segmentation problem, which can be applied for precomputing process in many tasks of image processing Given an input image, we can build a sparse undirected graph of pixels and pick some pixels with predefined labels as sources By running the Dijkstra’s algorithm on the whole graph, the closest source with the corresponding label from all pixels can be determined efficiently in terms of both time performance and accuracy The main contributions include the idea of parallel precomputing the local shortest distances in a neighborhood of each pixel to build an undirected graph and combining label-correcting methods with the Dijkstra’s algorithm to reduce the running time Our experimental results on real-world images demonstrate significant improvement to the existing image segmentation methods in terms of both time performance and accuracy Keywords: Shortest distance; Dijkstra’s algorithm; Graph; Parallel; Image segmentation; Label correcting INTRODUCTION In recent years, the task of image segmentation has attracted a much attention in a variety of interactive applications [1] Roughly speaking, the ultimate goal of the task is to extract objects from images with as few user interactions as possible Many image segmentation algorithms have been proposed in the literature and could be classified into different groups as edge detection [2], thresholding [3] and clustering based [4] segmentations However, designing an automated image segmentation technique which can be applied fully in real-world applications is still a challenging problem The main reason is about the large requirement of input information from user to define the ground truth of objects, especially when the input images contain multiple objects with different colors or when users need to change the input information frequently to archive satisfactory results And for the iris/vessel detection problems [5,6], it is difficult to enhance multiscale vessellike structures, especially on the images with low contrast or high homogeneity of illumination conditions In contrast, the graph based approaches [7] have been broadly used in largescale image database applications and they have become a major trend in image segmentation Greig et al [8] evaluated two-colour or binary images by using Ford-Fulkerson algorithm for finding the maximum flow in a capacitated network Felzenszwalb determined the similarity of image regions by measuring the internal variation between neighboring pixels [9] Recently, Wei et al [10] presented a multi-source variant of the Dijkstra's algorithm which allows us to perform interactive segmentation User can predefine labels for a small number of pixels (seeds) and then the closest seed from each unlabeled pixel is determined in an accurately way (see Fig.1) Their computational approach is promising since it requires the initiating settings from user only once Journal of Military Science and Technology, Special Issue, No.60A, 05 - 2019 41 Electronics & Automation (a) (b) Figure Seeds and represent two different labels and (a) By finding the closest seeds, the whole pixels in the image can be properly marked (b) Figure is borrowed from [10] All of the existing graph based algorithms use only single thread to check the local connectivity of the embedded graph before each propagation of the Dijkstra’s algorithm And because the Dijkstra’s algorithm performs at most one iteration per node, its required extra overhead (e.g., to extract the node with minimum label) depends on the total number of nodes in the whole graph However, with the rapid development of scanning technology, the input images for segmentation could have very large size nowadays It means the computational cost is expensive for the classical Dijkstra's algorithm and it may not be executed interactively The speed time of the Dijkstra's algorithm is more concerned in the cases when segmentation must be performed multi time on sets of image databases for analyzing purposes In this paper, we focus on improving the Dijkstra’s algorithm in term of time performance Our framework is based on the idea of partitioning an image into regions by using multiple-source Dijkstra's algorithm for label checking, such that all pixels in each region have the same label The complete framework consists of two stages In the first stage, the input image is mapped into a closed undirected graph where its local connectivity is precomputed parallelly It allows us to break down one of the most expensive part of the Dijkstra’s algorithm into sub-problems which can be easily solved in multi-threading mode In the second stage, there are some pixel with individual label mapped to will be set as seeds for the Dijkstra's algorithm In each iteration, nodes with new distance information are inserted into or extracted from a set/list using incremental sorting operations Once the algorithm ruminates, the label of all unprocessed pixels is changed to the label of their closest seed by computing their shortest distances from the seeds To avoid using expensive internal sorting in linked set/list as common and reduce the extra overhead, we adapted the label correcting strategies on priority queue Thanks for that the propagating speed is boosted up significantly and our framework can be applied in interactive applications We have evaluated our framework on real-world images and observed promising results The rest of the paper is organized as follows Section reviews related work on image segmentation and the Dijkstra's algorithm Section explains about the graph generation and how to efficiently perform the Dijkstra's algorithm on it with the label correcting strategies applied Section contains our experimental results and discusses the merits and limitations of our work Finally, Section concludes the paper 42 N N Phuc, …, H H Huy, “Improved Dijkstra’s algorithm … segmentation problem.” Research RELATED WORKS 2.1 Graph-based segmentation Image segmentation is a wide range topic Therefore, in this section we will briefly discuss only some of the classical interactive and graph-based methods in recent years For the graph-based algorithms, the image segmentation is treated as a graph partitioning problem In such approaches, a graph G=(V, E) needs to be prebuilt where each node vi V of the graph corresponds to a pixel pi of the input image Certain nodes vi, vj V are connected by edges eij E whose weights measure the distance between the corresponding pixels pi, pj in some feature space There are several weighting functions can be found in [1,11] In our work, an exponential weighting function is introduced for gray images as from [11]: ( Ii I j )2 wij e (1) 1 where Ii indicates the value of intensity at pixel pi and is the sensitivity coefficient When the input image is given in RGB mode, Eq (1) is modified as below with Ci is the color vector at pi: ( Ci C j )( Ci C j )T wij e 1 (2) In the graph-based approaches, each component (or region) in a segment S corresponds to a connected subgraph G' = (V; E'), where E' E This means any segmentation is induced by a subset E' E The quality of a segmentation can be measured in different ways, but in general we prefer all the elements to be similar in the same component to be dissimilar in different components In other words, the edges between two vertices should have relatively low weights in the same component and higher weights in different components The earliest graph-based algorithms use fixed thresholds plus local measures for segmenting Zahn [13] proposed an algorithm based on the minimum spanning tree of the graph This algorithm has been extended for point clustering with the weights are computed based on distances between points Another algorithm proposed in [14] uses a measure based only on the nearest neighbors of each node in graph to decide which edges to break in partitioning However, the local variability around nodes alone may not be enough to get a reasonable measure of image variability over arbitrarily large regions of the input image In the graph cuts based algorithms [1], the chosen foreground/background seeds are treated as sources for max-flow/min-cut operations to returns a set of edges with minimum total weight as boundary One of the main issues in the graph cuts is that the boundary may take a shortcut over a protruded section of the object in an attempt to minimize boundary length Moreover, the segmentation results are strongly sensitive to noise and becomes fragmented on images with low quality In the random walks based algorithms [15], the weight of an edge eij was treated as the probability of a particle at node vi traveling to its neighboring node vj Given the Journal of Military Science and Technology, Special Issue, No.60A, 05 - 2019 43 Electronics & Automation seeds, one may compare the probability that a particle initiating at any pixel travels first to the foreground or background seeds and then assign that pixel to the corresponding label A common problem of random walks based algorithms is the tendency to produce segmentations that are overly likely to provide balance between the foreground and background, which may make images hard to be properly segmented 2.2 Multi-source Dijkstra's algorithm The graph based approaches naturally link the segmentation problem on an image and the shortest path problem on a graph, since the label of each pixel pi is set depends on the shortest distances between sources and the corresponding node vi Although the numerical techniques [16,17] have advantage in fast computing the shortest distances, their accuracy and stability are still the common issues we need to concerns about for segmentation of small or non-trivial shapes In contrast, the computational solutions, includes the Dijkstra's algorithm [18], are more reliable and they are widely used for especially in iris/vessels detecting applications Recently, Wei et al [10] adopted a multi-source variant of the Dijkstra's algorithm to change the label on pixels by running once Their idea is to find the closest seed for each pixel but not the shortest distance from all of the seeds Suppose the certain weighted graph G = (V, E) is given and there exists two kinds of seeds S1 = {f1, f2,…, fm} and S2 = {b1, b2,…, bn}, where fi denotes the foreground seeds and bi is the marked background seeds Firstly, all of the seeds are initialized with their distance to zero, and labeled with and respectively before putting them into a list Q During the Dijkstra’s expansion, the pixel’s label is marked as same as its closest seed’s label Thus, the foreground can be finally segmented from the background when all pixels in the image are marked with their certain labels Nowadays, with the rapid development of modern image capture/scanning technology, the input images for segmentation are often very large, so the embedded graphs could have millions of nodes or more It means the Dijkstra's algorithm could be the bottleneck of the partitioning process and hard to be used in real-time applications In addition, for the analysis purposes, the partitioning process needs to be carried out multiple times until the results match the user's requirements, so designing graphs with simple data structures for storing is a good way to reduce computing resource and running time THE PROPOSED METHOD 3.1 Efficient graph construction The graph construction is required only once before running the Dijkstra's algorithm, then all information about the local connectivity is stored in conventional incidence lists Each node stores its incident edges and each edge stores two incident nodes plus its length struct Node { unsigned int id; vector id_incident_edges; } 44 N N Phuc, …, H H Huy, “Improved Dijkstra’s algorithm … segmentation problem.” Research struct Edge { unsigned int id_v1, id_v2; //index of the related start and end vertices double length; //edge's length } In graph based algorithms, the 4-connected and 8-connected neighborhood structures are usually used to transfer the local connectivity from the input image into its corresponding graph Thus, only weights of the edges that connect two adjacent nodes can be computed using Eq (1) or (2) Additionally, we can increase area of the local neighborhood to speed up the convergence of the Dijkstra's algorithm, the trade-off is that some disconnected similar objects could be linked because of the wrong labels marked on the pixels between these objects Our construction algorithm is shown in Algorithm For each vertex vi V, we compute distances from vi in a local neighborhood (vi) Then all these distances are taken to form the edges from vi Computing the distances is the bottleneck of the entire graph construction algorithm due to the expensive math functions In our implementation, the distances, i.e the edges' length, can be computed in parallel on the CPU/GPU: the graph data structure is accessible to all threads in the read-only manner, and each thread maintains its own data To avoid the written conflicts, the thread for node vi produces directed edges originated from vi The gain of the constructing time strongly depends on the number of threads on CPU as it is shown in Table Table The total time for edges’ construction on images with different size Tsingle_thread is the time required in the classical Dijkstra’s algorithm which runs on single thread Ttwo_threads and Tfour_threads are our constructing time for edges on two threads and four threads respectively Image size 640*480 Tsingle_thread (ms) 40 Ttwo_threads (ms) 17 Tfour_threads (ms) 16 1080*720 92 185 255 2428 54 123 173 1515 36 95 130 998 1920*1080 2560*1080 3000*4000 Algorithm 1: Parallel Graph Constructing Algorithm Input: Image Output: Connected graph G = (V, E) initialize all nodes vi in V and edges ei in E parallel for each pixel pi in //each loop runs on on separate thread for each pixel pj in the neighborhood (pi) of pi Compute distance dij between pi and pj using Equation (1) or (2) Update dij to edge eij end for Journal of Military Science and Technology, Special Issue, No.60A, 05 - 2019 45 Electronics & Automation end for return graph G 3.2 Improving performance of the Dijkstra's algorithm To compute the shortest distances from a source, say v1, to all of the other nodes in graph G, the Dijkstra’s algorithm maintains a label vector (d1, , d|V |) and a set of nodes C, called the "candidate list", starting with d1 = 0, di = ∞ for all i ≠ and C = {v1} It iteratively processes the nodes from C and terminates only when C is empty Upon the termination, the shortest distance from v1 to node vi is equal to label di In each iteration, it takes the front node v* with the smallest label from C to update distances to v*'s neighborhood Different data structures of C (for example, binary or Fibonacci heaps) can be used to determine the node with smallest label in C Since each node enters and exits C exactly once, the Dijkstra’s algorithm needs |V| iterations to complete In our work, the label correcting methods are adopted to maintain C as a priority queue with multiple entrances, from which nodes can enter or exit in constant time O(1) Node insertion follows the small label to the front (SLF) scheme [19] and node extraction follows the large label last (LLL) scheme [20]: when a node vj enters the priority queue C, its label dj is compared with the label di of the top node vi of C Node vj is placed at the top of C if dj ≤ di; otherwise vj is placed at the bottom of C Node vi remains at the top of C while its label is less than a threshold; otherwise it is sent to the bottom of C The threshold is usually set as the mean label value of all nodes in C The Dijkstra’s algorithm performs |V| iterations at most and requires some extra overhead to extract the node with minimum label per iteration In contrast, the SLF-LL methods requires more than |V| iterations but the overhead per iteration is smaller The performance of the Dijkstra's algorithm and the SLF-LLL methods is evaluated on several images for comparison As Table shows, the SLF-LLL methods can almost double the runtime performance achieved by the Dijkstra’s algorithm for the same image Table Performance of the Dijkstra's algorithm (TDijkstra), and our SLF/LLLenhanced method (TSLF/LLL) on same graphs |V| TDijkstra TSLF/LLL 200K 0.20 s 0.08 s 400K 0.37 s 0.18 s 548K 0.48 s 0.26 s 800K 0.70 s 0.33 s 1.28M 1.09 s 0.51 s Algorithm 2: SLF/LLL-enhanced Dijkstra's algorithm Input: Graph G=(V, E) with sources S={s1, s2, …, sc} Output: The shortest distance vector D = {d1, d2, , d|V|} and the label vector L = {l1, l2, , l|V|} initialize D to ∞; 46 N N Phuc, …, H H Huy, “Improved Dijkstra’s algorithm … segmentation problem.” Research initialize L to 0; initialize status vector P = {p1, p2, , p|V|} to false initialize priority queue C as empty for each source si in S initialize di to initialize li to i initialize pi to true add si to C 10 end for 11 initialize to 12 while C is not empty 13 while true //extract minimum element from C using LLL policy 14 extract minimum element vi from C 15 16 17 18 19 20 21 22 23 24 25 26 27 28 if di greater than then size of C add node vi to bottom of C else set equal to ( - di) set pi equal to false break end if end while for each node vj in the neighborhood (vi) of vi if dj greater than (di+ eij) then initialize dtemp equal to dj set dj equal to (di+ eij) set lj equal to li if pj is false then //add element to C using SLF policy 29 add node vj to C 30 set pj equal to true 31 set equal to ( + dj) 32 else 33 set equal to ( - dtemp) 34 set equal to ( + dj) 35 end if 36 end if 37 end for 38 end while 39 return vectors D and L EXPERIMENTAL RESULTS We implemented our framework on a 64-bit PC with an Intel Xeon cores CPU and 32GB memory The parallelized parts are implemented to run on CPU using Journal of Military Science and Technology, Special Issue, No.60A, 05 - 2019 47 Electronics & Automation multi-threading OpenMP interface Our results are compared with [10] to illustrate the time performance We observed that our parallel implementation on a quadcore CPU was up to times faster than when using a single-core, thus it allows interactive manipulation on image with million pixels (see Table 3) Table Comparing results of graph based algorithms TW is the running time of Wei’s algorithm [10] Our required memory is shown in the third column The last column reports executing time of our framework (Tours) Image size TW Memor y Tours 640*480 0.30 s 14.7 MB 0.10 s 1080*720 0.65 s 37.3 MB 0.30 s 1920*1080 1.15 s 99.5 MB 0.57 s 2560*1080 2.60 s 132.7 MB 1.10 s As we adopted the multi-source Dijkstra's algorithm, our segmentation results should have similar quality in comparison with results of the method proposed in [10] The main advantage obtained is about the time efficiency which allows our work be more interactive and suitable for real-time applications As we divided the construction of embedded graph from the main Dijkstra’s algorithm, this process can be effectively solved on multi-cores CPUs The precomputing process costs less than 20% of total running time of the whole algorithm on our computer with threads CPU This ratio can be smaller if we have more thread available on CPUs, and it means the expensive computational part is not the bottleneck of the Dijkstra’s algorithm anymore Thus, the main performance boost comes from the SLF/LLL-enhanced strategy as Table shows (a) (b) (c) (d) Figure The procedure of our framework (a) and (c) are the original RGB and gray images from BSDS300 database The corresponding segmentation results are in (b, (d) Our experiment is completed on different image datasets Some of our segmentation results are shown in Figures 2, and In the Fig 2, there are groups of pixels picked at the background, red and yellow areas as the seeds for multiple objects segmentation It is noted that the user’s labeled seeds can be freely settled and the precomputing graph needs to be processed only once, so the gain is more noticed in cases we want to modify the previous seeds and rerun the algorithm until the results are satisfied This is the main advantage of our work, especially for interactive image segmentation problems Our proposed framework is also tested on DRIVE database images for vessels detection problem Fig shows the results in segmenting small vessels from the retinal 48 N N Phuc, …, H H Huy, “Improved Dijkstra’s algorithm … segmentation problem.” Research images Each pixel finds the nearest seed from itself, so the quality of segmentation strongly depends on the connectivity of similar regions in the input images It can be seen that the small vessel pixels are segmented properly with high speed, but as part of the trade-off, there could be some tiny areas appeared in background For some complex images with some physically unconnected pixels or noises, the segmentation results are drastically influenced Fortunately, we can change the seeds setting (see Fig 3) or applied some preprocessing technics to improve the quality of segmentation In the Fig 4, the input image was preprocessed using noise filters from [21], and thanks to it, our framework can produce satisfactory segmentation results (a) (b) (c) (d) Figure The segmentation results for vessels detection problem (a) is the original image with seeds given The corresponding segmentation result is in (b) With one more seed added in (c), the vessels’ background can be properly detected (b) (a) (b) Figure Fast 3-labels Iris Segmentation result (b) The original image (a) is from the present CASIA V1 iris database CONCLUSION In this paper, we introduce an improved multi-source Dijkstra's algorithm for image segmentation with high performance in both time and user's convenience aspects The first improvement is about designing an efficient way for graph constructing, which can be carried out in a parallel fashion on CPUs Instead of solving the shortest distance problem multiple times, we describe this process through a multi-source variant of the Dijkstra's algorithm and replace its inserting/extracting policies by the SLF/LLL rules to reduce the overhead in its propagation Plus a comfortable seed selection for users, we show that these improvements make our algorithm performs remarkably well on a variate range of large images with multiple objects contained inside Journal of Military Science and Technology, Special Issue, No.60A, 05 - 2019 49 Electronics & Automation Although our work can out-perform other graph based algorithms running on CPUs with only a small additional memory usage, it is still challenging to detect the tiny objects in noise images In our algorithm, picking correct seeds is an important task to guarantee the optimal performance, hence it is highly desirable to pre-process the input images if they are strongly noised For the common images from modern scanning/capturing devices, our algorithm can efficiently produce satisfactory segmentation results REFERENCES [1] Y Y Boykov, M Jolly, “Interactive graph cuts for optimal boundary region segmentation of objects in n-d images”, in: Proceedings Eighth IEEE International Conference on Computer Vision ICCV 2001, volume 1, 2001, pp 105-112 vol.1 [2] I Sobel, “An isotropic 3x3 image gradient operator”, Presentation at Stanford A.I Project 1968 2014) [3] R Kohler, “A segmentation system based on thresholding”, Computer Graphics and Image Processing 15 (1981) 319-338 [4] M E Celebi, H A Kingravi, P A Vela, “A comparative study of e_cient initialization methods for the k-means clustering algorithm”, Expert Systems with Applications 40 (2013) 200-210 [5] R Alshehhi, P Marpu, W Woon, “An automatic cognitive graph-based segmentation for detection of blood vessels in retinal images”, Mathematical Problems in Engineering 2016 (2016) 1-15 [6] F Alkoot, “A review on advances in iris recognition methods”, International Journal of computer engineering research (2012) 1-9 [7] P F Felzenszwalb, D P Huttenlocher, “Effcient graph-based image segmentation”, International Journal of Computer Vision 59 (2004) 167-181 [8] D Greig, B Porteous, A Seheult, “Exact maximum a posteriori estimation for binary images”, Journal of the Royal Statistical Society, Series B 51 (1989) 271-279 [9] P F Felzenszwalb, D P Huttenlocher, “Image segmentation using local variation”, in: Proceedings 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat No.98CB36231), 1998, pp 98-104 [10] X Wei, W Lu, W Xing, “A rapid multi-source shortest path algorithm for interactive image segmentation”, Multimedia Tools and Applications 76 (2017) 21547-21563 [11] L Grady, “Random walks for image segmentation”, IEEE Trans Pattern Anal Mach Intell 28 (2006) 1768-1783 [12] C T Zahn, “Graph-theoretical methods for detecting and describing gestalt clusters”, IEEE Transactions on Computers C-20 (1971) 68-86 [13] P F Felzenszwalb, D P Huttenlocher, “Image segmentation using local variation”, in: Proceedings 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat No.98CB36231), 1998, pp 98-104 [14] X Yang, Y Su, R Duan, H Fan, S Y Yeo, C Lim, L Zhong, R S Tan, “Cardiac image segmentation by random walks with dynamic shape constraint”, IET Computer Vision 10 (2016) 79-86 [15] N Forcadel, C Le Guyader, C Gout, “Generalized fast marching method: applications to image segmentation”, Numerical Algorithms 48 (2008) 189-211 50 N N Phuc, …, H H Huy, “Improved Dijkstra’s algorithm … segmentation problem.” Research [16] R Tsai, S Osher, “Level set methods and their applications in image science”, Communications in mathematical sciences (2003) [17] E W Dijkstra, “A note on two problems in connexion with graphs”, Numer Math (1959) 269-271 [18] D P Bertsekas, “A simple and fast label correcting algorithm for shortest paths”, Networks 23 (1993) 703-709 [19] D P Bertsekas, F Guerriero, R Musmanno, “Parallel asynchronous labelcorrecting methods for shortest paths”, Journal of Optimization Theory and Applications 88 (1996) 297-320 [20] P Patidar, M Gupta, S Srivastava, A K Nagawat, “Image de-noising by various filters for different noise”, International Journal of Computer Applications (2010) TÓM TẮT CẢI TIẾN THUẬT TOÁN DIJKSTRA ĐA NGUỒN TRONG PHÂN VÙNG ẢNH Bài báo trình bày thuật tốn Dijkstra đa nguồn cải tiến nhằm giải cách nhanh chóng, hiệu tốn phân vùng ảnh có chứa nhiều đối tượng áp dụng cho mục đích tiền tính tốn nhiệm vụ xử lý ảnh Từ ảnh đầu vào, nhóm tác giả xây dựng đồ thị đóng vơ hướng từ điểm ảnh lựa chọn số điểm ảnh nguồn gán nhãn cho trước Thơng qua thuật tốn Dijkstra đồ thị, điểm nguồn gần điểm ảnh với nhãn tương ứng xác định cách hiệu xác Trong phạm vi báo, tác giả đề xuất số cải tiến bao gồm việc song song hóa q trình xây dựng đồ thị từ khoảng cách ngắn cục quanh điểm ảnh áp dụng quy tắc hiệu chỉnh nhãn cho thuật tốn Dijkstra đồ thị nhằm giảm thời gian tính tốn cần thiết Kết thử nghiệm ảnh thực tế cho thấy thuật toán đề xuất báo cải thiện đáng kể độ xác tốc độ tính tốn phân vùng ảnh Từ khóa: Khoảng cách ngắn nhất; Thuật tốn Dijkstra; Đồ thị; Tính tốn song song; Phân vùng ảnh; Hiệu chỉnh nhãn Received 13th March 2019 Revised 06th May 2019 Accepted 15th May 2019 Author affiliations: Information Technology Department, Ministry of Public Security, Vietnam; Military Technical Academy, Vietnam; Hanoi University of Science and Technology, Vietnam; Academy of Military science and technology, Vietnam * Corresponding author: phucnguyenh46@gmail.com Journal of Military Science and Technology, Special Issue, No.60A, 05 - 2019 51 ... running time THE PROPOSED METHOD 3.1 Efficient graph construction The graph construction is required only once before running the Dijkstra''s algorithm, then all information about the local connectivity... Figure The segmentation results for vessels detection problem (a) is the original image with seeds given The corresponding segmentation result is in (b) With one more seed added in (c), the vessels’... the retinal 48 N N Phuc, …, H H Huy, ? ?Improved Dijkstra’s algorithm … segmentation problem. ” Research images Each pixel finds the nearest seed from itself, so the quality of segmentation strongly