Level Fusion: Formation Association Metric with Uncertainty STEPHEN C STUBBERUD Boeing Corporation Anaheim, CA 92805 UNITED STATES OF AMERICA KATHLEEN A KRAMER Electrical Engineering Program University of San Diego 5998 Alcalá Park San Diego, CA 92110-2492 UNITED STATES OF AMERICA Abstract – Level data fusion, also referred to as Situational Assessment, defines relationships between entities In a target tracking scenario, one part of interpreting the relationships is the formation that related entities form One way to interpret formation has been to use the image processing technique of invariant moments These moments can be used to compare the actual formation of the targets to known formations In the previous implementation of this metric, the invariant moments for the tracks were computed based on the estimated location However, tracks are often estimated with both a location and uncertainty This uncertainty can have an impact in the comparison of the formations In this paper, we begin the incorporation of uncertainty into the formation metric and determine its potential for benefit in formation estimation Keywords – Invariant moments, Level fusion, Formation association, Information fusion, Uncertainty Introduction Level fusion, also known as Situational Assessment, develops and interprets the relationships between entities [5] One approach to implement an automated Level fusion system was presented in [7] Figure shows the proposed functional flow of a Level fusion system, allowing the problem to be decomposed into smaller and more easily addressable components Detect Predict Associate Hypothesis Generation Update Hypothesis Management Fig.1: A Level Fusion Architecture One of the key components that was defined in this approach was that of association Unlike Level association that compares measurements to existing tracks as discussed in [2], Level association also incorporates the interpretation of the relationships that are developed between entities To implement the Figure approach, the concept of a state representation to define the Level object was presented The proposed state was defined as a b c d where xunit y a unit xunit yunit # ofclass1 M # ofclass1 b M # ofclassn M # ofclass n In [8] and [9], association metrics were developed for each component While each of these techniques has been shown to work well as defined, the implementations were developed without incorporating the uncertainty associated with the tracks The question of how uncertainty can be incorporated into these metrics has been raised as a logical next step in the development of an automated Level fusion system In this paper, we begin this investigation Our first step in the use of uncertainty in the association metrics investigates the inclusion of uncertainty into the formation metric Section summarized the formation metric and how we can interpret uncertainty in this context This is followed in Section with the proposed approaches and associated reasoning Section summarizes the results of our tests cases while Section presents our conclusions and proposed continuation of this research The Formation Metric And Viewing Uncertainty Tracking systems are often developed using a Kalman filter [1] The state estimates of the target localizations are computed as a mean value and an associated error covariance In Figure 2, the targets are plotted over time as points that represent the mean estimates It was from this interpretation of the results, points plotted on a display that an operator can interpret, that the initial concept of image processing as an approach to interpret the formation was first considered The upper left corner shows the last set of position estimates that are of interest formation c formation change group extent1 d group extent 2 Fig 2: Tracking of Targets Over Time To make the determination, point locations for each group were compared against an image of a known formation using the following seven invariant moments: (1) 20 02 (1) (2) 20 02 112 (2) (3) 30 312 3 21 03 (4) 30 12 21 03 2 (3) (4) (5) 30 312 30 12 2 30 12 21 03 (5) 321 03 21 03 2 3 30 12 21 03 (6) 20 02 30 12 21 03 (6) 11 30 12 21 03 2 3 21 03 30 12 2 30 12 21 03 30 312 21 03 (7) 2 3 30 12 21 03 where pq x x y y p q f ( x, y ) dxdy m10 m00 m y 01 m00 x m pq (8) (9) (10) x p y q f ( x, y )dxdy (11) These moments, as described in [3] and [4], will be the same for two images if they can be considered translated, rotational, and scaled versions of each other To create a metric for this comparison, we employed a norm of the error between the two sets of the moments In [9], we showed that this approach was useful even in the presence of noisy results This is important because these so-called noisy formations emulate the real behavior of vehicles in the field Also, the estimates from tracking systems are corrupted by noise The formation metric combines the error of the seven invariant moments between the estimated formation and those of a known formation, using a norm, as shown in equation (12) (1) known (1) (2) known (2) (3) known (3) m f (4) known (4) (5) known (5) (6) known (6) (7) known (7) (12) While the point estimates have performed well in previous experiments, the concept of incorporating the uncertainty into the metric has been raised as a potential method for improvement A number of the techniques have been discussed The seemingly most promising approaches to the problem are 1) to use the uncertainty ellipse for the target estimates to derive the metrics and compare to results using the point estimates and 2) to create a convex hull from the target group and compare that to a formation’s convex hull Here, we address the first approach As with the point-to-point comparison of [9], we again look at the display of the Level fusion or track objects While the target localization is considered a point, the error covariance can be represented as an error ellipse The Kalman filter assumes that the distribution of the target estimate is Gaussian Ellipses, similar to those of Figure 3, are often drawn around the target’s estimate to represent this distribution Each ellipse typically represents two standard deviations from the mean and is also referred to as a 2-sigma ellipse The 2-sigma ellipse represents a 0.865 probability that the target shall be in that region scaled, rotated, or translated, these ellipses are considered identical This allows an ellipse, rather than a point, to serve as the image for each target By looking at a point as a scaled version of a larger ellipse, we can compare the results of the moments for each Implementation of Uncertainty in the Formation Metric In image processing, the density function f(x,y) used in equations (8) and (11) is replaced by an intensity function In [9], the intensity function was set to 1 f ( x, y ) 0 Fig 3: Associated uncertainty of position shown as a 2-sigma ellipse The ellipse is determined from the positive definite covariance matrix, C The eigenvalues of this 2x2 matrix represent the semi-major and semi-minor axes of the ellipse The rotation of the ellipse is based on the eigenvector of the larger eigenvalue Mathematically, the matrix C is defined as x x y C y x y (13) for x, y target location else (14) This made implementation simple in that integration was performed over a small set of delta functions Here, an intensity function must be defined and a numerical integration must be implemented for each ellipse Numerical integration was performed by approximating the ellipse as a set of points Figure shows an example of the technique that was used where x and y are the standard deviations of the xand y-coordinate of the estimates Fig 5: Points Defined Over Area of Ellipse Next, we selected three approaches for computing the intensity values of each point Each value is computed based on the Gaussian distribution of the target at that point Fig 4: Four ellipses that are the same according to the invariant moments In Figure 4, we see four scaled ellipses Each has the same invariant moments as the others Whether x x T 12 e y y 2 det(C ) x x C 1 y y (15) Then, the values of the points for a given ellipse are normalized to sum to Similar to method 1, but the values are weighted according to the area of the subellipse associated with that point The values are normalized to sum to Each value is based on a uniform distribution over the ellipse Summary of Results For our evaluation, we defined five primary formations: horizontal column, vertical column, a box, a wide wedge, and a narrow wedge Figures 6, 7, and show the latter three of these These three were used to evaluate the effectiveness of each of the uncertainty techniques Fig 8: Narrow Wedge Formation Test cases were generated by applying Gaussiandistributed error to the standard formations using different levels of noise Different numbers of subellipses (5, 10, 15, and 20) were also tested In most cases seven targets were in the formation, but cases of only five were also tested Each test run included 52 different noise sequences This number was selected based upon 90-90 results of order statistics [6] Measures of effectiveness included whether the correct formation was identified, the average metric observed over the 52 tests, the minimum metric observed, and the maximum One such test case included applying noise to the box formation Figure shows the randomly generated noise and associated ellipses for one of the sequences within the test run These tests were based upon using 10 sub-ellipses and a maximum noise sigma of 0.5 Fig 6: Box Formation Fig 9: Box Formation With Random Noise On Each Target Fig 7: Wide Wedge Formation Table shows the results observed in that case The four elements of each entry correspond respectively to the metric basis used – techniques 1, 2, and (defined above), and the point-based metric that does not take uncertainty into account In this case all techniques resulted in an accurate identification of formation into the metric Technique number will provide the basis of this approach We also intend on investigating the convex hull approach and to use a multiple point-to-point technique Form box References: [1] Blackman, S., Multiple-Target Tracking with Radar Applications, Artech House, Norwood, MA, 1986 [2] Blackman, S and R Popoli, Design and Analysis of Modern Tracking Systems, Artech House, Norwood, MA, 1999 [3] Hu, M., “Visual Pattern Recognition by Moment Invariants,” IRE Transactions on Information Theory, pp 179-187, February 1962 [4] Maitra, S., “Moment Invariants,” Proceedings of the IEEE, Vol 67, No 4, pp 697 - 699, April 1979 [5] Steinberg A., C Bowman, F White, “Revisions to the JDL Data Fusion Model”, Proceedings of the SPIE Sensor Fusion: Architectures, Algorithms, and Applications III, pp 430-441, 1999 [6] Stubberud, A.R., Class Notes, Engineering Probability:ECE 186, University of California, Irvine 1996 [7] Stubberud, S., P.J Shea, and D Klamer, "Data Fusion: A Conceptual Approach to Level Fusion ( Situational Awareness)," Proceedings of SPIE, Aerosense03, Orlando, FL., April, 2003 [8] Stubberud, S., P.J Shea, and D Klamer, "Metrics for Level Fusion Association," Proceedings of Fusion 2003, Cairns, Australia, July, 2003 [9] Stubberud, S and P.J Shea, "More Metrics for Level Fusion Association," Proceedings of the 16th International Conference on Systems Engineering 2003, Coventry, England, September, 2003 wide wedge narrow wedge 0.0032 0.0004 0.0058 0.0011 7.2257 7.2328 7.2221 7.2287 4.2539 4.2551 4.2496 4.2590 max 0.1035 0.1038 0.1151 0.0903 7.2822 7.2822 7.2779 7.2880 4.3160 4.3163 4.3104 4.3236 mean 0.0413 0.0365 0.0481 0.0359 7.2550 7.2570 7.2519 7.2582 4.2826 4.2850 4.2786 4.2868 Table 1: Metric Results from Box Formation Test Case Results from numerous test runs showed that technique provided the best results among the three uncertainty techniques However, the results obtained were similar to those obtained from the point-based technique Varying the number of ellipses did not improve performance significantly Conclusions In this paper, we addressed the initial use of uncertainty in the Level fusion formation metric Our initial results indicate that, while the methods are accurate, they provide little, if any, improvement over the point-to-point technique originally developed The use of the added information of uncertainty should, however, provide an improvement We plan to continue to look at the incorporation of uncertainty ... 20 03, Coventry, England, September, 20 03 wide wedge narrow wedge 0.00 32 0.0004 0.0058 0.0011 7 .22 57 7 .23 28 7 .22 21 7 .22 87 4 .25 39 4 .25 51 4 .24 96 4 .25 90 max 0.1035 0.1038 0.1151 0.0903 7 .28 22 7 .28 22. .. 0.0903 7 .28 22 7 .28 22 7 .27 79 7 .28 80 4.3160 4.3163 4.3104 4. 323 6 mean 0.0413 0.0365 0.0481 0.0359 7 .25 50 7 .25 70 7 .25 19 7 .25 82 4 .28 26 4 .28 50 4 .27 86 4 .28 68 Table 1: Metric Results from Box Formation Test... (1) 20 02 (1) (2) 20 02 1 12 (2) (3) 30 3 12 3 21 03 (4) 30 12 21 03 2 (3) (4) (5) 30 3 12 30 12 2