Multisensor fusion and integration in the wake of big data, deep learning and cyber physical system

However, in a distributedenvironment, a number of critical issues arise that are yet to be addressed and solved,including 1 the difficulty of estimating exact cross-correlations among mul

Lecture Notes in Electrical Engineering 501

Volume 501

Board of Series editors

Leopoldo Angrisani, Napoli, Italy

Marco Arteaga, Coyoacán, México

Bijaya Ketan Panigrahi, New Delhi, India

Samarjit Chakraborty, München, Germany

Jiming Chen, Hangzhou, P.R China

Shanben Chen, Shanghai, China

Tan Kay Chen, Singapore, Singapore

Rüdiger Dillmann, Karlsruhe, Germany

Haibin Duan, Beijing, China

Gianluigi Ferrari, Parma, Italy

Manuel Ferre, Madrid, Spain

Sandra Hirche, München, Germany

Faryar Jabbari, Irvine, USA

Limin Jia, Beijing, China

Janusz Kacprzyk, Warsaw, Poland

Alaa Khamis, New Cairo City, Egypt

Torsten Kroeger, Stanford, USA

Qilian Liang, Arlington, USA

Tan Cher Ming, Singapore, Singapore

Wolfgang Minker, Ulm, Germany

Pradeep Misra, Dayton, USA

Sebastian Möller, Berlin, Germany

Subhas Mukhopadyay, Palmerston North, New Zealand

Cun-Zheng Ning, Tempe, USA

Toyoaki Nishida, Kyoto, Japan

Federica Pascucci, Roma, Italy

Yong Qin, Beijing, China

Gan Woon Seng, Singapore, Singapore

Germano Veiga, Porto, Portugal

Haitao Wu, Beijing, China

Junjie James Zhang, Charlotte, USA

SCOPUS, MetaPress, Springerlink **

Lecture Notes in Electrical Engineering (LNEE) is a book series which reports the latest research and developments in Electrical Engineering, namely:

• Communication, Networks, and Information Theory

• Computer Engineering

• Signal, Image, Speech and Information Processing

• Circuits and Systems

• Bioengineering

• Engineering

The audience for the books in LNEE consists of advanced level students, researchers, and industry professionals working at the forefront of their fields Much like Springer’s other Lecture Notes series, LNEE will be distributed through Springer ’s print and electronic publishing channels For general information about this series, comments or suggestions, please use the contact address under “service for this series”.

To submit a proposal or request further information, please contact the appropriate Springer Publishing Editors:

Asia:

China, Jessie Guo, Assistant Editor (jessie.guo@springer.com) (Engineering)

India, Swati Meherishi, Senior Editor (swati.meherishi@springer.com) (Engineering)

Japan, Takeyuki Yonezawa, Editorial Director (takeyuki.yonezawa@springer.com)

(Physical Sciences & Engineering)

South Korea, Smith (Ahram) Chae, Associate Editor (smith.chae@springer.com)

(Physical Sciences & Engineering)

Southeast Asia, Ramesh Premnath, Editor (ramesh.premnath@springer.com)

(Electrical Engineering)

South Asia, Aninda Bose, Editor (aninda.bose@springer.com) (Electrical Engineering)

Europe:

Leontina Di Cecco, Editor (Leontina.dicecco@springer.com)

(Applied Sciences and Engineering; Bio-Inspired Robotics, Medical Robotics, Bioengineering; Computational Methods & Models in Science, Medicine and Technology; Soft Computing; Philosophy of Modern Science and Technologies; Mechanical Engineering; Ocean and Naval Engineering; Water Management & Technology)

(christoph.baumann@springer.com)

(Heat and Mass Transfer, Signal Processing and Telecommunications, and Solid and Fluid Mechanics, and Engineering Materials)

North America:

Michael Luby, Editor (michael.luby@springer.com) (Mechanics; Materials)

More information about this series at http://www.springer.com/series/7818

Songhwai Oh

Editors

Multisensor Fusion

and Integration in the Wake

of Big Data, Deep Learning and Cyber Physical System

An Edition of the Selected Papers

from the 2017 IEEE International Conference

on Multisensor Fusion and Integration

for Intelligent Systems (MFI 2017)

123

Seoul National UniversitySeoul

Korea (Republic of)

ISSN 1876-1100 ISSN 1876-1119 (electronic)

Lecture Notes in Electrical Engineering

ISBN 978-3-319-90508-2 ISBN 978-3-319-90509-9 (eBook)

https://doi.org/10.1007/978-3-319-90509-9

Library of Congress Control Number: 2018940915

© Springer International Publishing AG, part of Springer Nature 2018

This work is subject to copyright All rights are reserved by the Publisher, whether the whole or part

of the material is concerned, speci fically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission

or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed.

The use of general descriptive names, registered names, trademarks, service marks, etc in this publication does not imply, even in the absence of a speci fic statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use.

The publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication Neither the publisher nor the authors or the editors give a warranty, express or implied, with respect to the material contained herein or for any errors or omissions that may have been made The publisher remains neutral with regard to jurisdictional claims in published maps and institutional af filiations.

Printed on acid-free paper

This Springer imprint is published by the registered company Springer International Publishing AG part of Springer Nature

The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland

Multisensor fusion and integration is playing a critical role in harnessing the smarttechnologies as we ride the big wave of the 4th Industrial Revolution Deployment

of the Internet of Things, Cyber-Physical Systems and Robotics in distributedenvironment is rapidly rising as our society seeks to transition from being ambient

to being smart and, at the same time, to enable human to curate information andknowledge between ubiquitous and collective computing environments Whatsurround us are the networks of sensors and actuators that monitor our environment,health, security and safety, as well as the service robots, intelligent vehicles andautonomous systems of ever heightened autonomy and dependability with inte-grated heterogeneous sensors and actuators Developing fundamental theories andadvancing implementation tools to address the emerging key issues in multisensorfusion and integration in the wake of big data and deep learning would make theabove transition smooth and rewarding

This volume is an edition of the papers selected from the 13th IEEE InternationalConference on Multisensor Integration and Fusion, IEEE MFI 2017, held in Daegu,Korea, 16–22 November 2017 Only 17 papers out of the 112 papers accepted forIEEE MFI 2017 were chosen and requested for revision and extension to beincluded in this volume The 17 contributions to this volume are organized into twochapters: Chapter 1 is dedicated to the theories in data and information fusion indistributed environment and Chapter 2 to the multisensor fusion in robotics To helpreaders understand better, a chapter summary is included in each chapter as anintroduction

It is the wish of the editors that readers find this volume informative andenjoyable We would also like to thank Springer-Verlag for undertaking the pub-lication of this volume

Sukhan LeeHanseok KoSonghwai Oh

v

Multi-sensor Fusion: Theory and Practice

Covariance Projection as a General Framework of Data Fusion

and Outlier Removal 5Sukhan Lee and Muhammad Abu Bakr

State Estimation in Networked Control Systems with Delayed

and Lossy Acknowledgments 22Florian Rosenthal, Benjamin Noack, and Uwe D Hanebeck

Performance of State Estimation and Fusion with Elliptical

Motion Constraints 39Qiang Liu and Nageswara S V Rao

Relevance and Redundancy as Selection Techniques

for Human-Autonomy Sensor Fusion 52Justin D Brody, Anna M R Dixon, Daniel Donavanik,

Ryan M Robinson, and William D Nothwang

Classification of Reactor Facility Operational State Using SPRT

Methods with Radiation Sensor Networks 76Camila Ramirez and Nageswara S V Rao

Improving Ego-Lane Detection by Incorporating Source Reliability 98Tran Tuan Nguyen, Jens Spehr, Jonas Sitzmann, Marcus Baum,

Sebastian Zug, and Rudolf Kruse

Applying Knowledge-Based Reasoning for Information Fusion

in Intelligence, Surveillance, and Reconnaissance 119Achim Kuwertz, Dirk Mühlenberg, Jennifer Sander, and Wilmuth Müller

Multiple Classifier Fusion Based on Testing Sample Pairs 140Gaochao Feng, Deqiang Han, Yi Yang, and Jiankun Ding

vii

Multi-sensor Fusion Applications in Robotics

Bayesian Estimator Based Target Localization in Ship Monitoring

System Using Multiple Compact High Frequency

Surface Wave Radars 157Sangwook Park, Chul Jin Cho, Younglo Lee, Andrew Da Costa,

SangHo Lee, and Hanseok Ko

SLAM-Based Return to Take-Off Point for UAS 168Daniel Bender, Wolfgang Koch, and Daniel Cremers

Underwater Terrain Navigation During Realistic Scenarios 186

Mårten Lager, Elin A Topp, and Jacek Malec

Supervised Calibration Method for Improving Contrast

and Intensity of LIDAR Laser Beams 210Mohammad Aldibaja, Noaki Suganuma, Keisuke Yoneda, Ryo Yanase,

and Akisue Kuramoto

Multi-object Tracking Based on a Multi-layer Particle Filter

for Unclustered Spatially Extended Measurements 219Johannes Buyer, Martin Vollert, Mihai Kocsis, Nico Sußmann,

and Raoul Zöllner

Ensemble Kalman Filter Variants for Multi-Object Tracking

with False and Missing Measurements 239Fabian Sigges and Marcus Baum

Fall Detection with Unobtrusive Infrared Array Sensors 253Xiuyi Fan, Huiguo Zhang, Cyril Leung, and Zhiqi Shen

Subtle Hand Action Recognition in Factory Based

on Inertial Sensors 268Yanyan Bao, Fuchun Sun, Xinfeng Hua, Bin Wang, and Jianqin Yin

Kinematics, Dynamics and Control of an Upper Limb

Rehabilitation Exoskeleton 284Qingcong Wu and Ziyan Shao

Author Index 299

Multi-sensor Fusion: Theory and Practice

Sukhan Lee and Hanseok Ko

Multisensor fusion and integration in a distributed environment is becoming of utmostimportance, especially, in the wake of the growing deployment of Internet of Things(IoT) as well as Cyber Physical Systems (CPS) Although the fundamental theorybehind multisensor fusion and integration has been well-established through severaldecades of investigations, in practice, there still remain a number of technical chal-lenges to overcome, in particular, for dealing with multisensor fusion and integration in

a distributed environment Specifically speaking, multisensor fusion with the knowncross-correlations among multiple data sources can be handled ideally, for instance, byBar-Shalom Campo and Generalized Millman’s formula However, in a distributedenvironment, a number of critical issues arise that are yet to be addressed and solved,including (1) the difficulty of estimating exact cross-correlations among multiple datasources due to the physical relationships possibly existing among their observations aswell as the possible double counting by sharing prior information or data sources,(2) the presence of inconsistency or outliers among data sources, (3) the existence oftransmission delays as well as data losses and (4) the incorporation of various con-straints that may be available among states and observations into fusion The paperscollected for this chapter are to address some of the critical issues as described above in

a theoretical and/or a practical point of view, as follows:

The paper, entitled“Covariance Projection as General Framework of Data Fusionand Outlier Removal,” by Sukhan Lee and Muhammad Abu Bakr proposes a generalframework of distributed data fusion for distributed sensor networks of arbitraryredundancies, where inconsistent data are identified simultaneously within the frame-work The paper, entitled “State Estimation in Networked Control Systems withDelayed and Lossy Acknowledgments,” by Florian Rosenthal, Benjamin Noack andUwe D Hanebeck deals with the state estimation in networked control systems wherethe control inputs and measurements transmitted via networks as well as theacknowledgements packets sent by the actuator upon reception of control inputs aresubject to data losses and random transmission delays The paper, entitled “Perfor-mance of State Estimation and Fusion with Elliptical Motion Constraints,” byQiang Liu and Nageswara Rao investigates target tracking in the presence of ellipticalnonlinear constraints on its motion dynamics, where the state estimates generated bysensors are considered to be sent over long-haul lossy links to a remote fusion center.The paper, entitled “Relevance and Redundancy as Selection Techniques forHuman-Autonomy Sensor Fusion,” by Justin David Brody, Anna Marie Rogers Dixon,Daniel Donavanik, Ryan M Robinson and William D Nothwang addresses theproblem of sensor fusion in a human-autonomy system where the dynamic nature ofsensors makes it difficult to model their variability The paper examines the application

of information theoretic entities, such as the relevance between sensors and targetclasses and the redundancy among the selected sensors, as the criteria for evaluating theimportance for fusion The paper, entitled “Classification of Reactor Facility

Operational State Using SPRT Methods with Radiation Sensor Networks,” byNageswara Rao and Camila Ramirez deals with the problem of inferring the opera-tional status of a reactor facility using measurements from a radiation sensor networkwhere sensor measurements are inherently random with the parameters determined bythe intensity at the sensor locations The paper, entitled“Applying Knowledge-BasedReasoning for Information Fusion in Intelligence, Surveillance, and Reconnaissance,”

by Wilmuth Muller, Achim Kuwertz, Dirk Muhlenberg and Jennifer Sander presents amethod of high-level data fusion combining probabilistic information processing withlogical and probabilistic reasoning This is to support human operators in their situa-tional awareness for improving their capabilities of making efficient and effectivedecisions The paper, entitled “Multiple Classifier Fusion Based on Testing SamplePairs,” by Gaochao Feng, Deqiang Han, Yi Yang, and Jiankun Ding presents a multipleclassifier system operated under the classification based on testing sample pairs, wherefuzzy evidential reasoning is used to implement multiclass classification fusion Thepaper, entitled“Improving Ego-Lane Detection by Incorporating Source Reliability,”

by Tran Tuan Nguyen, Jens Spehr, Jian Xiong, Marcus Baum, Sebastian Zug andRudolf Kruse proposes an efficient and sensor-independent metric which provides anobjective and intuitive self-assessment for the entire road estimation process at multiplelevels, including individual detectors, lane estimation and the target applications

Framework of Data Fusion and Outlier Removal

Sukhan Lee(&)and Muhammad Abu BakrIntelligent Systems Research Institute, Sungkyunkwan University,

Gyeonggi-do, Suwon 440-746, South Korea{lsh1,abubakr}@skku.edu

Abstract A fundamental issue in sensor fusion is to detect and remove outliers

as sensors often produce inconsistent measurements that are difficult to predictand model The detection and removal of spurious data is paramount to thequality of sensor fusion by avoiding their inclusion in the fusion pool In thispaper, a general framework of data fusion is presented for distributed sensornetworks of arbitrary redundancies, where inconsistent data are identifiedsimultaneously within the framework By the general framework, we mean that it

is able to fuse multiple correlated data sources and incorporate linear constraintsdirectly, while detecting and removing outliers without any prior information.The proposed method, referred to here as Covariance Projection (CP) Method,aggregates all the state vectors into a single vector in an extended space Themethod then projects the mean and covariance of the aggregated state vectorsonto the constraint manifold representing the constraints among state vectors thatmust be satisfied, including the equality constraint Based on the distance fromthe manifold, the proposed method identifies the relative disparity among datasources and assigns confidence measures The method provides an unbiased andoptimal solution in the sense of Minimum Mean Square Error (MMSE) fordistributed fusion architectures and is able to deal with correlations and uncer-tainties among local estimates and/or sensor observations across time Simulationresults are provided to show the effectiveness of the proposed method in iden-tification and removal of inconsistency in distributed sensors system

Keywords: Covariance projection method
Constraint manifold

Data fusion
Distributed sensor network
Inconsistent data

1 Introduction

Multisensor data fusion is to obtain a more meaningful and precise estimate of a state

by combining data from multiple sources One of the inherent issues in multisensordata fusion is that of uncertainty in sensor measurements The sensor uncertainties maycome from impreciseness and noise in the measurements, as well as, from ambiguitiesand inconsistencies present in the environment The fusion methodologies should beable to model such uncertainties and combine data to provide a consistent and accuratefused solution

© Springer International Publishing AG, part of Springer Nature 2018

S Lee et al (Eds.): MFI 2017, LNEE 501, pp 5 –21, 2018.

https://doi.org/10.1007/978-3-319-90509-9_1

Recently, distributed data fusion [1, 2] is widely explored in diverse fields ofengineering and control due to its superior performance over the centralized fusion interms offlexibility, robustness to failure and cost-effectiveness in infrastructure andcommunication However, the distributed architecture needs to address statisticaldependency among the local estimates received from multiple nodes for fusion This isdue to the fact that local state estimates at individual nodes can be subject to sameprocess noise [3] and to double counting, i.e., sharing same data sources among them[4] Ignoring such statistical dependency or cross-correlation among multiple nodesleads to inconsistent results, causing divergence in data fusion [5].

The fusion methodologies assume that the sensor measurements are affected byGaussian noise only and thus the covariance of the estimate provides a goodapproximation of all the disturbances affecting the sensor measurements However, inreal applications, the sensor measurements may not only be affected by noise but alsofrom unexpected situations such as short duration spike faults, sensor glitch, permanentfailure or slowly developing failure due to sensor elements [6] Since these types ofuncertainties are not attributable to the inherent noise, they are difficult to model Due

to these uncertainties, the estimates provided by sensor nodes in a distributed networkmay be spurious and inconsistent Fusing these imprecise estimates with correct esti-mates can lead to severely inaccurate results [7] Hence, a data validation scheme isrequired to identify and eliminate the outliers from the fusion pool

Detection of inconsistency needs either a priori information often in the form ofspecific failure model(s) or data redundancy [1] The model-based approaches [1,8]uses the generated residuals between the model outputs and actual measurements todetect and remove faults For instance in [9], Nadaraya-Watson estimator and a prioriobservations are used to validate sensor measurements Similarly, a priori system modelinformation as a reference is used to detect failures infiltered estimates [10] Researchershave also used fuzzy logic [11] and neural network [12] based approaches for sensorvalidation However, model-based methods either require an explicit mathematicalmodel or need tuning and training for data validation This restricts the usage of thesemethods in the case where prior information is not available or unmodeled failureoccurs A method to detect spurious data based on Bayesian framework is proposed in[13] The method adds a term to the Bayesian formulation which has the effect ofincreasing the posterior distribution when measurement from one of the sensor isinconsistent with respect to the other However, the method assumes independence ofthe sensor estimates in its analysis and may lead to incorrect rejection of true estimates orincorrect retaining of false estimates

This paper presents a general data fusion framework, referred to as CovarianceProjection (CP) Method, tofind an optimal and consistent fusion solution for multiplecorrelated data sources The proposed method provides a framework for identifying andremoving outliers in a distributed sensor network where only the sensor estimates may

be available at the fusion center

1.1 Problem Statement

In a distributed architecture [1, 2], the sensors are often equipped with a trackingsystem to provide local estimates of some quantity of interest in the form of mean and

covariance Assume that each local system predicts the underlying states using lowing equation,

fol-xk ¼ Axk 1 þ Buk 1 þ wk 1where A is the system matrix, B input matrix, uk 1 input vector and^xk 1 is the statevector The system process is affected by zero mean Gaussian noise wk 1 withcovariance matrix Q The sensor measurements are approximated as,

zki ¼ Hixk þ vki þ eki; i ¼ 1; ; nwhere vki is Gaussian noise with covariance matrices Ri; i ¼ 1; 2; ; n The sensormeasurements are also affected by unmodeled faults eki The state prediction of eachlocal system is updated by its own sensor measurement to compute local state estimates

as (^xk; Pk) The local estimates are then communicated among sensor nodes or sent to acentral node for obtaining a global estimate However, the local estimates may becorrelated due to common process noise [3] or double counting [4] Furthermore, theestimates provided by local systems may be spurious and inconsistent due to theunmodeled sensor faults As stated in the introduction, the majority of work needs apriori information in the form of particular failure model(s) to detect sensor faults[9, 10] While in a distributed architecture, the fusion node may have access to theestimated mean and covariance of the data sources only Moreover, the cross-correlation among data sources is overlooked in traditional data validation schemes andoutliers removal is mostly based on heuristics [13]

This paper presents a general framework to validate and fuse correlated anduncertain data from multiple sources without any prior information The proposedmethod assigns confidence measure to multiple data sources based on the distance fromthe constraint manifold The method then statistically removes the inconsistent sensorestimates of arbitrary dimensions and correlations

2 Proposed Approach

Consider unbiased estimates^x1and^x2, of the true state x, with covariances P1, P2andcross-covariance matrix P12 The statistical distribution, that is, the mean and covari-ance from individual sensors in RN is aggregated such that it is transformed to anextended space ofR2Nalong with the equality constraint between the two data sources,that is,

Whitening Transform (W) is a linear

transformation that can be defined as, W ¼ D1=2ET, where D and E is the respectiveeigenvalue and eigenvector matrix of P Applying Whitening transform, we get,

^xW ¼ W^x; PW ¼ WPWT ¼ I; MW ¼ WMFigure1(b) shows the transformation of the ellipsoid into a unit circle after W Themean and covariance are then projected on the constraint manifold MW to get a fusedresult in the transformed space as shown in Fig.1(b) Inverse Whitening Transform isapplied to obtain the optimal fused mean and covariance in the original space as,

in (2) and (3), we get the closed-form simplification of fused mean and covariance for

The details of the simplification are provided in Appendix1 Using the values of

M; ^x; and P from (1) in (4) and (5), we get the CP fused mean and covariance of twosensor estimates as,

~x ¼ Pð 2  P21Þ Pð 1 þ P2  P12  P21Þ1^x1

þ Pð 1  P21Þ Pð 1 þ P2  P12  P21Þ1^x2

ð6Þ

~P ¼ P1  Pð 1  P12Þ Pð 1 þ P2  P12  P21Þ1ðP1  P21Þ ð7ÞGiven n sensor estimatesð^x1; P1Þ, ð^x2; P2Þ; ; ð^xn; PnÞ of a true state x 2 RN withknown cross-covariance Pij; i; j ¼ 1; ; n, (4) and (5) can be used to provide theoptimal fused mean and covariance with M ¼ I½N1; IN2; ; INnT

Where IN is theidentity matrix and N the dimension of individual data source The proposed CPmethod provides an unbiased and optimal fused solution in the sense of MinimumMean Square Error (MMSE) for a multisensor system of arbitrary redundancies.Theorem 1: The fused estimate ~x given by the CP method in Eq (2) is an unbiasedestimator of x, that is, Eð Þ ¼ E x~x ð Þ:

Proof: Using (2) we can write

x  ~x ¼ W1P

rW xð  ^xÞTaking expectation on both sides, we get

Theorem 2: The fused covariance ~P of the CP method is smaller than the individualcovariances, that is, ~P  Pi; i ¼ 1; 2; ; n:

Proof: From Eq (5), we can write

where M is the constraint among data sources and Mi ¼ I½Ni; 0; ; 0T

is the straint matrix for Pi The equality holds for Pi ¼ Pij; that is, ~P ¼ Pi; when Pi ¼ Pij;

con-j ¼ 1; 2; ; n

Since the estimates of the state provided by sensors in a distributed architecture arecorrelated, computation of cross-covariance Pij; is needed to compute the fused mean(4) and covariance (5) The cross-covariance between the sensor estimates can becomputed as [15],

ij represent the cross covariance of the previous cycle between sensor i and j

3 Con fidence Measure of Data Sources

The working of fusion algorithms is based on assumption that the input sensor mates are consistent and consequently fails in the case of inconsistent estimates Hence,

esti-a desti-atesti-a vesti-alidesti-ation scheme is required to identify esti-and eliminesti-ate the outliers before fusion.The proposed approach identifies relative disparity and confidence measure of themulti-sensory data by utilizing the relationship among data sources Assuming that thedata sources can be represented jointly as a multivariate normal distribution, theconfidence of data sources can be measured by calculating the distance from theconstraint manifold as depicted in Fig.2 Suppose that we have n Gaussian datasources inRN with corresponding joint mean and covariance matrices as,

^x ¼

^xN1

^xN2

^xNn

264

37

P1 P12 P1n

PT 12

PT 1n

P2

Pn

266

377

Then the distance d from the manifold representing confidence measure can be puted as,

The point on the manifold is given as,

~x ¼ P2ðP1 þ P2Þ1^x1 þ P1ðP1 þ P2Þ1^x2Simplifying we get,

Theorem 3: For N dimension of n data sources, the d distance (9) follow a chi-squareddistribution with nN degrees of freedom (DOF), that is, d v2ð Þ.Nn

Proof: From (9) we can write

where y¼ W ^x  ~xð Þ  N 0; 1ð Þ is an independent standard normal distribution For Ndimensions of state vector, the right-hand side of (12) is PN

i ¼1y2i, thus distance dfollows a chi-square distribution with N DOF, that is, d v2ð Þ For n data sourcesNwith N states,

d v2ð ÞnNSince d is a chi-square distribution with nN DOF, then for any significance level

a 2 0; 1ð Þ, v2

að Þ is defined such that the probability,Nn

P d v2

að ÞNn ¼ aHence, to have a confidence of 100 1  að Þ percent, d should be less than respectivecritical value A value of a ¼ 0:05 is assumed in this paper unless specified.Chi-square table [16] can be used to obtain the critical value for the confidence distancewith a particular significance level and DOF

3.1 Inconsistency Detection and Exclusion

To obtain reliable and consistent fusion results, it is important that the inconsistentestimates in a multisensor distributed system be identified and excluded before fusion.For this reason, at each time step when the fusion center receives computed estimatesfrom sensor nodes, distance d is calculated A computed distance d less than the criticalvalue mean that we are confident about the closeness of sensor estimates and that theycan be fused together to provide better estimate of the underlying states On the otherhand, a distance d greater than or equal to the critical value indicate spuriousness of thesensor estimates At least one of the sensor estimate is significantly different than theother sensor estimates To exclude the outliers, a distance from the manifold is com-puted for every estimate and compared with the respective critical values For n sensorestimates the hypothesis and decision rule are summarized as follow,

Hypotheses: H0: ^x1¼ ^x2 ¼


¼ ^xn

H1: ^x16¼ ^x2 6¼


6¼ ^xnDecision Rule: Accept H0if d \ v2

að ÞNnReject H0 if d  v2

að ÞNn

If the hypothesis H0 is accepted then the estimates are optimally fused using (4) and(5) On the other hand, rejection of null hypothesis means that at least one of the sensorestimate is significantly different than the other sensor estimates The next step is toidentify the inconsistent sensor estimates A distance from the manifold is computed foreach of the estimates as,

di ¼ ^xð i ~xÞT

P1

i ð^xi ~xÞ; i¼ 1; 2; ; n

The outliers are identified and eliminated based on the respective critical value, that is,

of correlation coefficients The y-axis shows the percentage of rejection of the nullhypothesis H0 Figure3(b) shows the distance d with changing correlation coefficientfrom−1 to 1 It can be noted that ignoring the cross-correlation in distance d result inunderestimated or overestimated confidence and may lead to incorrect rejection of truenull hypothesis (Type I error) or incorrect retaining of false null hypothesis (Type IIerror) The proposed framework inherently takes care of any cross-correlation amongmultiple data sources in the computation of distance d

Fig 3 Effect of correlation on d distance (a) Percentage of rejecting the null hypothesis H0withdifferent correlation values (b) d distance with correlationq 2 1; 1½ :

Example: Consider a numerical simulation with the constant state,

xk ¼ 10Three sensors are used to estimate the state xk, where the measurements of thesensors are corrupted with respective variance of R1; R2 and R3 The values for theparameters assumed in the simulation are,

Q ¼ 2; R1 ¼ 0:5; R2 ¼ 1; R1 ¼ 0:9The sensors measurements are assumed to be cross-correlated It is also assumed thatthe sensor 1, sensor 2 and sensor 3 measurements are independently affected byunmodeled random noise and produce inconsistent data for 33, 33 and 34% of the timerespectively The sensors compute local estimates of the state and send it to the fusioncenter Three strategies for fusing the local sensor estimates are compared: (1) CP,which fuses the three sensor estimates using (4) and (5) without removing outliers,(2) CP WO-d means the outliers were identified and rejected based on (13) with

r2

12¼ 0 before fusion, that is, correlation in computation of d is ignored and, (3) CPWO-dC, reject the outliers based on (13) with taking into account the cross-correlation.Figure4 shows the fused solution of three sensors when the estimate provided bysensor 2 is in disagreement with sensor 1 and 3 It can be observed from Fig.4 thatneglecting the cross-correlation in CP WO-d result in Type II error, that is, all the threeestimates are fused despite the fact that estimate 2 is inconsistent CP WO-dC correctlyidentifies and eliminates the spurious estimate before the fusion process Figure 5

Fig 4 Three sensors fusion when the estimate of sensor 2 is inconsistent Neglecting thecross-correlation results in Type II error

shows the estimated state after fusion of three sensors estimates for 100 samples It can

be seen that the presence of outliers greatly affects the outcome of multisensor datafusion As depicted in Fig.5, eliminating outliers before fusion can improve the esti-mation performance The fused samples of CP WO-d and CP WO-dC on average liescloser to the actual state Figure5also shows the fusion performance when outliers areidentified with and without cross-correlation It can be noted that inconsideration ofcorrelation affects the estimation quality because of Type I and Type II error

4 Simulation Results

In this section, simulation results are provided to demonstrate the effectiveness of theproposed method for fusion of spurious data The performance is assessed by root meansquare error (RMSE) over the simulation time computed as,

where L is the length of simulation and V is the Monte-Carlo runs

Consider a target tracking scenario characterized by the following dynamic systemmodel,

Fig 5 Estimated state after three sensor fusion in presence of inconsistent estimates

with the state vector xk 1 ¼ s v½ T

, where s and v are the position and velocity of thetarget at time t respectively T is the sampling period and assumed as 3 s The systemprocess is affected by zero mean Gaussian noise wk 1with covariance matrix Q Threesensors are employed to track the movement of the target, where the sensor mea-surements are approximated by the following equation,

zki ¼ 10 01

xk þ vki þ eki; i ¼ 1; 2; 3 ð15Þ

The measurements of the sensors are corrupted by noise vkiwith respective covariance

of Ri; i ¼ 1; ; 3 The covariance of the process noise assumed is Q ¼ 10 andsensor measurement noises are,

R1 ¼ diag 50; 30ð Þ; R2 ¼ diag 70; 20ð Þ; R3 ¼ diag 10; 60ð Þ

The control input uk 1 ¼ 1 if v \ 30 otherwise it is changed to −1 until v \ 5: It isassumed that the sensor 1, sensor 2 and sensor 3 measurements are independentlyaffected by unmodeled random noise eki for 33, 33 and 34% of the time respectivelyand thus the estimates provided by sensors are sometimes spurious

Starting from an initial value, in each time step the individual sensor uses local stateprediction, that is, (14) to predict the state of the target and then update the stateprediction by its own sensor measurements obtained through (15) The local estimatesare assumed to be correlated and (8) is used to calculate the track-to-trackcross-correlation The estimated states and covariances by each sensor are sent to thefusion center, where they are fused by CP Method, which takes care of thecross-correlation among the estimates The three fusion strategies of CP (fusion withoutoutlier removal), CP WO-d (outlier removal without considering cross-correlation) and

CP WO-dC (taking care of correlation in outlier removal) are compared based onRMSE between the actual state value and fused estimate of the state for 1000 MonteCarlo runs In the simulation setup, the inconsistency is detected with significance level

a = 0.05 Figure6(a) and (b) illustrate the RMSE of the target position and velocityrespectively versus time Table1summarizes the average RMSE for 1000 Monte Carloruns

Figure6and Table1shows the efficacy of the proposed method in identifying andremoving outliers It can be observed that the presence of outliers deteriorates the fusionperformance of multisensor data fusion Eliminating the outliers before fusion greatlyimprove the estimation quality Figure6and Table1also shows the fusion performancewhen outliers are identified with and without consideration of cross-correlation in dis-tance d It can be noted that inconsideration of correlation affects the estimation qualitybecause of Type I and Type II error

Fig 6 Illustration of distributed multisensor data fusion in presence of inconsistent estimates.(a) Position RMSE (b) Velocity RMSE.

5 Conclusion

Sensors often produce inconsistent and spurious data Detection and removal of suchinconsistencies before fusion is essential for accurate state estimation In this paper, wepropose a general approach to the fusion of correlated and uncertain data sources Theproposed method provides an unbiased and optimal fusion rule for arbitrary sensors in

a distributed sensor architecture The method automatically detects and removeinconsistent estimates from multiple data sources by assigning statistical confidencemeasure Simulation results verified the effectiveness of the proposed method in theidentification of spuriousness in distributed sensor data It was shown that the proposedmethod improves the estimation quality by effectively identifying and removing theincorrect sensor data It was also observed that consideration of cross-correlation by theproposed method in the detection of outliers result in lower RMSE due to avoidance ofType I and II errors

Acknowledgments The original idea of the proposed approach is due to Sukhan Lee Thisresearch was supported, in part, by the“Space Initiative Program” of National Research Foun-dation (NRF) of Korea (NRF-2013M1A3A3A02042335), sponsored by the Korean Ministry ofScience, ICT and Planning (MSIP), and in part, by the“3D Visual Recognition Project” of KoreaEvaluation Institute of Industrial Technology (KEIT) (2015-10060160), and in part, by the

“Robot Industry Fusion Core Technology Development Project” of KEIT (R0004590)

The ~P in (A5) is the projection of the ellipsoid on the equality constraint Projecting it

on the subspace of individual data source will result in fused covariance as,

~P ¼ M TP1M1

ðA6ÞSimilarly, using definitions of various components in fused mean (A1), we have,

^x2 ~x ¼ I  Ph 1ðP1þ P2Þ1i^x2 Ph 2ðP1þ P2Þ1i^x1

^x2 ~x ¼ P2ðP1þ P2Þ1½^x1 ^x2 ðB3ÞPutting (B2) and (B3) in (B1) and simplifying, we get,

12 Abbaspour, A., Aboutalebi, P., Yen, K.K., Sargolzaei, A.: Neural adaptive observer-basedsensor and actuator fault detection in nonlinear systems: application in UAV ISA Trans 67,

317–329 (2017)

13 Kumar, M., Garg, D., Zachery, R.: A method for judicious fusion of inconsistent multiplesensor data IEEE Sens J 7, 723–733 (2007)

14 Lee, S., Bakr, M.A.: An optimal data fusion for distributed multisensor systems In:Proceedings of the 11th International Conference on Ubiquitous Information Managementand Communication - IMCOM 2017, pp 1–6 ACM Press, New York (2017)

15 Shin, V., Lee, Y., Choi, T.: Generalized Millman’s formula and its application for estimationproblems Signal Process 86, 257–266 (2006)

16 Walpole, R., Myers, R., Myers, S., Ye, K.: Probability and Statistics for Engineers andScientists Prentice Hall, Upper Saddle River (1993)

Systems with Delayed and Lossy

Acknowledgments

Florian Rosenthal(B), Benjamin Noack, and Uwe D Hanebeck

Intelligent Sensor-Actuator-Systems Laboratory (ISAS),

Institute for Anthropomatics and Robotics,Karlsruhe Institute of Technology (KIT), Karlsruhe, Germany

{florian.rosenthal,noack}@kit.edu, uwe.hanebeck@ieee.org

Abstract In this article, we are concerned with state estimation in

Net-worked Control Systems where both control inputs and measurementsare transmitted over networks which are lossy and introduce randomtransmission delays We focus on the case where acknowledgment pack-ets transmitted by the actuator upon reception of applicable controlinputs are also subject to delays and losses, as opposed to the com-mon notion of TCP-like communication where successful transmissionsare acknowledged instantaneously and without losses As a consequence,the state estimator in the considered setup has only partial and belatedknowledge concerning the actually applied control inputs which results inadditional uncertainty We derive an estimator by extending an existingapproach for the special case of UDP-like communication which main-tains estimates of the applied control inputs that are incorporated intothe estimation of the plant state The presented estimator is compared

to the original approach in terms of Monte Carlo simulations where itsincreased robustness towards imperfect knowledge of the underlying net-works is indicated

Keywords: State estimation·Networked control systems·Delays

Packet losses·Markov jump linear systems·IMM filter

1 Introduction

Networked Control Systems (NCSs), such as the one sketched in Fig.1, are a cial class of control loops where the individual components, i.e., plant/actuator,sensor, and controller, communicate over packet-based and typically general-purpose networks such as WiFi or Ethernet In comparison to traditional con-trol loops, where dedicated point-to-point connections are utilized, such sys-tems profit from reduced expenditure for installation and maintenance, and fromenhanced flexibility and reliability [1] On the downside, they have to cope witheffects and constraints induced by the networks and constraints like randomc

spe- Springer International Publishing AG, part of Springer Nature 2018

S Lee et al (Eds.): MFI 2017, LNEE 501, pp 22–38, 2018.

packet losses, delays and limited bandwidth Since these factors affect the all system performance and stability, communication and control should not beaddressed independently from each other [2 4] Consequently, several controlmethods have been proposed in recent years that explicitly consider the under-

over-lying network Among these, the approach of sequence-based control has gained

much attention [5 9] Here the idea is to compute control inputs for the next,sayN, time steps in addition to the one for the current time instant By trans- mitting this control sequence in a single data packet which is buffered at the

actuator upon reception, the problem of delayed or missing control inputs can

be alleviated Such controllers, often called predictive controllers, are usuallybased on model predictive control approaches [6,7], or adapted from nominalcontrollers which disregard the network [10,11] Also, controllers which directlyminimize a quadratic cost function with respect to the control sequence havebeen proposed [8,9] Since most of the derived control algorithms either explic-itly demand a state estimate or assume a perfectly known state or noise-freeplants and measurements, state estimation is generally required in an NCS.This article is an extended version of our previous paper [12], where we devel-oped an estimator based on the minimum mean squared error (MMSE) criterionfor a given predictive controller in an NCS scenario Due to the presence of thenetworks, the estimator is confronted with the problem that measurements andcontrol inputs can arrive delayed or get lost so that out-of-sequence and burstarrivals are probable In particular, the resulting uncertainty about the actualapplied control inputs poses a major challenge

Network

Controller

PlantSensor Actuator

Fig 1 Schematical overview of a Networked Control System.

Estimators being able to cope with delayed or absent measurements havebeen proposed, for instance, in [13–15], whereas the problem of estimation sub-ject to missing control inputs has been investigated in [16,17] Moayedi et al [18]presented a filter for Networked Control Systems which can handle both delayedand missing control inputs and measurements Yet, here the probabilities that aparticular control input is actually applied are assumed to be time-invariant andhave to be known beforehand As opposed to this, the filter developed in [19],where a setup similar to ours was considered, utilizes an estimate of the currentlybuffered control sequence for the state estimation We consider the case where theactuator is able to acknowledge data packets, i.e., control sequences, that weresuccessfully transmitted from the controller, which is in contrast to [19] wheresuch acknowledgments are not provided Additionally, we take into account that

the acknowledgment packets sent from the actuator to the controller also sufferfrom random delays and losses Consequently, the setup in [19] can be seen as

a special case of the scenario we consider in that acknowledgments are provided

by the actuator but always get lost

Outline This article is structured as follows First, in Sect.2we give a detaileddescription of the considered scenario Then, in Sect.3 we derive an estimatorbased on a formal model of the considered problem The performance of thepresented estimator is then assessed in Sect.4 Finally, Sect.5 concludes thiswork

Notation Throughout the article, vectors will be indicated by underlined

let-ters (x) while random vectors will be underlined and in bold (x) To denote

matrices, we will employ boldface capital letters, e.g., A We use In to denotethen-dimensional identity matrix, 0 to denote zero matrices of arbitrary dimen-

sion, and a subscript k, e.g., x k, to indicate the time step Transposition of

a vector or a matrix is indicated by xT and AT Finally, δ i,j stands for theKronecker delta, i.e.,δ i,j= 1 ifi = j and 0 otherwise.

2 Problem Formulation

Consider an NCS where both plant and sensor are linear and described by

x k+1= Ak x k+ Bk u k+w k ,

y k= Ck x k+v k , (1)

withx k ∈ IR n, andy k ∈ IR mstate and measurement, respectively, andu k ∈ IR l

the control input provided by a given controller The zero mean white noisesequencesw kandv kare Gaussian and independent of each other with covariance

matrices Wk and Vk The initial plant statex0 is Gaussian with mean ˆx0 and

covariance matrix Σ0and is independent ofw iandv j Furthermore, we assumethat all components are synchronized and that the networks assign time stamps

to data packets upon transmission

The actuator is collocated with the plant and connected to the controller via

a lossy network (CA-network), which means that each transmitted data packetcan experience a (potentially unbounded) delay or get lost By interpreting losses

as infinite delays, we can model the delay of a packet that is sent from thecontroller to the actuator at time k by the random variable τ CA

k ∈ IN0 Weadditionally assume that the τ CA

k are independent and identically distributed(i.i.d.) with known probability mass function (PMF)f CA In order to account forthese network-induced effects, the controller does not only transmit the currentcontrol inputu k at timek but also predicted control inputs for the next N time

steps Consequently, the data packet sent to the actuator consists of the controlsequence

where u k+i|k, i = 0, , N denotes the control input computed at time k and

to be applied at time k + i The buffer located at the actuator side employs the so called past packets rejection logic [1]: From the set of all received controlsequences, the one with the largest time index, that is, only the most recentsequence, is maintained while all others are discarded The control inputs pro-vided by this sequence are then successively applied at the corresponding timesteps until a newer sequence reaches the actuator However, in case of subsequentpacket losses or large delays it may occur that the next control sequence arrivestoo late so that the control inputs from the buffered sequence are not applicableanymore In such a case, the default input u df

k = 0 is applied

Remark 1 Applying the default control input u df

k = 0 is known as zero-inputstrategy in literature Another common alternative is to apply the previous con-trol input, i.e.,u df

k =u k−1, which is known as hold-input strategy While the firstone is mathematically more convenient, it has been shown in [20] that even forscalar systems and when only packet dropouts are considered neither strategy issuperior

Each time the stored control sequence is replaced by a more recent one, anacknowledgment (ACK) is sent back by the actuator to indicate a successfultransmission of the corresponding sequence It is important to emphasize thatthese ACKs are application layer acknowledgments, since not every received datapacket is acknowledged by the actuator, but only that one containing the actuallyutilized control sequence From the perspective of the underlying CA-networkthey are just regular payload to be delivered Consequently, the ACKs are alsosubject to delays and losses (infinite delays) which are modeled by the i.i.d.random variables τ AC

k with PMFf AC Due to this acknowledgment procedure,the controller is able to infer applied control inputs upon the reception of ACKsfrom the actuator

Remark 2 Note that, besides the actuator acknowledgment procedure, the

transport layer protocol employed by the CA-network might send out dedicatedacknowledgment packets upon successful reception of data packets A commonexample is the TCP protocol, while UDP is an example for a transport layer pro-tocol that does not acknowledge received packets TCP implementations enhancethe reliability of the communication compared to UDP, since they issue packetretransmissions in case such an acknowledgment packet is delayed or missing.However, it is known that this behavior often poses a severe problem for rela-tively short transfers [21,22] Also, data losses are traded for large delays, which

is typically not desired in Networked Control Systems [3]

Remark 3 In NCS literature the notion of UDP-like networks is used to describe

transmissions where received data packets are not acknowledged by the receiver,

while the term TCP-like refers to idealized transmission schemes that

acknowl-edge successful transmissions instantaneously and without losses [17] In this

regard, the setup we consider could be summarized as UDP-like network withapplication layer acknowledgments.

Finally, at each time step, a sensor takes a noisy measurement of the state andsends it over another network (SC-network) to an estimator which is attached

to the controller Delays and losses in this network are described by the i.i.d.random variablesτ SC

k with given PMFf SC, so that at each time instant multiplemeasurements (or none) can arrive at the estimator Note that in contrast tothe CA-network, (i) all delayed packets do provide valuable information aboutpast states and hence should be processed by the estimator and (ii) this net-work appears deterministic for the estimator since the packet delays are knowndue to the assigned time stamps However, as the estimator’s buffer is finite,only up to M ∈ IN measurements can be stored at the same time As will be

discussed in Sect.3, an appropriate approach to deal with burst and sequence arrivals of measurements is to maintain a fixed measurement history.The following assumption is thus justified

out-of-Assumption 1 Measurements with a delay larger than M − 1 time steps are

discarded by the estimator upon reception

Remark 4 Discarding measurements according to the above assumption always

results in a suboptimal estimator However, for the case that all measurementpackets either arrive with a delay of at mostM −1 time steps or do not arrive at

all, that is, for the case that no measurements have to be discarded, the optimalestimator was presented [14]

The complete setup is depicted in Fig.2 Our goal is to design an estimator which,

at each time stepk, supplies the controller with an estimate ˆx e

k of the plant statebased on the MMSE criterion for the given setup We do so by extending thefilter proposed by Fischer et al [19] for a UDP-like CA-network

BufferActuatorPlant

Fig 2 Considered NCS Setup A control sequenceU k computed by the controller istransmitted to the actuator which buffers the most recent sequence From this sequence,the control inputu k corresponding to the current time step is selected and applied tothe plant

3 Derivation of the Proposed Estimator

The estimator from [19] relies on a stochastic model which jointly describesthe CA-network and the actuator as dynamical system With this model and

a suitable state augmentation, the considered NCS is then expressed in terms

of a Markov jump linear system (MJLS) [23] As we build upon this estimator,

we provide a condensed summary of the resulting model in Sect.3.1 A moredetailed derivation can be found in [8,19] We then present the proposed esti-mator in Sect.3.2

Main ingredients of the network-actuator model are a vector η k which passes all control inputs from the sequencesU k−N , , U k−1that are still appli-cable at time k or later, and an additional discrete, scalar random variable θ k.Formally,η k is given by

Fig 3 Visualization of the elements of η k (encircled) for N = 2 Applicable control

inputs for the same time step are shown one below another

Fig 4 Illustration of the relationship betweenη k+1 (dashed), η k (solid), and U k for

N = 2 Applicable control inputs for the same time step are shown one below another.

where k − N ≤ t ≤ k, we have that θ k ∈ {0, , N + 1} and that θ k =N + 1

corresponds to the case when the buffer ran empty and the default inputu df

wherep j=f CA(j) denotes the probability that a control sequence arrives with

a delay ofj time steps, and q j = 1− j

i=0 p i By means of η k and θ k, we canwrite the actual applied control input according to

u k= Hk η k+ Jk U k , (6)

which is a MJLS with parameter θ k , usually referred to as the mode of the

system Recall from (4) thatθ k ∈ {0, , N + 1}, the augmented system hence

a variety of approximations to the optimal solution have been proposed, rangingfrom LMMSE estimators [26–28] to approaches which at each time instant main-tain only a fixed number of hypotheses about the mode history by applying somehypothesis reduction strategy [29] Among the latter, the Interacting Multiple Model (IMM) filter [24] has gained much attraction as it exhibits a good trade-

off between estimation quality and complexity A variant of the IMM filter for

a UPD-like NCS scenario, which corresponds to this extreme case I k =∅, hasbeen introduced in [19] As already mentioned, we will build upon this estimatorand generalize it to the caseI k ⊂ S k

Another merit of the IMM is that it is widely employed in (multi-)targettracking applications, where, as in Networked Control Systems, delayed andout-of-sequence arrivals of sensor data are common Hence, a lot of work hasbeen conducted in this community to handle these issues, yielding IMM filterswhere retrodiction techniques are used to incorporate both arbitrarily delayedmeasurements and out-of-sequence arrivals [30] Note that this is in contrast toother estimators for MJLS that have been developed to cope with lost or delayed

measurements For instance, the estimators proposed in [31,32] only considerpacket losses, and the MMSE estimator from [33] assumes fixed measurementdelays On the downside, applying retrodiction in our setup necessitates that the

system matrix Ak in (1) is invertible which is not always given Fischer et al.[19] thus proposed to adapt the approach from [34] and to instead maintain

a history of past estimates which is updated upon the reception of a delayedmeasurement Besides being simple, this approach has the advantage that it isinherently suited for dealing with burst arrivals of measurements which can be,for instance, processed one by one Moreover, it can be easily extended to dealwith delayed mode observations We introduce this extension in the following

In essence, the IMM filter is composed of a bank of Kalman filters, onefor each mode, which are individually reinitialized at each time step by mixingall mode-conditioned estimates from the previous time step [29] For the givensystem (7) with N + 2 different modes the IMM filter thus requires N + 2

individual filters, so that the state estimate is maintained as a Gaussian mixturedistribution withN + 2 components Each component is weighted according to

the estimated mode probability distributionπ k At the end of each measurementupdate, the mode distribution is updated according to the mode-conditionedmeasurement likelihoods The point estimate ˆx e

kfor the controller is then simplythe mean of the mixture Suppose now, that at time k, the estimator can infer

a mode realization θ t=L, t ≤ k, L ∈ {0, , N}, due to a received ACK The

proposed extension exploits that the distribution ofθ tthen reduces to

where e L+1 is the (N + 2)-dimensional unit vector with one at position L + 1

and zero elsewhere.1Please note that the mode realizationθ t=N +1 will never

be available to the filter since this indicates that at timet the default input was

applied In such a case, no applicable control sequence would have been received

in time by the actuator, and hence no ACK would have been sent back Notealso that, since the measurement equation in (7) is independent of the mode,

θ t only affects the prediction step at t + 1 Combined with Assumption1 thismeans that it is reasonable to discard all ACKs with a delay larger than M

time steps Integrating a delayed mode observation at time k finally consists of

updatingπ taccording to (8) and then recomputing the estimates fromt+1 to k.

This procedure is also well-suited to handle burst arrivals of ACKs, which meansthat multiple modes can be inferred at once Starting with the oldest one, theyare simply integrated into the recomputation of the state estimates one afteranother One cycle of the proposed estimator is summarized in Algorithm1.For a detailed description of the IMM-specific steps in lines 5, 9, 13, 14 and 16refer to, for instance, [24,29] We can clearly get from the algorithm that bothcomputational complexity and required memory increase with the buffer length

In particular, it necessary to store at least the mode observationsθ k−M , , θ k−1,the control sequencesU k−M , , U k−1, the measurementsy k−M +1 , , y k, andthe Gaussian mixture denoting the estimate from timek − M To represent the

1 The mixture then essentially degenerates into a single Gaussian.

latter,N +2 mode-conditioned means and covariance matrices and the estimated

mode distributionπ k−M must be stored

A reference implementation of the algorithm is available on github as part ofthe CoCPN-Sim simulation framework [35]

Algorithm 1 One Cycle of the Proposed IMM-based Estimator

Input: Estimate from timek − M, i.e., Gaussian mixture

Output: Point estimate ˆx e

k

1: fori = M − 1 to 0 do

3: Updateπ k−i−1according to (8)

5: Reinitialize the mode-conditioned Kalman filters

6: Createη k−i−1according to (2)

// Prediction Step

8: Compute mode-conditioned input by usingη k−i−1,U k−i−1in (6)9: Perform prediction using the mode-conditioned input

// Measurement Update

13: Perform measurement update usingy k−i

14: Evaluate measurement likelihood

in [19] it was shown that this approach is inferior to an IMM-based filter

4 Evaluation

In this section we assess the performance of the proposed estimator in a scenariosimilar to the one used in [19], namely controlling an inverted pendulum on a cartwhich operates in a transient state We compare the proposed estimator withthe original approach from [19] which does not have access to the partial mode

Table 1 Parameters of the inverted pendulum used in the evaluation.

Mass of the pendulum 0.5 kgCoefficient of friction for the cart 0.1 N s/mLength to pendulum center of mass 0.3 mMoment of inertia of the pendulum 0.006 kg m2Gravitational acceleration 9.81 m/s2

history I k Our aim is to evaluate to what extent the information advantage ofthe proposed filter results in improved estimates

To that end, consider the state of the pendulum given by x k =



s k s˙k φ k φ˙kT Here, s k denotes the position of the cart (in m) and φ k isthe deviation (in rad) of the pendulum from the upward equilibrium Lineariz-ing the nonlinear pendulum dynamics (cf., for instance, [36]) with parametersgiven in Table1 around the upward equilibrium and performing a subsequentdiscretization with sampling timet A=0.01 s results in the linear model (1) with

matrix for the calculation of the regulator gain L are given by

ˆ

x0=

0 0.2 0.2 0T

, Σ0= 0.5I4.

Delay in Time Steps

(b) f AC

Fig 5 PMFs of the packet delays Delays larger than five time steps in the SC-link

are treated as packet losses (infinite delay) by the estimators

The probability mass functionsf CA,f AC andf SC used to model the networksemployed in the simulations are depicted in Fig.5 In each simulation run theactual delay of each packet was independently drawn according to the corre-sponding PMF

We chose to set the length of the measurement history toM = 6 Accordingly,

measurements with a delay larger than five time steps and ACKs with a delaylarger than six time steps were discarded For the SC-network described by thePMFshown in Fig.5a the measurement loss rate was thus 4 For the ACKssent from the actuator to the controller, we decided to utilize a distribution (cf.Fig.5b) according to which delays larger than three time steps were very unlikely

As mentioned above, the plant operates in a transient state, i.e., set point changesoccur Thus, in each simulation run, the initial set point of the pendulum was[2 0 0 0]T which changed to [−2 0 0 0]T after 100 time steps and then changedback after another 100 time steps The length of the control sequences computed

by the controller was N + 1 = 7, resulting in an MJLS with 8 modes In each

run, the mode-conditioned Kalman filters of both estimators were initializedwith a Gaussian with mean ˆx0 and covariance matrix Σ0 and the initial modedistribution wasπ0 =e8∈ IR8 Note that neither estimator had impact on thecomputation of the control sequences because the control inputs were computedbased on the true state

In the first simulation, the true PMFf CA from Fig.5a was used in (5) to

compute the transition matrix T of the Markov chain θ k, while in the secondsimulation we assumed that the filters were completely unaware of the behavior

of the CA-link Hence, we employed a uniform PMFinstead in (5) to obtain T.

This decision was motivated by the time-varying nature of real networks, due towhich model mismatches are likely in practical applications

The simulation results in terms of the root mean squared error (RMSE) areshown in Fig.6 for the directly accessible statess k andφ k and in Fig.7for thenon-accessible states ˙s kand ˙φ k We can immediately see from the results that theperformance of both filters does not differ much most of the time In particularwith respect tos k andφ k, the information advantage of the proposed filter only