TEAM LinG MODEL-BASED SIGNAL PROCESSING MODEL-BASED SIGNAL PROCESSING James V Candy Lawrence Livermore National Laboratory University of California Santa Barbara, CA IEEE PRESS A JOHN WILEY & SONS, INC., PUBLICATION Copyright 2006 by John Wiley & Sons, Inc All rights reserved Published by John Wiley & Sons, Inc., Hoboken, New Jersey Published simultaneously in Canada No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as permitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, (978) 750-8400, fax (978) 750-4470, or on the web at www.copyright.com Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, (201) 748-6011, fax (201) 748-6008, or online at http://www.wiley.com/go/permission Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book, they make no representations or warranties with respect to the accuracy or completeness of the contents of this book and specifically disclaim any implied warranties of merchantability or fitness for a particular purpose No warranty may be created or extended by sales representatives or written sales materials The advice and strategies contained herein may not be suitable for your situation You should consult with a professional where appropriate Neither the publisher nor author shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages For general information on our other products and services or for technical support, please contact our Customer Care Department within the United States at (800) 762-2974, outside the United States at (317) 572-3993 or fax (317) 572-4002 Wiley also publishes its books in a variety of electronic formats Some content that appears in print may not be available in electronic formats For more information about Wiley products, visit our web site at www.wiley.com Library of Congress Cataloging-in-Publication Data: Candy, J V Model-based signal processing / James V Candy p cm ISBN-13 978-0-471-23632-0 (cloth) ISBN-10 0-471-23632-2 (cloth) Signal processing—Digital techniques—Textbooks I Title TK5102.9.C32 2005 621.382’2–dc22 2004065042 Printed in the United States of America 10 Praise the Lord, Jesus Christ, Our Savior! In times of great need and distress—He comforts us! CONTENTS Preface Acknowledgments Introduction 1.1 1.2 1.3 1.4 1.5 1.6 2.4 2.5 xxi Background / Signal Estimation / Model-Based Processing Example / Model-Based Signal Processing Concepts / 11 Notation and Terminology / 16 Summary / 16 MATLAB Notes / 16 References / 17 Problems / 17 Discrete Random Signals and Systems 2.1 2.2 2.3 xv 21 Introduction / 21 Deterministic Signals and Systems / 21 Spectral Representation of Discrete Signals / 24 2.3.1 Discrete Systems / 26 2.3.2 Frequency Response of Discrete Systems / 29 Discrete Random Signals / 32 2.4.1 Motivation / 32 2.4.2 Random Signals / 36 Spectral Representation of Random Signals / 44 vii viii CONTENTS 2.6 2.7 2.8 2.9 2.10 2.11 2.12 2.13 2.14 Discrete Systems with Random Inputs / 57 ARMAX (AR, ARX, MA, ARMA) Models / 60 Lattice Models / 71 Exponential (Harmonic) Models / 79 Spatiotemporal Wave Models / 83 2.10.1 Plane Waves / 83 2.10.2 Spherical Waves / 87 2.10.3 Spatiotemporal Wave Model / 89 State-Space Models / 92 2.11.1 Continuous State-Space Models / 92 2.11.2 Discrete State-Space Models / 98 2.11.3 Discrete Systems Theory / 102 2.11.4 Gauss-Markov (State-Space) Models / 105 2.11.5 Innovations (State-Space) Models / 111 State-Space, ARMAX (AR, MA, ARMA, Lattice) Models / 112 State-Space and Wave Model Equivalence / 120 Summary / 124 MATLAB Notes / 124 References / 125 Problems / 127 Estimation Theory 3.1 3.2 3.3 3.4 3.5 Introduction / 135 3.1.1 Estimator Properties / 136 3.1.2 Estimator Performance / 137 Minimum Variance (MV ) Estimation / 139 3.2.1 Maximum a Posteriori (MAP) Estimation / 142 3.2.2 Maximum Likelihood (ML) Estimation / 143 Least-Squares (LS ) Estimation / 147 3.3.1 Batch Least Squares / 147 3.3.2 LS : A Geometric Perspective / 150 3.3.3 Recursive Least Squares / 156 Optimal Signal Estimation / 160 Summary / 167 MATLAB Notes / 167 References / 167 Problems / 168 Equivalence 135 ix CONTENTS AR, MA, ARMAX, Lattice, Exponential, Wave Model-Based Processors 175 4.1 4.2 Introduction / 175 AR (All-Pole) MBP / 176 4.2.1 Levinson-Durbin Recursion / 179 4.2.2 Toeplitz Matrices for AR Model-Based Processors / 185 4.2.3 Model-Based AR Spectral Estimation / 187 4.3 MA (All-Zero) MBP / 191 4.3.1 Levinson-Wiggins-Robinson (LWR) Recursion / 193 4.3.2 Optimal Deconvolution / 198 4.3.3 Optimal Time Delay Estimation / 201 4.4 Lattice MBP / 207 4.5 ARMAX (Pole-Zero) MBP / 213 4.6 Order Estimation for MBP / 220 4.7 Case Study: Electromagnetic Signal Processing / 227 4.8 Exponential (Harmonic) MBP / 238 4.8.1 Exponential MBP / 240 4.8.2 SVD Exponential MBP / 247 4.8.3 Harmonic MBP / 250 4.9 Wave MBP / 262 4.10 Summary / 271 MATLAB Notes / 272 References / 272 Problems / 275 Linear State-Space Model-Based Processors 5.1 5.2 5.3 5.4 5.5 5.6 5.7 5.8 281 State-Space MBP (Kalman Filter) / 281 Innovations Approach to the MBP / 284 Innovations Sequence of the MBP / 291 Bayesian Approach to the MBP / 295 Tuned MBP / 299 Tuning and Model Mismatch in the MBP / 308 5.6.1 Tuning with State-Space MBP Parameters / 308 5.6.2 Model Mismatch Performance in the State-Space MBP / 312 MBP Design Methodology / 318 MBP Extensions / 327 5.8.1 Model-Based Processor: Prediction-Form / 327 5.8.2 Model-Based Processor: Colored Noise / 329 5.8.3 Model-Based Processor: Bias Correction / 335 664 Kalman filter, 175, 281, 284, 290, 291, 293, 300, 318, 358, 372, 513 gain matrix, 112 Kalman-Szego-Popov, 495 Kirchoff current equations, 304 Kullback-Leibler Information, 221 Kurtosis, 395, 397 Lagrange multiplier, 259 Laplace transform, 25, 28, 29, 32, 79, 96, 97 Laplacian, 87 Laplacian operator, 84 Laser experiment, 204 ultrasonics, 562, 570, 627 Lattice, 71, 176, 179, 187, 460 algorithms, 456 filter, 207, 212, 451 method, 212 model, 73, 207, 210, 451, 455 processor, 185, 452 recursion, 207, 208, 460 structure, 454 technique, 210 transformations, 118 Law of large numbers, 222 Layered homogeneous medium, 72, 203 Least mean-squared, 430, 565 Least-squares, 147, 152, 153, 167, 243 approach, 142 estimate, 142, 148, 172, 221 estimation, 142, 147, 167, 248 problem, 150, 153, 155, 156 techniques, 456 Level of significance, 302, 576 Levinson all-pole filters, 209 recursion, 75, 207, 212 Levinson-Durbin, 176, 182, 191, 212 recursion, 179, 183, 185, 186, 193, 194, 276 Levinson-Wiggins-Robinson, 191, 548, 565 INDEX Likelihood, 139, 142 function, 144, 392 method, 143 ratio, 537 detectors, 295 solutions, 490 Linear algebra, 97 combiner, 486 discrete-time system, 27 dispersive waves, 610 dynamical equations, 573 least-squares, 257 prediction, 185, 194, 251 theory, 245 predictor, 176, 448 state-space models, 490, 493, 495 systems theory, 34 time-invariant, 96, 98 time-invariant system, 96, 104 time-varying, 95 time-varying state-space representation, 101 transformations, 43 Linearization, 313, 367, 369, 377, 537 Linearized, 413 filter, 370 measurement perturbation, 369 model, 367 process model, 369 Local iteration, 385, 411 Localization, 7, 202, 266, 269, 471, 600, 608, 628 problem, 598, 607, 628 Logarithmic a posteriori probability, 372 Log-likelihood, 144, 222 equation, 145 function, 224 ratio, 267 Lumped physical, Lyapunov equation, 106 Machine condition monitoring, 447 INDEX Markov, 49 parameters, 104 process, 41 sequence, 104 Matched filtering, 57 Matched-field processing, 265 Matched-field processor, 268 Matching vector, 268 Matrix difference equation, 102 differential equation, 98 exponential, 97 series, 354 fraction transfer function form, 226 gradient filter, 492 inversion lemma, 119, 158, 218, 297, 389, 438 Matrix square root, 111, 398, 641 Maximum a posteriori, 142, 143, 145, 146, 167, 170, 295, 607, 609, 621 estimate, 146, 615 Maximum entropy method, 550 spectral estimate, 448 Maximum likelihood, 142, 144, 145, 167, 170, 236, 251, 263, 489, 601 estimate, 143–145, 302 Maximum log-likelihood, 223 Mean, 398, 634 log-likelihood, 224 propagation recursion, 63 stationary, 41 Mean-squared, 432 convergent, 137 error, 161, 277, 424, 425, 432, 439 criterion, 141, 423, 429, 444, 451, 564 estimate, 321 Measure of quality, Measurement, 93, 101, 173 covariance, 108 matrix, 91, 354 equation, 336, 586 filter, 281 665 filtering, 340 gradient weighting matrix, 502 instrumentation, 18, 319, 353 jacobian, 386, 515 linearization, 372 matrix, 93 mean vector, 106 model, 18, 82, 122, 251, 284, 328, 339, 351, 353, 444, 460, 462, 571, 584, 598, 602 noise, 296, 319, 327, 462 covariance, 309 dynamics, 330 nonlinearities, 385, 390, 392 perturbation, 369 power spectrum, 108 space, 291, 329 system, 102, 170, 332 model, 319, 330 uncertainties, 353 variance, 106 vector, 93 Mechanical system, 573 Michelson interferometer, 567 Minimal, 95 polynomial, 104 realizations, 104 Minimization, 444 Minimum error variance, 299, 306, 357, 474, 549 estimate, 300, 313, 338 eigenvalue, 255 mean-squared error, 432, 484, 548 norm, 258, 259 variance, 167, 293, 560 design, 299, 321, 560 distortionless response, 545 estimate, 139–143, 147, 148, 170, 172, 177, 285, 286, 288, 294, 329, 339, 547, 564, 593, 601 processor, 308 solution, 612 Misadjustment, 432, 438 Mismatch, 314, 315, 576 666 Modal, 427 coefficients, 599 coordinates, 428, 429 form, 427 functions, 596, 600, 601 matrix, 427 simulation, 574 state space, 525 transformation, 427 Mode propagation, 597 Model development, 319 order, 225, 229, 240 recursion, 208 reference, 564 set, 314 validation, 229 Model-based, 191 algorithm, 587 application, 593 approach, 1, 8, 150, 175, 201, 204, 207, 571, 577, 594, 599, 601, 621, 627, 628 array processing, 584, 586 dispersive processor, 628 wave, 607 estimation, 186 framework, 219, 460, 498, 530, 627 identifier, 340, 341 localization, 522, 599, 601, 603, 606, 607 problem, 599 matched-field processing, 266 methods, 187 parameter estimation, 240 parametric approach, 175, 176 predictor, 467 processing, 14, 92, 150, 175, 285, 295, 445, 556, 571, 583, 595, 608, 627, 628, 647 package, 617 processor, 4, 9, 18, 112, 198, 207, 214, 281, 291, 292, 299, 322, 358, INDEX 439, 571, 584, 595, 609, 614, 617, 647 scheme, 432 solution, 584, 607, 609, 613 spectral density, 189 spectral estimation, 187, 189 wave enhancer, 617 estimation problem, 262 Model-reference processor, 565, 568, 570 Modeler, 321 Modeling error, 241, 313, 314, 317 inaccuracies, 302 Modulation signal, 89, 449 Moment of inertia ratio, 540 Moments, 394, 395 Momentum vector, 541 Monochromatic wave model, 614 Monte Carlo, 378 approach, 327 method, 322, 323 Motion compensation problem, 149 Moving array, 584 Moving average, 62, 114, 115 model, 74, 119, 276 Multichannel, 108, 191, 226, 571, 584 classifier, 577 Multihypothesis, 576 Multipath, 364 Multiple input–multiple output, 489 Multiple-input, 385 Multiple-output, 385 Multivariable Gauss-Markov representation, 496 Multivariable representation, 96 transfer function, 96 Multivariate, 398 MUSIC, 257, 264 Mutual information, 221, 636 Narrowband, 89, 90, 468 model, 92 INDEX sinusoid, 447 wave model, 90 Natural modes, 235 Navigation, 404 Near-field, 88, 597 Neural network, 416 Newton, 481, 483 direction, 421 recursion, 434 Newton’s method, 237 Newton-Rhapson, 388, 411 Neyman-Pearson, 295, 576 theorem, 267 Noise averaging, 257 cancelation, 462 canceler, 460, 462, 463, 465, 469, 470, 475, 479, 485, 486 covariance, 320, 321, 500, 507 estimates, 509 matrices, 91, 491, 497 eigenvalues, 257 eigenvector, 253–255, 256 matrix, 258 filter, 463 models, 284, 571 ratio, 308, 309 sequences, 329 sources, 319 spectrum, 478 statistics, 308, 313, 507 subspace, 253, 255, 257, 258, 263 vectors, 253 Nondestructive evaluation, 202, 203, 561, 570, 627 Nonlinear, 122, 490, 584, 586 bearing model, 383 case, 321 cost function, 387 discrete-time state-space representation, 100 dynamic equations, 319 dynamic system, 95, 513, 514 filtering, 377, 381, 392 functions, 321 667 least-squares, 236, 600 measurement, 373 model, 313, 321, 369, 397, 401, 514 model-based processors, 392, 411 optimization, 595, 599, 628 optimization problem, 240 parameter estimator, 513, 531 process, 397, 398 processors, 401, 410 state estimation, 392, 411 state-space models, 491 processor, 385 systems, 512 stochastic vector difference equations, 367 systems, 367, 385, 413, 518 trajectory estimation problem, 401 transformation, 393–395, 399 vector functions, 367 system, 612 Nonparametric, 28 impulse response, 30 Nonrandom constant, 145 Nonrecursive, 28 Nonstationary, 108, 322, 419, 434, 463, 554, 575 prediction error, 554 Nonwhite, 337, 576 process, 333 Normal, 13 equations, 155, 179, 192, 550, 564 form, 117 mode propagation model, 522, 595, 602, 607 process, 42 state-space form, 118 Normal-mode representation, 522 pressure-field propagation model, 525 Normal-modes, 595 668 Normalization, 434 condition, 396 constraint, 393 Normalized correlation, 520 covariance, 301 gaussian random vector, 638 innovations variance, 302 least mean-square, 434 Nuclear waste, 621 Null hypothesis, 301 Numerical implementation, 518 Numerically stable, 516, 646 Nyquist sampling theorem, 99 Observability matrix, 104 Observable, 102, 312, 587 Observation well, 621, 622 Observer canonical form, 113, 114 Ocean acoustic, 522, 594, 599, 607 application, 531 normal-mode solutions, 595 problem, 522 environment, 522 propagation medium, 595 On-line, 513 One-step convergence, 436 predicted estimate, 177 prediction error, 291 Optics, 205 Optimal, 9, 300, 322, 443 batch solution, 187 Bayesian processor, 608 corrected estimate, 294 covariance, 322 decomposition, 550 deconvolution, 200 design, 321, 323 dispersive wave model-based processor, 613 estimate, 161–164, 166, 180, 191, 199, 205, 294, 419, 463, 486, 609 filter, 478 INDEX least-squares parameter estimate, 242 noise filter, 478 performance, 327 processor, 204, 324 signal estimation, 160, 167 processing, 150 solution, 428 Wiener solution, 164, 167, 196 Optimality, 291 condition, 507 Optimization, 236, 387, 420, 599 problem, 600 theory, 434 Optimum solution, 194, 431 Order, 205, 220, 230, 248, 478, 565 estimation, 180, 183, 220 recursive, 198 tests, 230, 237 Ordered moments, 396 Orthogonal, 43, 129, 140, 152, 153, 161, 172, 186, 251, 253–255, 286, 293, 329, 427 complement, 153 decomposition, 152, 154, 163, 167, 294, 641, 644 property, 329 projection, 152, 153, 161, 207 matrix, 153 theory, 153 subspace, 152 Orthogonality, 140, 291, 452 condition, 141, 142, 153, 154, 162, 177, 191, 199, 208, 242, 244, 245, 264, 292 property, 254, 286, 287, 288, 290, 292, 294, 330, 451 relations, 245 Orthogonalization, 293, 643 Orthonormal, 427 basis, 155 Output covariance matrix, 484, 496 error, 236 INDEX Parameter, 28, 554 adaptive, 10 change, 556 estimation, 143, 213, 215, 219, 225, 237, 279, 425, 431, 446, 489, 513, 516, 520, 521, 535, 580, 595, 599, 603 iteration, 425, 431, 447, 454, 456, 459 vector, 420, 431, 493, 592 Parametric approach, 175 change detector, 554 detector, 558, 561 estimator, 513 iteration, 434 methods, 175 processor, 175, 220, 221, 279, 444 Parametrically adaptive, 491, 495, 500, 512, 522, 536, 599, 601, 606, 607, 652 model-based processor, 512, 516, 522, 528, 531, 583, 607 Parsimony, 223 Partial correlation coefficient, 181 differential equation, 621, 624, 628 fraction expansion, 117, 238 Partitions, 515 Passive localization, 382 problem, 594 Passive sonar processing, 470 Peak detection, 254 Penalty function, 394 Performance analysis, 321, 322, 424 Perturbation model, 370, 415 trajectory, 368 Phase, 615 change, 553, 556 Change Detector, 557, 558, 560 fronts, 610 functions, 609 modulation, 537 speed, 610 669 velocity, 615 Phased array radar, 405 Phenomenological models, Physical system, 92, 99, 100 variables, 94 Physics, 491 Physics-based, 491, 522, 531, 627 applications, 628 processing, 491 Piecewise constant, 343, 514 Pisarenko harmonic decomposition, 255 Planar wavefront, 265 Plane wave, 8, 85, 91, 123, 262, 470, 584, 585, 587, 608 measurement model, 586 model, 123, 583 propagation, 10 signal, Plasma, 476 estimation, 477 pulse, 475 PM system, 538 Pneumatic bubbler, 351 Polar coordinates, 381 Pole, 28, 31, 32, 59, 98, 235, 237, 309 Pole-zero, 76, 134, 220, 443 adaptive filter, 481 solutions, 443 form, 31 Polynomial range model, 560 Polytope optimizer, 606 Position estimation, 404 Positive definite, 185, 186 Posterior, 142, 143, 397 density, 612 mean, 398 predicted (state) residual, 400 Postexperimental design, 478 Power estimator, 254, 257, 260 function, 254, 259 matrix, 252 method, 251, 258, 264 670 Power (Continued) spectral density, 46, 49, 81 spectrum, 34, 44, 57, 130, 416 Pre-whitening, 187, 193 Precession frequency, 550, 556 half-cone angle, 540 Precision, 12, 136, 354 Predicted cross-covariance, 401 error covariance, 284, 507, 508, 514 estimate, 299, 378 estimation error covariance, 289 gain, 328 measurement, 284, 400, 514, 575 perturbation, 378 state error covariance, 328 state estimate, 330 variances, 589 Prediction, 284, 289, 467, 470 distance, 467 equations, 328, 329, 335, 586, 644 error, 177, 180, 207, 214, 217, 220, 224, 230, 236, 241, 299, 446, 493, 553, 554, 561 approach, 213 correlations, 553 covariance, 494 criterion, 214, 229 filter, 213, 219 gradient, 177 method, 481 model, 185, 501 sequence, 237 tests, 229 variance, 179, 187, 209, 225, 493, 550 filters, 176 form, 327, 328, 330, 362, 499 gain, 362 horizon, 470, 471 phase, 284 Predictor, 278, 466–468, 491, 500, 502 coefficients, 187, 246, 248, 249, 451 model, 500 INDEX polynomials, 277 Predictor-corrector, 281, 282, 327, 362 form, 281, 362, 516 Pressure measurements, 352, 357 Pressure-field, 524, 584, 596, 59 measurement, 522, 526, 592, 601 model, 524, 599 Principal components, 260 Prior, 142 Probabilistic axioms, 636 chain rule, 40, 41, 636 Probability, 36 density function, 144 distribution function, 37, 392, 632 function, 36, 631 mass function, 36, 38, 42, 508, 632, 635 space, 36, 38 theory, 634 Process, 101 dynamics errors, 315 matrix, 93 model, 313, 319, 351, 400, 571 noise, 284, 332 covariance, 304, 308, 507–510, 617 statistics, 507 dynamics, 330 estimator, 510 Process-to-measurement noise, 308 Processor, 308, 321, 322, 327, 331, 358 accuracy, 327 bandwidth, 309 design, 323, 358 gain, 507 model, 322 statistics, 303 Projection, 152, 153, 161, 258 matrix, 152, 153, 258 operator, 258 theory, 167 Prony harmonic decomposition, 246 method, 240 normal equations, 245 INDEX problem, 248 SVD, 260 technique, 238, 240, 246, 248, 251 Propagating waves, 83 Propagation, 522 direction, 85 dynamics, 319 model, 268, 565 speed, 591 Propagator, 600 Pulse transfer function, 61, 62 representation, 134 Pump drawdown test, 621 Purely random, 48 Quadratic prediction error criterion, 492 Quality, 179 Quasi-Newton methods, 421 Quasi-stationary, 322, 468, 553, 575 Radar, 535, 608 cross section, 541 signature data, 559 system, 362 tracking system, 475 Radiolysis effects, 351 Random, 21 amplitude, 171 inputs, 57, 105, 339 noise, 14, 240 parameter, 142, 165 process, 38 sequence, 251 signal, 5, 21, 22, 34–36, 38, 41, 44, 48, 53, 54, 57, 63, 71, 105, 135, 160, 167, 175, 275 time function, 37 variable, 37, 38, 42, 129, 172, 631 vector, 169, 393 vector space, 162 walk, 514 Range, 409 curvature, 590 671 depth, 601 estimation, 584, 590 measurements, 409 polynomial, 547 Range-depth function, 599, 600, 602, 606 parameters, 603, 604 Rank, 104 condition, 104 Rational form, 29 function, 28 lattice form, 76, 77, 120 recursion, 120 Rayleigh-distributed, 171 RC-circuit problem, 14, 498 Realization, 38, 631 problem, 104 Recursion, 438, 504 Recursive, 28, 159, 167 approach, 156 estimation, 156 extended least-squares, 229 filtered solution, 287 form, 156, 157, 160, 437, 493, 494 identification, 341 least-squares, 156, 157, 237, 341, 436, 481 maximum likelihood, 229 parameter estimators, 213 prediction error, 491, 495 approach, 531 method, 213, 474 model, 491 method, 553 processor, 392, 397 solution, 191, 193, 285, 287 Recursive-in-time, 216, 473, 474, 552, 553, 556, 560, 627 Reduced-order model, 313 Reentry vehicle, 540 dynamics, 556, 561 response, 542 radar signatures, 627 672 Reference channel, 470, 548 input, 463 measurement, 369, 371 noise, 462 response, 570 signal, 465, 469, 470 state, 377 trajectory, 367, 368, 370, 375, 377, 385, 386 Reflection, 181 coefficient, 71, 76, 186, 202, 207–209, 277, 451, 454 iteration, 460 Relative delay times, 88, 590 Repeated least squares, 221 Representation theorem, 59, 60, 330 Residual, 218, 321 errors, 443 sequence, 11 Residue, 32, 235 theorem, 166 Resolvant, 98 matrix, 97 Resonances, 237, 577 Resonant frequencies, 576 Response time, 98 Riccati equation, 362 RMS error, 357 modeling errors, 529 standard error, 529 Root, 98, 242, 255, ,257, 449 Rule-of-thumb, 104 S -plane, 98 Sample correlation function, 321, 619 mean, 300 space, 36, 631 variance, 301, 306 estimators, 303 Sampled data, 100 model, 100 INDEX discrete system, 100 Sampling interval, 31, 32, 80 theory, 79 Sampling-resampling, 392 Satellite communications, 405 Scalar performance index, 323 Schmidt-Kalman filter, 343 Schur-Cohn stability technique, 208 Search direction, 422 Second order statistics, 292 Seismic exploration, 202 waves, 608 Self-tuning filter, 470, 487 Sensitivity analysis, 323 Sensor array, 88, 590 measurement model, 262 Sensor dynamics, 572 Separation constants, 84 Separation of variables, 84, 87, 122 Sequential, 156, 167 measurement processing, 644 processing, 379, 641, 643, 644 Shallow ocean acoustic application, 628 water ocean experiment, 595 Shear arrivals, 563 Short-time Fourier transform, 474, 553 Sigma points, 393, 397, 398 Signal, aperiodic, 22 characteristics, 419 covariance matrix, 91 continuous, 22 deterministic, 21, 22 direction vectors, 263 discrete, 21 eigenvectors, 253–255 energy, 478 enhancement, 5, 8, 600, 609 error test, 226, 233 estimation, 5, 176, 197, 419 model, 443 periodic, 22 processing, 21, 33, 95, 299 INDEX random, 22 representations, 188 sampled, 22 sinusoidal, 23 subspace, 152–154, 253, 257, 260 vector, 153 unit step, 26 Significance level, 301, 302 Singular value decomposition, 153, 154, 167 values, 167, 248 Singularity expansion method, 228, 235 Sinusoid, 23, 26 Sinusoidal covariance, 471 disturbances, 478 model, 82 signal, 3, 34 Sinusoids, 189 SNAP, 523 SNAP model, 529 SNR, 2, 319 Solution mass, 353 Sonar, 382 applications, 628 system, 471 Sonic waves, 608 Sound speed profile, 522, 523, 596 model, 525 Space vehicles, 539 Space-time, 83 acoustic array processing, 594 array processing, 584 equations, 623 impulse response, 202 signal, 83 wave, 90 Space-varying, 527 Spatial covariance matrix, 263 estimation, 470 frequency, 262 variable, 85 localization problem, 263 state vector, 124 673 Spatio-temporal, 83 velocity field, 616 wave model, 89, 90, 262 model, 124 signal, 83, 262, 263, 616 Spectral analysis, 556 covariance matrix, 92 decomposition, 253, 263 density, 48 estimation, 10, 175, 187–189, 220, 448, 545, 550, 551, 556, 627, 652 factor, 496 factorization, 59, 112, 166, 497 theorem, 59, 496 line, 470 match, 478 peaks, 264, 478, 551 shaping, 60 Spectrogram, 449, 473, 475, 553, 556–558, 560, 627 Spectrum, 545, 551 simulation procedure, 59 Speed of convergence, 429 Spherical coordinates, 87 wave, 87, 88, 91, 263, 584 equation, 87, 590 processors, 628 Spherical wavefront, 262, 268 Spherically spreading propagation models, 583 Spin rate, 540 Square root methods, 644 processors, 644 Squared error criterion, 142 Stability, 98, 429, 433, 440 test, 75, 218, 443 Stabilized, 365 Stable, 27–29 estimate, 434 Standard error, 232 Standard Gauss-Markov model, 111 674 State, 93–95 covariance, 106, 320 equations, 96, 97 estimate, 290, 312, 315, 320, 370, 377 estimation error, 288, 290, 296, 303, 315 estimation problem, 285, 295 information, 399 input transfer matrix, 97 mean vector, 106 orthogonality condition, 293 perturbation, 370, 372 predictor gradient weighting matrix, 502, 504 propagation, 328 transition matrix, 98, 101, 102, 414, 426, 597 mechanism, 169 variable, 93, 95, 397 variance, 106 vector, 95, 97, 353 State-space, 95, 123, 176, 281, 284, 291–293, 304, 308, 405, 411, 491, 495, 592, 594, 595, 597, 599, 627 description, 608 equation, 426, 428 estimators, 651 feed-forward lattice form, 118 feedback lattice form, 119 form, 134, 353, 367, 454, 491, 523, 571, 583, 597 formulation, 610 framework, 614 model, 95, 99, 284, 289, 299, 312, 313, 489, 535, 585, 586, 595, 611, 628, 641 model-based processor, 290 postprocessor, 648 preprocessor, 648 propagator, 526, 600 rational lattice form, 119 representation, 94, 95, 101, 281, 284, 427, 492, 522, 573, 608, 609 INDEX techniques, 430 wave model, 120 Stationary, 41, 107, 198, 419, 443, 468 noise processes, 291 process, 50, 59 signal, 423 Statistical consistency, 509 hypothesis test, 300 performance, 571 test, 237 tests, 299, 321, 553, 616, 618, 619 Statistically independent, 430 white, 306, 575 Steady-state, 108, 303, 312, 324, 358, 361, 424, 495, 533 gain, 510 processor, 361 Wiener solution, 440 Step-size, 237, 419, 421, 422, 428, 429, 431–433, 441, 454, 459, 486, 565 adjustment, 456 Stochastic approximation, 430, 509 deconvolution problem, 342 gradient, 422, 423, 431, 435, 440, 445, 455, 459, 463, 481, 483, 485, 486 algorithm, 425, 428–431, 455 technique, 424 Newton, 440 process, 36, 37, 38, 41, 44, 92, 135, 284, 378, 633, 635 realization, 496, 534 Stopping rule, 386 Storage coefficient, 622, 626 Storage tank, 351, 358 Storativity, 621 Structural displacements, 575 failure detection, 577, 581, 583 failures, 627 model, 534, 579 process model, 573, 574 INDEX response, 577 system, 94, 573, 575–577 Suboptimal, 521 method, 343 processor, 323, 324, 327 state estimates, 326 Subspace, 152, 253, 263 decomposition, 258 Subsurface hydrology, 621 Sufficient, 137 statistic, 145 Sum decomposition, 53, 58, 59, 130, 166, 496 Sum-squared error, 436, 438 criterion, 147 model error, 240 prediction error, 242 Surface displacement, 562, 563 Synthetic aperture, 584 array processing, 523, 628 System linear time invariant, 27 identification, 188, 465, 531, 571 problem, 338 model, 100 order, 220 Systems theory, 32, 41, 95, 97, 102, 105 Target, 415 bearings, 583 localization, 250 Taylor series, 100, 215, 368, 369, 387, 392, 397, 414, 420, 492, 527, 531, 536, 586 Temporal frequency, Test statistic, 301, 302 Threshold, 268 Time average, 42 Time averages, 209, 452 Time averaging, 445 Time constant, 432 Time delay, 88, 91, 201–203, 416, 590–592 675 Time reverse, 128 series analysis, 175 Time-to-failure, 577 Time-correlated, 327, 329, 330 noise, 330 Time-frequency, 557, 560 estimation, 556 representation, 473 Time-invariant 27 predictor, 499 systems, 102 Time-uncorrelated, 287 Time-varying, 377, 419, 536 bandwidth, 308 matrices, 370 polynomial, 448 problem, 448 Toeplitz, 246 correlation matrix, 550, 564 matrix, 176, 178, 179, 185, 186, 191–193, 198, 549 structure, 234 Tomography, 522 Towed array, 583 Tracking, 202, 411, 448, 471 filter, 411 problem, 381, 382, 404, 410 radar, 539 telescope, 19 Training sequence, 416 Trajectory, 149, 159 motion compensation, 547, 556, 560 Transfer function, 3, 29, 31, 61, 73, 74, 76, 96, 101, 104, 117, 127, 167, 276, 486, 540, 560 matrix, 97 Transform, 25 Transformation matrix, 173, 286, 459 Transformed statistics, 398 Transient, 344 data, 468 performance, 311, 312 plasma pulse, 478 problems, 342 676 Transient (Continued) pulse, 479, 481 signal, 477 Transition matrix, 97, 309, 426, 454 Transmissivity, 621, 622, 626 Trend, 149 estimation, 560 removal, 149, 159 Trial-and-error process, 327 Triangular matrix, 185 True measurement, 332 Truncated SVD, 260 Truth model, 320, 322, 323, 574 Tune, 303, 327 Tuned, 300 Tuning, 308, 322, 354 example, 324 problem, 311, 339 U-D factorization method, 282 factorized form, 439, 516 Ultrasonic signal, 562, 567 waves, 562 Unbiased covariance estimator, 245 estimate, 169, 431 estimator, 136, 291 Unconditionally unbiased, 136, 140 Uncorrelated, 161 gaussian variable, 43 innovations, 458 noise, 111 output, 467 Uniform distribution, 42 random variable, 42 Unit circle, 31, 53, 247, 428 impulse, 22 ramp, 22 step, 22 Unitary matrix, 167, 427 Unity constraint, 258 INDEX Unknown input, 343 Unobservable, 102 Unpredictable, 289 Unscented Kalman filter, 392, 397 model-based processor, 392 transformation, 393, 394 Van der Monde matrix, 243 Variance, 12, 65, 634 Vector measurements, 322 recursion, 445 space, 150 Vibrating structure, 577 Wave, 72 source/target parameters, 262 dynamics, 617 equation, 84, 87, 595 estimation, 609 model, 72, 90 propagation, 608 Wave-field enhancement problem, 621 Waveguide, 595 Wavelength, 85 Wavenumber, 8, 262, 524, 584, 590, 596, 610, 614 vector, 85, 87, 91 spectrum, 523 Wavenumber-frequency, 83 space, 85 Weighted least-squares estimate, 147, 150 quadratic cost function, 493 sample variance estimator, 494 sum-squared error, 492 sum-squared residual, 302, 322, 560, 575 Weighting matrix, 299 White, 105, 111, 162, 177, 230, 284, 290, 292, 295, 306, 314, 321, 340, 344, 385, 391, 409, 414, 575 gaussian noise, 49, 473, 552 gaussian sequence, 50 677 INDEX noise, 48, 59, 131, 191, 275, 330 prediction errors, 554 Whiteness, 50, 292, 322, 391, 507, 510, 554, 555 detector,556, 558, 561 test, 220, 233, 301, 302, 306, 575, 577, 618 Whitening filter, 165, 276 Wide-sense stationary, 41 Wiener, 142, 443 filter, 165, 166, 191, 275, 358, 460, 547, 549, 560 problem, 458 solution, 142, 165, 192, 193, 198, 203, 275, 423, 424, 437, 459, 468, 497 Wiener-Kalman filtering, 112 Wiener-Khintchine, 47, 52 Wold decomposition, 59, 61 Yule-Walker, 178 Z-transform, 58, 101, 104, 108 region of convergence, 25 unit circle, 25 Zero mean, 130, 275, 284, 290, 292, 306, 314, 321, 344, 409, 414, 416, 448, 554, 575, 619 test, 301, 391 misadjustment, 439 Zero-mean/whiteness test, 321, 322, 326, 409, 504, 510, 520, 593, 619 Zero-state, 97 Zeros, 28, 31, 32, 59 Adaptive and Learning Systems for Signal Processing, Communications, and Control Editor: Simon Haykin Beckerman / ADAPTIVE COOPERATIVE SYSTEMS Candy / MODEL-BASED SIGNAL PROCESSING Chen and Gu / CONTROL-ORIENTED SYSTEM IDENTIFICATION: An H∞ Approach Cherkassky and Mulier / LEARNING FROM DATA: Concepts, Theory, and Methods Diamantaras and Kung / PRINCIPAL COMPONENT NEURAL NETWORKS: Theory and Applications ă Hansler and Schmidt / ACOUSTIC ECHO AND NOISE CONTROL: A Practical Approach Haykin / UNSUPERVISED ADAPTIVE FILTERING: Blind Source Separation Haykin / UNSUPERVISED ADAPTIVE FILTERING: Blind Deconvolution Haykin and Puthussarypady / CHAOTIC DYNAMICS OF SEA CLUTTER Haykin and Widrow / LEAST-MEAN-SQUARE ADAPTIVE FILTERS Hrycej / NEUROCONTROL: Towards an Industrial Control Methodology ă Hyvarinen, Karhunen, and Oja / INDEPENDENT COMPONENT ANALYSIS ´ Kanellakopoulos, and Kokotovic ´ / NONLINEAR AND ADAPTIVE Kristic, CONTROL DESIGN Mann / INTELLIGENT IMAGE PROCESSING Nikias and Shao / SIGNAL PROCESSING WITH ALPHA-STABLE DISTRIBUTIONS AND APPLICATIONS Passino and Burgess / STABILITY ANALYSIS OF DISCRETE EVENT SYSTEMS ´ Sanchez-Pe˜ na and Sznaier / ROBUST SYSTEMS THEORY AND APPLICATIONS Sandberg, Lo, Fancourt, Principe, Katagiri, and Haykin / NONLINEAR DYNAMICAL SYSTEMS: Feedforward Neural Network Perspectives ´ nez, and Passino / STABLE ADAPTIVE CONTROL Spooner, Maggiore, Ordo˜ AND ESTIMATION FOR NONLINEAR SYSTEMS: Neural and Fuzzy Approximator Techniques Tao / ADAPTIVE CONTROL DESIGN AND ANALYSIS ´ / ADAPTIVE CONTROL OF SYSTEMS WITH ACTUATOR AND Tao and Kokotovic SENSOR NONLINEARITIES Tsoukalas and Uhrig / FUZZY AND NEURAL APPROACHES IN ENGINEERING Van Hulle / FAITHFUL REPRESENTATIONS AND TOPOGRAPHIC MAPS: From Distortion- to Information-Based Self-Organization Vapnik / STATISTICAL LEARNING THEORY Werbos / THE ROOTS OF BACKPROPAGATION: From Ordered Derivatives to Neural Networks and Political Forecasting Yee and Haykin / REGULARIZED RADIAL BIAS FUNCTION NETWORKS: Theory and Applications ... strategy called the model- based approach provides the essence of model- based Model- Based Signal Processing, by James V Candy Copyright 2006 John Wiley & Sons, Inc INTRODUCTION signal processing [1].. .MODEL- BASED SIGNAL PROCESSING MODEL- BASED SIGNAL PROCESSING James V Candy Lawrence Livermore National Laboratory University of California Santa Barbara, CA IEEE PRESS A JOHN WILEY & SONS, ... the model- based approach” to signal processing for a variety of useful model- sets, including what has become popularly termed “physics -based models It presents a unique viewpoint of signal processing