77un/ Edition DIGITAL SIGNAL PROCESSING Principles, Algorithms, m l Applications J o h n G Proakis Dimitris G M anolakis Digital Signal Processing Principles, Algorithms, and Applications Third E dition John G Proakis Northeastern U niversity Dimitris G Manolakis Boston C ollege PRENTICE-HALL INTERNATIONAL, INC This edition may be sold only in those countries to which it is consigned by Prentice-Hall International It is not to be reexported and it is not for sale in the U S.A , Mexico, or Canada © 1996 by Prentice-Hall, Inc Simon & Schuster/A Viacom Company U pper Saddle River, New Jersey 07458 All rights reserved No part of this book may be reproduced, in any form or by any means, without permission in writing from the publisher The author and publisher of this book have used their best efforts in preparing this book These efforts include the development, research, and testing of the theories and programs to determine their effectiveness The author and publisher make no warranty of any kind, expressed or implied, with regard to these programs or the documentation contained in this book The author and publisher shall not be liable in any event for incidental or consequential damages in connection with, or arising out of the furnishing, performance, or use of these programs Printed in the United States of America 10 ISBN 0-13-3TM33fl-cl Prentice-Hall International (U K ) Limited L ondon Prentice-Hall of Australia Pty Limited, Sydney Prentice-Hall Canada, Inc., Toronto Prentice-Hall Hispanoamericana S.A., M exico Prentice-Hall of India Private Limited, N ew D elhi Prentice-Hall of Japan, Inc., T okyo Simon & Schuster Asia Pie, Ltd., Singapore Editora Prentice-Hall Brasil, Ltda., R io de Janeiro Prentice-Hall, Inc, Upper Saddle River, N ew Jersey Contents PREFACE xiii 1 INTRODUCTION 1.1 S ignals, S ystem s, and S ignal P ro cessin g 1.1.1 Basic Elements of a Digital Signal Processing System 1.1.2 A dvantages of Digital over Analog Signal Processing, 1.2 C lassificatio n o f Signals 1.2.1 Multichannel and Multidimensional Signals 1.2.2 Continuous-Time Versus Discrete-Time Signals 1.2.3 Continuous-Valued Versus Discrete-Valued Signals 10 1.2.4 Determ inistic Versus Random Signals, 11 1.3 T h e C o n c e p t o f F re q u e n c y in C o n tin u o u s -T im e an d D isc re te -T im e S ignals 14 1.3.1 Continuous-Time Sinusoidal Signals, 14 1.3.2 Discrete-Time Sinusoidal Signals 16 1.3.3 Harmonically Related Complex Exponentials, 19 1.4 A n a lo g -to -D ig ita l an d D ig ita l-to -A n a lo g C o n v e rs io n 21 1.4.1 Sampling of Analog Signals, 23 1.4.2 The Sampling Theorem , 29 1.4.3 Q uantization of Continuous-Am plitude Signals, 33 1.4.4 Quantization of Sinusoidal Signals, 36 1.4.5 Coding of Quantized Samples, 38 1.4.6 Digital-to-Analog Conversion, 38 1.4.7 Analysis of Digital Signals and Systems Versus Discrete-Time Signals and Systems, 39 S u m m a ry a n d R e fe re n c e s Problems 39 40 iii iv Contents DISCRETE-TIME SIGNALS AND SYSTEMS 2.1 D isc rete-T im e S ignals 43 2.1.1 Some Elem entary Discrete-Time Signals, 45 2.1.2 Classification of Discrete-Time Signals, 47 2.1.3 Simple Manipulations of Discrete-Time Signals, 52 2.2 D isc re te -T im e S ystem s 56 2.2.1 Input-O utput Description of Systems, 56 2.2.2 Block Diagram Representation of Discrete-Time Systems, 59 2.2.3 Classification of Discrete-Time Systems, 62 2.2.4 Interconnection of Discrete-Tim e Systems, 70 2.3 A n alysis o f D isc re te -T im e L in e a r T im e -In v a ria n t S ystem s 72 2.3.1 Techniques for the Analysis of Linear Systems, 72 2.3.2 Resolution of a Discrete-Time Signal into Impulses, 74 2.3.3 Response of LTI Systems to A rbitrary Inputs: The Convolution Sum, 75 2.3.4 Properties of Convolution and the Interconnection of LTI Systems, 82 2.3.5 Causal Linear Tim e-Invariant Systems 86 2.3.6 Stability of Linear Tim e-Invariant Systems, 87 2.3.7 Systems with Fim te-D uration and Infinite-Duration Impulse Response 90 2.4 D isc rete-T im e System s D e s c rib e d by D iffe re n c e E q u a tio n s 91 2.4.1 Recursive and Nonrecursive Discrete-Tim e Systems, 92 2.4.2 Linear Time-Invariant Systems Characterized by Constant-Coefficient Difference Equations, 95 2.4.3 Solution of Linear Constant-Coefficient Difference Equations 100 2.4.4 The Impulse Response of a Linear Tim e-Invariant Recursive System, 108 2.5 Im p le m e n ta tio n o f D isc re te -T im e S ystem s 111 2.5.1 Structures for the Realization of Linear Tim e-Invariant Systems, 111 2.5.2 Recursive and Nonrecursive Realizations of FIR Systems, 116 2.6 C o rre la tio n of D isc re te -T im e S ignals 118 2.6.1 Crosscorrelation and A utocorrelation Sequences, 120 2.6.2 Properties of the A utocorrelation and Crosscorrelation Sequences, 122 2.6.3 Correlation of Periodic Sequences, 124 2.6.4 Com putation of Correlation Sequences, 130 2.6.5 Input-O utput Correlation Sequences, 131 2.7 S u m m ary a n d R e fe re n c e s Problems 135 134 43 Contents THE Z-TRANSFORM AND ITS APPLICATION TO THE ANALYSIS OF LTI SYSTEMS 3.1 T h e r-T n sfo rm 151 3.1.1 The Direct ^-Transform 152 3.1.2 The inverse : -Transform, 160 3.2 P ro p e rtie s o f th e ; -T n sfo rm 3.3 R a tio n a l c-T ran sfo rm s 172 3.3.1 Poles and Zeros, 172 3.3.2 Pole Location and Time-Domain Behavior for Causal Signals 178 3.3.3 The System Function of a Linear Tim e-Invariant System 181 3.4 In v e rs io n o f th e ^ -T n sfo rm 184 3.4.1 The Inverse ; -Transform by Contour Integration 184 3.4.2 The Inverse ;-Transform by Power Series Expansion 186 3.4.3 The Inverse c-Transform by Partial-Fraction Expansion 188 3.4.4 Decomposition of Rational c-Transforms 195 3.5 T h e O n e -sid e d ^ -T n sfo rm 197 3.5.1 Definition and Properties, 197 3.5.2 Solution of Difference Equations 201 3.6 A n aly sis o f L in e a r T im e -In v a ria n t S ystem s in th e D o m a in 3.6.1 Response of Systems with Rational System Functions 203 3.6.2 Response of P ole-Z ero Systems with Nonzero Initial Conditions 204 3.6.3 Transient and Steady-State Responses, 206 3.6.4 Causality and Stability 208 3.6.5 P ole-Z ero Cancellations 210 3.6.6 M ultiple-Order Poles and Stability 211 3.6.7 The Schur-C ohn Stability Test, 213 3.6.8 Stability of Second-Order Systems 215 3.7 S u m m ary an d R e fe re n c e s P ro b le m s 151 161 203 219 220 FREQUENCY ANALYSIS OF SIGNALS AND SYSTEMS 4.1 F re q u e n c y A n aly sis o f C o n tin u o u s-T im e Signals 230 4.1.1 The Fourier Series for Continuous-Time Periodic Signals 232 4.1.2 Power Density Spectrum of Periodic Signals 235 4.1.3 The Fourier Transform for Continuous-Time Aperiodic Signals, 240 4.1.4 Energy Density Spectrum of Aperiodic Signals 243 4.2 F re q u e n c y A n aly sis o f D isc re te -T im e Signals 247 4.2.1 The Fourier Series for Discrete-Time Periodic Signals, 247 230 Contents V) 4.2.2 4.2.3 4.2.4 4.2.5 4.2.6 4.2.7 4.2.8 Power Density Spectrum of Periodic Signals 250 The Fourier Transform of Discrete-Time Aperiodic Signals 253 Convergence of the Fourier Transform 256 Energy Density Spectrum of Aperiodic Signals, 260 Relationship of the Fourier Transform to the i-Transform , 264 The Cepstrum, 265 The Fourier Transform of Signals with Poles on the Unit Circle, 267 4.2.9 The Sampling Theorem Revisited, 269 4.2.10 Frequency-Domain Classification of Signals: The Concept of Bandwidth, 279 4.2.11 The Frequency Ranges of Some N atural Signals 282 4.2.12 Physical and M athematical Dualities 282 4.3 P ro p e rtie s of th e F o u rie r T n s fo rm fo r D isc re te -T im e S ignals 286 4.3.1 Symmetry Properties of the Fourier Transform, 287 4.3.2 Fourier Transform Theorems and Properties, 294 4.4 F re q u e n c y -D o m a in C h a c te ristic s of L in e a r T im e -In v a ria n t S ystem s 305 4.4.1 Response to Complex Exponential and Sinusoidal Signals: The Frequency Response Function 306 44.2 Steady-State and Transient Response to Sinusoidal Input Signals 314 4.4.3 Steady-State Response to Periodic Input Signals, 315 4.4.4 Response to Aperiodic Input Signals 316 4.4.5 Relationships Between the System Function and the Frequency Response Function 319 4.4.6 Com putation of the Frequency Response Function 321 4.4.7 Input-O utput Correlation Functions and Spectra, 325 4.4.8 Correlation Functions and Power Spectra for Random Input Signals 327 4.5 L in e a r T im e -In v a ria n t S ystem s as F re q u e n c y -S e le c tiv e F ilters 330 Ideal Filter Characteristics, 331 4.5.1 4.5.2 Lowpass, Highpass, and Bandpass Filters, 333 4,5.3 Digital Resonators, 340 4.5.4 Notch Filters, 343 4.5.5 Comb Filters 345 4.5.6 All-Pass Filters 350 4.5.7 Digital Sinusoidal Oscillators, 352 4.6 In v e rse S y stem s an d D e c o n v o lu tio n 355 4.6.1 Invertibility of Linear Tim e-Invariant Systems, 356 4.6.2 Minimum-Phase Maximum-Phase, and Mixed-Phase Systems 359 4.6.3 System Identification and Deconvolution, 363 4.6.4 Hom om orphic Deconvolution 365 vii Contents 4.7 S u m m ary a n d R e fe re n c e s P ro b le m s 368 THE DISCRETE FOURIER TRANSFORM: ITS PROPERTIES AND APPLICATIONS 5.1 F re q u e n c y D o m a in Sam pling: T h e D isc re te F o u rie r T n s fo rm 394 5.1.1 Frequency-Dom ain Sampling and Reconstruction of Discrete-Time Signals 394 5.1.2 The Discrete Fourier Transform (DFT) 399 5.1.3 The D FT as a Linear Transform ation 403 5.1.4 Relationship of the DFT to O ther Transforms, 407 5.2 P ro p e rtie s o f th e D F T 409 5.2.1 Periodicity Linearity, and Symmetry Properties, 410 5.2.2 Multiplication of Two DFTs and Circular Convolution 415 5.2.3 Additional DFT Properties, 421 5.3 L in e a r F ilte rin g M e th o d s B ased on th e D F T 5.3.1 Use of the DFT in Linear Filtering 426 5.3.2 Filtering of Long Data Sequences 430 5.4 F re q u e n c y A n aly sis o f S ignals U sing th e D F T 5.5 S u m m ary an d R e fe re n c e s P ro b le m s 367 394 425 433 440 440 EFFICIENT COMPUTATION OF THE DFT: FAST FOURIER TRANSFORM ALGORITHMS 448 6.1 E fficien t C o m p u ta tio n of th e D F T : F F T A lg o rith m s 448 6.1.1 Direct Com putation of the DFT, 449 6.1.2 D ivide-and-Conquer Approach to Com putation of the DFT 450 6.1.3 Radix-2 FFT Algorithms 456 6.1.4 Radix-4 FFT Algorithms 465 6.1.5 Split-Radix FFT Algorithms, 470 6.1.6 Im plem entation of FFT Algorithms 473 6.2 A p p lic a tio n s o f F F T A lg o rith m s 475 6.2.1 Efficient Com putation of the D FT of Two Real Sequences 475 6.2.2 Efficient Com putation of the D FT of a Z N -Point Real Sequence, 476 6.2.3 Use of the FFT Algorithm in Linear Filtering and Correlation, 477 6.3 A L in e a r F ilte rin g A p p ro a c h to C o m p u ta tio n o f th e D F T 6.3.1 The Goertzel Algorithm, 480 6.3.2 The Chirp-z Transform Algorithm, 482 479 viii Contents 6.4 Q u a n tiz a tio n E ffects in the C o m p u ta tio n o f th e D F T 486 6.4.1 Quantization Errors in the Direct Com putation of the DFT 487 6.4.2 Quantization Errors in FFT Algorithms 489 6.5 S u m m ary an d R e fe re n c e s P ro b le m s 493 494 500 IMPLEMENTATION OF DISCRETE-TIME SYSTEMS 7.1 S tru c tu res fo r th e R e a liz a tio n o f D isc re te -T im e S ystem s 7.2 S tru c tu res fo r F IR System s 502 7.2.1 Direcl-Form Structure, 503 7.2.2 Cascade-Form Structures 504 7.2.3 Frequency-Sampling S tructures1 506 7.2.4 Lattice Structure 511 500 S tru c tu re s for IIR S ystem s 519 7.3.1 Direct-Form Structures 519 7.3.2 Signal Flow Graphs and Transposed Structures 521 7.3.3 Cascade-Form Structures, 526 7.3.4 Parallel-Form Structures 529 7.3.5 Lattice and Lattice-Ladder Structures for IIR Systems, 531 S tate-S p a ce System A n aly sis a n d S tru c tu re s 539 7.4.1 State-Space Descriptions of Systems Characterized by Difference Equations 540 7.4.2 Solution of the State-Space Equations 543 7.4.3 Relationships Between Input-O utput and State-Space Descriptions, 545 7.4.4 State-Space Analysis in the z-Domain, 550 7.4.5 Additional State-Space Structures 554 R e p re s e n ta tio n of N u m b e rs 556 7.5.1 Fixed-Point Representation of Numbers 557 7.5.2 Binary Floating-Point R epresentation of Numbers 561 7.5.3 E rrors Resulting from R ounding and Truncation 564 Q u a n tiz a tio n of F ilte r C o e fficien ts 569 7.6.1 Analysis of Sensitivity to Quantization of Filter Coefficients 569 7.6.2 Q uantization of Coefficients in FIR Filters 578 7.7 R o u n d -O ff E ffects in D igital F ilte rs 582 7.7.1 Limit-Cycle Oscillations in Recursive Systems 583 7.7.2 Scaling to Prevent Overflow, 588 7.7.3 Statistical Characterization of Q uantization Effects in Fixed-Point Realizations of Digital Filters 590 7.8 S u m m ary a n d R e fe re n c e s P ro b le m s 600 598 ix Contents 8.1 G e n e l C o n s id e tio n s 614 8.1.1 Causality and Its Implications 615 8.1.2 Characteristics of Practical Frequency-Selective Filters 619 8.2 D e sig n o f F IR F ilters 620 8.2.1 Symmetric and Antisym m eiric FIR Filters, 620 8.2.2 Design of Linear-Phase FIR Filters Using Windows, 623 8.2.3 Design of Linear-Phase FIR Filters by the Frequency-Sampling M ethod, 630 8.2.4 Design of Optimum Equiripple Linear-Phase FIR Filters, 637 8.2.5 Design of FIR Differentiators, 652 8.2.6 Design of Hilbert Transformers, 657 8.2.7 Comparison of Design M ethods for Linear-Phase FIR Filters, 662 8.3 D esig n o f I I R F ilters F ro m A n a lo g F iiters 666 8.3.1 IIR Filter Design by Approxim ation of Derivatives 667 8.3.2 IIR Filter Design by Impulse Invariance 671 8.3.3 IIR Filter Design by the Bilinear Transform ation, 676 8.3.4 The M atched-; Transform ation, 681 8.3.5 Characteristics of Commonly Used Analog Filters 681 8.3.6 Some Examples of Digital Filter Designs Based on the Bilinear Transform ation 692 8.4 F re q u e n c y T n s fo rm a tio n s 692 8.4.1 Frequency Transform ations in the Analog Dom ain, 693 8.4.2 Frequency Transform ations in the Digital Dom ain 698 8.5 D esig n o f D ig ital F ilters B a sed on L e a st-S q u a re s M e th o d 8.5.1 Pade Approxim ation Method, 701 8.5.2 Least-Squares Design Methods, 706 8.5.3 FIR Least-Squares Inverse (W iener) Filters, 711 8.5.4 Design of IIR Filters in the Frequency Dom ain, 719 8.6 S u m m ary an d R e fe re n c e s P ro b le m s 614 DESIGN OF DIGITAL FILTERS 701 724 726 SAMPLING AND RECONSTRUCTION OF SIGNALS 9.1 S am p lin g o f B a n d p a ss S ignals 738 9.1.1 R epresentation of Bandpass Signals, 738 9.1.2 Sampling of Bandpass Signals, 742 9.1.3 Discrete-Time Processing of Continuous-Time Signals, 746 9.2 A n a lo g -to -D ig ita l C o n v e rsio n 748 9.2.1 Sample-and-Hold 748 9.2.2 Quantization and Coding, 750 9.2.3 Analysis of Q uantization Errors, 753 9.2.4 Oversampling A /D Converters, 756 738 R6 References and Bibliography A J 1970 “The Design of Digital Filters,” A use Telecommun Res., Vol 4, pp 29-34, May G ib b s , 1968 “An Algorithm for the Evaluation of Finite Trigonometric Series,” A m Math Monthly, Vol 65, pp 34-35, January G o e rtz e l, G I., ed 1986.1 Schiir Methods in Operator Theory and Signal Processing, Birkhauser Verlag, Stuttgart, Germany G o h b e rg , B., and J o r d a n , K L„ J r 1986 “A Note on Digital Filter Synthesis,” Proc IE E E , Vol 56, pp 1717-1718, October G o ld , B., and - J o r d a n , K L., Jr 1969 “A Direct Search Procedure for Designing Finite Duration Impulse Response Filters,” IE E E Trans A u d io and Electroacoustics, Vol AU17, pp 33-36, March G o ld , B., and R a d e r , C M 1966 “Effects of Quantization Noise in Digital Filters.” Proc AF1PS 1966 Spring Joint Computer Conference, Vol , pp -2 G o ld , B., and G o ld , R a d e r, C M 1969 Digital Processing o f Signals, McGraw-Hill, New York R M., and K a i s e r , J F 1964 “Design of Wideband Sampled Data Filters,” Bell Syst Tech J., Vol 43, pp 1533-1546, July G o ld e n , I J 1971 “The Relationship Between Two Fast Fourier Transforms,” IE E E Trans Computers, Vol C-20, pp 310-317 G ood, G o r s k i-P o p ie l, J., ed 1975 Frequency Synthesis: Techniques and Applications, IEEE Press, New York D., K r a u s e , D J., and M o o r e , J B 1975 “Identification of Autoregressive-Moving Average Parameters of Time Series,” IE E E Trans Autom atic Control, Vol AC-20, pp 104-107, February G rau p e, A H., and M a r k e l , J D 1973 “Digital Lattice and Ladder Filter Synthesis,” IE E E Trans Acoustics, Speech, and Signal Processing, Vol ASSP-21, pp 491-500, December G ray , A H., and M a r k e l , J D 1976 “A Computer Program for Designing Digital Elliptic Filters,” IE E E Trans Acoustics, Speech, and Signal Processing, Vol ASSP-24, pp 529— 538, December G y , G ray , R M 1990 Source Coding Theory, Kluwer, Boston, MA O,, and S z e g o , G 1958 Teoplitz Forms and Their Applications, University of California Press, Berkeley, Calif G re n a n d e r, L J 1975 “Rapid Measurements of Digital Instantaneous Frequency,” IE E E Trans Acoustics, Speech, and Signal Processing, Vol ASSP-23, pp 207-222, April G r iff ith s , G u ille m in , G u p ta , E A 1957 Synthesis o f Passive Networks, Wiley, New York S C 1966 Transform and State Variable M ethods in Linear Systems, Wiley, New York H a m m in g , R W 1962 Numerical Methods fo r Scientists and Engineers, McGraw-Hill, New York H a y k in , S 1991 Adaptive Filter Theory, 2nd ed., Prentice Hall, Englewood Cliffs, N J and N i k i a s , C S 1985 “Improved Spectrum Performance via a Data-Adaptive Weighted Burg Technique,” IE E E Trans Acoustics, Speech, and Signal Processing, Vol ASSP-33, pp 903-910, August, H e l m e , B , References and Bibliography R7 D 1967 “Fast Fourier Transforms Method of Computing Difference Equations and Simulating Filters,” IE E E Trans A udio and Electroacoustics, Vol AU-15, pp 85-90 June H e lm s , H 1968 “Nonrecursive Digital Filters: Design Methods for Achieving Specifi cations on Frequency Response,” IE E E Trans A u d io and Electroacoustics, Vol AU-16, pp 336-342, September H e lm s , H D C W 1990 Probability and Stochastic Processes fo r Engineers, 2nd ed., Macmil lan, New York H e ls tro m , R W 1980 “The Cause of Line Splitting in Burg Maximum-Entropy Spectral Analysis,” IE E E Trans Acoustics, Speech, and Signal Processing, Vol ASSP-28, pp 692701, December H e r r in g , O 1970 “Design of Nonrecursive Digital Filters with Linear Phase,” Electron Lett., Vol , pp 328-329, November H erm an n , O., and S c h u e s s l e r , H W 1970a “Design of Nonrecursive Digital Filters with Minimum Phase,” Electron Lett., Vol , pp 329-330, November H e r r m a n n , O., and S c h u e s s l e r , H W 1970b “On the Accuracy Problem in the Design of Nonrecursive Digital Filters,” Arch Elek Ubertragung, Vol 24, pp 525-526 H e r r m a n n , O., R a b i n e r , L R , and C h a n , D S K (1973) “Practical Design Rules for Optimum Finite Impulse Response Lowpass Digital Filters,” Bell Syst Tech J., Vol 52, pp 769-799 July-August H erm an n , H ild e b n d , F B 1952 M ethods o f A pplied Mathematics, Prentice Hall, Englewood Cliffs, N.J E., O p p e n h e i m , A V., and S i e g e l , J 1971 “A New Technique for the Design of Nonrecursive Digital Filters,” Proc 5th A nnual Princeton Conference on Information Sciences and Systems, pp 64-72 H o f s te tte r , H o u s e h o l d e r , A S 1964 The Theory o f Matrices in Numerical Analysis, Blaisdell, Waltham, Mass S Y, 1977 “Minimum Uncorrelated Unit Noise in State Space Digital Filter ing.” IE E E Trans Acoustics, Speech, and Signal Processing, Vol ASSP-25, pp 273-281 August J a c k s o n , L B 1969 “An Analysis of Limit Cycles Due to Multiplication Rounding in Recursive Digital (Sub) Filters,” Proc 7th A nnual Allerton Conference on Circuit and System Theory, pp 69-78 H w ang, L B 1970a “On the Interaction of Roundoff Noise and Dynamic Range in Digital Filters,” Bell Syst Tech J., Vol 49, pp 159-184, February Jack so n , L B 1970b “Roundoff Noise Analysis for Fixed-Point Digital Filters Realized in Cascade or Parallel Form,” IE E E Trans A udio and Electroacoustics, Vol AU-18, pp 107-122, June J a c k s o n , L B 1976 “Roundoff Noise Bounds Derived from Coefficients Sensitivities in Digital Filters,” IE E E Trans Circuits and Systems, Vol CAS-23, pp 481-485, August J a c k s o n , L B 1979 “Limit Cycles on State-Space Structures for Digital Filters,” IE E E Trans Circuits and Systems, Vol CAS-26, pp 67-68, January Jackson, B„ L i n d g r e n , A G., and K jm , Y 1979 “Optimal Synthesis of Second-Order State-3pace Structures for Digital Filters,” IE E E Trans Circuits and Systems, Vol CAS-26, pp 149-153, March Ja c k s o n , L R8 References and Bibliography J a h n k e E a n d E m d e , F 1945 Tables o f Functions, 4th ed Dover, New York V K and C r o c h i e r e , R E 1984 “Quadrature Mirror Filter Design in the Time Domain," IE E E Trans Acoustics, Speech, and Signal Processing, Vol ASSP-32, pp 353361, April J a in G M., and San Francisco J e n k in s , W a tts , D G 1968 SpectraI Analysis and Its Applications, Holden-Day, D 1980 “A Filter Family Designed for Use in Quadrature Mirror Filter Banks,” IE E E International Conference on Acoustics, Speech, and Signal Processing, pp 291-294, April J o h n s t o n , J 1982 “The Application of Spectral Estimation Methods to Bearing Estima tion Problems,” Proc IE E E , Vol 70, pp 1018-1028, September Jo h n so n , D H R H 1976 “Autoregression Order Selection,” Geophysics, Vol 41, pp 771-773, August Jo n es, J u r y E I 1964 Theory and Applications o f the z-Transform M ethod, Wiley, New York T 1974 “A View of Three Decades of Linear Filter Theory." IE E E Trans Infor mation Theory, Vol IT-20, pp 146-181, March K a ila th , T 1981 Lectures on Wiener and Kalman Filtering, 2nd Printing, Springer-Verlag, New York K a ila th , T 1985 “Linear Estimation for Stationary and Near-Stationary Processes.” in M odem Signal Processing, T Kailath, ed Hemisphere Publishing Corp., Washington, D.C K a ila th T 1986 “A Theorem of I Schur and Its Impact on Modern Signal Processing," in Gohberg (1986) K a ila th , T V i e i r a , A C G, and M o r f , M 1978 “Inverses of Toeplitz Operators, Innova tions, and Orthogonal Polynomials.” S IA M Rev., Vol 20, pp 1006-1019 K a ila th , K a is e r J F 1963 “Design Methods for Sampled Data Filters." Proc First Allerton Con ference on Circuit System Theory, pp 2 -2 , November J F 1966 “Digital Filters,” in System Analysis by Digital Computer, F F Kuo and J F Kaiser, Eds., Wiley, New York K a is e r, N., and T h e o d o r i d i s , S 1987 “Fast Adaptive Least-Squares Algorithms for Power Spectral Estimation,” IE E E Trans Acoustics, Speech, and Signal Processing, Vol ASSP-35, pp 661-670, May K a lo u p ts id is , R E 1960 “A New Approach to Linear Filtering and Prediction Problems." Trans A SM E , J Basic Eng., Vol 82D, pp 35-45, March K a lm a n , R E., and B u c y , R S 1961 “New Results in Linear Filtering Theory,” Trans ASM E, J Basic Eng., Vol 83, pp 95-108 K a lm a n , R L 1980 “Inconsistency of the AIC Rule for Estimating the Order of Autore gressive Models,” IE E E Trans Autom atic Control, Vol AC-25, pp 996-998, October K ashyap J., and B r u z z o n e , S P 1979 “Order Determination for Autoregressive Spectral Es timation." Record o f the 1979 R A D C Spectral Estimation W orkshop, pp 139-145, Griffin Air Force Base, Rome, N.Y K aveh, M., and L i p p e r t , G A 1983 “An Optimum Tapered Burg Algorithm for Linear Prediction and Spectral Analysis,” IE E E Trans Acoustics, Speech, and Signal Processing, Vol ASSP-31, pp 438-444, April K aveh, References and Bibliography R9 S M 1980 “A New ARMA Spectral Estimator,” IE E E Trans Acoustics, Speech, and Signal Processing, Vol ASSP-28, pp 585-588, October K ay, K ay, S M 1988 M odern Spectral Estimation, Prentice Hall, Englewood Cliffs, N J K ay , S, M , and M a r p l e , S L , J r 1979 “Sources of and Remedies for Spectral Line Splitting in Autoregressive Spectrum Analysis,” Proc 1979 IC A SSP , pp -1 S M , and M a r p l e , S L., J r 1981 “Spectrum Analysis: A Modern Perspective,” Proc IE E E , Vol 69, pp 1380-1419, November K ay, K e s le r, S B., ed 1986 M odem Spectrum Analysis II, IEEE Press, New York B., and O l c a y t o , E M 1968 “Coefficient Accuracy and Digital Filter R e sponse,” IE E E Trans Circuit Theory, Vol CT-15, pp 31-41, March K n o w le s , J H 1988 “New Split Levinson, Schur, and Lattice Algorithms for Digital Signal Processing,” Proc 1988 International Conference on Acoustics, Speech, and Signal Pro cessing, pp 1640-1642, New York, April K rish n a , R E 1969 “Asymptotic Properties of the Autoregressive Spectral Estimator,” Ph.D dissertation, Department of Statistics, Stanford University, Stanford, Calif K ro m er, and Hu, Y H 1983 “A Highly Concurrent Algorithm and Pipelined Architec ture for Solving Toeplitz Systems,” IE E E Trans Acoustics, Speech, and Signal Processing, Vol ASSP-31, pp 66-76, January K u n g , S Y „ S Y., W h i t e h o u s e , H J , and K a i l a t h , T., eds 1985 V L S I and Modern Signal Processing, Prentice Hall, Englewood Cliffs, N.J K ung, R T 1971 “Data Adaptive Spectral Analysis Methods,” Geophysics, Vol 36, pp 661-675, August L aco ss, S W , and M c C l e l l a n , J H 1980 “Frequency Estimation with Maximum Entropy Spectral Estimators,” IE E E Trans Acoustics, Speech, and Signal Processing, Vol ASSP28, pp 716-724, December L ang, 1947 “The Wiener RMS Error Criterion in Filter Design and Prediction,” J Math Phys., Vol 25, pp 261-278 L e v in so n , N L evy, H„ and L essm an, F 1961 Finite Difference Equations, Macmillan, New York B 1971 “Effect of Finite Word Length on the Accuracy of Digital Filters— A Review,” IE E E Trans Circuit Theory, Vol CT-18, pp 670-677, November L iu , J 1975 “Linear Prediction: A Tutorial Review,” Proc IE E E , Vol 63, pp 561— 580, April M a k h o u l, 1978 “A Class of All-Zero Lattice Digital Filters: Properties and Applica tions,” IE E E Trans Acoustics, Speech, and Signal Processing, Vol ASSP-26, pp 304-314, August M arkel, J D., and G r a y , A H., Jr 1976 Linear Prediction o f Speech, Springer-Verlag, New York M a k h o u l, J S L , J r 1980 “A New Autoregressive Spectrum Analysis Algorithm.” IE E E Trans Acoustics, Speech, and Signal Processing, Vol ASSP-28, pp 441-454, August M a rp le , S L., J r 1987 Digital Spectral Analysis with Applications, Prentice Hall, Englewood Cliffs, N J a r z e t t a , T L 1983 “A New Interpretation for Capon’s Maximum Likelihood Method of Frequency-Wavenumber Spectral Estimation,” IE E E Trans Acoustics, Speech, and Signal Processing, Vol ASSP-31, pp 445-449, April M a rp le , M R10 References and Bibliography T L., and Lang, S W 1983 “New Interpretations for the MLM and DASE Spectral Estimators,” Proc 1983 IC A SSP , pp 844-846, Boston, April M a rz e tta , T L , and L a n g , S W 1984 “Power Spectral Density Bounds.” IE E E Trans Information Theory, Vol IT-30, pp 117-122, January M a rz e tta , S J., and New York M aso n , Z im m e rm a n , H J 1960 Electronic Circuits, Signals and Systems, Wiley, J H 1982 “Multidimensional Spectral Estimation,” Proc IE E E , Vol 70, pp 1029-1039, September M c C le lla n , R N 1983 “Application of the Maximum-Likelihood Method and the Maxi mum Entropy Method to Array Processing,” in Nonlinear M ethods o f Spectral Analysis, 2nd ed., S Haykin, ed., Springer-Verlag, New York M cD onough, D., and C o o p e r , G R 1984 Continuous and Discrete Signal and System Analysis, 2nd ed Holt Rinehart and Winston, New York M c G ille m , C M e d itc h , J E 1969 Stochastic Optimal Linear Estimation and Control McGraw-Hill, New York L., M u l l i s , C T , and R o b e r t s , R A 1981 “Low Roundoff Noise and Normal Realizations of Fixed-Point IIR Digital Filters.” IE E E Trans Acoustics, Speech, and Signal processing, Vol ASSP-29, pp 893-903, August M ills , W A., and L e e , D T 1977 “Ladder Forms for Identification and Speech Processing,” Proc 1977 IE E E Conference Decision and Control, pp 1074-1078, New Orleans, La., December M o r f , M „ V ie ir a , C T , and R o b e r t s , R A 1976a “Synthesis of Minimum Roundoff Noise Fixed* Point Digital Filters,” IE E E Trans Circuits and Systems, Vol CAS-23, pp 551-561, September M u llis , C T „ and R o b e r t s , R A 1976b “Roundoff Noise in Digital Filters: Frequency Transformations and Invariants,” IE E E Trans Acoustics, Speech and Signal Processing, Vol ASSP-24, pp 538-549, December M u llis , 1985 “Fast M L M Power Spectrum Estimation from Uniformly Spaced Corre lations,” IE E E Trans Acoustics, Speech, and Signal Proc., Vol ASSP-33, pp 1333-1335, October M u s ic u s , B W I 1981 “Extension to the Maximum Entropy Method III,” Proc 1st A SSP Workshop on Spectral Estimation, pp 1.7.1—1.7.6, Hamilton, Ontario, Canada, August N ew m an, and R a g h u v e e r , M R 1987 “Bispectrum Estimation: cessing Framework,” Proc IE E E , Vol 75, pp 869-891, July N i k i a s , C L , A Digital Signal Pro C L„ and S c o t t , P D 1982 “Energy-Weighted Linear Predictive Spectral Es timation: A New Method Combining Robustness and High Resolution.” IE E E Trans Acoustics, Speech, and Signal Processing, Vol ASSP-30, pp 287-292, April N ik ia s , A H 1976 “Spectral Analysis of a Univariate Process with Bad Data Points, via Maximum Entropy and Linear Predictive Techniques,” N U SC Technical Report TR-5303, New London, Conn., March N u tta ll, H 1928 “Certain Topics in Telegraph Transmission Theory,” Trans A IE E , Vol 47, pp 617-644, April N y q u is t, A V 1978 Applications o f Digital Signal Processing, Prentice Hall, Englewood Cliffs, N J O p p e n h e im , References and Bibliography R11 A V., and S c h a f e r , R W 1989 Discrete-Time Signal Processing, Prentice Hall, Englewood Cliffs, N J O p p e n h e im , O p p e n h e im , A V., and W e in s te in , C W 1972 “Effects of Finite Register Length in Digital Filters and the Fast Fourier Transform,” Proc IE E E , Vol 60, pp 957-976 August O p p e n h e im , A V , and W i l l s k y , A S 1983 Signals and Systems, Prentice Hall, Englewood Cliffs, N J P a p o u lis , A 1962 The Fourier Integral and Its Applications, McGraw-Hall, New York A 1984 Probability, R andom Variables, and Stochastic Processes, 2nd ed., McGraw-Hill, New York P a p o u lis , P a r k e r , S R., and H ess, S F 1971 “Limit-Cycle Oscillations in Digital Filters,” IE E E Trans Circuit Theory, Vol CT-18, pp 687-696, November P a r k s , T W., and M c C l e l l a n , J H 1972a “Chebyshev-Approximation for Nonrecursive Digital Filters with Linear P h a s e ,” IE E E Trans Circuit Theory, Vol C T -1 , pp -1 , M a rc h T W., and M c C l e l l a n , J H 1972b “A Program for the Design of Linear Phase Finite Impulse Response Digital Filters,” IE E E Trans A udio and Electroacoustics, Vol AU-20, pp 195-199, August P a rk s, E 1957 “On Consistent Estimates of the Spectrum of a Stationary Time Series,” A m Math Stat., Vol 28, pp 329-348 P a rze n , E 1974 “Some Recent Advances in Time Series Modeling,” IE E E Trans A u to matic Control, Vol AC-19, pp 723-730, December P a rze n , K L, and T r e i t e l , S 1969 “Predictive Deconvolution-Theory and Practice,” Geo physics, Vol 34, pp 155-169 P eacock, Z., J r 1987 Probability, Random Variables, and Random Signal Principles, 2nd ed., McGraw-Hill, New York P e e b le s , P V F 1973 “The Retrieval of Harmonics from a Convariance Function,” Geophys J R Astron Soc., Vol 33, pp 347-366 P isa re n k o , 1981 Autoregressive M ethods o f Spectral Analysis, E.E degree thesis, Depart ment of Electrical and Computer Engineering, Northeastern University, Boston, May P o o le , M A P r ic e , R 1990 “Mirror FFT and Phase FFT Algorithms,” unpublished work, Raytheon Research Division, May P r o a k is , J G 1995 Digital Communications, 3rd ed., McGraw-Hill, New York R a b i n e r , L R., and S c h a e f e r , R W 1974a “On the Behavior of Minimax Relative Error FIR Digital Differentiators,” Bell Syst Tech J., Vol 53, pp 333-362, February R a b i n e r , L R , and S c h a f e r , R W 1974b “On th e Behavior of Minimax F I R Digital Hilbert Transformers,” Bell SysL Tech J., Vol 53, pp 363-394, February R a b i n e r , L R , and S c h a f e r , R W 1978 Digital Processing o f Speech Signals, Prentice Hall, Englewood Cliffs, N J R a b i n e r , L R., S c h a f e r , R W , and R a d e r , C M 1969 “The Chirp ^-Transform Algorithm and Its Applications,” Bell S yst Tech J., Vol 48, pp 1249-1292, May-June R a b i n e r , L R , G o l d , B., a n d M c G o n e g a l, C A 1970 “An Approach to the Approxima tion Problem for Nonrecursive Digital Filters,” IE E E Trans A u d io and Electroacoustics, Vol AU-18, pp 83-106, June R12 References and Bibliography W 1975 “ F I R Digital Filter Design Tech niques Using Weighted Chebyshev Approximation,” Proc IE E E , Vol 63, pp 595-610, April R a b i n e r , L R , M c C l e l l a n , J H , a n d P a r k s , T 1970 An Improved Algorithm f o r High-Speed Auto-correlation with Appli cations to Spectral Estimation,” IE E E Trans A udio and Electroacoustics, Vol AU-18, pp 439-441, December R a d e r, C M C M., and B r e n n e r , N M 1976 “A New Principle for Fast Fourier Transformation,” IE E E Trans Acoustics, Speech and Signal Processing, Vol ASSP-24, pp 264-266, June R a d e r, C M., and G o l d , B 1967a “Digital Filter Design Techniques in the Frequency Domain,” Proc IE E E , Vol 55, pp 149-171, February R ad er, C M., and G o l d , B 1967b “Effects of Parameter Quantization on the Poles of a Digital Filter,” Proc IE E E , Vol 55, pp 688-689, May R ad er, 1984 “Digital Methods for Conversion Between Arbitrary Sampling Fre quencies,” IE E E Trans Acoustics, Speech, and Signal Processing, Vol ASSP-32, pp 577591, June R a m s ta d , T A 1957 General Computational M ethods o f Chebyshev A pproxim ation, Atomic Energy Translation 4491, Kiev, USSR R e m e z, E Y a J 1983 “A Universal Prior for the Integers and Estimation by Minimum D e scription Length,” Ann Stat., Vol 11, pp 417-431 R js s a n e n R o b e r t s , R A , and M u llis , C T, 1987 Digital Signal Processing, Addison-Wesley, Reading, Mass R o b in s o n , E A 1962 R andom Wavelets and Cybernetic Systems, Charles Griffin, London E A 1982 “A Historical Perspective of Spectrum Estimation,” Proc IE E E , Vol 70, pp 885-907, September R o b in so n , E A., and T r e i t e l , S 1978 “Digital Signal Processing in Geophysics,” in A p plications o f Digital Signal Processing, A V Oppenheim, ed., Prentice Hall, Englewood Cliffs, N.J R o b in s o n , and wood Cliffs, N.J R o b i n s o n , E A , T r e i t e l , S 1980 Geophysical Signal Analysis, Prentice Hall, Engle A., and K a i l a t h , T 1986 “ESPRIT: A Subspace Rotation Approach to Estimation of Parameters of Cisoids in Noise,” IE E E Trans Acoustics, Speech, and Signal Processing, Vol ASSP-34, pp 1340-1342, October R o y , R , P a u l r a j , R J , M a c K a y , K J a y a n t , N W., and K im , T 1988 “Image Coding Based on Selective Quantization of the Reconstruction Noise in the Dominant Sub-band,” Proc 1988 IE E E International Conference on Acoustics, Speech, and Signal Processing, pp 765768, April S a fra n e k , H 1979 “Statistical Properties of AR Spectral Analysis,” IE E E Trans Acoustics, Speech, and Signal Processing, Vol ASSP-27, pp 402-409, August S a k a i, I W , and K a i s e r , J F 1972 “A Bound on Limit Cycles in Fixed-Point Im plementations of Digital Filters,” IE E E Trans A udio and Electroacoustics, Vol AU-20, pp 110-112, June S a n d b e rg , E H,, and A l e x a n d e r J T 1978 “High Resolution Spectral Analysis of Sinusoids in Correlated Noise,” Proc 1978 IC A SSP, pp 349-351, Tulsa, Okla., April 10-12 S a to riu s , References and Bibliography R13 W., and R a b i n e r , L R 1973 “A Digital Signal Processing Approach to Inter polation,” Proc IE E E , Vol 61, pp 692-702, June S c h a f e r , R H., and G o c k l e r , H 1981 “A Comprehensive Survey of Digital Transmul tiplexing Methods,” Proc IE E E , Vol 69, pp 1419-1450 S c h e u e rm a n n , R D 1981 “A Signal Subspace Approach to Multiple Emitter Location and Spectral Estimation,” Ph.D dissertation, Department of Electrical Engineering, Stanford University, Stanford, Calif., November S c h m id t, R D 1986 “Multiple Emitter Location and Signal Parameter Estimation,” IE E E Trans A ntennas and Propagation, Vol AP 34, pp 276-280, March S c h m id t, J P., and M c C l e l l a n , J H 1984 “Maximum Entropy Power Spectrum Estimation with Uncertainty in Correlation Measurements,” IE E E Trans Acoustics, Speech, and Signal Processing, Vol ASSP-32, pp 410-418, April S c h o tt, I 1917 “On Power Series Which Are Bounded in the Interior of the Unit Circle,” J Reine Angew Math., Vol 147, pp 205-232, Berlin For an English translation of the paper, see Gohberg 1986 S ch u r, 1898 “On the Investigation of Hidden Periodicities with Application to a Supposed Twenty-Six-Day Period of Meteorological Phenomena,” Terr Mag Vol 3, pp 13-41, March S c h u s te r , S ir A r t h u r and F e t t w e i s A 1973 “Digital Filters with True Ladder Configuration.” Int J Circuit Theory A ppL, Vol 1, pp 5-10 March S e d l m e y e r , A , S h a n k s , J L 1967 “Recursion Filters for Digital Processing," Geophysics, Vol 32, p p 3 , February Shannon, C E 1949 “Communication in the Presence of Noise,” Proc IRE , pp 10-21, January D H., ed 1986 Analog-Digital Conversion H andbook, Prentice Hall, Englewood Cliffs, N.J S h e in g o ld , S ie b e rt, W M 1986 Circuits, Signals and Systems, McGraw-Hill, New York R C 1967 “A Method for Computing the Fast Fourier Transform with Auxil iary Memory and Limit High speed Storage.” IE E E Trans A udio and Electroacoustics, Vol AU-15, pp 91-98, June S in g le to n , R C 1969 “An Algorithm for Computing the Mixed Radix Fast Fourier Trans form,” IE E E Trans A u dio and Electroacoustics, Vol AU-17, pp 93-103, June S in g le to n , M J T., and B a r w e l l , T P 1984 “A Procedure for Designing Exact Reconstruc tion Filter Banks for Tree Structured Subband Coders,” Proc 1984 IE E E International Conference on Acoustics, Speech, and Signal Processing, pp 27.1.1-27.1.4, San Diego, March S m ith , M J T., and E d d i n s , S L 1988 “Subband Coding of Images with Octave Band Tree Structures,” Proc 1987 IE E E International Conference on Acoustics, Speech, and Signal Processing, pp 1382-1385, Dallas, April S m ith , K 1965 “The Equivalence of Digital and Analog Signal Processing,” Inf Control, Vol , pp 455-467, October S te ig litz , K 1970 “Computer-Aided Design of Recursive Digital Filters,” IE E E Trans A u d io and Electroacoustics, Vol AU-18, pp 123-129, June S te ig litz , R14 References and Bibliography T G 1966 “High Speed Convolution and Correlation,” 1966 Spring Joint Com puter Conference, A F IP S Proc., Vol 28, pp 229-233 S to c k h a m , S to re r J E 1957 Passive Network Synthesis, McGraw-Hill, New York S w a r z tr a u b e r P 1986 “Symmetric FFT’s,” Mathematics o f Computation, Vol 47, pp 323- 346 July D N 1979a “A Comparison Between Burg’s Maximum Entropy Method and a Nonrecursive Technique for the Spectral Analysis of Deterministic Signals,” J Geophys Res Vol 84, pp 679-685, February S w in g le r, D N 1979b “A Modified Burg Algorithm for Maximum Entropy Spectral Anal ysis,” Proc IE E E , Vol 67, pp 1368-1369, September S w in g le r, D N 1980 “Frequency Errors in MEM Processing,” IE E E Tram Acoustics, Speech, and Signal Processing, Vol ASSP-28, pp 257-259, April S w in g le r, G 1967 Orthogonal Polynomials, 3rd ed., Colloquium Publishers, no 23, American Mathematical Society, Providence, R.l Szego, 1981 “A Comparison of the Least-Squares Method and the Burg Method for Autoregressive Spectral Analysis,” IE E E Trans A ntennas and Propagation Vol AP29, pp 675-679, July T h o rv a ld s e n , T H 1975 “Autoregressive Model Fitting with Noisy Data by Akaike's Information Criterion," IE E E Trans Information Theory, Vol IT-21, pp - , July Tong, H 1977 “More on Autoregressive Model Fitting with Noise Data b y Akaike's Infor mation Criterion," IE E E Trans Information Theory, Vol IT-23, pp 409-410 May Tong, T re tte r S A 1976 Introduction to Discrete-Time Signal Processing Wiley, New York D W and K u m a r e s a n , R 1982 “Estimation of Frequencies of Multiple Sinusoids: Making Linear Prediction Perform Like Maximum Likelihood,” Proc IE E E , Vol 70, pp 975-989 September T u fts , T J., and B i s h o p , T N 1975 “Maximum Entropy Spectral Analysis and Autore gressive Decomposition,7' Rev Geophys Space Phys., Vol 13, pp 183-200, February U lr y c h T J„ and C l a y t o n , R W 1976 “Time Series Modeling and Maximum Entropy,” Phys Earth Planet Inter., Vol 12, pp 188-200, August U lry c h , P P 1990 “Multirate Digital Filters, Filter Banks, Polyphase Networks, and Applications: A Tutorial,” Proc IE E E , Vol 78, pp 56-93, January V a id y a n a th a n , V a id y a n a th a n , P P 1993 Multirate Systems and Filter B anks, Prentice Hall, Englewood Cliffs, N.J J 1984 “Multi-dimensional Sub-band Coding: Some Theory and Algorithms,” Signal Processing, Vol , pp 97-112, April V e tte r li, J 1987 “A Theory of Multirate Filter Banks,” IE E E Trans Acoustics, Speech, and Signal Processing, Vol ASSP-35, pp 356-372, March V e tte rli, A C G 1977 “Matrix Orthogonal Polynomials with Applications to Autoregressive Modeling and Ladder Forms,” Ph.D dissertation, Department of Electrical Engineering, Stanford University, Stanford, Calif., December V ie ir a , G 1931 “On Periodicity in Series of Related Terms,” Proc R Soc., Ser A Vol 313, pp 518-532 W a lk e r , M., and K a i l a t h , T 1985 “Detection of Signals by Information Theoretic Criteria.” IE E E Trans Acoustics, Speech, and Signal Processing, Vol ASSP-32, pp 387-392, April W ax, References and Bibliography W e i n b e r g , L 1962 R15 Network Analysis and Synthesis, M cG raw-H ill, N ew York P D 1967 “The Use of Fast Fourier Transform for the Estimation of Power Spectra: A Method Based on Time Averaging over Short Modified Periodograms,” IE E E Trans A u d io and Electroacoustics, Vol AU-15, pp 70-73, June i e n e r , N 1949 Extrapolation, Interpolation and Sm oothing o f Stationary Time Series with Engineering Applications, Wiley, New York W e lc h , W N., and P a l e y , R E A C 1934 Fourier Transforms in the Complex Domain American Mathematical Society, Providence, R.I W i n o g r a d , S 1976 "On Computing the Direte Fourier Transform,” Proc Natl Acad S e t, Vol 73, pp 105-106 W i n o g r a d , S 1978 “On Computing the Discrete Fourier Transform,” Math Comp., Vol 32, pp 7 -1 9 W o l d , H 1938 A Study in the Analysis o f Stationary Time Series, reprinted by Almqvist & Wiksell, Stockholm, 1954 W o o d , L C., and T r e i t e l , S 1975 “Seismic Signal Processing,” Proc IE E E , Vol 63, pp 649661, April W o o d s , J W , and O ’ N e i l , S D 1986 “Subband Coding of Images,” IE E E Trans Acoustics, Speech, and Signal Processing, Vol ASSP-34, pp 1278-1288, October Y o u l a , D„ and K a z a n j i a n , N 1978 “Bauer-Type Factorization of Positive Matrices and the Theory of Matrices Orthogonal on the Unit Circle,” IE E E Trans Circuits and Systems, Vol CAS-25, pp 57-69, January Y ule, G U 1927 “On a Method of Investigating Periodicities in Disturbed Series with Special References to Wolfer’s Sunspot Numbers,” Philos Trans R Soc London, Ser A Vol 226, pp 267-298, July Z a d e h , L A., and D e s o e r , C A 1963 Linear System Theory: The State-Space Approach, McGraw-Hill, New York Z v e r e v , A I 1967 H andbook o f Filter Synthesis, Wiley, New York W ie n e r , Accumulator, 58 Akaike information criterion (AIC), 932 Algorithms, Chiip-z, 482-486 FFT, 448-475 Goertzel, -4 Remez, 645-647 Aliasing frequency-domain, 22, 276-279 time-domain 276 Alternation theorem, 643 Amplitude, 16 Analog-to-digital (A/D) converter, 5, 21, 748-762 oversampling, 756-762 Analog signals (see Signals) Antialiasing filter, 746-747 Autocorrelation of deterministic signals, 118-133 of random signals, 327-329, A3 Autocovariance, A4 Autoregressive-moving average (ARMA) process, 855, 922 autocorrelation of, 856 Autoregressive (AR) process, 855, 922 autocorrelation of, 856 Averages autocorrelation, A3 autocovariance A4 ensemble, A3-A5 for discrete-time signals, A6-A7 expected value, A3 moments, A3 power, A3 time-averages, A8-A10 Backward predictor, 515 860 Bandlimited signals, 281 Bandpass filter, 331 337-338 Bandpass signal, 280, 738-742 Bandwidth, 279-282 619 Bartlett's method (see Power spectrum estimation) Bessel filter, 690-691 Bibliography, R1-R15 Bilinear transformation 676-680 Binary codes, 752 11 Blackman-Tukey method (see Power spectrum estimation) Burg algorithm (see Power spectrum estimation) Butterworth filters, 682-683 Canonic form 114 Capon method ( s e e Power spectrum estimation) Carrier frequency 739-740 Cauchy integral theorem 160 Causal signals, 87 Causal systems, 68-69, 86-87, 208-209 Causality, implications of, 615-618 Cepstral coefficients 854 Cepstrum, 265-266 Characteristic polynomial, 101 547 Chebyshev filters, 683-689 Chirp signal, 484 Chirp-z transform algorithm 482-486 Circular convolution, 415-420 Coding, 22-38 Comb filter, 345-349 Complex envelope, 740 Constant-coefficient difference equations, 95-111 solution of, 100-111 Continuous-time signals exponentials 19-20 sampling of, 1-24, 269-279 sampling theorem for, 29-31, 269-279 Convolution (linear), 75-82 circular, 415-420 properties, 82-85 sum, 75 Correlation, 1J 8—133 A3 autocorrelation, 122, 325-330, A3 computation, 130-131 cross-correlation, 120, 325-327 of periodic signals 124, 130 properties, 122—124 Coupled-form oscillator, 352-354 Cross-power density spectrum, A6 Dead band, 584 Decimation, 784-787 See also Sampling rate conversion Deconvolution 266, 355-359, 363-365 homomorphic 266 365-367 Delta modulation, 758 Difference equations - 11 constant coefficient, 100-111 solution, 100-111 homogeneous solution 100-103 particular solution 103-104 for recursive systems 95 I ll total solution 105-111 from one-sided ;-translomi 201-202 Differentiator, 652 design of, 652-657 Digital resonator 340 Digital sinusoidal oscillator 353-354 Digital-to-analog (D/A) converter 5, 22, -7 oversampling 774 Dirichlet conditions for Fourier series, 234 for Fourier transform 243 Discrete Fourier transform (DFT) 399-402 computation, 49-473 butterfly, 460, 464, 466 decimination-in-frequency FFT algorithm, 461—464 decimination-in-time FFT algorithm, 456-461 direct, 449—450 divide-and-conquer method, 450-473 in-place computations, 461 radix-2 FFT algorithms, 456-464 radix-4 FFT algorithms, 465—469 shuffling of data, 461 split radix 70 -47 via linear filtering 479-486 definition, 401 IDFT, 401 implementation o f FFT algorithm, 473-479 properties, 09 -42 circular convolution, 415-420 circular correlation, 423 circular frequency shift, 422 circular time shift, 421-422 complex conjugate, 423 linearity, 410 multiplication, 15 -42 12 Index Parseval's theorem, 424 periodicity, 410 symmetry, 413—415 table 415 time reversal 421 relationship to Fourier series 407 409 relationship to Fourier transform, 407 relationship to £-transform, 408 use in frequency analysis, 433 -44 use in linear filtering, 425-433 Discrete-time signals, 9, 43-55 antisymmetric (odd), correlation, IS—133 definition, 9, 43 exponential, 46-47 frequency analysis of 247-264 nonperiodic, 50 periodic 14-19 random, 12 representation of, 44 sinusoidal, 16-18 symmetric (even), 51 unit ramp, 45 unit sample, 45 unit step, 45 Discrete-time systems, 56-71 causal, 8-6 9, 86-87 dynamic, 62 finite-duration impulse response 90-91 finite memory, 62, 90-91 implementation of, 500-556 infinite-duration impulse response 90-91 infinite memory, 62, 90-91 linear, 65 memoryless, 60 noncausal, 8-69 nonlinear, 67 nonrecursive, 94-95 recursive, 92-93 relaxed, 59 representation, 44 shift-invariant, 63-65 stability test for, 213-219 stability triangle, 216 stable (BIBO), 69-70, 87-90 static, 62 time-invariant, 63-65 unit sample (impulse) response, 76-82 unstable, 9-70, 87-90 Distortion amplitude, 317 delay 332 harmonic, 378 phase, 317 Down sampling, 54, 55 See also Sampling rate conversion Dynamic range, 35, 561, 751 Eigenfunction, 307 Eigenvalue 307, 547 Eigenvector, 547 Elliptic fitters, 689-690 Emigy definition, 47 density spectrum, 243-246, 260-264 partial 391 signal 47-49 Energy density spectrum, 243-246, 260-264 computation 897-902 Ensemble, AI averages, A3-A3 Envelope, 740-741 complex, 740 Envelope delay, 332 Ergodic, A8 correlation-ergodic, A9-A10 mean-ergodic, A8-A9 Estimate (properties) asymptotic bias, 904 asymptotic variance, 904 bias, 904 consistent, 904 variance, 904 See also Power spectrum estimation Fast Fourier transform (FFT) algorithms, 448-475 application to correlation, 77-479 efficient computation of DFT, 448-475 linear filtering, 477-479 implementation 473-475 minor FFT, 473 phase FFT, 473 radix-2 algorithm 456-464 decimation-in-frequcncy 461-464 decimation-in-time 456-461 radix-4 algorithm, 465-469 split-radix, 470-473 Fibonacci sequence, 201, 548-549, 553 difference equation, 201-202 state-space form, 548-549, 553 Filter bandpass, 331, 337-338 definition, 317, 330-332 design of linear-phase FIR, 620-665 transition coefficient for, C1-C5 design of HR filters, 330-354, 666-692 all pass, 350-352 comb, 345-349 notch, 343-344 by pole-zero placement, 333-354 resonators (digital), 340-343 distortion 317 distortionless, 332 frequency-selective, 331 highpass, 331 333-334 ideal 331-332 least squares inverse, 711-718 lowpass, 331, 333-334 nonideal, 332-333 passband ripple, 619 stopband ripie, 619 transition band, 619 prediction error filter, 512 858 smoothing, 39 structures, 500-556 Wiener filter, 715, 880-890 n h e r banks, 825-831 critically sampled, 829 quadrature minor, 833-841 uniform DFT, 826-829 Filter transformations, 338-340, 692-700 analog domain 693-698 digital domain, 698 -70 lowpass-lo-highpass, 338-340 Filtering of long data sequences, 430-433 overlap-add method for, 430-432 overlap-save method for, 430-431 via DFT, 430-433 Final prediction error (FPE) criterion, 931-932 Final value theorem, 200 FIR filters antisymmetric, 620-622 design, 620-665 comparison of methods, 662-665 differentiators, 652-657 equiripple (Chebyshev) approximation, 637-661 frequency sampling method, 630-637 Hilbert transformers, 657-662 window method, 623-630 linear phase property, 621-623 symmetric, 62(W>22 FIR filter structures, 502-519 cascade form, 504-505 direct form, 503-504 conversion to lattice form, 518-519 frequency sampling form, 506-510 lattice form, 511-561, 877 conversion to direct form, 516-517 transposed form, 25-526 FIR systems, 90, 94, 115 First-order hold, 768-769 Fixed-point representation, 557-560 Floating-point representation, 561-564 Flowgraphs, 521-524 Folding frequency, 28, 274-275 Forced response, 96-97 Forward predictor, 515, 857-858 Fourier series, 20, 232-240, 247-250 coefficients of, 234-235, 247-248 for continuous-time periodic signals, 232-240 for discrete-time periodic signals, 247-250 Fourier transform, 240-243, 253-256 of continuous-time aperiodic signals, 240-243 convergence of, 256-259 of discrete-time aperiodic signals, 253-256 inverse, 242 properties convolution, 297-298 correlation, 298 differentiation, 303-304 frequency shifting, 300 linearity, 294-295 modulation, 300-302 multiplication, 302-303 Paraeval’s theorem, 302 Index 13 Fourier transform (continued) symmetry, 287-294 table, 304 time-reversal, 297 time-shifting, 296 relationship to ;-transform, 264-265 of signals with poles on unit circle, 267-268 Frequency, 14-18 alias, 18, 26 content, 29 folding, 28, 274-275 fundamental range, 19 highest 19 negative 15 normalized, 24 positive, 15 relative, 24 Frequency analysis continuous-time aperiodic signals, 240-243 continuous-time periodic signals, 232-240 discrete-time aperiodic signals, 253-256 discrete-time periodic signals, 247-250 dualities, 282-286 for LTI systems, 305-330 table of formulas for, 285 Frequency response 311 computation 321—325 to exponentials, 306-314 geometric interpretation of, 321-325 magnitude of, 311 phase of 311 relation to system function, 319-321 lo sinusoids, 311-314 Frequency transformations (see Filter transformations) Fundamental period, 17 Gibbs phenomenon, 259, 629 Goertzel algorithm, 480-481 Granular noise, 753 Group (envelope) delay, 332 Harmonic distortion 378, 779-780 High-frequency signal 280 Hilbert transform, 618 Hilbert transformer, 657-662, 739 Homomorphic deconvolution, 365-367 system, 366 HR filters design from analog filters, 666-692 by approximation of derivatives, 667-671 by bilinear transformation, 676-680, 692 by impulse invariance, 671-676 by maiched-z transformation, 681 least-squares design methods, 706-724 frequency domain optimization, 719-724 least squares inverse, 711— 718 Prony’s (least squares), 706-708 Shanks' (least squares), 709-710 Padi approximation 701-705 pole-zero placement, 333-354 IIR filter structures, 519-556 cascade form, 526-528 direct form, 519-521 lattice-ladder, 531-539, 78-880 parallel fonn, 529-531 second-order modules, 527 state-space forms, 539-556 transposed forms 521-526 Impulse response 108-110 Initial value theorems, 172 Innovations process, 852-854 Interpolation, 30-31, 273, 784, 787-790 first-order hold, 768-771 function, 30, 763 ideal, 0-31 273 linear, 38, 768-774 See also Sampling-rate conversion Inverse filter, 355-356 Inverse Fourier transform, 242, 256 Inverse system, 355, 357 Inverse ’-transform, 160-172, 184-197 by contour integration, 160-161, 184-186 integral formula, 160 partial fraction expansion, 188-197 power series, 186-188 Lattice filters, 511-516, 531-539, 859 876-880 AR structures, 877-878 ARMA structure, 878-880 MA structure, 857-859 Leakage, 434, 899 Least squares filter design, 706-724 inverse filter, 711-718 Levinson-Durbin algorithm, 716, 865-868 generalized, 868 893 split Levinson 891 Limit cycle oscillations, 583-587 Linear filtering based on DFT, 425-433 overlap-add method, 430-432 overlap-save method, 430-431 Linear interpolation, 768-774 Linear prediction 512, 857-876 backward, 860-863 forward 857-860 lattice filter for, 859-860 normal equations for, 864 properties of, 873-876 maximum-phase, 843-844 minimum-phase, 842-843 orthogonality 844 whitening, 844 Linear prediction filter (see Linear prediction) LTI systems moving average, 115, 117 second order, 115-116 structures 111-118 canonic form, 114 direct form I 111-112 direct form II, 113-114 nonrecursive, 116-118 recursive, 116-118 weighted moving average, 115 Low-frequency signal, 279 Lowpass filter, 331 Maximal ripple filters 644 Maximum entropy method 928 Maximum-phase system, 361-362 Mean square estimation, 882 orthogonality principle, 884-886 Minimum description length (MDL), 932 Minimum-phase system, 359-362 Minimum variance estimate, 942-946 Mixed-phase system, 361-362 Moving-average filter, 309 Moving-average (M A) process, 855, 922 autocorrelation of, 857 Moving-average signal, 115 Multichannel signal Multidimensional signal, -8 Narrowband signal 281 Natural response 97 Natural signals 282-283 Noise subspace, 951 Noise whitening filter, 854 Normal equations, 864 solution of, 864-873 Levinson-Durbin algorithm, 865-868 Schur algorithm 868-872 Number representation, 556-564 fixed-point 557-561 floating point, 61-564 Nyquist rate, 30 One’s complement 558 One-sided z-transform, 197-202 Orthogonality principle, 884-885 Oscillators (sinusoidal generators) CORDIC algorithm for, 354 coupled-form, 353-354 digital 352 Overflow, 588-589 Overtap-add method, 430 -43 Overlap-save method, 430-431 Overload noise, 418 Oversampling A/D, 756-762 Oversampling D/A, 774 Paley-Wiener theorem, 616 Parseval's relations aperiodic (energy) signals, 244, 260 302 DFT 424 periodic (power) signals, 236, 251 Partial energy, 363 Partial fraction expansion (tee Inverse r-transform) Periodogram, 02 -90 estimation of, 902 -90 mean value 903 14 Index variance, 903 Phase 14, 741 maximum 359-363 minimum, 359-363 mixed, 359-363 response, 311 Pisarenko method, 948-950 Poles, 172 complex conjugate, 193-194, 218-219 distinct, 178-179, 189-191 217 location, 178-181 multiple-order, 179, 191-192 Polyphase filters, 797-800 for decimation, 800 for interpolation, 797 Power definition, 49 signal 50 Power density spectrum, 235-240 definition 236 estimation of (see Power spectrum estimation) periodic signals, 235-240, 250-253 random signals, A5-A7 rectangular pulse train, 237-240 Power spectrum estimation Capon (minimum variance) method 942-945 direct method 899 eigenanalysis algorithms, 950-959 ESPRIT 953-955 MUSIC, 952 order selection, 955-956 Pisarenko, 948-950 experimental results, 936-942 from finite data, 902-908 indirect method, 899 leakage, 899 nonparametric methods, 908 -92 Bartlett, 910-911, 917 Blackman-Tukey, 913-916, 918-919 computational requirements, 919-920 performance characteristics, 916-919 Welch, 911-913, 917-918 parametric (model-based) methods, 920-942 AR model, 924 AR model order selection, 1-932 ARMA model, 924, 34-936 Burg method, 925-928 least-squares, 929-930 MA model, 924, 933-934 maximum entropy method 928 model parameters, 923-924 modified Burg, 928 relation to linear prediction 923-924 sequential least squares, 930-931 Yule-Walker 925 use of DFT, 906-908 Prediction coefficients, 857 Prediction-error filter, 512, 858 properties of, 873-876 Principal eigenvalues, 951 Probability density function, A1-A3 Probability distribution function B1-B2 Prony's method, 706-708 Pseudorandom sequences Barker sequence 148 maximal-length shift register sequences, 148-149 Quadrature components, 740 Quadrature mirror filters for perfect reconstruction, 833-841 for subband coding, 832 Quality, 916-919 of Bartlett estimate, 917 of Blackman-Tukey estimate, 918-919 of Welch estimate, 917-918 Quantization, 21-22, 33-38, 750-753 in A/D conversion 750-753 differential, 756 differential predictive 757 dynamic range, 35, 561, 751 error 37 42, 582-598 in filler coefficients, 569-582 rounding 35, 565-567 truncation 35 564-565 level, 35 750 resolution 35 561 step size, 35 Quantization effects in A/D conversion 37-38, 753-756 in computation of DFT, 486-493 direct computation 487-489 FFT algorithms 489-493 in filter coefficients, 569-582 fixed-point numbers, 557-560 one’s complement, 558-559 sign-magnitude 558 table of bipolar codes, 752 Iwo's complement 559-560 floating-point numbers, 561-564 limit cycles, 583-587 dead band, 584 overflow, 88-589 zero-input, 584 scaling to prevent overflow, 588 -58 statistical characterization, 590-598 Quantizer midrise, 750 midtread 750 resolution 750-752 uniform 750 Random number generators B I-B Gaussian random variable B4-B6 subroutine for B6 Random processes 327-330 A I-A 10 averages, A3-A8 autocorrelation A3 autocovariance, A4 for discrete-time signals, A6-A7 expected value, A3 moments A3 power A3 cofielation-crgodic, A 9-A 10 discrete-time A6-A7 ergodic, A8 jointly stationary, A2-A3 mean-ergodic, A8-A9 power density spectrum, A5-A6 response of linear systems, 327-330 autocorrelation, 327-329 expected value, 328 power density spectrum, 329-330 sample function, A stationary, A3 wide-sense, A3 time-averages, A8-A9 Random signals (see Random processes) Rational z-transforms, 188-196 poles, 172-174 zeros, 172-174 Recursive systems, 116-118 References, R1-R15 Reflection coefficients, 512, 536 863-864 Resonator (see Digital resonator) Reverse (reciprocal) polynomial 515, 861 backward system function 515, 861 Round-off error, 565-567, 590-598 Sample function Al Sample-and-hold, 748-749, 765 Sampling, 9, 21 23, 269-279, 742-746 aliasing effects 7-28, 271-279 of analog signals, 23-33, 269-279 742-746 of bandpass signals 742-746 of discrete-time signals 782-845 frequency, 23 frequency domain, 394-399 interval, 23 Nyquist rate, 30 period, 23 periodic, 23 rate, 23 of sinusoidal signals, 24-28 theorem, 29-30 time-domain, 4-2 8, 269-279 uniform, 23 Sampling-rate conversion, 782-845 applications of, 821-845 for DFT filter banks, 825-831 for interfacing, 823 for lowpass filters, 824 for oversampling A/D and D/A 843-844 for phase shifters 821-822 for subband coding, 831-832 for transmultipiexing, 841-843 by arbitrary factor 815-821 of bandpass signals, 810-815 decimation, 784-787 filter design for, 792-806 interpolation, 784 787-790 multistage, 806-810 polyphase filters for, 797-800 by rational factor, 790-792 Sampling theorem, 29-30, 269-279 Schur algorithm, 868-872 pipelined architecture for, 872-873 split-Schiir algorithm 892 15 Schur-Cohn stability test 213-215 conversion to lattice coefficients 213-214 Shanks' method, 709-710 Sigma-delta modulation 758 Sign magnitude representation, 558 Signal flowgraphs, 521-526 Signals, 2-3 analog, g antisymmetric, 51 aperiodic 50 bandpass, 280 738-742 complex envelope 740 envelope, 741 quadrature components 740 continuous-time deterministic 11 digital 11 discrete-time, 9, 43-55 electrocardiogram (ECG), equivalent lowpass 740 harmonically related 19 multichannel multidimensional, natural 282 frequency ranges 2K2-283 periodic, 15 random, 12, AI-A 10 correlation-ergodic A9-A10 ergodic A9 expected value of, A4 mean-ergodic A9-A10 moments of A4-A7 statistically independent A4 strict-sense stationary' A3 time-averages, A 8-A I0 wide-sense stationary A3 unbiased A8 unconelated A4 seismic, 283 sinusoidal 14 speech 2-3 symmetric, 51 Signal subspace, 951 Sinusoidal generators {.tee Oscillators) Spectrum, 230-232 analysis 232 estimation of 232 896-959 line, 237 Set also Power spectrum estimation Split-radix algorithms 470-473 Stability of LTI systems, 208-217 of second-order systems, 215-217 Stability triangle 216 State-space analysis, 539-566 definition of stale, 540 for difference equations 540-542 LTI state-space model 542 output equation 542 relation to impulse response, 551-553 Index solution of state-space equations, 543-544 state equations, 542 state space, 541 state-space realizations cascade form, 555 coupled form, 556 minimal, 546 normal (diagonal) form, 555 parallel form, 555 state transition matrix, 544 state variables, 539 z-domain, 550-554 zero-input response, 544 zero-state response 544 Steady-state response 206-207, 314-316 Structures 111-118 direct form I, 111-112 direct form II, 113-114 Subband coding, 831-833 Superposition principle, 65 Superposition summation, 76 System, 3, 56-59 dynamic, 62 finite memory, 62 infinite memory, 62 inverse, 356 invertible, 356 relaxed, 59 Syslem function 181-184, 319-321 of all-pole system, 183 of all-zero system, 182-183 of LTI systems, 182-183 relation to frequency response, 319-321 System identification, 355, 363-364 System modeling, 855 System responses forced, 96-97 impulse, 108-110 natural (free), 97, 204 o f relaxed pole-zero systems, 172-184 steady-stale, 206-207 of systems with initial conditions 204-206 transient, 107, 206-207 zero-input, 97 zero-state, 96 Toeplitz matrix, 865, 883 Time averages, A8-A10 Time-limited signals, 281 Transient response 107, 206-207 314-315 Transition band, 619 Transposed structures, 521-526 Truncation error, 35, 564-565 Two’s complement representation, 559 Uniform distribution, 487-488, 565-568, 755 Unit circle, 265, 267 Unit sample (impulse) response 108-110 Unit sample sequence, 45 Variability 916 Variance, 487-lXX, 591-593 Welch method 911-913, 917-918, 919-920 Wideband signal, 281 Wiener filters, 715, 880-890 for filtering, 8 FIR structure, 715, 881-884 IIR structure, 885-889 noncausal 89-890 for prediction 881 for smoothing 881 Wiener-Hopf equation, 882 Wiener-Khintchine theorem 299 Window functions, 626 Wold representation, 854 Wolfer sunspot numbers, 10 autocorrelation, 127-128 graph 128 table, 127 Yule-Walker equations, 857 modified 935 Yule-Walker method, 925 Zero-input linear, 98 Zero-input response, 97 Zero-order hold 38 765 Zero padding 400 Zero-state linear, 98 Zeros, 172 Zoom frequency analysis, 850-851 Z - Irans forms definition, 151-152 bilateral (two-sided), 151-152 unilateral (one-sided), 197-202 inverse, 160-172 184-197 by contour integration, 160-161, 184-186 by partial fraction-expansion, 188-197 by power series, 186-188 properties, 161-172 convolution 168-169 correlation, 169-170 differentiation, 166-167 initial value theorem, 172 linearity 161-163 multiplication, 170-171 Parseval’s relation, 171-172 scaling 164-165 table of, 173 time reversal, 166 time shifting, 163-164 rational, 172-184 region of convergence (ROC), 152-160 relationship of Fourier transform, 264-265 table of, 174 ... Analog signal processor Analog output signal Figure 1.2 A nalog signal processing Sec 1.1 Signals, Systems, and Signal Processing Analog output signal Analog input signal Digital input signal. .. SAMPLING AND RECONSTRUCTION OF SIGNALS 9.1 S am p lin g o f B a n d p a ss S ignals 738 9.1.1 R epresentation of Bandpass Signals, 738 9.1.2 Sampling of Bandpass Signals, 742 9.1.3 Discrete-Time Processing. .. Analysis of Digital Signals and Systems Versus Discrete-Time Signals and Systems, 39 S u m m a ry a n d R e fe re n c e s Problems 39 40 iii iv Contents DISCRETE-TIME SIGNALS AND SYSTEMS 2.1 D