signals and systems

Signals and Systems Richard Baraniuk NONRETURNABLE NO REFUNDS, EXCHANGES, OR CREDIT ON ANY COURSE PACKETS Signals and Systems Course Authors: Richard Baraniuk Contributing Authors: Thanos Antoulas Richard Baraniuk Adam Blair Steven Cox Benjamin Fite Roy Ha Michael Haag Don Johnson Ricardo Radaelli-Sanchez Justin Romberg Phil Schniter Melissa Selik John Slavinsky Michael Wakin Produced by: The Connexions Project http://cnx.rice.edu/ Rice University, Houston TX Problems? Typos? Suggestions? etc http://mountainbunker.org/bugReport c 2003 Thanos Antoulas, Richard Baraniuk, Adam Blair, Steven Cox, Benjamin Fite, Roy Ha, Michael Haag, Don Johnson, Ricardo Radaelli-Sanchez, Justin Romberg, Phil Schniter, Melissa Selik, John Slavinsky, Michael Wakin This work is licensed under the Creative Commons Attribution License: http://creativecommons.org/licenses/by/1.0 Table of Contents 1 Introduction 2.1 Signals Represent Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 2 Signals and Systems: A First Look 3.1 System Classifications and Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7 3.2 Properties of Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 3.3 Signal Classifications and Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .14 3.4 Discrete-Time Signals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 3.5 Useful Signals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 3.6 The Complex Exponential . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .30 3.7 Discrete-Time Systems in the Time-Domain . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 3.8 The Impulse Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 3.9 BIBO Stability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .40 3 Time-Domain Analysis of CT Systems 4.1 Systems in the Time-Domain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 4.2 Continuous-Time Convolution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 4.3 Properties of Convolution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 4.4 Discrete-Time Convolution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 4 Linear Algebra Overview 5.1 Linear Algebra: The Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 5.2 Vector Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 5.3 Eigenvectors and Eigenvalues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .70 5.4 Matrix Diagonalization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .76 5.5 Eigen-stuff in a Nutshell . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 5.6 Eigenfunctions of LTI Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79 5 Fourier Series 6.1 Periodic Signals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 6.2 Fourier Series: Eigenfunction Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 6.3 Derivation of Fourier Coefficients Equation . . . . . . . . . . . . . . . . . . . . . . . . . . . 88 6.4 Fourier Series in a Nutshell . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .89 6.5 Fourier Series Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93 6.6 Symmetry Properties of the Fourier Series . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96 6.7 Circular Convolution Property of Fourier Series . . . . . . . . . . . . . . . . . . . . . . . 100 6.8 Fourier Series and LTI Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102 6.9 Convergence of Fourier Series . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104 6.10 Dirichlet Conditions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106 6.11 Gibbs’s Phenomena . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108 6.12 Fourier Series Wrap-Up . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .111 6 Hilbert Spaces and Orthogonal Expansions 7.1 Vector Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 7.2 Norms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115 7.3 Inner Products . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .118 7.4 Hilbert Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120 7.5 Cauchy-Schwarz Inequality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .120 7.6 Common Hilbert Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128 7.7 Types of Basis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131 iv 7.8 Orthonormal Basis Expansions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135 7.9 Function Space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139 7.10 Haar Wavelet Basis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140 7.11 Orthonormal Bases in Real and Complex Spaces . . . . . . . . . . . . . . . . . . . . . 143 7.12 Plancharel and Parseval’s Theorems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148 7.13 Approximation and Projections in Hilbert Space . . . . . . . . . . . . . . . . . . . . . 151 7 Fourier Analysis on Complex Spaces 8.1 Fourier Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153 8.2 Fourier Analysis in Complex Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154 8.3 Matrix Equation for the DTFS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .163 8.4 Periodic Extension to DTFS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164 8.5 Circular Shifts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168 8.6 Circular Convolution and the DFT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .178 8.7 DFT: Fast Fourier Transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183 8.8 The Fast Fourier Transform (FFT) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184 8.9 Deriving the Fast Fourier Transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185 8 Convergence 9.1 Convergence of Sequences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189 9.2 Convergence of Vectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .190 9.3 Uniform Convergence of Function Sequences . . . . . . . . . . . . . . . . . . . . . . 194 9 Fourier Transform 10.1 Discrete Fourier Transformation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195 10.2 Discrete Fourier Transform (DFT) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196 10.3 Table of Common Fourier Transforms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .199 10.4 Discrete-Time Fourier Transform (DTFT) . . . . . . . . . . . . . . . . . . . . . . . . . 199 10.5 Discrete-Time Fourier Transform Properties . . . . . . . . . . . . . . . . . . . . . . . . . 200 10.6 Discrete-Time Fourier Transform Pair . . . . . . . . . . . . . . . . . . . . . . . .200 10.7 DTFT Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202 10.8 Continuous-Time Fourier Transform (CTFT) . . . . . . . . . . . . . . . . . . . . . . . . 204 10.9 Properties of the Continuous-Time Fourier Transform . . . . . . . . . . . . . . . .207 10 Sampling Theorem 11.1 Sampling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211 11.2 Reconstruction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215 11.3 More on Reconstruction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219 11.4 Nyquist Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222 11.5 Aliasing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223 11.6 Anti-Aliasing Filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227 11.7 Discrete Time Processing of Continuous TIme Signals . . . . . . . . . . . . . . . . . .228 11 Laplace Transform and System Design 12.1 The Laplace Transforms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235 12.2 Properties of the Laplace Transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239 12.3 Table of Common Laplace Transforms . . . . . . . . . . . . . . . . . . . . . . . . . 239 12.4 Region of Convergence for the Laplace Transform . . . . . . . . . . . . . . . . . . . . 240 12.5 The Inverse Laplace Transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242 12.6 Poles and Zeros . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243 12 Z-Transform and Digital Filtering 13.1 The Z Transform: Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247 v 13.2 Table of Common Z-Transforms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251 13.3 Region of Convergence for the Z-transform . . . . . . . . . . . . . . . . . . . . . . . . . . .252 13.4 Inverse Z-Transrom . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 260 13.5 Rational Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263 13.6 Difference Equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .265 13.7 Understanding Pole/Zero Plots on the Z-Plane . . . . . . . . . . . . . . . . . . . . . . .268 13.8 Filter Design using the Pole/Zero Plot of a Z-Transform . . . . . . . . . . . . . . . 272 13 Homework Sets 14.1 Homework #1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 277 14.2 Homework #1 Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 280 vi 1 1 Cover Page .1.1 Signals and Systems: Elec 301 summary: This course deals with signals, systems, and transforms, from their theoretical mathematical foundations to practical implementation in circuits and computer algorithms. At the conclusion of ELEC 301, you should have a deep understanding of the mathematics and practical issues of signals in continuous and discrete time, linear time-invariant systems, convolution, and Fourier transforms. Instructor: Richard Baraniuk 1 Teaching Assistant: Michael Wakin 2 Course Webpage: Rice University Elec301 3 Module Authors: Richard Baraniuk, Justin Romberg, Michael Haag, Don Johnson Course PDF File: Currently Unavailable 1 http://www.ece.rice.edu/∼richb/ 2 http://www.owlnet.rice.edu/∼wakin/ 3 http://dsp.rice.edu/courses/elec301 2 [...]... to discrete-time signals, and will prove useful subsequently Discrete-time systems can act on discrete-time signals in ways similar to those found in analog signals and systems Because of the role of software in discrete-time systems, many 26 CHAPTER 2 SIGNALS AND SYSTEMS: A FIRST LOOK more different systems can be envisioned and “constructed” with programs than can be with analog signals In fact, a... not, discrete-time systems are ultimately constructed from digital circuits, which consist entirely of analog circuit elements Furthermore, the transmission and reception of discrete-time signals, like e-mail, is accomplished with analog signals and systems Understanding how discrete-time and analog signals and systems intertwine is perhaps the main goal of this course 3.5 Useful Signals Before looking... discussions like energy signals vs power signals have been designated their own module for a more complete discussion, and will not be included here 15 Cascaded LTI Systems Figure 2.11: The order of cascaded LTI systems can be interchanged without changing the overall effect Parallel LTI Systems Figure 2.12: Parallel systems can be condensed into the sum of systems 16 CHAPTER 2 SIGNALS AND SYSTEMS: A FIRST... confidence On the other hand, a random signal has a lot of uncertainty about its behavior The future values of a random signal cannot be acurately predicted and can usually only be guessed based on the averages of sets of signals 2.3.2.7 Right-Handed vs Left-Handed A right-handed signal and left-handed signal are those signals whose value is zero between a given variable and positive or negative infinity... 2 Signals and Systems: A First Look 3.1 System Classifications and Properties 2.1.1 Introduction In this module some of the basic classifications of systems will be briefly introduced and the most important properties of these systems are explained As can be seen, the properties of a system provide an easy way to separate one system from another Understanding these basic difference’s between systems, and. .. 2.1.2 Classification of Systems Along with the classification of systems below, it is also important to understand the Classification of Signals 2.1.2.1 Continuous vs Discrete This may be the simplest classification to understand as the idea of discrete-time and continuous-time is one of the most fundamental properties to all of signals and system A system where the input and output signals are continuous... equation, Equation 2.7, to be true 2.3.2.4 Causal vs Anticausal vs Noncausal Causal signals are signals that are zero for all negative time, while anitcausal are signals that are zero for all positive time Noncausal signals are signals that have nonzero values in both positive and negative time 18 CHAPTER 2 SIGNALS AND SYSTEMS: A FIRST LOOK Figure 2.15 (a) (b) Figure 2.16: (a) A periodic signal with... real numbers: ∞ ≤ f (t) ≤ −∞ 23 (a) (b) Figure 2.21: (a) Right-handed signal (b) Left-handed signal 3.4 Discrete-Time Signals So far, we have treated what are known as analog signals and systems Mathematically, analog signals are functions having countinuous quantities as their independent variables, such as space and time Discrete-time signals are functions defined on the integers; they are sequences... deal with such symbolic valued (pg ??) signals and systems as well As with analog signals, we seek ways of decomposing real-valued discrete-time signals into simpler components With this approach leading to a better understanding of signal structure, we can exploit that structure to represent information (create ways of representing information with signals) and to extract information (retrieve the... difference) equation means that the parameters of the system are not changing over time and an input now will give the same result as the same input later 2.2.3 ”Linear Time-Invariant (LTI) Systems Certain systems are both linear and time-invariant, and are thus referred to as LTI systems As LTI systems are a subset of linear systems, they obey the principle of superposition In the figure below, we see the . Page .1.1 Signals and Systems: Elec 301 summary: This course deals with signals, systems, and transforms, from their theoretical mathematical foundations to practical implementation in circuits and computer. fundamental properties to all of signals and system. A system where the input and output signals are continuous is a continuous system , and one where the input and ouput signals are discrete is a discrete. ”Linear Time-Invariant (LTI) Systems Certain systems are both linear and time-invariant, and are thus referred to as LTI systems. As LTI systems are a subset of linear systems, they obey the principle