Digital signal processing: Lecture 8 Discrete Fourier Transformation –II

Produced by Qiangfu Zhao (Since 1995), All rights reserved © DSPLec081 Digital signal processing: Lecture 8 Discrete Fourier Transformation –II Chapter 4: Fast Fourier TransformationProduced by Qiangfu Zhao (Since 1995), All rights reserved © DSPLec082 Topics of last lecture • Discrete Fourier transformation (DFT) • Linearity • Periodicity • Periodicity Symmetry • Direct computation of DFT • 離散フーリエ変換 (DFT) • DFTの線形性 • DFTの周期性 • DFTの対称性 • DFTの直接的計算Produced by Qiangfu Zhao (Since 1995), All rights reserved © DSPLec083 Topics of this lecture • Fast Fourier transform. • Computational cost of FFT. • Convolution based on FFT. • Filtering based on FFT. • Crosscorrelation based on FFT. • 高速フーリエ変換 • FFTの計算量 • FFTに基づく畳み込み • FFTに基づくフィルタリ ング • FFTに基づく相関Produced by Qiangfu Zhao (Since 1995), All rights reserved © DSPLec084 The fast Fourier transform (FFT) • The basic idea is to use the symmetric and periodic properties of ej2πknN. • Suppose that N=2p. • The DFT can be divided into two parts as follows: • Since the computational cost for each part is (N2)2, and we can combine the results using N computations, the total cost can be reduced. ∑ ∑ ∑ = = − = = = + n odd kn N n even kn N N n kn X (k) x(n)WN x(n)W x(n)W 1 0Produced by Qiangfu Zhao (Since 1995), All rights reserved © DSPLec085 ( ) ( ) ( )( ) ( )( ) (2 )( ) (2 1)( ) (2 )( ) (2 1)( ) ( ) ( ) ( ) ( ) 0 1 2 2 1 0 1 2 1 0 2 0 2 2 1 0 2 1 0 2 2 1 0 2 2 1 0 2 1 0 X k W X k x r W W x r W x r W W x r W x r W W x r W X k x n W x n W x n W kN kr N N r kN kr N r N kr N N r kN kr N r N kr N r N kN kr N r N n odd kn N n even kn N N n kn N = + = + = + + = + + = = + ∑ ∑ ∑ ∑ ∑ ∑ ∑ ∑ ∑ − = − = − = − = − = − = = = − =Produced by Qiangfu Zhao (Since 1995), All rights reserved © DSPLec086 ( 2) ( ) ( ), 0,1,..., 2 1 ( ) ( ) ( ), 0,1,..., 2 1 We can find ( ) as follows: Also, since In the above derivation, we have used the property 0 1 0 1 2 2 2 2 (2 ) (2 ( 2)) + = − = − = + = − = − = = = + − − X k N X k W X k k N X k X k W X k k N X k W W W e e W kN kN k N N kN N j N j N N π πProduced by Qiangfu Zhao (Since 1995), All rights reserved © DSPLec087 Graphically, we haveProduced by Qiangfu Zhao (Since 1995), All rights reserved © DSPLec088 This process can be continuedButterfly operation • The division is continued until N=2. • For N=2, two outputs can be obtained from two inputs using the butterfly operation. Produced by Qiangfu Zhao (Since 1995), All rights reserved © DSPLec089 ( ) ( ) ( ) (4.21) ( ) ( ) ( ) (4.20) 1 1 X r X q W X r X q X q W X r kN s s kN s s = − = + + +Flow of FFT (Fig. 4.6 p. 56) Produced by Qiangfu Zhao (Since 1995), All rights reserved © DSPLec0810Produced by Qiangfu Zhao (Since 1995), All rights reserved © DSPLec0811 Computational cost of FFT • An Npoint DFT can be divided into two N2point DFTs. • Computation amount is reduced (if N>2) to N+2(N2)2 from N2. • Similarly, the N2point DFT can be divided into two N4point DFTs. • This division can be continued until there are only two points. • The total number of multiplications is (N2)log2N. • The total number of additions is Nlog2N.Find the convolution sum of two Npoint discrete signals (p. 62) ∑= = − N k y n x k x n k N y n N x n x n 0 1 2 1 2 ( ) ( ) ( ) Result : An 2 point discrete signal ( ) calculated by Given :Two point discrete signals ( ) and ( ). Produced by Qiangfu Zhao (Since 1995), All rights reserved © DSPLec0812Find the convolution sum with FFT • If we find the convolution sum directly, the number of computations will be O(N2). • If we use FFT, the computation cost can be reduced to Nlog2N. • For this purpose, we need the following steps: – Step 1: Extend each signal to 2N1 points; – Step 2: Find the DFT of each signal; – Step 3: Multiply the results; – Step 4: Find the IDFT. • The cost will be 2xNlog2(2N)=O(Nlog2N). Produced by Qiangfu Zhao (Since 1995), All rights reserved © DSPLec0813 (p. 62)Example 4.3 p. 6263 Produced by Qiangfu Zhao (Since 1995), All rights reserved © DSPLec0814Find the response of an FIR filter Produced by Qiangfu Zhao (Since 1995), All rights reserved © DSPLec0815 ∑= = − N k y n h k x n k x n h n 0 ( ) ( ) ( ) Response of the system calculated by Result : ( ) : Input of the system : ( ) : Impulse response of an LTI system Given : Merit: We can use FFT to reduce the computational cost. Problem: The length of x(n) can be much longerFiltering with an FIR system Produced by Qiangfu Zhao (Since 1995), All rights reserved © DSPLec0816 Taken from Digital Signal Processing, written by Oppenheim, and translated by H. Date.Produced by Qiangfu Zhao (Since 1995), All rights reserved © DSPLec0817 Overlapadd method for finding the response of an FIR filter using convolution. Taken from Digital Signal Processing, written by Oppenheim, and translated by H. Date.Find the crosscorrelation of two signals • Crosscorrelation is a measure of similarity between two signals or patterns. • Crosscorrelation can be used to find features in an unknown signal by comparing it to a known one. It has applications in pattern (e.g. image and text) detection, which is useful for information retrieval. • The crosscorrelation is similar to the convolution of two signals. Convolution involves reversing a signal, then shifting it and multiplying by another signal. • Both crosscorrelation and convolution can be found efficiently using FFT. Produced by Qiangfu Zhao (Since 1995), All rights reserved © DSPLec0818Produced by Qiangfu Zhao (Since 1995), All rights reserved © DSPLec0819 Definition of crosscorrelation correlation is called auto correlation. conjugate. If ( ) and ( ) are the same, the cross where the superscript asterisk indicates the complex ( ) ( ) ( ) ( ) correction is defined by Given two signals ( ) and ( ), their cross 1 2 2 1 1 2 1 2 x n x n x n x n x m x n m x n x n ∞∑m =−∞ ⊗ = +Produced by Qiangfu Zhao (Since 1995), All rights reserved © DSPLec0820 Finding the crosscorrelation using FFT • Similar to convolution theorem, we have • Therefore, we can find the crosscorrelation efficiently using FFT as follows: – Step 1: Extend the lengths (durations) of the two signals to 2N1; – Step 2: Find the DFT of the two signals. – Step 3: Find the complex conjugate of DFTx1(n). – Step 4: Multiplication. – Step 5: Find the IDFT. DFTx1(n)⊗ x2(n) = DFTx1(n) ⋅DFTx2(n)Homework Produced by Qiangfu Zhao (Since 1995), All rights reserved © DSPLec0821 • Revise the program given in p. 63, and find x3(n)=x1(n)x2(n), using FFT, where is the convolution sum, and x1(n) and x2(n) are given below. – To understand the basic steps, see slide 13. – Plot and print ｘ1,x2, and x3; and their FFTs. – Submit the hardcopies before the deadline. x1=1 2 3 4 5 6 7 8; x2=1 2 3 4 5 4 3 2;Quiz and selfevaluation 0 0.2 0.4 0.6 0.8 1 1.2 T1 T2 T4 T3 T5 Name: Student ID: . • For a N=16 point DFT, how many percent of computations can we reduce if we use FFT? The rate of reduction is calculated as follows: Answer: • How about the reduction rate for N=64? Answer: % Cost of DFT Cost of DFT Cost of FFT R =

Digital signal processing: Lecture Discrete Fourier Transformation –II Chapter 4: Fast Fourier Transformation Produced by Qiangfu Zhao (Since 1995), All rights reserved â DSP-Lec08/1 Topics of last lecture ã Discrete Fourier transformation (DFT) • Linearity • Periodicity • Periodicity Symmetry • Direct computation of DFT • 離散フーリエ変換 (DFT) • DFTの線形性 • DFTの周期性 • DFTの対称性 • DFTの直接的計算 Produced by Qiangfu Zhao (Since 1995), All rights reserved © DSP-Lec08/2 Topics of this lecture • Fast Fourier transform • Computational cost of FFT • Convolution based on FFT • Filtering based on FFT • Cross-correlation based on FFT 高速フーリエ変換 FFTの計算量 FFTに基づく畳み込み FFTに基づくフィルタリ ング • FFTに基づく相関 • • • • Produced by Qiangfu Zhao (Since 1995), All rights reserved © DSP-Lec08/3 The fast Fourier transform (FFT) • The basic idea is to use the symmetric and periodic properties of e-j2πkn/N • Suppose that N=2p • The DFT can be divided into two parts as follows: N −1 X (k ) = ∑ x(n)WNkn = n =0 kn x ( n ) W ∑ N + n = even kn x ( n ) W ∑ N n = odd • Since the computational cost for each part is (N/2)2, and we can combine the results using N computations, the total cost can be reduced Produced by Qiangfu Zhao (Since 1995), All rights reserved © DSP-Lec08/4 N −1 X (k ) = ∑ x(n)W n =0 = = kn N ∑ x(n)W n = even N/ −1 N/ −1 r =0 r =0 kr k x r W W ( )( ) + ∑ N N N/ −1 kr k x r W W ( )( ) + ∑ N N r =0 = = kn N N/ −1 ∑ x (r )(W r =0 ) +W kr N + ∑ x(n)W n = odd kn N kr x r W ( )( + ∑ N) N/ −1 kr x ( )( ) r W + ∑ N r =0 N/ −1 k N ∑ x (r )(W r =0 N ) kr = X (k ) + W X (k ) k N Produced by Qiangfu Zhao (Since 1995), All rights reserved © DSP-Lec08/5 In the above derivation, we have used the property W =e N − j ( 2π / N ) =e − j ( 2π /( N / )) = WN Also, since WNk = −WNk + N / We can find X ( k ) as follows : X ( k ) = X ( k ) + W X ( k ), k = 0,1, , N / − k N X ( k + N / 2) = X ( k ) − WNk X ( k ), k = 0,1, , N / − Produced by Qiangfu Zhao (Since 1995), All rights reserved © DSP-Lec08/6 Graphically, we have Produced by Qiangfu Zhao (Since 1995), All rights reserved © DSP-Lec08/7 This process can be continued Produced by Qiangfu Zhao (Since 1995), All rights reserved â DSP-Lec08/8 Butterfly operation ã The division is continued until N=2 • For N=2, two outputs can be obtained from two inputs using the butterfly operation X s +1 ( q) = X s ( q) + WNk X ( r ) ( 4.20) X s +1 ( r ) = X s ( q) − WNk X ( r ) ( 4.21) Produced by Qiangfu Zhao (Since 1995), All rights reserved © DSP-Lec08/9 Flow of FFT (Fig 4.6 p 56) Produced by Qiangfu Zhao (Since 1995), All rights reserved â DSP-Lec08/10 Computational cost of FFT ã An N-point DFT can be divided into two N/2-point DFTs • Computation amount is reduced (if N>2) to N+2(N/2)2 from N2 • Similarly, the N/2-point DFT can be divided into two N/4-point DFTs • This division can be continued until there are only two points • The total number of multiplications is (N/2)log2N • The total number of additions is Nlog2N Produced by Qiangfu Zhao (Since 1995), All rights reserved © DSP-Lec08/11 Find the convolution sum of two N-point discrete signals (p 62) Given : Two N - point discrete signals x1 (n) and x2 (n) Result : An N - point discrete signal y (n) calculated by N y (n) = ∑ x1 (k ) x2 (n − k ) k =0 Produced by Qiangfu Zhao (Since 1995), All rights reserved © DSP-Lec08/12 Find the convolution sum with FFT • If we find the convolution sum directly, the number of computations will be O(N2) • If we use FFT, the computation cost can be reduced to Nlog2N • For this purpose, we need the following steps: – – – – Step 1: Extend each signal to 2N-1 points; Step 2: Find the DFT of each signal; Step 3: Multiply the results; Step 4: Find the IDFT (p 62) • The cost will be 2xNlog2(2N)=O(Nlog2N) Produced by Qiangfu Zhao (Since 1995), All rights reserved © DSP-Lec08/13 Example 4.3 p 62-63 Produced by Qiangfu Zhao (Since 1995), All rights reserved © DSP-Lec08/14 Find the response of an FIR filter Given : h(n) : Impulse response of an LTI system x(n) : Input of the system : Result : Response of the system calculated by N y ( n) = ∑ h( k ) x ( n − k ) k =0 Merit: We can use FFT to reduce the computational cost Problem: The length of x(n) can be much longer! Produced by Qiangfu Zhao (Since 1995), All rights reserved © DSP-Lec08/15 Filtering with an FIR system Taken from Digital Signal Processing, written by Oppenheim, and translated by H Date Produced by Qiangfu Zhao (Since 1995), All rights reserved © DSP-Lec08/16 Overlap-add method for finding the response of an FIR filter using convolution Taken from Digital Signal Processing, written by Oppenheim, and translated by H Date Produced by Qiangfu Zhao (Since 1995), All rights reserved © DSP-Lec08/17 Find the cross-correlation of two signals • Cross-correlation is a measure of similarity between two signals or patterns • Cross-correlation can be used to find features in an unknown signal by comparing it to a known one It has applications in pattern (e.g image and text) detection, which is useful for information retrieval • The cross-correlation is similar to the convolution of two signals Convolution involves reversing a signal, then shifting it and multiplying by another signal • Both cross-correlation and convolution can be found efficiently using FFT Produced by Qiangfu Zhao (Since 1995), All rights reserved © DSP-Lec08/18 Definition of cross-correlation Given two signals x1 ( n ) and x2 ( n ), their cross correction is defined by x1 ( n ) ⊗ x2 ( n ) = ∞ * x ∑ ( m ) x2 ( n + m ) m = −∞ where the superscript asterisk indicates the complex conjugate If x1 ( n ) and x2 ( n ) are the same, the cross correlation is called auto - correlation Produced by Qiangfu Zhao (Since 1995), All rights reserved © DSP-Lec08/19 Finding the cross-correlation using FFT • Similar to convolution theorem, we have DFT[ x1 (n) ⊗ x2 (n)] = DFT[ x1 (n)]* ⋅ DFT[ x2 (n)] • Therefore, we can find the cross-correlation efficiently using FFT as follows: – Step 1: Extend the lengths (durations) of the two signals to 2N-1; – Step 2: Find the DFT of the two signals – Step 3: Find the complex conjugate of DFT[x1(n)] – Step 4: Multiplication – Step 5: Find the IDFT Produced by Qiangfu Zhao (Since 1995), All rights reserved â DSP-Lec08/20 Homework ã Revise the program given in p 63, and find x3(n)=x1(n)*x2(n), using FFT, where * is the convolution sum, and x1(n) and x2(n) are given below – To understand the basic steps, see slide 13 – Plot and print ｘ1,x2, and x3; and their FFTs – Submit the hardcopies before the deadline x1=[1 8]; x2=[1 2]; Produced by Qiangfu Zhao (Since 1995), All rights reserved â DSP-Lec08/21 Quiz and self-evaluation ã For a N=16 point DFT, how many percent of computations can we reduce if we use FFT? The rate of reduction is calculated as follows: T1 1.2 0.8 0.6 R= Cost of DFT - Cost of FFT % Cost of DFT 0.4 T5 T2 0.2 Answer: • How about the reduction rate for N=64? T4 T3 Answer: Name: Student ID: ... reserved © DSP-Lec 08/ 11 Find the convolution sum of two N-point discrete signals (p 62) Given : Two N - point discrete signals x1 (n) and x2 (n) Result : An N - point discrete signal y (n) calculated...Topics of last lecture • Discrete Fourier transformation (DFT) • Linearity • Periodicity • Periodicity Symmetry • Direct computation... DSP-Lec 08/ 15 Filtering with an FIR system Taken from Digital Signal Processing, written by Oppenheim, and translated by H Date Produced by Qiangfu Zhao (Since 1995), All rights reserved © DSP-Lec 08/ 16