Audio DSP Dr. Deepa Kundur University of Toronto Dr. Deepa Kundur (University of Toronto) Audio DSP 1 56 Intro to Audio Signals Amplitude and Loudness Sound I Sound: vibration transmitted through a medium (gas, liquid, solid and plasma) composed of frequencies capable of being detected by ears. I Note: sound cannot travel through a vacuum. I Human detectable sound is often characterized by air pressure variations detected by the human ear. I The amplitude, frequency and relative phase of the air pressure signal components determine (in part) the way the sound is perceived. Dr. Deepa Kundur (University of Toronto) Audio DSP 2 56 Intro to Audio Signals Amplitude and Loudness Sinusoids and Sound: Amplitude I A fundamental unit of sound is the sinusoidal signal. xa(t) = A cos(2πF0t + θ); t 2 R I A ≡ volume I F0 ≡ pitch (more on this . . . ) I θ ≡ phase (more on this . . . ) Dr. Deepa Kundur (University of Toronto) Audio DSP 3 56 Intro to Audio Signals Amplitude and Loudness Sound Volume I Volume = Amplitude of sound wavesaudio signals I quoted in dB, which is a logarithmic measure; 10 log(A2) I no soundnull is −1 dB I Loudness is a subjective measure of sound psychologically correlating to the strength of the sound signal. I the volume is an objective measure and does not have a onetoone correspondence with loudness I perceived loudness varies from persontoperson and depends on frequency and duration of the sound Dr. Deepa Kundur (University of Toronto) Audio DSP 4 56Intro to Audio Signals Amplitude and Loudness Music Volume Dynamic Range Tests conducted for the musical note: C6 (F0 = 1046:502 Hz). Dynamic Level Decibels Threshold of hearing 0 ppp (pianissimo) 40 p (piano) 60 f (forte) 80 fff (fortississimo) 100 Threshold of pain 120 Dr. Deepa Kundur (University of Toronto) Audio DSP 5 56 Intro to Audio Signals Frequency and Pitch Sinusoids and Sound: Frequency I A fundamental unit of sound is the sinusoidal signal. xa(t) = A cos(2πF0t + θ); t 2 R I A ≡ volume I F0 ≡ pitch I θ ≡ phase (more on this . . . ) Dr. Deepa Kundur (University of Toronto) Audio DSP 6 56 Intro to Audio Signals Frequency and Pitch Pure Frequency I Q: What type of sound does a pure frequency produce? I A: A pure tone with a single pitch. I Q: Can any instrument produce a pure tone by playing a single note? I A: No. Dr. Deepa Kundur (University of Toronto) Audio DSP 7 56 Intro to Audio Signals Frequency and Pitch Tuning Forks I A tuning fork is a twopronged instrument that is an acoustic resonator. It is usually made out of steel and resonates at a specific constant pitch which is a function of the length of the prongs. I Striking the tuning fork will produce the required sounds although initially there may be overtones that die out quickly. I A very common tuning fork used by musicians produces the A note (F0 = 440 Hz), which is international concert pitch used to tune orchestras. Dr. Deepa Kundur (University of Toronto) Audio DSP 8 56Intro to Audio Signals Frequency and Pitch Frequency and Pitch I Sinusoids can be represented either as: xa(t) = A cos(2πF0t + θ); t 2 R or for mathematical convenience when interpreting as Fourier signal components as: xa(t) = Aej(2πF0t+θ); t 2 R I Pitch is directly related to the frequency F0. I To be able to hear a frequency F0, it has to be in the human audible range. Dr. Deepa Kundur (University of Toronto) Audio DSP 9 56 Intro to Audio Signals Frequency and Pitch Harmonically Related Frequencies and Pitch Scientific Designation Frequency (Hz) k for F0 = 8:176 C1 32.703 4 C2 65.406 8 C3 130.813 16 C4 (middle C) 261.626 32 C5 523.251 64 C6 1046.502 128 C7 2093.005 256 C8 4186.009 512 C1 C2 C3 C4 C5 C6 C7 C8 Dr. Deepa Kundur (University of Toronto) Audio DSP 10 56 Intro to Audio Signals Frequency and Pitch Harmonically Related Frequencies I Recall harmonically related sinusoids have the following analytic form for k 2 Z: xa;k(t) = A cos(2πkF0t + θ) or xa;k(t) = Aej(2πkF0t+θ) I They are used in the context of the Fourier Series to build periodic signals: x(t) = 1 X k=−1 X(k)ej(2πkF0t) Dr. Deepa Kundur (University of Toronto) Audio DSP 11 56 Intro to Audio Signals Frequency and Pitch Signature Sounds I Q: If two different people sing the same note or two different instruments play the same note, why do they sound different? I The notes are not pure tones. There are natural overtones and undertones that provide distinguishing signatures that can be viewed in the associated spectra. Dr. Deepa Kundur (University of Toronto) Audio DSP 12 56Intro to Audio Signals Frequency and Pitch Fourier Transforms of the Same Note 0 f Instrument A 0 f Instrument B 0 f Tuning Fork Dr. Deepa Kundur (University of Toronto) Audio DSP 13 56 Intro to Audio Signals Frequency and Pitch Human Audible Range I Hearing is usually limited to frequencies between 20 Hz and 20 kHz. I The upper limit decreases with age. I The audible frequency range is different for animals Dr. Deepa Kundur (University of Toronto) Audio DSP 14 56 Intro to Audio Signals Frequency and Pitch Animal Audible Range Species Approx Range (Hz) human 20 20,000 dog 67 45,000 rabbit 360 42,000 bat 2,000 110,000 goldfish 20 3,000 Reference: R.R. Fay (1988), Hearing in Vertebrates: A Psychophysics Databook. Dr. Deepa Kundur (University of Toronto) Audio DSP 15 56 Intro to Audio Signals Phase and Sound Sinusoids and Sound: Phase I A fundamental unit of sound is the sinusoidal signal. xa (t) = A cos(2πF0t + θ); t 2 R I A ≡ volume I F0 ≡ pitch I θ ≡ phase Dr. Deepa Kundur (University of Toronto) Audio DSP 16 56Intro to Audio Signals Phase and Sound Phase and Sound Consider a general sound signal x(t) that is comprised of frequency components each with a specific phase shift. x(t) = Z−1 1 X(f )ej2πf tdf I jX(f )j: relative volume of a sinusoidal component I X(f ): relative phase of a sinusoidal component Dr. Deepa Kundur (University of Toronto) Audio DSP 17 56 Intro to Audio Signals Phase and Sound Phase and Sound I If x(t) is the general sound signal, then x(−t) is the sound signal in reverse. I Q: Do x(t) and x(−t) sound similar? I A: No. Dr. Deepa Kundur (University of Toronto) Audio DSP 18 56 Intro to Audio Signals Phase and Sound Phase and Sound I Recall, from the continuoustime Fourier transform (CTFT) that for a real signal x(t): x(t) F X(f ) x(−t) F X(−f ) and X(f ) = X ∗(−f ) Dr. Deepa Kundur (University of Toronto) Audio DSP 19 56 Intro to Audio Signals Phase and Sound Phase and Sound I Taking the magnitude and phase of both sides we have: X(f ) = X ∗(−f ) jX(f )j = jX ∗(−f )j = jX(−f )j X(f ) = X ∗(−f ) = −X(−f ) I Conjugate Symmetry (for real signals x(t)): I CTFT magnitude is even I CTFT phase is odd Dr. Deepa Kundur (University of Toronto) Audio DSP 20 56Intro to Audio Signals Phase and Sound Phase and Sound I Therefore, for x(t) F X (f ) x(−t) F X (−f ) I jX (f )j = jX (−f )j ) the CTFT magnitudes for forward and reverse sound signals are exactly the same. I X (f ) 6= X (−f ) ) the CTFT phases for forward and reverse sound signals are different. I Therefore, the relative phase of the sinusoidal components of sound contains very salient perceptual information much like for images. Dr. Deepa Kundur (University of Toronto) Audio DSP 21 56 Intro to Audio Signals Auditory Masking Auditory Masking I occurs when the perceived quality of one (primary) sound is affected by the presence of another (secondary) sound I Simultaneous masking: the secondary sound is heard at the same time as the primary sound I Can be exploited (as we see in an upcoming lab) to mask nonideal signal processing. Dr. Deepa Kundur (University of Toronto) Audio DSP 22 56 Audio Digital Signal Processing Analog and Digital Audio Why Digitize Audio? I Fidelity of digital audio is much higher than analog audio. I Manipulation tools for digital audio are much more sophisticated than those available for analog audio. I Compression of digital audio provides significantly reduced storage requirements. I Storage of digital audio (e.g., CDs) are much more convenient and compact. I Duplication of digital audio is exact in contrast to analog audio. Dr. Deepa Kundur (University of Toronto) Audio DSP 23 56 Audio Digital Signal Processing Analog and Digital Audio Benefits of Digital Audio I Convenient recording, enhancement, massproduction and distribution. I CDs, online stores such as iTunes, etc. I data files are distributed instead of physical media storing the information such as records and tapes. Dr. Deepa Kundur (University of Toronto) Audio DSP 24 56Audio Digital Signal Processing Analog and Digital Audio Concerns about Digital Audio I Convenient recording, enhancement, massproduction and distribution. I unlawful manipulation of recorded audio is difficult to detect I piracy: unlawful copying and redistribution of copyrighted content Dr. Deepa Kundur (University of Toronto) Audio DSP 25 56 Audio Digital Signal Processing Analog and Digital Audio Analog vs. Digital Audio: Analog Audio System Analog audio signal Transmission Storage Loudspeaker Transducer (e.g., microphone) I microphone: converts sound into an electrical signal; air pressure motion of conductorcoil magnetic field electrical signal I loudspeaker: converts electrical signal into acoustic waves; electrical signal magnetic field motion air pressure Dr. Deepa Kundur (University of Toronto) Audio DSP 26 56 Audio Digital Signal Processing Analog and Digital Audio Analog vs. Digital Audio: Analog Audio System Analog audio signal Transmission Storage Loudspeaker Transducer (e.g., microphone) I associated circuits suffer from inherent noise (noise floor) I capacitance and inductance of the circuits limit bandwidth, and resistance limits amplitude Dr. Deepa Kundur (University of Toronto) Audio DSP 27 56 Audio Digital Signal Processing Analog and Digital Audio Analog vs. Digital Audio: Digital Audio Chain Analog audio signal Digital audio signal Transmission Storage DA Converter AD Converter Error Correction Coding (ECC) ECC Decoding I fidelity limited by quantization noise I bandwidth limited by sampling rate I dynamic range limited by bit resolution Dr. Deepa Kundur (University of Toronto) Audio DSP 28 56Audio Digital Signal Processing Analog and Digital Audio Digitizing Audio AD Processing for Transmission Storage DA Analog audio input (from microphone transducer) Bandlimited analog audio signal Sampled data signal Analog audio output Ctstime dstamp “staricase” signal Digital signal {0100101} Digital signal {0110001} Audio DSP System Antialiasing Filter Sample and Hold Reconstruction Filter Dr. Deepa Kundur (University of Toronto) Audio DSP 29 56 Audio Digital Signal Processing Analog and Digital Audio Digitizing Audio AD Processing for Transmission Storage DA Analog audio input (from microphone transducer) Bandlimited analog audio signal Sampled data signal Analog audio output Ctstime dstamp “staricase” signal Digital signal {0100101} Digital signal {0110001} Audio DSP System Antialiasing Filter Sample and Hold Reconstruction Filter Antialiasing Filter: I ensures that analog audio input does not contain frequency components higher than half of the sampling frequency (to avoid aliasing) I Example: C6713 DSP, Fs = 8 kHz, therefore antialiasing filter must have a passband of 0 Hz to 4000 Hz. Dr. Deepa Kundur (University of Toronto) Audio DSP 30 56 Audio Digital Signal Processing Analog and Digital Audio Digitizing Audio t 2 3 2 1 1 2 3 4 2 4 Input Signal t 2 3 2 1 1 2 3 4 2 4 Antialiased Signal Dr. Deepa Kundur (University of Toronto) Audio DSP 31 56 Audio Digital Signal Processing Analog and Digital Audio Digitizing Audio AD Processing for Transmission Storage DA Analog audio input (from microphone transducer) Bandlimited analog audio signal Sampled data signal Analog audio output Ctstime dstamp “staricase” signal Digital signal {0100101} Digital signal {0110001} Audio DSP System Antialiasing Filter Sample and Hold Reconstruction Filter Sample and Hold: I holds a sampled analog audio value for a short time while the AD converts and interprets the value as a digital Dr. Deepa Kundur (University of Toronto) Audio DSP 32 56Audio Digital Signal Processing Analog and Digital Audio Digitizing Audio t 2 3 2 1 1 2 3 4 2 4 1 2 0 x(t) Antialiased Signal t 2 3 2 1 1 2 3 4 2 4 Sampled Data Signal antialiased signal Dr. Deepa Kundur (University of Toronto) Audio DSP 33 56 Audio Digital Signal Processing Analog and Digital Audio Digitizing Audio AD Processing for Transmission Storage DA Analog audio input (from microphone transducer) Bandlimited analog audio signal Sampled data signal Analog audio output Ctstime dstamp “staricase” signal Digital signal {0100101} Digital signal {0110001} Audio DSP System Antialiasing Filter Sample and Hold Reconstruction Filter AD: I converts a sampled data audio value into a digital number, in part, through quantization of the amplitude Dr. Deepa Kundur (University of Toronto) Audio DSP 34 56 Audio Digital Signal Processing Analog and Digital Audio Digitizing Audio t 2 3 2 1 1 2 3 4 2 4 Sampled Data Signal antialiased signal t 2 3 2 1 1 2 3 4 2 4 Digital Signal sampled data signal Dr. Deepa Kundur (University of Toronto) Audio DSP 35 56 Audio Digital Signal Processing Analog and Digital Audio Digitizing Audio AD Processing for Transmission Storage DA Analog audio input (from microphone transducer) Bandlimited analog audio signal Sampled data signal Analog audio output Ctstime dstamp “staricase” signal Digital signal {0100101} Digital signal {0110001} Audio DSP System Antialiasing Filter Sample and Hold Reconstruction Filter Processing for TransmissionStorage: I transmissionstorage contains inherent nonidealities that cause errors in the receivedretrieved data symbols I error correction coding (ECC) is employed to add redundancy to the digital signal so that errors can be compensated for during decoding Dr. Deepa Kundur (University of Toronto) Audio DSP 36 56Audio Digital Signal Processing Analog and Digital Audio Error Correction Coding Example: Nrepetition code Input Signal Bit Coded Sequence 0 0 0 0 · ·· 0 | {z } N zeros 1 1 1 1 · · · 1 | {z } N ones Therefore, for N = 3 the following input signal sequence: 0 0 1 would be coded as follows: 0 0 0 0 0 0 1 1 1: Dr. Deepa Kundur (University of Toronto) Audio DSP 37 56 Audio Digital Signal Processing Analog and Digital Audio Error Correction Coding Q: How would you interpret receiving the following coded sequence (with possible error): 1 1 1 0 1 0 0 0 0? 1 1 1 | {z } 1 0 1 0 | {z } 0 0 0 0 | {z } 0 A: Decoding can make use of majority vote logic. Dr. Deepa Kundur (University of Toronto) Audio DSP 38 56 Audio Digital Signal Processing Analog and Digital Audio Error Correction Coding Coder for N = 3: Input Signal Bit Coded Sequence 0 0 0 0 1 1 1 1 Majority vote logic decoder for N = 3: Received Coded Seq Decoded Signal Bit 0 0 0 0 0 0 1 0 0 1 0 0 0 1 1 1 1 0 0 0 1 0 1 1 1 1 0 1 1 1 1 1 Dr. Deepa Kundur (University of Toronto) Audio DSP 39 56 Audio Digital Signal Processing Analog and Digital Audio Digitizing Audio AD Processing for Transmission Storage DA Analog audio input (from microphone transducer) Bandlimited analog audio signal Sampled data signal Analog audio output Ctstime dstamp “staricase” signal Digital signal {0100101} Digital signal {0110001} Audio DSP System Antialiasing Filter Sample and Hold Reconstruction Filter DA: I converts a digital audio signal into a staircaselike signal for further reconstruction Dr. Deepa Kundur (University of Toronto) Audio DSP 40 56Audio Digital Signal Processing Analog and Digital Audio Digitizing Audio t 2 3 2 1 1 2 3 4 2 4 1 2 0 x(t) Digital Signal sampled data signal t 2 3 2 1 1 2 3 4 2 4 Staircase Signal digital signal sampled data signal Dr. Deepa Kundur (University of Toronto) Audio DSP 41 56 Audio Digital Signal Processing Analog and Digital Audio Digitizing Audio AD Processing for Transmission Storage DA Analog audio input (from microphone transducer) Bandlimited analog audio signal Sampled data signal Analog audio output Ctstime dstamp “staricase” signal Digital signal {0100101} Digital signal {0110001} Audio DSP System Antialiasing Filter Sample and Hold Reconstruction Filter Reconstruction Filter: I converts a staircaselike signal into an analog filter through lowpass filtering I depending on the application the filter can be similar to the antialiasing filter, or may be very cheap (e.g., compact disk receivers), or may using a different sampling rate for special effects Dr. Deepa Kundur (University of Toronto) Audio DSP 42 56 Audio Digital Signal Processing Analog and Digital Audio Digitizing Audio t 2 3 2 1 1 2 3 4 2 4 Staircase Signal digital signal sampled data signal t 2 3 2 1 1 2 3 4 2 4 Reconstructed Signal antialiased signal Dr. Deepa Kundur (University of Toronto) Audio DSP 43 56 Audio Digital Signal Processing Analog and Digital Audio Digitizing Audio The quality of digitizing audio is related to the following parameters: I sampling rate (Hz) I bit depth (bitssample) and dynamic range (related to number of quantization levels) I mono vs. stereo Dr. Deepa Kundur (University of Toronto) Audio DSP 44 56Audio Digital Signal Processing Analog and Digital Audio Digitizing Audio Note: For the same cost, digital audio provides higher signaltonoise ratio or lower meansquare error between the real sound and what is recordedplayed. I It is less expensive to increase sampling rate and quantization depth (i.e., reduce quantization noise) than to use less noisy analog circuitry (i.e., reduce noise floor) I When signals are represented digitally the natural noise in the circuits can be circumvented via error correction coding. Thus, it is possible to have near perfect storagetransmission. Dr. Deepa Kundur (University of Toronto) Audio DSP 45 56 Audio Digital Signal Processing Audio Quality Audio Quality and Sampling Rate Audio Quality as a Function of Sampling Rate: Sampling Rate (Hz) Quality Similar to 8,000 telephone 11,025 AM radio 22,050 FM radio 44,100 CD 48,000 DAT Dr. Deepa Kundur (University of Toronto) Audio DSP 46 56 Audio Digital Signal Processing Audio Quality Audio Quality, Sampling Rate, and Bit Depth Audio Quality as a Function of Sampling Rate, Bit Depth and StereoMonophony: Sampling Rate (Hz) Bit Depth StereoMono Quality 8,000 8 mono telephone 11,025 8 stereo low 22,050 8 stereo · 22,050 16 mono · 22,050 16 stereo · 44,100 16 mono good 44,100 16 stereo CD quality Dr. Deepa Kundur (University of Toronto) Audio DSP 47 56 Audio Digital Signal Processing Audio Quality Audio Quality Q: Why do some people insist that analog audio is superior to digital audio? A: What they think sounds good isn’t the exact original sound, but a nonlinearly distorted version generated from the analog components. Note: Some digital audio companies now make digital amplifiers that mimic the distortion from analog audio amplifiers. Quality of audio is a qualitative and psychological measure that is userspecific. Dr. Deepa Kundur (University of Toronto) Audio DSP 48 56Audio Digital Signal Processing Audio Equalizers Audio Equalization I Equalization ≡ Equalisation ≡ EQ I amplifying or attenuation different frequency components of an audio signal I Example: basstreble control in inexpensive car radios I Common goals of equalization: I provide fine granularity of frequency amplificationattenuation control without affecting adjacent frequencies. I correct for unwanted frequency attenuationamplification during recording processes I enhancing the presence of certain sounds I reducing the presence of unwanted signals such as noise Dr. Deepa Kundur (University of Toronto) Audio DSP 49 56 Audio Digital Signal Processing Audio Equalizers Equalizer Design Basics 1. Determine the processing band of your audio signal. I human audible range is: 20 Hz to 20 kHz I if sampling rate of a DSP is Fs then, the bandwidth of the audio signal to process is: 20 to F2s Hz I Example: Fs = 16; 000 Hz 1 8000 20 20 8000 Dr. Deepa Kundur (University of Toronto) Audio DSP 50 56 Audio Digital Signal Processing Audio Equalizers Equalizer Design Basics 2. Determine the granularity of your equalizer (i.e., number of frequency bands to independently control). I one approach might be to equally partition the audio signal bandwdith I more popular approaches suited to human auditory system models have bands that increase in width by two I Example: 3 frequency bands 8000 3000 1000 20 20 1000 3000 8000 Dr. Deepa Kundur (University of Toronto) Audio DSP 51 56 Audio Digital Signal Processing Audio Equalizers Equalizer Design Basics 3. Design your bandpass filters. I each bandpass filter is independently setcontrolled from the others I ideally, many people would like shelving EQ I Example: Ideal bandpass filters 1 8000 3000 1000 20 20 1000 3000 8000 Dr. Deepa Kundur (University of Toronto) Audio DSP 52 56Audio Digital Signal Processing Audio Equalizers Equalizer Design Basics 3. Design your bandpass filters. I each bandpass filter is independently setcontrolled from the others I ideally, many people would like shelving EQ I Example: Bell EQ 1 8000 3000 1000 20 20 1000 3000 8000 Dr. Deepa Kundur (University of Toronto) Audio DSP 53 56 Audio Digital Signal Processing Audio Equalizers Common Types of Equalizers I All bell filters and many other bandpass filters can be characterized by three parameters: I center frequency I width of the bell curve I gain (i.e. peak) of the bell curve 1 8000 3000 1000 20 20 1000 3000 8000 width peak amplitude center frequency Dr. Deepa Kundur (University of Toronto) Audio DSP 54 56 Audio Digital Signal Processing Audio Equalizers Common Types of Equalizers I Parametric Equalizers: the center frequency, passband width and peak amplitude can be independently selected for each filter I most powerful EQ, predominantly used for recording and mixing I Graphic Equalizers: the center frequency and passband width of each filter are preset; the gains of each filter can be independently controlled I used for live applications such as concerts Dr. Deepa Kundur (University of Toronto) Audio DSP 55 56 Audio Digital Signal Processing Audio Equalizers Common Types of Equalizers I Notch Filters: the passband width is small and fixed for each filter; center frequencies and gains are variable. I used in multimedia applicationsaudio mastering Dr. Deepa Kundur (University of Toronto) Audio DSP 56 56
Intro to Audio Signals Amplitude and Loudness Sound Sound: vibration transmitted through a medium (gas, liquid, solid and plasma) composed of frequencies capable of being detected by ears Audio DSP Note: sound cannot travel through a vacuum Dr Deepa Kundur Human detectable sound is often characterized by air pressure variations detected by the human ear The amplitude, frequency and relative phase of the air pressure signal components determine (in part) the way the sound is perceived University of Toronto Dr Deepa Kundur (University of Toronto) Audio DSP Intro to Audio Signals / 56 Dr Deepa Kundur (University of Toronto) Amplitude and Loudness Intro to Audio Signals Sinusoids and Sound: Amplitude / 56 Amplitude and Loudness Sound Volume Volume = Amplitude of sound waves/audio signals quoted in dB, which is a logarithmic measure; 10 log(A2 ) A fundamental unit of sound is the sinusoidal signal xa (t) = A cos(2πF0 t + θ), Audio DSP no sound/null is −∞ dB t∈R Loudness is a subjective measure of sound psychologically correlating to the strength of the sound signal the volume is an objective measure and does not have a one-to-one correspondence with loudness perceived loudness varies from person-to-person and depends on frequency and duration of the sound A ≡ volume F0 ≡ pitch (more on this ) θ ≡ phase (more on this ) Dr Deepa Kundur (University of Toronto) Audio DSP / 56 Dr Deepa Kundur (University of Toronto) Audio DSP / 56 Intro to Audio Signals Amplitude and Loudness Intro to Audio Signals Music Volume Dynamic Range Frequency and Pitch Sinusoids and Sound: Frequency Tests conducted for the musical note: C6 (F0 = 1046.502 Hz) A fundamental unit of sound is the sinusoidal signal Dynamic Level Threshold of hearing ppp (pianissimo) p (piano) f (forte) fff (fortississimo) Threshold of pain Dr Deepa Kundur (University of Toronto) Decibels 40 60 80 100 120 Audio DSP Intro to Audio Signals xa (t) = A cos(2πF0 t + θ), A ≡ volume F0 ≡ pitch θ ≡ phase (more on this ) / 56 Dr Deepa Kundur (University of Toronto) Frequency and Pitch Audio DSP Intro to Audio Signals Pure Frequency / 56 Frequency and Pitch Tuning Forks A tuning fork is a two-pronged instrument that is an acoustic resonator It is usually made out of steel and resonates at a specific constant pitch which is a function of the length of the prongs Q: What type of sound does a pure frequency produce? A: A pure tone with a single pitch Q: Can any instrument produce a pure tone by playing a single note? A: No Dr Deepa Kundur (University of Toronto) t∈R Audio DSP / 56 Striking the tuning fork will produce the required sounds although initially there may be overtones that die out quickly A very common tuning fork used by musicians produces the A note (F0 = 440 Hz), which is international concert pitch used to tune orchestras Dr Deepa Kundur (University of Toronto) Audio DSP / 56 Intro to Audio Signals Frequency and Pitch Intro to Audio Signals Frequency and Pitch Harmonically Related Frequencies and Pitch Scientific Designation C1 C2 C3 C4 (middle C) C5 C6 C7 C8 Sinusoids can be represented either as: t∈R xa (t) = A cos(2πF0 t + θ), or for mathematical convenience when interpreting as Fourier signal components as: xa (t) = Ae j(2πF0 t+θ) , t∈R Pitch is directly related to the frequency F0 To be able to hear a frequency F0 , it has to be in the human audible range Dr Deepa Kundur (University of Toronto) Frequency and Pitch Audio DSP Intro to Audio Signals C1 / 56 Frequency and Pitch C2 C3 Dr Deepa Kundur (University of Toronto) Frequency (Hz) 32.703 65.406 130.813 261.626 523.251 1046.502 2093.005 4186.009 C4 C5 C6 C7 Audio DSP Intro to Audio Signals Harmonically Related Frequencies k for F0 = 8.176 16 32 64 128 256 512 C8 10 / 56 Frequency and Pitch Signature Sounds Recall harmonically related sinusoids have the following analytic form for k ∈ Z: xa,k (t) = A cos(2πkF0 t + θ) Q: If two different people sing the same note or two different instruments play the same note, why they sound different? or xa,k (t) = Ae j(2πkF0 t+θ) The notes are not pure tones There are natural overtones and undertones that provide distinguishing signatures that can be viewed in the associated spectra They are used in the context of the Fourier Series to build periodic signals: ∞ X (k)e j(2πkF0 t) x(t) = k=−∞ Dr Deepa Kundur (University of Toronto) Audio DSP 11 / 56 Dr Deepa Kundur (University of Toronto) Audio DSP 12 / 56 Intro to Audio Signals Frequency and Pitch Intro to Audio Signals Frequency and Pitch Human Audible Range Fourier Transforms of the Same Note Tuning Fork f Hearing is usually limited to frequencies between 20 Hz and 20 kHz The upper limit decreases with age Instrument A f The audible frequency range is different for animals Instrument B f Dr Deepa Kundur (University of Toronto) Audio DSP Intro to Audio Signals 13 / 56 Frequency and Pitch Audio DSP Intro to Audio Signals Animal Audible Range Species human dog rabbit bat goldfish Dr Deepa Kundur (University of Toronto) 14 / 56 Phase and Sound Sinusoids and Sound: Phase Approx Range (Hz) 20 - 20,000 67 - 45,000 360 - 42,000 2,000 - 110,000 20 - 3,000 A fundamental unit of sound is the sinusoidal signal xa (t) = A cos(2πF0 t + θ), t∈R A ≡ volume F0 ≡ pitch θ ≡ phase Reference: R.R Fay (1988), Hearing in Vertebrates: A Psychophysics Databook Dr Deepa Kundur (University of Toronto) Audio DSP 15 / 56 Dr Deepa Kundur (University of Toronto) Audio DSP 16 / 56 Intro to Audio Signals Phase and Sound Intro to Audio Signals Phase and Sound Phase and Sound Phase and Sound Consider a general sound signal x(t) that is comprised of frequency components each with a specific phase shift ∞ X (f )e j2πf t df x(t) = If x(t) is the general sound signal, then x(−t) is the sound signal in reverse −∞ Q: Do x(t) and x(−t) sound similar? A: No |X (f )|: relative volume of a sinusoidal component ∠X (f ): relative phase of a sinusoidal component Dr Deepa Kundur (University of Toronto) Audio DSP Intro to Audio Signals 17 / 56 Phase and Sound Dr Deepa Kundur (University of Toronto) Audio DSP Intro to Audio Signals Phase and Sound 18 / 56 Phase and Sound Phase and Sound Taking the magnitude and phase of both sides we have: Recall, from the continuous-time Fourier transform (CTFT) that for a real signal x(t): F x(t) ←→ X (f ) F X (f ) = X ∗ (−f ) |X (f )| = |X ∗ (−f )| = |X (−f )| ∠X (f ) = ∠X ∗ (−f ) = −∠X (−f ) x(−t) ←→ X (−f ) and Conjugate Symmetry (for real signals x(t)): X (f ) = X ∗ (−f ) Dr Deepa Kundur (University of Toronto) Audio DSP CTFT magnitude is even CTFT phase is odd 19 / 56 Dr Deepa Kundur (University of Toronto) Audio DSP 20 / 56 Intro to Audio Signals Phase and Sound Intro to Audio Signals Phase and Sound Auditory Masking Auditory Masking Therefore, for F x(t) ←→ X (f ) occurs when the perceived quality of one (primary) sound is affected by the presence of another (secondary) sound F x(−t) ←→ X (−f ) |X (f )| = |X (−f )| ⇒ the CTFT magnitudes for forward and reverse sound signals are exactly the same ∠X (f ) = ∠X (−f ) ⇒ the CTFT phases for forward and reverse sound signals are different Simultaneous masking: the secondary sound is heard at the same time as the primary sound Can be exploited (as we see in an upcoming lab) to mask non-ideal signal processing Therefore, the relative phase of the sinusoidal components of sound contains very salient perceptual information much like for images Dr Deepa Kundur (University of Toronto) Audio DSP Audio Digital Signal Processing 21 / 56 Analog and Digital Audio Audio DSP Audio Digital Signal Processing Why Digitize Audio? 22 / 56 Analog and Digital Audio Benefits of Digital Audio Fidelity of digital audio is much higher than analog audio Manipulation tools for digital audio are much more sophisticated than those available for analog audio Compression of digital audio provides significantly reduced storage requirements Storage of digital audio (e.g., CDs) are much more convenient and compact Duplication of digital audio is exact in contrast to analog audio Dr Deepa Kundur (University of Toronto) Dr Deepa Kundur (University of Toronto) Audio DSP 23 / 56 Convenient recording, enhancement, mass-production and distribution CDs, online stores such as iTunes, etc data files are distributed instead of physical media storing the information such as records and tapes Dr Deepa Kundur (University of Toronto) Audio DSP 24 / 56 Audio Digital Signal Processing Analog and Digital Audio Audio Digital Signal Processing Concerns about Digital Audio Analog and Digital Audio Analog vs Digital Audio: Analog Audio System Transducer (e.g., microphone) Analog audio signal Convenient recording, enhancement, mass-production and distribution unlawful manipulation of recorded audio is difficult to detect piracy: unlawful copying and redistribution of copyrighted content Transmission/ Storage Loudspeaker microphone: converts sound into an electrical signal; air pressure → motion of conductor/coil → magnetic field → electrical signal loudspeaker: converts electrical signal into acoustic waves; electrical signal → magnetic field → motion → air pressure Dr Deepa Kundur (University of Toronto) Audio DSP Audio Digital Signal Processing 25 / 56 Dr Deepa Kundur (University of Toronto) Analog and Digital Audio Audio Digital Signal Processing Analog vs Digital Audio: Analog Audio System Transducer A/D Converter Analog audio signal Transmission/ Storage Loudspeaker capacitance and inductance of the circuits limit bandwidth, and resistance limits amplitude Audio DSP Analog and Digital Audio Error Correction Coding (ECC) Digital audio signal 27 / 56 Transmission/ Storage ECC Decoding D/A Converter associated circuits suffer from inherent noise (noise floor) Dr Deepa Kundur (University of Toronto) 26 / 56 Analog vs Digital Audio: Digital Audio Chain (e.g., microphone) Analog audio signal Audio DSP fidelity limited by quantization noise bandwidth limited by sampling rate dynamic range limited by bit resolution Dr Deepa Kundur (University of Toronto) Audio DSP 28 / 56 Audio Digital Signal Processing Analog and Digital Audio Audio Digital Signal Processing Digitizing Audio Digitizing Audio Audio DSP System Antialiasing Filter Analog audio input (from microphone transducer) Analog and Digital Audio Audio DSP System Sample and Hold Bandlimited analog audio signal Processing for Transmission/ Storage A/D Sampled data signal Digital signal {0100101} Antialiasing Filter Reconstruction D/A Filter Digital Cts-time dst-amp signal “staricase” signal {0110001} Analog audio output Analog audio input (from microphone transducer) Sample and Hold Bandlimited analog audio signal Processing for Transmission/ Storage A/D Sampled data signal Digital signal {0100101} D/A Reconstruction Filter Digital Cts-time dst-amp signal “staricase” signal {0110001} Analog audio output Anti-aliasing Filter: ensures that analog audio input does not contain frequency components higher than half of the sampling frequency (to avoid aliasing) Example: C6713 DSP, Fs = kHz, therefore anti-aliasing filter must have a passband of Hz to 4000 Hz Dr Deepa Kundur (University of Toronto) Audio DSP Audio Digital Signal Processing 29 / 56 Analog and Digital Audio 30 / 56 Analog and Digital Audio Digitizing Audio Input Signal Audio DSP System Antialiasing Filter -3 Audio DSP Audio Digital Signal Processing Digitizing Audio -4 Dr Deepa Kundur (University of Toronto) -2 -1 t Analog audio input (from microphone transducer) Sample and Hold Bandlimited analog audio signal A/D Sampled data signal Processing for Transmission/ Storage Digital signal {0100101} D/A Reconstruction Filter Digital Cts-time dst-amp signal “staricase” signal {0110001} Analog audio output -2 Anti-aliased Signal Sample and Hold: holds a sampled analog audio value for a short time while the A/D converts and interprets the value as a digital -4 -3 -2 -1 t -2 Dr Deepa Kundur (University of Toronto) Audio DSP 31 / 56 Dr Deepa Kundur (University of Toronto) Audio DSP 32 / 56 Audio Digital Signal Processing Analog and Digital Audio Audio Digital Signal Processing Digitizing Audio Digitizing Audio Anti-aliased Signal Audio DSP System Antialiasing Filter -4 -3 Analog and Digital Audio -2 -1 t Analog audio input (from microphone transducer) Sample and Hold Bandlimited analog audio signal Processing for Transmission/ Storage A/D Sampled data signal Digital signal {0100101} D/A Reconstruction Filter Digital Cts-time dst-amp signal “staricase” signal {0110001} Analog audio output -2 Sampled Data Signal anti-aliased signal -4 -3 -2 A/D: converts a sampled data audio value into a digital number, in part, through quantization of the amplitude -1 t -2 Dr Deepa Kundur (University of Toronto) Audio DSP Audio Digital Signal Processing 33 / 56 Dr Deepa Kundur (University of Toronto) Analog and Digital Audio Audio Digital Signal Processing Digitizing Audio Analog and Digital Audio -2 Audio DSP System Antialiasing Filter anti-aliased signal -3 34 / 56 Digitizing Audio Sampled Data Signal -4 Audio DSP -1 t Analog audio input (from microphone transducer) Sample and Hold Bandlimited analog audio signal A/D Sampled data signal Processing for Transmission/ Storage Digital signal {0100101} D/A Reconstruction Filter Digital Cts-time dst-amp signal “staricase” signal {0110001} Analog audio output -2 Digital Signal -4 -3 -2 Processing for Transmission/Storage: transmission/storage contains inherent non-idealities that cause errors in the received/retrieved data symbols error correction coding (ECC) is employed to add redundancy to the digital signal so that errors can be compensated for during decoding sampled data signal -1 t -2 Dr Deepa Kundur (University of Toronto) Audio DSP x(t) 35 / 56 Dr Deepa Kundur (University of Toronto) Audio DSP 36 / 56 Audio Digital Signal Processing Analog and Digital Audio Audio Digital Signal Processing Analog and Digital Audio Error Correction Coding Error Correction Coding Example: N-repetition code Q: How would you interpret receiving the following coded sequence (with possible error): Input Signal Bit Coded Sequence 0 0 ··· 1 1 0 0? N zeros 1 ··· 1 111 010 000 N ones Therefore, for N = the following input signal sequence: 0 would be coded as follows: 0 0 0 1 Dr Deepa Kundur (University of Toronto) Audio DSP Audio Digital Signal Processing 37 / 56 A: Decoding can make use of majority vote logic Dr Deepa Kundur (University of Toronto) Analog and Digital Audio Audio DSP Audio Digital Signal Processing Error Correction Coding 38 / 56 Analog and Digital Audio Digitizing Audio Coder for N = 3: Audio DSP System Input Signal Bit Coded Sequence 000 111 Antialiasing Filter Analog audio input (from Majority vote logic decoder for N = 3: Received 0 0 1 1 Dr Deepa Kundur (University of Toronto) Coded Seq 00 01 10 11 00 01 10 11 microphone transducer) Decoded Signal Bit 0 1 1 Audio DSP Sample and Hold Bandlimited analog audio signal A/D Sampled data signal Processing for Transmission/ Storage Digital signal {0100101} D/A Reconstruction Filter Digital Cts-time dst-amp signal “staricase” signal {0110001} Analog audio output D/A: converts a digital audio signal into a “staircase”-like signal for further reconstruction 39 / 56 Dr Deepa Kundur (University of Toronto) Audio DSP 40 / 56 Audio Digital Signal Processing Analog and Digital Audio Audio Digital Signal Processing Digitizing Audio Digitizing Audio Digital Signal -4 -3 Analog and Digital Audio -2 Audio DSP System Antialiasing Filter sampled data signal -1 t Analog audio input (from microphone transducer) Sample and Hold Bandlimited analog audio signal Processing for Transmission/ Storage A/D Sampled data signal Digital signal {0100101} D/A Reconstruction Filter Digital Cts-time dst-amp signal “staricase” signal {0110001} Analog audio output -2 Staircase Signal Reconstruction Filter: converts a “staircase”-like signal into an analog filter through lowpass filtering depending on the application the filter can be similar to the anti-aliasing filter, or may be very cheap (e.g., compact disk receivers), or may using a different sampling rate for special effects digital signal -4 -3 -2 -1 -2 Dr Deepa Kundur (University of Toronto) t sampled data signal Audio DSP Audio Digital Signal Processing Dr Deepa Kundur (University of Toronto) 41 / 56 Analog and Digital Audio Audio DSP Audio Digital Signal Processing Digitizing Audio 42 / 56 Analog and Digital Audio Digitizing Audio Staircase Signal digital signal -4 -3 -2 -1 -2 The “quality” of digitizing audio is related to the following parameters: sampling rate (Hz) bit depth (bits/sample) and dynamic range (related to number of quantization levels) mono vs stereo t sampled data signal Reconstructed Signal -4 -3 -2 -1 anti-aliased signal t -2 Dr Deepa Kundur (University of Toronto) Audio DSP x(t) 43 / 56 Dr Deepa Kundur (University of Toronto) Audio DSP 44 / 56 Audio Digital Signal Processing Analog and Digital Audio Audio Digital Signal Processing Audio Quality Digitizing Audio Audio Quality and Sampling Rate Note: For the same cost, digital audio provides higher signal-to-noise ratio or lower mean-square error between the real sound and what is recorded/played Audio Quality as a Function of Sampling Rate: It is less expensive to increase sampling rate and quantization depth (i.e., reduce quantization noise) than to use less noisy analog circuitry (i.e., reduce noise floor) When signals are represented digitally the natural noise in the circuits can be circumvented via error correction coding Thus, it is possible to have near perfect storage/transmission Dr Deepa Kundur (University of Toronto) Audio DSP Audio Digital Signal Processing 45 / 56 Audio Quality Dr Deepa Kundur (University of Toronto) Audio DSP Stereo/Mono mono stereo stereo mono stereo mono stereo Audio DSP 46 / 56 Audio Quality Audio Quality Q: Why some people insist that analog audio is superior to digital audio? Audio Quality as a Function of Sampling Rate, Bit Depth and Stereo/Monophony: Bit Depth 8 16 16 16 16 Dr Deepa Kundur (University of Toronto) Audio Digital Signal Processing Audio Quality, Sampling Rate, and Bit Depth Sampling Rate (Hz) 8,000 11,025 22,050 22,050 22,050 44,100 44,100 Sampling Rate (Hz) Quality Similar to 8,000 telephone 11,025 AM radio 22,050 FM radio 44,100 CD 48,000 DAT A: What they think sounds good isn’t the exact original sound, but a nonlinearly distorted version generated from the analog components Quality telephone low · · · good CD quality Note: Some digital audio companies now make digital amplifiers that mimic the distortion from analog audio amplifiers Quality of audio is a qualitative and psychological measure that is user-specific 47 / 56 Dr Deepa Kundur (University of Toronto) Audio DSP 48 / 56 Audio Digital Signal Processing Audio Equalizers Audio Digital Signal Processing Audio Equalization Equalizer Design Basics Equalization ≡ Equalisation ≡ EQ Determine the processing band of your audio signal amplifying or attenuation different frequency components of an audio signal Example: bass/treble control in inexpensive car radios Common goals of equalization: provide fine granularity of frequency amplification/attenuation control without affecting adjacent frequencies correct for unwanted frequency attenuation/amplification during recording processes enhancing the presence of certain sounds reducing the presence of unwanted signals such as noise Dr Deepa Kundur (University of Toronto) Audio Equalizers Audio DSP Audio Digital Signal Processing 49 / 56 human audible range is: 20 Hz to 20 kHz if sampling rate of a DSP is Fs then, the bandwidth of the audio signal to process is: 20 to F2s Hz Example: Fs = 16, 000 Hz -8000 -20 Dr Deepa Kundur (University of Toronto) Audio Equalizers 8000 Audio DSP Audio Digital Signal Processing Equalizer Design Basics 20 50 / 56 Audio Equalizers Equalizer Design Basics Determine the granularity of your equalizer (i.e., number of frequency bands to independently control) Design your bandpass filters each bandpass filter is independently set/controlled from the others ideally, many people would like shelving EQ Example: Ideal bandpass filters one approach might be to equally partition the audio signal bandwdith more popular approaches suited to human auditory system models have bands that increase in width by two Example: frequency bands -8000 -8000 Dr Deepa Kundur (University of Toronto) -3000 -1000 -20 Audio DSP 20 1000 3000 -3000 -1000 -20 20 1000 3000 8000 8000 51 / 56 Dr Deepa Kundur (University of Toronto) Audio DSP 52 / 56 Audio Digital Signal Processing Audio Equalizers Audio Digital Signal Processing Equalizer Design Basics Audio Equalizers Common Types of Equalizers All bell filters and many other bandpass filters can be characterized by three parameters: Design your bandpass filters each bandpass filter is independently set/controlled from the others ideally, many people would like shelving EQ Example: Bell EQ center frequency width of the bell curve gain (i.e peak) of the bell curve center frequency -8000 -3000 Dr Deepa Kundur (University of Toronto) -1000 -20 20 1000 3000 Audio DSP Audio Digital Signal Processing -8000 8000 53 / 56 Audio Equalizers -3000 Dr Deepa Kundur (University of Toronto) -1000 -20 20 1000 3000 8000 Audio DSP Audio Digital Signal Processing Common Types of Equalizers peak amplitude width 54 / 56 Audio Equalizers Common Types of Equalizers Parametric Equalizers: the center frequency, passband width and peak amplitude can be independently selected for each filter most powerful EQ, predominantly used for recording and mixing Graphic Equalizers: the center frequency and passband width of each filter are pre-set; the gains of each filter can be independently controlled Notch Filters: the passband width is small and fixed for each filter; center frequencies and gains are variable used in multimedia applications/audio mastering used for live applications such as concerts Dr Deepa Kundur (University of Toronto) Audio DSP 55 / 56 Dr Deepa Kundur (University of Toronto) Audio DSP 56 / 56