XI
ImageandVideo
Processing
JanBiemond
DelftUniversityofTechnology
RussellM.Mersereau
GeorgiaInstituteofTechnology
51ImageProcessingFundamentals IanT.Young,JanJ.Gerbrands,andLucasJ.vanVliet
Introduction
•
DigitalImageDefinitions
•
Tools
•
Perception
•
ImageSampling
•
Noise
•
Cameras
•
Displays
•
Algorithms
•
Techniques
•
Acknowledgments
52StillImageCompression TorA.Ramstad
Introduction
•
SignalDecomposition
•
QuantizationandCodingStrategies
•
FrequencyDomain
Coders
•
FractalCoding
•
ColorCoding
53ImageandVideoRestoration A.MuratTekalp
Introduction
•
Modeling
•
ModelParameterEstimation
•
Intra-FrameRestoration
•
Multiframe
RestorationandSuperresolution
•
Conclusion
54VideoScanningFormatConversionandMotionEstimation GerarddeHaan
Introduction
•
Conversionvs.Standardization
•
ProblemswithLinearSamplingRateConversion
AppliedtoVideoSignals
•
AlternativesforSamplingRateConversionTheory
•
MotionEstimation
•
MotionEstimationandScanningFormatConversion
55VideoSequenceCompression OsamaAl-Shaykh,RalphNeff,DavidTaubman,and
AvidehZakhor
Introduction
•
MotionCompensatedVideoCoding
•
DesirableFeatures
•
Standards
56DigitalTelevision Kou-HuTzou
Introduction
•
EDTV/HDTVStandards
•
HybridAnalog/DigitalSystems
•
ErrorProtectionand
Concealment
•
TerrestrialBroadcasting
•
SatelliteTransmission
•
ATMTransmissionofVideo
57StereoscopicImageProcessing ReginaldL.Lagendijk,RuggeroE.H.Franich,andEmile
A.Hendriks
Introduction
•
AcquisitionandDisplayofStereoscopicImages
•
DisparityEstimation
•
Compres-
sionofStereoscopicImages
•
IntermediateViewpointInterpolation
58ASurveyofImageProcessingSoftwareandImageDatabases StanleyJ.Reeves
ImageProcessingSoftware
•
ImageDatabases
59VLSIArchitecturesforImageCommunications P.PirschandW.Gehrke
c
1999byCRCPressLLC
Introduction
•
Recent Coding Schemes
•
Architectural Alternatives
•
Efficiency Estimation of Al-
ternative VLSI Implementations
•
Dedicated Architectures
•
Programmable Architectures
•
Con-
clusion
I
MAGE AND VIDEO SIGNAL PROCESSING is quite different from other forms of signal pro-
cessing for a variety of reasons. The most obvious difference lies in the fact that these signals
are two or three dimensional. This means that some familiar techniques used for processing
one-dimensional signals, for example, those that require factorization of polynomials, have to be
abandoned. Other techniques for filtering, sampling, and transform computation have to be mod-
ified. Even more compromises have to be made, however, because of the signals’ size. Images and
sequences of images can be huge. For example, processing sequences of color images each of which
contains 780 rows and 1024 columns at a fr ame rate of 30 frames per second requires a data rate
of 72 megabytes per second. Successful image processing techniques reward careful attention to
problemrequirements,algorithmiccomplexity,andmachinearchitecture. The pastdecadehasbeen
particularly exciting as each new wave of faster computing hardware has opened the door to new
applications. This is a trend that will likely continue for some time.
The following chapters, w ritten by experts in their fields, highlight the state-of-the-art in several
aspects of image and video processing. The range of topics is quite broad. While it includes some
discussionsoftechniquesthatgobackmorethanadecade,theemphasisisoncurrentpractice. There
is some danger in this, because the field is changing veryrapidly,but,ontheotherhand,many of the
concepts on which these current techniques are based should be around for some time.
Chapter 51 is a very long and thorough discussion of image processing fundamentals. For a
novice to the field, this material is important for a complete understanding. It discusses the basics
of how images differ from other types of signals and how the limitations of cameras, displays, and
the human visual system affect the kinds of processing that can be done. It also defines the basic
theory of multidimensional digital signal processing, particularly with respect to how linear and
nonlinearfiltering, transformcomputation,andsamplingaregeneralizedfromtheone-dimensional
case. Other topics treated include statistical models for images, models for recording distortions,
histogram-based methods for image processing, and image segmentation.
Probablythemostvisibleimageprocessingisoccurringinthedevelopmentofstandardsforimage
and video compression. JPEG, MPEG, and digital television are all highly visible success stories.
Chapter52looksatmethodsforstillimagecompressionincludingJPEG,wavelet, andfractalcoders.
Imagecompressionissuccessful because image samples arespatiallycorrelated with their neighbors.
Operatorssuchasthediscretecosinetransform(DCT)largelyremovethiscorrelationandcapturethe
essenceofanimageblockinafewparametersthatcanbequantizedandtransmitted. The transform
domain also enables these coders to exploit limitations in the human visual system. Chapters 55 and
56 extend these approaches to video and television compression, respectively. Video compression
achieves significant additional compression gains by exploiting the temporal redundancy that is
presentin videosequences. Thisisdonebyusingsimplemodelsformodelingobjectmotionwithina
scene,usingthesemodelstopredictthecurrentframe,andthenencodingonlythemodelparameters
and the quantized prediction errors.
Images are often distor ted when they are recorded. This might be caused by out-of-focus optics,
motion blur, camera noise, or coding errors. Chapter 53 looks at methods for image and v ideo
restoration. This is the most mathematically based area of image processing, and it is also one of the
areas with the longest history. It has applications in the analysis of astronomical images, in forensic
c
1999 by CRC Press LLC
imaging, and in the production of high-quality stills from video sequences.
Chapter54looksatmethodsformotionestimationandvideoscanconversion. Motionestimation
is a key technique for removing temporal redundancy in image sequences and, as a result, it is a key
component in all of the video compression standards. It is also, however, a highly time-consuming
numerically ill-posed operation. As a result it continues to be highly studied, particularly with
respect to more sophisticated motion models. A related problem is the problem of scanning format
conversion. Thisisamajorissueintelevisionsystemswherebothinterlacedandprogressivelyscanned
images are encountered.
Chapter 57 explores stereoscopic and multiview image processing. Traditional image processing
assumes that only one camera is present. As a result depth information in a three-dimensional scene
is lost. When explicit depth information is needed, multiple cameras can be used. Differences in the
displacementofobjects in the left and right images can be converted to depth measurements. Mam-
mals do this naturally with their two eyes. Stereoscopic image processing techniques are becoming
increasinglyusedinproblemsofcomputervisionandcomputergraphics. This chapterdiscussesthe
state-of-the-art in this emerging area.
The final two chapters in this section, Chapters 58 and 59, look at software and hardware systems
fordoingimageprocessing. Chapter58providesanoverviewofarepresentativesetofimagesoftware
packages that embody the core capabilities required by many image processing applications. It also
providesalistofInternetaddressesforanumberofimagedatabases. Chapter59providesanoverview
of VLSI architectures for implementing many of the video compression standards.
c
1999 by CRC Press LLC
. and 1024 columns at a fr ame rate of 30 frames per second requires a data rate
of 72 megabytes per second. Successful image processing techniques reward. chapterdiscussesthe
state-of-the-art in this emerging area.
The final two chapters in this section, Chapters 58 and 59, look at software and hardware systems
fordoingimageprocessing.