Video COdec design developing image and video compression systems iain richardson

he Freya and Hugh Copyright (02002 by John Wiley & Sons Ltd, Baffins Lane, Chichester, West Sussex PO19 IUD, England National Intemutionnl 01 243 179117 ( -1-44) 1243 779177 e-mail (for orders and customer service enquiries): cs-books@wiley.co.uk Visit our Home Page on http:Nwww.wileyeurope.coin All Rights Reserved No part of this publication may be reproduced, stored in a retrieval system, or Wimsmitted, in any forin or by any means, electronic, mechanical, photocopying, recording, scanniiig or othcrwwe, except under the terms of the Copyright, Designs and Parents Act 1988 or under the terms of a licence issued by the Copyright Licensing Agency Ltd, 90 Tottenhain Court Road, London, UK WIP OLP, without the permission in writing of the publislier Neilher the authors nor John Wiley & Sons Lld accept any responsibility or liability for loss or daniagc occasioned to any person or property through using the material, instructions, methods or ideas contained herein, or acting or refraining from acting as a result of such use The authors and disclaim all implied warranties, including merchantability of fitness for any Designations used by companies to distinguish heir prod re often claillied a8 tradcrnarks In all instances where John Wilcy & Sons IS aware of a claim, tie product names appear in initial capital or capital letters Readers, however should contact the appinpnate companies for more complete information regarding trademarks and registration Other Wiley Edzron'ul Ojzces John Wiley & Sons, Inc., 605 Thud Avenue, New York, NY 10158-0012, USA WILEY-VCH Verlag GmhH, Pappelallee 3, D-69469 Wcinheim, Geiinany John Wiley & Sons Australia Ltd, 33 Park Road, Milton Queenrland 4064, Australia John Wiley & Sons (Asia) Pte Ltd, Clement1 Loop #02-01, Jin Xing Distripark, Singapore 129809 John Wiley & Sous (Canada) Ltd, 22 Worcester Road, Rexdalc, Ontario M9W 1L1, Canada ~~~s~ Library ~ a ~ a l o ~ uin i n~g u ~ ~ i c aData ~ion A catalogue record for this book IS available from the British Lihrary ISBN 41 48553 Typeset m 10/12 'Times by Thomson Press (India) Ltd., New Delhl Printed and bound in Great Bntain by Antony Rowe Ltd, Chqpenh'm, Wiltshirc This book is printed on acid-free paper responsibly manufxctured froiu sustainable forestry, in which at least two trees are planted for each one used for paper production I Image and Video Compression 1.2 Video CODEC Design 1.3 Structure of this Book ital Intr an 2.2 Concepts, Capture and Display 2.2.1 The Video Image 2.2.2 Digital Video 2.2.3 Video Capture 2.2.4 Sampling 2.2.5 Display 2.3 Colour Spaces 2.3.1 R G B , 2.3.2 YCrCb 2.4 The Human Visual System 2.5 Video Quality 2.5.1 Subjective Quality Measurement 2.5.2 Objective Quality Measurement 2.6 Standards for Representing Digital Video 2.7 Applications 2.7.1 P.for.s Sununary eferences e 2 5 7 10 11 12 16 16 17 19 23 24 25 25 26 s 3.1 Introduction 3.1.1 Do We Need Compression? 3.2 Image and Video Compression 3.2.1 DPCM (Djfferential Pulse Code Modulation) 32.2 Transform Coding 2.3 Motion-compensated Prediction 3.2.4 Model-based Coding 3.3 ImageCOaEC 3.3.1 Transform Coding uantisation 27 28 30 31 31 32 33 33 35 CONTENTS Vi 3.3.3 Entropy Coding ding icing nsated Predicti~)n uantisation and Entropy E n c ( ~ d i.i.~.~ 3.5 3.4.4 Decoding Sumrnary 4.1 n.o ction 4.2 The ~ n ~ e r n a ~ ~~ toann~~l ~Bodies r d s 4.2.1 The Expert Groups 4.2.2 The Staiidardisation Process 4.2.3 ~ ~ i d e ~ s t a nand d i nUsing ~ the S t ~ d a r ~ .s JPEG (Joint Photographic Experts Group) 4.3.1 JPEG 4.3.2 Motion P E G 4.3.3 PEG-2000 g Picture Experts Group) -1 -2 4.4.3 ~ ~ E ~ - .4 Summary eferences s: 37 40 41 42 43 45 45 45 47 47 48 50 50 51 51 56 56 58 58 64 67 76 76 5.1 lntroduction 261 263 5.3.1 Featurcs .263 Optional ModeslH.263+ H.263 Profiles 5.5 H.26E 5.6 Perforniance of the Video Coding Standards 5.7 uni.iary e ~ ~ r e n c.e.s 79 80 80 81 81 86 87 90 91 92 93 6.1 Introduction 94 ion and Compensation ents for Motion Estimation and ompeiisa on 94 95 97 rence Energy 99 ation 102 6.4 Fast Search 102 6A.l Three-Step Search (TSS) CONTEJTI'S vii 6.4.2 Logarithic Search 6.4.3 Cross Search ation Algorithms 6.6 Sub-Fixel Motion Estimation Frames iction 6.7.2 Backwards Prediction 6.7.3 ctional Prediction 6.7.4 le Reference Frames 6.8 Enhancements to the Motion Model rest Neighbows Search nt Outside the Reference Picture lock Motion Compensation (OBMC) on Models ware Implementations ntations 6.10 S u m m y References Introduction Discrete Cosine Transform Discrete Wavelet Transform Fast Algorithms for th CT 7.4.1 Separable Tran rrls 7.4.2 Flowgraph Algorithms 7.4.3 Distributed Algorithms 4.4.4 Other DCT A ~ ~ ~ r i t .h ~ .s 7.5 I n ~ p l e ~ e n tthe i i ~DCT ~ 7.5.1 UCT 5.2 DCT uantisation ser 7.1 7.2 7.3 7.4 II ementation iantisation eferences 103 i04 105 105 107 109 111 113 113 113 113 114 115 115 115 116 116 117 117 122 125 125 127 127 133 138 138 140 144 145 146 246 148 150 152 153 156 157 160 161 8.1 ~ntroaucuon 8.2 Data Symbols 8.21 ~ u ~ - ~ e Coding v e l 163 164 164 .II CONTENTS 8.2.2 Other Symbols 8.3 Huffman Coding 8.3.1 ‘True’ 13uffman Coding 8.3.2 Moclified Huffman Coding 8.3.3 Table Design 8.3.4 Entropy Coding Example 8.3.5 Vzuiable Length Encoder Design 8.3.6 Variable Length Decoder Design 8.3.7 Dealing with Errors 8.4 Aritbnietk Coding 8.4.1 lniplementation h u e s 8.5 S u i n ~rya eferences 9.2 Pre-filtering 92.1 Camera Noise 9.2.2 CamernMovement 9.3 Post-filtering 167 169 i69 174 174 177 180 184 186 188 191 192 193 195 195 196 198 199 9.3.1 Image ~ i s ~ o ~ ~ i o .n 9.3.2 De-blocking Filters 9.3.3 De-ringing Filters 9.3.4 Error Concealment Filters Summary eferevrces., 199 206 207 208 208 209 y 211 212 212 215 217 220 226 226 228 231 232 232 uction te and Distortion 10.2.2 The hiipoitilnce of Rate Control 10.2.2 Rate-Distortion Performai~ce 10.2.3 The Kate-Dis~o~tion Problem 10.2.4 Practical Rate Controll Methods 30.3 ~ o n l p ~ ~ l a ~ iComplexity onal 10.3.1 Computational Complexity and Video Quality 10.3.2 Variable Complexity Algorithms 10.3.3 Complexity-Rate Cone01 Sumiiiary ~erences 235 s and Constraints 235 11.2.1 QoS Kequireinellts for Coded Video 11 2.2 Practical QoS Performance 11.2.3 Effect of QoS Constraints on Coded Video 235 239 241 CONTENTS iX 244 silience 244 11.3.3 Delay 247 249 EG-2 S y s t ~ m s / ~ r a n s p.o ~249 Multimedia Conferencing 252 Summary 254 ferences 255 uction 12.2 Cienerd-purpose Processors abilities tirnedia Support roceswrs rs 12.9 Summary ace 13.2.1 Video In/Out 13.2.2 Coded Data InlOut 13.2.3 Control Parmeters 2.4 Status Parameters sign of a Software CQDEC 3.1 Design Goals 13.3.2 Specification and Partitio g 13.3.3 Designing the Furictiona ocks 133.4 Improving Performance 3.5 Testing sign of a Hardware CO EC 13.4.1 Design Goals 13.4.2 Specification and Parlitioniiig 13.4.3 Designing the Functional Blocks 13.4.4 Testing Summary f ~ r ~ ~ .c e s 257 258 258 260 262 263 264 266 267 269 270 71 27 271 271 274 276 277 278 278 279 282 283 284 284 284 285 286 286 287 287 14.1 Introduction 289 VIDEO CODEC DESIGN 11 P Pir\,ch and H -J Stolbeig, 'VLSI implementations of image and video multimedia processing syctcrrts', IEEE Transaetiiwzs on Circuit5 and Sytprn,! for I'ideo Technology, (7), November 1998, pp 878-891 12 A V Wu, K R Liu, A Raghupathy and S C Liu,System Architecture of a Massively Parallel Progmmmubk Video CO-Pmcessor, Technical Report ISR 'I'R 95-34, IJniversity ol Maryla~d, 1995 This book has concentrated on the design of video CODECs that are compatible with current standards (in p ~ t ~ c ~ i l ~ , 6-4 and H.263) and on the current ‘state of the at’in video coding ~echno1o~y.l Video coding i s a fast-moving subject and cui-rent research in the field moves beyond the bounds 01‘ the iiitcrnational standards; at the same time, improvements in processing technology will soon make it possible to i~plenienttecliiqaes that were previously considered too complex This final chapter reviews trends in video coding staiidards, research and plaifornis g 011 two ru;rill area: upd MPEG organisation is at pr -4 is a large and cornplcx sta tandards and a new standard, with many fariictions an tools that go well beyond the basic H.263-like functionality of the popular ‘siiiiple profil CODEC It was originally designed with continual evolution in mind: as new t e c ~ n i ~ u eand s applications become mature, extra tools and profiles con~inue to be added to C’r-4 set of standards ent work, for example, has inch profiles that su e of the emerging Trite -based applications for MFEG-4 the more advanced elenients of G-4 (such as sprite coding and ~ n [ ~ d e l ~ ~ca s e d not yet widely used in practice, partly for reasons of coiiiplexity As these element more popular (perhaps due to increased processor capabilities), it may be ion in the standard will need to be modified and updated G 2 b ~ i i ~ on d s the coding tools PEG-4 and the content d e s c ~ p ~ itool i~n standard to provide a ‘frame for ~ ~ i u l t i communication ~ ~ e ( ~ ~ ~ ~ The c o ~ i i ~ ~has i~~ moved e e beyond the details of coding and description to an a n ~ b ~effort t i ~ ~ ~ to ~ ~ a n ~ a r aspects c ~ i s ~of the complete multimedia ”delivery chain’, from creation to ‘consumption’ (viewing or interacting with the data) This process inay include the s ~ ~ ~ d a rof~new ~ icoding s a ~ and ~ ~compression ~ ~ tools The Video ~~)~~~~ E X ~ F TGroup S of the I‘TU continues to develop the biaiidxds The recently added Aniiexes V3 W and X of H.263 are expecte major revisions to this standard The main ongoing effort has to finalise the first version of W.26L: the core tools of the standard (described in Chapter 5) are reasonably well defined, but there is further work required to convert these into a published international standard The technical aspects of 771.2611were scheduled to be finalised during 2001 ’ 290 FlJTURE DEVlXOPMENTS is now an initiative between MPEG and VCEG to jointly develop a new coding standard based on H.26L3 14.3 VIDEO CODING RESEARCH Video coding technology remains a very active area for researchers Research in this field falls into two main categories, ‘applied‘ research into the practical implementation of established video coding techniques and ‘speculative’ research into new and emerging coding algorithms As a guide to the subjects that are nmently popular in the research community, it is interesting to examine the papers presented at the 2001 Picture Coding Symposium (a specialist forum for image and video coding research) The total of 110 papers included: * 22 papers on the implementation and optimisation of the popular block DCT-based video coding standards; e 11 papers on transmission issues; * papers on quality measurement and quality metrics; e 22 papers on content-based and object-based coding (including MPEG-4 object-based coding); e papers on wavelet-based coding of video sequence; a papers on coding of 3D/multi-view video (Note that some papers were difficult to categorise.) This cross-section of topics implies that much of the current research effort focuses on practical implementation issues for the popular block-based coding standards The object-based functions of the MPEG-4 standard attract a lot of research interest and the feeling is that there are still a number of practical problems to solve (such as reliable, automatic segmentation of video scenes into video object planes) before these tools become widely adopted by the multimedia industry A surprisingly small number of papers were presented on ‘blue sky’ research into novel coding methods It is important to research and develop the next generation of video coding algorithms; at the same time, there is clearly a lot of scope for improving and optimising the current generation of coding technology 14.4 PLATFORM TRENDS Chapter 12 summarised the key features of a range of platforms for video CODEC implementation There is some evidence of convergence between some of these platforms; for example, PC processor manufacturers continue to add instructions and features that were formerly encountered in special-purposevideo or media processors However, it is likely that APPLICATION TRENDS 291 there will continue to be distinct classes of platform for video coding, possibly along the following lines: PC processors with media processing functions and increasing use of basdware co- processing (e.g in video display cards) More ‘streamlined’ processors (e.g embedded processors with internal or external multimedia support, or media processors) for ernbedded niullimedia applications edicated hardware CODECs (with limited programmability) for efficient iniplementat ~ o nof ‘mass-niaket’ applications such 2.1sdigital TV dccoding There is still a place in the market for dedicated hardware designs but at the same time there is a trend towards flexible, embedded designs for new applications such as mobile multimedia The increasing use of ‘system on a chip’ (SoC) techniques, with which a complex 1C design can be rapidly put together from Intellectual Property building blocks, should make it possible to quickly reconfigure and redesign a ‘dedicated‘ hardware CUDEC This will be necessary if dedicated designs are to continue to compete with the flexibility of embedded or general-purpose processors IC Predicting future directions for multimedia applications is notoriously difficult Few of the ‘interactive’ applications that were proposed in the early 1990s, for example, have gained a significant market presence The largest markets for video coding at present are probably digjtal television broadcasting and video (both utilising MPEG-2 coding) lnternet by the limited Internet connections experienced video is gaining popularity, but is EG-4 coding for video compression, storage and by most users There are some signs playback may experience a boom in popularity similar to PEG Layer Audio (‘MP3’ audio) However, much work needs to be done on the management and protection of intellectual property rights before this can take place Video conferencing via the Internet (typically using the H.323 protocol family) is becoming more widely used and may gain further acceptancc with increases in processor and connection performance It has yet to approach the popularity of’ communication via voice e-mail and text niessaging There are two application areas that are currently of interest to developers and communications providers, at opposite ends of the bandwidth spectrum: Very low power, very low bandwidth video for hand-held mobile devices (one or the hoped-fbr ‘killer applications’ for the costly third-generation mobile networks) The challenge here is to provide usable, low-cost video services that could match thc popularity of mobile telephony High bandwidth, high quality video coding for applimtiuns such as: (a) ‘Imiraersive’video conferencing, for example displaying conference p a ~ i c ~ ~on~ a~ n t s video ‘wall’ as if they were sitting across a table from each other The eventual goal i s a video conference meeting that is almost indistinguishable from a lace-to-face meeting 29 FUTURE D E V E L O P ~ ~ N T S gh definition television (IDTV approximately twice the resolution of ITU-R 601 ‘standard’ digital lelevision) Coding nietliods (past of MPEG-2) have been stand~disedfor several years but this technoltrgy has not yet taken hold in the marketplace (6) Digital cinema offers an alternative to the reels of projector film that are still u distributioii and display of cineina filins There i s cursenlly an effort by the committee (among o s) to develop standxd(s) to support cinema-quality coding of video and audio G’s requirements document for digital cinema4 specifies ‘visually lossless’ compression (i.e no loss should be discernible by a human observer in a movie theatre) of frames contaiiiing up to 16 million pixels at f‘rame ralcs of up to 150 Hz T n coniparison, an ITU-K 601 framc contains around 0.5 million pixels Coding and decoding at cinema fidelity are likely to be extremely demanding and wilf pose some difficult challenges for CODEC developers An interesting by-produc~of the ‘mainstream’ video coding applications and standards is the growing list of new and innovative applications for digital video Some exarnples include the use of ‘live’ video in computer games; video ‘chat’ on a large bcale with multiple participants; video surveillance in increasingly hostile environments (such as in an oil well or inside the body of a patient); 3-D video conferencing; video conferencing for groups with special requirements (for example deaf users); and many others Early experiences have taught designers of digital video app~ ica~ ~ that ons an application will only be sucGessfu1 if users find it to be a usable, useful iniproveinent over existing technology In many cases the design o€ the user interfrtce is as impor~iitas, or more iinpo~~ant than, the efiiciency of a video coding algori~~in Usabili~yi s a vital hut often overlooked requirement for any new video-based application The aim of this book has been to introduce seadess to the concepts, s t a n d ~ d sdesign , tcchniques and practical considerations behind the design of video coding and comnunication systems A questiou that i s often raised i s whether the huge worldwide effort in video coding research and dev~~opmeiit will continue to be necessary, since transi~~ssion bandw id~ smay perhaps reach the point at which compression becomes unnecessary Video and multimedia applications have only begun to make a significant impact on businesses and consumers since the late 1990s Despite continued improvements in resource^ such processing power, storage and bandwidth, these resources continue to be stretched by iiicrea~ingdemands for hi~ll-quality~ realistic ~ i i ~ l ~ icommun~cali~I~s ~~dia with more Functiona~~~y There i s still a large gap between the expectati~nsof the user aiid the capbilities of present-day video applications and this gap shows no sign of ~ i n ~ n i s ~ n g As digital video increases its share of the market, consumer demands fos ~ ighe~ - q~ ial~ ry, richer ~ u l ~ m services e d ~ ~ will ~ continue to increase the gap (providing better qualiiy and ~ i i n c t ~ o n a lwithin it~ the limits of bandwi rocessing power) requires, among other things, coritinued ~~nproveiiients in video design In the past, market researchers have overeestimnted the rate of take-up of multimedia applications such as digital TV and video conferencing aiid i t remains to be seen whether lhere i s a real demand for some of the newer video services wcli as mobile video Sorlle REFERENCES 293 interesting trends (for example, the continued popularity of MJPEG video ~ O ~ because ~ C s of their design simplicity and inherent error resilience) imply that the video coiimunications market is likely to continue to be driven more by user needs than by impressive research dcvelopmcnts This in turn implies that only some of the recent developments in vidcn coding (such as object-based coding, content-based tools, media processors and so on) will survive However, video coding will remain a core element of the growing multimedia communications market Platforms, algorithms and techniques for video coding will continue to change and evolve It is hoped that this book will help to make the subject of video CODEC design accessible to a wider auclience of designers, developers, integrators and users T Ebrahimi and M Kunt, ‘Visual data compression for multimcdia applications: an overview’, Proceedings ofthe lEEE, 86(h), June 1998 ISOlIEC JTCIISI:29/WGll N4318, ‘MPEG-21 overview’, Sydney, July 2001 iTU-’I’Q6/SGl(1VCEG-L45, ‘H.26LTest Model Long-term number (TML-6) draft O’, March 2001 ISOlIEC J ~ C l / S C ~ N433 ~ 1 , ‘Digital cinema requirements’, Sydney, July 2001 Bhaskaran, V and R Konstantinides, Iinage and Video Compression Standards: Algoritlzms and Arclritectures, Kluwer 997 Ghaiih;lri, M Wdeo Cixhg: An htroduction fo Stundcird Corlecs, IEE Press, 1999 Girocl, R., C Greaner and R Niernatin (ed5), PrincipEes of Image Analysis and SyntheJic., Rlnwer, 2000 Haskell, B., A Purl and A Netravali, Digztul Vtdeo: An Introduction to MPEG-2, C h a p a ~& ? Hall, 1996 Netravali, A aiid B Haskell, Digiluf Picturrs: R~~p~i7/"sentolion, Compression and Standunls, Plenum Press, 1995 Parhi, K R and T Nlshiimi (ed\), Drgital Signal Processing for hfultimetlzii Sy rtcms, Marcel Deklcer, 1999 Pennebalcer, W, B and J L Mitcheil, JPEG: Still Iinugcz Data Colriperrion Stcrnilut-d,Van No Reinhold, 1993 Pennebaker, W B., J I, Mitchell, C Fogg and D LcCall, MPEG Digitui Video Comprewion Stamfurd, Chapman & Hall, 1997 h r i , A and T Chen (eds), Multimediu Sysrenis, Standards and Networks, Marcel Dekker, 2000 10 Rao, K R and Hwang, Technique7 and Standnrils for Image, Video and Audio Coding, Prentrce Hall, 1997 I1 Rao, K R and P Yip, Discrele Co.cine Xrunsjorm, Acadcmic Preys, 1990 12 Riley, M.J and I G Richadon, Digital Video Conzmunicntious,t Artech Housc, Fcbruaqr 1997 This Page Intentionally Left Blank :2 :0 (sampling) : : (sampling) :4 : (sampling) APT aritlimctic coding artefact block malching blocking B-picture channcl coding chronlinance CIF CODEC colour space DCT DFD DPC DSC DVD UWT entropy coding error concealment field flowgraph full search C;OB GOP H.261 H.263 W.26L HDTV ~ u ~ i coding ~ a n saiiipling mcthod : chrominance components havc half the hoiimiital and vertical resolution of luniiimnce component sarnplriig method : chroininance components have half the horizontal resolution of luminance component sampling method : chrominmcc components have sanie resolution as luminance component application programmiiig inteitace coding method to reduce redundancy visual distortion in an imiigc binary alpha block, indicates the boundaries of a region (MPBG-4 Visual) a codec ~iiiple~ientiiig a basic set of features from a standard motion estimatioii carried out o n rectangular picture area5 square or rectangular distortion areas in an image coded prcture predicted using bidirectional motion conipensation emor control coding colour difference component comnion intermediate format, a colour image format CQderlDECodei pair method of rcprcsenting colour images discrete cosine tran $form displaced ft ame di ft'erence {residual imagc after motion compensation} differential pulse coclc modulation double shrnulus continuous quality scale, a scalc and method for suhjeciive quality meawwmxit digital versatile disk discrete wavelet transforni coding method to reducc redundancy post-processing of a decoded image to remove or reduce visible error effects odd- or cvcn-numbered 1nie.s from a video iniagc pictorial repi-escntation of a traiisform algonthm (or the algonthm It\elf) a motion estimation algonthm group of block.;, a rectangular region of a coded picture group of pictures, a set of coded video images stanclard [or video coding standard for video coding 'Long-term' dandard for video coding high definition television coding method to reduce redundancy 29 FIVS inter-frame (coding) interlaced (video) intra-frame (codiiig) IS0 ITU TTU-R 601 JPEG JPEG-2000 ia:r latency loop filter MGU nicdia processor memory bandwidth MJPEC mobon compensation motion evtimation motion vector MPEG MPEG- objective quality OBMC profile progressive (video) pruning (Iransfom) PS N rate control RTP RVLC scalable coding shoi? header (MPEG-4) slice atati stical redundaiicy subjectivc quality subjective redundancy sub-pixel (motion compensation) CLOSSrnY human visual system, the system by whxh humans percleve and interpret visual imagcs coding of video frames wing temporal prediction or compensation video data represented as a series of ficlda coditig o f video frames without tcniporal prediction Inlernational Standards Organisation International Telecoimuaication Union a colour video linage format Joint Photographic Experts Group, a commiltee of ISO; also aii image coding standard an image coding standard Karnuhen-IJoeve transform delay through a comniuiiication system spatial filter placed within encoding or decoding feedback loop mulli-point control unit, controls a multi-party conference processor with katures specific to inu~timed~a coding and processing Data transfer rate tolfrom RAM Syztem of coding a video sequence using P E G intra-frame compression prediction of a vrdco frame with modelling of motion cstimation of relative iiiotion between two or iiiore video frames vcctor indicating displaced block or region to bc used Lor motion coinpensation Motion Picture Experts Group, a conimittce of IS0 a video coding standad a video coding srandard a video coding standard vjsu;il quality measured by algorithtn(s) overlapped block motion compensation a \et of f'unctionai capabilitics (of a vidco CODEC) video data represented as a series of complete frames ieducing the number of calculated transform coefficients peak signal to noise ratio, an objective quality measure quaifer cominon intermediate format quality o f service reduce the precision of a scalar or vector quantity control of bit rate o€ encoded video signal measure of CODEC performance (distortion at a range of coded bit rates) red/green/blue colour space round shilrp edgca in a decoded image reat-timc protocol, a tr;.msport protocol for real-time data reversible variable length code coding a vignal into a number of layers coding mode that i s Cunctionally identical to W.263 ('baselmc') single instruction multiple data a region o f a coded picture redundancy due to the statistical distribution of dtltia visutil quality as perceived by human obscrver(s) redundancy due to coniponents of the data that arc subjectively insignificant motion-coinpensated prediction from a reference area that may be formed by interpolating between integer-valued pixel positioni, GLOSSARY test model TSS VCA VCEG video packet (MFEG-4) video processor VLC VLD VIE VLTW vm VO (MPEG-4) VOP (MPEG-4) VQEG YCrCb 299 a software model and document that describe a reference implementation of a video coding standard Ux-ee-step search, a motion estirrration algorithm v i a b l e complexity algorithm Video Coding Experts Group, a committee of ITU coded unit suitable for packetisabon processor with Peakires specific to video coding and processing variable length cock variable length decoder variable length encoder very long instruction word very large scale integrtited circuit video objcct video object plane Video Quality Experts Group luminanceked chsominancelblue chrominance colour space This Page Intentionally Left Blank Application Progranirniiig Interlace, 276 artefacts blocking, 299 ringing, 200 block matching, 43, 95 blockiness See artefacts:hiocking B-picture, 59 chrominance, I2 CODEC entropy, 37,45, 163 image, 33 video, 41 coded block pattern, 70, 167 coding arithmetic, 188 channel, 29 contcnt-based, 70 entropy, 163 field, 64 Hufhan, 169 losslcss, 28 lossy, 28 mesh, 74 model based, 32 object-based, 70 run-lcvel, 37, 164 scalable, 65, 73 shape, 70 source, 29 spite, 73 transform, 31, 127 colour space, 10 RGR, 1 YCrCh, 12 complexity complexity-rate control, 23 oompu ta tiona I, 226 variable complexity algorithms, 228 compression image and video, 28 DCT, 31, 127 basis functions, 130 distributed, 144 fast algorithms, 138 Aowgraph, 140 forwwd, 127 hardware, 148 inverse, 127 pruned, 228 software 146 design hardware CODEC, 284 perlormance, 283 software CODEC, 278 Digital Cinema, 291 Digital Versatile Disk, 24 displaced framc difference, 94 DPCM, 30 error concealment, 244 error resilience, 73, 81, 244 errors, 244 filters deblocking, 82, 206 de-ringing, 207 error concealment, 208 loop, 202 pre, 195 stabilization, 198 formats 4CIF, 24 ClF, 24 ITU-R 601,23 QCIF, 24 frame rates, 17, COB.S m Group of Blocks Group of Blocks, 70 Group of Pictures, 62 N.261, X0 H.263, 80 annexes, 81 baseline, 81 302 I“ H.263 (Continued) PB-framcs, 82 profiles, 86 H.26L X I 289 H.323,252 high definition television, 67 Human Visual System, 16 HVS See Iluman Visual Systeni intcrface coded data 274 control and stalus, 276, 27’9 to video CQDEC, 271 video, 271 inteifmiie, 41 ~tit~ma~ional Standards Organisation, 47 International Telecommiiiiications Union, 47 intrafi ilme, I-picture, 59 ISO Sec International Standards Organisation ITU See lnternational ‘relecomiiiunications Union JPEG, 51 baseline CODEC, 51 hierarchical, 55 lossless, 54 N l o h t , 56 JPEG2000, 56 KLT 31 latency, 240 237, 243 luminaiicc, 12 inemoiy bandwidth, 274 MJPEG See JPEGmotion motion compcnsation, 43, 94 estimation, 43 94, 109 Cross Search, 304 fiill search, 99 hardware, 122 hierarchical, 107 Logarithmic Search, 103 nearest iieighboui s scarch, I OS OTA, 105 performance, 100 soffware, 117 sub-pixel, 1I Three Step Search, 102 vectors, 43, 94, 161 MPEG, 47 MPEG- 1, 58 syntax 61 MPEG-2, 64 Program Stream, 250 systems 249 Transport Stream, 250 video, 64 MPEG-21, 49, 289 MPEG-4, 67 Binary Alpha Blocks I profiles and levels, 14 Short Header, 68 Vety Low Bitrate Video core, 68 Video Object, 68 Video Object Plane, 6X video packet, 73 MWC-7, 49, 289 Multipoint Control Unit, 253 OBMC See motion compensation prediction backwards, I 13 bidirectional, 13 forward, 11 processors co-processor 267 DSP, 260 embedded, 262 general purpose, 257 media, 263 PC, 257 video signal, 264 profiles and levels, 66, 74 quality, 16 USCQS, 17 ITU-K 500- 10 I7 objective, 19 PSNR, 19 recency, I8 subjective, 17 Quality of Service, 235 yuantiration, 35, 150 scale factor, 35 vector, 157 rate, 212 control, 212, 220 Lagrangian optimization, 21 rate-distortion 17 Red Time Protocol, 252, 254 redundancy statistical, 29 subjcctive 30 referencc picture selection, 84, 247 INDEX re-ordering pictures, 60 modified scan, 166 zigzag, 166, 37 RGB SPCcolour space ringing See artefacts: ringing RVLC See variable length codes sampling 4-2-0, 212, 12 4-2-2, 212, 12 - 4 12 spatial, temporal, acdability See coding: scalable Single lristruction Multiple Data, 258 slice, 63, 83 sourcc model, 28 still image, sub pixel motion estimation See motion: estimation test model, 50 transform DG?: See DCT 303 fractal, 35 integer, 145 wavelet, 35, 57, 133 variable length codes reversible, 73, 187 table dcsign, 174 universal, 175 variable length decoder, 184 variable length encoder, 180 vectors See rnotion:vectors Very Long Instruction Word, 263 video capture, digital, interlaced, 9, 64 progressive, stereoscopic, 7, 65 Video Coding Experts Group, 48 Video Quality Experts Group, 19 VOP See MPEG-4: Video Object Plane wavelet transform See transfom:wavdet YCrCb See colour space ... cominoiily used image resolutions and gives an approximately equivdent analogue video quality: VHS video, broadcast TV and high-definition TV A moving video image is formed by scampling the video vignal... Component Design ecction 3: System Design Structure of the book and H.26L, explajns the concepts of the ZTU-T video coding 63 and the emerging H.26L The chapter ends with a comparison of sin image and. .. are planted for each one used for paper production I Image and Video Compression 1.2 Video CODEC Design 1.3 Structure