Lecture 3 Block
Data CompressionLecture 3Block Coding Alexander Kolesnikov Shortcoming of Huffman codesAlphabet: a, b.pa=p=0.99, pb=q=0.011) Entropy H1=-(p*log2(p)+q*log2(q))=0.081 bits/pel2) Huffman code: pa=’0’, pb=’1’ Bitrate R1 = 1*p+1*q = p+q = 1 bit/pel!Make a new alphabet blocking symbols! Block coding: n=2New alphabet: ’A’=’aa’, ’B’=’ab’, ’C’=’ba’, ’D’=’bb’pA=p2=0.9801, pB=pq=0.0099, pC=pq=0.0099, pD= q2=0.0001 1) Entropy: H2=-(0.9801*log2(0.9801) + 0.0099*log2(0.0099) + + 0.0099*log2(0.0099) + 0.0001*log2(0.0001))= =(0.0284+0.0659+0.0659+0.0013)/2= 0.081 bits/pelWhy H2=H1? 2) Huffman code: cA=’0’, cB=’10’, cC=’110’, cD=’111’ LA=1, LB=2, LC=2, LD=3 Bitrate R2 = (1*pA+2*pB+3*pC+3*pD)/2=0.515 bits/pel Block coding: n=3’A’=’aaa’ -> pA=p3 ’B=’aab’, ’C’=’aba’, ’D’=’baa’ -> pB= pC=pD=p2q’E’=’abb’, ’F’=’bab’, ’G’=’bba’ -> pE= pF=pG=pq2 ’H’=’bbb’ -> pH=q3Huffman code: cA=’0’, cB=’10’, cC=’110’, cD=’1110’ cE=’111100, cB=’111101’, cG=’111110’, cH=’111111’ Entropy H3?Bitrate: R3 = (1*pA+2*pB+3*pC+4*pD+6*(pE+pF+pG+pH))/3= = 0.353 bits/pel Block coding: n→ ∞pa=p=0.99, pb=q=0.01Entropy Hn=0.081 bits/pel Bitrate for Hufman coder: n= 1: R1 = 1.0 bit 2 symbols in alphabet n= 2: R2 = 0.515 bits 4 symbols in alphabet n= 3: R3 = 0.353 bits 8 symbols in alphabetIf block size n → ∞? Hn ≤ Rn < Hn+1/nProblem - alphabet size and Huffman table size grows exponentially with number of symbols n blocked.∑∑==+≤≤NinnNinnnnBpBpnRBpBpn121*21)(log)(1)(log)(1 Block coding: Example 2, n=11) Entropy H=-((56/64)*log2(56/64)+(8/64)*log2(8/64))=0.544 bits/pel2) Huffman code: a= ’0’; b=’1’ Bitrate: R= 1 bit/pel pa=56/64pb=8/64 Block coding: Example 2, n=41) Entropy H=-((12/16)*log2(12/16)+(4/16)*log2(4/16))/4=0.203 bits/pel 2) Huffman code: A=’0’, B=’1’ Bitrate R = (1*pA+1*pB)/4=0.250 bits/pelpA=12/16pB=4/16 Exercise Thực hiện mã khối với n=2;pa=56/64pb=8/64 . cG=’111110’, cH=’111111’ Entropy H3?Bitrate: R3 = (1*pA+2*pB +3* pC+4*pD+6*(pE+pF+pG+pH)) /3= = 0 .35 3 bits/pel Block coding: n→ ∞pa=p=0.99, pb=q=0.01Entropy. alphabet n= 2: R2 = 0.515 bits 4 symbols in alphabet n= 3: R3 = 0 .35 3 bits 8 symbols in alphabetIf block size n → ∞? Hn ≤ Rn < Hn+1/nProblem -