ALGORITHMIC INFORMATION THEORY - CHAPTER 6 pps

[...]... enable us to use Theorem I2 to show that there is a computer D with these properties: ( HD (s) = ; lg PC (s) + 1 (6. 1) PD (s) = 2lg PC (s) < PC (s) 6. 3 BASIC IDENTITIES ( HD (s=t) = ; lg PC (s=t) + 1 PD (s=t) = 2lg PC (s=t) < PC (s=t): 169 (6. 2) By applying Theorem I0(a,b) to (6. 1) and (6. 2), we see that Theorem I4 holds with c = sim(D) + 2 How does the computer D work? First of all, it checks whether... possibilities for de ning random in nite strings and real numbers, and we study them at length in Chapter 7 To anticipate, the undecidability of the halting problem is a fundamental theorem of recursive function theory In algorithmic information theory the corresponding theorem is as follows: The base-two representation of the probability that U halts is a random (i.e., maximally complex) in nite string... computer 6. 3 BASIC IDENTITIES 171 Theorem I5(b) enables one to reformulate results about H as results concerning P , and vice versa it is the rst member of a trio of formulas that will be completed with Theorem I9(e,f) These formulas are closely analogous to expressions in classical information theory for the information content of individual events or symbols Shannon and Weaver (1949)] Theorem I6 (There... Theorem I2 is satised, the requirements (6. 3) indeed determine a computer, and the proof of (6. 1) and Theorem I4(a) is complete (b) If D has been given the free data t , it enumerates Tt without repetitions and simulates the computer determined by the set of all requirements of the form f(s n + 1) : \PC (s=t) > 2;n " 2 Ttg (6. 4) = f(s n + 1) : PC (s=t) > 2;n g : CHAPTER 6 PROGRAM SIZE 170 Thus (s n) is taken.. .6. 3 BASIC IDENTITIES 167 only if the interval requested is longer than the total length of the unassigned part of 0 1), i.e., only if the requirements are inconsistent Q.E.D Note The preceding proof may be considered to involve a computer memory \storage allocation" problem We have one unit of storage, and all requests for storage request a power of two of storage, i.e., one-half unit, one-quarter... of degree To the question \How random is s?" one must reply indicating how close 6. 4 RANDOM STRINGS 177 H (s) is to the maximum possible for strings of its size A string s is most random if H (s) is approximately equal to jsj + H (jsj) As we shall see in the next chapter, a good cut-o to choose between randomness and non-randomness is H (s) jsj The natural next step is to de ne an in nite string to... otherwise, which immediately yields (6. 2) However, we must check that the requirements (6. 4) on D satisfy the Kraft inequality and are consistent X ;jpj lg PC (s=t) 2 =2 < PC (s=t): D(p t )=s Hence X D(p t ) is de ned 2;jpj < X s PC (s=t) 1 by Theorem I0(k) Thus the hypothesis of Theorem I2 is satis ed, the requirements (6. 4) indeed determine a computer, and the proof of (6. 2) and Theorem I4(b) is complete... essentially the entire formalism of information theory Results such as these can now be obtained e ortlessly: H (s1) H (s1=s2) + H (s2=s3) + H (s3=s4) + H (s4) + O(1) H (s1 s2 s3 s4) = H (s1=s2 s3 s4) + H (s2=s3 s4) + H (s3=s4) + H (s4) + O(1): However, there is an interesting class of identities satis ed by our H function that has no parallel in classical information theory The simplest of these is H (H (s)=s)... t) = O(1) which implies H (H (s : t)=s t) = O(1): And of course these identities generalize to tuples of three or more strings 6. 4 Random Strings In this section we begin studying the notion of randomness or algorithmic incompressibility that is associated with the program-size complexity measure H Theorem I10 (Bounds on the complexity of positive integers) (a) P 2;H (n) 1 n Consider a computable... is a necessary and su cient condition for the existence a computer C determined by the requirements f(n f (n)) : n n0g is satis ed It follows that H (n) f (n) + sim(C ) for all n n0 Q.E.D CHAPTER 6 PROGRAM SIZE 1 76 Remark H (n) can in fact be characterized as a minimal function computable in the limit from above that lies just on the borderline between the convergence and the divergence of X ;H (n) .