Lecture Notes for CS 2110 Introduction to Theory of Computation Next: Forward Lecture Notes for CS 2110 Introduction to Theory of Computation Robert Daley Department of Computer Science University of Pittsburgh Pittsburgh, PA 15260 ● Forward ● Contents ● 1. Introduction ● 1.1 Preliminaries ● 1.2 Representation of Objects ● 1.3 Codings for the Natural Numbers ● 1.4 Inductive Definition and Proofs ● 2. Models of Computation ● 2.1 Memoryless Computing Devices ● 2.2 Digital Circuits ● 2.3 Propositional Logic ● 2.4 Finite Memory Devices ● 2.5 Regular Languages ● 3. Loop Programs ● 3.1 Semantics of LOOP Programs ● 3.2 Other Aspects ● 3.3 Complexity of LOOP Programs http://www.cs.pitt.edu/~daley/cs2110/notes/cs2110w.html (1 of 3) [12/23/2006 12:00:41 PM] Lecture Notes for CS 2110 Introduction to Theory of Computation ● 4. Primitive Recursive Functions ● 4.1 Primitive Recursive Expressibility ● 4.2 Equivalence between models ● 4.3 Primitive Recursive Expressibility (Revisited) ● 4.4 General Recursion ● 4.5 String Operations ● 4.6 Coding of Tuples ● 5. Diagonalization Arguments ● 6. Partial Recursive Functions ● 7. Random Access Machines ● 7.1 Parsing RAM Programs ● 7.2 Simulation of RAM Programs ● 7.3 Index Theorem ● 7.4 Other Aspects ● 7.5 Complexity of RAM Programs ● 8. Acceptable Programming Systems ● 8.1 General Computational Complexity ● 8.2 Algorithmically Unsolvable Problems ● 9. Recursively Enumerable Sets ● 10. Recursion Theorem ● 10.1 Applications of the Recursion Theorem ❍ 10.1.1 Machine Learning ❍ 10.1.2 Speed-Up Theorem ● 11. Non-Deterministic Computations ● 11.1 Complexity of Non-Deterministic Programs ● 11.2 NP-Completeness ● 11.3 Polynomial Time Reducibility ● 11.4 Finite Automata (Review) ● 11.5 PSPACE Completeness http://www.cs.pitt.edu/~daley/cs2110/notes/cs2110w.html (2 of 3) [12/23/2006 12:00:41 PM] Lecture Notes for CS 2110 Introduction to Theory of Computation ● 12. Formal Languages ● 12.1 Grammars ● 12.2 Chomsky Classification of Languages ● 12.3 Context Sensitive Languages ● 12.4 Linear Bounded Automata ● 12.5 Context Free Languages ● 12.6 Push Down Automata ● 12.7 Regular Languages ● Bibliography ● Index Next: Forward Bob Daley 2001-11-28 ©Copyright 1996 Permission is granted for personal (electronic and printed) copies of this document provided that each such copy (or portion thereof) is accompanied by this copyright notice. Copying for any commercial use including books, journals, course notes, etc., is prohibited . http://www.cs.pitt.edu/~daley/cs2110/notes/cs2110w.html (3 of 3) [12/23/2006 12:00:41 PM] Forward Next: Contents Up: Lecture Notes for CS 2110 Introduction to Theory Previous: Lecture Notes for CS 2110 Introduction to Theory Forward These notes have been compiled over the course of more than twenty years and have been greatly influenced by the treatments of the subject given by Michael Machtey and Paul Young in An Introduction to the Genereal Theory of Algorithms and to a lesser extent by Walter Brainerd and Lawrence Landweber in Theory of Computation. Unfortunately both these books have been out of print for many years. In addition, these notes have benefited from my conversations with colleagues especially John Case on the subject of the Recursion Theorem. Rather than packaging these notes as a commercial product (i.e., book), I am making them available via the World Wide Web (initially to Pitt students and after suitable debugging eventually to everyone). Next: Contents Up: Lecture Notes for CS 2110 Introduction to Theory Previous: Lecture Notes for CS 2110 Introduction to Theory Bob Daley 2001-11-28 ©Copyright 1996 Permission is granted for personal (electronic and printed) copies of this document provided that each such copy (or portion thereof) is accompanied by this copyright notice. Copying for any commercial use including books, journals, course notes, etc., is prohibited . http://www.cs.pitt.edu/~daley/cs2110/notes/cs2110w_node1.html [12/23/2006 12:01:17 PM] Contents Next: 1. Introduction Up: Lecture Notes for CS 2110 Introduction to Theory Previous: Forward Contents ● Contents ● 1. Introduction ❍ 1.1 Preliminaries ❍ 1.2 Representation of Objects ❍ 1.3 Codings for the Natural Numbers ❍ 1.4 Inductive Definition and Proofs ● 2. Models of Computation ❍ 2.1 Memoryless Computing Devices ❍ 2.2 Digital Circuits ❍ 2.3 Propositional Logic ❍ 2.4 Finite Memory Devices ❍ 2.5 Regular Languages ● 3. Loop Programs ❍ 3.1 Semantics of LOOP Programs ❍ 3.2 Other Aspects ❍ 3.3 Complexity of LOOP Programs ● 4. Primitive Recursive Functions ❍ 4.1 Primitive Recursive Expressibility ❍ 4.2 Equivalence between models ❍ 4.3 Primitive Recursive Expressibility (Revisited) ❍ 4.4 General Recursion ❍ 4.5 String Operations ❍ 4.6 Coding of Tuples ● 5. Diagonalization Arguments ● 6. Partial Recursive Functions ● 7. Random Access Machines ❍ 7.1 Parsing RAM Programs ❍ 7.2 Simulation of RAM Programs ❍ 7.3 Index Theorem ❍ 7.4 Other Aspects ❍ 7.5 Complexity of RAM Programs ● 8. Acceptable Programming Systems http://www.cs.pitt.edu/~daley/cs2110/notes/cs2110w_node2.html (1 of 2) [12/23/2006 12:01:34 PM] Contents ❍ 8.1 General Computational Complexity ❍ 8.2 Algorithmically Unsolvable Problems ● 9. Recursively Enumerable Sets ● 10. Recursion Theorem ❍ 10.1 Applications of the Recursion Theorem ■ 10.1.1 Machine Learning ■ 10.1.2 Speed-Up Theorem ● 11. Non-Deterministic Computations ❍ 11.1 Complexity of Non-Deterministic Programs ❍ 11.2 NP-Completeness ❍ 11.3 Polynomial Time Reducibility ❍ 11.4 Finite Automata (Review) ❍ 11.5 PSPACE Completeness ● 12. Formal Languages ❍ 12.1 Grammars ❍ 12.2 Chomsky Classification of Languages ❍ 12.3 Context Sensitive Languages ❍ 12.4 Linear Bounded Automata ❍ 12.5 Context Free Languages ❍ 12.6 Push Down Automata ❍ 12.7 Regular Languages ● Bibliography ● Index Next: 1. Introduction Up: Lecture Notes for CS 2110 Introduction to Theory Previous: Forward Bob Daley 2001-11-28 ©Copyright 1996 Permission is granted for personal (electronic and printed) copies of this document provided that each such copy (or portion thereof) is accompanied by this copyright notice. Copying for any commercial use including books, journals, course notes, etc., is prohibited . http://www.cs.pitt.edu/~daley/cs2110/notes/cs2110w_node2.html (2 of 2) [12/23/2006 12:01:34 PM] 1. Introduction Next: 1.1 Preliminaries Up: Lecture Notes for CS 2110 Introduction to Theory Previous: Contents 1. Introduction Goal To learn the fundamental properties and limitations of computability (i.e., the ability to solve problems by computational means) Major Milestones Invariance in formal descriptions of computable functions Church's Thesis Undecidability by computer programs of any dynamic (i.e., behavioral) properties of computer programs based on their text Major Topics Models of computable functions Decidable vs undecidable properties Feasible vs infeasible problems P NP Formal Languages (i.e., languages whose sentences can be parsed by computer programs) ● 1.1 Preliminaries ● 1.2 Representation of Objects ● 1.3 Codings for the Natural Numbers ● 1.4 Inductive Definition and Proofs Next: 1.1 Preliminaries Up: Lecture Notes for CS 2110 Introduction to Theory Previous: Contents http://www.cs.pitt.edu/~daley/cs2110/notes/cs2110w_node3.html (1 of 2) [12/23/2006 12:01:55 PM] 1. Introduction Bob Daley 2001-11-28 ©Copyright 1996 Permission is granted for personal (electronic and printed) copies of this document provided that each such copy (or portion thereof) is accompanied by this copyright notice. Copying for any commercial use including books, journals, course notes, etc., is prohibited . http://www.cs.pitt.edu/~daley/cs2110/notes/cs2110w_node3.html (2 of 2) [12/23/2006 12:01:55 PM] 1.1 Preliminaries Next: 1.2 Representation of Objects Up: 1. Introduction Previous: 1. Introduction 1.1 Preliminaries We will study a variety of computing devices. Conceptually we depict them as being ``black boxes'' of the form Figure 1.1:Black box computing device where x is an input object of type X (i.e., x X) and y is an output object of type Y. Thus, at this level the device computes a function f : X Y defined by f (x) = y. ● For some computing devices the function f will be a partial function which means that for some inputs x the function is not defined (i.e., produces no output). In this case we write f (x) . Similarly, we write f (x) whenever f on input x is defined. ● The set of all inputs on which the function f is defined is called its domain (denoted by dom f), and is given by dom f = {x : f (x) }. ● Also, the range of a function f (denoted by ran f), and is given by ran f = {y : x dom f, y = f (x)}. We will also be interested in computing devices which have multiple inputs and outputs, i.e., which can be depicted as follows: Figure 1.2:Multiple input-output computing device http://www.cs.pitt.edu/~daley/cs2110/notes/cs2110w_node4.html (1 of 3) [12/23/2006 12:02:07 PM] 1.1 Preliminaries where x 1 , , x n are objects of type X 1 , , X n (i.e., x 1 X 1 , , x n X n ), and y 1 , , y m are objects of type Y 1 , , Y m . Thus, the device computes a function f : X 1 x x X n Y 1 x x Y m defined by f (x 1 , , x n ) = (y 1 , , y m ). Here we use X 1 x x X n to denote the cartesian product, i.e., X 1 x x X n = {(x 1 , , x n ) : x 1 X 1 , , x n X n }. ● We also use X n to denote the cartesian product when X 1 = X 2 = = X n = X. ● Of course, since X 1 x x X n is just some set X and Y 1 x x Y m is some set Y, the situation with multiple inputs and outputs can be viewed as a more detailed description of a single input-output device where the inputs are n-tuples of elements and the outputs are m-tuples of elements. ● We use i n to denote x i , x i + 1 , , x n where i n, and n to denote 1 n (i.e., x 1 , , x n ). Besides viewing computing devices as mechanisms for computing functions we are also interested in them as mechanisms for computing sets. ● Given a set X the characteristic function of X (denoted by ) is given by http://www.cs.pitt.edu/~daley/cs2110/notes/cs2110w_node4.html (2 of 3) [12/23/2006 12:02:07 PM] [...]... function of the set X, i.e., f = 2 its domain is equal to X, i.e., X = dom f In this case we say that the device is an acceptor (or a recognizer) for the set X 3 its range is equal to X, i.e., X = ran f In this case we say that the device is a generator for the set X Next: 1.2 Representation of Objects Up: 1 Introduction Previous: 1 Introduction Bob Daley 200 1-1 1-2 8 ©Copyright 1996 Permission is granted for. .. normal form) expression if it is a disjunction of terms A monomial is a one-term DNF expression A boolean expression is a CNF (conjunctive normal form) expression if it is a conjunction of clauses The previous theorem is proved by constructing a DNF expresssion for any given boolean function Next: 2.2 Digital Circuits Up: 2 Models of Computation Previous: 2 Models of Computation Bob Daley 200 1-1 1-2 8 ©Copyright... Models of Computation Previous: 1.4 Inductive Definition and Proofs Bob Daley 200 1-1 1-2 8 ©Copyright 1996 Permission is granted for personal (electronic and printed) copies of this document provided that each such copy (or portion thereof) is accompanied by this copyright notice Copying for any commercial use including books, journals, course notes, etc., is prohibited http://www.cs.pitt.edu/ ~daley/ cs2110 /notes/ cs2110w_node8.html... 1010 122 22 11 1011 : 1 4 The function 1 3 and 1 2 The codings via 0 211 23 are one -to- one and onto The coding via * is not 010 = 10 , providing the one -to- one and onto map is defined inductively as follows: http://www.cs.pitt.edu/ ~daley/ cs2110 /notes/ cs2110w_node6.html (1 of 3) [12/23/2006 12:02:29 PM] 1.3 Codings for the Natural Numbers (0) = (x) = d1 dj, and let k Next, suppose that jbe the greatestinteger... Models of Computation Previous: 2.1 Memoryless Computing Devices Bob Daley 200 1-1 1-2 8 ©Copyright 1996 Permission is granted for personal (electronic and printed) copies of this document provided that each such copy (or portion thereof) is accompanied by this copyright notice http://www.cs.pitt.edu/ ~daley/ cs2110 /notes/ cs2110w_node10.html (1 of 2) [12/23/2006 12:03:03 PM] 2.2 Digital Circuits Copying for. .. a concept for a class of animals which inclues lions and tigers and bears! http://www.cs.pitt.edu/ ~daley/ cs2110 /notes/ cs2110w_node11.html (1 of 2) [12/23/2006 12:03:06 PM] 2.3 Propositional Logic Next: 2.4 Finite Memory Devices Up: 2 Models of Computation Previous: 2.2 Digital Circuits Bob Daley 200 1-1 1-2 8 ©Copyright 1996 Permission is granted for personal (electronic and printed) copies of this document... state q0 Therefore, such a device can be abbreviated as a tuple M= , , Q, , , q0 We depict M schematically as follows: Figure 2.6:Schematic for Finite State Automaton While this model of a finite memory device clearly models the computation of functions f : with finite memory, we need only consider a restricted form which are acceptors for languages over (i.e., subsets of strings from set of specially... Numbers Up: 1 Introduction Previous: 1.1 Preliminaries Bob Daley 200 1-1 1-2 8 ©Copyright 1996 Permission is granted for personal (electronic and printed) copies of this document provided that each such copy (or portion thereof) is accompanied by this copyright notice Copying for any commercial use including books, journals, course notes, etc., is prohibited http://www.cs.pitt.edu/ ~daley/ cs2110 /notes/ cs2110w_node5.html... use am to denote the word of length m consisting of m a's as a language , then X Y = {x y : x X and y Y} X(0) = { } X(n + 1) = X(n) X, for n 0 http://www.cs.pitt.edu/ ~daley/ cs2110 /notes/ cs2110w_node5.html (1 of 2) [12/23/2006 12:02:24 PM] as follows: 1.2 Representation of Objects X* = X(n) X+ = X(n) Thus X(n)is the set of all ``words'' of length nover the ``alphabet'' X Next: 1.3 Codings for the... Natural Numbers Bob Daley 200 1-1 1-2 8 ©Copyright 1996 Permission is granted for personal (electronic and printed) copies of this document provided that each such copy (or portion thereof) is accompanied by this copyright notice Copying for any commercial use including books, journals, course notes, etc., is prohibited http://www.cs.pitt.edu/ ~daley/ cs2110 /notes/ cs2110w_node6.html (3 of 3) [12/23/2006 12:02:29