Notes on Discrete Mathematics

Notes on Discrete Mathematics James Aspnes 2016-08-03 16:41 Contents Table of contents i List of figures xiv List of tables xv List of algorithms xvi Preface xvii Internet resources xviii Introduction 1.1 So why I need to learn all this nasty mathematics? 1.2 But isn’t math hard? 1.3 Thinking about math with your heart 1.4 What you should know about math 1.4.1 Foundations and logic 1.4.2 Basic mathematics on the real numbers 1.4.3 Fundamental mathematical objects 1.4.4 Modular arithmetic and polynomials 1.4.5 Linear algebra 1.4.6 Graphs 1.4.7 Counting 1.4.8 Probability 1.4.9 Tools 1 3 4 6 7 Mathematical logic 2.1 The basic picture 2.1.1 Axioms, models, and inference rules 9 i CONTENTS 2.2 2.3 2.4 2.5 2.6 Set 3.1 3.2 3.3 2.1.2 Consistency 2.1.3 What can go wrong 2.1.4 The language of logic 2.1.5 Standard axiom systems and models Propositional logic 2.2.1 Operations on propositions 2.2.1.1 Precedence 2.2.2 Truth tables 2.2.3 Tautologies and logical equivalence 2.2.3.1 Inverses, converses, and contrapositives 2.2.3.2 Equivalences involving true and false Example 2.2.4 Normal forms Predicate logic 2.3.1 Variables and predicates 2.3.2 Quantifiers 2.3.2.1 Universal quantifier 2.3.2.2 Existential quantifier 2.3.2.3 Negation and quantifiers 2.3.2.4 Restricting the scope of a quantifier 2.3.2.5 Nested quantifiers 2.3.2.6 Examples 2.3.3 Functions 2.3.4 Equality 2.3.4.1 Uniqueness 2.3.5 Models 2.3.5.1 Examples Proofs 2.4.1 Inference Rules 2.4.2 Proofs, implication, and natural deduction 2.4.2.1 The Deduction Theorem Natural deduction 2.5.1 Inference rules for equality 2.5.2 Inference rules for quantified statements Proof techniques ii 10 10 11 11 12 13 15 16 17 19 21 22 23 24 25 25 26 26 27 27 28 30 31 31 32 32 33 34 35 37 37 38 39 39 42 theory 47 Naive set theory 47 Operations on sets 49 Proving things about sets 50 CONTENTS 3.4 3.5 3.6 3.7 3.8 iii Axiomatic set theory Cartesian products, relations, and functions 3.5.1 Examples of functions 3.5.2 Sequences 3.5.3 Functions of more (or less) than one argument 3.5.4 Composition of functions 3.5.5 Functions with special properties 3.5.5.1 Surjections 3.5.5.2 Injections 3.5.5.3 Bijections 3.5.5.4 Bijections and counting Constructing the universe Sizes and arithmetic 3.7.1 Infinite sets 3.7.2 Countable sets 3.7.3 Uncountable sets Further reading 52 53 55 55 56 56 56 57 57 57 57 58 60 60 62 62 63 The real numbers 4.1 Field axioms 4.1.1 Axioms for addition 4.1.2 Axioms for multiplication 4.1.3 Axioms relating multiplication and addition 4.1.4 Other algebras satisfying the field axioms 4.2 Order axioms 4.3 Least upper bounds 4.4 What’s missing: algebraic closure 4.5 Arithmetic 4.6 Connection between the reals and other standard algebras 4.7 Extracting information from reals 64 65 65 66 68 69 70 71 73 73 74 75 Induction and recursion 5.1 Simple induction 5.2 Alternative base cases 5.3 Recursive definitions work 5.4 Other ways to think about induction 5.5 Strong induction 5.5.1 Examples 5.6 Recursively-defined structures 5.6.1 Functions on recursive structures 77 77 79 80 80 81 82 83 84 CONTENTS 5.6.2 5.6.3 iv Recursive definitions and induction Structural induction Summation notation 6.1 Summations 6.1.1 Formal definition 6.1.2 Scope 6.1.3 Summation identities 6.1.4 Choosing and replacing index variables 6.1.5 Sums over given index sets 6.1.6 Sums without explicit bounds 6.1.7 Infinite sums 6.1.8 Double sums 6.2 Products 6.3 Other big operators 6.4 Closed forms 6.4.1 Some standard sums 6.4.2 Guess but verify 6.4.3 Ansatzes 6.4.4 Strategies for asymptotic estimates 6.4.4.1 Pull out constant factors 6.4.4.2 Bound using a known sum Geometric series Constant series Arithmetic series Harmonic series 6.4.4.3 Bound part of the sum 6.4.4.4 Integrate 6.4.4.5 Grouping terms 6.4.4.6 Oddities 6.4.4.7 Final notes 84 85 86 86 87 87 87 89 89 91 91 91 92 92 93 94 95 96 97 97 97 98 98 98 98 98 98 99 99 99 Asymptotic notation 100 7.1 Definitions 100 7.2 Motivating the definitions 100 7.3 Proving asymptotic bounds 101 7.4 Asymptotic notation hints 102 7.4.1 Remember the difference between big-O, big-Ω, and big-Θ 102 7.4.2 Simplify your asymptotic terms as much as possible 103 CONTENTS Number theory 8.1 Divisibility and division 8.2 Greatest common divisors 8.2.1 The Euclidean algorithm for computing gcd(m, n) 8.2.2 The extended Euclidean algorithm 8.2.2.1 Example 8.2.2.2 Applications 8.3 The Fundamental Theorem of Arithmetic 8.3.1 Applications 8.4 Modular arithmetic and residue classes 8.4.1 Arithmetic on residue classes 8.4.2 Division in Zm 8.4.3 The Chinese Remainder Theorem 8.4.4 The size of Z∗m and Euler’s Theorem 8.5 RSA encryption 106 106 108 108 109 109 109 112 112 113 114 115 116 119 120 Relations 9.1 Representing relations 9.1.1 Directed graphs 9.1.2 Matrices 9.2 Operations on relations 9.2.1 Composition 9.2.2 Inverses 9.3 Classifying relations 9.4 Equivalence relations 9.4.1 Why we like equivalence relations 9.5 Partial orders 9.5.1 Drawing partial orders 9.5.2 Comparability 9.5.3 Lattices 9.5.4 Minimal and maximal elements 9.5.5 Total orders 9.5.5.1 Topological sort 9.5.6 Well orders 9.6 Closures 7.5 7.4.3 Remember the limit trick Variations in notation 7.5.1 Absolute values 7.5.2 Abusing the equals sign v 103 104 104 104 122 122 122 123 124 124 125 125 125 128 128 130 130 131 131 132 132 135 136 CONTENTS vi 9.6.1 Examples 139 10 Graphs 10.1 Types of graphs 10.1.1 Directed graphs 10.1.2 Undirected graphs 10.1.3 Hypergraphs 10.2 Examples of graphs 10.3 Local structure of graphs 10.4 Some standard graphs 10.5 Subgraphs and minors 10.6 Graph products 10.6.1 Functions 10.7 Paths and connectivity 10.8 Cycles 10.9 Proving things about graphs 10.9.1 Paths and simple paths 10.9.2 The Handshaking Lemma 10.9.3 Characterizations of trees 10.9.4 Spanning trees 10.9.5 Eulerian cycles 140 141 141 141 142 143 144 144 147 149 150 151 153 155 155 156 156 160 160 11 Counting 162 11.1 Basic counting techniques 163 11.1.1 Equality: reducing to a previously-solved case 163 11.1.2 Inequalities: showing |A| ≤ |B| and |B| ≤ |A| 163 11.1.3 Addition: the sum rule 164 11.1.3.1 For infinite sets 165 11.1.3.2 The Pigeonhole Principle 165 11.1.4 Subtraction 165 11.1.4.1 Inclusion-exclusion for infinite sets 166 11.1.4.2 Combinatorial proof 166 11.1.5 Multiplication: the product rule 167 11.1.5.1 Examples 168 11.1.5.2 For infinite sets 168 11.1.6 Exponentiation: the exponent rule 168 11.1.6.1 Counting injections 169 11.1.7 Division: counting the same thing in two different ways170 11.1.8 Applying the rules 172 11.1.9 An elaborate counting problem 173 CONTENTS vii 11.1.10 Further reading 176 11.2 Binomial coefficients 176 11.2.1 Recursive definition 177 11.2.1.1 Pascal’s identity: algebraic proof 178 11.2.2 Vandermonde’s identity 179 11.2.2.1 Combinatorial proof 179 11.2.2.2 Algebraic proof 180 11.2.3 Sums of binomial coefficients 180 11.2.4 The general inclusion-exclusion formula 181 11.2.5 Negative binomial coefficients 182 11.2.6 Fractional binomial coefficients 183 11.2.7 Further reading 183 11.3 Generating functions 183 11.3.1 Basics 183 11.3.1.1 A simple example 183 11.3.1.2 Why this works 184 11.3.1.3 Formal definition 185 11.3.2 Some standard generating functions 188 11.3.3 More operations on formal power series and generating functions 188 11.3.4 Counting with generating functions 189 11.3.4.1 Disjoint union 189 11.3.4.2 Cartesian product 190 11.3.4.3 Repetition 190 Example: (0|11)∗ 190 Example: sequences of positive integers 190 11.3.4.4 Pointing 192 11.3.4.5 Substitution 192 Example: bit-strings with primes 193 Example: (0|11)* again 193 11.3.5 Generating functions and recurrences 193 11.3.5.1 Example: A Fibonacci-like recurrence 194 11.3.6 Recovering coefficients from generating functions 194 11.3.6.1 Partial fraction expansion and Heaviside’s cover-up method 196 Example: A simple recurrence 196 Example: Coughing cows 197 Example: A messy recurrence 198 11.3.6.2 Partial fraction expansion with repeated roots200 Solving for the PFE directly 200 CONTENTS Solving for the PFE using the extended coverup method 11.3.7 Asymptotic estimates 11.3.8 Recovering the sum of all coefficients 11.3.8.1 Example 11.3.9 A recursive generating function 11.3.10 Summary of operations on generating functions 11.3.11 Variants 11.3.12 Further reading viii 202 203 204 204 205 208 209 209 12 Probability theory 210 12.1 Events and probabilities 211 12.1.1 Probability axioms 211 12.1.1.1 The Kolmogorov axioms 212 12.1.1.2 Examples of probability spaces 213 12.1.2 Probability as counting 213 12.1.2.1 Examples 214 12.1.3 Independence and the intersection of two events 214 12.1.3.1 Examples 215 12.1.4 Union of events 216 12.1.4.1 Examples 216 12.1.5 Conditional probability 217 12.1.5.1 Conditional probabilities and intersections of non-independent events 217 12.1.5.2 The law of total probability 218 12.1.5.3 Bayes’s formula 219 12.2 Random variables 219 12.2.1 Examples of random variables 219 12.2.2 The distribution of a random variable 220 12.2.2.1 Some standard distributions 221 12.2.2.2 Joint distributions 222 Examples 222 12.2.3 Independence of random variables 223 12.2.3.1 Examples 223 12.2.3.2 Independence of many random variables 224 12.2.4 The expectation of a random variable 224 12.2.4.1 Variables without expectations 225 12.2.4.2 Expectation of a sum 226 Example 226 12.2.4.3 Expectation of a product 226 CONTENTS 12.2.5 12.2.6 12.2.7 12.2.8 12.2.9 ix 12.2.4.4 Conditional expectation Examples 12.2.4.5 Conditioning on a random variable Markov’s inequality 12.2.5.1 Example 12.2.5.2 Conditional Markov’s inequality The variance of a random variable 12.2.6.1 Multiplication by constants 12.2.6.2 The variance of a sum 12.2.6.3 Chebyshev’s inequality Application: showing that a random variable is close to its expectation Application: lower bounds on random variables Probability generating functions 12.2.7.1 Sums 12.2.7.2 Expectation and variance Summary: effects of operations on expectation and variance of random variables The general case 12.2.9.1 Densities 12.2.9.2 Independence 12.2.9.3 Expectation 13 Linear algebra 13.1 Vectors and vector spaces 13.1.1 Relative positions and vector addition 13.1.2 Scaling 13.2 Abstract vector spaces 13.3 Matrices 13.3.1 Interpretation 13.3.2 Operations on matrices 13.3.2.1 Transpose of a matrix 13.3.2.2 Sum of two matrices 13.3.2.3 Product of two matrices 13.3.2.4 The inverse of a matrix Example 13.3.2.5 Scalar multiplication 13.3.3 Matrix identities 13.4 Vectors as matrices 13.4.1 Length 227 228 230 231 232 232 232 233 234 235 235 236 237 237 237 239 239 241 241 242 243 243 244 245 246 247 248 249 249 249 250 251 252 253 253 255 255 APPENDIX G THE NATURAL NUMBERS 357 Proof Expanding the definitions gives (∃yx = y + y) → (¬∃zSx = z + z) This is an implication at top-level, which calls for a direct proof The assumption we make is ∃yx = y +y Let’s pick some particular y that makes this true (in fact, there is only one, but we don’t need this) Then we can rewrite the right-hand side as ¬∃zS(y + y) = z + z There doesn’t seem to be any obvious way to show this (remember that we haven’t invented subtraction or division yet, and we probably don’t want to) We are rescued by showing the stronger statement ∀y¬∃zS(y+y) = z+z: this is something we can prove by induction (on y, since that’s the variable inside the non-disguised universal quantifier) Our previous lemma gives the base case ¬∃zS(0 + 0) = z + z, so we just need to show that ¬∃zS(y + y) = z + zimplies¬∃zS(Sy + Sy) = z + z Suppose that S(Sy + Sy) = z + z for some z [“suppose” = proof by contradiction again: we are going to drive this assumption into a ditch] Rewrite S(Sy + Sy) to get SSS(y + y) = z + z Now consider two cases: Case z = Then SSS(y + y) = + = 0, contradicting our first axiom Case z = Sw Then SSS(y + y) = Sw + Sw = SS(w + w) Applying the second axiom twice gives S(y + y) = w + w But this contradicts the induction hypothesis Since both cases fail, our assumption must have been false It follows that S(Sy + Sy) is not even, and the induction goes through G.5 Defining more operations Let’s define multiplication (·) by the axioms: • · y = • Sx · y = y + x · y Some properties of multiplication: • x · = • · x = x • x · = x • x · y = y · x • x · (y · z) = (x · y) · z APPENDIX G THE NATURAL NUMBERS 358 • x = ∧ x · y = x · z → y = z • x · (y + z) = x · y + x · z • x ≤ y → z · x ≤ z · y • z = ∧ z · x ≤ z · y → x ≤ y (Note we are using as an abbreviation for S0.) The first few of these are all proved pretty much the same way as for addition Note that we can’t divide in N any more than we can subtract, which is why we have to be content with multiplicative cancellation Exercise: Show that the Even(x) predicate, defined previously as ∃yy = x + x, is equivalent to Even (x) ≡ ∃yx = · y, where = SS0 Does this definition make it easier or harder to prove ¬Even (S0)? Bibliography [BD92] Dave Bayer and Persi Diaconis Trailing the dovetail shuffle to its lair Annals of Applied Probability, 2(2):294–313, 1992 [Ber34] George Berkeley THE ANALYST; OR, A DISCOURSE Addressed to an Infidel MATHEMATICIAN WHEREIN It is examined whether the Object, Principles, and Inferences of the modern Analysis are more distinctly conceived, or more evidently deduced, than Religious Mysteries and Points of Faith Printed for J Tonson, London, 1734 [Big02] Norman L Biggs Discrete Mathematics Oxford University Press, second edition, 2002 [Bou70] N Bourbaki Théorie des Ensembles Hermann, Paris, 1970 [Ded01] Richard Dedekind Essays on the Theory of Numbers The Open Court Publishing Company, Chicago, 1901 Translated by Wooster Woodruff Beman [Die10] R Diestel Graph Theory Springer, 2010 [Fer08] Kevin Ferland Discrete Mathematics Cengage Learning, 2008 Graduate Texts in Mathematics [Gen35a] Gerhard Gentzen Untersuchungen über das logische Schließen i Mathematische zeitschrift, 39(1):176–210, 1935 [Gen35b] Gerhard Gentzen Untersuchungen über das logische Schließen ii Mathematische Zeitschrift, 39(1):405–431, 1935 [GKP94] Ronald L Graham, Donald E Knuth, and Oren Patashnik Concrete Mathematics: A Foundation for Computer Science Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USA, 2nd edition, 1994 359 BIBLIOGRAPHY 360 [GVL12] Gene H Golub and Charles F Van Loan Matrix Computations Johns Hopkins University Press, 4th edition, 2012 [Hal58] Paul R Halmos Finite-Dimensional Vector Spaces Van Nostrand, 1958 [Hau14] Felix Hausdorff Grundzüge der Mengenlehre Veit and Company, Leipzig, 1914 [Kol33] A.N Kolmogorov Grundbegriffe der Wahrscheinlichkeitsrechnung Springer, 1933 [Kur21] Casimir Kuratowski Sur la notion de l’ordre dans la théorie des ensembles Fundamenta Mathematicae, 2(1):161–171, 1921 [Mad88a] Penelope Maddy Believing the axioms I Journal of Symbolic Logic, 53(2):48–511, June 1988 [Mad88b] Penelope Maddy Believing the axioms II Journal of Symbolic Logic, 53(3):736–764, September 1988 [NWZ11] David Neumark, Brandon Wall, and Junfu Zhang Do small businesses create more jobs? New evidence for the United States from the National Establishment time series The Review of Economics and Statistics, 93(1):16–29, February 2011 [Pea89] Ioseph Peano Arithmetices Principia: Nova Methodo Exposita Fratres Bocca, Rome, 1889 [Pel99] Francis Jeffrey Pelletier A history of natural deduction rules and elementary logic textbooks Available at http://www.sfu.ca/ ~jeffpell/papers/pelletierNDtexts.pdf, 1999 [Ros12] Kenneth Rosen Discrete Mathematics and Its Applications McGraw-Hill Higher Education, seventh edition, 2012 [Say33] Dorothy Sayers Murder Must Advertise Victor Gollancz, 1933 [Sch01] Eric Schechter Constructivism is difficult American Mathematical Monthly, 108:50–54, 2001 [Sol05] Daniel Solow How to Read and Do Proofs: An Introduction to Mathematical Thought Processes Wiley, 2005 BIBLIOGRAPHY 361 [Sta97] Richard P Stanley Enumerative Combinatorics, Volume Number 49 in Cambridge Studies in Advanced Mathematics Cambridge University Press, 1997 [Str05] Gilbert Strang Linear Algebra and its Applications Cengage Learning, 4th edition, 2005 [SW86] Dennis Stanton and Dennis White Constructive Combinatorics Undergraduate Texts in Mathematics Springer, 1986 [TK74] Amos Tversky and Daniel Kahneman Judgment under uncertainty: Heuristics and biases Science, 185(4157):1124–1131, September 1974 [Wil95] Andrew John Wiles Modular elliptic curves and Fermat’s Last Theorem Annals of Mathematics, 141(3):443–551, 1995 Index O(f (n)), 100 Ω(f (n)), 100 Θ(f (n)), 100 ↔, 14 N, 64, 74 Q, 64 R, 64 Z, 74 ¬, 13 ω(f (n)), 100 →, 14 ∨, 13 ∧, 14 D, 74 k-permutation, 169 n-ary relation, 122 o(f (n)), 100 abelian group, 65, 270 absolute value, 76 absorption law, 22 abuse of notation, 66, 105 accepting state, 59 acyclic, 154 addition, 65 addition (inference rule), 36 additive identity, 270 additive inverse, 270 adjacency matrix, 124 adjacent, 144 adversary, 28 affine transformation, 262 aleph-nought, 60 aleph-null, 60 aleph-one, 60 aleph-zero, 60 algebra, 74 linear, 243 algebraic field extension, 272 algebraically closed, 73 alphabet, 59 and, 14 annihilator, 68 ansatz, 96 antisymmetric, 125 antisymmetry, 70 arc, 140 arguments, 25 associative, 65, 270 associativity, 65 of addition, 65 of multiplication, 67 associativity of addition, 355 asymptotic notation, 100 atom, 211 automorphism, 151 average, 225 axiom, 9, 34 Axiom of Extensionality, 48 axiom schema, 52, 351 axiomatic set theory, 52 axioms field, 65 for the real numbers, 64 362 INDEX 363 Chinese Remainder Theorem, 116 class base case, 78 equivalence, 126 basis, 258 closed, 74 Bayes’s formula, 219 closed form, 93 Bayesian interpretation, 210 closed walk, 153 Bernoulli distribution, 221 closure, 136 biconditional, 14 reflexive, 136 big O, 100 reflexive symmetric transitive, 137 big Omega, 100 reflexive transitive, 137 big Theta, 100 symmetric, 137 bijection, 57 transitive, 137 bijective, 57 CNF, 23 binary relation, 122 codomain, 54 bind, 26 column, 247 binomial coefficient, 170, 176 column space, 261 binomial distribution, 221 column vector, 255 binomial theorem, 176 combinatorics, 162 bipartite, 293 enumerative, 162 bipartite graph, 142 commutative, 65, 270 blackboard bold, 64 commutative group, 65 block, 126 commutative ring, 270 commutativity, 65 cancellation, 355 of addition, 65 Cantor pairing function, 61 of multiplication, 67 Cantor’s diagonalization argument, 62 commutativity of addition, 355 cardinal arithmetic, 60 comparability, 70 cardinal numbers, 60 comparable, 128, 130 cardinality, 58, 162 compatible, 250 Cartesian product, 54 complement, 49 case analysis, 21 complete bipartite, 146 Catalan number, 207 complete graph, 144 categorical graph product, 150 completeness, 34 Cauchy sequence, 59 complex, 64 ceiling, 55, 75 complex number, 64 Central Limit Theorem, 222 component central limit theorem, 236 strongly-connected, 137 chain rule, 347 composed, 56 Chebyshev’s inequality, 235 composite, 106 Chernoff bounds, 236 composition, 56 Peano, 350 INDEX of relations, 124 compound proposition, 14 comprehension restricted, 52 conclusion, 34 conditional expectation, 227 conditional probability, 217 congruence mod m, 113 conjuctive normal form, 23 conjunction (inference rule), 36 connected, 152 connected components, 152 connected to, 152 consistent, 10 constants, 11 constructive, 41 contained in, 49 continuous random variable, 241 contraction, 149 contradiction, 17 contrapositive, 19 converges, 91 converse, 19 coordinate, 243 coprime, 108 countable, 62 counting two ways, 170 covariance, 234 CRC, 276 cross product, 150 cube, 147 Curry-Howard isomorphism, 21 cycle, 146, 153 simple, 153 cyclic redundancy check, 276 DAG, 123, 154 decision theory, 225 Dedekind cut, 59 deducible, 34 364 Deduction Theorem, 37 definite integral, 348 definition recursive, 77, 78 degree, 144, 271 dense, 72 density, 241 joint, 241 derivative, 346 deterministic finite state machine, 59 diagonalization, 62 die roll, 214 differential, 346 dimension, 243, 247, 259 of a matrix, 124 directed acyclic graph, 123, 137, 154 directed graph, 122, 140, 141 directed multigraph, 141 discrete probability, 211 discrete random variable, 220 disjoint, 60 disjoint union, 60 disjunctive normal form, 24 disjunctive syllogism, 36 distribution, 220 Bernoulli, 221 binomial, 221 geometric, 221 joint, 222 marginal, 222 normal, 222 Poisson, 221 uniform, 221 distribution function, 240 distributive, 270 divergence to infinity, 346 divided by, 67 divides, 106 divisibility, 128 division, 67 INDEX division algorithm, 83, 107 for polynomials, 271 divisor, 106, 107 DNF, 24 domain, 25, 54 dot product, 256 double factorial, 287 downward closed, 59 dyadics, 74 edge, 122 parallel, 122, 140 edges, 140 egf, 209 Einstein summation convention, 91 element minimum, 308 elements, 47, 247 empty set, 47 endpoint, 141 entries, 247 enumerative combinatorics, 162 equality, 31 equivalence class, 126, 127 equivalence relation, 125 Euclidean algorithm, 108 extended, 109 Euler’s Theorem, 119 Euler’s totient function, 119 Eulerian cycle, 154 Eulerian tour, 154 even, 113 even numbers, 61 event, 211 exclusive or, 13 existential quantification, 26 existential quantifier, 26 expectation, 224 exponential generating function, 209 extended Euclidean algorithm, 109 365 extended real line, 72 extension of a partial order to a total order, 132 factor, 106 factorial, 168, 287 double, 287 false positive, 219 Fermat’s Little Theorem, 120 field, 65, 269 finite, 113, 268 Galois, 273 ordered, 70 field axioms, 65 field extension, 272 field of fractions, 74 finite field, 113, 268 finite simple undirected graph, 140 first-order logic, 25 floating-point number, 65 floor, 55, 75, 107 flush, 301 formal power series, 185 fraction, 64, 74 fractional part, 76 frequentist interpretation, 210 full rank, 264 function symbol, 31 functions, 54 Fundamental Theorem of Arithmetic, 112 Galois field, 273 gcd, 108 Generalized Continuum Hypothesis, 61 generating function, 185 probability, 237 generating functions, 183 INDEX geometric distribution, 221 Goldbach’s conjecture, 31 graph, 140 bipartite, 123, 142 directed, 122 directed acyclic, 123, 137 simple, 122, 140 two-path, 296 undirected, 140 graph Cartesian product, 149 Graph Isomorphism Problem, 150 greatest common divisor, 108 greatest lower bound, 72, 131 group, 65, 270 abelian, 270 commutative, 65, 270 Hamiltonian cycle, 154 Hamiltonian tour, 155 Hasse diagram, 130 head, 141 homomorphism, 115, 151 graph, 293 hyperedges, 142 hypergraph, 142 hypothesis, 34 hypothetic syllogism, 36 identity, 66 additive, 270 for addition, 66 for multiplication, 67 multiplicative, 270 identity element, 92 identity matrix, 251 if and only if, 14 immediate predecessor, 130 immediate successor, 130 implication, 14 in-degree, 144 366 incident, 144 inclusion-exclusion, 181 inclusion-exclusion formula, 166 inclusive or, 13 incomparable, 130 incompleteness theorem, 35 indefinite integral, 348 independent, 214, 223, 241 pairwise, 234 index of summation, 86 indicator variable, 219 indirect proof, 19 induced subgraph, 147 induction, 77 induction hypothesis, 78 induction schema, 78, 351 induction step, 78 inequality Chebyshev’s, 235 inference rule, 35 inference rules, 9, 34 infimum, 72 infinite descending chain, 135 infinitesimal, 72 infix notation, 122 initial state, 59 initial vertex, 122, 141 injection, 57 injective, 57 integer, 64 integers, 74 integers mod m, 113 integral definite, 348 indefinite, 348 Lebesgue, 348 Lebesgue-Stieltjes, 224 integration by parts, 349 intersection, 49 intuitionistic logic, 21 INDEX invariance scaling, 70 translation, 70 inverse, 19, 57, 67 additive, 270 multiplicative, 270 of a relation, 125 invertible, 251 irrational, 64 irreducible, 272 isomorphic, 56, 75, 126, 150 isomorphism, 150 367 linearly independent, 257 little o, 100 little omega, 100 logical equivalence, 17 logically equivalent, 17 loops, 141 lower bound, 72, 86 lower limit, 86 lower-factorial, 169 magma, 270 marginal distribution, 222 Markov’s inequality, 231 join, 108, 131 mathematical maturity, 1, joint density, 241 matrix, 123, 247 joint distribution, 221, 222, 240 adjacency, 124 maximal, 131 Kolmogorov’s extension theorem, 213 maximum, 131 measurable, 240 labeled, 143 measurable set, 212 lambda calculus, 63 meet, 108, 131 lattice, 108, 131 mesh, 150 law of non-contradiction, 21 method of infinite descent, 81 law of the excluded middle, 21 minimal, 131 law of total probability, 218 minimum, 131 lcm, 108 minimum element, 308 least common multiple, 108 minor, 149 least upper bound, 71, 131 minus, 66 Lebesgue integral, 348 model, 10, 16, 33 Lebesgue-Stieltjes integral, 224 model checking, 16 lemma, 34, 37, 354 modulus, 107 length, 146, 152, 255 modus ponens, 24, 35, 36 lex order, 129 modus tollens, 36 lexicographic order, 129 monoid, 270 LFSR, 268 multigraph, 142 limit, 91, 344 multinomial coefficient, 171 linear, 87, 226, 259 multiplicative identity, 270 linear algebra, 243 multiplicative inverse, 109, 115, 270 linear combination, 257 multiset, 128 linear transformation, 249, 259 multivariate generating functions, 193 linear-feedback shift register, 268 INDEX naive set theory, 47 natural, 64 natural deduction, 38 natural number, 64 natural numbers, 47, 74, 350 negation, 13, 66 negative, 66, 70 neighborhood, 144 node, 140 non-constructive, 21, 41, 53 non-negative, 70 non-positive, 70 normal, 258 normal distribution, 222, 241 notation abuse of, 105 asymptotic, 100 number complex, 64 floating-point, 65 natural, 64 rational, 64 real, 64 number theory, 106 O big, 100 o little, 100 octonion, 64 odd, 113 odd numbers, 61 Omega big, 100 omega little, 100 on average, 225 one-to-one, 57 one-to-one correspondence, 57 onto, 57 368 or, 13 exclusive, 13 inclusive, 13 order lexicographic, 129 partial, 128 pre-, 129 quasi-, 129 total, 70, 128, 132 ordered field, 70 ordered pair, 53 orthogonal, 257 orthogonal basis, 258 orthonormal, 258 out-degree, 144 outcome, 212 over, 67 pairwise disjoint, 165 pairwise independent, 234 parallel edge, 122 parallel edges, 140 partial order, 128 strict, 128 partially ordered set, 128 partition, 59, 126 Pascal’s identity, 177 Pascal’s triangle, 178 path, 146, 152 Peano axioms, 350 peer-to-peer, 144 permutation, 169 k-, 169 pgf, 209, 237 plus, 65 Poisson distribution, 221 poker deck, 301 poker hand, 214 pole, 203 polynomial ring, 271 INDEX poset, 128 product, 129 positive, 70 power, 153 predecessor, 130 predicate logic, 11 predicates, 11, 25 prefix, 129 premise, 34 preorder, 129 prime, 82, 106 prime factorization, 112 probability, 210, 211 conditional, 217 discrete, 211 probability distribution function, 220 probability generating function, 209, 237 probability mass function, 220, 240 probability measure, 212 probability space, 212 uniform, 213 product, 250 product poset, 129 product rule, 167 projection, 264 projection matrix, 266 proof by contraposition, 19 proof by construction, 41 proof by example, 41 proposition, 12 compound, 14 propositional logic, 11, 12 provable, 34 pseudo-ring, 270 Pythagorean theorem, 257 quantifiers, 26 quantify, 11 369 quasiorder, 129 quaternion, 64 quotient, 107 quotient set, 126 radius of convergence, 203 random bit, 214 random permutation, 214 random variable, 219, 239 continuous, 241 discrete, 220 range, 54, 57 rank, 264 ranking, 176 rational, 64 rational decision maker, 225 rational functions, 203 rational number, 64 reachable, 152 real number, 64 recursion, 77 recursive, 84 recursive definition, 77, 78 recursively-defined, 83 reflexive, 125 reflexive closure, 136 reflexive symmetric transitive closure, 137 reflexive transitive closure, 137 reflexivity, 70 reflexivity axiom, 32 regression to the mean, 229 regular expression, 197 relation, 122 n-ary, 122 binary, 122 equivalence, 125 on a set, 122 relatively prime, 108 remainder, 107 INDEX representative, 114, 127 residue class, 113 resolution, 23, 36 resolution proof, 24 resolving, 23 restricted comprehension, 52 restriction, 169 rig, 270 ring, 68, 70, 270 commutative, 270 polynomial, 271 rng, 270 round-off error, 65 row, 247 row space, 261 row vector, 255 RSA encryption, 120 Russell’s paradox, 48 scalar, 245, 246, 253 scalar multiplication, 246 scalar product, 253 scaling, 245, 262 scaling invariance, 70 second-order logic, 352 selection sort, 133 self-loops, 140 semigroup, 270 semiring, 70, 270 sequence, 247 set comprehension, 48 set difference, 49 set theory axiomatic, 52 naive, 47 set-builder notation, 48 shear, 263 sigma-algebra, 212 signature, 33 signum, 76 370 simple, 141, 152 simple cycle, 153 simple induction, 77 simple undirected graph, 141 simplification (inference rule), 36 sink, 122, 141 size, 57 sort selection, 133 topological, 132 soundness, 34 source, 122, 141 span, 257 spanning tree, 160 square, 247 square product, 149 standard basis, 258 star graph, 146 state space, 59 statement, 11 Stirling number, 170 strict partial order, 128 strong induction, 81 strongly connected, 152, 153 strongly-connected component, 137 strongly-connected components, 153 structure, 32 sub-algebra, 74 subgraph, 147 induced, 147 sublattice, 113 subscript, 55 subset, 49 subspace, 257 substitution (inference rule), 39 substitution axiom schema, 32 substitution rule, 32, 39 subtraction, 66 successor, 130 sum rule, 164 INDEX supremum, 72 surjection, 57 surjective, 56, 57 symmetric, 125, 249 symmetric closure, 137 symmetric difference, 49 symmetry, 32 syntactic sugar, 55 tail, 141 tautology, 17 terminal vertex, 122, 141 theorem, 9, 34, 37, 354 Wagner’s, 149 theory, 9, 33 Theta big, 100 Three Stooges, 47 topological sort, 132 topologically sorted, 154 total order, 70, 128, 132 totally ordered, 128 totient, 119 transition function, 59 transition matrix, 249 transitive, 125 transitive closure, 137, 153 transitivity, 32, 70 translation invariance, 70 transpose, 249 tree, 154, 156 triangle inequality, 256 trichotomy, 70 truth table, 16 proof using, 17 tuples, 55 turnstile, 34 two-path graph, 296 uncorrelated, 234 371 uncountable, 63 undirected graph, 140, 141 uniform discrete probability space, 213 uniform distribution, 221 union, 49 unit, 106 unit vector, 256 universal quantification, 26 universal quantifier, 26 universe, 49 universe of discourse, 25 unranking, 176 upper bound, 71, 86 upper limit, 86 valid, 35, 36 Vandermonde’s identity, 179 variable indicator, 219 variance, 232 vector, 243, 246, 255 unit, 256 vector space, 243, 246 vertex, 122 initial, 122 terminal, 122 vertices, 140 Von Neumann ordinals, 58 Wagner’s theorem, 149 weakly connected, 153 web graph, 144 weight, 184 well order, 135 well-defined, 354 well-ordered, 58, 80 Zermelo-Fraenkel set theory with choice, 52 ZFC, 52 Zorn’s lemma, 135 ... Propositional logic Propositional logic is the simplest form of logic Here the only statements that are considered are propositions, which contain no variables Because propositions contain no... product 226 CONTENTS 12.2.5 12.2.6 12.2.7 12.2.8 12.2.9 ix 12.2.4.4 Conditional expectation Examples 12.2.4.5 Conditioning on a random variable ... Imagine printing out an execution trace that showed every operation a typical $500 desktop computer executed in one (1) second If you could read one operation per second, for eight hours every day,