No bullshit guide to linear algebra Ivan Savov April 15, 2016 No bullshit guide to linear algebra by Ivan Savov Copyright © Ivan Savov, 2015, 2016 All rights reserved Published by Minireference Co Montréal, Québec, Canada minireference.com | @minireference | fb.me/noBSguide For inquiries, contact the author at ivan.savov@gmail.com Near-final release v0.9 hg changeset: 295:aea569652164 ISBN 978-0-9920010-1-8 (TODO: generate new ISBN for v1.0) 10 Contents Preface vii Introduction 1 Math fundamentals 1.1 Solving equations 1.2 Numbers 1.3 Variables 1.4 Functions and their inverses 1.5 Basic rules of algebra 1.6 Solving quadratic equations 1.7 The Cartesian plane 1.8 Functions 1.9 Function reference 1.10 Polynomials 1.11 Trigonometry 1.12 Trigonometric identities 1.13 Geometry 1.14 Circle 1.15 Solving systems of linear equations 1.16 Set notation 1.17 Math problems Vectors 2.1 Vectors 2.2 Basis 2.3 Vector products 2.4 Complex numbers 2.5 Vectors problems 90 91 99 100 103 108 11 15 17 20 24 28 31 37 54 58 63 65 67 69 73 80 Intro to linear algebra 111 3.1 Introduction 111 3.2 Review of vector operations 117 i 3.3 3.4 3.5 3.6 Matrix operations Linearity Overview of linear algebra Introductory problems 121 126 131 135 Computational linear algebra 4.1 Reduced row echelon form 4.2 Matrix equations 4.3 Matrix multiplication 4.4 Determinants 4.5 Matrix inverse 4.6 Computational problems 136 137 148 152 156 167 174 Geometrical aspects of linear 5.1 Lines and planes 5.2 Projections 5.3 Coordinate projections 5.4 Vector spaces 5.5 Vector space techniques 5.6 Geometrical problems algebra 178 178 186 191 196 207 217 219 219 230 241 246 253 Theoretical linear algebra 7.1 Eigenvalues and eigenvectors 7.2 Special types of matrices 7.3 Abstract vector spaces 7.4 Abstract inner product spaces 7.5 Gram–Schmidt orthogonalization 7.6 Matrix decompositions 7.7 Linear algebra with complex numbers 7.8 Theory problems 254 255 268 274 278 285 289 295 310 Applications 8.1 Balancing chemical equations 8.2 Input–output models in economics 8.3 Electric circuits 8.4 Graphs 8.5 Fibonacci sequence 8.6 Linear programming 314 315 317 318 324 326 329 Linear transformations 6.1 Linear transformations 6.2 Finding matrix representations 6.3 Change of basis for matrices 6.4 Invertible matrix theorem 6.5 Linear transformations problems iii 8.7 8.8 8.9 8.10 8.11 8.12 Least squares approximate Computer graphics Cryptography Error correcting codes Fourier analysis Applications problems solutions 330 339 351 363 372 386 387 387 394 400 407 10 Quantum mechanics 10.1 Introduction 10.2 Polarizing lenses experiment 10.3 Dirac notation for vectors 10.4 Quantum information processing 10.5 Postulates of quantum mechanics 10.6 Polarizing lenses experiment revisited 10.7 Quantum physics is not that weird 10.8 Applications 10.9 Quantum mechanics problems 408 409 415 422 428 431 445 449 454 469 End matter Conclusion Social stuff Acknowledgements General linear algebra links 471 471 473 473 473 Probability theory 9.1 Probability distributions 9.2 Markov chains 9.3 Google’s PageRank algorithm 9.4 Probability problems A Answers and solutions B Notation Math notation Set notation Vectors notation Complex numbers notation Vector space notation Notation for matrices and matrix operations Notation for linear transformations Matrix decompositions Abstract vector space notation 475 494 494 495 495 496 496 497 498 498 499 Concept maps Figure 1: This concept map illustrates the prerequisite topics of high school math covered in Chapter and vectors covered in Chapter Also shown are the topics of computational and geometrical linear algebra covered in Chapters and iv Figure 2: Chapter covers linear transformations and their properties Figure 3: Chapter covers theoretical aspects of linear algebra v Figure 4: Matrix operations and matrix computations play an important role throughout this book Matrices are used to implement linear transformations, systems of linear equations, and various geometrical computations Figure 5: The book concludes with three chapters on linear algebra applications In Chapter we’ll discuss applications to science, economics, business, computing, and signal processing Chapter on probability theory and Chapter 10 on quantum mechanics serve as examples of advanced subjects that you can access once you learn linear algebra vi 485 Solutions to selected exercises E6.2 Start from the formula for B [MT ]B and multiply it by left and by B [✶]B from the right: B[ ✶]B B [MT ]B B [✶]B = = Recall that B[ ✶]B E6.3 Since yiw ∗ ✶]B B[ B [✶]B B [MT ]B B [ ✶]B ✶]B B from the [✶]B B [MT ]B is the inverse matrix of = B B[ B [✶]B so the two matrices cancel out [MT ]B [xi ]B , you can recover [xi ]B by hitting yiw the inverse of MT in the basis B : [xi ]B = MT−1 B B output, compute [yi ]B = B [MT ]B MT−1 yiw ∗ B B yiw ∗ ∗ with To find the correct E6.4 Let c1 , c2 , , cn be the n columns of A Since the columns of A form a basis for Rn , any vector b ∈ Rn can be written as a unique linear combination of the columns of A: b = x1 c1 + x2 c2 + · · · + xn cn for some coefficients x1 , x2 , , xn Reinterpreting the last equation as a matrix product in the column picture, we conclude that Ax = b has a unique solution x E6.5 Let Bs = {ˆ e1 , eˆ2 , , eˆn } be the standard basis for Rn Since Ax = b has a solution x for every possible b, it is possible to find the solutions xi in n equations of the form Axi = eˆi , for i ∈ {1, 2, , n} Now construct the matrix B that contains the solutions xi as columns: B = [x1 , , xn ] Observe that AB = A[x1 , , xn ] = [Ax1 , , Axn ] = [ˆ e1 , , eˆn ] = ✶n The equation BA = ✶n implies B ≡ A−1 and thus A is invertible Answers to problems P6.1 Im(T ) = span {(1, 1, 0), (0, −1, 2)} P6.2 −1 P6.3 D = 0 0 0 0 0 0 Solutions to selected problems P6.1 Applying T to the input vector (1, 0) produces (1, − 0, · 0) = (1, 1, 0), and the input vector (0, 1) produces the output (0, − 1, · 1) = (0, −1, 2) Thus, Im(T ) = span {(1, 1, 0), (0, −1, 2)} ⊆ R3 P6.2 We apply L to both e2x cos x and e2x sin x L(e2x cos x) = 2e2x cos x − e2x sin x + e2x cos x = 3e2x cos x − e2x sin x L(e2x sin x) = 2e2x sin x + e2x cos x + e2x sin x = e2x cos x + 3e2x sin x The first corresponds to the vector [3, −1],with respect to the given basis, and the second corresponds to the vector [1, 3] with respect to the given basis These vectors are columns of the matrix representing L Thus, the matrix representing L with respect to the given basis is −1 Chapter solutions Answers to exercises E7.3 See math.stackexchange.com/a/29374/46349 E7.6 α = −3 and β = E7.7 (1, 1)T and (1, −2)T are eigenvectors of L E7.8 No E7.9 det(A) = 15, 11 −5 − −1 A−1 = 0 ; det(B) = xz, B −1 = x y − xz z ; det(C) = 1, C −1 = 486 5 E7.10 v = (1, 0, 0, 1) E7.11 V = [1, 0, 0, 1] E7.12 The solution space is one-dimensional and spanned by f (t) = e−t E7.13 Yes e1 = (4, 2), e2 = (−1, 2) E7.15 eˆ1 = ( √1 , √1 , 0), eˆ2 = ( √1 , − √1 , eˆ3 = (− √1 , E7.17 √1 , √1 ) 31 √ Q = √2 −1 + i; (b) −14 + 23i; √1 , e ˆ2 = 2 1 √ −√ √1 − √1 , and R √2 √1 (c) 25 (26 + 7i) ˆ1 = E7.16 e E7.14 √ ), and 6√ 6 ˆ3 = √ (3x − 1) 3/2x, and e √ √1 √1 2 √3 √1 E7.18 (a) = 6 √2 0 Solutions to selected exercises E7.1 The characteristic polynomial of the matrix has degree n, and an nth -degree polynomial has at most n distinct roots E7.3 Proof by contradiction Assume you have n distinct eigenvalues λi and n eigenvectors {ei } which are linearly dependent: i=1 αi ei = with some αi = If a nonzero combination of αi could give the zero vector as a linear combination, the equation (A − λn I) ( αi ei ) = (A − λn I)0 = would be true However, if you expand the expression on the left, you’ll see it’s not equal to zero E7.4 Multiply both sides of the eigenvalue equation Aeλ = λeλ by A to obtain AAeλ = λAeλ = λ2 eλ Thus λ2 is an eigenvalue of A2 E7.5 Multiply both sides of the eigenvalue equation Aeλ = λeλ by A−1 to obtain A−1 Aeλ = λA−1 eλ After A−1 cancels with A, we’re left with the equation eλ = λA−1 eλ Dividing both sides of the equation by λ we obtain λ eλ = A−1 eλ , −1 −1 which shows that λ is an eigenvalue of A E7.6 The characteristic polynomial of A is pA (λ) = x2 − βx − α If we want the eigenvalues of A to be and 3, we must choose α and β so that pA (λ) = (λ − 1)(λ − 3) Expanding the factored expression, we find (λ − 1)(λ − 3) = λ2 − 4λ + 3, so α = −3 and β = E7.7 First we compute L(1, 1)T = (5, 5)T = 5(1, 1)T so (1, 1)T is an eigenvector, with eigenvalue λ = Next we compute L(1, −1)T = (1, 3)T = α(1, −1)T so (1, −1)T is not an eigenvector We can also compute L(1, −2)T = (−1, 2)T = −1(1, −2)T , which implies (1, −2)T is an eigenvector with eigenvalue −1 Since × matrix can have at most two eigenvectors, we don’t need to check (2, −1)T — we know its not an eigenvector E7.8 A matrix A is Hermitian if it satisfies A† = A, which is not the case for the given matrix E7.12 The solutions to the differential equation f (t) + f (t) = are of the form f (t) = Ce−t , where C is an arbitrary constant Since any solution in the solution space can be written as a multiple of the function e−t , we say the solution space is spanned by e−t Since one function is sufficient to span the solution space, the solution space is one-dimensional E7.13 Consider an arbitrary second-degree polynomial p(x) = a0 + a1 x + a2 x2 We can rewrite it as p(x) = a0 + a1 [(x − 1) + 1] + a2 [(x − 1) + 1]2 = a0 + a1 (x − 1) + a1 + a2 [(x − 1)2 + 2(x − 1) + 1] = (a0 + a1 + a2 )1 + (a1 + 2a2 )(x − 1) + a2 (x − 1)2 Since we’ve expressed an arbitrary polynomial of degree in the desired form, the answer is yes In other words, the set {1, x − 1, (x − 1)2 } is a basis for the vector space of polynomials of degree at most 487 E7.14 The exercise does not require us to normalize the vectors, so we can leave the first vector as is v1 = e1 = (4, 2) Next, we calculate e2 using the formula e2 = v2 − Πe1(v2 ), which corresponds to removing the component of v2 that lies in the direction of e1 Using the projection formula we obtain Πe1(v2 ) ≡ (4,2)·(1,3) (4, 2) (4,2) 4+6 (4, 2) = 12 (4, 2) = (2, 1) Thus e2 = v2 − Πe1(v2 ) = = 16+4 (1, 3) − (2, 1) = (−1, 2) Verify that e1 · e2 = Answers to problems P7.1 (a) λ1 = 6, λ2 = −1; (b) λ1 = −2, λ2 = 2, λ3 = √ √ ϕ ≡ 1+2 = 1.6180339 ; λ2 = − ϕ = 1−2 = −0.6180339 and λ2 = − ϕ P7.4 X = Q−1 = √ 5+ 10 √ 5− 10 √ 5 √ − 55 P7.2 λ1 = P7.3 λ1 = ϕ P7.5 (a) λ1 = 5, λ2 = 4; (b) λ1 = 21 (5 + (5)), λ2 = 21 (5 − (5)); (c) λ1 = 3, λ2 = λ3 = and (d) λ1 = −3, λ2 = −1, λ3 = P7.6 (a) λ1 = 1, eλ1 = (1, 1)T , T λ2 = −1, eλ2 = (1, −1) ; (b) λ1 = 3, eλ1 = (1, 3, 9)T , λ2 = 2, eλ2 = (1, 2, 4)T , 2 10 −1 P7.7 A10 = λ3 = −1, eλ3 = (1, −1, 1)T 765854 282722 706805 341771 4y0 + z0 )T P7.9 = P7.8 (x∞ , y∞ , z∞ )T = 16 (x0 + 4y0 + z0 , x0 + 4y0 + z0 , x0 + (a) Yes; (b) No; (c) Yes P7.10 No P7.11 No P7.14 eˆ1 = (0, 1), eˆ2 = (−1, 0) P7.15 eˆ1 = ( √1 , √1 ), eˆ2 = (− √1 , √1 ) P7.16 eˆ1 = ( √3 , √1 ), −1 eˆ2 = ( √ , 2 −5 √3 ) 10 101 P7.17 A = QΛQ−1 = = 10 0 −3 100 √ P7.18 a) 5;b) + 3i; c) + 5i; d) 26 P7.19 A + B = 10 10 00 −3 0 20 −1 02 8+3i −5+3i ; CB = 3+8i 2−i 45+3i −33+31i 16−4i 16+8i 8−i ; (2 + i)B = 9+7i P7.20 (a) λ1 = + i and −15+5i √ √ λ2 = − i; (b) λ1 = 3i and λ2 = 3i; (c) λ1 = + 8i and λ2 = − 8i 100 000 000 000 000 001 P7.22 d = P7.23 P7.25 e1 = , e2 = 000 010 000 d = P7.24 (a) Yes (b) No (c) Yes (d) No (e)√Yes (f ) Yes 1+i Q = −1+i , Λ = 00 02 P7.26 (a) 9; (b) 3; (c) 10 2 P7.21 , e3 = Solutions to selected problems P7.2 To find the eigenvalues of the matrix A we must find the roots of it’s characteristic polynomial p(λ) = det(A − λ✶) = det 1 λ − 0 λ = det 1−λ 1 −λ Using the determinant formula det ac db = ad − bc, we find the characteristic polynomial of A is p(λ) = λ2 − λ − The eigenvalues of A are the roots λ1 and λ2 of this equation, which we can find using the formula for solving quadratic equations we saw in Section 1.6 (see page 27) P7.3 The vector e1 is an eigenvector of A because Ae1 = Now observe the following interesting fact: This means we can write Ae1 = 11 10 1 ϕ (1 ϕ + = ϕ ) ϕ 1 ϕ = 1 1+ ϕ 11 = 10 ϕ 1 + ϕ12 = ϕ+1 = ϕ ϕ2 , which shows that e1 is an eigenvector of A and it corresponds to eigenvalue λ1 = ϕ Similar reasoning shows Ae2 = − ϕ e2 so e2 is an eigenvector of A that corresponds to eigenvalue λ2 = − ϕ 488 P7.4 The eigendecomposition of matrix A is A = QΛQ−1 The unknown matrix X is the inverse matrix of the matrix Q = ϕ 1 −ϕ = 2√ 1+ √ − 1+2 To find we can start from the array [ Q | ✶ ] and perform row operations until we obtain [ ✶ | Q−1 ] P7.6 TODO: add steps of solution P7.7 First we decompose A as the product of three matrices A = QΛQ−1 , where = Q is a matrix of eigenvectors, and Λ contains the eigenvalues of A A = 25 −1 Q−1 −2 −3 0 power Λ10 = A10 = −1 7 7 Since the matrix Λ is diagonal, we can compute it’s fifth Thus expressing the calculation of A10 59049 0 1048576 −2 1 59049 −1 7 1048576 7 = 765854 706805 282722 341771 P7.8 The eigenvalues of M are 14 , 21 , and 1, and its eigendecomposition is M = QΛQ−1 We can compute (x∞ , y∞ , z∞ )T using M ∞ (x0 , y0 , z0 )T To compute M ∞ , we can compute Λ∞ The 41 and 12 eigenspaces will disappear, so we’ll be left only with the subspace of the eigenvalue M ∞ = QΛ∞ Q−1 , and each row of this matrix has the form [ 16 , 64 , 61 ] See bit.ly/eigenex001 for details P7.9 To check if the matrix is orthogonal the transpose of that matrix should act as an inverse such that OT O = ✶ P7.10 A vector space would obey · (a1 , a2 ) = (0, 0) (the zero vector), but we have · (a1 , a2 ) = (0, a2 ) = (0, 0), so (V, R, +, ·) is not a vector space P7.11 The vector addition operation is not associative: we have ((a1 , a2 ) + (b1 , b2 )) + (c1 , c2 ) = (a1 + 2b1 + 2c1 , a2 + 3b2 + 3c2 ) but (a1 , a2 ) + ((b1 , b2 ) + (c1 , c2 )) = (a1 + 2b1 + 4c1 , a2 + 3b2 + 9c2 ) P7.12 We proceed using the following chain of inequalities: u+v = u + v, u + v = u, u + u, v + v, u + v, v ≤ u + u, v + v ≤ u +2 u v + v 2 =( u + v ) Therefore we obtain the equation u + v ≤ ( u + v )2 , and since u and v are non-negative numbers, we can take the square root on both sides of the equation to obtain u + v ≤ u + v P7.13 If v = 0, then the inequality holds trivially If v = 0, we can start from the following which holds for any c ∈ C: ≤ u − cv, u − cv = u, u − cv − c v, u − cv = u, u − c u, v − c v, u + |c|2 v, v This is true in particular when c = ≤ u, u − u,v v,v , so we continue u, v u, v u, v u, v | u, v |2 v, v − + v, v v, v v, v ≤ u, u − | u, v |2 | u, v |2 | u, v |2 − + v, v v, v v, v ≤ u, u − | u, v |2 v, v ≤ u, u v, v − | u, v |2 , 489 from which we conclude | u, v |2 ≤ u v Taking the square root on both sides we obtain the the Cauchy–Schwarz inequality | u, v | ≤ u v P7.14 Let v1 = (0, 1) and v2 = (−1, 0) We don’t really need to perform the Gram–Shmidt procedure since v2 is already perpendicular to v1 and both vectors have unit length P7.15 We’re given the vectors v1 = (1, 1) and v2 = (0, 1) and want to perform the Gram–Shmidt procedure We pick e1 = v1 = (1, 1) and after normalization e1 · v2 )ˆ e1 = we have eˆ1 = ( √1 , √1 ) Next we compute e2 = v2 − Πeˆ1 (v2 ) = v2 − (ˆ 2 ( −1 , 12 ) Normalizing e2 we obtain eˆ2 = (− √1 , 2 √1 ) P7.16 Let v1 = (3, 1) and v2 = (−1, 1) We start by identifying e1 = v1 , then perform Gram–Shmidt process to find e2 from v2 : e2 = v2 − Πeˆ1(v2 )ˆ e1 = v2 − e1 · v2 e1 e1 e1 = −2 , 5 Now we have two orthogonal vectors and we can normalize them to make them −1 √3 unit length We obtain the vectors ( √3 , √1 ) and ( √ , ) which form an 10 10 10 10 orthogonal basis P7.17 First you have to find eigenvalues of the given matrix.1,1 Then find an eigenvector for each eigenvalue Then construct a matrix Q composed of the eigenvectors Finally, write the eigendecomposition of the form A = QΛQ−1 √ P7.18 a) 32 + 42 = 5; b) Only the sign of imaginary part changes √ so it becomes + 3i; c) 3i − + + 2i = + 5i; d) |3i − 4i − 5| = | − − i| = 26 P7.21 First of all we must determine dimensionality of the vector space in question The general vector space of × matrices has dimensions, but a diagonal matrix A satisfies aij = for all i = j, which corresponds to the following six constraints {a12 = 0, a13 = 0, a21 = 0, a23 = 0, a31 = 0, a32 = 0} The vector space of diagonal matrices is therefore three dimensional The answer given is the standard basis Other answers are possible so long as they span the same space, and there are three of them P7.22 A matrix A ∈ R3×3 is symmetric if and only if AT = A This means we can pick the entries on the diagonal arbitrarily, but the symmetry requirement leads to the constraints a12 = a21 , a13 = a31 , and a23 = a32 Thus space of × symmetric matrices is six dimensional P7.23 A Hermitian matrix H is a matrix with complex coefficients that satisfies H = H † , or equivalently hij = hji , for all i, j A priori the space of × matrices with complex coefficients is 18 dimensional, for the real and imaginary parts of each of the nine coefficients The Hermitian property imposes two types of constraints Diagonal elements must be real if we want hii = hii to be true, which introduces three constraints Once we pick the real and imaginary part of an off-diagonal element aij , we’re forced to choose aji = aij , which are another constraints Thus, the vector space of × Hermitian matrices is 18 − − = dimensional P7.24 To prove a matrix is nilpotent we can either compute its powers to see if we get the zero matrix To prove a matrix is not nilpotent, you can show it has a non-zero eigenvalue (a) This matrix is nilpotent because its square is the zero matrix (b) The matrix is not nilpotent because its characteristic polynomial p(λ) = λ2 − 6λ + is different from the all-zero eigenvalues characteristic polynomial p(λ) = (λ − 0)(λ − 0) = λ2 (c) This matrix is nilpotent because it squares to the zero matrix (d) The matrix is not nilpotent because it has non-zero eigenvalues 490 (e) Yes, the cube of this matrix is the zero matrix (f ) Yes, the square of this matrix is the zero matrix P7.25 The characteristic polynomial of A is pA (λ) = λ2 − 2λ = λ(λ − 2) so the eigenvalues are and The eigenvector for λ = is e0 = (−1 + i, 2)T The eigenvector for λ = is e2 = (1+i, 2)T As A has two one-dimensional eigenspaces, an eigenbasis is given by {(−1 + i, 2)T , (1 + i, 2)T } So A is diagonalizable with 1+i Q = −1+i and Λ = 00 02 2 P7.26 (a) Using linearity: v1 , 2v2 + 3v3 = v1 , v2 + v1 , v3 = + = (b) Using linearity in both entries: 2v1 − v2 , v1 + v3 = v1 , v1 + v1 , v3 − v2 , v1 − v2 , v3 = v1 , v1 + v1 , v3 − v1 , v2 − v2 , v3 = + − + = (c) We start with 13 = v2 , v1 + v2 = v2 , v1 + v2 , v2 , and since √ we know v1 , v2 = 3, we obtain + v2 = 13, so v2 = 10 and v2 = 10 P7.27 (a) There are several ways to answer this question One way is to note that the dimension of P2 is and so if Ba is a linearly independent set, then it is a basis as it has elements If a(1 + ix) + b(1 + x + ix2 ) + c(1 + 2ix) = then (a + b + c)1 + (ia + b + 2ic)x + ibx2 = But the standard basis of P2 is linearly independent and so we must have a + b + c = 0, ia + b + 2ic = and ib = The last equation implies b = and then the first two imply both a and c are zero As the only linear combination of distinct elements of Ba which sums to zero is the trivial sum, Ba is linearly independent Chapter solutions Answers to exercises E8.1 4Al + 3O2 → 2Al2 O3 E8.2 Fe(OH)3 + 3HCl → FeCl3 + 3H2 O E8.3 √ √ √ N N an+1 = λ1 = ϕ = 1+2 E8.4 aN = √1 1+2 − √1 1−2 an 5 S(m∗ ) = 4704.63 E8.6 S(m ∗ ) = 433.54 E8.7 c = xG = (1, 0, 1, 0, 1, 0, 1) limn→∞ E8.5 E8.8 A × identity matrix next to an all-ones vectors Solutions to selected exercises E8.4 Another expression is Fn = ϕn −(−ϕ)−n √ E8.9 To find the inner product e1 (x), e2 (x) , we must compute the integral L π 2π π sin( L x) sin( L x) dx We can change variables y = L x to simplify the inteL gral, changing also dx → π dy and the limits We thus have e1 (x), e2 (x) = k 0π sin(y) sin(2y) dy, where k = L Using the double angle formula we find π sin(y) sin(2y) = sin(y)2 sin(y) cos(y) = sin2 (y) cos(y) We proceed using the substitution u = sin(y), du = cos(y)dy to obtain 2k sin2 (y) cos(y) dy = 2k u2 du = 2k u3 = 2k sin3 (y) Finally, we evaluating the expression at the endpoints: 3 e1 (x), e2 (x) = 2k sin3 (π) − sin3 (0) = 0, confirming orthogonality E8.10 The coefficient b0 corresponds to the basis function sin( 0π t), which is zero T everywhere Thus, the integral b0 = T1 0T f (t) sin( 2π0 t) dt, always produces a T zero result b0 = On the other hand, the zero-frequency cos function is constant t) = 1, and the coefficient a0 = T1 0T f (t)1 dt corresponds to the average cos( 2π0 T value of the function f (t) on the interval [0, T ] 491 Answers to problems Solutions to selected problems P8.1 Decompose the formula for cn into a its real part and imaginary parts: T ei cn = 2πn t T , et f (t) dt = T T T = Re = = T T f (t)e−i 2πn t T dt T T f (t)e−i 2πn t T dt T + Im f (t)Re e−i 2πn t T dt + T 2πn t T f (t) cos dt − T T T f (t)e−i 2πn t T dt i f (t)Im e−i 2πn t T i dt T T f (t) sin − 2πn t i dt T We can recognize the real part of cn as the cosine coefficients an of the Fourier series, and the imaginary part of cn as the negative of the sine coefficients bn of the Fourier series: Re {cn } = Re Im {cn } = Im T T T f (t)e−i 2πn t T dt = T f (t)e−i 2πn t T dt = T T T f (t) cos T 2πn t T dt = an , f (t) sin − 2πn t dt = −bn T Thus we have shown cn = an − ibn Using the simple definition of the coefficients cn above, the synthesis equation for the complex Fourier transform is a somewhat awkward expression f (t) = c0 + ∞ t −i 2πn T cn n=1 e 2πn −i T t cn Many textbooks describe complex + 12 −1 n=−∞ e Fourier series in terms of the two-sided coefficients cn , n ∈ Z defined as c0 = c0 , cn = c−n = c n c n = = (an (an − ibn ) for n ≥ 1, + ibn ) for n ≥ Using the coefficients cn , the synthesis equation for the complex Fourier series is the simpler expression f (t) = ∞ t −i 2πn T cn n=−∞ e Chapter solutions Answers to exercises E9.1 (a) No; weights don’t add to one (b) Yes (c) No; contains a negative , 14 , 13 E9.5 pX = 34 , 14 , 13 E9.6 number E9.4 pX = 34 61 61 61 61 61 61 pX ∞ ∞ = (0.2793, 0.1457, 0.2768, 0.02, 0.2781)T ∞ Solutions to selected exercises E9.4 Use ev = C.eigenvects()[0][2][0] to extract the eigenvector that correspond to the eigenvalue λ = 1, then normalize the vector by its 1-norm to make 34 14 13 it a probability distribution pinf = ev/ev.norm(1) = , , 61 61 61 492 E9.5 Define the matrix C then use ev = (C-eye(3)).nullspace()[0] to obtain an eigenvector that corresponds the eigenvalue λ = To obtain a probability dis34 14 13 , 61 , 61 Using the nullspace method tribution, compute ev/ev.norm(1) = 61 is a more targeted approach for finding stationary distributions than using the eigenvects method We don’t need to compute all the eigenvectors of C, just the one we’re interested in E9.6 You can see the solution at this URL: bit.ly/21GOUCe Answers to problems P9.1 Pr({heads, heads, heads}) = pX ∞ = ( 31 , 23 )T P9.2 pX = ( 12 , 12 )T , pX = ( 38 , 85 )T , Solutions to selected problems P9.1 Substitute p = 21 and n = into the expression pn P9.2 Defined Xi to be probability distribution of the weather in Year i The transition matrix for the weather Markov chain is M = weather in Year using pX = M (1, 0)T = ( 21 , 12 )T 2 4 We obtain the The weather in Year is pX = M (1, 0)T = ( 83 , 58 )T The long term (stationary) distribution is pX ∞ M ∞ (1, 0)T = ( 31 , 23 )T = Chapter 10 solutions Answers to exercises E10.1 u = (α, 0, 0β)T (column vector), v T = (0, a, b,√ 0) (row vector) E10.2 |w = 1|0 + i|1 − |2 , w| = 0| − i 1| − 2|, w = E10.3 det(A) = −2 E10.6 See solution E10.7 Pr({−}|ψ) = (α−β)2 Solutions to selected exercises E10.5 Note XY is not the same as Y X E10.6 We can rewrite the definition of |+ and |− in order to obtain expressions for the elements of the standard basis |0 = √1 |+ + √1 |− , and |1 = √1 |+ − √1 |− 2 2 The remainder of the procedure is straightforward: |01 − |10 = |0 |1 − |1 |0 = √1 |+ − = + √1 |− √1 |+ √1 |+ − √1 |− − √ |− √1 |+ + √1 |− ✘ − |+ |− + |− |+ − ✘ ✘ |+✘ |+ |−✘ |− ✘ ✘ − |+ |− + |− |+ + ✘ ✘ −✘ |+✘ |+ |−✘ |− − 2|+ |− + 2|− |+ = −(|+ |− − |− |+ ) = = |+ |− − |− |+ 493 The last equality holds because the global phase of the states doesn’t matter Answers to problems P10.1 No, since Q is not unitary P10.2 HH|0 = |0 and HH|1 = |1 P10.3 parameters P10.4 α ∈ [0, 1] and ϕ ∈ [0, 2π] P10.5 θ ∈ [0, π] and ϕ ∈ [0, 2π] P10.6 |R3 = |ψ , since the state was teleported to Bob’s register Solutions to selected problems P10.1 Quantum operators are unitary Since QQ† = ✶, Q is not unitary and it cannot be implemented by any physical device The boss is not always right! P10.2 Let’s first see what happens to |0 when we apply the operator HH The result of the first H applied to |0 is H|0 = |+ = √1 (|0 + |1 ) Then applying the second H operator we get HH|0 = H|+ = ing the H operation gives √1 √2 |0 √1 √1 (|0 + |1 ) + √1 (|0 2 √1 (H|0 + H|1 ) Apply- − |1 ) , which simplifies to = |0 So HH|0 = |0 A similar calculation shows HH|1 = |1 P10.3 Starting from the four degrees of freedom for a general two-dimensional complex vectors, we must subtract one degree of freedom for each of the constraints: one because we’re ignoring global phase, and one because we require |α|2 + |β|2 = 1: d.f − α real − { |ψ = 1} = d.f A qubit |ψ has only two degrees of freedom In other words two parameters are sufficient to describe any qubit P10.5 These are the angles on the Bloch sphere P10.6 Check this video for a sketch of the solution: youtu.be/3wZ35c3oYUE Appendix B Notation This appendix contains a summary of the notation used in this book Math notation Expression Read as Used to denote a, b, x, y = ≡ a+b a−b a × b ≡ ab a2 ≡ aa a ≡ aaa an √ a ≡ a2 √ a ≡ a3 a/b ≡ ab a−1 ≡ a1 is equal to is defined as a plus b a minus b a times b a squared a cubed a exponent n square root of a cube root of a a divided by b one over a variables expressions that have the same value new variable definitions the combined lengths of a and b the difference in lengths between a and b the area of a rectangle the area of a square of side length a the volume of a cube of side length a a multiplied by itself n times the side length of a square of area a the side length of a cube with volume a a parts of a whole split into b parts division by a f (x) f −1 f ◦g f of x f inverse f compose g θ, φ sin, cos, tan % theta, phi sin, cos, tan percent ex ln(x) ax loga (x) e to the x natural log of x a to the x log base a of x the function f applied to input x the inverse function of f (x) function composition; f ◦ g(x) ≡ f (g(x)) the the the the exponential function base e logarithm base e exponential function base a logarithm base a angles trigonometric ratios proportions of a total; a% ≡ 494 a 100 SET NOTATION 495 Set notation You don’t need a lot of fancy notation to math, but it really helps if you know a little bit of set notation Symbol { } | N Z Q R C Fq Read as Denotes the set such that the naturals the integers the rationals the reals definition of a set describe or restrict the elements of a set the set N ≡ {0, 1, 2, } Also N+≡ N\{0} the set Z ≡ { , −2, −1, 0, 1, 2, 3, } the set of fractions of integers the set of real numbers the set of complex numbers the set {0, 1, 2, 3, , q − 1} finite field subset subset or equal union intersection S set minus T a in S a not in S for all x there exists x there doesn’t exist x ⊂ ⊆ ∪ ∩ S\T a∈S a∈ /S ∀x ∃x x one set strictly contained in another containment or equality the combined elements from two sets the elements two sets have in common the elements of S that are not in T a is an element of set S a is not an element of set S a statement that holds for all x an existence statement a non-existence statement An example of a condensed math √ statement that uses set notation is “ m, n ∈ Z such that m = 2,” which reads “there don’t exist n √ integers m and n whose fraction equals 2.” Since we identify the set of fraction of integers with the rationals, √ √ this statement is equivalent to the shorter “ ∈ / Q,” which reads “ is irrational.” Vectors notation Expression R n v (vx , vy ) vxˆı + vy ˆ v ∠θ v θ vˆ ≡ vv u·v u×v Denotes the set of n-dimensional real vectors a vector vector in component notation vector in unit vector notation vector in length-and-direction notation length of the vector v angle the vector v makes with the x-axis unit length vector in the same direction as v dot product of the vectors u and v cross product of the vectors u and v COMPLEX NUMBERS NOTATION 496 Complex numbers notation Expression C i Re{z} = a Im{z} = b √ |z|∠ϕz |z| = a2 + b2 ϕz = tan−1 (b/a) z¯ = a − bi Cn Denotes the set of complex numbers C ≡√{a + bi | a, b ∈ R} the unit imaginary number i ≡ −1 or i2 = −1 real part of z = a + bi imaginary part of z = a + bi polar representation of z = |z| cos ϕz + i|z| sin ϕz magnitude of z = a + bi phase or argument of z = a + bi complex conjugate of z = a + bi the set of n-dimensional complex vectors Vector space notation Expression U, V, W W ⊆V {v ∈ V | cond } span{v1 , , } dim(U ) R(M ) N (M ) C(M ) N (M T ) rank(M ) nullity(M ) Bs {e1 , , en } {ˆ e1 , , eˆn } B [✶]B ΠS ΠS ⊥ Denotes vector spaces vector space W subspace of vector space V subspace of vectors in V satisfying condition cond span of vectors v1 , , dimension of vector space U row space of M null space of M column space of M left null space of M rank of M ; rank(M ) ≡ dim(R(M )) = dim(C(M )) nullity of M ; nullity(M ) ≡ dim(N (M )) the standard basis an orthogonal basis an orthonormal basis the change-of-basis matrix from basis B to basis B projection onto subspace S projection onto the orthogonal complement of S NOTATION FOR MATRICES AND MATRIX OPERATIONS 497 Notation for matrices and matrix operations Expression m×n R A aij |A| A−1 AT ✶ AB Av wTA uTv uv T ref(A) rref(A) rank(A) A∼A R1 , R2 , ER [A | b ] [A | B ] Mij adj(A) (ATA)−1AT Cm×n A† Denotes the set of m × n matrices with real coefficients a matrix entry in the ith row and j th column of A determinant of A, also denoted det(A) matrix inverse matrix transpose identity matrix; ✶A = A✶ = A and ✶v = v matrix-matrix product matrix-vector product vector-matrix product vector-vector inner product; uTv ≡ u · v vector-vector outer product row echelon form of A reduced row echelon form of A rank of A ≡ number of pivots in rref(A) matrix A obtained from matrix A by row operations row operations, of which there are three types: → Ri ← Ri + kRj : add k-times row j to row i → Ri ↔ Rj : swap rows i and j → Ri ← mRi : multiply row i by constant m elementary matrix that corresponds R; R(M ) ≡ ER M augmented matrix containing matrix A and vector b augmented matrix array containing matrices A and B minor associated with entry aij See page 167 adjugate matrix of A See page 169 generalized inverse of A See page 330 the set of m × n matrices with complex coefficients Hermitian transpose; A† ≡ (A)T NOTATION FOR LINEAR TRANSFORMATIONS 498 Notation for linear transformations Expression n m T :R →R MT ∈ Rm×n Dom(T ) ≡ Rn CoDom(T ) ≡ Rm Im(T ) ≡ C(MT ) Ker(T ) ≡ N (MT ) S ◦ T (x) M ∈ Rm×n TM : Rn → Rm TM T : Rm → Rn Denotes linear transformation T (a vector function) matrix representation of T domain of T codomain of T the image space of T the kernel of T composition of linear transformations; S ◦ T (x) ≡ S(T (x)) ≡ MS MT x an m × n matrix the linear transformation defined as TM (v) ≡ M v the adjoint linear transformation TM T (a) ≡ aTM Matrix decompositions Expression n×n A∈R pA (λ) ≡ |A − λ✶| λ1 , , λ n Λ ∈ Rn×n eλ1 , , eλn Q ∈ Rn×n A = QΛQ−1 A = OΛOT B ∈ Rm×n σ1 , σ , Σ ∈ Rm×n u1 , , um U ∈ Rm×m v1 , , V ∈ Rn×n B = U ΣV T Denotes a matrix (assume diagonalizable) characteristic polynomial of A eigenvalues of A = roots of pA (λ) ≡ n i=1 (λ − λi ) diagonal matrix of eigenvalues of A eigenvectors of A matrix whose columns are eigenvectors of A eigendecomposition of A eigendecomposition of a normal matrix a generic matrix singular values of B matrix of singular values of B left singular vectors of B matrix of left singular vectors of B right singular vectors of B matrix of right singular vectors of B singular value decomposition of B ABSTRACT VECTOR SPACE NOTATION 499 Abstract vector space notation Expression Denotes (V, F, +, ·) abstract vector space of vectors from the set V , whose coefficients are from the field F , addition operation “+” and scalar-multiplication operation “· ” abstract vectors inner product of vectors u and v norm of u distance between u and v u, v, w u, v u d(u, v) ... No bullshit guide to linear algebra Ivan Savov April 15, 2016 No bullshit guide to linear algebra by Ivan Savov Copyright © Ivan Savov, 2015,... solutions B Notation Math notation Set notation Vectors notation Complex numbers notation Vector space notation Notation... (x) serves to approximate the function f (x) near xo Using linear algebra techniques to model nonlinear phenomena can be understood as a multivariable generalization of this idea Linear models