Chapter 5 eigenvalue eigenvector (1)

Eigenvalues, eigenvectors, diagonalization Phan Thi Khanh Van E-mail: khanhvanphan@hcmut.edu.vn May 13, 2021 (Phan Thi Khanh Van) Eigenvalues, eigenvectors, diagonalization May 13, 2021 / 26 Table of contents Eigenvalues, eigenvectors Diagonalization Leslei model: Population growth Markov chain Application of singular value decomposition (SVD) (Phan Thi Khanh Van) Eigenvalues, eigenvectors, diagonalization May 13, 2021 / 26 Eigenvalues, eigenvectors Let A ∈ Mn be a square matrix λ is called an eigenvalue of A if there exist x = 0, x ∈ Rn satisfying: Ax = λx x is then called an eigenvector of A associated with λ Example A= 2 Are e1 = , e2 = , e3 = eigenvectors of A? 1 −1 Ae2 = Ae3 = Ae1 = 2 1 = = 3e1 - eigenvector with λ = 3 = = λe2 - e2 is not an eigenvector −1 = = −e3 - eigenvector with λ = −1 −1 (Phan Thi Khanh Van) Eigenvalues, eigenvectors, diagonalization May 13, 2021 / 26 Properties If x is an eigenvector of A associated with the eigenvalue λ then: αx, α = is also an eigenvector of A assosiated with λ λm is an eigenvalue of Am , x is the corresponding eigenvector If A is invertible, then λ = and 1/λ is an eigenvalue of A−1 , x is the corresponding eigenvector Example Given A = , is λ = an eigenvalue of A? Consider the equation Ax = λx = 1.x = x 0 ⇔ (A − I )x = ⇔ ∼ 0 0 x1 = m x2 = We can find x = (m, 0) = (if m = 0) such that Ax = x Hence, λ = is an eigenvalue of A ⇔ (Phan Thi Khanh Van) Eigenvalues, eigenvectors, diagonalization May 13, 2021 / 26 Remark λ is an eigenvalue of A ∈ Mn ⇔ ∃x = 0, x ∈ Rn satisfying: Ax = λx ⇔ (A − λI )x = has nontrivial solution ⇔ det(A − λI ) = - characteristic equation of A The polynomial PA (λ) = det(A − λI ) is called the characteristic polynomial of A Example   1 m 1 Find m such that is an eigenvalue of A = −2 −1 −2 is an eigenvalue of A ⇔ |A − 2I | = −1 m ⇔ −2 −1 = ⇔ 6m − = ⇔ m = −1 −4 (Phan Thi Khanh Van) Eigenvalues, eigenvectors, diagonalization May 13, 2021 / 26 How to find the eigenvectors, eigenvalues Find PA (λ) = |A − λI |, and then solve |A − λI | = to find λ For every λi solve the system (A − λi I )x = to find the corresponding eigenvectors x The subspace of Rn of all eigenvectors assosiated with λi is called the eigenspace of A assosiated with λi , and is denoted by Eλi Theorem All vector in the bases of the eigenspaces form a linearly independent set (Phan Thi Khanh Van) Eigenvalues, eigenvectors, diagonalization May 13, 2021 / 26 Example Find all eigenvalues and eigenvectors of: A = 2−λ = (2 − λ)(3 − λ) − 12 3−λ = λ2 − 5λ − = ⇔ λ = −1 ∨ λ = λ = −1: Consider the system (A + I )x = 3 −1 ⇔x =m , Eigenspace: E−1 = span 4 λ = 6: Consider the system (A − 6I )x = −4 3 ⇔x =m , Eigenspace: E6 = span −3 |A − λI | = (Phan Thi Khanh Van) Eigenvalues, eigenvectors, diagonalization −1 May 13, 2021 / 26 Example   6  Find all eigenvalues and eigenvectors of: A = −2 −12 −11 |A − λI | = −λ3 + trace(A)λ2 − (A11 + A22 + A33 )λ + |A| = −λ3 − 4λ2 − (6 − 23 + 10)λ + 10 = ⇔ λ = −5 ∨ λ = −1 ∨ λ = λ  = −5: Consider the system  (A + 5I )x =  0 35 21  −2 11  ∼  −2 11  −12 −6 0 10 ⇔ x = m(−3, −6, 10), Eigenspace: E−5 = span {(−3, −6, 10)} , λ  = −1: Consider (A +  I )x =  2 2 0  −2 ⇔ 0  9 −12 −10 0 −16 −16 ⇔ x = m(−1, −2, 2) Eigenspace: E−1 = span {(−1, −2, 2)} (Phan Thi Khanh Van) Eigenvalues, eigenvectors, diagonalization May 13, 2021 / 26 λ = 2: Consider (A −  2I )x =  0  −1 −1  −2 0  ⇔ −12 −13 0 −4 −1 ⇔ x = m(10, −1, 4) Eigenspace: E2 = span {(10, −1, 4)} , dim(E2 ) = (Phan Thi Khanh Van) Eigenvalues, eigenvectors, diagonalization May 13, 2021 / 26 Example  3 Find all eigenvalues, eigenvectors of A = −3 −5 −3 3  |A − λI | = −λ3 − 3λ2 − (4 + − 8)λ + = ⇔ λ = −2, (multiplicity =2), ∨λ = 1, (multiplicity=1) λ = 1: Consider (A  − I )x =  3  −3 −6 −3  ⇔ x = m(1, −1, 1) 3 0 E1 = span {(1, −1, 1)}, dim(E1 ) = λ = −2: Consider (A   + 2I )x = 3  −3 −3 −3  ∼ 1 3 ⇔ x = m(−1, 1, 0) + n(−1, 0, 1) E−2 = span {(−1, 1, 0), (−1, 0, 1)}, dim(E−2 ) = (Phan Thi Khanh Van) Eigenvalues, eigenvectors, diagonalization May 13, 2021 10 / 26 Exercises Find the eigenvalues, eigenvectors of the following matrices   15 −18 −16 A =  −12 −8  −4 −6   −1 −1 A = 2 −2 −4 −4 −3 (Phan Thi Khanh Van) Eigenvalues, eigenvectors, diagonalization May 13, 2021 12 / 26 Similar matrices A matrix A is called similar to B if there exists an invertible matrix P : A = PBP −1 Diagonalizable matrix Let A ∈ Mn be a square matrix A is called diagonalizable if it is similar to a diagonal matrix: ∃P : P −1 AP = D, where D is a diagonal matrix Theorem A is diagonalizable ⇔ A has n independent eigenvectors If A has exactly n distinct eigenvalues then A is diagonalizable Theorem If A is diagonalizable then we can choose P = [e1 en ] where the column vectors ei are independent eigenvectors of A, D = diag (λ1 λn ), λi − corresponding eigenvalues (Phan Thi Khanh Van) Eigenvalues, eigenvectors, diagonalization May 13, 2021 13 / 26 How to diagonalize a matrix Solve |A − λI | = to find the eigenvalues λ1 , λn Solve the system (A − λ)x = to find the corresponding eigenvectors of A: e1 , e2 en   λ1  λ2    D =   , P = [e1 e2 en ] (col vectors)   0 (Phan Thi Khanh Van) λn Eigenvalues, eigenvectors, diagonalization May 13, 2021 14 / 26 Consider the examples 3,4,5,6 Example 3: A has independent eigenvectors ⇒ A is diagonalizableA = PDP −1 −1 −1 D= ,P= Example 4: A has3 distinct eigenvalues A = PDP −1   ⇒ A is diagonalizable  −5 0 −3 −1 10    D = −1 , P = −6 −2 −1 0 10 Example 5: A has3 independent ⇒ A is diagonalizableA = PDP −1  eigenvectors  −2 0 −1 −1    −1 D = −2 , P = 0 1 Example 6: A is not diagonalizable (Phan Thi Khanh Van) Eigenvalues, eigenvectors, diagonalization May 13, 2021 15 / 26 Remark If A is diagonalizable A = PDP −1 then An =(PDP −1 )(PDP −1 ) (PDP −1 ) = PD n P −1  λn1  λn   −1  = P  P   0 λnn Consider Example A2018 = PD 2018 P −1 (Phan Thi Khanh Van)  2018  0  P −1 =P 2018 0 Eigenvalues, eigenvectors, diagonalization May 13, 2021 16 / 26 Exercises Diagonalize the following matrices (if it is possible) A= −1   −2 A = 1  Find A2017   −6 A = 1 −2 1   A = −4 0 −2 (Phan Thi Khanh Van)   −1 −1 A = −3 −1 −3 Find B : B = A 0 A = 1 −1 1   2 A = 2 6   −1 −1 A = −1 −1 −1 −1 Eigenvalues, eigenvectors, diagonalization May 13, 2021 17 / 26 Population growth Matrices can be used to form models for population growth The first step in this process is to group the population into age classes of equal duration For instance, if the maximum life span of a member is L years, then the age classes are represented by the n intervals: First age class: [0, Ln ), Second (n−1)L , L) The number of population age class [ Ln , 2L n ), n−th age class: [ n members in each age class is then represented   by the age distribution x1 Number in first age class  x2 Number in second age class  vector    xn Number in n-th age class (Phan Thi Khanh Van) Eigenvalues, eigenvectors, diagonalization May 13, 2021 18 / 26 Example A population of rabbits raised in a research laboratory has the characteristics listed below (a) Half of the rabbits survive their first year Of those, half survive their second year The maximum life span is years (b) During the first year, the rabbits produce no offspring The average number of offspring is during the second year and during the third year The laboratory population now consists of 24 rabbits in the first age class, 24 in the second, and 20 in the third Find a stable age distribution vector To find a stable age distribution vector, we have to find x : Ax = λx |A − λI | = ⇔ λ = −1 ∨ λ = Choosing the positive value λ = To find a corresponding eigenvector, row reduce the matrix (A − 2I ) to   16 obtain x =   (Phan Thi Khanh Van) Eigenvalues, eigenvectors, diagonalization May 13, 2021 19 / 26 Example Two competing companies offer satellite television service to a city with 100000 households The transition matrix is   0.7 0.15 0.15 A = 0.2 0.8 0.15 (see the 0.1 0.05 0.7 figure below for the changes in satellite subscriptions each year) Company A now has 10000 subscribers and Company B has 15000 subscribers Using the method of diagonalizing the transition matrix to find the numbers of subscribers after 50 years Round each answer to the nearest integer (Phan Thi Khanh Van) Eigenvalues, eigenvectors, diagonalization May 13, 2021 20 / 26 Let x, y be the numbers of subscribers of companies A and B, and z = 100000 − (x + y ) be the numbers of households that don’t use any services time: x0 = 10000, y0 = 15000, z0 = 75000   At this  10000, x0 y0  =  15000  75000 z0     x100 x0 After 50 years y100  = A50 y0  z100 z0 Use the method of diagonalization: A = PDP −1 , we can find A50 = PD 50 P −1 (Phan Thi Khanh Van) Eigenvalues, eigenvectors, diagonalization May 13, 2021 21 / 26 SVD Let A be an m × n matrix Then AAT and AT A are symmetric matrices Hence, they can be orthogonally diagonalized: AAT = QD1 Q T , AT A = PD2 P T We can easily prove that AAT and AT A have the same set of nonzero eigenvalues and these values are positive: λ21 > λ22 > > λ2r > Then, D - an m × n matrix; A = QΣP T (SVD), where Σ = 0 D = diag ([λ1 , λ2 , λr ])- a diagonal matrix of order r λi are called singular values of A, Q = [q1 qm ], P = [p1 pn ] ⇒ A = λ1 q1 p1T + λ2 q2 p2T + + λr qr prT A = Qr DPrT -compact SVD of A (Phan Thi Khanh Van) Eigenvalues, eigenvectors, diagonalization May 13, 2021 22 / 26 Example Find the SVD of A = 2 −2 17 25 D1 =  13 AT A = 12 , Orthogonally diagonalize: AAT = QD1 Q T where 17 1 , Q = √12 −1  12 13 −2, Orthogonally diagonalize: −2     −2 √ √1 25 0 18  23  A = PD2 P T , where D2 =  0 , P =  √12 √−1  18 0 √ 18 AAT = Hence, the SVD of A: A = QΣP T , where Σ = (Phan Thi Khanh Van) Eigenvalues, eigenvectors, diagonalization 0 May 13, 2021 23 / 26 Application of SVD in image compression The singular values σi are ordered σ1 ≥ σ2 ≥ ≥ σn ≥ Significant compression of the image is possible if the spectrum of singular values has only a few very strong entries A ≈ Ak = σ1 q1 p1T + σ2 q2 p2T + + σk qk pkT Example We have an image flower jpg Transform the image into a matrix using Matlab: A=rgb2gray(imread(’flower.jpg’));, A is 432 × 432 Change to the double type: A = im2double(A); Find SVD of A: [U S V]=svd(A); sigma = diag(S); (Phan Thi Khanh Van) Eigenvalues, eigenvectors, diagonalization May 13, 2021 24 / 26 The original image: figure; subplot(4,2,1); imshow(A); title(’Original’); Choose the number k to approximate A: ranks=[200,100,50,30,20,10,5]; for i = : length(ranks) approxsigmas = sigma; approxsigmas(ranks(i):end)=0; ns=length(sigma); approxS = S; approxS(1:ns,1:ns)=diag(approxsigmas); approxflower = U*approxS*V’; subplot(4,2,i+1),imshow(approxflower); title(sprintf(’When k = %s \n’, ranks(i))); end (Phan Thi Khanh Van) Eigenvalues, eigenvectors, diagonalization May 13, 2021 25 / 26 (Phan Thi Khanh Van) Eigenvalues, eigenvectors, diagonalization May 13, 2021 26 / 26 ... 150 00, z0 = 750 00   At this  10000, x0 y0  =  150 00  750 00 z0     x100 x0 After 50 years y100  = A50 y0  z100 z0 Use the method of diagonalization: A = PDP −1 , we can find A50... = eigenvectors of A? 1 −1 Ae2 = Ae3 = Ae1 = 2 1 = = 3e1 - eigenvector with λ = 3 = = λe2 - e2 is not an eigenvector −1 = = −e3 - eigenvector with λ = −1 −1 (Phan Thi Khanh Van) Eigenvalues, eigenvectors,... (Phan Thi Khanh Van) Eigenvalues, eigenvectors, diagonalization May 13, 2021 / 26 Example Find all eigenvalues and eigenvectors of: A = 2−λ = (2 − λ)(3 − λ) − 12 3−λ = λ2 − 5? ? − = ⇔ λ = −1 ∨ λ