Chapter 3 inner product space (1)

Inner product spaces Phan Thi Khanh Van E-mail: khanhvanphan@hcmut.edu.vn May 13, 2021 (Phan Thi Khanh Van) Inner product spaces May 13, 2021 / 20 Table of contents Inner product The orthogonal/perpendicular complement Vector projection Gram- Schmidt orthogonalization (Phan Thi Khanh Van) Inner product spaces May 13, 2021 / 20 Inner product Let V be a vector space The inner product of vectors u, v ∈ V is a real number satisfying following axioms: ∀u ∈ V : (u, u) ≥ 0, ” = ” occurs ⇔ u = ∀α ∈ K , u, v ∈ V : (αu, v ) = α(u, v ) ∀u, v ∈ V : (u, v ) = (v , u) ∀u, v , w ∈ V : (u + v , w ) = (u, w ) + (v , w ) A vector space V + inner product = inner product space Rn + inner product = Euclidean space In Rn , the standard inner product (dot product) is: (x, y ) = x1 y1 + x2 y2 + x3 y3 + + xn yn Example In R2 : (x, y ) = x1 y1 + 2x2 y1 + 2x1 y2 + 5x2 y2 is an inner product In P2 [R] : (p, q) = In R3 p(x)q(x)dx is an inner product : (x, y ) = x1 y1 − x2 y1 − x1 y2 + 3x2 y2 + 5x3 y3 is an inner product (Phan Thi Khanh Van) Inner product spaces May 13, 2021 / 20 Norm of a vector u = (u, u) Distance between vectors d(u, v ) = u − v Angle between vectors cos(u, v ) = (u,v ) u v Orthogonal vectors u ⊥ v ⇔ u, v = π ⇔ cos(u, v ) = A vector having norm is called a unit vector Cauchy-Schwatz inequality: |(u, v )| ≤ u v Triangle inequality: u + v ≤ u + v (Phan Thi Khanh Van) Inner product spaces May 13, 2021 / 20 Example In R given an inner product: (x, y ) = x1 y1 + x2 y1 + x1 y2 + 3x2 y2 + 5x3 y3 , x = (1, 2, −1), y = (−1, 0, 1) Determine: (x, 3y + x) x The distance, angle between vectors 2x + y and x − y a unit vector parallel to x 3y + x = 3(−1, 0, 1) + (1, 2, −1) = (−2, 2, 2) ⇒ (x, 3y + x) = −2 + 1.2 − 2.2 + 3.2.2 − 5.1.2 = −2 Remark The inner product of x = (x1 , x2 , x3 ) and y = (y1 , y2 , y3 ) can be represented as    1 y1 (x, y ) = x1 x2 x3 1 0 y2  = x.A.y T 0 x3 So, we can compute: (x, 3y + x) = x.A.(3y + x)T = −2 (Phan Thi Khanh Van) Inner product spaces May 13, 2021 / 20 √ √ x = (x, x) = x.A.x T = 22 d(2x √ + y , x − y ) = (2x + y ) − (x − y ) = x + 2y = (−1, 2, 1) = 14 The angle α between vectors: 2x + y = (1, 4, −1) and x − y = (2, 2, −2): (2x+y ,x−y ) √ 46 √23 cos α = 2x+y x−y = 62.44 = 682 α = arccos √23 682 Let u be a unit vector parallel to x We have that u = αx and u = |α| x = ⇒ α = ± x1 ⇒u=± x x (Phan Thi Khanh Van) √ = ± (1,2,−1) = ±( √122 , √222 , − √122 ) 22 Inner product spaces May 13, 2021 / 20 Example In an inner product space V with the inner product (x, y ) x = 2, y = 3, (x, y ) = π3 Find d(x − 3y , 2x + y ) d(x − 3y , 2x + y ) = x − 3y − 2x − y = − x − 4y = (x + 4y , x + 4y ) = (x, x) + 4(x, y ) + 4(y , x) + 16(y , y ) = = x + x y cos (x, y ) + 16 y √ 22 + 8.2.3 12 + 16.32 = 172 (Phan Thi Khanh Van) Inner product spaces May 13, 2021 / 20 A vector orthogonal to a vector set M u ⊥ M ⇔ (u, v ) = 0, ∀v ∈ M (orthogonal to any vector in M) A vector orthogonal to a vector space A vector u is called orthogonal to a vector space U if it is orthogonal to any vector v in U: (u, v ) = 0, ∀v ∈ U If U = span{M}, then u ⊥ U ⇔ u ⊥ M, (u ⊥ U ⇔ u ⊥ to the spanning set of U) orthogonal vector sets M ⊥ N ⇔ (u, v ) = 0, ∀u ∈ M, v ∈ N orthogonal spaces Given spaces U = span{M}, V = span{N}, U ⊥ V ⇔ (u, v ) = 0, ∀u ∈ U, v ∈ V ⇔ M ⊥ N, (2 spaces are orthogonal ⇔ spanning sets are orthogonal) (Phan Thi Khanh Van) Inner product spaces May 13, 2021 / 20 Orthogonal set M is called an orthogonal set if: ∀u, v ∈ M : (u, v ) = (mutually orthogonal/perpendicular) Property: An orthogonal set without vector is LI Orthonormal set M is called an orthonormal set if it is orthogonal and ei = 1, ∀ei ∈ M (Phan Thi Khanh Van) Inner product spaces May 13, 2021 / 20 Example In R given an inner product: (x, y ) = x1 y1 − x2 y1 − x1 y2 + 4x2 y2 + 3x3 y3 a Let F = span{f1 = (1, 1, −1), f2 = (6, 0, 1)} and u = (1, 2, m) Find m such that u ⊥ F b Let W = {x ∈ R |x1 + x2 − x3 = 0, 3x1 + x3 = 0} Find m such that u ⊥ W a b u ⊥ f1 (u, f1 ) = ⇔ u ⊥ f2 (u, f2 ) =    −1 1 −1  −1 0   = ⇔ m = ⇔ 0 m Find the basis of W : x ∈ W ⇔ x = (−m, 4m, 3m) A basis of W : {(−1, 4, 3)}  −1 x ⊥ U ⇔ u ⊥ (−1, 4, 3) ⇔ −1 −1 0 ⇔ m = − 29 u⊥F ⇔ (Phan Thi Khanh Van) Inner product spaces   0   = m May 13, 2021 10 / 20 The orthogonal complement In an inner product space V , let U be a subspace The orthogonal/perpendicular complement of U is the set of all vectors orthogonal to U: U ⊥ := {x ∈ V |x ⊥ U} Theorem For any vector u ∈ V , u can be uniquely represented as: u = x + y , where x ∈ U and y ∈ U ⊥ dimU + dimU ⊥ = dimV (Phan Thi Khanh Van) Inner product spaces May 13, 2021 11 / 20 Example In R3 with the inner product (x, y ) = x1 y1 + 3x2 y2 + 4x3 y3 , let U be U = span{(1, 2, −1), (0, 1, 2), (2, 3, −4)} Find the dimension and one basis of U ⊥   x ⊥ u1 ⊥ x = (x1 , x2 , x3 ) ∈ U ⇔ x ⊥ u2   x ⊥u    3    x1 −4 −1 0  ⇔ 0  0 0 x2  = ⇔  x3 −4 0 −16   −4 −4  ⇔ ⇔ 3 0 −3 −8 The general solution: x = (60m, −8m, 3m) ⇒ dim(U ⊥ ) = One basis for U ⊥ : {(60, −8, 3)} (Phan Thi Khanh Van) Inner product spaces May 13, 2021 12 / 20 Example In R3 with the inner product (x, y ) = x1 y1 + x1 y2 + x2 y1 + 4x2 y2 + x3 y3 , find a basis, the dimension of U ⊥ where: U = {x ∈ R3 |x1 + x2 − x3 = 0, 2x1 + x2 + x3 = 0} First, we will find a basis of U: 1 −1 1 −1 ⇔ 1 0 -1 General solution: x = (−2m, 3m, m)T , a basis of U : {(−2, 3, 1)} Next, we will find the general vector x ∈ U ⊥ ⇔ x ⊥ U ⇔ (x, (−2, 3, 1)) =0   1 x1 ⇔ −2 1 0 x2  = ⇔ 10 0 x3 General solution x = m(−10, 1, 0) + n(−1, 0, 1) One basis for U ⊥ : {(−10, 1, 0), (−1, 0, 1)}, dim(U ⊥ ) = (Phan Thi Khanh Van) Inner product spaces May 13, 2021 13 / 20 Exercises In R4 with the dot product, find the dimension, a basis of U ⊥ where: U = {x ∈ R |x1 + x2 − x4 = 0, 2x1 + x2 + x3 = 0} In R4 with the dot product, find the dimension, a basis of U ⊥ where: U =< (1, −2, 1, 0), (1, 2, 3, 4) > In R3 with the inner product: (x, y ) = x1 y1 + x1 y2 + x2 y1 + 4x2 y2 + x3 y3 , find the dimension, a basis of U ⊥ where U =< (1, − 3), (2, 1, 0) > (Phan Thi Khanh Van) Inner product spaces May 13, 2021 14 / 20 Vector projection Let U be a subspace of an inner product space V Any vector z in U can be uniquely represented as z = x + y , x ∈ U, y ∈ U ⊥ x is called the vector projection/ vector resolution of z onto U Denote: prU (z) y is called the vector rejection of z The distance from z to U: d(z, U) = y = z − prU (z) (Phan Thi Khanh Van) Inner product spaces May 13, 2021 15 / 20 Example In R3 with the inner product (x, y ) = x1 y1 + 2x2 y2 + 3x3 y3 , given U = span{(1, 2, −1), (2, 1, 0)}, z = (2, −3, 4) Find prU (z), d(z, U) Let z = x + y , x ∈ U, y ∈ U ⊥ , e1 = (1, 2, −1), e2 = (2, 1, 0) x ∈ U ⇒ x = αe1 + βe2 y ∈ U⊥ ⇒ Then, we have (y , e1 ) = (y , e2 ) = (z, e1 ) = α(e1 , e1 ) + β(e2 , e1 ) (z, e2 ) = α(e1 , e2 ) + β(e2 , e2 ) α = − 10 −22 = 12α + 6β ⇔ β=3 −2 = 6α + 6β 11 10 Therefore, prU (z) == − 10 (1, 2, −1) + 3(2, 0, 1) = ( , − , ) or 20 And d(z, U) = z − prU (z) = ( −2 , , 3) = (Phan Thi Khanh Van) Inner product spaces √ 816 May 13, 2021 16 / 20 Exercises In R with the dot product, given U = {x ∈ R |x1 + x2 − x3 = 0}, z = (1, 2, 1) Find prU (z) In R given an inner product (x, y ) = x1 y1 + x2 y1 + x1 y2 + 3x2 y2 − 2x2 y3 − 2x3 y2 + 3x3 y3 , a subspace U =< (1, 1, 2), (2, −1, 1) >, a vector z = (−8, 7, 3) Find prU (z), d(z, U) (Phan Thi Khanh Van) Inner product spaces May 13, 2021 17 / 20 Orthogonal basis In an inner product space V , a basis E = {e1 , e2 , , en } is called an orthogonal basis for V , if it is mutually orthogonal (ei ⊥ ej ∀i, j) Orthonormal basis In an inner product space V , a basis E = {e1 , e2 , , en } is called an orthonormal basis for V , if it is orthogonal (ei ⊥ ej ), and ei = 1, ∀i, j Gram-Schmidt orthogonalization In an inner product space V , E = {e1 , e2 , , en } is a basis of V We will find an orthogonal basis F from E using Gram-Schmidt algorithm: f1 = e1 f2 = e2 − prf1 (e2 ) = e2 − (e2 ,f1 ) (f1 ,f1 ) f1 ,f1 ) f3 = e3 − prf1 (e3 ) − prf2 (e3 ) = e3 − (e (f1 ,f1 ) f1 − fn = en − prf1 (en ) − prf2 (en ) − prfn−1 (en ) Normalizing: gi = (Phan Thi Khanh Van) fi fi (e3 ,f2 ) (f2 ,f2 ) f2 , ∀i = n, we obtain an orthonormal basis Inner product spaces May 13, 2021 18 / 20 Example In R3 with the dot product, let E = {e1 = (1, 1, 1), e2 = (1, 2, −1)} be a basis Find an orthonormal basis F from E Orthogonalizing E using Gram-Schmidt algorithm: f1 = e1 = (1, 1, 1) ,f1 ) f2 = e2 − (e (f1 ,f1 ) f1 = (1, 2, −1) − (1, 1, 1) = ( , , − ) Choose f2 = (1, 4, −5) An orthogonal basis F = {(1, 1, 1), (1, 4, −5)} An orthonormal basis √ √ √ √ √ √ F = {(1/ 3, 1/ 3, 1/ 3), (1/ 42, 4/ 42, −5/ 42)} (Phan Thi Khanh Van) Inner product spaces May 13, 2021 19 / 20 Thank you for your attention! (Phan Thi Khanh Van) Inner product spaces May 13, 2021 20 / 20 ... w ) A vector space V + inner product = inner product space Rn + inner product = Euclidean space In Rn , the standard inner product (dot product) is: (x, y ) = x1 y1 + x2 y2 + x3 y3 + + xn yn ... 22 Inner product spaces May 13, 2021 / 20 Example In an inner product space V with the inner product (x, y ) x = 2, y = 3, (x, y ) = ? ?3 Find d(x − 3y , 2x + y ) d(x − 3y , 2x + y ) = x − 3y... is an inner product In P2 [R] : (p, q) = In R3 p(x)q(x)dx is an inner product : (x, y ) = x1 y1 − x2 y1 − x1 y2 + 3x2 y2 + 5x3 y3 is an inner product (Phan Thi Khanh Van) Inner product spaces