Linear algebra học AI từ cơ bản đến nâng cao

Operating system Linear Algebra and Applications Quang Vinh Dinh PhD in Computer Science AI VIETNAM AI Course 2022  Project Description  Matrix Operations  Delving into Numpy  Least Squares Estima.

AI VIETNAM AI Course 2022 Linear Algebra and Applications Quang-Vinh Dinh PhD in Computer Science Outline       Project Description Matrix Operations Delving into Numpy Least Squares Estimation Cosine Similarity Singular Value Decomposition Pipeline Binary BoW: for word appearing for word non-appearing Text Representation BoW: frequency of each word occurring ❖ Bag-of-Words (BoW) vocabulary_size = Corpus doc1 = “deep learning book” doc2 = “machine learning algorithm” [‘deep’, ‘learning’, ‘book’] Tokenization [‘machine’, ‘learning’, ‘algorithm’] doc3 = “learning from scratch” [‘learning’, ‘ai’, ‘from’, ‘scratch’] doc4 = “ai vietnam” [‘ai’, ‘vietnam’] Vocabulary = deep learning book machine algorithm from scratch vietnam same size 👉 Given a string = “vietnam machine learning deep learning book” deep learning book machine algorithm from scratch vietnam BoW 1 0 0 Binary BoW 1 1 0 0 Outline       Project Description Matrix Operations Delving into Numpy Least Squares Estimation Cosine Similarity Singular Value Decomposition AI VIETNAM AI Course 2022 Vector & Matrix Matrix Vector 𝑛 is a natural number Matrix A has the shape of rectangle ℛ is a set of real numbers Has 𝑚 rows and n columns 𝑣Ԧ has a length of n and contain real numbers Use capital letter 𝑣Ԧ ∈ ℛ 𝑛 A ∈ ℛ 𝑚×𝑛 𝑣1 ℛ 𝑣Ԧ = 𝑣2 ∈ ℛ = ℛ 𝑣3 ℛ 𝑎11 𝑎12 ℛ ℛ A = 𝑎21 𝑎22 ∈ ℛ ℛ = ℛ 3×2 𝑎31 𝑎32 ℛ ℛ Vector Addition 𝑣1 𝑣Ԧ = … 𝑣3 𝑢1 𝑢= … 𝑢3 𝑣1 𝑢1 𝑣1 + 𝑢1 … 𝑣Ԧ + 𝑢 = … + … = 𝑣3 𝑢3 𝑣3 + 𝑢3 data x data y result 6 + = 10 12 AI VIETNAM AI Course 2022 Vector Operations Vector subtraction 𝑣1 𝑣Ԧ = … 𝑣𝑛 𝑢1 𝑢= … 𝑢𝑛 𝑣1 𝑢1 𝑣1 − 𝑢1 … 𝑣Ԧ + 𝑢 = … − … = 𝑣𝑛 𝑢𝑛 𝑣𝑛 − 𝑢𝑛 data x data y result - = 4 Vector Operations Multiply with a number 𝑢1 𝛼𝑢1 𝛼𝑢 = 𝛼 … = … 𝑢𝑛 𝛼𝑢𝑛 data result 2 * = Length of a vector 𝑢 = 𝑢12 + ⋯ + 𝑢𝑛2 AI VIETNAM AI Course 2022 Matrix Operations 𝑎11 … 𝑎1𝑛 A = … … 𝑎𝑚1 … 𝑎𝑚𝑛 𝑏11 … 𝑏1𝑛 B = … … 𝑏𝑚1 … 𝑏𝑚𝑛 Addition (𝑎11 + 𝑏11 ) … (𝑎1𝑛 + 𝑏1𝑛 ) … … A+B= (𝑎𝑚1 + 𝑏𝑚1 ) … (𝑎𝑚𝑛 + 𝑏𝑚𝑛 ) A Subtraction A−B= 𝑎11 − 𝑏11 … (𝑎1𝑛 − 𝑏1𝑛 ) … … 𝑎𝑚1 − 𝑏𝑚1 … (𝑎𝑚𝑛 − 𝑏𝑚𝑛 ) C B + A = 12 12 C B _ = Example VT 𝐴= 2 −2 mxn 𝑈= 1 18 − 18 18 − − Σ= 0 U Σ mxm mxn = A 𝑉𝑇 = 1 2 −1 2 nxn VT A = B mxn U Σ mxm mxn nxn B=𝐴𝑇 Step 1: Compute eigenvalues and eigenvectors from 𝐵𝑇 𝐵 𝑇 𝐵 𝐵𝑣𝑖 = 𝜆𝑖 𝑣𝑖 𝐵𝐵𝑇 𝑢𝑖 = 𝜆𝑖 𝑢𝑖 𝐵𝑇 𝐵𝑣𝑖 = 𝜆𝑖 𝑣𝑖 𝐵𝑣𝑖 = σ𝑖 𝑢𝑖 Σ Step 2: Compute eigenvectors from 𝐵𝐵𝑇 𝐵𝑣𝑖 = σ𝑖 𝑢𝑖 𝐵 = 𝐴𝑇 = 𝑈Σ𝑉 𝑇 U VT VT A U Σ mxm mxn = B mxn nxn B=𝐴𝑇 Step 1: Compute eigenvalues and eigenvectors from 𝐴𝐴𝑇 𝑇 𝐴𝐴 𝑣𝑖 = 𝜆𝑖 𝑣𝑖 𝐵𝐵𝑇 𝑢𝑖 = 𝜆𝑖 𝑢𝑖 𝐵𝑇 𝐵𝑣𝑖 = 𝜆𝑖 𝑣𝑖 𝐵𝑣𝑖 = σ𝑖 𝑢𝑖 Σ Step 2: Compute eigenvectors from 𝐴𝑇 𝐴 𝐴𝑇 𝑣𝑖 = σ𝑖 𝑢𝑖 𝐴 = (𝑈Σ𝑉 𝑇 )𝑇 = 𝑉Σ 𝑇 𝑈 𝑇 U VT Example VT A = B mxn U Σ mxm mxn 𝑇 B=𝐴 𝐴= 2 𝐵𝐵𝑇 𝑢𝑖 = 𝜆𝑖 𝑢𝑖 𝐵=𝐴 = 𝑇 𝐵𝑇 𝐵𝑣𝑖 = 𝜆𝑖 𝑣𝑖 𝐵𝑣𝑖 = σ𝑖 𝑢𝑖 −2 2 −2 𝐵 = 𝐴𝑇 = 𝑈Σ𝑉 𝑇 𝑈= 1 18 − Σ= 0 18 18 nxn − − 𝑉𝑇 = 1 2 −1 2 Example VT A U Σ mxm mxn = B mxn 𝑇 B=𝐴 𝐴= 2 𝐵𝐵𝑇 𝑢𝑖 = 𝜆𝑖 𝑢𝑖 𝐵=𝐴 = 𝑇 𝐵𝑇 𝐵𝑣𝑖 = 𝜆𝑖 𝑣𝑖 𝐵𝑣𝑖 = σ𝑖 𝑢𝑖 −2 2 −2 𝐴 = (𝑈Σ𝑉 𝑇 )𝑇 = 𝑉Σ 𝑇 𝑈 𝑇 𝑈= 1 18 − Σ= 0 18 18 nxn − − 𝑉𝑇 = 1 2 −1 2 AI VIETNAM All-In-One Course Singular Value Decomposition  Revisit VT A mxn Year 2022 = U Σ mxm mxn nxn 101 AI VIETNAM All-In-One Course Singular Value Decomposition  Revisit Σ’ A_r mxn Year 2022 = VT U’ mxk kxn nxn 102 AI VIETNAM All-In-One Course Singular Value Decomposition  Example 𝐴= 6 1 −0.53, −0.55, −0.44, −0.47 −0.18, 0.03, −0.63, 0.76 𝑈= −0.42, 0.83, −0.21, −0.3 −0.71, −0.08, 0.61, 0.34 15.39, 0, 0, 4.01, Σ= 0, 0, 2.45 0, 0, Year 2022 −0.38, −0.73, −0.57 𝑉 𝑇 = 0.58, 0.29, −0.76 0.72, −0.62, 0.32 103 AI VIETNAM All-In-One Course Singular Value Decomposition  Example: Reconstruction 𝐴= 6 1 15.39, 0, 0, 4.01, Σ= 0, 0, 2.45 0, 0, −0.53, −0.55, −0.44, −0.47 −0.18, 0.03, −0.63, 0.76 𝐴_𝑟 = −0.42, 0.83, −0.21, −0.3 −0.71, −0.08, 0.61, 0.34 1.0e+00 1.0e−16 = 4.0e+00 5.0e+00 Year 2022 6.0e+00 3.0e+00 6.0e+00 7.0e+00 6.0e+00 1.0e+00 1.0e+00 7.0e+00 −0.53, −0.55, −0.44, −0.47 −0.18, 0.03, −0.63, 0.76 𝑈= −0.42, 0.83, −0.21, −0.3 −0.71, −0.08, 0.61, 0.34 −0.38, −0.73, −0.57 𝑉 = 0.58, 0.29, −0.76 0.72, −0.62, 0.32 𝑇 15.39, 0, −0.38, −0.73, −0.57 0, 4.01, 0.58, 0.29, −0.76 0, 0, 2.45 0.72, −0.62, 0.32 0, 0, Error = 1.0e-14 104 17 AI VIETNAM All-In-One Course Singular Value Decomposition  Example: Reconstruction 𝐴= 6 −0.53, −0.55, −0.44, −0.47 −0.18, 0.03, −0.63, 0.76 𝑈= −0.42, 0.83, −0.21, −0.3 −0.71, −0.08, 0.61, 0.34 1 15.39, 0, 0, 4.01, Σ= 0, 0, 2.45 0, 0, Compression k=2 Year 2022 −0.53, −0.55 −0.18, 0.03 15.39, 0, 𝐴_𝑟 = −0.42, 0.83 0, 4.01, −0.71, −0.08 1.78 1.1 = 4.37 3.93 5.33 2.05 5.68 7.92 6.34 1.49 1.16 6.53 0 −0.38, −0.73, −0.57 𝑉 = 0.58, 0.29, −0.76 0.72, −0.62, 0.32 𝑇 −0.38, −0.73, −0.57 0.58, 0.29, −0.76 0.72, −0.62, 0.32 Error = 2.45 105 18 AI VIETNAM All-In-One Course Singular Value Decomposition  Example: Reconstruction 𝐴= 6 −0.53, −0.55, −0.44, −0.47 −0.18, 0.03, −0.63, 0.76 𝑈= −0.42, 0.83, −0.21, −0.3 −0.71, −0.08, 0.61, 0.34 1 15.39, 0, 0, 4.01, Σ= 0, 0, 2.45 0, 0, Compression k=1 Year 2022 −0.53 −0.18 𝐴_𝑟 = 15.39, −0.42 −0.71 3.07 1.04 = 2.42 4.12 5.98 2.02 4.71 8.02 0, 4.67 1.58 3.68 6.27 −0.38, −0.73, −0.57 𝑉 = 0.58, 0.29, −0.76 0.72, −0.62, 0.32 𝑇 −0.38, −0.73, −0.57 0.58, 0.29, −0.76 0.72, −0.62, 0.32 Error = 4.703 106 19 AI VIETNAM All-In-One Course Singular Value Decomposition  Application: Image compression Year 2022 Input Image Output (k=370) Output (k=300) Output (k=200) Output (k=100) Output (k=50) Output (k=30) Output (k=10) 107 20 ... arr=data) 40 AI VIETNAM AI Course 2022 Numpy  Some important functions arr1 arr2 -1 -1 result Year 2022 41 AI VIETNAM AI Course 2022  Some important functions Year 2022 Numpy AI VIETNAM AI Course...