Đang tải... (xem toàn văn)
Operating system Linear Algebra and Applications Quang Vinh Dinh PhD in Computer Science AI VIETNAM AI Course 2022 Project Description Matrix Operations Delving into Numpy Least Squares Estima.
AI VIETNAM AI Course 2022 Linear Algebra and Applications Quang-Vinh Dinh PhD in Computer Science Outline Project Description Matrix Operations Delving into Numpy Least Squares Estimation Cosine Similarity Singular Value Decomposition Pipeline Binary BoW: for word appearing for word non-appearing Text Representation BoW: frequency of each word occurring ❖ Bag-of-Words (BoW) vocabulary_size = Corpus doc1 = “deep learning book” doc2 = “machine learning algorithm” [‘deep’, ‘learning’, ‘book’] Tokenization [‘machine’, ‘learning’, ‘algorithm’] doc3 = “learning from scratch” [‘learning’, ‘ai’, ‘from’, ‘scratch’] doc4 = “ai vietnam” [‘ai’, ‘vietnam’] Vocabulary = deep learning book machine algorithm from scratch vietnam same size 👉 Given a string = “vietnam machine learning deep learning book” deep learning book machine algorithm from scratch vietnam BoW 1 0 0 Binary BoW 1 1 0 0 Outline Project Description Matrix Operations Delving into Numpy Least Squares Estimation Cosine Similarity Singular Value Decomposition AI VIETNAM AI Course 2022 Vector & Matrix Matrix Vector 𝑛 is a natural number Matrix A has the shape of rectangle ℛ is a set of real numbers Has 𝑚 rows and n columns 𝑣Ԧ has a length of n and contain real numbers Use capital letter 𝑣Ԧ ∈ ℛ 𝑛 A ∈ ℛ 𝑚×𝑛 𝑣1 ℛ 𝑣Ԧ = 𝑣2 ∈ ℛ = ℛ 𝑣3 ℛ 𝑎11 𝑎12 ℛ ℛ A = 𝑎21 𝑎22 ∈ ℛ ℛ = ℛ 3×2 𝑎31 𝑎32 ℛ ℛ Vector Addition 𝑣1 𝑣Ԧ = … 𝑣3 𝑢1 𝑢= … 𝑢3 𝑣1 𝑢1 𝑣1 + 𝑢1 … 𝑣Ԧ + 𝑢 = … + … = 𝑣3 𝑢3 𝑣3 + 𝑢3 data x data y result 6 + = 10 12 AI VIETNAM AI Course 2022 Vector Operations Vector subtraction 𝑣1 𝑣Ԧ = … 𝑣𝑛 𝑢1 𝑢= … 𝑢𝑛 𝑣1 𝑢1 𝑣1 − 𝑢1 … 𝑣Ԧ + 𝑢 = … − … = 𝑣𝑛 𝑢𝑛 𝑣𝑛 − 𝑢𝑛 data x data y result - = 4 Vector Operations Multiply with a number 𝑢1 𝛼𝑢1 𝛼𝑢 = 𝛼 … = … 𝑢𝑛 𝛼𝑢𝑛 data result 2 * = Length of a vector 𝑢 = 𝑢12 + ⋯ + 𝑢𝑛2 AI VIETNAM AI Course 2022 Matrix Operations 𝑎11 … 𝑎1𝑛 A = … … 𝑎𝑚1 … 𝑎𝑚𝑛 𝑏11 … 𝑏1𝑛 B = … … 𝑏𝑚1 … 𝑏𝑚𝑛 Addition (𝑎11 + 𝑏11 ) … (𝑎1𝑛 + 𝑏1𝑛 ) … … A+B= (𝑎𝑚1 + 𝑏𝑚1 ) … (𝑎𝑚𝑛 + 𝑏𝑚𝑛 ) A Subtraction A−B= 𝑎11 − 𝑏11 … (𝑎1𝑛 − 𝑏1𝑛 ) … … 𝑎𝑚1 − 𝑏𝑚1 … (𝑎𝑚𝑛 − 𝑏𝑚𝑛 ) C B + A = 12 12 C B _ = Example VT 𝐴= 2 −2 mxn 𝑈= 1 18 − 18 18 − − Σ= 0 U Σ mxm mxn = A 𝑉𝑇 = 1 2 −1 2 nxn VT A = B mxn U Σ mxm mxn nxn B=𝐴𝑇 Step 1: Compute eigenvalues and eigenvectors from 𝐵𝑇 𝐵 𝑇 𝐵 𝐵𝑣𝑖 = 𝜆𝑖 𝑣𝑖 𝐵𝐵𝑇 𝑢𝑖 = 𝜆𝑖 𝑢𝑖 𝐵𝑇 𝐵𝑣𝑖 = 𝜆𝑖 𝑣𝑖 𝐵𝑣𝑖 = σ𝑖 𝑢𝑖 Σ Step 2: Compute eigenvectors from 𝐵𝐵𝑇 𝐵𝑣𝑖 = σ𝑖 𝑢𝑖 𝐵 = 𝐴𝑇 = 𝑈Σ𝑉 𝑇 U VT VT A U Σ mxm mxn = B mxn nxn B=𝐴𝑇 Step 1: Compute eigenvalues and eigenvectors from 𝐴𝐴𝑇 𝑇 𝐴𝐴 𝑣𝑖 = 𝜆𝑖 𝑣𝑖 𝐵𝐵𝑇 𝑢𝑖 = 𝜆𝑖 𝑢𝑖 𝐵𝑇 𝐵𝑣𝑖 = 𝜆𝑖 𝑣𝑖 𝐵𝑣𝑖 = σ𝑖 𝑢𝑖 Σ Step 2: Compute eigenvectors from 𝐴𝑇 𝐴 𝐴𝑇 𝑣𝑖 = σ𝑖 𝑢𝑖 𝐴 = (𝑈Σ𝑉 𝑇 )𝑇 = 𝑉Σ 𝑇 𝑈 𝑇 U VT Example VT A = B mxn U Σ mxm mxn 𝑇 B=𝐴 𝐴= 2 𝐵𝐵𝑇 𝑢𝑖 = 𝜆𝑖 𝑢𝑖 𝐵=𝐴 = 𝑇 𝐵𝑇 𝐵𝑣𝑖 = 𝜆𝑖 𝑣𝑖 𝐵𝑣𝑖 = σ𝑖 𝑢𝑖 −2 2 −2 𝐵 = 𝐴𝑇 = 𝑈Σ𝑉 𝑇 𝑈= 1 18 − Σ= 0 18 18 nxn − − 𝑉𝑇 = 1 2 −1 2 Example VT A U Σ mxm mxn = B mxn 𝑇 B=𝐴 𝐴= 2 𝐵𝐵𝑇 𝑢𝑖 = 𝜆𝑖 𝑢𝑖 𝐵=𝐴 = 𝑇 𝐵𝑇 𝐵𝑣𝑖 = 𝜆𝑖 𝑣𝑖 𝐵𝑣𝑖 = σ𝑖 𝑢𝑖 −2 2 −2 𝐴 = (𝑈Σ𝑉 𝑇 )𝑇 = 𝑉Σ 𝑇 𝑈 𝑇 𝑈= 1 18 − Σ= 0 18 18 nxn − − 𝑉𝑇 = 1 2 −1 2 AI VIETNAM All-In-One Course Singular Value Decomposition Revisit VT A mxn Year 2022 = U Σ mxm mxn nxn 101 AI VIETNAM All-In-One Course Singular Value Decomposition Revisit Σ’ A_r mxn Year 2022 = VT U’ mxk kxn nxn 102 AI VIETNAM All-In-One Course Singular Value Decomposition Example 𝐴= 6 1 −0.53, −0.55, −0.44, −0.47 −0.18, 0.03, −0.63, 0.76 𝑈= −0.42, 0.83, −0.21, −0.3 −0.71, −0.08, 0.61, 0.34 15.39, 0, 0, 4.01, Σ= 0, 0, 2.45 0, 0, Year 2022 −0.38, −0.73, −0.57 𝑉 𝑇 = 0.58, 0.29, −0.76 0.72, −0.62, 0.32 103 AI VIETNAM All-In-One Course Singular Value Decomposition Example: Reconstruction 𝐴= 6 1 15.39, 0, 0, 4.01, Σ= 0, 0, 2.45 0, 0, −0.53, −0.55, −0.44, −0.47 −0.18, 0.03, −0.63, 0.76 𝐴_𝑟 = −0.42, 0.83, −0.21, −0.3 −0.71, −0.08, 0.61, 0.34 1.0e+00 1.0e−16 = 4.0e+00 5.0e+00 Year 2022 6.0e+00 3.0e+00 6.0e+00 7.0e+00 6.0e+00 1.0e+00 1.0e+00 7.0e+00 −0.53, −0.55, −0.44, −0.47 −0.18, 0.03, −0.63, 0.76 𝑈= −0.42, 0.83, −0.21, −0.3 −0.71, −0.08, 0.61, 0.34 −0.38, −0.73, −0.57 𝑉 = 0.58, 0.29, −0.76 0.72, −0.62, 0.32 𝑇 15.39, 0, −0.38, −0.73, −0.57 0, 4.01, 0.58, 0.29, −0.76 0, 0, 2.45 0.72, −0.62, 0.32 0, 0, Error = 1.0e-14 104 17 AI VIETNAM All-In-One Course Singular Value Decomposition Example: Reconstruction 𝐴= 6 −0.53, −0.55, −0.44, −0.47 −0.18, 0.03, −0.63, 0.76 𝑈= −0.42, 0.83, −0.21, −0.3 −0.71, −0.08, 0.61, 0.34 1 15.39, 0, 0, 4.01, Σ= 0, 0, 2.45 0, 0, Compression k=2 Year 2022 −0.53, −0.55 −0.18, 0.03 15.39, 0, 𝐴_𝑟 = −0.42, 0.83 0, 4.01, −0.71, −0.08 1.78 1.1 = 4.37 3.93 5.33 2.05 5.68 7.92 6.34 1.49 1.16 6.53 0 −0.38, −0.73, −0.57 𝑉 = 0.58, 0.29, −0.76 0.72, −0.62, 0.32 𝑇 −0.38, −0.73, −0.57 0.58, 0.29, −0.76 0.72, −0.62, 0.32 Error = 2.45 105 18 AI VIETNAM All-In-One Course Singular Value Decomposition Example: Reconstruction 𝐴= 6 −0.53, −0.55, −0.44, −0.47 −0.18, 0.03, −0.63, 0.76 𝑈= −0.42, 0.83, −0.21, −0.3 −0.71, −0.08, 0.61, 0.34 1 15.39, 0, 0, 4.01, Σ= 0, 0, 2.45 0, 0, Compression k=1 Year 2022 −0.53 −0.18 𝐴_𝑟 = 15.39, −0.42 −0.71 3.07 1.04 = 2.42 4.12 5.98 2.02 4.71 8.02 0, 4.67 1.58 3.68 6.27 −0.38, −0.73, −0.57 𝑉 = 0.58, 0.29, −0.76 0.72, −0.62, 0.32 𝑇 −0.38, −0.73, −0.57 0.58, 0.29, −0.76 0.72, −0.62, 0.32 Error = 4.703 106 19 AI VIETNAM All-In-One Course Singular Value Decomposition Application: Image compression Year 2022 Input Image Output (k=370) Output (k=300) Output (k=200) Output (k=100) Output (k=50) Output (k=30) Output (k=10) 107 20 ... arr=data) 40 AI VIETNAM AI Course 2022 Numpy Some important functions arr1 arr2 -1 -1 result Year 2022 41 AI VIETNAM AI Course 2022 Some important functions Year 2022 Numpy AI VIETNAM AI Course...
