Linear Algebra for Data Science Why Learn It Linear algebra is a very fundamental area in mathematics But we use it wide range of application with respect to machine learning We use it to solve some s.
Why Learn It: Linear algebra is a very fundamental area in mathematics But we use it wide range of application with respect to machine learning We use it to solve some specific types of problems in machine learning algorithm What is scaler quantity: Scalar quantity is defined as the physical quantity with magnitude and no direction Some physical quantities can be described just by their numerical value (with their respective units) without directions (they don’t have any direction) The addition of these physical quantities follows the simple rules of the algebra Here, only their magnitudes are added Example: A bike is moving with the speed of 5Km/h is a scaler quantity because it has only magnitude What is vector quantity: A vector quantity is defined as the physical quantity that has both direction as well as magnitude Example: A bike is moving with the speed of 5Km/h in east direction is a vector quantity because it has both magnitude and direction as well Represent vector graphically: To represent vector graphically We only care about the magnitude in the direction We not necessarily not care about Where we start and where we place it In [1]: import matplotlib.pyplot as plt plt.plot([0,5],[1,1]) #vector v1 plt.plot([1,6],[2,2]) #vector v2 plt.plot([8,-8],[0,0],'k ') plt.plot([0,0],[8,-8],'k ') plt.grid() plt.axis((-8,8,-8,8)) plt.show() Explanation: As you can see we have v1 and v2 vector we not care about where to start we just care about the magnitude in the direction Here v1 and v2 are same because both have same magnitude and direction is "Horizontal" direction Real coordinate space: Coordinate Geometry is even more useful in space than in the plane, since it is much harder to draw figures in 3D for accurate visualization Notation: Vectors will be written either as rows or columns when only vector operations are concerned If matrices enter the picture, then there is a difference between row vectors (row matrices) and column vectors (column matrices) In general, one writes O or sometimes for the zero vector In 2-space one often writes O or for (0, 0), I or i for (1, 0) and J or j for (0, 1) In 3-space one oftern writes O or for (0, 0, 0), I or i for (1, 0, 0) and J or j for (0, 1, 0), K or k for (0, 0, 1) In n-space, even when n = 1, or 3, one writes vectors e1, , en, where ek is the vector in n-space with all entries = except for in the k entry Addition of vectors: The process of adding two or more vectors is called vector addition Consider a vector A and B “When two vectors are represented by two sides of a triangle in magnitude and direction were taken in the same order then the third side of that triangle represents in magnitude and direction the resultant of the vectors.” In [2]: import numpy as np import matplotlib.pyplot as plt v1=np.array([1,2]) v2=np.array([3,4]) v3=v1+v2 print("v3=",v3) plt.plot([0,v1[0]],[0,v1[1]],'r',label='v1') plt.plot([0,v2[0]],[0,v2[1]],'b',label='v2') plt.plot([0,v2[0]],[0,v2[1]],'b',label='v2') plt.plot([0,v3[0]],[0,v3[1]],'g',label='v3') plt.plot([8,-8],[0,0],'k ') plt.plot([0,0],[8,-8],'k ') plt.grid() plt.axis((-8,8,-8,8)) plt.legend() plt.show() v3= [4 6] Multiply a vector by a scalar: When a vector is multiplied by a scalar quantity, then the magnitude of the vector changes in accordance with the magnitude of the scalar but the direction of the vector remains unchanged Suppose we have a vector a, then if this vector is multiplied by a scalar quantity k then we get a new vector with magnitude as |ka|and the direction remains same as the vector a if k is positive and if k is negative then the direction of k becomes just opposite of the direction of vector a Now let us understand visually the scalar multiplication of the vector Let us take the values of ‘k ‘to be = 2,3,-3,−12 and so on In [3]: v1=np.array([3,4]) a=0.5 v2=v1*a plt.plot([0,v1[0]],[0,v1[1]],'r-',label='v1') plt.plot([0,v2[0]],[0,v2[1]],'b ',label='v2') plt.plot([8,-8],[0,0],'k ') plt.plot([0,0],[8,-8],'k ') plt.grid() plt.grid() plt.axis((-8,8,-8,8)) plt.legend() plt.show() In [4]: v1=np.array([3,4]) a=-0.5 v2=v1*a plt.plot([0,v1[0]],[0,v1[1]],'r-',label='v1') plt.plot([0,v2[0]],[0,v2[1]],'b ',label='v2') plt.plot([8,-8],[0,0],'k ') plt.plot([0,0],[8,-8],'k ') plt.grid() plt.axis((-8,8,-8,8)) plt.legend() plt.show() Length of a vector: The length of the directed segment determines the numerical value of the vector is called the length of vector AB The magnitude of a vector is the length of the vector In [5]: v1=np.array([1,2,3,4,5]) length=np.sqrt(np.dot(v1,v1)) print("The length of vector v1 is: ",length) The length of vector v1 is: 7.416198487095663 Dot product: A vector has magnitude (how long it is) and direction: Here are two vectors: We will multiply these two vectors by using dot product: And the result is a number (called a "scalar" so we know it is not a vector) Angle between two vector: In [6]: v1 = np.array([8,4]) v2 = np.array([-4,8]) ang = np.rad2deg(np.arccos( np.dot(v1,v2) / (np.linalg.norm(v1)*np.linalg.norm(v2)) )) plt.plot([0,v1[0]] , [0,v1[1]] , 'r' , label = 'v1') plt.plot([0,v2[0]]+v1[0] , [0,v2[1]]+v1[1], 'b' , label = 'v2') plt.plot([16,-16] , [0,0] , 'k ') plt.plot([0,0] , [16,-16] , 'k ') plt.grid() plt.axis((-16, 16, -16, 16)) plt.legend() plt.title('Angle between Vectors - %s' %ang) plt.show() Distance of a point from origin Distance between two points: Projection of vectors: Equation of a line (2-D)/plane (3-D)/Hyperplane (n-D) Equation of circle(2-D),sphere(3-D),hypersphere(n-D): Equation of an elipse(2-D),ellipsoid(3-D) and hyperellipsoid(nD) Row vector: row matrix is a × m matrix, that is, a matrix consisting of a single row of m elements Column vector: column matrix is an m × matrix, that is, a matrix consisting of a single column of m elements, Matrix: A matrix is a rectangular arrangement of numbers into rows and columns In [7]: A = np.array([[1,2,3,4] , [5,6,7,8] , [10 , 11 , 12 ,13] , [14,15,16,17]]) A A Out[7]: array([[ 1, 2, 3, 4], [ 5, 6, 7, 8], [10, 11, 12, 13], [14, 15, 16, 17]]) Zero matrix: A matrix is known as a zero or null matrix if all of its elements are zero In [8]: A=np.zeros(9).reshape(3,3) A Out[8]: array([[0., 0., 0.], [0., 0., 0.], [0., 0., 0.]]) Ones matrix: A matrix is known as a ones matrix if all of its elements are one In [9]: A=np.ones(9).reshape(3,3) A Out[9]: array([[1., 1., 1.], [1., 1., 1.], [1., 1., 1.]]) Identity matrix In linear algebra, the identity matrix of size n is the n × n square matrix with ones on the main diagonal and zeros elsewhere It is denoted by Iₙ In [10]: A=np.eye(9) A Out[10]: array([[1., [0., [0., [0., [0., [0., [0., [0., [0., 0., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 0., 0., 0., 0., 0., Diagonal Matrix 0., 0., 0., 0., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 1., 0., 0.], 0.], 0.], 0.], 0.], 0.], 0.], 0.], 1.]]) a diagonal matrix is a matrix in which the entries outside the main diagonal are all zero; the term usually refers to square matrices In [11]: A=np.diag([1,2,3,4,5,6,7,8]) A Out[11]: array([[1, [0, [0, [0, [0, [0, [0, [0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 4, 0, 0, 0, 0, 0, 0, 0, 0, 5, 0, 0, 0, 0, 0, 0, 0, 0, 6, 0, 0, 0, 0, 0, 0, 0, 0, 7, 0, 0], 0], 0], 0], 0], 0], 0], 8]]) Matrix Addition Matrix addition is the operation of adding two matrices by adding the corresponding entries together In [12]: M = np.array([[1,2,3],[4,-3,6],[7,8,0]]) N = np.array([[1,1,1],[2,2,2],[3,3,3]]) print("\n First Matrix (M): \n", M) print("\n Second Matrix (N): \n", N) C = M+N print("\n Matrix Addition (M+N): \n", C) First Matrix (M): [[ 3] [ -3 6] [ 0]] Second Matrix (N): [[1 1] [2 2] [3 3]] Matrix Addition (M+N): [[ 4] [ -1 8] [10 11 3]] Matrix subtraction Subtraction of matrix is possible by subtracting the element of another matrix if they have the same order This new matrix is the difference between the above two matrices So, the difference between two matrices is obtained by subtracting the corresponding elements of the given matrices In [13]: M = np.array([[1,2,3],[4,-3,6],[7,8,0]]) N = np.array([[1,1,1],[2,2,2],[3,3,3]]) print("\n First Matrix (M): \n", M) print("\n Second Matrix (N): \n", N) C = M-N print("\n Matrix Subtraction (M-N): \n", C) First Matrix (M): [[ 3] [ -3 6] [ 0]] Second Matrix (N): Second Matrix (N): [[1 1] [2 2] [3 3]] Matrix Subtraction (M-N): [[ 2] [ -5 4] [ -3]] Matrices Scalar Multiplication The term scalar multiplication refers to the product of a real number and a matrix In scalar multiplication, each entry in the matrix is multiplied by the given scalar In [14]: M = np.array([[1,2,3],[4,-3,6],[7,8,0]]) C = 10 print("\n Matrix (M): \n", M) print("\nMatrices Scalar Multiplication: \n", C*M) Matrix (M): [[ 3] [ -3 6] [ 0]] Matrices Scalar Multiplication: [[ 10 20 30] [ 40 -30 60] [ 70 80 0]] Transpose of a matrix The transpose of a matrix is an operator which flips a matrix over its diagonal; that is, it switches the row and column indices of the matrix A by producing another matrix, often denoted by Aᵀ In [15]: M = np.array([[1,2,3],[4,-3,6],[7,8,0]]) print("\n Matrix (M): \n", M) print("\nTranspose of M: \n", np.transpose(M)) Matrix (M): [[ 3] [ -3 6] [ 0]] Transpose of M: [[ 7] [ -3 8] [ 0]] Determinant of a matrix In [16]: M = np.array([[1,2,3],[4,-3,6],[7,8,0]]) print("\n Matrix (M): \n", M) print("\nDeterminant of M: ", np.linalg.det(M)) Matrix (M): [[ 3] [ -3 6] [ 0]] Determinant of M: 195.0 Rank of a matrix In [17]: M = np.array([[1,2,3],[4,-3,6],[7,8,0]]) print("\n Matrix (M): \n", M) print("\nRank of M: ", np.linalg.matrix_rank(M)) Matrix (M): [[ 3] [ -3 6] [ 0]] Rank of M: Inverse of matrix A In [18]: M = np.array([[1,2,3],[4,-3,6],[7,8,0]]) print("\n Matrix (M): \n", M) print("\nInverse of M: \n", np.linalg.inv(M)) Matrix (M): [[ 3] [ -3 6] [ 0]] Inverse of M: [[-0.24615385 0.12307692 0.10769231] [ 0.21538462 -0.10769231 0.03076923] [ 0.27179487 0.03076923 -0.05641026]] Matrix Multiplication (pointwise multiplication) In [19]: M = np.array([[1,2,3],[4,-3,6],[7,8,0]]) N = np.array([[1,1,1],[2,2,2],[3,3,3]]) print("\n First Matrix (M): \n", M) print("\n Second Matrix (N): \n", N) print("\n Point-Wise Multiplication of M & N: \n", M*N) First Matrix (M): [[ 3] [ -3 6] [ 0]] Second Matrix (N): [[1 1] [2 2] [3 3]] Point-Wise Multiplication of M & N: [[ 3] [ -6 12] [21 24 0]] Sum of all elements in a matrix In [20]: N = np.array([[1,1,1],[2,2,2],[3,3,3]]) print("\n Matrix (N) ==> \n", N) print ("\nSum of all elements in a Matrix: ") print (np.sum(N)) Matrix (N) ==> [[1 1] [2 2] [3 3]] Sum of all elements in a Matrix: 18 Column-Wise Addition In [21]: N = np.array([[1,1,1],[2,2,2],[3,3,3]]) print("\n Matrix (N): \n", N) print ("\nColumn-Wise summation: ") print (np.sum(N,axis=0)) Matrix (N): [[1 1] [2 2] [3 3]] Column-Wise summation: [6 6] Row-Wise Addition In [22]: N = np.array([[1,1,1],[2,2,2],[3,3,3]]) print("\n Matrix (N) ==> \n", N) print ("\nRow-Wise summation ==>") print (np.sum(N,axis=1)) Matrix (N) ==> [[1 1] [2 2] [3 3]] Row-Wise summation ==> [3 9] ... sometimes for the zero vector In 2-space one often writes O or for (0, 0), I or i for (1, 0) and J or j for (0, 1) In 3-space one oftern writes O or for (0, 0, 0), I or i for (1, 0, 0) and J or j for. .. or j for (0, 1, 0), K or k for (0, 0, 1) In n-space, even when n = 1, or 3, one writes vectors e1, , en, where ek is the vector in n-space with all entries = except for in the k entry Addition... A=np.ones(9).reshape(3,3) A Out[9]: array([[1., 1., 1.], [1., 1., 1.], [1., 1., 1.]]) Identity matrix In linear algebra, the identity matrix of size n is the n × n square matrix with ones on the main diagonal