34 LINEAR ALGEBRA FOR 3D GRAPHICS CHAPTER 2 coordinates (see Section 2.4), M we M ow takes a vertex from object coordinates to eye coordinates. Notice the order of application: v = M ow v is applied first, followed by v = M we v = M we M ow v. A compound transformation can in principle be evaluated in two orders. You can do it fromrighttoleft(M we (M ow v)), that is, transform the vertex through each of the coordi- nate systems one at a time, or from left to right ((M we M ow )v), that is, collapse the trans- formations into one before applying them to the vertex. Let us now analyze which order is more efficient. If you take the first approach, repeatedly transforming each vertex, you will need 16mn multiplications, where m is the number of transformations and n the number of ver- tices. If you take the second approach, you need 64(m −1) multiplications to collapse the matrices, and 16n multiplications to apply the transformation to each vertex. Therefore, if 64(m − 1) + 16n<16mn, it makes sense to compound the matrices before applying the result to the vertices. For example, if you have 20 points and 3 transformations, the approach of transforming each point 3 times would require 16 · 3 · 20 = 960 multipli- cations, while first combining the transformations and then applying them would only require 64 · (3 − 1) + 16 · 20 = 448 multiplications. You can see that it almost always makes more sense to collapse the transformations before applying them to the vertices. Repeating this analysis for the number of additions rather than multiplications only changes the constant factors in the above inequality (from 64 to 48, and from 16 to 12), thus not changing the outcome in a significant way. 2.2.4 TRANSFORMING NORMAL VECTORS Even though vectors in general are transformed the same way as points, normal vectors must be transformed differently. Normal vectors are not just any direction vectors, they are defined by the surface so that a normal vector is perpendicular to the surface. If M is used to transform the surface, then normals must be transformed by the transpose of the inverse of M, that is, (M −1 ) T ,orM −T for short. Figure 2.4 illustrates why simply using M does not work: on the left, we have a line and a vector that is perpendicular to it. On the right, both the line and the vector have been scaled by two in y. Obviously the vector is not perpendicular to the line any longer. The correctly transformed vector, shown by the dotted line, remains normal to the surface. The key observation is that normal vectors are not really independent directions; instead, they are de fined to be perpendicular to the underlying surface. That is, assume v 1 and v 2 are points on a surface and that they are close enough that t = v 1 − v 2 is a tangent vector on the surface, while n is the normal vector. The normal is perpendicular to the tangent, so the dot product n T t must remain zero. Now, if we transform the surface points v 1 and v 2 by M, the tangent vector becomes Mt, and the dot product n T (Mt) is not necessarily zero any longer. Then how should SECTION 2.3 AFFINE TRANSFORMATIONS 35 Figure 2.4: On the left we have a line and a normal vector that is perpendicular to it. On the right both have been scaled by 2 in y, and the normal is obviously not perpendicular to the line any more. The correctly transformed normal is shown dashed. we transform the normal to keep it perpendicular to the tangent? All we need to do is to eliminate M from the equation by inserting M −1 : n T M −1 Mt = n T t. (2.16) Recalling from Equation (2.12) that (AB) T = B T A T , we can rewrite (n T M −1 ) T as (M −1 ) T n, and see that the normals need to be tr ansformed by the inverse transpose of M. There is a very common special case where Mn is parallel to M −T n, namely when M consists only of rotations, translations, and uniform scalings (these transformations are introduced in Section 2.3.1). In such a case transforming the normals does not require inverting and transposing M. However, if nonuniform scaling or shearing is involved, the transposed inverse formula must be used. In this case, the normal vectors will gen- erally not be unit length after the transformation, and will have to be renormalized; see Equation (2.4). 2.3 AFFINE TRANSFORMATIONS Affine transformations are the most widely used modeling operations, and they are sim- pler than the full 4 × 4 matrices that also encode projections: their bottom row is always [ 0001 ] . This section covers the different types of affine transformations: transla- tion, rotation, scaling, and shearing. We also give examples how to use transformations for hierarchical modeling. 2.3.1 TYPES OF AFFINE TRANSFORMATIONS There are four basic component transformations that are affine, and all their combina- tions are affine as well. They are translation, rotation, scaling, and shearing, and are illus- trated in Figure 2.5. They are commonly used to position and orient objects in a scene, or change their size or shape. 36 LINEAR ALGEBRA FOR 3D GRAPHICS CHAPTER 2 Translation Rotation Scaling Shearing Figure 2.5: The four types of affine transformations: translation, rotation, scaling, shearing. The black shape is the original, the gray after it has been transformed. Note that all transformations take place with respect to the origin. Translation by aconstant offset t x t y t z T movesavertexa to a x + t x a y + t y a z + t z T , and is expressed by the matrix T(t x , t y , t z ) = ⎡ ⎢ ⎢ ⎢ ⎣ 100t x 010t y 001t z 0001 ⎤ ⎥ ⎥ ⎥ ⎦ . (2.17) As seen in Figure 2.5 the shape is not changed, it is simply relocated. A translation does not affect a direction vector: directions have w = 0 and the only components that differ from the identity matrix are in the last column which corresponds to the w. The inverse of translation is simply another translation to the opposite direction: T −1 (t x , t y , t z ) = T(−t x , − t y , − t z ). (2.18) A rotation matrix rotates a point around an axis that passes through the origin. Here is the rotation matrix around the x axis by angle θ: R x (θ) = ⎡ ⎢ ⎢ ⎢ ⎣ 10 0 0 0 cos(θ) −sin(θ)0 0 sin(θ) cos(θ)0 00 0 1 ⎤ ⎥ ⎥ ⎥ ⎦ . (2.19) Similar matrices can be created for rotations around the y and z axes. The rotation axis always goes through the origin, and the positive direction of rotation is chosen using the right-hand rule, as shown in Figure 2.2. Thus, if you look along the axis toward the origin, positive rotation of the other axes is counterclockwise. Rotations around the main axes are easiest to write down. By combining three main axis rotations, an arbitrary rotation around an arbitrary axis can be created. This technique is called the Euler angles after Leonhard Euler who developed the method in the eighteenth SECTION 2.3 AFFINE TRANSFORMATIONS 37 century. People have used several axis orders, for example by first rotating around x axis, followed by z axis, then again x axis. Euler rotations can get stuck in a singularity known as gimbal lock, where the value of the third rotation becomes ir relevant. An alternative to Euler angles is to use quaternions [Sho85] that can be obtained from an arbitrary rotation axis and rotation angle. Quaternions do not suffer from gimbal lock, and since quaternions are generally quicker to calculate and simpler to interpolate, they are usually preferred over Euler angles for rotations in computer graphics. A quaternion is a 4D imaginary number with three imaginary components and one real component q = xi + yj + zk + w, (2.20) but we usually represent a quaternion as a four-vector xyzw . Every 3D rotation can be expressed by a unit rotation axis ˆ a and a rotation angle θ, and using them we can construct a matching unit quaternion as ˆ q( ˆ a, θ) = sin(θ/2)a x sin(θ/2)a y sin(θ/2)a z cos(θ/2) . (2.21) From the normalized unit quaternion ˆ q we obtain the following rotation matrix: R( ˆ q) = ⎡ ⎢ ⎢ ⎢ ⎣ w 2 + x 2 − y 2 − z 2 2xy − 2wz 2xz + 2wy 0 2xy + 2wz w 2 − x 2 + y 2 − z 2 2yz − 2wx 0 2xz − 2wy 2yz + 2wx w 2 − x 2 − y 2 + z 2 0 0001 ⎤ ⎥ ⎥ ⎥ ⎦ . (2.22) Note, however, that quaternions do not have a one-to-one mapping to 3D rotations: both ˆ q and − ˆ q (all components are negated) represent the same rotation. You can check this from the Equation (2.22): all the entries consist of a quadratic combination of quaternion components, so negating them all at the same time yields the same rotation matrix. How- ever, when we later interpolate quaternions in Section 4.1.2 it is very impor tant not to flip the quaternion signs, since the interpolation path from a rotation ˆ p to ˆ q is different than the path from ˆ p to − ˆ q. Rotations are a member of a special class of matrices, orthogonal matrices: all the columns are unit vectors, and they are perpendicular to each other (this also holds for the rows). Let us see what happens when we multiply R T R. The first row of R T is the same as the first column of R, and since it is a unit vector, dotted with itself it produces 1. Further, since it is perpendicular to the rest of the columns, the first row of R T R yields [ 1000 ] . Similarly, the other rows consist ofa1atthediagonal and 0 elsewhere. From this we see that the inverse of a rotation equals its transpose, that is, R −1 = R T . (2.23) 38 LINEAR ALGEBRA FOR 3D GRAPHICS CHAPTER 2 Whereas translation and rotation are so-called rigid transformations, that is, they do not change the shape or size of objects, scaling does change the size and potentially also shape. The scaling matrix is S(s x , s y , s z ) = ⎡ ⎢ ⎢ ⎢ ⎣ s x 000 0 s y 00 00s z 0 0001 ⎤ ⎥ ⎥ ⎥ ⎦ , (2.24) which scales the x componentofavectorbys x , the y component by s y , and the z compo- nent by s z .Ifs x = s y = s z , S is a uniform scaling which changes the size but not the shape, as in Figure 2.5. If the factors are not the same, then the shape is also changed. The inverse of scaling is equivalent to scaling down by the same factor: S −1 (s x , s y , s z ) = S(1/s x , 1/s y , 1/s z ). (2.25) Shearing is the last type of affine transformations, and also the least frequently used in computer graphics. A shearing matrix is obtained by changing one of the zeros in the upper-left 3 × 3 corner of the identity matrix to a non-zero value. An example is the matrix H 01 (h) = ⎡ ⎢ ⎢ ⎢ ⎣ 1 h 00 0100 0010 0001 ⎤ ⎥ ⎥ ⎥ ⎦ , (2.26) where the term in the first row and second column is set to h = 1 to create the shear in the last image of Figure 2.5. In this case, the greater the y component of the input, the more the x component is sheared to the right. Shearing always distorts the shape. The inverse of shearing is obtained by negating the same shear: H −1 ab (h) = H ab (−h). (2.27) One of the most common compound modeling transformations is rigid motion, which changes the location and orientation of an object, but retains its shape and size. Rigid motion consists of an arbitrary sequence of rotations and translations, and has the form M = Rt 0 1 , (2.28) where R is a 3 × 3 matrix, t is a 3 × 1 column vector, and 0 = [ 000 ] . M is equivalent to a rotation by R followed by a translation t. It has a simple inverse, obtained from the inverse of compound transformations (see Equation (2.15)) and the inverses of rotation and translation: start with a translation by −t followed by arotation by R −1 = R T , yielding Rt 0 1 −1 = R T −R T t 0 1 . (2.29) SECTION 2.3 AFFINE TRANSFORMATIONS 39 2.3.2 TRANSFORMATION AROUND A PIVOT As shown in Figure 2.5, all these operations are done using the origin as the pivot, the center of rotation. In the second image the triangle rotates around the origin, so it is both re-oriented and moved to a different location. In the third image the triangle grows and every point moves away from the origin. In the last image the vertical position is unchanged, but each point is sheared along the x axis and moved away toward the positive direction of the x axis. Of course, if the scaling would shrink the shape it would move toward the orig in, or if the sign of the shearing was flipped the movement would be to the opposite direction. However, often a transformation should be applied with respect to a pivot point,suchas the center or one of the corners of the shape. By default, the origin is the only point that does not move in rotation, scaling, or shearing. The trick is to move the pivot first to the origin, apply the transformation, and move the pivot back to where it was. For rotation, for example, the sequence would be T(p) RT(−p),wherep is the pivot. A more com- plicated example would scale the shape not only around a pivot, but also along a coor- dinate system that has been rotated by 30 ◦ about the z axis. Here the correct sequence would be T (p) R z (30 ◦ ) SR z (−30 ◦ ) T(−p). Reading from right to left, it first moves the pivot to the origin, then applies an inverse rotation to align the desired direction of the scaling with the coordinate axes, performs the scaling, then rotates back and moves the pivot back. 2.3.3 EXAMPLE: HIERARCHICAL MODELING The basic transformations can be nested to model objects in a hierarchical fashion. Figure 2.6 gives a simple example, a crane, which has been constructed from three boxes and two disks that hide the cracks that the boxes would make at the joints. The following pseudocode shows how an immediate mode API such as OpenGL would call the modeling operations to draw the crane of Figure 2.6. For simplicity we stay in 2D. The methods translate, rotate, and scale modify the current transformation matrix M by multiplying it from the right by a new matrix T, R,orS, respectively for each method. For example, the new matrix after rotate is obtained by M = MR. We have two functions available, draw_box() and draw_unit_disk(),whichdraw a box and a disk, respectively, extending from −1to1inbothx and y. Note that the sides of the box and the diameter of the disk have length 2, not 1. Here we also introduce the concept of a matrix stack. push_matrix() stores the cur- rent transformation matrix into a stack, from which it can be recovered by a matching pop_matrix(). At first we save the current transformation. We are going to modify the current coordinate system, so we want to be able to restore the original for drawing the rest of the scene: push_matrix() # save the current matrix 40 LINEAR ALGEBRA FOR 3D GRAPHICS CHAPTER 2 8 5 (0, 0) 1.5 1 1 4 6 2 458 708 (4, 5) Figure 2.6: A simple crane made of three boxes and two disks (to hide the seams at the joints). The origins of the coordinate systems of the base, upper arm, and the lower arm are indicated with the bold arrows that make a right angle. We start with a translation that places the crane: translate( x, y ) # move the whole crane Next we draw the base: push_matrix() scale( 4, 5/2 ) # scale the box to be8x5 translate( 0, 1 ) # move the box up so its bottom is at origin draw_box() # draw a box from — 1 to 1 in x and y pop_matrix() # restore the matrix at the time of last push The call to draw_box() is the last one before restoring the current transformation using pop_matrix(). Reading backward, we draw a box that would extend from −1to1in x and −1to1iny. However, before that we call translate which moves the box up byonesoitextendsfrom0to2iny. In other words, it now sits on top of the origin. This origin is now a good pivot for scaling the box to extend it from −4to4inx and from0to5iny. Note that if we did not have the matrix stack available, we should man- ually undo the modeling transformations by issuing translate( 0, — 1 ) and SECTION 2.3 AFFINE TRANSFORMATIONS 41 scale( 1/4, 2/5 ) at the end. Also note that even though we write in the code first scale, then translate, and finally draw, the primitives that are drawn are translated first, and scaled second, that is, the order of writing and evaluating is reversed. Now we translate the coordinate system to the upper right corner of the base so we can place the first disk there: translate( 4, 5 ) # move the origin to the upper right corner # first disk push_matrix() scale( 3/2 ) # scale the disk to have radius 3/2==1.5 draw_unit_disk() # draw a disk with radius 1 pop_matrix() Next we use the current origin as the rotation pivot for the upper arm. We rotate the upper arm by angle_1 degrees, which in this example is +45 degrees: rotate( angle_1 ) # rotate upper arm (45 degrees) The upper arm is then modified so that its origin is at the left end ofa6by2box,andthe boxisdrawn: push_matrix() scale( 3, 1 ) # scale the box to be6x2 translate( 1, 0 ) # move left center of the box to the origin draw_box() # draw a box from — 1 to 1 in x and y pop_matrix() Next we translate to the end of the upper arm. It is important to notice that we are cur- rently in the coordinate system of the upper arm, so we simply move the origin 6 units to the right along the x axis: translate( 6, 0 ) # move the origin to the end of upper arm There we draw the second disk that is already just the right size: draw_unit_disk() # draw a disk with radius 1 Finally, we rotate the coordinate system for the lower arm, draw the lower arm, clean up the transformations, and exit: rotate( angle_2 ) # rotate lower arm ( — 70 degrees) push_matrix() scale( 2, 1/2 ) # scale the box to be4x1 translate( 1, 0 ) # move left center of the box to the origin draw_box() # draw a box from — 1 to 1 in x and y pop_matrix() pop_matrix() # final pop to restore coordinate axes as they were 42 LINEAR ALGEBRA FOR 3D GRAPHICS CHAPTER 2 By calling this routine with different values of (x,y), angle_1 and angle_2 it is possible to move andanimate the crane. (x,y)moves everything, angle_1 rotates both the upper armand thelower arm attachedtoit, whileangle_2 onlyrotates thelower arm. 2.4 EYE COORDINATE SYSTEM Objects in world coordinates or object coordinates must be tr ansformed into the eye coordinate system, also known as camera coordinates, for rendering. By definition, the cam- era in OpenGL ES and M3G lies at the origin of the eye coordinate system, views down along the negative z axis, with the positive x axis pointing to the right and the positive y pointing up. Rather than modeling directly in the eye coordinates, it is easier to model objects in world coordinates, and start w ith a world-to-eye coordinate transformation. To calcu- late this transformation, we first study the structure of affine transformations and how they transform homogeneous points v. We exploit that structure to directly write down the eye-to-world transformation M ew , which we finally invert to obtain the world-to-eye transformation M we . We begin by asserting that M ew is an affine transformation and therefore has the last row [ 0001 ] : M ew = ⎡ ⎢ ⎢ ⎢ ⎣ abcd efgh ijkl 0001 ⎤ ⎥ ⎥ ⎥ ⎦ . (2.30) The first three columns of M ew are direction vectors, since their w components are zero, while the last column is a point as its w = 1. Looking at Figure 2.7 (and ig noring t and u for now), we want to choose values for M ew so that it transforms any point in the eye coordinate system xyz to the world coordinate system XYZ. Let us start from the origin o; its coordinates in xyz are [ 0001 ] T by definition. This extracts the last column of M ew : M ew [ 0001 ] T = dhl1 T = o x o y o z 1 T . Hence the last column of M ew is the location of o expressed in XYZ coordinates. Similarly, the first column is the directional vector x , again in XYZ coordinates: M ew [ 1000 ] T = [ aei0 ] T = x x x y x z 0 T , and the second and third columns are y and z, respectively, expressed in XYZ coordinates. Now we can take an ar bitrary point v = v x v y v z 1 T in eye coordinates xyz, and M ew produces the world coordinates XYZ for the same point. Let us see how this works. The last component w = 1 takes us to the origin o. From the origin, we move a distance v x to the direction of x,fromtherev y times y, and finally v z times z. To summarize, with the matrix from Equation (2.30) we get the 3D point o + v x x + v y y + v z z. SECTION 2.4 EYE COORDINATE SYSTEM 43 z y x u o t Y X Z Figure 2.7: Defining an eye coordinate system (lowercase xyz) using the camera origin o, target t, and view-up vector u, all defined in the world coordinate system (uppercase XYZ). We can now write down the whole eye-to-world transformation as M ew = ⎡ ⎢ ⎢ ⎢ ⎣ x x y x z x o x x y y y z y o y x z y z z z o z 0001 ⎤ ⎥ ⎥ ⎥ ⎦ (2.31) from which we obtain the world-to-eye matrix using the inversion formula of Equation (2.29): M we = ⎡ ⎢ ⎢ ⎢ ⎣ x x x y x z −x T o y x y y y z −y T o z x z y z z −z T o 000 1 ⎤ ⎥ ⎥ ⎥ ⎦ . (2.32) You can place this in the beginning of the rendering loop, and model the rest of the scene in world coordinates. The compound transformation moves the objects into the eye coor- dinates, ready for projection into the frame buffer. What remains is to calculate the coordinate axes xyz in the world coordinate system. In Figure 2.7 the camera is located at o, is looking toward the target point t, and is oriented so that the view-up vector u is aligned with the up direction on the image plane. Note that the view-up vector must point at some other direction than the viewing direction t−o for the following to work. Since the camera is viewing along the negative z axis, z must be the direction vector that extends from t to o, and we assign z = o −t ||o−t|| . We know that y must be on the same plane as z and the view-up vector u, while x must be perpendicular to that plane. We also know . first moves the pivot to the origin, then applies an inverse rotation to align the desired direction of the scaling with the coordinate axes, performs the scaling, then rotates back and moves the. to 1 in x and y pop_matrix() pop_matrix() # final pop to restore coordinate axes as they were 42 LINEAR ALGEBRA FOR 3D GRAPHICS CHAPTER 2 By calling this routine with different values of (x,y),. coordinates or object coordinates must be tr ansformed into the eye coordinate system, also known as camera coordinates, for rendering. By definition, the cam- era in OpenGL ES and M3G lies at the