3D Graphics with OpenGL ES and M3G- P33 ppt

304 BASIC M3G CONCEPTS CHAPTER 13 (pre)ortotheright(post) of the current R; scaling and translation are order-independent. These methods take the same parameters as their setter counterparts. Transformable also defines a getter for each of the four components, as well as for the composite transformation C: void getTranslation(float[] translation) void getOrientation(float[] angleAxis) void getScale(float[] scale) void getTransform(Transform transform) void getCompositeTransform(Transform transform) Note that there is indeed only one getter for each component, not separate ones for tx, ty, tz, angle, and so on. Consistent with the API conventions, the values are filled in to a float ar ray or Transform object designated by the user, thus facilitating object reuse. Rotations Rotations in Transformable are specified in the axis-ang le format, which is very intuitive, but unfortunately less robust and sometimes less convenient than quaternions. There are no utility methods in the API to convert between the two representations, but luckily this is quite simple to do in your own code. Denoting a normalized axis-angle pair by [ ˆ a θ] = [a x a y a z θ], and a unit quaternion by ˆ q = [q v q w ] = [q x q y q z q w ], the conversions are as follows: [q v q w ] = [ ˆ a sin(θ/2) cos(θ/2)] (13.3) [ ˆ a θ] =  q v /  1 − q w 2 2 acos(q w )  . (13.4) Both formulas assume the input axis or quaternion to be normalized, and will produce normalized output.If the axis-angle outputis intended forM3G, however,you do not need to normalize the axis because M3G will do that in any case. You may therefore skip the square root term, yielding a significantly faster conversion from quaternion to axis-angle: [a θ] = [q v 2 acos(q w )]. (13.5) In other words, only the rotation angle needs to be computed, because q v can be used as the rotation axis as such. Remember that the angle needs to be degrees, and if your acos() returns the angle in radians, the resulting θ must be multiplied with $180/π. Note that the input quaternion must still be normalized, or else acos(q w ) will yield an incorrect value for θ. A quaternion can be normalized just like any other vector: ˆ q = q/  q x 2 + q y 2 + q z 2 + q w 2 . (13.6) SECTION 13.3 MATRICES AND TRANSFORMATIONS 305 Pitfall: Due to the unfortunate lack of inverse trigonometric functions in mobile Java, youwillhavetowriteanacos routine yourself, or download one from the web. In any case, ensure that the routine has sufficient precision, or otherwise you will not get smooth animation. Pivot transformations The scale, orientation, and translation components are not sufficient to fully represent affine transformations (that is, all 3 × 4 matrices). For that, we would also need scale orientation O and pivot translation P. Scale orientation provides for scaling along arbitrary orthonormal axes, rather than just the primary axes. Pivot translation allows any 3D point to be set as the center of rotation, instead of just the origin. The composite transformation would then become C = TPRP −1 OS O −1 M (13.7) Scale orientation and pivot translation were omitted from M3G, because they are rarely needed in practice. They would also add substantial storage and processing overhead to scene graph nodes, even when not used. If you do need a full affine t ransformation, however, you can composite one in your own code, and feed it in as the generic M component. This works for both texture transformations and node transformations. For node transformations, you have an easier alternative: use some extra Group nodes and split the transformation into multiple parts, depending on which components you need to control separately. Typically, P and O remain constant, while the translation, rotation, and scale are animated. In this case, you could assign the components into one extra Group and the original leaf node as shown in Figure 13.1. Note that T and P are combined into T A . Group A Node B T A 5 T P R A 5 R S A 5 I M A 5 I T B 5 P 21 R B 5 O S B 5 S M B 5 O 21 Figure 13.1: Using an extra Group node to implement rotation around a pivot point (PRP −1 ) and oriented scaling (OSO −1 ). 306 BASIC M3G CONCEPTS CHAPTER 13 This works fine withtranslate, but if you usesetTranslation, you must remember to factor in the pivot translation. 13.4 Object3D Object3D is an abstract base class for all objects that can be part of a 3D scene. As shown in Figure 12.3, all but four classes in the whole API are derived from Object3D. It defines properties and capabilities that are common to all scene graph objects, including the ability to be keyframe-animated, duplicated, or imported from a file. 13.4.1 ANIMATING Arguably the single most powerful method in Object3D is animate(int time), which invokes the built-in keyframe animation system. The animate method first updates all animated properties in the Object3D that it is called on, then in all Object3Ds that are referenced from it, and so on in a recursive manner. In other words, animate updates all objects that are reachable from the initial object by following a chain of references. There are two kinds of references that are not followed, though: those that go upward in the scene graph, and those that go sideways, jumping from one branch to another. This is all very intuitive—if you animate a particular branch of the scene graph, you expect all objects in that branch to be updated, not just the root node. On the other hand, you do not expect the rest of the scene to be animated with it. Recalling that World is derived from Object3D, and correctly deducing that there must be a chain of references from a World object to everything in the scene graph, it follows that we can animate an entire scene with this single line of code: myWorld.animate(myWorldTime); This updates all animated properties in myWorld to the given time. You can then render the updated scene as such, or further animate it using ordinary Java code. animate is typically called once per frame, and the world time increased by the number of millisec- onds elapsed since the previous frame. The keyframe animation system is described in detail in Chapter 16. 13.4.2 ITERATING AND CLONING There are three other methods in Object3D that follow the same principles of traversing the scene graph as animate. Let us start with getReferences, which helps you iterate through the scene graph in your own code: int getReferences(Object3D[] references) SECTION 13.4 OBJECT3D 307 This method retrieves all objects that are directly referenced by an Object3D, again not including references to parent nodes or other branches. The integer return value tells you how many such direct references there are. Typically you would first invoke getReferences(null) to get the number of references, then allocate an array of sufficient size and call the method again to get the list of objects. Note that there may be duplicate objects in the list, as each reference is treated separately. To illustrate a case when getReferences is highly useful, consider the following utility method that retrieves all objects of the given class type that are reachable from the given root object, eliminating duplicates along the way. For example, to retrieve all cameras in the scene, you would call getUniqueObjects(cameras, myWorld, Class.forName("javax.microedition.m3g.Camera")), just making sure that the cameras array is large enough. // A recursive method to find all Object3Ds of given ’type’ that // are reachable from ’root’. Returns the number of unique, // matching objects and inserts them into the ’found’ array. // This method is not very efficient: it takes O(N^2) time and // O(N) memory, where N is the number of objects traversed. // On the other hand, finding objects in the scene graph is // typically a one−off process. // int getUniqueObjects(Object3D[] found, Object3D root, Class type) { int i, numUnique = 0; // Retrieve the scene graph objects that are directly referenced // by ’root’ and process them recursively. Note that we create // a new ’references’ array at each step of the recursion; this // is not recommended as a general practice, but in this case we // are favoring brevity and clarity over efficiency. // Object3D[] references = new Object3D[root.getReferences(null)]; root.getReferences(references); for (i=0; i < references.length; i++) { numUnique += getUniqueObjects(found, references[i], type); } // Check whether ’root’ is an instance of ’type’, and if so, // insert it at the end of the ’found’ array, provided that // there is at least one empty slot left. Then loop through // the array, checking if ’root’ has been inserted before. // If not, we let it remain in the array and increase the // count of unique objects by one. // if (type.isAssignableFrom(root.getClass()) && numUnique < found.length) { found[numUnique] = root; for (i=0; found[i] != root; i++); 308 BASIC M3G CONCEPTS CHAPTER 13 if (i == numUnique) numUnique++; } return numUnique; } That is just fifteen lines of code, excluding comments, of which only five are spent actually traversing the scene graph. If it were not for getReferences, those five lines would be replaced with a horrendous switch-case construct, probably using up several hun- dred lines of code. The second utility method in Object3D that deals with chains of references is duplicate. This method creates a copy of the object that it is called on. For most objects that amounts to a shallow copy, in which the instance variables and references are copied, but the referenced objects themselves are not. However, for non-leaf scene graph nodes, a deep copy is necessary to maintain the integrity of the duplicated scene graph branch. Given the rule that scene graph nodes can have at most one parent in M3G, how should we go about cloning a Group node, for instance? The children of that group cannot be in both the new and the old group at the same time, so there are two choices: leave the new group empty, or clone the entire subtree. M3G does the latter, because creating a “duplicate” that has none of the original contents would be rather misleading and pointless. Now that you know how duplicate works, you may wonder why it has to exist in the first place. We already have clone in java.lang.Object, right? Wrong: there is no clone in CLDC/MIDP. It does exist on the higher-end CDC platform, though, and may some day exist on CLDC. On such platforms, Object3D will include both duplicate and clone. However, the behavior of clone on Object3D and its derived classes is left undefined in the M3G specification, so we recommend using duplicate instead. 13.4.3 TAGS AND ANNOTATIONS The one remaining scene graph traversal method, find(int userID), is best utilized together with the M3G file format. Importing content from .m3g files is discussed in the next section, but what is important for understanding the benefits of find is that since you can load an ar bitrarily large number of objects in one go, locating the objects that you need may be difficult. If there are a thousand meshes in the scene, how do you identify the flashlight object that your player character should be able to pick up? That is when find and userID come in handy. Finding objects by ID Despite being just a simple integer, the userID is a very powerful tool that allows the graphics designer and application programmer to synchronize their work. First, the designer assigns a unique userID for each object that needs individual treatment at SECTION 13.4 OBJECT3D 309 runtime. For instance, the player character might be tagged with the ID 1000, her right hand with 1010, and different items that she can hold with IDs 2000-2010. The designer then exports the 3D scene into a .m3g file. Finally, at runtime, the scene is loaded in and the relevant objects retrieved using find: Node player = (Node)myWorld.find(1000); Node playerRightHand = (Node)player.find(1010); Mesh flashlight = (Mesh)myWorld.find(2000); Mesh knife = (Mesh)myWorld.find(2010); Formally, Object3D.find retrieves the first Object3D that has the given userID and is reachable from the object where the search is started. The definition of being reachable is the same as with animate. Adding metadata The UserObject is another mechanism that allows the designer to communicate with the runtime engine. The user object is simply an arbitrary block of metadata that can be associated with any scene graph object. The interpretation of that metadata is up to you; M3G itself never touches the user object. You can associate a user object with an Object3D either at runtime, in which case it can be anything derived from java.lang. Object, or through the M3G file format, in which case it will be a java.util.Hashtable filled with byte[] elements that are keyed by Integers. It is then up to the application to parse the byte arrays to extract their meaning. Advanced annotations As an advanced example that leverages both the UserID and the UserObject,letus associate symbolic names with those bones of a SkinnedMesh that we need to manipu- late at runtime. The names of the bones have been agreed on by the designer and the programmer, and they are of the form “left_arm.” The designer identifies the relevant bones in the authoring tool, assigns the agreed-upon names and any arbitrary userIDsto them, and finally, with a little support from the exporter, stores the (userID, name) pairs into the UserObject field of the SkinnedMesh. At runtime, having loaded the M3G file, we first retrieve the UserObject that has by now taken the form of a Hashtable with (Integer,byte[]) pairs. In this case, the integers are actually our user IDs, while the by te arrays are the names. We then iterate through the hash table: take the user ID, find the bone that corresponds to it, and finally build a new hash table (called bones) that associates each bone w ith its name: // Load the SkinnedMesh and get the table of (userID, name) pairs, // then set up a new hash table that will associate bones with 310 BASIC M3G CONCEPTS CHAPTER 13 // their symbolic names. // SkinnedMesh creature = (SkinnedMesh)Loader. load("/creature.m3g")[0]; Hashtable names = (Hashtable)creature.getUserObject(); Hashtable bones = new Hashtable(names.size()); // For each UserID in the (userID, name) table: // 1. Get the name corresponding to that UserID. // 2. Convert the name from a byte array into a Java String. // 3. Find the bone Node corresponding to the UserID. // 4. Insert the String and the Node into the new hash table. // Enumeration userIDs = names.keys(); while (userIDs.hasMoreElements()) { Integer userID = (Integer)userIDs.nextElement(); String name = new String( (byte[])names.get(userID) ); Node bone = (Node)creature.find(userID.intValue()); bones.put(name, bone); } // Finally, bind the (name, bone) table as the UserObject // of our SkinnedMesh, replacing the (userID, name) table // (assuming that it will not be needed anymore). // mySkinnedMesh.setUserObject(bones); Now that we have some semantics associated with our bones, it becomes a breeze to animate any specific part of the character in our main loop. For example, to move the left arm into a certain position relative to the left shoulder, you just do this: Node leftArm = (Node)bones.get("left_arm"); leftArm.translate( ); Note that if you are going to update leftArm very often, it may be smart to cache it in an instance variable rather than looking it up from the hash table every time. Annotating bones—or any scene graph objects, for that matter—with symbolic names is a good idea, because it allows the designer to change the scene representation without the programmer having to change the application code. If the application relies on the left arm being a certain number of steps from the skinned mesh root, for example, things will break down immediately if the artist decides to add a few fix-up bones to the shoulder region. Furthermore, using plain-text names rather than just integers leaves less room for typing errors for both the artist and the programmer, and is of great help in debugging. SECTION 13.5 IMPORTING CONTENT 311 13.5 IMPORTING CONTENT The easiest way to construct a scene graph in M3G is to import it from the JAR package or a network URL using the built-in Loader. The loader can import individual images from PNG and JPEG files, and complete or partial scene graphs from M3G files. There are numerous M3G exporters available for 3ds Max, Maya, Softimage, Lightwave, and Blender; see the companion web site for an up-to-date list. For a step-by-step tutorial on creating some M3G content in Blender, see Chapter 10 of Mobile 3D Graphics: Learning 3D Graphics with the Java Micro Edition by Claus H ¨ ofele [H ¨ o07]. We begin this section by explaining how to use the Loader, then proceed to discuss the M3G file format in enough detail to get you started with your own content processing tools. 13.5.1 Loader The Loader will accept at least PNG and M3G files on any device, but JPEG is also supported on many devices. 5 Loading a PNG or JPEG yields a single Image2D object (see also Section 13.2). M3G files, on the other hand, can store anything up to and including an array of complete scene graphs. They may even include an arbitrary number of other files by reference. The number of objects per file and the total size of those objects are bounded only by the available memory in each device. Methods Loader is one of the four special classes in M3G that are not derived from Object3D (see the class diagram in Figure 12.3). Moreover, it cannot be instantiated, and its only members are these two static load methods: Object3D[] load(String name) Object3D[] load(byte[] data, int offset) The first variant loads the content from a named resource, which can be a file in the JAR package or a URL (typically of the form http:// ). Named resources are treated as case-sensitive, and must specify an absolute path (e.g., /bg.png rather than just bg.png). This method is what you will probably use in most cases. The other form of load reads the data from a byte array. This method may be useful in special cases: if you need to embed some art assets into your Java class files for whatever reason, this method allows you to do that. It also lets you manually load and preprocess your content before forwarding it to the loader. For example, you could make it harder to 5 At least on those that conform to the MSA (JSR 248) specification. 312 BASIC M3G CONCEPTS CHAPTER 13 rip off or reverse-engineer your assets by keeping themin encrypted form, only decrypting them at runtime. Both load methods are synchronous, i.e., they only return when the entire contents of the given input have been successfully loaded, along with any files that are included by reference. It is therefore not possible to render or otherwise process a partially loaded file. There is also no way to get any progress information from the loader. If loading takes a lot of time in your application and you need a good progress bar, our advice is to split your assets into multiple files that are loaded separately. Output Both load methods return an Object3D array. This array contains the root objects of the file. A root object is one that is not referenced by any other object in the file. There may be an arbitrary number of root objects in a file, and the root objects may, in turn, reference an arbitrary number of non-root objects. Finding the objects that you need from within that mass of objects is made easier by the Object3D methods find(int userID) and getReferences, as discussed in the prev ious section. All the returned objects, whether root or non-root, are guaranteed to be consistent, meaning that you might as well have constructed them with the API without getting any exceptions thrown. However, they are not guaranteed to be in a renderable condition. For example, there may be textures or other components missing, causing an exception if you try to render the scene. This is fully intentional, as it allows you to store a partial scene graph in a file and patch in the missing pieces at runtime. Pitfall: The peak memory consumption of many applications occurs at the setup stage, and asset loading plays a big role in that. Let us hypothesize what may be happening behind the scenes when you load a PNG image into a MIDP Image and then convert it into an Image2D. The PNG file is first decompressed from the JAR; then the PNG itself is decompressed into the Image; finally, the Image is copied into Image2D.In the worst case, you m ay have one compressed copy and two uncompressed copies of the image in memory at the same time! The temporary copies are of course released in the end, but that does not help with your peak memory consumption. If you keep running out of memory while loading, we would suggest you to split your content into parts that are loaded separately. Example The following code fragment gives a concrete example of using the Loader.Thecode first loads the file myscene.m3g from the /res subdirectory of the JAR package, and then uses the runtime type information provided by Java to find the first World object SECTION 13.5 IMPORTING CONTENT 313 among the potentially many root objects. Note that if there are any Worlds in the file, they are guaranteed to be among the root objects. try { Object3D[] objects = Loader.load("/res/myscene.m3g"); for (int i=0; i < objects.length; i++) { if (objects[i] instanceof World) { myWorld = (World)objects[i]; break; } } } catch (Exception e) { } Catching any exceptions that might occur during the loading is not only good program- ming practice, but actually required: the Loader may throw an IOException, which is a special kind of exception in that it must be either explicitly caught or declared to be thrown to the calling method. You cannot just not mention it and let the caller take care of it. Note that the above example will not work reliably if there are several World objects in the file, because the Loader does not guarantee that they are returned in any particular order. On the other hand, if we are certain that there is never going to be more than one root object in the file, that being our desired World, we can omit the for loop altogether and just do this: myWorld = (World)Loader.load("/res/myscene.m3g")[0]; Using Loader.load is so straightforward that there is not much else to say about it, so let us continue with the M3G file format instead. You may now want to fast-forward to the next chapter if you do not intend to debug any .m3g files or develop any M3G content processing tools in the foreseeable future. 13.5.2 THE FILE FORMAT The graphics industry is certainly not suffering from a shortage of file formats. There must be a thousand 3D file formats in existence, so why did the M3G standardization group have to invent yet another one? Well, probably for the same reasons as most of the file format designers before us: we needed a format that matches the runtime API perfectly, not just so-and-so. We wanted to leave as little room for interpretation as possible, and that is best achieved if every object and field in the file format has a one-to-one mapping with the API. One of the key features of the file format is its simplicity. We opted for quick and easy reading and writing, rather than extensibility, maximum compression, random access, streaming, error concealment, or a number of other things that are best handled elsewhere or not required at all. You will no doubt value that decision if you ever need to write your . UserID and the UserObject,letus associate symbolic names with those bones of a SkinnedMesh that we need to manipu- late at runtime. The names of the bones have been agreed on by the designer and. getReferences(Object3D[] references) SECTION 13.4 OBJECT3D 307 This method retrieves all objects that are directly referenced by an Object3D, again not including references to parent nodes or other. programmer, and they are of the form “left_arm.” The designer identi es the relevant bones in the authoring tool, assigns the agreed-upon names and any arbitrary userIDsto them, and finally, with a