3D Graphics with OpenGL ES and M3G- P39 potx

364 THE M3G SCENE GRAPH CHAPTER 15 Table 15.1: RayIntersection member functions for querying data about the closest inter- section when picking. The surface normal at the intersection point is interpolated if vertex normals are present, otherwise its value is undefined. Function Data returned float getDistance() The distance from the origin of the ray to the intersection point. The unit corresponds to the length of the direction vector of the ray— see getRay below. Node getIntersected() The Mesh or Sprite3D object intersected. float getNormalX() float getNormalY() float getNormalZ() The X, Y,orZ component of the surface nor- mal at the intersection point. void getRay(float[] ray) The origin, in elements 0 to 2, and direction, inelements3to5ofray, of the pick ray. int getSubmeshIndex() The index of the submesh intersected. Always zero for sprites. float getTextureT(int index) float getTextureS(int index) The S or T texture coordinate, for texture unit index, at the intersection point. For sprites, this is the pointwithin the sprite crop rectangle, with (0, 0) being at the top left corner of the displayed area and (1, 1) at the bottom right. Performance tip: Picking complex models can be very slow. To speed it up, use sepa- rate picking geometry. Make two Mesh objects for your model: one for rendering, and another, simplified one, for picking. Disable picking on the rendering model, and dis- able rendering on the picking model. Group them together and use the transformation of the Group node to transform them. 15.6 OPTIMIZING PERFORMANCE It is easy to fill your scene graph with more objects than M3G can handle with reason- able performance. When building real-life applications, you will almost certainly run into problems where M3G is not drawing your world quite as quickly as you would like. While M3G seems to offer very few tools explicitly geared for helping you with performance problems, there are a few relatively easy things you can do with the scene graph that may help you along the way. SECTION 15.6 OPTIMIZING PERFORMANCE 365 15.6.1 VISIBILITY OPTIMIZATION M3G as a specification takes no position on visibility optimization; it neither mandates nor recommends any specific method. However, most if not all commercially deployed implementations are doing view frustum culling, either on a per-mesh basis, or also hierarchically at the group node level. Pitfall: Early M3G-enabled devices from many vendors were plagued with bugs related to view frustum culling, causing meshes and/or sprites to disappear when they were definitely not supposed to. Some devices are known to exhibit problems when a VertexBuffer is shared by multiple Mesh objects, others fail to update the bound- ing box for animated meshes, and so forth. If you are developing for older devices and are experiencing weird visibility problems, we suggest that you pay a visit to the handset vendor’s developer forums. As a rule, any visibility optimization that a generic scene graph API does, a carefully crafted game specific engine can do better, and do it without burning cycles on opti- mizations that are not helpful with the particular game genre that the engine is designed for. This is one of the reasons that generic scene graphs have not met with great success on PCs, game consoles, and other native platforms. But again, the mobile Java platform is different. A complex culling scheme written in Java is not likely to compare well with a scheme that is running in native code, even if the complex system would be able to cull a larger percentage of objects. Fortunately, the developer has the option to combine the best of both worlds: to employ a spatial data structure that is optimized for the specific application, but also leverage retained-mode rendering. The scene can be organized into an appropriate spatial hierar- chy, for example an octree, by using Group nodes. The application must do a pre-render traversal of the scene graph, marking the potentially visible and definitely hidden groups of objects by setting or clearing the rendering enable flag s of the corresponding Group nodes with Node.setRenderingEnable. M3G can then render the potentially vis- ible set efficiently in one render(World) call. This approach is likely to perform much better than rendering the leaf nodes one by one in immediate mode. 15.6.2 SCOPE MASKS Every Node object in M3G has a 32-bit integer scope mask that you can freely set with setScope. Scope masks are another way to reduce the workload of M3G in com- plex scenes. You can use scope masks to exclude objects from rendering or picking, as well as reduce the number of lig hts affecting an object. When rendering via any of the render variants, the scope mask of each rendered mesh or sprite is compared with the scope mask of the current camera, and the object is ren- dered only if the masks share at least one bit that is set to 1. The same is done for each 366 THE M3G SCENE GRAPH CHAPTER 15 light source prior to computing lighting, in immediate as well as retained mode. Also, the scope parameter of pick is similarly compared with the scope mask of each object, and the object is only tested if the masks share at least one bit. Note, however, that scope masks are not inherited—the scope masks for Group nodes are therefore effectively ignored. To give a concrete example, imagine a scene w ith lots of objects and light sources scattered among multiple rooms. Each light can only interact with a small subset of the objects, so it is advantageous for you to inform M3G about that in advance. If you can determine which rooms each light can illuminate, you can encode that information into the scope masks: static final int LIVING_ROOM = 1<<0; static final int KITCHEN = 1<<1; static final int BEDROOM = 1<<2; static final int STAIRCASE = 1<<3; static final int HALL = 1<<4; Light kitchenLight, bedroomLight, livingRoomLight, hallLight; kitchenLight.setScope(KITCHEN|LIVING_ROOM|HALL); bedroomLight.setScope(BEDROOM|STAIRCASE); livingRoomLight.setScope(LIVING_ROOM|KITCHEN); hallLight.setScope(HALL|STAIRCASE|KITCHEN); Then, for each object you can just mark the room it is in: sofa.setScope(LIVING_ROOM); armchair.setScope(LIVING_ROOM); kitchenSink.setScope(KITCHEN); bed.setScope(BEDROOM); Finally, you can dynamically set the scope of the camera based on which rooms it can see from its current position: camera.setScope(LIVING_ROOM|HALL); Now, when you render, you will already have significantly reduced your set of visible objects and light interactions, saving a lot of work for the M3G implementation. You will still have to determine the potentially visible rooms each time the camera moves, though. Essentially, scope masks let you segment your scene into 32 groups or regions that are independent of the scene graph topology. For each camera and light, you can then decide which of those 32 groups it can see or affect. Similarly, for picking you can choose to detect different kinds of objects at different times. 32 groups may not sound like much, but since the scoping mechanism is completely orthogonal to the scene topology, you can use both scoping and Group nodes together when segmenting the scene. 16 CHAPTER ANIMATION IN M3G The final part of our tour of the M3G API is animation. M3G boasts a simple, yet flexible, animation engine with advanced features geared for the needs of modern games. The thinking behind the eng ine is to provide you with the basic building blocks you need, while leaving you in charge of using them in the way that best suits your application. You can use the animation engine as such or build a more comprehensive animation system on top of it, whichever best suits your particular needs. Fundamentally, animation in M3G is nothing more than a way to quickly set a number of object properties to pre-programmed, time-dependent values upon command. All of the properties modified by animation can be equally modified by calling setters and getters. The animation engine merely adds a conceptual model on top, enables pre-authoring of complex animations through the binary file format, and provides better performance via its native math engine. For background information on the animation concepts we will be dealing with in this chapter, please refer to Chapter 4. 16.1 KEYFRAME ANIMATION: KeyframeSequence The principal component of the M3G animation engine is keyframe animation (see Section 4.1). You can create keyframe se quences for most settable properties and 367 368 ANIMATION IN M3G CHAPTER 16 Skinned Mesh Group Keyframe Sequence Keyframe Sequence Keyframe Sequence Keyframe Sequence Keyframe Sequence Keyframe Sequence Animation Track Animation Track Animation Controller Animation Controller Animation Track Animation Track Animation Track Animation Track Animation Track Keyframe Sequence Group Group LightGroup Group … Mesh Target = COLOR Target = ORIENTATION … Keyframe Sequence Animation Track Animation Controller Target = TRANSLATION Figure 16.1: A SkinnedMesh object with animations applied to the translation of the root node and the orientations of the bone nodes. The AnimationTrack class links a keyframe sequence into its target property, while AnimationController supplies timing and control data. attach them to different objects for animation. Figure 16.1 shows an example of how the different animation classes link together; we will describe the classes in detail in the rest of this chapter. The KeyframeSequence class lets you create sequences of parameter values placed on a timeline and choose how to interpolate between them. For example, KeyframeSequence(10, 3, KeyframeSequence.LINEAR) creates a sequence of ten 3-vector keyframes for linear interpolation. Table 16.1 lists the available interpola- tion types, which are explained inmore detail inSections 4.1.1 and4.1.2. Thekeyfr ame val- ues for STEP, LINEAR, and SPLINE can be vectors with arbitrarily many components— including scalars as1-vectors—whereas SLERP and SQUAD require quaternion keyframes represented as 4-vectors. In all cases, all keyframes must have the same dimensionality within any single sequence. SECTION 16.1 KEYFRAME ANIMATION: KeyframeSequence 369 Table 16.1: Types of keyframe interpolation available in M3G. Name Description STEP Step function, no interpolation. LINEAR Linear interpolation. SPLINE Spline interpolation. SLERP Linear interpolation of quaternions. SQUAD Spline interpolation of quaternions. Note that when creating a keyframe sequence, you must decide both the number of keyframes and the interpolation type beforehand. This allows M3G to select all the required resources at construction time, avoiding potentially costly data reorganization later on. While fixing the number of keyframes may seem restrictive, you can still modify them dynamically and use only a subset of the sequence at any given time. More on that later. Setting up the keyframes After creating your new KeyframeSequence, you will want to put some data into it. Often, you will be loading pre-authored animations from resource files using the Loader class, but you could as well want to create your sequences at runtime for a number of reasons—for example, if you are streaming the animation data over a network connection or generating it procedurally at runtime. You can set your desired values to the keyframes by calling, int time, float[]value, where index is the index of the keyframe in the sequence, time is the time position of the keyframe, and value is a floating-point vector that gives the actual data value to be assigned to the keyframe. The key frame times are given relative to the beginning of the sequence, and must be monotonically increasing so that no keyframe has a time preceding any of the previous keyframes. In other words, the keyframes must be on the timeline in the same order as their indices. It is acceptable to have multiple keyframes at the same point in time, though. pitfall: The M3G file format offers a choice between 16-bit integer and 32-bit floating- point keyframes to enable potential space savings in the JAR file. However, some early M3G implementations (including the Sun Wireless Toolkit) exhibit problems when loading 16-bit keyframes, so you may find that your target platform can only use the floating-point variant. pitfall: There are no getters for keyframe times or values in M3G 1.0; they were only added in 1.1. The same goes for the number of keyframes and components, the interpolationmode,andthevalidrange.However,youcanextractthatdatabyabusingthe exceptions defined in setValidRange,setKeyframeandObject3D.animate. 370 ANIMATION IN M3G CHAPTER 16 The interpolation mode can be inferred by interpolating between known keyframes. Extracting all the hidden data takes some 200 lines of code and is dog-slow, but you can do it if you must for some pressing reason, such as for debugging purposes. Our companion web site includes example code implementing a class that you can readily use for this purpose. Note that we are talking about time without specifying any time unit. This is intentional, since M3G leaves the choice of time units to you. Feel free to use milliseconds, seconds, or microseconds for your keyframes, and M3G will happily produce the results you want as long as you are consistent. In fact, you can even use different time units for different sequences if you happen to need that sort of thing. Refer to Section 16.3 for information on doing that. Duration and looping After setting all your keyfr ames, you must also set the duration of your keyframe sequence by calling setDuration. This information is used by M3G when you want your keyframe sequences to loop, but it also serves as a sanity check for your keyframe times. pitfall: Your keyframe sequences are not valid until a valid duration is set. This concerns all sequences, regardless of whether they loop or not. We mentioned looping sequences, and often you will use exactly those. For example, the motion of a helicopter rotor is easily defined using just two quaternion keyframes that repeat over and over. To enable looping, call setRepeatMode(Keyframe- Sequence.LOOP)—the default value of KeyframeSequence.CONSTANT means that the sequence maintains the value of the first or last keyframe when sampled outside of the beginning or end of the sequence, respectively. With looping enabled, the keyframes are conceptually repeated in time at intervals of the sequence duration d, to infinity and beyond in both directions. An example is shown in Figure 16.2. The effect is exactly the same as replicating each key frame at every time instant ( ,t− 2d, t− d, t, t+d, t+2d, ), where t is the original time of the keyframe, but without any additional memory use. To wr ap up the basic steps of keyframe sequence creation, let us create a looping sequence that you can use to perpetually rotate an object about the Z axis at a steady pace. This sequence will interpolate quaternion keyframes, using spherical linear interpolation, so that the target object completes one revolution every one thousand time units: static float kf0[4] = { 0.f, 0.f, 0.000f, 1.0f }; static float kf1[4] = { 0.f, 0.f, 0.866f, — 0.5f }; static float kf2[4] = { 0.f, 0.f, — 0.866f, — 0.5f }; KeyframeSequence rotationSeq = new KeyframeSequence(3, 4, KeyframeSequence.SLERP); SECTION 16.1 KEYFRAME ANIMATION: KeyframeSequence 371 virtual keyframes real keyframes virtual keyframes -d 0d2d Figure 16.2: A looping KeyframeSequence: virtual keyframes are created around the original sequence at intervals of duration d. rotationSeq.setDuration(2000); rotationSeq.setRepeatMode(KeyframeSequence.LOOP); rotationSeq.setKeyframe(0, 0, kf0); rotationSeq.setKeyframe(1, 667, kf1); rotationSeq.setKeyframe(2, 1333, kf2); The keyframe values are unit quaternions corresponding to rotations of 0, 240, and 480 degrees, respectively—since we are using quaternions, we will complete two revo- lutions in 3D when rotating one full circle in the quaternion space (refer to Sections 2.3.1 and 4.1.2). Hence, we set the duration to 2000 time units to match our desired time for one revolution. Alternatively, we could set keyframes at 0, 180, and 360 degrees at 0, 500, and 1000 milliseconds, respectively, and set the sequence duration at 1000 milliseconds. How- ever, that would introduce a discontinuit y into our interpolated quaternion data, which would cause awkward jumps in the animation if we later tried to blend (Section 16.5.2) our rotation with another orientation sequence. Valid keyframe range One more feature of KeyframeSequence is the valid range. As we already hinted, this lets you choose a subset of the entire sequence. That subset is then used exactly like a sequence comprising just those keyframes. For example, calling setValidRange (3, 7) would only take the keyframes with indices 3 to 7 into account when you use the sequence. You can also specify a valid range that wraps around: setValidRange (8, 2) would treat your 10-keyframe sequence as if it comprised keyframes 8, 9, 0, 1, 372 ANIMATION IN M3G CHAPTER 16 and 2, in that order—keyframe 8 would then have to have the lowest time value for the sequence to be valid. The valid range is useful if, for any reason, you want to dynamically modify the contents or length of your animation sequence, as it saves you from creating new sequences and inducing garbage collection. We will show some example use cases later on. 16.2 ANIMATION TARGETS: AnimationTrack A KeyframeSequence on its own does very little—you need something to connect the animation to. In M3G parlance, each animatable property on each object is called an animation target. To connect a keyfr ame sequence with an animation target, you create an AnimationTrack object and add it to your target object via Object3D.addAnimationTrack. Animatable properties are enumerated in the AnimationTrack class as listed in Table 16.2. AnimationTrack associates a keyframe sequence with one of these prop- erties. To create a track for animating the position of the scene graph node myNode using the KeyframeSequence object mySequence, for example, you could use this piece of code: AnimationTrack myPositionTrack; myPositionTrack = AnimationTrack(mySequence, AnimationTrack.POSITION); myNode.addAnimationTrack(myPositionTrack); We can also animate myNode with the rotation sequence we created in the earlier example: AnimationTrack rotationTrack; rotationTrack = new AnimationTrack(rotationSeq, AnimationTrack.ORIENTATION); myNode.addAnimationTrack(rotationTrack); Every object can be animated w ith multiple tracks at the same time, so doing both of the above would make myNode both move and rotate. All properties are animated as floating-point values interpolated from the keyfr ames. For boolean properties, values of 0.5 or higher are interpreted as “tr ue,” and values under 0.5 as “false.” Values for certain properties, such as colors, are clamped between 0 and 1 after interpolation. For the ORIENTATION property, the quaternion values are automatically normalized after interpolation. Performance tip: Quaternions do not require SLERP or SQUAD interpolation. Due to the automatic normalization, using plain LINEAR or SPLINE interpolation will make you only lose the constant velocity property. For many animations this is good SECTION 16.2 ANIMATION TARGETS: AnimationTrack 373 Table 16.2: List of animatable properties in M3G 1.1. Name Applicable properties ALPHA Alpha component of Background, Material , and VertexBuffer color; Node alpha factor. AMBIENT_COLOR Ambient color in Material. COLOR Color of Light, Background, Fog, and VertexBuffer; Texture2D blend color. CROP Cropping rectangle in Background and Sprite3D. DENSITY Fog density. DIFFUSE_COLOR Diffuse Material color. EMISSIVE_COLOR Emissive Material color. FAR_DISTANCE Far distance in Camera and Fog. FIELD_OF_VIEW Camera field of view. INTENSITY Light intensity. MORPH_WEIGHTS MorphingMesh weights. NEAR_DISTANCE Near distance in Camera and Fog. ORIENTATION Transformable orientation. PICKABILITY Picking enable flag of Node. SCALE Transformable scale. SHININESS Material shininess. SPECULAR_COLOR Specular Material color. SPOT_ANGLE Spotlight angle of Light. SPOT_EXPONENT Spotlight exponent of Light. TRANSLATION Transformable translation. VISIBILITY Rendering enable flag in Node. enough, and if you can use simpler interpolation on most of the animations, you will save valuable processing time for other things. For example, try the simple LINEAR or SPLINE modes on your bone orientations. pitfall: There is something of an infamous problem regarding SLERP interpolation in several M3G 1.0 implementations as well as early content authoring tools, as discussed in Section 12.3.2 This issue was resolved in M3G 1.1, but if you want your content to be compatible with the faulty M3G 1.0 implementations as well as with the correctly implemented ones, you must take extra precautions in constructing your SLERP keyframe sequences: make sure that the angle between the orientations described by adjacent keyframes is less . classes link together; we will describe the classes in detail in the rest of this chapter. The KeyframeSequence class lets you create sequences of parameter values placed on a timeline and choose. move and rotate. All properties are animated as floating-point values interpolated from the keyfr ames. For boolean properties, values of 0.5 or higher are interpreted as “tr ue,” and values under. per-mesh basis, or also hierarchically at the group node level. Pitfall: Early M3G-enabled devices from many vendors were plagued with bugs related to view frustum culling, causing meshes and/ or