264 EGL CHAPTER 11 100 90 23 45 Length of the test run (%) Figure 11.2: Duration with frame rate capped to 15 FPS but with different features enabled. 2 = textured and lit, 3 = textured with mipmaps and lit, 4 = nontextured and lit, 5 = nontextured, no lighting. Studying Figure 11.1 one can see that dropping the frame rate has the biggest effect on how long the batteries last (about 30%). Enabling mipmapping also saves some energy by allowing more coherent memory access patterns, as can be seen in Figure 11.2. Disabling texture mapping has some effect, as well as dropping lighting altogether. Although on this particular hardware the optimizations in examples 2–5 did not provide sig nificant wins, on some other hardware the effect may be more noticeable, especially if the vertex pipeline is not hardware accelerated. 11.10 EXAMPLE ON EGL CONFIGURATION SELECTION The criteria for optimal EGL configuration selection logic depend on the application, so no generic optimal solution can be implemented. If it were possible, it would already be implemented in EGL! In this example, the primary goal is to find a configuration that matches at least the minimum color buffer and depth buffer bit depths requested by the application. A secondary goal is to find a configuration that has at least as many stencil bits as requested by the application. If such a configuration does not exist, stencil selec- tion will be completely ignored by the rest of the code. Finally, a configuration with the best antialiasing quality is selected among the configurations that otherwise match the requirements. The example function does not return any information about the config- uration that was selected, as these can be queried using the various glGet functions. An alternate way for compactly specifying the requirements in the application would be to let the application specify both the minimum and optimal requirements for the attributes, and then sort the requirements in the order of importance. In this case SECTION 11.10 EXAMPLE ON EGL CONFIGURATION SELECTION 265 the configuration that matches the optimal requirements would be searched for, and if multiple candidates are found, the one with the best antialiasing support would be picked. If no configurations fulfill the optimal requirements, a configuration that matches at least the minimum requirements would be selected. If no configuration is found that supports even the minimum requirements, none would be returned, and the application would have to exit. In the first code snippet, we construct a list of configuration attributes and filter out with eglChooseConfig the configurations that clearly do not match our surface type requirements (set by the caller), color depth, and depth buffer bits. If stencil bits are requested, they are also set as a requirement to the attribute list. EGLConfig select_config( int surfacetype, int framebuf_bits, int depthbuf_bits, int stencil_bits ) { EGLBoolean err; EGLint amount; EGLint attrib_list[5*2]; EGLConfig configs[64], best_config; EGLint *ptr = &attrib_list[0]; *ptr++ = EGL_SURFACE_TYPE; *ptr++ = surfacetype; *ptr++ = EGL_BUFFER_SIZE; *ptr++ = framebuf_bits; *ptr++ = EGL_DEPTH_SIZE; *ptr++ = depthbuf_bits; if( stencil_bits ) { *ptr++ = EGL_STENCIL_SIZE; *ptr++ = stencil_bits; } *ptr++ = EGL_NONE; err = eglChooseConfig( eglGetDisplay( EGL_DEFAULT_DISPLAY ), &attrib_list[0], &configs[0], 64, &amount); Now, amount contains the number of configurations that fulfill our requirements. If no configurations were returned, a new call to eglChooseConfig is made with an attribute list where the stencil requirement is dropped. if( amount == 0 ) { attrib_list[6] = EGL_NONE; err = eglChooseConfig( eglGetDisplay( EGL_DEFAULT_DISPLAY ), &attrib_list[0], &configs[0], 64, &amount ); } 266 EGL CHAPTER 11 At this stage, we either have a list of configurations supporting stencil, or we have configurations that do not support stencil, or we have zero configurations if the basic requirements are not met. If no configurations exist, we just exit the code. Otherwise, we continue by finding the one with the best antialiasing, i.e., most samples per pixel. if( amount>0) { int i, best_samples; best_samples = 0; best_config = configs[0]; for(i=0;i<amount; i++ ) { int samp; eglGetConfigAttrib( eglGetDisplay( EGL_DEFAULT_DISPLAY ), configs[i], EGL_SAMPLES, &samp ); if( samp > best_samples ) { best_config = configs[i]; best_samples = samp; } } return best_config; } else { return (EGLConfig) 0; } } PART III M3G This page intentionally left blank 12 CHAPTER INTRODUCING M3G Practically all mobile phones sold in developed countries are equipped with Java Micro Edition (Java ME), making it the most widely deployed application platform in the his- tory of computing. A rapidly growing subset of those devices come pre-installed with M3G (Mobile 3D Graphics API for Java ME; also known as JSR 184) [JCP05]. As of 2007, there are more than a dozen device vendors shipping M3G-enabled devices, with yearly shipments in the order of hundreds of millions. To get hold of such a device, just pick up— for example—any Nokia or Sony Ericsson phone with a quarter-VGA display (240 × 320 pixels). This chapter introduces M3G, putting it in the context of the mobile Java environment, OpenGL ES, and other scene graph engines. The later chapters will get you started with programming on M3G. Our presentation is aligned with Part I of this book, and builds on concepts introduced there. In other words, we assume that you are familiar with the OpenGL (ES) rendering model, scene graphs, keyframe animation, mesh deformation, and so on. Reading Part II will give you further insight to the inner workings of M3G, but it is not a prerequisite for understanding the following chapters or for utilizing M3G in pr actice. This book is not about teaching Java programming; you should already have working knowledge of objects, classes, inheritance, exceptions, garbage collection, Java virtual machines, and other basic concepts of object-oriented programming. Neither is this book about mobile Java per se; we do not explain how to use its 2D graphics libraries, or how to wr ite well-behaving applications. Familiarity with these topics may help, but is not strictly 269 270 INTRODUCING M3G CHAPTER 12 necessary, as our example framework (available on the companion web site) takes care of all the non–3D-related code. 12.1 OVERVIEW The Mobile 3D Graphics API provides Java programmers with an efficient and compact interface for managing and rendering 3D scenes. It is based on the established OpenGL rendering pipeline, yet designed for Java with an object-oriented mindset, providing for a shallow learning curve for beginners, and high productivity for seasoned 3D program- mers. Due to its retained-mode design, it minimizes the performance overhead of the Java virtual machine. The strict specifications and rigorous conformance tests of M3G ensure application portability from one device to another, allowing developers to reach hundreds of millions of devices from different vendors with a reasonable effort. 12.1.1 MOBILE JAVA Figure 12.1 shows an overview of the mobile Java software architecture and the position- ing of M3G in it. At the top of the diagram, we have the applications, called midlets in this environment. The term originates from MIDP, the Mobile Information Device Pro- file, which is show n in the diagram just beneath the application layer. MIDP defines the structuring and packaging of mobile Java applications, as well as the basic features that are available to them in the runtime execution environment. All midlets must adhere to an event-driven framework that is intended to make them better behaved with respect to shared resources and system events, such as incoming phone calls. To this end, midlets cannot have a main method, but must instead implement M3G MIDP Other JSRs OpenGL ES VM (CLDC/CDC) Java applications (midlets) Figure 12.1: The mobile Java software stack. SECTION 12.1 OVERVIEW 271 a set of event handlers like startApp and pauseApp, as shown in Figure 12.2. Along with the application framework, MIDP also provides basic facilities for controlling the display, polling the keypad, rendering 2D graphics and text, accessing the network, play- ing back audio, and so on. A contemporary mobile handset has a dozen or so built-in APIs that are standardized under various Java Specification Requests. The most widely available ones include the Mobile Media API (JSR 135), the Wireless Messaging API (JSR 120), and of course M3G. Some devices also include vendor-specific packages, residing in namespaces other than the standard javax.microedition. For example, the fairly widespread Nokia UI API resides in the com.nokia.mid.ui package. Going back to Figure 12.1 and proceeding to the bottom layer there, we first encounter OpenGL ES. As discussed in Section 5.2.4, M3G is conceptually based on OpenGL ES, PAUSED ACTIVE DESTROYED new startApp pauseApp destroyApp destroyApp Figure 12.2: The life cycle of an application in Java MIDP. The application may be in one of three states: paused, active, or destroyed. State transitions are controlled by the application framework, and signaled to the midlet via the three event handlers shown in the diagram. All midlets must implement these event handlers, as well as a constructor; some of the methods may be empty, though. The midlet is responsible for acquiring and releasing its resources as appropriate upon each event. 272 INTRODUCING M3G CHAPTER 12 but some implementations use tailor-made software rasterizers instead. All hardware- accelerated devices are probably using OpenGL ES, though. Finally, at the bottom right of Figure 12.1, we have the Java Virtual Machine (VM) with its core libraries. The core libraries are defined in CLDC (Connected Limited Device Con- figuration) for typical devices, or the less limited CDC on some high-end devices. There are other flavors of mobile Java than the one presented here, mainly in Japan, but the CLDC/MIDP combination is so dominant that we will use it as a synonym to Java Micro Edition in this book. Besides, M3G has been deployed on all variants of mobile Java, as well as on desktop Java, and it works the same way on all of them. Compared to desktop Java, some of the most important differences in mobile Java (CLDC/MIDP) are the lack of the Java Native Interface (JNI), lack of dynamic class load- ing, and limited built-in libr aries. These restrictions are in place to help guarantee secu- rity and to reduce hardware requirements. As a result, you cannot include native code with your midlet, load and unload classes to optimize memory use, or load classes over the network to dynamically extend your application. You must also implement some basic things like inverse trigonometric functions in your own code. Appendix B provides further information on the inner workings of Java virtual machines. 12.1.2 FEATURES AND STRUCTURE M3G can be thought of as an object-oriented interface to OpenGL ES at the low level, and as a link to digital content creation tools—such as 3ds Max or Maya from Autodesk, 1 Softimage, 2 or the freely available Blender 3 —at the high level. Figure 12.3 shows the class diagram of M3G. All the classes are defined in the javax. microedition.m3g package. We w ill refer to this figure in the later chapters as we discuss each class in detail. The base class of the hierarchy is Object3D; all objects that can be rendered or be part of a scene graph are derived from it. These objects are col- lectively known as scene g raph objects, and they form the bulk of the API. There are only four classes that are not derived from Object3D: Graphics3D takes care of all ren- dering; Loader is for importing art assets and scenes from files or over the network; Transform represents a generic 4 × 4 matrix; and RayIntersection is used for picking objects in the scene. At its foundation, M3G wraps coherent blocks of OpenGL ES state into retained-mode objects that are controlled by the M3G engine, and can thus be stored and processed completely in native code (see Section 5.3). Classes that can be considered simple wr ap- pers for OpenGL concepts are indicated by the dashed outline in Figure 12.3. Nearly all 1 www.autodesk.com 2 www.softimage.com 3 www.blender.org SECTION 12.1 OVERVIEW 273 Object3D Material Fog Texture2D PolygonMode Background Compositing Mode Image2D Transformable Graphics3D Loader Transform RayInter section Node Group World Mesh Skinned Mesh Morphing Mesh Animation Track Animation Controller Keyframe Sequence Appearance VertexArray IndexBuffer Triangle StripArray Sprite3D Camera Light VertexBuffer Figure 12.3: The M3G class hierarchy consists of 30 classes, all but four of them derived from Object3D. Classes that are simple wrappers for OpenGL ES and EGL functionality are demarcated by the dashed line. The other classes provide capabilities that are beyond the scope of OpenGL, such as object and scene representation, keyframe animation, and content loading. features of OpenGL ES 1.0 are available through M3G, although a few were abstracted into a simplified form, e.g., blending and depth testing, and certain rarely used features were dropped altogether, e.g., logic ops, points, and lines. Also, to provide developers with a less fragmented platform, anything that is optional in OpenGL ES or poorly supported in hardware was left out, e.g., stencil buffer ing. Refer back to Figure 3.1 to see how the rendering pipeline differs from that of OpenGL ES 1.1. Building on the core rendering features, M3G defines a scene graph where the retained- mode components can be linked with each other to form complete objects, groups of objects, and ultimately an entire scene. M3G supports the types of scene graph nodes that one would expect to find in a scene graph API, including Camera, Light, Group , and a basic rigid-body Mesh. In addition, there are two deformable variants of Mesh: the SkinnedMesh that is animated by a bone hierarchy, and the MorphingMesh that is deformed by linear blending of morph targets. Figure 12.4 illustrates how these and other high-level features relate to the OpenGL ES vertex pipeline. The scene graph nodes also include Sprite3D, which is useful for 2D billboards and overlays, and World, which is the scene graph root. A simple example of a scene graph is shown in Figure 12.5. To keep the scene graph lightweight and uncomplicated, M3G does not include explicit support for terrains, shadows, portals, particles, and other advanced high-level features. . site) takes care of all the non 3D- related code. 12.1 OVERVIEW The Mobile 3D Graphics API provides Java programmers with an efficient and compact interface for managing and rendering 3D scenes. It. only four classes that are not derived from Object3D: Graphics3 D takes care of all ren- dering; Loader is for importing art assets and scenes from files or over the network; Transform represents a generic. of the test run (%) Figure 11.2: Duration with frame rate capped to 15 FPS but with different features enabled. 2 = textured and lit, 3 = textured with mipmaps and lit, 4 = nontextured and lit,