314 BASIC M3G CONCEPTS CHAPTER 13 own exporters, viewers, or optimization tools for M3G content, or simply need to debug an existing file. The simplicity is achieved by four primary means: First, there is a one-to-one mapping between object types in the file format and Java classes in the run-time API. Second, there are no forward references, so objects can only refer to objects that reside earlier in the file. Third, compression is based on the widely available zlib, rather than specialized encod- ings for different data types, or complex algorithms such as arithmetic coding. Finally, the Loader has an all-or-nothing policy: a file is either loaded completely or not at all. No attempt is made to recover from errors. We will now go through the structure of the file format, shown at a high level in Figure 13.2. To shed light on individual details, we will refer to the example file in Figure 13.3. Note that what follows is not intended to be an exhaustive description or a replacement for the specification, but only to give you an idea of how things are laid out. Having read this, you should be able to more quickly dig up whatever details you need in the spec. File structure M3G files are easy to recognize in a text editor by their file identifier, <<JSR184>>, located at the very beginning. To be exact, the identifier consists of that string and a few special characters to quickly catch file transmission errors, for a total of twelve bytes (in hexadecimal): AB 4A 53 52 31 38 34 BB 0D 0A 1A 0A. File Identifier File Section Object Data Header Section Scene Section n Compressed: {0, 1} Total Section Length Uncompressed Length Object 0 Object n Checksum Object Type (SkinnedMesh = 16) Length Data Object3D Transformable Node Mesh SkinnedMesh XREF Section Scene Section 0 Figure 13.2: M3G files are divided into sections, each containing one or more objects, which are further divided into fields. The fields are laid out in the order of class inheritance; for example, a SkinnedMesh object is derived from Object3D, Transformable, Node, and Mesh, as shown in Figure 12.3. SECTION 13.5 IMPORTING CONTENT 315 ‘«’ ‘4’‘J’ ‘S’ ‘R’ ‘1’ ‘8’ ‘»’ 0x0D 0x0A 0x1A 0x0A 0 TotalSectionLength=30 0 ObjectLength=12UncompressedLength=30 ApproximateSize=16612 null Section Checksum 0 TotalSectionLength=26 255 ObjectLength=8UncompressedLength=26 Section Checksum 0 TotalSectionLength=44 17 ObjectLength=26 UserID=0x1234 UncompressedLength=44 false false ObjectIndex=2 AnimationTrackCount=0 UserParameterCount=0 228 241 241 210 2100 0 0 1 0 true TotalFileSize=112 ‘/’ ‘b’ ‘g’ ‘.’ ‘p’ ‘n’ ‘g’ null Section Checksum Object3D fields Transformable fields Texture2D fields ObjectType 0: Header File Identifier Header Section XREF Section Scene Section ObjectType 255: XREF ObjectType 17: Texture2D Blend mode: REPLACE Wrap mode: (REPEAT, REPEAT) Blend color: (0, 0, 0) Filter mode: (NEAREST, NEAREST) Figure 13.3: The complete contents of our example file, byte by byte. There is one root object in the file, a Texture2D, and one non-root object, an Image2D that is loaded from the external file /bg.png. All sections are uncompressed. Object data are shown in gray boxes, section data in white. Beyond the file identifier, M3G files are divided into s ections that contain one or more objects each. Sections may be individually compressed with zlib; this allows you to selectively compress the sections for which it makes most sense. Each section also has an Adler32 checksum (provided by zlib)toallowtheLoader to quickly reject a corrupt file without parsing it further. Note that the loader will make no attempt to recover the contents of a damaged file, but will simply throw an IOException. There are three kinds of sections. The header section must be present in every file, must be uncompressed, and must be located right after the file identifier. The external refer- ence section is optional, but must immediately follow the header section if present. The rest of the file is composed of an arbitrary number of scene sections. Any legal M3G file must have at least one section besides the header, and must not have any empty sections. 316 BASIC M3G CONCEPTS CHAPTER 13 Data types The file format has data t ypes corresponding to the built-in boolean, byte, short, int, and float types in Java. Booleans are encoded as a single byte, such that 1 indicates true, 0 indicates false , and other values are disallowed. The integer and float types are stored so that the least significant byte comes first. Besides the basic types, M3G files may also contain null-terminated strings, 3-element floating-point vectors, 4 × 4 matrices, RGB and RGBA colors encoded at 8 bits per color component, references to other objects, and arbitrary-length arrays of any basic or com- pound type. Object references Upon loading, objects are read sequentially from the beginning of the file and assigned a running number as their index. The first object in the file, which is a special header object, gets the index one, the first actual scene object gets the index two, and so on. The index zero is reserved for null references. Objects can only reference other objects that reside at lower indices, i.e., those that have already been imported. This is to guarantee that the Loader can parse any M3G file from start to finish in one pass, and also to allow it to type-check the references immedi- ately. Note that the lack of forward references means that a scene graph loaded from a file can never have cycles, although they are permitted in the runtime scene graph for node alignment; see Section 15.3.2. Header section The header section contains exactly one object, the headerobject, which cannot be present anywhere else in the file. As shown in Figure 13.3, the header object begins with a two-byte version number, identifying variants of the file format. The only legal version number at this time is 1.0. Note that the file format does not necessarily have the same version number as the M3G API. Following the version number is a boolean flag telling whether the external reference sec- tion is present (in our example, it is). The header object also stores the total size of the file, both with and without the external references. The size that includes the external refer- ences is regarded to be a hint and need not be accurate, so as to allow the referenced files to be modified independently of the root file. In our example, the root file is 112 bytes (exactly), and the externally referenced PNG is (estimated to be) 16500 by tes. The final item in the header object is the AuthoringField where the authoring tool or the author herself may store an arbitrary null-terminated string of text, such as a copyright notice. In our example the field is left empty, containing just the terminating null. SECTION 13.5 IMPORTING CONTENT 317 External reference section The external reference section stores one or more external references, or XREFs for short. External references allow you to build up your scene from a collection of separate M3G and image files. Images are typically included by reference rather than embedded into the host file, because dedicated image formats provide better compression than plain zlib. A minor disadvantage with external image files is that they have n o user ID or user parameters. External references are simply null-terminated strings pointing at named resources, such as network URLs or files in the JAR package. Each external reference yields exactly one root-level Object3D. Our example file in Figure 13.3 has just one XREF, pointing at /bg.png in the JAR package. It will be imported as a single Image2D. M3G files may reference an arbitrary number of other M3G files, which in turn may freely reference another set of M3G files, and so on, but the references are not allowed to form a cycle. Also, if you intend an M3G file to be referenced from another, make sure that it only has one root object. If there are many root objects, the Loader w ill pick only the first one and discard the rest. Scene sections Scene sections store the actual scene graph—or several scene graphs. Each scene sec- tion can contain one or more objects, and again each object corresponds to exactly one Object3D in the runtime API. The contents of each object are generally the same as that of the corresponding class in the API. In our example file, there is one scene section, containing a single Texture2D. The base class data for Object3D comes first, followed by the other base class Transformable. The data for the concrete class is stored last. This is the case with all ty pes of objects in the file format. For simplicity, we have no animation tracks or user parameters in our example, and no texture matrix in Transformable. The two false entries in Transformable indicate that the TRScomponents as well as the M component will assume their default values, i.e., the identity matrix. The fields of the Texture2D object itself are pretty obvious. The main thing to note is that the image in /bg.png must have power-of- two dimensions. Also note that the Image2D is assigned the object index 2, because the header object always gets the index one, and zero is reser ved for null. Special compression We said in the beginning of this section that there are no special compression formats for different data types in the M3G file format, just zlib for everything, but that is 318 BASIC M3G CONCEPTS CHAPTER 13 not st rictly true. The VertexArray and KeyframeSequence classes do in fact have special encodings as an option. However, the encodings are almost trivial. Vertex arrays—including colors, normals, texture coordinates and vertex positions—can be compressed with delta encoding. This means that each vertex attribute is stored as a dif- ference vector relative to the previous value. The differences are unsigned, so for exam- ple a difference of −2 is encoded as 254 (in case of a byte array) or 65534 (in case of a short array). Thus, the deltas take up the same number of bits as the r aw integers (8 or 16 bits per component), making the whole encoding seem rather pointless. How- ever, the deltas tend to have fewer significant bits, causing the same bit patterns to repeat more often across the array. This, in turn, allows zlib to compress the array more effi- ciently. Note that delta encoding and zlib are both lossless. Keyframes, which are 32-bit float values in raw format, can be encoded by quantizing them to 16-bit or 8-bit integers which are then scaled and offset using a bias value. The quantized keyframes consume only a half or a quarter of the file size compared to the raw format, and that is further reduced by zlib. Floating-point bit patterns, on the other hand, are poorly compressed by zlib. 14 CHAPTER LOW-LEVEL MODELING IN M3G M3G builds upon the common low-level concepts set forth in Chapter 3. It offers most of the same functionality that OpenGL ES provides for native applications, but with an object-oriented Javainterface. Some features are slightly more abstracted in order to reduce API complexity, but the underlying rendering pipeline, be that implemented in software or hardware, can be shared with OpenGL ES. Also, while familiarity with OpenGL ES is not a prerequisite for understanding M3G, existing knowledge on OpenGL ES will not be wasted on M3G. In this chapter, we walk through the lowest-level parts of M3G. By the end of the chapter, you will know how to use M3G to draw polygon meshes in immediate mode, similarly to OpenGL ES. The components discussed here will also serve as building blocks for the higher-level functions which are covered in the following chapters. 14.1 BUILDING MESHES Meshes in M3G are built out of vertex array and buffer objects, triangle buffer objects, and shading parameters specified in various rendering state objects. 14.1.1 VertexArray Low-level modeling begins by defining your vertex data. The VertexArray class stores an array of vectors that can then be used for any per-vertex data: positions, normals, 319 320 LOW-LEVEL MODELING IN M3G CHAPTER 14 colors, or texture coordinates. The class constructor is VertexArray(int numVertices, int numComponents, int componentSize), where the parameters are the number of vertices, number of components per vertex, and size of the data type used, respectively. componentSize is 1 for byte,and2forshort data. For a mesh with 100 vertices having vertex positions and colors only, for example, you could create two arrays: myPositions = new VertexArray(100, 3, 2); // 16-bit positions myColors = new VertexArray(100, 4, 1); // 8-bit RGBA colors Vertex data is loaded into the ar rays using the set function, which copies a range of vertex values from a byte or short array: void set(int firstVertex, int numVertices, byte[] values) void set(int firstVertex, int numVertices, short[] values) Pitfall: If you plan on reading data back from VertexArray, you may soon find that the get method for that is not included in M3G 1.0—it was one of the many getters dropped to minimize the API footprint. The get methods were added to VertexArray in M3G 1.1, but if you abso- lutely need equivalent functionality with M3G 1.0, it can be done using the Transform.transform method as described in Section 13.3. Even then, you will only get the vertex data in floating-point format, not the original 8-bit or 16-bit integers. Now, assume that you have your positions in a short array myPositionData and your colors in a byte arr ay myColorData. The arrays should have at least 300 and 400 elements, respectively. We can then load the data values for all 100 vertices into the previously created vertex arrays: myPositions.set(0, 100, myPositionData); myColors.set(0, 100, myColorData); M3G makes a copy of the data you load into a VertexArray,somyPositionData and myColorData can be discarded at this point. In fact, all data in M3G is stored internally—client-side arrays are only referenced when copying data from them. This allows M3G to internally organize the data in the most efficient way. 14.1.2 VertexBuffer Once you have the vertex arrays you need, they must be combined into a VertexBuffer to form the actual vertices. The constructor for VertexBuffer simply creates an empty set of vertices. The necessary component arrays are added using the setPositions, setNormals, setColors, and setTexCoords functions. Note that there are certain restrictions on what kind of vertex data you can use for SECTION 14.1 BUILDING MESHES 321 Table 14.1: Supported vertex array types in M3G (ticks), relative to OpenGL ES 1.1 (ticks and crosses). The grayed-out boxes indicate combinations that are supported in neither API. Byte Short Fixed Float 2D 3D 4D Vertices ✓✓ ✘ ✘✘✓✘ Texcoords ✓✓ ✘ ✘✓✓✘ Normals ✓✓ ✘ ✘ ✓ Colors ✓ ✘✘ ✓✓ each vertex component—those are summarized in Table 14.1. The setters for colors and normals are trivial, only taking in the array you wish to use for that vertex com- ponent. Normals are automatically normalized. For positions, however, additional scale and bias values must be supplied: void setPositions(VertexArray positions, float scale, float[] bias) Since M3G only supports 8- and 16-bit vertices, scale and bias let you map the quantized vertices into a wider floating-point domain. Before M3G uses any of your vertex data, each quantized vertex v i is converted into an equivalent floating-point vertex v i : v i = sv i + b (14.1) where s and b are the values of s cale and bias, respectively. This way, you can author your model in floating point, quantize the vertices to 16 or 8 bits, and still use the resulting VertexArray in M3G as you would the original data. The precision is sufficient for just about any model, while the memory usage is only a half or one-quarter of full floating-point vertices. M3G implementations are also made more efficient, as there is no need to implement a full floating-point vertex pipeline. Again, using our example arrays, let us set up a VertexBuffer: myVertices = new VertexBuffer(); myVertices.setColors(myColors); myVertices.setPositions(myPositions, 100.f / (1<<16), null); In this case, we set bias to null, signaling a default bias of (0, 0, 0).Thescale parameter scales our 16-bit position data so that the coordinates span 100 units in floating point domain. If the full 16-bit range is utilized in myPositions, our model therefore extends from −50 to +50 on each coordinate axis. Perfor mance tip: Always make use of the full range available for your vertex positions and texture coordinates. Quantize your floating-point coordinates so that the y fill the 8- or 16-bit numeric range optimally, then map the data back to floating point by using the scale and bias values. There is no additional runtime cost from doing this, but it will let you achieve the maximum precision possible. 322 LOW-LEVEL MODELING IN M3G CHAPTER 14 Texture coordinates take one more additional parameter, the index of the texturing unit: void setTexCoords(int index, VertexArray texcoords, float scale, float[] bias) When using multi-texturing, VertexBuffer must contain a set of texture coordi- nates for each texture unit. Of course, you can—and often will—also set the same VertexArray for each texture unit; no data replication is required. Arrays can also be shared between any number of vertex buffers, and nothing prevents you from, for exam- ple, using the same array for both vertex normals and texture coordinates. Perfor mance tip: Always prefer multi-texturing over multi-pass rendering. With multi- texturing, you get multiple layers of texture by only rendering the geometry once, whereas multi-pass rendering incurs the transformation and lighting overhead for each pass. Blending and other frame buffer processing will also add to this overhead. Vertex positions are the only piece of data that is required for all rendering. If you want to use lighting, your VertexBuffer will also need normal vectors, and we already mentioned that texture coordinates are required to apply a texture. For vertex colors, you have a choice of using either a per-vertex color array or a single color for the entire buffer, set using setDefaultColor. To construct a buffer where all the vertices are red, you can do: myVertices = new VertexBuffer(); myVertices.setPositions(myPositions, 100.f / (1<<16), null); myVertices.setDefaultColor(0xFF0000); Pitfall: M3G versions 1.0 and 1.1 specify slightly different error handling for vertex components, with M3G 1.0 being more strict. It throws exceptions for many cases of missing data, such as texturing w ithout valid texture coordinates. This was viewed as an extra burden on implementations, and in M3G 1.1 error checking was relaxed so that vertex normals and texture coordinates default to some undefined value if the respective arrays do not exist during rendering. As an application developer, you should be aware of the fact that you may not get an exception if you are missing a required piece of vertex data. Instead, your rendering results will simply be incorrect. The reason may be tricky to identify even if you know what to look for, so if you want to use lighting or texturing, remember to supply those normals or texture coordinates! Perfor mance tip: Note that you can use set at any time to modify the contents of a VertexArray, and you can also use the setters on VertexBuffer to change the arrays being used. Be aware, however, that this may not be cheap, as the implementation may have to recompute bounding volumes or other cached data that is dependent on SECTION 14.1 BUILDING MESHES 323 the vertex values. As a rule, create your vertex arrays and buffers during the initialization phase of your application and only modify their contents after that if you really need to. 14.1.3 IndexBuffer AND RENDERING PRIMITIVES Vertices alone do not let you render anything. You also need to specify the kind of prim- itives you want to draw. For this purpose, M3G has the abstract IndexBuffer class that is specialized for each type of primitive. With the primitive types, M3G takes rather an ascetic approach, as the only kind of primitive currently supported is a triangle strip (see Section 3.1.1). Comparing to OpenGL ES in Table 14.2, we see that this is quite a cut in features. Lines and points were dropped because they would have added a lot of complexity to support quite a few use cases; if necessary, they can be emulated with triangles, which is h ow most renderers implement them in any case. The reasoning behind supporting triangles as strips only was that they are an efficient primitive both for storing a shape and for rendering it, and most M3G content will come from authoring tools that can easily generate the strips. It was perceived that quite a bit of implementation complexity could be dropped this way. Looking back now, this was one of the decisions where features were a little too aggressively cut down in the effort to minimize complexity—having to use triangle strips instead of triangle lists is quite an annoyance when generating meshes in code, for example. The TriangleStripArray class lets you construct an array of several triangle strips. You have a choice of two flavors of strips: implicit and explicit. The former assumes that the source vertices are ordered sequentially in the ascending order of indices, and you only have to specify the star ting vertex and the length of each subsequent strip: TriangleStripArray(int firstVertex, int[] stripLengths) The number of entries in the stripLengths array gives the number of strips. For example, if stripLengths is {3, 4, 3}, the call TriangleStripArray(2, stripLengths) will create three strips with the indices {2, 3, 4}, {5, 6, 7, 8}, and {9, 10, 11}. This Table 14.2: Supported rendering primitives in M3G, relative to OpenGL ES 1.1. The grayed-out boxes indicate combinations that are supported in neither API. Byte Short Implicit Strip Fan List Triangles ✘✓ ✓ ✓✘✘ Lines ✘✘ ✘ ✘✘✘ Points ✘✘ ✘ ✘ Points sprites ✘✘ ✘ ✘ . indicates true, 0 indicates false , and other values are disallowed. The integer and float types are stored so that the least significant byte comes first. Besides the basic types, M3G files may also contain. software or hardware, can be shared with OpenGL ES. Also, while familiarity with OpenGL ES is not a prerequisite for understanding M3G, existing knowledge on OpenGL ES will not be wasted on M3G. In. header object also stores the total size of the file, both with and without the external references. The size that includes the external refer- ences is regarded to be a hint and need not be accurate,