194 OPENGL ES TRANSFORMATION AND LIGHTING CHAPTER 8 /* Scene ambient light and single-sided lighting */ glLightModelfv( GL_LIGHT_MODEL_AMBIENT, dark_red ); glLightModelf( GL_LIGHT_MODEL_TWO_SIDE, GL_FALSE ); /* Set up the directional light */ glLightfv( GL_LIGHT0, GL_POSITION, dir_light ); glLightfv( GL_LIGHT0, GL_AMBIENT, dark_gray ); glLightfv( GL_LIGHT0, GL_DIFFUSE, white ); glLightfv( GL_LIGHT0, GL_SPECULAR, red_transp ) glLightf( GL_LIGHT0, GL_CONSTANT_ATTENUATION, 1.f ); glLightf( GL_LIGHT0, GL_LINEAR_ATTENUATION, 0.f ); glLightf( GL_LIGHT0, GL_QUADRATIC_ATTENUATION, 0.f ); glLightf( GL_LIGHT0, GL_SPOT_CUTOFF, 180.f ); /* Set up the spot light */ glLightfv( GL_LIGHT1, GL_POSITION, spot_light ); glLightfv( GL_LIGHT1, GL_AMBIENT, white ); glLightfv( GL_LIGHT1, GL_DIFFUSE, white ); glLightfv( GL_LIGHT1, GL_SPECULAR, white ) glLightf( GL_LIGHT1, GL_CONSTANT_ATTENUATION, 0.f ); glLightf( GL_LIGHT1, GL_LINEAR_ATTENUATION, 1.f ); glLightf( GL_LIGHT1, GL_QUADRATIC_ATTENUATION, 0.f ); glLightf( GL_LIGHT1, GL_SPOT_CUTOFF, 40.f ); glLightf( GL_LIGHT1, GL_SPOT_EXPONENT, 10.f ); glLightfv( GL_LIGHT1, GL_SPOT_DIRECTION, spot_dir ); /* Set up materials */ glMaterialfv( GL_FRONT_AND_BACK, GL_AMBIENT, blueish ); glMaterialfv( GL_FRONT_AND_BACK, GL_DIFFUSE, blueish ); glMaterialfv( GL_FRONT_AND_BACK, GL_SPECULAR, red_transp ); glMaterialfv( GL_FRONT_AND_BACK, GL_EMISSION, black ); glMaterialf( GL_FRONT_AND_BACK, GL_SHININESS, 15.f ); /* Don’t forget to normalize normals! Faster if already normalized. */ glDisable( GL_NORMALIZE ); glEnable( GL_LIGHT0 ); glEnable( GL_LIGHT1 ); glEnable( GL_LIGHTING ); /* We use normalized coordinates -> can be used also as normals */ glVertexPointer( 3, GL_FIXED, 0, vertices ); glNormalPointer( GL_FIXED, 0, vertices ); glEnableClientState( GL_VERTEX_ARRAY ); glEnableClientState( GL_NORMAL_ARRAY ); glDrawArrays( GL_TRIANGLES, 0, 512*3 ); 9 CHAPTER OPENGL ES RASTERIZATION AND FRAGMENT PROCESSING This chapter covers ever ything that happens after the transformation and lighting pipeline. First the primitives are clipped and culled, and the surviving ones are raster- ized into frag ments. Clipping has already been described in Section 3.3 and will not be repeated here. Texture-mapping, if enabled, is applied to each fragment, and the rest of the fragment pipeline is executed: fog and antialiasing, followed by the alpha, depth, and sten- cil tests. Finally, the various buffers (color, depth, stencil) are updated. The color buffer updates may use blending or logical operations. 9.1 BACK-FACE CULLING Back-face culling is used to discard triangles that are facing away from the viewer. This is a useful optimization as roughly half of the triangles of an opaque closed mesh are hidden by the front-facing ones. Culling these early in the pipeline increases the render- ing performance. Culling is controlled globally by glEnable and glDisable using GL_CULL_FACE as the argument. Culling affects only triangles; points and lines are never back-face culled. The user may select which face should be culled by calling void glCullFace(GLenum mode) 195 196 OPENGL ES RASTERIZATION AND FRAGMENT PROCESSING CHAPTER 9 with either GL_FRONT, GL_BACK,orGL_FRONT_AND_BACK as the argument. The last token culls all triangles. Which side of a triangle is considered to be the front is defined using glFrontFace, described in Section 8.3.3. Culling is conceptually performed during the triangle setup, just prior to rasterization. However, implementations may choose to do this already earlier in the pipeline, and potentially be able to skip, e.g., lighting calculations, for the culled triangles. By default culling is disabled. The following example enables culling for the clockwise- oriented faces: glFrontFace( GL_CCW ); glEnable( GL_CULL_FACE ); glCullFace( GL_BACK ); 9.2 TEXTURE MAPPING Texture mapping plays a fundamental part in the OpenGL ES rendering pipeline. Although the texturing model is slightly simplified from the desktop, it is still a very powerful mechanism that allows many interesting effects. Texturing is conceptually per- formed during rasterization and prior to the rest of the fragment pipeline. However, some implementations may internally postpone it until after the depth and stencil tests to avoid texturing fragments that would be discarded by these tests. Texturing for the currently active texture unit is enabled or disabled with the GL_TEXTURE_2D flag. 9.2.1 TEXTURE OBJECTS Texture maps are stored in texture objects. The idea is that you first create a texture object, and then set up the various values that relate to the texture, e.g., the bitmap image to use, or the filtering and blending modes. Finally, just before drawing the primitives, you activate the relevant texture object. Eachtextureobjectisreferredbyatexture name which acts as a handle for texture data and state. The name can be any positive integer (zero is reserved and refers to the default texture). You can ask the driver to provide you a list of unused names with void glGenTextures(GLsizei n, GLuint * textures) which returns n new names, stored in the array textures.Tocreateanewtextureobject, or to reactivate a previously created one, call void glBindTexture(GL_TEXTURE_2D, GLuint texture) where texture is the name of the texture object. In desktop OpenGL other targets, such as 1D and 3D textures are possible, but OpenGL ES only supports two-dimensional textures. SECTION 9.2 TEXTURE MAPPING 197 When a texture map is no longer needed, the resources consumed by it should be freed. This is done by a call to void glDeleteTextures(GLsizei n, const GLuint * textures) which deletes the n textures in the array textures. If any of these textures is currently bound to one of the texturing units, that unit will have the default texture object (texture 0) assigned to it. Note that in a multi-context environment where different GL contexts share textures, the resources are freed only when a texture is not actively used in any of the contexts. As a summary, the following code shows a typical pattern of how texture objects are used: GLuint tex_handle[2]; glGenTextures( 2, &tex_handle[0] ); /* set up the textures */ glBindTexture( GL_TEXTURE_2D, tex_handle[0] ); /* specify texture data, filtering, etc. */ glTexImage2D( ); glBindTexture( GL_TEXTURE_2D, tex_handle[1] ); glTexImage2D( ); /* now ready to draw, reactivate one of the objects */ glEnable( GL_TEXTURE_2D ); glBindTexture( GL_TEXTURE_2D, tex_handle[1] ); glDrawArrays( ); glDisable( GL_TEXTURE_2D ); /* release the resources when not needed any longer */ glDeleteTextures( 2, &tex_handle[0] ); 9.2.2 SPECIFYING TEXTURE DATA In OpenGL ES the texture image data is managed by the server, which means that the data needs to be copied to it from the client. At this time the image data is usually converted into the internal format best suited for texture mapping. Since both the copy and conversion operations take time, a texture should be created once and used multiple times before being deleted. void glTexImage2D(GL_TEXTURE_2D, GLint level, GLenum internalformat, GLsizei width, GLsizei height, GLint border, GLenum format, GLenum type, const GLvoid * pixels) copies texture image data from client-side memory to the server-side texture object. If mipmapping is not used, or this image is the base mipmap level, level should be zero (see Section 9.2.3 for mipmapping and levels). Since OpenGL ES does not support texture borders, the value of border must be 0. The dimensions of the texture, in pixels, are given by width and height, and they have to be powers of two (1, 2, 4, , 512, 1024, ) but 198 OPENGL ES RASTERIZATION AND FRAGMENT PROCESSING CHAPTER 9 they do not have to be the same (for example 32 × 64 is a valid size). The two format parameters, internalformat and format, must be the same, and they must be one of the formats in the table below. The table also lists the data types that can be matched with the formats. The numbers that are part of a token tell how many bits are allocated to each of the R, G, B, and A channels. pixels contains a pointer to the data, the first data row corresponds to the bottom texel row. Only byte-based data types work the same way on all platforms, as short-based data types are interpreted according to the native platform endianess. Short-based texture data is accessed by the GL implementation as if it were accessed through a native short integer pointer. If the texture data comes from a data file that was stored using the other endi- aness, texture data must be byte-swapped before any texture upload functions are used. See Table 9.2. If pixels is NULL, the server will reserve memory to hold the image data but no image data is copied. The data can be subsequently loaded by using void glTexSubImage2D(GL_TEXTURE_2D, GLint level, GLint xoffset, GLint yoffset, GLsizei width, GLsizei height, GLenum format, GLenum type, const GLvoid * pixels) which updates a subimage within a previously defined texture image. Parameters level, format, type, and pixels are the same as for glTexImage2D, and the format needs to Table 9.1: Texture formats and types. Texture Format Data type GL_LUMINANCE GL_UNSIGNED_BYTE GL_ALPHA GL_UNSIGNED_BYTE GL_LUMINANCE_ALPHA GL_UNSIGNED_BYTE GL_RGB GL_UNSIGNED_BYTE GL_UNSIGNED_SHORT_5_6_5 GL_RGBA GL_UNSIGNED_BYTE GL_UNSIGNED_SHORT_4_4_4_4 GL_UNSIGNED_SHORT_5_5_5_1 Table 9.2: Endianess in a data file. Byte Offset 0 1 Texel 0 0 Big Endian HI: R 4 R 3 R 2 R 1 R 0 G 5 G 4 G 3 LO: G 2 G 1 G 0 B 4 B 3 B 2 B 1 B 0 Little Endian LO: G 2 G 1 G 0 B 4 B 3 B 2 B 1 B 0 HI: R 4 R 3 R 2 R 1 R 0 G 5 G 4 G 3 SECTION 9.2 TEXTURE MAPPING 199 match the original format of the texture map. The lower left corner is at (xoffset, yoffset) for the width × height subimage. The pixel data is copied from the memory block pointed by pixels. By default each row of pixels must be aligned to a word boundary, i.e., the alignment must be to an even 4 bytes. The alignment can be changed by void glPixelStorei(GL_UNPACK_ALIGNMENT, GLint param) where the allowed values for param are 1 (byte-alignment), 2 (rows aligned to even- numbered by tes), 4 (word alignment), and 8 (rows start on double-word boundaries). If your pixel data is continuous, setting the alignment to 1 will always work, but may be slower for unaligned image data. As OpenGL ES only supports power-of-two texture maps and there is no support for GL_PACK_ROW_LENGTH and GL_PACK_SKIP_PIXELS for glPixelStorei, there is no way to copy a general image that does not have power-of-two dimensions into a texture directly. There are two ways around this problem. The first one is to allo- cate a memory buffer and to copy power-of-two subpieces of image data into it, then call glTexSubImage2D to construct the texture from these pieces. Another way is to allo- cate the next larger power-of-two buffer and to copy the original image into a subregion of this buffer, then load the texture data using a single call to glTexImage2D. Which one is faster depends on the texture upload performance and size of the texture, e.g., if the source image size is 260 × 260, the next power of two size is 512 × 512 which would almost quadr uple the amount of data to be transferred. Copying from the frame buffer If you first render an image that you would like to use as a texture map, you can copy it directly from the frame buffer. Since both the texture maps and the frame buffer reside on the server, i.e., the graphics subsystem, doing the whole operation on the server can be much more efficient than reading the color buffer to the client and then copying the data back to the server using glTexImage2D. void glCopyTexImage2D(GL_TEXTURE_2D, GLint level, GLenum internalformat, GLint x, GLint y, GLsizei width, GLsizei height, GLint border) copies a width × height block of pixels with a lower left corner at (x, y) in the color buffer into the currently bound texture. The level, internalfor mat, and border arguments are iden- tical to those of glTexImage. The parameter internalformat must be compatible with the color buffer format according to Table 9.3. Note, however, that glCopyTexImage2D has to flush the graphics pipeline and com- plete all previous graphics calls, so calling it in the middle of a rendering pass may have a negative performance impact. 200 OPENGL ES RASTERIZATION AND FRAGMENT PROCESSING CHAPTER 9 Table 9.3: Texture formats compatible with the color buffer formats. Color Texture Format Buffer GL_ALPHA GL_LUMINANCE GL_LUMINANCE_ALPHA GL_RGB GL_RGBA A √ ———— L— √ ——— LA √√ √ —— RGB — √ — √ — RGBA √√ √ √√ void glCopyTexSubImage2D(GL_TEXTURE_2D, GLint level, GLint xoffset, GLint yoffset , GLint x, GLint y, GLsizei width, GLsizei height) is a variant that takes a screen-aligned width ×height pixel rectangle with lower left corner at (x, y), and replaces a block of the same size in the texture map, with lower left corner at (xoffset, yoffset). Compressed texture formats In order to save texture memory, OpenGL ES supports the concept of compressed tex- ture formats. Currently the only supported format uses paletted textures, which contain a palette of colors in the header, followed by a sequence of indices into the palette, one index for each texel. The indices can either use 4-bit indexing for 16 colors, or 8-bit indexing for 256 colors. The palette entries can be either RGB colors stored in 888 or 565 formats, or RGBA colors stored using 8888, 4444, or 5551 formats. void glCompressedTexImage2D(GL_TEXTURE_2D, GLint level, GLenum internalformat, GLsizei width, GLsizei height, GLint border, GLsizei imageSize, const GLvoid * data) is similar to glTexImage2D except that imageSize gives the length of data, the com- pressed client-side image data. inter nalformat indicates the format of the compressed data and has to be one of GL_PALETTE4_RGB8_OES, GL_PALETTE8_RGB8_OES, GL_PALETTE4_R5_G6_B5_OES, GL_PALETTE8_R5_G6_B5_OES, GL_PALETTE4_RGBA8_OES, GL_PALETTE8_RGBA8_OES, GL_PALETTE4_RGBA4_OES, GL_PALETTE8_RGBA4_OES, GL_PALETTE4_RGB5_A1_OES, GL_PALETTE8_RGB5_A1_OES. The level may be zero if the texture contains only the base level. A negative number indi- cates the number of mipmap levels in the texture (see the next section about texture filtering). SECTION 9.2 TEXTURE MAPPING 201 Figure 9.1: Level 0 of the texture in a grid. An implementation may support other, proprietary compressed texture formats as exten- sions. Those formats may compress better and provide more colors than paletted textures, but they are less portable as they are not universally supported. The specification defines an entry point for void glCompressedTexSubImage2D(GL_TEXTURE_2D, GLint level, GLint xoffset, GLint yoffset, GLsizei width, GLsizei height, GLenum format, GLsizei imageSize, const GLvoid * pixels ), but currently there are no Khronos-specified formats that support this. The reason is that most implementations expand the paletted texture immediately (and possibly recompress using a proprietary internal format), and being able to update subimages would require also storing the palette and format, creating additional memory overhead. The following example creates a paletted texture. The colors are 32-bit RGBA, 16 of them are stored in the beginning of the texture map. The base mipmap level is 8 × 8 texels, so the palette is followed by 8 · 8/2 = 32 bytes of 4-bit indices. The PALTEX macro packs two such indices into a single unsigned byte. The texture contains also 3 mipmap levels (4 ×4, 2 ×2, and 1 ×1). The decompressed texture looks like that in Figure 9.1. For more details about mipmapping, see the next section. #define PALTEX(left,right) ((left << 4) | (right)) static const GLubyte palette_texture[] = { /* 16-entry palette with 32bpp colors */ 0, 0, 0,255, 10,10,10,255, 20,20,20,255, 30,30,30,255, 40,40,40,255, 50,50,50,255, 60,60,60,255, 70,70,70,255, 202 OPENGL ES RASTERIZATION AND FRAGMENT PROCESSING CHAPTER 9 80,80,80,255, 90,90,90,255, 100,100,100,255, 110,110,110,255, 120,120,120,255, 130,130,130,255, 140,140,140,255, 150,150,150,255, /* mipmap level 0 (base) is (8x8), one palette index is 4 bits */ PALTEX(0,2), PALTEX(4,6), PALTEX(8,10), PALTEX(12,14), PALTEX(0,2), PALTEX(4,6), PALTEX(8,10), PALTEX(12,14), PALTEX(0,2), PALTEX(4,6), PALTEX(8,10), PALTEX(12,14), PALTEX(0,2), PALTEX(4,6), PALTEX(8,10), PALTEX(12,14), PALTEX(0,2), PALTEX(4,6), PALTEX(8,10), PALTEX(12,14), PALTEX(0,2), PALTEX(4,6), PALTEX(8,10), PALTEX(12,14), PALTEX(0,2), PALTEX(4,6), PALTEX(8,10), PALTEX(12,14), PALTEX(0,2), PALTEX(4,6), PALTEX(8,10), PALTEX(12,14), /* mipmap level 1 is 4x4 */ PALTEX(1,5), PALTEX(9,13), PALTEX(1,5), PALTEX(9,13), PALTEX(1,5), PALTEX(9,13), PALTEX(1,5), PALTEX(9,13), /* mipmap level 2 is 2x2 */ PALTEX(3,11), PALTEX(3,11), /* the last mipmap level (3) is 1x1 */ PALTEX(7,7) }; /************************************************** * Prepare compressed texture. * |level|+1 is the number of mipmap levels. * Here we have: | — 3|+1 = 4 levels. ***************************************************/ glGenTextures( 1, &texture_handle ); glBindTexture( GL_TEXTURE_2D, texture_handle ); glCompressedTexImage2D( GL_TEXTURE_2D, — 3, GL_PALETTE4_RGBA8_OES, 8, 8, 0, sizeof(palette_texture), palette_texture ); 9.2.3 TEXTURE FILTERING The basic ideas in texture filtering were introduced in Section 3.4.1 and illustrated in Figure 3.13. The texture coordinates are interpolated for each fragment within a primitive. The coordinates are interpreted so that 0 at the first component (s) maps to the left side of the first texel on a given row of the texture map, and 1.0 maps to the right side of the last texel on that row. The row is determined by the t-coordinate, 0 corresponding to the bottom of the bottom row, and 1.0 to the top of the top row. The texture fetch and filtering machinery then has to come up w ith a filtered color value from the texture map. SECTION 9.2 TEXTURE MAPPING 203 Basic filtering modes The basic filtering choices are point sampling and linear interpolation. Point sampling simply returns the value of the texel nearest to the inter polated texture coordinate, while linear interpolation takes a weighted average of the neighboring texel values. These filter- ing modes can be set separately for magnification where one texel maps to several pixels and for minification where several texels map to a single pixel. They can be set by calling void glTexParameter{ifx}( GL_TEXTURE_2D, GLenum pname, T param) void glTexParameter{ifx}v( GL_TEXTURE_2D, GLenum pname, const T * param) where pname is either GL_TEXTURE_MAG_FILTER or GL_TEXTURE_MIN_FILTER and param is either GL_NEAREST or GL_LINEAR. Mipmap specification If a drastic minification is required, that is, a large number of texels would project to a single pixel, neither of those sampling approaches works well. The selection of which texel (or the interpolation of which four texels) is used would essentially be random. This would create both v i sual artifacts and result in inefficient memory access patterns. Mipmapping provides a solution by providing prefiltered texture maps that can be chosen so that the pixel-texel size ratio is sufficiently close to one. There are three ways of specifying mipmaps: give them one level at a time for regular textures, ask the system to automatically generate them (from OpenGL ES 1.1 onward), or provide them all at once for compressed textures. If the levels are given one at a time, they are given with glTexImage2D and other related commands using the parameter level. The base level is zero, while level 1 needs to be half the size both in width and height, unless one of them is already 1. As an example, for a 64 × 32 texture the level 1 mipmap is 32 × 16,level2is16 × 8, and so on until level5is2 × 1 and the final level 6 is a 1 × 1 texture. The texture will not be complete until all mipmaps have been given, they have correct sizes as described above, and the texture formats of the different levels match. Incomplete texture behaves as if texturing was disabled for the texture units where the texture is bound. OpenGL ES 1.1 supports automatic generation of mipmap levels. The levels are typically obtained by averaging four texels at a finer level to create one texel at a coarser level. Auto- matic mipmap level generation is not enabled by default, but can be enabled with glTexParameteri( GL_TEXTURE_2D, GL_GENERATE_MIPMAP, GL_TRUE ); and disabled with glTexParameteri( GL_TEXTURE_2D, GL_GENERATE_MIPMAP, GL_FALSE ); When automatic mipmap generation is activated, the server automatically recomputes the contents of all mipmap levels whenever the base level is updated. . the name of the texture object. In desktop OpenGL other targets, such as 1D and 3D textures are possible, but OpenGL ES only supports two-dimensional textures. SECTION 9.2 TEXTURE MAPPING 197 When. resources consumed by it should be freed. This is done by a call to void glDeleteTextures(GLsizei n, const GLuint * textures) which deletes the n textures in the array textures. If any of these. GL_PALETTE8_RGB8_OES, GL_PALETTE4_R5_G6_B5_OES, GL_PALETTE8_R5_G6_B5_OES, GL_PALETTE4_RGBA8_OES, GL_PALETTE8_RGBA8_OES, GL_PALETTE4_RGBA4_OES, GL_PALETTE8_RGBA4_OES, GL_PALETTE4_RGB5_A1_OES, GL_PALETTE8_RGB5_A1_OES. The