154 PERFORMANCE AND SCALABILITY CHAPTER 6 Transferring a mobile application and associated data over the network is often both slow and expensive, and in many cases the network operators pose strict limits for the sizes of the applications. For these reasons it is important to ensure that the graphics assets do not consume any more space than necessary. Texture maps usually consume a lion’sshare of all the graphics data, so special attention should be paid to them. In general, textures should either be procedurally generated, or an efficient lossy compression scheme such as DCT or wavelet compression (as utilized by the JPEG and JPEG 2000 formats, respectively) should be applied to them. PART II OPENGL ES AND EGL This page intentionally left blank 7 CHAPTER INTRODUCING OPENGL ES This chapter covers the fundamentals of the OpenGL ES 1.x standard. First we take a brief look at its history and design principles, followed by an introduction to the resources available online. Then we examine the different profiles and versions of the API, give an overview of the different versions, and take a look at the main OpenGL ES conventions. 7.1 KHRONOS GROUP AND OPENGL ES The Khronos Group was formed in 2000 to create the specification of OpenML, an open standard media authoring API. The consortium later expanded to include new work- ing groups for several mobile media standards. The OpenGL ES working group was the first such group. Over time it has created several versions of a compact 3D graphics API (OpenGL ES stands for OpenGL for Embedded Systems). Nowadays, working groups exist for defining interfaces for 2D vector graphics (OpenVG), sound processing (OpenSL ES), video playback (OpenMAX), and application portability (OpenKODE). 3D content file formats are covered by the COLLADA working group, and other new groups are formed on a regular basis to meet the demands of the industry. OpenGL ARB (Archi- tecture Review Board) joined Khronos in September 2006. This makes it easier for both the desktop and embedded versions of OpenGL to develop in aligned directions. The goal of Khronos is the timely creation of royalty-free open multimedia standards. The first ver- sions of the APIs were created at quite a brisk pace; now the updates will be created when there is a real market need for them. 157 158 INTRODUCING OPENGL ES CHAPTER 7 Khronos is a consortium of more than 120 member companies, including, e.g., graphics technology providers, phone manufacturers, operating system vendors, content creators, and operators. There are two levels of membership. The Promoter-level members, some 15 companies, act as the board of directors and decide which standards are created, and ratify the results of the different working groups. Contributor companies participate in the working groups to define the standards. Additionally, adopters may create implemen- tations of the APIs. The OpenGL ES working group was formed in May 2002 to create an API for rendering 3D graphics on embedded devices. The desktop OpenGL standard was chosen as a start- ing point since it is widely available, well known, and has extremely well-specified, solid architecture. However, the desktop version has been evolving since the early 1990s, and has bloated over the years with redundant and legacy features. Removing old features is not a problem for applications that are written once, such as most games, whereas the development life cycle of large CAD systems and other professional applications may be decades, and removing old features would greatly increase maintenance costs. The goal of the OpenGL ES working group was to create a cleaned-up and t rimmed- down version of OpenGL. Innovation in the marketplace would still be enabled through an extension mechanism, and new versions of the specification would be rolled out as the market evolves. The first version, OpenGL ES 1.0, was introduced at the SIGGRAPH conference in July 2003. Version 1.1 followed at SIGGRAPH 2004. A draft version 2.0 and the ES 1.x Extension Pack came out a year later. The final 2.0 specification was completed in late 2006. The OpenGL ES working group consists of some 20 to 30 individuals representing differ- ent member companies. The group has weekly teleconferences, an active mailing list, and a few multi-day face-to-face meetings each year. In addition to the specification itself, the group also produces manual pages and the official conformance tests. These tests are used to raise the quality of OpenGL ES implementations in the market, as any implementation wanting to use the OpenGL ES trademark needs to pass them. Also, before the Promoters ratify any specification, at least one, and preferably two, working implementations have to be created. This ensures that the specifications also work in practice. After its first introduction, OpenGL ES has been widely adopted by the mobile industry. It is an integral part of a number of oper ating systems, and implementations have been shipped on tens of millions of devices. For example, all new Symbian and BREW devices support OpenGL ES, and outside the mobile phone world Sony is using it as the graphics API of PlayStation 3. 7.2 DESIGN PRINCIPLES When the OpenGL ES working group began its work of designing the API, several prin- ciples were laid out. One of the goals was to keep the API as compact as possible. This SECTION 7.3 RESOURCES 159 would allow software implementations of OpenGL ES to fit into a very small amount of code. Indeed, there exist implementations of OpenGL ES 1.0 with footprints of less than 50kB of binary code. The working g roup chose OpenGL 1.3 as the starting point and engaged in the long pro- cess of removing redundant functionality. In the world of graphics technology, OpenGL is a well-established standard, created in 1992 and originally based on SGI’s IrisGL. Due to the policy of maintaining backward-compatibility between versions, OpenGL has col- lected a lot of excess baggage over the years. Newer and better techniques have replaced older approaches and new hardware designs have made old ones obsolete. As a result, multiple different ways exist to accomplish the same goal. For example, there are half a dozen distinct ways for rendering a set of triangles! When creating the OpenGL ES specification, the goal was to identify only the most current and relevant variants and expose them. Apart from removing unnecessary functionality, there are other ways to make OpenGL slimmer. Almost all OpenGL API calls accept parameters in a variety of data formats. The working group decided to trim the number down considerably. In most cases, only a single integer format and a sing le floating-point variant is supported. Rarely used and outdated operations, such as support for indexed colors, or selection and feedback modes, were removed. Several features whose implementation burden was found to be excessive (for example, display lists) were dropped. The rule-of-thumb was that features hard to emulate in the application code were mostly retained, whereas convenience func- tions were considerably trimmed. Therefore high-level operations such as evaluators were dropped, but the fragment processing pipeline was kept almost in its entirety. Some features were also added to the API to support lower-end devices. All double- precision floating-point data types were replaced by single-precision, and a variant accept- ing fixed-point integer input was provided for all functions taking in floats. There is one fundamental difference between the OpenGL ES and its desktop variant. To prevent OpenGL ES from becoming bloated, the working group decided that backward- compatibility between versions is retained only between minor versions of the API. In other words, OpenGL ES 1.1 is backward-compatible with OpenGL ES 1.0, while 2.0 is not compatible with any of the 1.x versions (though OpenGL ES 2.0 is still similar enough that 1.x drivers should be implementable for 2.0 hardware). Although this creates several “families” of OpenGL ES, it allows new major versions to start from a clean slate, and to come up with more elegant designs. 7.3 RESOURCES Several free resources are available for those interested in using OpenGL ES. Additional resources are available for Khronos members. We list some of them below. 160 INTRODUCING OPENGL ES CHAPTER 7 7.3.1 DOCUMENTATION All of the official OpenGL ES documentation is hosted at Khronos Group’s web site. 1 Yo u will find there the most up-to-date versions of OpenGL ES and EGL specifications, refer- ence manuals, and official header files. The original OpenGL ES specification described the differences with respect to the desktop OpenGL (1.0 was based on desktop 1.3; 1.1 was based on desktop 1.5), but in 2007 a complete stand-alone specification was created. Another good source of general OpenGL material is at www.opengl.org. The www.khronos.org site offers a number of additional resources, such as presenta- tions from conferences and trade shows about OpenGL ES programming and optimiza- tion. The site also contains tutorials and example source code for sample OpenGL ES programs. If you are planning to implement OpenGL ES yourself, you should become a Khronos member. The members gain access to the OpenGL ES Conformance Test source code. This is needed in order to claim official conformance to the specification. Members also get the source code of the OpenGL ES 1.0 sample implementation, and get to parti- cipate on the internal technical mailing list dedicated for members. 7.3.2 TECHNICAL SUPPORT If the various specifications, reference manuals, and tutorials cannot answer all of your questions, do not despair: the Internet is full of people willing to help you. There are sev- eral web forums where OpenGL ES experts can answer your trickiest questions. The offi- cial one is hosted by Khronos. 2 Also, different handset vendors and chip manufacturers have their own for ums; we list them in this book’s companion web site. 7.3.3 IMPLEMENTATIONS There are several free implementations of OpenGL ES available; links to them can be found from the book’s companion web site. Vincent is an open source implementation of OpenGL ES 1.1, and it runs on several handheld operating systems. Hybrid’s Rasteroid package contains free implementations of both OpenGL ES 1.1 and OpenVG 1.0, and runs on desktop Windows and many Windows Mobile and S60 devices. Other implementations include an OpenGL ES 1.0 Linux sample reference implemen- tation that runs on top of desktop OpenGL 1.3, and PowerVR’s implementation. Several handset vendors also provide SDKs that include OpenGL ES: Nokia’s S60, SonyEricsson’s Symbian UIQ 3 SDK, and Qualcomm’s BREW. 1 www.khronos.org/opengles/ 2 www.khronos.org/message_boards/ SECTION 7.4 API OVERVIEW 161 Additionally, OpenGL ES libraries come pre-installed on many newer PDAs and smartphones. The high-end ones, such as Nokia’s N93 and N95, have even full hardware acceleration. All devices that are based on S60 2nd edition FP2, or later, have at least a software-based OpenGL ES implementation. 7.4 API OVERVIEW OpenGL ES comes in several flavors. Various profiles have been defined to cater to the needs of different market segments. Later versions introduce additional features, and different vendors provide various extensions. 7.4.1 PROFILES AND VERSIONS Two important concepts to understand about OpenGL ES are profiles and versions.Pro- files are used to create different subsets to target very distinct use cases. Currently there are three major profiles of OpenGL ES: the Common profile (also known as CM), the Common Lite (CL) profile, and the Safety-Critical (SC) profile. The first two are similar; the main difference is the omission of all floating-point entry points from the Common Lite profile. The Safety-Critical profile differs significantly from the other two profiles and is targeted primarily for the aviation market. It uses OpenGL 1.3 as its starting point and removes functionality not needed in cockpit displays while retaining functionality needed by legacy applications in this specialized market, e.g., display lists. We will not cover the Safety-Critical profile in this book, instead we focus on the Common and Common Lite profiles. Each profile has versions indicated by two numbers, the major and minor version num- bers. For example, in OpenGL ES 1.0 the major version number is 1 and the minor one 0. The specifications sharing the same major number are backward-compatible. This means that an application written with the OpenGL ES 1.0 specification will compile, link, and run unmodified using an OpenGL ES 1.1 implementation. However, the same application will not run on an OpenGL ES 2.0 implementation. 7.4.2 OPENGL ES 1.0 IN A NUTSHELL The most compact way to cover OpenGL ES is to examine where it differs from its desk- top cousin, although this approach assumes familiarity with the desktop version. While OpenGL ES misses a number of function entry points and even major parts of the desktop OpenGL functionality, the underlying rendering model, terminology, and key concepts are the same. 162 INTRODUCING OPENGL ES CHAPTER 7 Floating-point and fixed-point values OpenGL is at its core a floating-point API. However, most mobile devices do not support a hardware floating-point unit. Two changes were made to keep the API simple and efficient. First, the double data type (a 64-bit high-precision floating-point number) is not supported. Many functions that take doubles as arguments already had a variant accept- ing floats, and for the rest (e.g., glFrustum) a new variant accepting float arguments (e.g., glFrustumf) was created. Second, a new fixed-point data type was created: type GLfixed, with a correspoding enum GL_FIXED. GLfixed is a number stored in a 32-bit integer, but interpreted so that the first 16 bits describe a signed two’s complement integer value, and the last 16 bits describe the fractional component. In other words, you can convert between GLfixed and GLfloat by multiplying a GLfloat by 2 16 or by dividing a GLfixed by 2 16 . Finally, for each function that takes in floats, a variant was created that takes in fixed- point arguments (e.g., glFrustumx). For details on fixed-point programming, refer to Appendix A. As in OpenGL, the GLfloat type for passing arguments to the engine is an IEEE float, but inside the engine the representation can be different and only needs to keep the accu- racy of 1 part in 10 5 . The positional and normal coordinates must store magnitudes up to 2 32 , while the colors and texture coordinates must only store magnitudes up to 2 10 . Note that these requirements also apply when using the fixed-point functions. In practice, in a well-implemented software engine the performance savings of using the fixed-point versions of the functions are not significant. The major exception is with ver- tex data, which should never be given in floating point, unless you know that both your CPU and GPU have hardware support for floating-point data. Vertex data Originally, the rendering primitives in OpenGL were described by issuing a begin com- mand for a set of primitives, and then updating the current vertex positions, normal vec- tors, colors, or texture coordinates in an arbitrary order, and finally ending the primitive. This creates a very complicated state machine that does not run at an optimal speed. In current OpenGL versions, the vertex data is provided through vertex arrays and is ren- dered using calls to glDrawElements or glDrawArrays. OpenGL ES adopted only these simpler and more efficient approaches. Vertex data can be specified using byte, short, float,orfixed, whereas int and double are not supported. Note that unlike on desktop OpenGL, the vertices can also be specified in bytes. Fog coordinates and secondary colors have been dropped from the specification. SECTION 7.4 API OVERVIEW 163 Primitives Triangles, lines, and points are supported as lists, strips, or fans. Quads and polygons are not supported, as they are too trivial to implement inside an application, and their spec- ification is ambiguous if the vertices are not all on the same plane. glPolygonMode, which allows the same geometry to be drawn either as triangles, lines, or points, is not sup- ported; hence the concept of edge flags, denoting which edges to draw in the line drawing mode, does not exist either. Also, 2D primitives such as rectangles are not provided as they can be emulated with a pair of triangles. Line and polygon stippling are not supported; if needed, the y can be emulated using texture mapping. Transformation and lighting The most significant changes to the matrix stacks of desktop OpenGL are dropping of the color matrix stack and reducing the minimum depth of the modelview stack from 32 to 16. OpenGL ES does not support transposed matrices. Texture coordinate gener- ation has been omitted as it can be emulated in the applications. Only RGBA colors (or their subsets) are provided; there is no support for indexed colors. The lighting pipeline has been left largely intact with a few exceptions. There is no sup- port for secondary colors, local viewer lighting model, or color material tracking except for GL_AMBIENT_AND_DIFFUSE. Two-sided lighting has been retained, but separate front and back materials cannot be specified. Texturing OpenGL ES supports only 2D texture mapping since 1D maps are trivial to emulate using 2D maps, and 3D maps are too resource intensive to include at this stage. The imple- mentation burden of cube mapping was deemed too high both for software and hard- ware implementations. Texture borders, proxies, priorities, and LOD clamping are not included. Texture formats must match internal formats, and only the five most important formats are supported. Texture addressing modes are limited to GL_CLAMP_TO_EDGE and GL_REPEAT. OpenGL ES adds support for paletted textures, an efficient way of compressing texture data. This extension introduces a number of new internal texture formats. Reading back texture data, i.e., support for glGetTexImage, is not allowed. Multi- texturing is supported, but an implementation does not have to provide more than one texturing unit. OpenGL ES 1.1 raises this to a minimum of two units. Fragment pipeline OpenGL ES does not make major changes to the fragment pipeline. Both stencil and depth buffering are supported, although stencil buffering is left optional, and not many . this creates several “families” of OpenGL ES, it allows new major versions to start from a clean slate, and to come up with more elegant designs. 7.3 RESOURCES Several free resources are available. The original OpenGL ES specification described the differences with respect to the desktop OpenGL (1.0 was based on desktop 1.3; 1.1 was based on desktop 1.5), but in 2007 a complete stand-alone. OpenGL ES 1.x standard. First we take a brief look at its history and design principles, followed by an introduction to the resources available online. Then we examine the different profiles and