Direct3D ShaderX Vertex and Pixel Shader Tips and Tricks Edited by Wolfgang F Engel Wordware Publishing, Inc Team LRN Library of Congress Cataloging-in-Publication Data Direct3d ShaderX : vertex and pixel shader tips and tricks / edited by Wolfgang F Engel p cm Includes bibliographical references and index ISBN 1-55622-041-3 Computer games Programming Three-dimensional display systems Direct3D I Engel, Wolfgang F QA76.76.C672 D57 2002 794.8'15265 dc21 2002006571 © 2002, Wordware Publishing, Inc All Rights Reserved 2320 Los Rios Boulevard Plano, Texas 75074 No part of this book may be reproduced in any form or by any means without permission in writing from Wordware Publishing, Inc Printed in the United States of America ISBN 1-55622-041-3 10 0502 RenderMan is a registered trademark of Pixar Animation Studios krass engine is a trademark of Massive Development 3D Studio Max is a registered trademark of Autodesk/Discreet in the USA and/or other countries GeForce is a trademark of NVIDIA Corporation in the United States and/or other countries Some figures in this book are copyright ATI Technologies, Inc., and are used with permission Other product names mentioned are used for identification purposes only and may be trademarks of their respective companies All inquiries for volume purchases of this book should be addressed to Wordware Publishing, Inc., at the above address Telephone inquiries may be made by calling: (972) 423-0090 Team LRN iii Contents Foreword xvi Acknowledgments xvii Part 1: Introduction to Shader Programming Fundamentals of Vertex Shaders Wolfgang F Engel What You Need to Know/Equipment Vertex Shaders in the Pipeline Why Use Vertex Shaders? Vertex Shader Tools NVIDIA Effects Browser 2/3 NVIDIA Shader Debugger Shader City 10 Vertex Shader Assembler 10 NVIDIA NVASM — Vertex and Pixel Shader Macro Assembler 10 Microsoft Vertex Shader Assembler 11 Shader Studio 11 NVLink 2.x 12 NVIDIA Photoshop Plug-ins 13 Diffusion Cubemap Tool 14 DLL Detective with Direct3D Plug-in 15 3D Studio Max 4.x/gmax 1.1 16 Vertex Shader Architecture 16 High-Level View of Vertex Shader Programming 18 Check for Vertex Shader Support 19 Vertex Shader Declaration 19 Set the Vertex Shader Constant Registers 22 Write and Compile a Vertex Shader 22 Application Hints 26 Complex Instructions in the Vertex Shader 27 Putting It All Together 27 Swizzling and Masking 31 Guidelines for Writing Vertex Shaders 32 Compiling a Vertex Shader 33 Create a Vertex Shader 34 Set a Vertex Shader 34 Free Vertex Shader Resources 35 Summary 35 What Happens Next? 35 Team LRN iv Contents Programming Vertex Shaders 38 Wolfgang F Engel RacorX The Common Files Framework Check for Vertex Shader Support Vertex Shader Declaration Set the Vertex Shader Constant Registers The Vertex Shader Compile a Vertex Shader Create a Vertex Shader Set a Vertex Shader Free Vertex Shader Resources Non-Shader Specific Code Summary RacorX2 Create a Vertex Shader Summary RacorX3 Vertex Shader Declaration Set the Vertex Shader Constant Registers The Vertex Shader Directional Light Diffuse Reflection Summary RacorX4 Vertex Shader Declaration Set the Vertex Shader Constants The Vertex Shader Specular Reflection Non-Shader Specific Code Summary RacorX5 Point Light Source Light Attenuation for Point Lights Set the Vertex Shader Constants The Vertex Shader Summary What Happens Next? 38 40 42 42 43 43 45 46 47 47 47 49 49 51 53 53 54 54 55 57 57 59 59 59 60 61 62 65 66 66 66 66 67 68 70 70 Fundamentals of Pixel Shaders 72 Wolfgang F Engel Why Use Pixel Shaders? Pixel Shaders in the Pipeline Pixel Shader Tools Microsoft Pixel Shader Assembler MFC Pixel Shader Team LRN 72 74 79 79 80 v Contents ATI ShadeLab 80 Pixel Shader Architecture 81 Constant Registers (c0-c7) 82 Output and Temporary Registers (ps.1.1-ps.1.3: r0+r1; ps.1.4: r0-r5) 82 Texture Registers (ps.1.1-ps.1.3: t0-t3; ps.1.4: t0-t5) 82 Color Registers (ps.1.1-ps.1.4: v0+v1) 83 Range 83 High-Level View of Pixel Shader Programming 84 Check for Pixel Shader Support 84 Set Texture Operation Flags (D3DTSS_* flags) 85 Set Texture (with SetTexture()) 86 Define Constants (with SetPixelShaderConstant()/def ) 86 Pixel Shader Instructions 87 Texture Address Instructions 88 Arithmetic Instructions 101 Instruction Modifiers 110 Instruction Modifiers 116 Instruction Pairing 119 Assemble Pixel Shader 120 Create a Pixel Shader 120 Set Pixel Shader 121 Free Pixel Shader Resources 121 Summary 121 What Happens Next? 122 Programming Pixel Shaders 125 Wolfgang F Engel RacorX6 Check for Pixel Shader Support Set Texture Operation Flags (with D3DTSS_* flags) Set Texture (with SetTexture()) Define Constants (with SetPixelShaderConstant()/def) Pixel Shader Instructions Per-Pixel Lighting Assemble Pixel Shader Create Pixel Shader Set Pixel Shader Free Pixel Shader Resources Non-Shader Specific Code Summary RacorX7 Define Constants (with SetPixelShaderConstant()/def) Pixel Shader Instructions Summary RacorX8 Set Texture Operation Flags (with D3DTSS_* flags) Team LRN 125 126 126 127 127 128 130 134 134 136 136 136 136 137 137 137 140 140 140 vi Contents Set Texture (with SetTexture()) Pixel Shader Instructions Summary RacorX9 Summary Further Reading 140 142 143 144 147 147 Basic Shader Development with Shader Studio 149 John Schwab Introduction What You Should Know Installation Directories Coordinate Systems Features Limitations Default Light Controls Create a Vertex Shader Step 1: Loading a Mesh Step 2: Transformations Step 3: Starting a Shader Step 4: Editing Code Step 5: Adding Some Color Step 6: Saving Your Work Step 7: Loading a Vertex Shader Step 8: Settings Step 9: Texture Control Create a Pixel Shader Step 1: Configuration Step 2: Writing a Pixel Shader Shaders Reference Workspace Menu Layout Files Shaders Dialog Vertex Shader Pixel Shader Declarations Constant Properties Dialog Materials Dialog Textures Dialog Browser Transformations Dialog Object Light Team LRN 149 149 149 149 150 150 151 152 152 152 153 153 154 154 155 155 156 157 157 159 159 159 160 160 160 161 161 162 162 163 163 164 165 165 165 166 166 vii Contents Camera Statistics Dialog Settings Dialog State Methods Camera Projection Configure Dialog Assets Further Info 166 167 167 167 168 168 169 169 Part 2: Vertex Shader Tricks Vertex Decompression in a Shader 172 Dean Calver Introduction Vertex Compression Overview Vertex Shader Data Types Compressed Vertex Stream Declaration Example Basic Compression Quantization Compression Decompression Practical Example Scaled Offset Compression Decompression Practical Example Compression Transform Compression Decompression Practical Example Optimizations Practical Example Advanced Compression Multiple Compression Transforms Compression Decompression Practical Example Sliding Compression Transform Compression Decompression Displacement Maps and Vector Fields Basic Displacement Maps Entering Hyperspace Conclusion Team LRN 172 172 173 174 174 174 174 175 175 175 175 176 176 176 177 178 178 179 180 180 180 181 181 181 182 182 183 184 184 186 187 viii Contents Shadow Volume Extrusion Using a Vertex Shader 188 Chris Brennan Introduction Creating Shadow Volumes Effect File Code Using Shadow Volumes with Character Animation 188 189 191 192 Character Animation with Direct3D Vertex Shaders 195 David Gosselin Introduction Tweening Skinning Skinning and Tweening Together Animating Tangent Space for Per-Pixel Lighting Per-Pixel Lighting Compression Summary 195 195 197 199 202 203 206 208 Lighting a Single-Surface Object 209 Greg James Vertex Shader Code 211 Enhanced Lighting for Thin Objects 213 About the Demo 214 Optimizing Software Vertex Shaders 215 Kim Pallister Introduction to Pentium Processor Architecture Introduction to the Streaming SIMD Extensions Optimal Data Arrangement for SSE Instruction Usage How the Vertex Shader Compiler Works Performance Expectations Optimization Guidelines Write Only the Results You’ll Need Use Macros Whenever Possible Squeeze Dependency Chains Write Final Arithmetic Instructions Directly to Output Registers Reuse the Same Temporary Register If Possible Don’t Implicitly Saturate Color and Fog Values Use the Lowest Acceptable Accuracy with exp and log Functions Avoid Using the Address Register When Possible Try to Order Vertices Profile, Profile, Profile… A Detailed Example Team LRN 216 217 218 220 221 221 221 222 222 223 223 223 223 223 224 224 224 ... ShaderX Vertex and Pixel Shader Tips and Tricks Edited by Wolfgang F Engel Wordware Publishing, Inc Team LRN Library of Congress Cataloging-in-Publication Data Direct3d ShaderX : vertex and pixel. .. Per -Pixel Strand-Based Illumination Per -Pixel Strand-Based Illumination with Colored Light and Base Map Per -Pixel Strand-Based Illumination with Four Colored Lights and. .. on Shader Studio, a tool for designing vertex and pixel shaders Fundamentals of Vertex Shaders This article discusses vertex shaders, vertex shader tools, and lighting and transformation with vertex