Cardboard VR Projects for Android Develop mobile virtual reality apps using the native Google Cardboard SDK for Android Jonathan Linowes Matt Schoen BIRMINGHAM - MUMBAI [ FM-1 ] Cardboard VR Projects for Android Copyright © 2016 Packt Publishing All rights reserved No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews Every effort has been made in the preparation of this book to ensure the accuracy of the information presented However, the information contained in this book is sold without warranty, either express or implied Neither the authors, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals However, Packt Publishing cannot guarantee the accuracy of this information First published: May 2016 Production reference: 1120516 Published by Packt Publishing Ltd Livery Place 35 Livery Street Birmingham B3 2PB, UK ISBN 978-1-78588-787-1 www.packtpub.com Credits for the Cover Image: Custom Illustration designed by eLearning Mind, LLC., www.eLearningMind.com, ELM creates interactive learning experiences using modern brain science and intuitively stunning design The DIY Virtual Reality article: http://elearningmind.com/diy-virtual-reality-world-of-warcraftthinks-inside-the-box-at-comic-con-2015/ [ FM-2 ] Credits Authors Project Coordinator Jonathan Linowes Nikhil Nair Matt Schoen Proofreader Safis Editing Reviewers Scott Dolim Indexer Oleksandr Popov Hemangini Bari Commissioning Editor Graphics Edward Gordon Kirk D'Penha Acquisition Editor Production Coordinator Reshma Raman Shantanu N Zagade Content Development Editor Sachin Karnani Cover Work Shantanu N Zagade Technical Editor Siddhi Rane Copy Editor Rashmi Sawant [ FM-3 ] About the Authors Jonathan Linowes is the owner of Parkerhill Reality Labs, a start-up VR/AR consultancy firm He is a VR and 3D graphics enthusiast, full-stack web developer, software engineer, successful entrepreneur, and teacher He has a fine arts degree from Syracuse University and a master's degree from the MIT Media Lab He has founded several successful start-ups and held technical leadership positions at major corporations, including Autodesk Inc He is also the author of the Unity Virtual Reality Projects book by Packt Publishing Matt Schoen is the cofounder of Defective Studios and has been making VR apps since the early DK1 days Still in the early stages of his career, he spent most of his time working on Unity apps and games, some for hire and some of his own design He studied computer engineering at Boston University and graduated with a BS in 2010, at which point he founded Defective with Jono Forbes, a high-school friend He has been making games and apps ever since Matt was the technical lead on Defective's debut game, CosmoKnots, and remains involved in Jono's pet project, Archean This is his first foray into authorship, but he brings with him his experience as an instructor and curriculum designer for Digital Media Academy Jono and Matt have recently joined Unity's VR Labs division, where they will be helping to create experimental new features which will shape the VR landscape for years to come [ FM-4 ] About the Reviewers Scott Dolim has worked on and off in 3D computer graphics for over 20 years, including a 5-year stint at Walt Disney Feature Animation in the 1990s More recently, for the last years, he has been actively involved in virtual reality development, mostly with Unity 3D Scott currently works at Google where he is the lead engineer of the Cardboard SDK for Unity Oleksandr Popov is a developer of numerous 3D apps, mainly live wallpapers, for Android devices His first experience with 3D for Android started in 2012 when with the release of Android 2.2, it became possible to create live wallpapers Since then, he has released about 15 of them in collaboration with his brother, Dmytro, who is responsible for creating 3D scenes After releasing each app, he gained more and more experience in OpenGL ES Basically, he tried almost every new feature of Android where 3D and OpenGL can be applied He started with live wallpapers in Android 2.2, then he added support of the daydream mode for them in 4.2 He started using some of the features of OpenGL ES 3.0 introduced in Android 4.3 And as soon as Google added support of custom watch faces for Android Wear 5.0, he and his brother created a set of 3D watch faces for smart watches too Of course, after Google announced Cardboard, he immediately decided to create VR apps for this platform as well He and his brother are also coauthors of the Deconstructing Google Cardboard Apps book by Bleeding Edge Press [ FM-5 ] www.PacktPub.com eBooks, discount offers, and more Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.PacktPub.com and as a print book customer, you are entitled to a discount on the eBook copy Get in touch with us at customercare@packtpub.com for more details At www.PacktPub.com, you can also read a collection of free technical articles, sign up for a range of free newsletters and receive exclusive discounts and offers on Packt books and eBooks TM https://www2.packtpub.com/books/subscription/packtlib Do you need instant solutions to your IT questions? PacktLib is Packt's online digital book library Here, you can search, access, and read Packt's entire library of books Why subscribe? • Fully searchable across every book published by Packt • Copy and paste, print, and bookmark content • On demand and accessible via a web browser [ FM-6 ] Table of Contents Preface ix Chapter 1: Virtual Reality for Everyone Why is it called Cardboard? The spectrum of VR devices Old fashioned stereoscopes Cardboard is mobile VR Desktop VR and beyond A gateway to VR The value of low-end VR Cardware! 11 Configuring your Cardboard viewer 14 Developing apps for Cardboard 16 Using Unity 16 Going native 17 An overview to VR best practices 19 Summary 21 Chapter 2: The Skeleton Cardboard Project What's in an Android app? APK files A Gradle build process A Java compiler The Android project structure Getting started with Android Studio Installing Android Studio The Android Studio user interface Creating a new Cardboard project Adding the Cardboard Java SDK [i] 23 23 24 24 26 26 29 29 29 33 37 Table of Contents The AndroidManifest.xml file The activity_main.xml file The MainActivity class Default onCreate Building and running Summary 40 45 46 48 49 51 Chapter 3: Cardboard Box 53 Creating a new project 54 Hello, triangle! 55 Introducing geometry 55 Triangle variables 57 onSurfaceCreated 58 Introducing OpenGL ES 2.0 58 Simple shaders 61 The compileShaders method 63 The prepareRenderingTriangle method 63 onDrawEye 65 Building and running 66 3D camera, perspective, and head rotation 67 Welcome to the matrix 67 The MVP vertex shader 70 Setting up the perspective viewing matrices 70 Render in perspective 71 Building and running 73 Repositioning the triangle 74 Hello, cube! 75 The cube model data 75 Cube code 77 Lighting and shading 80 Adding shaders 80 Cube normals and colors 82 Preparing the vertex buffers 84 Preparing the shaders 85 Adding a light source 87 Building and running the app 88 Spinning the cube 88 Hello, floor! 89 Shaders 89 Floor model data 91 Variables 91 [ ii ] Table of Contents onCreate 92 onSurfaceCreated 92 initializeScene 92 prepareRenderingFloor 93 onDrawEye 94 drawFloor 94 Hey, look at this! 95 The isLookingAtObject method 95 Summary 98 Chapter 4: Launcher Lobby Creating a new project Adding Hello Virtual World text overlay A simple text overlay Center the text using a child view Create stereoscopic views for each eye Controlling the overlay view from MainActivity Using a virtual screen Responding to head look Adding an icon to the view Listing installed Cardboard apps Queries for Cardboard apps Create the Shortcut class for apps Add shortcuts to OverlayView Using view lists in OverlayEye Highlighting the current shortcut Using the trigger to pick and launch the app Further enhancements Summary Chapter 5: RenderBox Engine 99 100 101 101 103 105 108 109 111 113 115 116 117 117 118 120 122 123 124 125 Introducing RenderBox – a graphics engine 126 Creating a new project 128 Creating the RenderBox package folder 129 Creating an empty RenderBox class 130 Adding the IRenderBox interface 132 Materials, textures, and shaders 133 Abstract material 134 The Math package 137 MathUtils 137 Matrix4 138 Quaternion 138 [ iii ] Chapter Finally, we can implement the customClear method: public static void customClear(float[] clearColor){ GLES20.glUseProgram(program); // Set the position buffer GLES20.glVertexAttribPointer(positionParam, 3, GLES20.GL_FLOAT, false, 0, vertexBuffer); GLES20.glUniform4fv(colorParam, 1, clearColor, 0); GLES20.glDrawElements(GLES20.GL_TRIANGLES, numIndices, GLES20.GL_UNSIGNED_SHORT, indexBuffer); } Rebuild the RenderBox module and copy the library file back to this VisualizeVR project Don’t forget to set trailsMode to true! Now when you run the app, it looks trippy and cool! Multiple simultaneous visualizations Now that we have a collection of visualizations, we can enhance the app to run more than one at a time and switch between them To support multiple concurrent visualizations, replace the activeViz variable in VisualizerBox with a list of visualizations: public List visualizations = new ArrayList