Ebook Programming interactivity Part 1

(BQ) Part 1 book Programming interactivity has contents Introducing interaction design, programming basics, processing, arduino, programming revisited, open frameworks, sound and audio, physical input, programming graphics.

Programming Interactivity A Designer’s Guide to Processing, Arduino, and openFrameworks Joshua Noble Beijing • Cambridge • Farnham • Köln • Sebastopol • Taipei • Tokyo Programming Interactivity by Joshua Noble Copyright © 2009 Joshua Noble All rights reserved Printed in the United States of America Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472 O’Reilly books may be purchased for educational, business, or sales promotional use Online editions are also available for most titles (http://my.safaribooksonline.com) For more information, contact our corporate/institutional sales department: (800) 998-9938 or corporate@oreilly.com Editor: Steve Weiss Production Editor: Sumita Mukherji Copyeditor: Kim Wimpsett Proofreader: Sumita Mukherji Production Services: Newgen Indexer: Ellen Troutman Zaig Cover Designer: Karen Montgomery Interior Designer: David Futato Illustrator: Robert Romano Printing History: July 2009: First Edition Nutshell Handbook, the Nutshell Handbook logo, and the O’Reilly logo are registered trademarks of O’Reilly Media, Inc Programming Interactivity, the image of guinea fowl, and related trade dress are trademarks of O’Reilly Media, Inc Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks Where those designations appear in this book, and O’Reilly Media, Inc., was aware of a trademark claim, the designations have been printed in caps or initial caps While every precaution has been taken in the preparation of this book, the publisher and author assume no responsibility for errors or omissions, or for damages resulting from the use of the information contained herein TM This book uses RepKover™, a durable and flexible lay-flat binding ISBN: 978-0-596-15414-1 [M] 1247251884 Table of Contents Preface xv Part I Introductions Introducing Interaction Design What This Book Is for Programming for Interactivity The Nature of Interaction Messages and Interaction Interfaces and Interaction Languages of Interaction Design and Interaction Art and Interaction Data Exchange and Exploration Working Process 10 12 13 15 19 Programming Basics 21 Why You’ll Read This Chapter More Than Once The Nature of Code Variables Simple Types Arrays Casting Operators Control Statements if/then for Loop while Loop continue break Functions 22 22 23 24 29 33 33 37 37 38 39 40 40 41 iii Defining a Function Passing Parameters to a Method Some Suggestions on Writing Functions Overloading Functions Objects and Properties Scope Review 41 42 44 44 46 49 50 Processing 53 Downloading and Installing Processing Exploring the Processing IDE The Basics of a Processing Application The setup() Method The draw() Method The Basics of Drawing with Processing The rect(), ellipse(), and line() Methods RGB Versus Hexadecimal The fill() Method The background() Method The line() Method The stroke() and strokeWeight() Methods The curve() Method The vertex() and curveVertex() Methods Capturing Simple User Interaction The mouseX and mouseY Variables The mousePressed() Method The mouseReleased() and mouseDragged() Methods The keyPressed and key Variables Importing Libraries Downloading Libraries Loading Things into Processing Loading and Displaying Images Displaying Videos in the Processing Environment Using the Movie Class Reading and Writing Files Running and Debugging Applications Exporting Processing Applications Conclusion Review 54 54 56 56 57 60 60 62 63 65 65 65 66 66 67 68 69 70 73 77 77 79 79 81 81 83 85 86 88 89 Arduino 91 Starting with Arduino Installing the IDE iv | Table of Contents 92 93 Configuring the IDE Touring Two Arduino Boards The Controller Duemilanove Versus Mini Touring the Arduino IDE The Basics of an Arduino Application The setup Statement The loop Method Features of the Arduino Language Constants Methods Arrays Strings How to Connect Things to Your Board Hello World Debugging Your Application Importing Libraries Running Your Code Running Your Board Without a USB Connection Review 96 97 97 97 102 105 106 106 108 109 110 111 112 115 117 122 124 126 126 127 Programming Revisited 129 Object-Oriented Programming Classes The Basics of a Class Class Rules Public and Private Properties Inheritance Processing: Classes and Files C++: Classes and Files cpp and h A Simple C++ Application Pointers and References Reference Pointer When to Use Pointers Large Data Objects Pointers and Arrays When Are You Going to Use This? Review 129 130 131 132 133 135 137 139 140 142 144 146 146 147 148 149 150 151 openFrameworks 153 Your IDE and Computer 154 Table of Contents | v Windows Mac OS X Linux Taking Another Quick Tour of C++ Basic Variable Types Arrays Methods Classes and Objects in C++ Getting Started with oF Touring an oF Application Methods Variables Creating “Hello, World” Drawing in 2D Setting Drawing Modes Drawing Polygons Displaying Video Files and Images Images Video Importing Libraries ofxOpenCv ofxVectorGraphics ofxVectorMath ofxNetwork ofxOsc Compiling an oF Program Compiling in Xcode Compiling in Code::Blocks Debugging an oF Application Using the printf Statement Using the GNU Debugger Using the Debugger in Xcode Using the Debugger in Code::Blocks Review 154 155 155 156 157 157 158 159 160 166 166 168 168 171 172 174 176 176 178 180 181 181 182 182 182 183 183 184 184 184 185 186 188 188 Part II Themes Sound and Audio 193 Sound As Feedback Sound and Interaction How Sound Works on a Computer Audio in Processing vi | Table of Contents 194 197 199 202 Instantiating the Minim Library Generating Sounds with Minim Filtering Sounds with Minim Sound in openFrameworks openFrameworks and the FMOD Ex Library The Sound Object Library The Magic of the Fast Fourier Transform Physical Manipulation of Sound with Arduino A Quick Note on PWM Creating Interactions with Sound Further Resources Review 202 204 208 214 221 228 233 238 239 242 242 243 Physical Input 245 Interacting with Physical Controls Thinking About Kinetics Getting Gear for This Chapter Controlling Controls The Button As an Electrical Object The Button As an Interactive Object The Button As a Value in Code Turning Knobs The Dial As an Interactive Object Potentiometers Using Lights Wiring an LED Detecting Touch and Vibration Reading a Piezo Sensor Getting Piezo Sensors Communicating with Other Applications Sending Messages from the Arduino openFrameworks Detecting Motion PIR Motion Sensor Reading Distance Reading Input from an Infrared Sensor Understanding Binary Numbers Binary Numbers Bits and Bit Operations Why Do You Need to Know Any of This? Detecting Forces and Tilt Introducing I2C What Is a Physical Interface? 245 246 247 248 248 248 248 249 249 249 251 252 253 254 255 259 262 263 265 265 267 269 270 270 271 273 273 278 283 Table of Contents | vii What’s Next Review 284 286 Programming Graphics 289 The Screen and Graphics Seeing Is Thinking, Looking Is Reading Math, Graphics, and Coordinate Systems Drawing Strategies Use Loops to Draw Use Arrays to Draw Draw Only What You Need Use Sprites Processing and Transformation Matrices Creating Motion Shaping the Gaze Setting the Mood Creating Tweens Using Vectors Using Graphical Controls ControlP5 Library Event Handling Importing and Exporting Graphics Using PostScript in Processing Using PostScript Files in oF What’s Next Review 289 292 293 296 296 298 303 303 303 307 308 308 310 315 325 326 326 328 329 330 333 334 10 Bitmaps and Pixels 337 Using Pixels As Data Using Pixels and Bitmaps As Input Providing Feedback with Bitmaps Looping Through Pixels Manipulating Bitmaps Manipulating Color Bytes Using Convolution in Full Color Analyzing Bitmaps in oF Analyzing Color Analyzing Brightness Detecting Motion Using Edge Detection Using Pixel Data Using Textures Textures in oF viii | Table of Contents 338 340 341 342 345 347 348 349 350 351 353 355 361 368 369 #include "ofMain.h" #include "Ball.h" #define numBalls class ch09vector : public ofBaseApp { public: void setup(); void update(); void draw(); void mouseMoved(int x, int y ); void drawConnectors(); ofxVec3f wind; ofxVec3f gravity; Ball b1; Ball b2; Ball b3; Ball* balls[3]; }; #endif Example 9-11 ch09Vector.cpp #include "ch09vector.h" void ch09vector::setup() { Here are the two forces that will be applied to each Ball, creating constant forces on them: gravity = ofxVec3f(0.0, 1.0f, 0.0); wind = ofxVec3f(0.5f, 0.0, 0.0); b1.color[0] = 0; b2.color[1] = 0; b3.color[2] = 0; balls[0] = &b1; //use a reference so we can access the pointer later balls[1] = &b2; balls[2] = &b3; } ofEnableAlphaBlending(); //for smooth animation, set vertical sync if we can ofSetVerticalSync(true); // also, frame rate: ofSetFrameRate(60); 322 | Chapter 9: Programming Graphics On each call to update(), the application adds the forces to the balls again so that their velocities and accelerations will reflect constant forces acting on them like wind and gravity: void ch09vector::update(){ b1.addForce(gravity); b2.addForce(gravity); b3.addForce(gravity); b1.addForce(wind); b2.addForce(wind); b3.addForce(wind); b1.updateBall(); b2.updateBall(); b3.updateBall(); } void ch09vector::draw(){ b1.drawFrame(); b2.drawFrame(); b3.drawFrame(); } drawConnectors(); void ch09vector::mouseMoved(int x, int y ){ gravity.set(float((x −500)/100), float((y - 500)/100), 0); } The drawConnectors() method draws a line from the center of each Ball drawn on the screen by reading the location vector of each Ball and using that to supply values to an ofLine() call: void ch09vector::drawConnectors() { int i, j; for(i=0; ilocation.y, balls[j]->location.x, balls[j]->location.y); } } } So far so good, but now you’ll add some collision detection to the balls Add the following to Example 9-11: void ch09vector::checkCollision() { int i, j; Using Vectors | 323 for(i=0; i balls[j]->location.x && balls[i]->location.x < balls[j]->location.x+100) { if (balls[i]->location.y+100 > balls[j]->location.y && balls[i]->location.y < balls[j]->location.y+100) { balls[i]->collision(balls[j]); } } } } } This determines whether the balls are actually colliding with one another You’ll also need to add a checkCollision() call to the update() method of the application The call to the collision() method of the Ball implies that you’ll be adding a collision() method: void Ball::collision(Ball* b1) { // this helps cut down on typing :) float m1 = mass; float m2 = b1->mass; // note pointer access using the -> float x1 = location.x; float x2 = b1->location.x; float y1 = location.y; float y2 = b1->location.y; // we'll just declare all these things we'll need at once float newMass, diff, angle, newX, newY, newVelocityX, newVelocityY, fy21, sign; newMass = m2/m1; newX = x2-x1; newY = y2-y1; newVelocityX = b1->velocity.x - velocity.x; newVelocityY = b1->velocity.y - velocity.y; // If the balls aren't heading at one another, we don't want to alter them // because they could be heading away from each other if ( (newVelocityX*newX + newVelocityY*newY) >= 0) return; fy21=1.0E-12*fabs(newY); if ( fabs(newX) ){ int numPts = pts.size(); output.setColor(0x0000ff); output.beginShape(); // begin the shape int rescaleRes = 6; //create nice smooth curves between points for(int i = 0; i < numPts; i++){ //we need to draw the first and last point //twice for a catmull curve if(i == || i == numPts −1){ output.curveVertex(pts[i].x, pts[i].y); } if(i % rescaleRes == 0) { output.curveVertex(pts[i].x, pts[i].y); } } output.endShape(); // end the shape } } // this will write the PS file to the data folder if(capture){ output.endEPS(); capture =false; } If the user presses the space bar, set the capture variable to true so that the draw() method will write to the ps file: void VectorGrfxApp::keyPressed(int key){ if(key == ' '){ capture = true; } } 332 | Chapter 9: Programming Graphics void VectorGrfxApp::mouseMoved(int x, int y ){ // make some values that will be used in the draw() method //to draw circles for(int i = 0; i < 30; i++) { angles[i] = * PI / steps * i + phaseoffset; } phaseoffset += PI / 180 * 1; } void VectorGrfxApp::mouseDragged(int x, int y, int button){ //we add a new point to our line pts.push_back(ofPoint()); int last = pts.size()-1; pts[last].x = x; pts[last].y = y; } // start storing the line void VectorGrfxApp::mousePressed(int x, int y, int button){ pts.clear(); //store the first point of the line pts.push_back(ofPoint()); pts[0].x = x; pts[0].y = y; } Vector graphics need not be limited to the screen By creating SVG and PostScript files you can create print ready graphics or lightweight graphics that can easily be stored, sent over the Internet to be displayed on other computers, or reloaded and manipulated later What’s Next There are far too many techniques to use in creating vector graphics for it to ever fit in a single book much less in a single chapter Depending on what you want to do, you might want to consider a few different directions Chapter 13 of this book covers OpenGL, which is really important to understand if you want to create 3D graphics or complex animations Although you can get away without knowing any OpenGL when working with Processing, it’s much more commonly used in oF applications, and any complex blending and drawing will most likely require it If you’re not familiar with vector math and linear algebra, then continuing to study a little more wouldn’t hurt Vector math is a very important topic to understand in graphics and computation I would recommend that anyone seriously interested in learning more about graphics programming study up on matrix math Beyond math, there are many algorithmic tricks to learn that can help you figure out how to things that you might not know how to The Graphics Gems series of books is an interesting What’s Next | 333 and important series that contains a lot of the core algorithms used in graphics programming I wouldn’t recommend that you buy the books, though, but I would recommend that you go to the online code repository and download the free source code there; it’s worth browsing All the books that have been published about Processing are excellent resources for learning more about graphics in general and Processing in particular In addition, the oF and Processing forums are both active and helpful Andrew Glassner is one of the giants of computer graphics, and in addition to the academic works for which he is best known, he has written three introductory books on graphics that might be interesting to you: Andrew Glassner’s Notebook (Morgan Kaufmann), Andrew Glassner’s Other Notebook (AK Peters), and Morphs, Mallards, and Montages (AK Peters) These are a less technical read and are full of interesting techniques and ideas Finally, you should look at the graphics of applications that you like, both on the Processing exhibition page of the Processing site, where many wonderful projects are highlighted, and on the oF website, where several of the most interesting projects are profiled along with example code Find artists using Processing, oF, other graphics applications such as Flash, Director, and even animation, and think about their graphics and animations both in terms of what makes them visually appealing and in terms of their construction Review There are two coordinates to consider when working with graphics in the screen: the Cartesian coordinate set used in mathematics and the screen coordinate set used when placing pixels on the screen Points on the screen are represented by two-dimensional or three-dimensional points Forces on the screen are represented as two or three-dimensional vectors, that is, a speed and a direction A few common strategies will help you organize your drawing code more effectively and create more efficient code: draw using loops, use arrays to store values you’ll need for an animation, and use sprite objects that can handle their own drawing routines Processing and oF use matrices to transform the drawing space that your graphics objects have been put into This is an easy and efficient way to assign the positions and rotations of many objects at once There are two simple descriptions for motion: organic and mechanical Organic motion has elements of randomness in it, accelerations and decelerations, while mechanical motion is more constant and repetitive Using tweening equations, like the ShapeTween library for Processing, helps you create more natural movement that has interesting acceleration and deceleration characteristics 334 | Chapter 9: Programming Graphics Vectors allow you to assign speed, assign direction, and check for collisions between graphical sprites The PVector class in Processing and the ofxVectorMath library for oF provide many of the mathematical methods that you’ll need in order to calculate positions and manipulate vectors Graphical controls such as sliders, buttons, scrollbars, and dials are a powerful and familiar way for users to give input and also for you to tune an animation or graphic Processing users should check out the ControlP5 library; oF users will want to look at the ofxSimpleGUI library Particles are another powerful strategy for working with graphics A particle system consists of three elements: the collection of particles, the emitter for the particles that places them on the screen, and the physics calculations that determine what those particles should once they are on the screen Graphics can also be imported and exported using SVG or EPS file formats that save out vector graphics The import and export of vector graphics is included in Processing, while in oF that functionality is provided by the ofxVectorGraphics library To save out ps files in Processing create a new instance of the SimplePostscript class When you call the open() method on the file it will create a ps file in the folder of the application The close() method of the SimplePostscript class writes the file The ofxVectorGraphics uses the beginEPS() and endEPS() in a similar way to begin and finish writing the file Review | 335 ... Debugger in Code::Blocks Review 15 4 15 5 15 5 15 6 15 7 15 7 15 8 15 9 16 0 16 6 16 6 16 8 16 8 17 1 17 2 17 4 17 6 17 6 17 8 18 0 18 1 18 1 18 2 18 2 18 2 18 3 18 3 18 4 18 4 18 4 18 5 18 6 18 8 18 8 Part II Themes Sound and Audio... Review 96 97 97 97 10 2 10 5 10 6 10 6 10 8 10 9 11 0 11 1 11 2 11 5 11 7 12 2 12 4 12 6 12 6 12 7 Programming Revisited 12 9 Object-Oriented Programming Classes... This? Review 12 9 13 0 13 1 13 2 13 3 13 5 13 7 13 9 14 0 14 2 14 4 14 6 14 6 14 7 14 8 14 9 15 0 15 1 openFrameworks 15 3 Your IDE and Computer 15 4 Table of