mobile image processing

Bernd Girod, David Chen, Derek Pang: EE368 Mobile Image Processing no. 1 Mobile image processing  Mobile application development on Android  Project 1: Contrast enhancement on viewfinder frames  Project 2: Augmentation of viewfinder frames  Project 3: Utilizing functions in the OpenCV library Bernd Girod, David Chen, Derek Pang: EE368 Mobile Image Processing no. 2 http://www.stanford.edu/~gtakacs/publication.html Mobile landmark recognition Bernd Girod, David Chen, Derek Pang: EE368 Mobile Image Processing no. 3 http://www.stanford.edu/~sstsai/research.html Mobile product recognition Bernd Girod, David Chen, Derek Pang: EE368 Mobile Image Processing no. 4 Spotlight on Android  Open source mobile platform developed by Google  Supported by Open Handset Alliance  13 mobile operators  22 handset manufacturers  20 semiconductor companies  17 software makers  Uses an open-source kernel and a virtual machine designed for mobile hardware  Largest share of smartphone market in the world  Android Market contains over 500k apps Bernd Girod, David Chen, Derek Pang: EE368 Mobile Image Processing no. 5 Distribution of worldwide smartphone sales Source: World Smartphone Sales to End Users, Gartner (Nov. 2011) 16.9% 15.0% 11.0% 4.6% 52.5% Android Symbian iOS RIM Other Bernd Girod, David Chen, Derek Pang: EE368 Mobile Image Processing no. 6 Android software stack [ Professional Android Application Development ] Linux Kernel Bernd Girod, David Chen, Derek Pang: EE368 Mobile Image Processing no. 7 Programming in Java  Android encourages high-level app development  Android uses Java as the main programming language  Inherits basic classes from conventional Java  String, Container, Math, IO, Network, …  Adds new classes specific to mobile devices  Camera, Telephony, Map, GPS, Speech, … Bernd Girod, David Chen, Derek Pang: EE368 Mobile Image Processing no. 8 Android programming with Eclipse Android SDK Java Runtime program installation and execution system messages programmer inputs data and messages Internet Eclipse IDE Bernd Girod, David Chen, Derek Pang: EE368 Mobile Image Processing no. 9 Class tutorials for mobile image processing  Android tutorials designed specifically for mobile image processing in the EE368 class  Tutorial #1: Basic Android Setup  Image processing-oriented introduction to Android  Explains how to download and install the different software packages (JDK, SDK, Eclipse) on your own computer  Shows how to build and run a viewfinder app that computes color histograms in real time  Tutorial #2: OpenCV for Android Setup  Builds on core skills developed in Tutorial #1  Explains how to download and install OpenCV for Android  Shows how to build viewfinder apps that detect circles and lines, detect feature keypoints, and track feature keypoints Bernd Girod, David Chen, Derek Pang: EE368 Mobile Image Processing no. 10 Class tutorials for mobile image processing http://ee368.stanford.edu/Android Support for Windows, Macintosh, and Linux in both tutorials [...]...Class tutorials for mobile image processing http://ee368.stanford.edu/Android Bernd Girod, David Chen, Derek Pang: EE368 Mobile Image Processing no 11 Different perspectives Text editor Class hierarchy Project files Errors and warnings Bernd Girod, David Chen, Derek Pang: EE368 Mobile Image Processing no 12 Structure of an Android project Programmer-defined... mPaintBlack); } Bernd Girod, David Chen, Derek Pang: EE368 Mobile Image Processing no 21 Running the program on an Android device Bernd Girod, David Chen, Derek Pang: EE368 Mobile Image Processing no 22 “Hello Viewfinder” application http://ee368.stanford.edu/Android/HelloViewfinder Bernd Girod, David Chen, Derek Pang: EE368 Mobile Image Processing no 23 Class Project: Plant Leaf Classification D... David Chen, Derek Pang: EE368 Mobile Image Processing no 16 “Hello Viewfinder” class hierarchy Hello Viewfinder (Activity) Preview (View) Handle incoming viewfinder frames and adjust camera parameters Bernd Girod, David Chen, Derek Pang: EE368 Manage two views and manage program open/close Draw on Top (View) Process viewfinder frames and draw modified frames Mobile Image Processing no 17 Preview class:... David Chen, Derek Pang: EE368 Mobile Image Processing no 19 “Hello Viewfinder” class hierarchy Hello Viewfinder (Activity) Preview (View) Handle incoming viewfinder frames and adjust camera parameters Bernd Girod, David Chen, Derek Pang: EE368 Manage two views and manage program open/close Draw on Top (View) Process viewfinder frames and draw modified frames Mobile Image Processing no 20 Draw On Top:... Chen, Derek Pang: EE368 Mobile Image Processing no 13 “Hello Viewfinder” project  Goals of this project     Learn how to create a simple Android project Learn how to display viewfinder frames Learn how to process viewfinder frames Full source available on class website:  http://ee368.stanford.edu/Android/HelloViewfinder Bernd Girod, David Chen, Derek Pang: EE368 Mobile Image Processing no 14 “Hello... and M Potter, Spring 2010 http://ee368.stanford.edu/Project_10 Bernd Girod, David Chen, Derek Pang: EE368 Mobile Image Processing no 24 Class Project: Jigsaw Puzzle Solver L Liang and Z Liu, Spring 2010 http://ee368.stanford.edu/Project_10 Bernd Girod, David Chen, Derek Pang: EE368 Mobile Image Processing no 25 ... { public void onPreviewFrame(byte[] data, Camera camera) { Pass data to DrawOnTop class } }); Data in YCbCr 4:2:0 format h/2 h w/2 w/2 h/2 w Cb Cr Y Bernd Girod, David Chen, Derek Pang: EE368 Mobile Image Processing no 18 Preview class: toggle states via touch screen // Define on touch listener this.setOnTouchListener(new OnTouchListener() { public boolean onTouch(View v, MotionEvent event) { if... frames and adjust camera parameters Bernd Girod, David Chen, Derek Pang: EE368 Manage two views and manage program open/close Draw on Top (View) Process viewfinder frames and draw modified frames Mobile Image Processing no 15 Hello Viewfinder Activity class public class HelloViewfinderActivity extends Activity { private Preview mPreview; private DrawOnTop mDrawOnTop; Subclass of Activity class will be