mobile image processing part 3

Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 3 no. 1 Mobile image processing – Part 3  Mobile application development on Android  Project 1: “Hello Image”, reading/writing/modifying images  Project 2: “Viewfinder EE368”, processing viewfinder frames  Project 3: “CVCamera”, utilizing OpenCV functions  Mobile application examples Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 3 no. 2 http://www.stanford.edu/~gtakacs/publication.html Mobile feature tracking Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 3 no. 3 http://www.stanford.edu/~dmchen/mvs.html Mobile book spine recognition Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 3 no. 4 Mobile book spine recognition Server Mobile Device Track Camera Pose … Low Motion High Motion Send Query Frame Query Vocab. Tree Check Geometry Send Response Time Wireless Network Viewfinder Frames Signal Processing and Linear Systems Extract Features Segment Spines Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 3 no. 5 Jon’s Java Imaging Library (JJIL)  Publicly available source code http://code.google.com/p/jjil/  Collection of common image processing functions  Images are processed through Sequences Rgb 3x3Average Rgb SubSample Sequence RgbImage RgbImage Sequence.Add (new Rgb3x3Average) Sequence.Add (new RgbSubSample) Sequence.Push (RgbImage) Sequence.GetFront() Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 3 no. 6 JJIL: 3x3 averaging filter Original Image Processed Image Rgb3x3Average Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 3 no. 7 JJIL: Horizontal differences Original Image Processed Image RgbAvgGray Gray8HorizSimpleEdge Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 3 no. 8 JJIL functions ApplyMaskRgb BinaryHeap Complex32Gray32 Complex32IFFT Fft1d Gray16Gray8 Gray16LinComb Gray16Threshold Gray32Div Gray32Gray8 Gray32Scale2Gray8 Gray32Threshold Gray3Bands2Rgb Gray82Gray32 Gray8Abs Gray8Add Gray8And Gray8CannyHoriz Gray8ConnComp Gray8Crop Gray8DeblurHorizHalftone Gray8DetectHaarMultiScale Gray8Fft Gray8GaussDeblurHoriz Gray8GaussHoriz Gray8GaussSmoothVert Gray8Gray32 Gray8Hist Gray8HistEq Gray8HistMatch Gray8HorizSimpleEdge Gray8HorizSum Gray8HorizVar Gray8HorizVertContrast Gray8InverseFilter Gray8LinComb Gray8Peak3x3 Gray8Reduce Gray8Shrink LinefitHough Rgb3x3Average RgbAvgGray RgbClip RgbCrop RgbDimMask RgbHorizGaussSmooth RgbHsv RgbMaskedAbsDiff RgbMaxContrast2Gray RgbMaxDiff RgbMeanStdDev RgbMinDiff RgbSelectGray RgbSubSample … Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 3 no. 9 “CVCamera” project  Goals of this project z Learn how to incorporate C/C++ code into an Android program z Learn how to utilize OpenCV library functions z Learn how to draw feature keypoints on viewfinder frames  Step-by-step instructions available in Tutorial #2: z http://ee368.stanford.edu/Android Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 3 no. 10 OpenCV: Open Source Computer Vision http://opencv.willowgarage.com/wiki/Welcome [...]... EE368 Mobile Image Processing – Part 3 no 22 Class Project: Mobile Panorama Creation T Chu, B Meng, and Z Wang, Spring 2010 http://ee368.stanford.edu/Project_10 Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 3 no 23 Class Project: Mobile Face Recognition G Davo K Sriadibhatla, and X Chao, Spring 2010 http://ee368.stanford.edu/Project_10 Bernd Girod, David Chen: EE368 Mobile Image Processing. .. David Chen: EE368 Mobile Image Processing – Part 3 no 19 CVCamera demo http://ee368.stanford.edu/Android Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 3 no 20 Communications between phone and server Server Phone HTTP Post (e.g., Image, Features) Wireless Network HTTP Get (e.g., Identification) Apache PHP MATLAB / C Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 3 no 21 HTTP... delete_Processor(long jarg1); Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 3 no 16 Project structure for “CVCamera” CVCamera Activity (Java) Manages user interface and program execution CVCamera JNI (Java) Interfaces with C/C++ code Processor (C++) Processes a “pool” of images OpenCV Library Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 3 no 17 Processor class: initialize feature... into Eclipse IDE Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 3 no 11 Project structure for “CVCamera” CVCamera Activity (Java) Manages user interface and program execution CVCamera JNI (Java) Interfaces with C/C++ code Processor (C++) Processes a “pool” of images OpenCV Library Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 3 no 12 CVCamera class: menu options public... Bernd Girod, David Chen: EE368 callback Feature extraction callbacks Mobile Image Processing – Part 3 no 14 Project structure for “CVCamera” CVCamera Activity (Java) Manages user interface and program execution CVCamera JNI (Java) Interfaces with C/C++ code Processor (C++) Processes a “pool” of images OpenCV Library Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 3 no 15 CVCamera JNI class:... Chen: EE368 Mobile Image Processing – Part 3 no 24 Android resources Developer website http://developer.android.com Class tutorials http://ee368.stanford.edu/Android Android-related office hours Wednesday, 3: 30pm – 5:00pm, SCIEN Lab Reference book Meier, Professional Android 2 Application Development Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 3 no 25 ... parameters), my_mserd(MSER detector parameters) {} void detectAndDrawFeatures (int idx, image_ pool* pool, int feature_type); }; Bernd Girod, David Chen: EE368 list Mobile Image Processing – Part 3 no 18 Processor class: detect and draw keypoints // Detect feature keypoints Mat grey_im = pool->getGrey(idx); Mat color_im = pool->getImage(idx); my_keypoints.clear(); my_mserd->detect(grey_im, my_keypoints); // Draw... onCreateOptionsMenu(Menu menu) { menu.add("FAST"); menu.add("STAR"); menu.add("SURF"); menu.add("MSER"); menu.add("Chess"); menu.add("Settings"); return true; } Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 3 no 13 CVCamera class: calling feature extractors public boolean onOptionsItemSelected(MenuItem item) { Stack of callback LinkedList defaultCallbackStack = functions new LinkedList();...OpenCV for Android Port of OpenCV to Android framework Over 2K optimized algorithms written in C/C++ Compiled with customized Android NDK (Crystax r4) Enables popular CV functions to be used on mobile images/videos Differences from regular Android programming Requires writing C/C++ code Requires writing Java Native Interface (JNI) wrappers Requires using Android NDK in addition to Android SDK Class