mobile image processing part2

Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 2 no. 1 Mobile image processing – Part 2  Mobile application development on Android  Project 1: “Hello Image”, reading/writing/modifying images  Project 2: “Viewfinder EE368”, processing viewfinder frames  Project 3: “CVCamera”, utilizing OpenCV functions  Mobile application examples Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 2 no. 2 Recognizing video at a glance (1) User snaps a photo of screen. (2) Our system identifies video and frame within the video. (3) User resumes video on the phone. Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 2 no. 3 http://www.stanford.edu/~dmchen/mvs.html Recognizing video at a glance Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 2 no. 4 Mobile device versus personal computer Feature Motorola DROID Typical PC Screen size 4.6 in × 2.4 in 25 in × 16 in Processor speed 550 MHz 3.0 GHz RAM 256 MB 4 GB Permanent storage 133 MB internal 8 GB external flash 500 GB disk Internet access Typically 3G or WiFi at hotspots WiFi or wired Ethernet Telephony Core feature Supplementary feature Camera 5 MP camera embedded External webcam Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 2 no. 5 “Viewfinder EE368” project  Goals of this project z Learn how to access frames from the viewfinder z Learn how to modify camera parameters z Learn how to augment the viewfinder frames  Step-by-step instructions available in Tutorial #1: z http://ee368.stanford.edu/Android Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 2 no. 6 Android manifest file … … <application android:icon="@drawable/icon" android:label="@string/app_name"> <activity android:name=".ViewfinderEE368" android:label="@string/app_name" android:screenOrientation="landscape“ android:theme="@android:style/Theme.NoTitleBar"> <intent-filter> <action android:name="android.intent.action.MAIN" /> <category android:name="android.intent.category.LAUNCHER" /> </intent-filter> </activity> </application> <uses-sdk android:minSdkVersion=“7" /> <uses-permission android:name="android.permission.CAMERA"/> Set the proper hardware permissions Landscape mode without title bar Main Activity Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 2 no. 7 Class hierarchy for “Viewfinder EE368” Viewfinder EE368 (Activity) Preview (View) Draw on Top (View) Manage two views and manage program open/close Handle incoming viewfinder frames and adjust camera parameters Augment viewfinder frames with a new layer of information on top Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 2 no. 8 Viewfinder class: full screen mode Viewfinder EE368 Icon and title bars visible Icon and title bars hidden public void onCreate(Bundle savedInstanceState) { getWindow().setFlags( WindowManager.LayoutParams.FLAG_FULLSCREEN, WindowManager.LayoutParams.FLAG_FULLSCREEN ); requestWindowFeature(Window.FEATURE_NO_TITLE); } Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 2 no. 9 Class hierarchy for “Viewfinder EE368” Viewfinder EE368 (Activity) Preview (View) Draw on Top (View) Manage two views and supervise program open/close Handle incoming viewfinder frames and adjust camera parameters Augment viewfinder frames with a new layer of information on top Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 2 no. 10 Timeline of events on the mobile device Time New frame New frame Capture Augment Preview DrawOnTop Capture Augment [...]... Mobile Image Processing – Part 2 no 21 Class Project: Mobile Image Completion H Wang and A Lin, Spring 2010 http://ee368.stanford.edu/Project_10 Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 2 no 22 Class Project: Mobile HDR Imaging J Mathe and T Wong, Spring 2010 http://ee368.stanford.edu/Project_10 Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 2 no 23 Mobile image processing. .. EE368 Mobile Image Processing – Part 2 no 18 Real-time debugging using DDMS (2) 0123456789ABC Device File system on device Messages on the device Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 2 no 19 Taking a screenshot of the phone (1) 0123456789ABC Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 2 no 20 Taking a screenshot of the phone (2) Bernd Girod, David Chen: EE368 Mobile. .. EE368 Mobile Image Processing – Part 2 no 15 DrawOnTop class: draw histogram bars Bin 0 Bin 1 Bin 2 Bin 3 Bin 4 // Draw red rectangle Rect rect = new Rect(left x, top y, right x, bottom y); myCanvas.drawRect(rect, myPaintRed); Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 2 no 16 Viewfinder EE368 demo http://ee368.stanford.edu/Android Bernd Girod, David Chen: EE368 Mobile Image Processing. .. Processing – Part 2 no 23 Mobile image processing – Part 2 Mobile application development on Android Project 1: “Hello Image , reading/writing/modifying images Project 2: “Viewfinder EE368”, processing viewfinder frames Project 3: “CVCamera”, utilizing OpenCV functions Mobile application examples Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 2 no 24 ... Girod, David Chen: EE368 Mobile Image Processing – Part 2 no 12 Class hierarchy for “Viewfinder EE368” Viewfinder EE368 (Activity) Manage two views and manage program open/close Preview (View) Draw on Top (View) Handle incoming viewfinder frames and adjust camera parameters Augment viewfinder frames with a new layer of information on top Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 2 no... PreviewCallback() { public void onPreviewFrame(byte[] data, Camera camera) { Pass data to DrawOnTop class } }); Data in YCbCr 4:2:0 format h/2 h w/2 w/2 h/2 w Cb Cr Y Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 2 no 11 Preview class: set camera parameters Preview (View) Camera.Parameters parameters = myCamera.getParameters(); parameters.setPreviewSize(640, 480); parameters.setPreviewFrameRate(15);... w Cr RGB 298.08 ⋅ Y 408.58 ⋅ Cr R= + − 222.921 256 256 298.08 ⋅ Y 100.29 ⋅ Cb 208.12 ⋅ Cr G= − − + 135.58 256 256 256 298.08 ⋅ Y 516.41 ⋅ Cb B= + − 276.84 256 256 Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 2 no 14 DrawOnTop class: create some paint brushes myPaintRed = new Paint(); myPaintRed.setStyle(Paint.Style.FILL); myPaintRed.setColor(Color.RED); myPaintRed.setTextSize(25);