Ch07 seam carving

Chapter – Seam Carving 7-1 Chapter Seam Carving Michael Rubinstein, MIT Shai Avidan, Mitsubishi Electric Research Labs Ariel Shamir, The Interdisciplinary Center & MERL Department of Mechatronics Chapter – Seam Carving 7-2 Contents Image resizing • Seam Carving algorithm Department of Mechatronics Chapter – Seam Carving 7-3 Display Devices Department of Mechatronics Chapter – Seam Carving 7-4 Content Retargeting PC Department of Mechatronics iPhone Chapter – Seam Carving 7-5 Page Layout Department of Mechatronics Chapter – Seam Carving 7-6 Simple Media Retargeting Operators Letterboxing Scaling ? Department of Mechatronics ? Chapter – Seam Carving 7-7 Content-aware Retargeting Operators “Important” content Contentaware Contentoblivious Department of Mechatronics Chapter – Seam Carving 7-8 Content-aware Retargeting Scale Input Department of Mechatronics “less-Important” content Crop Content-aware Chapter – Seam Carving 7-9 Image Retargeting • Problem statement: ã Input Image I with size n ì m, and new size n ì m ã Output Image of size n’ × m’ which will be “good representative” of the original image I • To date, no agreed definition, or measure, as to what a good representative is in this context! Department of Mechatronics Chapter – Seam Carving 7-10 Image/Video Retargeting • In large, we would expect: Adhere to the geometric constraints (display/aspect ratio) Preserve the important content and structures Limit artifacts Perhaps a new representation that will support different sizes? • Very ill-posed! o How we define important? Is there a universal ground truth? o Would different viewers think the same about a retargeted image? o What about artistic impression in the original content? Department of Mechatronics 7-36 Chapter – Seam Carving Changing Aspect Ratio Scale Department of Mechatronics 7-37 Chapter – Seam Carving Changing Aspect Ratio /Scale Department of Mechatronics 7-38 Chapter – Seam Carving Content Enhancement Department of Mechatronics 7-39 Chapter – Seam Carving Seam Carving in the Gradient Domain Department of Mechatronics Chapter – Seam Carving 7-40 Preserved Energy Energy 10% 30% 40% 75% While resizing: remove as many low energy pixels and as few high energy pixels! Department of Mechatronics Chapter – Seam Carving 7-41 Preserved Energy Average Pixel Energy Image Reduction If we measure the average energy of pixels in the image after applying a resizing operator… …the average should increase! While resizing: remove as many low energy pixels and as few high energy pixels! Department of Mechatronics Chapter – Seam Carving 7-42 Preserved Energy Average Pixel Energy Image Reduction crop Department of Mechatronics column seam pixel optimal Chapter Seam Carving 7-43 Both Dimensions? ã m ì n m ì n ã ã ã ã Remove horizontal seam first? Remove vertical seams first? Alternate between the two? The optimal order can be found!  Dynamic Prog (again) Department of Mechatronics Chapter – Seam Carving 7-44 Retargeting in Both Dimensions • The recursion relation: Department of Mechatronics Chapter – Seam Carving 7-45 Optimal Order Map Removal of horizontal seams Removal of vertical seams 13 16 19 16 17 22 28 19 31 25 35 24 28 29 ??? 32 35 33 41 38 35 Department of Mechatronics Chapter – Seam Carving 7-46 Optimal? • In a greedy sense… • Seam costs are not absolute Horizontal Seams o Consider HVV (how many possibly orderings?) o Cost(V) on HV not necessarily equal Cost(V) on VH o But we keep track of only one: min(HV,VH) … Department of Mechatronics Vertical Seams V VV H min(HV,VH) ? Chapter – Seam Carving 7-47 Image Expansion (Synthesis) Department of Mechatronics Chapter – Seam Carving 7-48 Image Expansion – take Scaling Department of Mechatronics 7-49 Chapter – Seam Carving Enlarged or Reduced? Department of Mechatronics Chapter – Seam Carving 7-50 Combined Insert and Remove Insert & remove seams Scaling Department of Mechatronics