New strategies for generating panoramic images for imperfect image series 8

Appendix A Intermediate Result of Seam Driven Stitching Approach In this supplemental material, we show the input images and intermediate results for the panoramas shown in Figure 4.6 in Chapter 4. The intermediate steps show the image alignment and the seam-cuts. Input images (#1) Input images (#2) Figure A.1: Two sets of input images used in Chapter 4. 87 Input images CHAPTER A. Intermediate Result of Seam Driven Stitching Approach Photoshop alignment Photoshop seam-cut Our alignment Our seam-cut 88 Figure A.2: Intermediate results for image set 1. Input images Photoshop alignment Photoshop seam-cut Our alignment Our seam-cut Figure A.3: Intermediate results for image set 2. 89 CHAPTER A. Intermediate Result of Seam Driven Stitching Approach 90 Appendix B Interface of the Interactive Panorama Correction Tool For the interactive correcting tool ,we developed a simple dialog based interface to help user switch between each mode and provide some visualized function for the users. Figure B.1 shows the design of this control dialog. This dialog is popped Figure B.1: The control dialog for the interactive panorama correction tools. 91 CHAPTER B. Interface of the Interactive Panorama Correction Tool (a) (b) (c) Figure B.2: An example of the display window in seam-editing mode. (a) The composite image with seam. (b) The content of the left side image of the overlapping region. (c)The content of the right side image of the overlapping region. up accompanied with the initial panorama in the result display window. From the figure we can see that there are three main modes for the post editing: observe, seam editing and warping. The user can switch among these modes by simply clicking the corresponding radio buttons. In observe mode, the user is enabled to drag and scale the whole panorama to a desired position. This helps user to find minor artifacts in the monitoring window. The seam editing mode and the warping mode are corresponding to the two introduced tools described in Chapter 5. When the seam-editing mode is selected, the shown panorama in the display window is dimmed out and the focusing overlapping region is highlighted. An example of seam-editing mode is shown in Figure B.2, we can see that the cursor is changed into a brush and can be used to mark the overlapping region by holding the left button of the mouse down. After the mouse button is released, the drawn stroke is considered as the input mark-up and the overlapping region is updated using the method described 92 (a) (b) Figure B.3: An example of the display window in warp mode. (a) The panorama before warping. There exists noticeable artifacts. The greed arrow indicates the moving direction of the brush. (b) The result after warping. in Section 5.2.2. At the same time, in seam-editing mode, when the user clicks the right button of the mouse, the overlapping region will be toggled among the left input image, right input image and the composite result as shown in B.2 (b) and (c). When the warp mode is selected, the focusing object is no more the overlapping region but the input image. Thus, the whole focusing image is highlighted with its neighboring overlapping region. Figure B.3 shows an example of the warp mode. The focusing image can be changed by clicking the “prev” and “next” button on the dialog. Similar as the seam-editing tool, a circular brush is used to warp the images. The user first adjusts the brush into a desired size. Then by holding the left button of the mouse down, the user can push the region inside the brush to a desired position. The snapping described in Section 5.2.3 is triggered when the brush has intersected with the cutting seam. 93 CHAPTER B. Interface of the Interactive Panorama Correction Tool 94 Appendix C Input Images for All Examples used in This Thesis Figure C.1: Input image series for Figure 3.9 Figure C.2: Input image series for Figure 3.10 95 CHAPTER C. Input Images for All Examples used in This Thesis Figure C.3: Input image series for Figure 3.11 Figure C.4: Input image series for Figure 4.1 Figure C.5: Input image series for Figure 5.6 Figure C.6: Input image series for Figure 5.7 96 Figure C.7: Input image series for Figure 5.8 97 CHAPTER C. Input Images for All Examples used in This Thesis 98 Bibliography [1] Autostitch, http://cvlab.epfl.ch/ brown/autostitch/auto-stitch.html. 1.1, 1.3, 2.1, 2.4.1, 2.4, 2.6, 3.5, 5.3 [2] Microsoft Image Composite Editor, http://research.micro- soft.com/enus/um/redmond/groups/ivm/ICE/. 1.1, 1.2, 1.3, 2.1, 2.4.2, 2.6, 3.5, 5.3 [3] A. Agarwala, M. Dontcheva, M. Agrawala, S. Drucker, A. Colburn, B. Curless, D. Salesin, , and M. Cohen. Interactive digital photo-montage. ACM Transactions on Graphics (TOG), 23(3):294–302, 2004. 1.2, 2.1, 2.4.1, 2.4.2, 2.4.2, 2.4.2, 2.5, 3.5, 5.2.1 [4] P. Anandan, P.J. Burt, K. Dana, M.W. Hansen, and G.S. van der Wal. Real-time scene stabilization and mosaic construction. 1994. 2.1 [5] Rakesh Kumar Anandan, Rakesh Kumar, P. An, Michal Irani, James Bergen, and Keith Hanna. Representation of scenes from collections of images. In In IEEE Workshop on Representations of Visual Scenes, pages 10–17, 1995. 2.1 [6] Shai Avidan and Ariel Shamir. Seam carving for content-aware image resizing. ACM Transactions on Graphics (SIGGRAPH), 26(3):10, 2007. 2.5 99 BIBLIOGRAPHY [7] Fady Badra, Ala Qumsieh, and Gregory Dudek. Rotation and zooming in image mosaicing. In IEEE Workshop on Applications of Computer Vision (WACV), pages 50–55, 1998. 2.2 [8] Connelly Barnes, Eli Shechtman, Adam Finkelstein, and Dan B Goldman. PatchMatch: A randomized correspondence algorithm for structural image editing. ACM Transactions on Graphics (SIGGRAPH), 28(3). 2.5 [9] Adrien Bartoli, Mathieu Perriollat, and Sylvie Chambon. Generalized thinplate spline warps. Internal Journal of Computer Vision (IJCV), 88(1):85–110, 2010. 2.5 [10] Sebastiano Battiato, Giovanni Gallo, Giovanni Puglisi, Salvatore Scellato, and Scuola Superiore Di Catania. Sift features tracking for video stabilization. In In IEEE International conference on Image Analysis and Processing (ICIAP), 2007. 2.1 [11] Adam Baumberg. Reliable feature matching across widely separated views. In IEEE International Conference on Computer Vistion and Pattern Recognision (CVPR), pages 774–781, 2000. 2.2 [12] James R. Bergen, P. Anandan, Keith J. Hanna, and Rajesh Hingorani. Hierarchical model-based motion estimation. In European Conference on Computer Vision (ECCV), pages 237–252, 1992. 2.1 [13] Yuri Boykov, Olga Veksler, and Ramin Zabih. Fast approximate energy minimization via graph cuts. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 23:1222–1239, 2001. 2.4.2, 2.5, 4.2.2 100 BIBLIOGRAPHY [14] Lisa Gottesfeld Brown. A survey of image registration techniques. ACM Computing Surveys, 24:325–376, 1992. 2.1 [15] M. Brown and D. Lowe. Recognising panoramas. In International Conference on Computer Vision (ICCV), 2003. 1.2, 2.1, 2.2, 2.3.2, 3.5, 5.2.1 [16] Matthew Brown and David G. Lowe. Automatic panoramic image stitching using invariant features. Internal Journal of Computer Vision (IJCV), 74:59–73, 2007. 1.2, 1.2, 2.1, 2.2, 2.3.2, 2.4.1, 3.3.1, 3.4.1, 4.2.1, 6.2 [17] Peter J. Burt and Edward H. Adelson. A multiresolution spline with application to image mosaics. ACM Transactions on Graphics (SIGGRAPH), 2(4):217– 236, 1983. 1.2, 2.4.1 [18] P.J. Burt and E.H. Adelson. A multiresolution spline with applications to image mosaics. ACM Transactions on Graphics (TOG), 2(4):217–236, 1983. 2.5 [19] D. Capel and A. Zisserman. Automated mosaicing with super-resolution zoom. In IEEE International Conference on Computer Vistion and Pattern Recognision (CVPR), 1998. 2.2 [20] Shenchang Eric Chen. Quicktime vr - an image-based approach to virtual environment navigation. In ACM SIGGRAPH, pages 26–33, 1995. 1.2, 2.1, 2.2, 2.3.3 [21] Jason J. Corso and Gregory D. Hager. Coherent regions for concise and stable image description. In IEEE International Conference on Computer Vistion and Pattern Recognision (CVPR), pages 184–190, 2005. 2.2 101 BIBLIOGRAPHY [22] Alexei A. Efros and Thomas K. Leung. Texture synthesis by non-parametric sampling. In International Conference on Computer Vision (ICCV), pages 1033– 1038, 1999. 2.5 [23] Raanan Fattal, Dani Lischinski, and Michael Werman. Gradient domain high dynamic range compression. ACM Transactions on Graphics (SIGGRAPH), 21(3):249–256, 2002. 2.4.1 [24] Martin A. Fischler and Robert C. Bolles. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM, 24:381–395, June 1981. 2.2.2, 3.3.1, 4.2, 5.2.1 [25] William T. Freeman and Edward H. Adelson. The design and use of steerable filters. IEEE Transaction on Pattern Analysis and Machine Intelligence, 13(9):891– 906, 1991. 2.2 [26] Junhong Gao and Michael S. Brown. An interactive image editing tool for correcting panoramas. In SIGGRAPH Asia -Technical Brief, 2012. 1.4 [27] Junhong Gao, Seon Joo Kim, and Michael S. Brown. Constructing image panoramas using dual-homography warping. In IEEE International Conference on Computer Vistion and Pattern Recognision (CVPR), pages 49–56, 2011. 1.4 [28] C A Glasbey and K V Mardia. A review of image-warping methods. Journal of Applied Statistics, 25(2):155–171, 1998. 2.5 [29] A. Andeshir Goshtasby. 2-D and 3-D Image Registration: for Medical, Remote Sensing, and Industrial Applications. Wiley-Interscience, 2005. 2.1 102 BIBLIOGRAPHY [30] R. Hartley and A. Zisserman. Multiview Geometry in Computer Vision. Cambridge University Press, 2004. 2.3.1 [31] Michal Irani, P. Anandan, and Steven C. Hsu. Mosaic based representations of video sequences and their applications. In International Conference on Computer Vision (ICCV), pages 605–611, 1995. 2.1 [32] Tat jen Cham and Roberto Cipolla. A statistical framework for long-range feature matching in uncalibrated image mosaicing. In IEEE International Conference on Computer Vistion and Pattern Recognision (CVPR), pages 442–447. IEEE Computer Society, 1998. 2.2 [33] J. Jia and C.-K. Tang. Image stitching using structure deformation. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 30(4):617– 631, 2008. 2.5 [34] Tat-Jun Chin Michael S. Brown Junhong Gao, Li Yu. Seam-driven image stitching. In EuroGraphics (EG) short, 2013. 1.4 [35] Ming-Chieh Lee, Wei-Ge Chen, C. B. Lin, Chuang Gu, T. Markoc, S. I. Zabinsky, and R. Szeliski. A layered video object coding system using sprite and affine motion model. IEEE Trans. Cir. and Sys. for Video Technol., 7(1), 1997. 2.1 [36] Anat Levin, Assaf Zomet, Shmuel Peleg, and Yair Weiss. Seamless image stitching in the gradient domain. In European Conference on Computer Vision (ECCV), volume 3024 of Lecture Notes in Computer Science, pages 377–389. Springer, 2004. 1.2, 2.1, 2.4.1, 6.2 [37] M. Lhuillier, L. Quan, H. Shum, , and H.T. Tsui. Relief mosaics by joint view 103 BIBLIOGRAPHY triangulation. In IEEE International Conference on Computer Vistion and Pattern Recognision (CVPR), 2001. 2.5 [38] S. Li. Markov Random Field Modeling in Image Analysis (2nd Edition). SpringerVerlag, 2001. 1.2 [39] Yin Li, Jian Sun, Chi keung Tang, and Heung yeung Shum. Lazy snapping. ACM Transactions on Graphics (TOG), 23:303–308, 2004. 2.4.2, 2.5 [40] Andrew Litvin, Janusz Konrad, and William C. Karl. Probabilistic video stabilization using kalman filtering and mosaicking. In In Symposium on Electronic Imaging, Image and Video Communications and Proc, pages 663–674, 2003. 2.1 [41] J.Y. Liu, J. Sun, and H.Y. Shum. Paint selection. ACM Transactions on Graphics (TOG), 69:1–7, 2009. 2.5 [42] David G. Lowe. Distinctive image features from scale-invariant keypoints. Internal Journal of Computer Vision (IJCV), 60:91–110, November 2004. 2.2, 2.2.1, 3.3.1, 4.2, 5.2.1 [43] Bruce D. Lucas and Takeo Kanade. An iterative image registration technique with an application to stereo vision. pages 674–679, 1981. 2.1 [44] Steve Mann and Rosalind W. Picard. Virtual bellows: Constructing high quality stills from video. In In IEEE International Conference on Image Processing, pages 363–367, 1994. 1.2, 2.1, 2.2 [45] Philip F McLauchlan and Allan Jaenicke. Image mosaicing using sequential bundle adjustment. Image and Vision Computing, pages 751–759, 1999. 2.1, 2.2 104 BIBLIOGRAPHY [46] Krystian Mikolajczyk and Cordelia Schmid. A performance evaluation of local descriptors. 2005. 2.2, 2.1 [47] Eric N. Mortensen and William A. Barrett. Intelligent scissors for image composition. In SIGGRAPH, 1995. 2.5 [48] S. Peleg and J. Herman. Panoramic mosaics by manifold projection. In IEEE International Conference on Computer Vistion and Pattern Recognision (CVPR), 1997. 2.5 [49] Patrick P´erez, Michel Gangnet, and Andrew Blake. Poisson image editing. ACM Transactions on Graphics (SIGGRAPH), 22(3):313–318, 2003. 1.2, 2.4.1 [50] PhotoShop. Adobe Photoshop CS5, http://www.adobe.com/products/photoshop. 1.1, 1.2, 1.3, 2.1, 2.4.2, 2.6, 3.5, 4.3, 5.3 [51] Y. Pritch, E. Kav-Venaki, and S. Peleg. Shift-map image editing. In IEEE International Conference on Computer Vision (ICCV), 2009. 2.5 [52] Carsten Rother, Vladimir Kolmogorov, and Andrew Blake. “GrabCut”: Interactive foreground extraction using iterated graph cuts. ACM Transactions on Graphics (TOG), pages 309–314, 2004. 2.4.2, 2.5 [53] Michael Rubinstein, Ariel Shamir, and Shai Avidan. Improved seam carving for video retargeting. ACM Transactions on Graphics SIGGRAPH), 27(3):1–9, 2008. 2.5 [54] Michael Rubinstein, Ariel Shamir, and Shai Avidan. Multi-operator media retargeting. ACM Transactions on Graphics (SIGGRAPH), 28(3):1–11, 2009. 2.5 105 BIBLIOGRAPHY [55] Harpreet Sawhney and Rakesh Kumar. True multi-image alignment and its application to mosaicing and lens distortion correction. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 1999. 2.1 [56] Scott Schaefer, Travis McPhail, and Joe D. Warren. Image deformation using moving least squares. ACM Transactions on Graphics (TOG), 25(3):533–540, 2006. 2.5 [57] Frederik Schaffalitzky and Andrew Zisserman. Multi-view matching for unordered image sets, or “how i organize my holiday snaps?”. In European Conference on Computer Vision (ECCV), pages 414–431, 2002. 2.2 [58] H.-Y. Shum and R. Szeliski. Construction of panoramic image mosaics with global and local alignment. Internal Journal of Computer Vision (IJCV), 36(2):101– 130, 2000. 2.4.2, 2.5 [59] Brian Summa, Julien Tierny, and Valerio Pascucci. Panorama weaving: fast and flexible seam processing. ACM Transactions on Graphics (SIGGRAPH), 2012. 5.2.2 [60] R. Szeliski. Fast surface interpolation using hierarchical basis functions. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 12(6):513–528, 1990. 2.4.1 [61] R. Szeliski. Image alignment and stitching: a tutorial. Foundations and Trends in Computer Graphics and Vision, 2:1–104, 2006. 1.2, 2.1, 2.1, 5.2.1 [62] Richard Szeliski. Image mosaicing for tele-reality applications. In IEEE Work- 106 BIBLIOGRAPHY shop on Applications of Computer Vision (WACV), pages 44–53. IEEE Computer Society, 1994. 2.3.1, 2.3.3 [63] Richard Szeliski. Video mosaics for virtual environments. IEEE Computer Graphics Application (CGA), 16(2):22–30, 1996. 2.3.1 [64] Richard Szeliski. Locally adapted hierarchical basis preconditioning. ACM Transactions on Graphics (SIGGRAPH), 25(3):1135–1143, 2006. 2.4.1 [65] Richard Szeliski and Sing Bing Kang. Direct methods for visual scene reconstruction. In In IEEE Workshop on Representations of Visual Scenes, pages 26–33, 1995. 1.2, 2.1, 2.2 [66] Richard Szeliski and Heung-Yeung Shum. Creating full view panoramic image mosaics and environment maps. In SIGGRAPH, pages 251–258, 1997. 2.1, 2.2, 2.3.2 [67] Laura Teodosio and Walter Bender. Salient video stills: Content and context preserved. In ACM Multimedia, pages 39–46, 1993. 2.1 [68] Matthew Uyttendaele, Ashley Eden, and Richard Szeliski. Eliminating ghosting and exposure artifacts in image mosaics. In IEEE International Conference on Computer Vistion and Pattern Recognision (CVPR), pages 509–516, 2001. 2.1 [69] Yu-Shuen Wang, Chiew-Lan Tai, Olga Sorkine, and Tong-Yee Lee. Optimized scale-and-stretch for image resizing. ACM Transactions on Graphics (Proceedings of ACM SIGGRAPH ASIA), 27(5), 2008. 3.4.1, 3.4.1 [70] Heung yeung Shum and Richard Szeliski. Construction of panoramic image 107 BIBLIOGRAPHY mosaics with global and local alignment. International Journal of Computer Vision,, 2000. 1.2, 2.1 [71] Guo-Xin Zhang, Ming-Ming Cheng, Shi Hu, and Ralph R. Martin. A shape-preserving approach to image resizing. Computer Graphics Forum, 28:1897–1906, October 2009. 3.4.1, 3.4.1 [72] Barbara Zitov´a and Jan Flusser. Image registration methods: a survey. Image and Vision Computing, 21:977–1000, 2003. 2.1 [73] I. Zoghlami, O. Faugeras, and R. Deriche. Using geometric corners to build a 2d mosaic from a set of images. In IEEE International Conference on Computer Vistion and Pattern Recognision (CVPR), pages 420–425, 1997. 2.2 108 [...]...Figure C.7: Input image series for Figure 5 .8 97 CHAPTER C Input Images for All Examples used in This Thesis 98 Bibliography [1] Autostitch, http://cvlab.epfl.ch/ brown/autostitch/auto-stitch.html 1.1, 1.3, 2.1, 2.4.1, 2.4, 2.6, 3.5, 5.3 [2] Microsoft Image Composite Editor, http://research.micro- soft.com/enus/um/redmond/groups/ivm/ICE/... (WACV), pages 50–55, 19 98 2.2 [8] Connelly Barnes, Eli Shechtman, Adam Finkelstein, and Dan B Goldman PatchMatch: A randomized correspondence algorithm for structural image editing ACM Transactions on Graphics (SIGGRAPH), 28( 3) 2.5 [9] Adrien Bartoli, Mathieu Perriollat, and Sylvie Chambon Generalized thinplate spline warps Internal Journal of Computer Vision (IJCV), 88 (1) :85 –110, 2010 2.5 [10] Sebastiano... and Joe D Warren Image deformation using moving least squares ACM Transactions on Graphics (TOG), 25(3):533–540, 2006 2.5 [57] Frederik Schaffalitzky and Andrew Zisserman Multi-view matching for unordered image sets, or “how do i organize my holiday snaps?” In European Conference on Computer Vision (ECCV), pages 414–431, 2002 2.2 [ 58] H.-Y Shum and R Szeliski Construction of panoramic image mosaics with... ACM SIGGRAPH ASIA), 27(5), 20 08 3.4.1, 3.4.1 [70] Heung yeung Shum and Richard Szeliski Construction of panoramic image 107 BIBLIOGRAPHY mosaics with global and local alignment International Journal of Computer Vision,, 2000 1.2, 2.1 [71] Guo-Xin Zhang, Ming-Ming Cheng, Shi min Hu, and Ralph R Martin A shape-preserving approach to image resizing Computer Graphics Forum, 28: 189 7–1906, October 2009 3.4.1,... Schmid A performance evaluation of local descriptors 2005 2.2, 2.1 [47] Eric N Mortensen and William A Barrett Intelligent scissors for image composition In SIGGRAPH, 1995 2.5 [ 48] S Peleg and J Herman Panoramic mosaics by manifold projection In IEEE International Conference on Computer Vistion and Pattern Recognision (CVPR), 1997 2.5 [49] Patrick P´ rez, Michel Gangnet, and Andrew Blake Poisson image editing... to image mosaics ACM Transactions on Graphics (SIGGRAPH), 2(4):217– 236, 1 983 1.2, 2.4.1 [ 18] P.J Burt and E.H Adelson A multiresolution spline with applications to image mosaics ACM Transactions on Graphics (TOG), 2(4):217–236, 1 983 2.5 [19] D Capel and A Zisserman Automated mosaicing with super-resolution zoom In IEEE International Conference on Computer Vistion and Pattern Recognision (CVPR), 19 98. .. A statistical framework for long-range feature matching in uncalibrated image mosaicing In IEEE International Conference on Computer Vistion and Pattern Recognision (CVPR), pages 442–447 IEEE Computer Society, 19 98 2.2 [33] J Jia and C.-K Tang Image stitching using structure deformation IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 30(4):617– 631, 20 08 2.5 [34] Tat-Jun Chin... An iterative image registration technique with an application to stereo vision pages 674–679, 1 981 2.1 [44] Steve Mann and Rosalind W Picard Virtual bellows: Constructing high quality stills from video In In IEEE International Conference on Image Processing, pages 363–367, 1994 1.2, 2.1, 2.2 [45] Philip F McLauchlan and Allan Jaenicke Image mosaicing using sequential bundle adjustment Image and Vision... pages 1033– 10 38, 1999 2.5 [23] Raanan Fattal, Dani Lischinski, and Michael Werman Gradient domain high dynamic range compression ACM Transactions on Graphics (SIGGRAPH), 21(3):249–256, 2002 2.4.1 [24] Martin A Fischler and Robert C Bolles Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography Commun ACM, 24: 381 –395, June 1 981 2.2.2, 3.3.1,... Intelligence, 13(9) :89 1– 906, 1991 2.2 [26] Junhong Gao and Michael S Brown An interactive image editing tool for correcting panoramas In SIGGRAPH Asia -Technical Brief, 2012 1.4 [27] Junhong Gao, Seon Joo Kim, and Michael S Brown Constructing image panoramas using dual-homography warping In IEEE International Conference on Computer Vistion and Pattern Recognision (CVPR), pages 49–56, 2011 1.4 [ 28] C A Glasbey . Input image series for Figure 3.11 Figure C.4: Input image series for Figure 4.1 Figure C.5: Input image series for Figure 5.6 Figure C.6: Input image series for Figure 5.7 96 Figure C.7: Input image. Tool 94 Appendix C Input Images for All Examples used in This Thesis Figure C.1: Input image series for Figure 3.9 Figure C.2: Input image series for Figure 3.10 95 CHAPTER C. Input Images for All Examples. Intermediate results for image set 1. 88 Input images Photoshop alignment Photoshop seam-cut Our alignment Our seam-cut Figure A.3: Intermediate results for image set 2. 89 CHAPTER A. Intermediate