3D SHAPE DESRIPTORS: 4D HYPERSPHERICAL HARMONICS “AN EXPLORATION INTO THE FOURTH DIMENSION” Bryan Bonvallet Michigan Technological University btbonval@mtu.edu Nikolla Griffin Western Michigan University nikolla.c.griffin@wmich.edu ABSTRACT Matching 3D images remains a challenging problem Most search engines on the internet use textual description to match images More sophisticated systems use shape descriptors that are automatically constructed from the original 3D shape Good shape descriptors must be insensitive to noise, orientation, scale, or translation They must be fast to compute, small in size, and easy to compare In this research, we are proposing a novel method for creating shape descriptors It is inspired from the descriptors using 3D spherical harmonics 3D spherical harmonics present the benefits of being insensitive to noise, orientation, scale, and translation, and of being relatively fast to compute On the other hand, they have the disadvantages of requiring 3D storage, compounding error, and failing to recognize inner rotations We address these problems by applying 4D hyperspherical harmonic decomposition A 3D object (mesh or point cloud) may be mapped to the 4D unit hypersphere The 4D hyperspherical harmonics have the same advantages of the 3D spherical harmonics, with added benefits because inner radius cuts are unnecessary KEY WORDS Dr Jia Li Oakland University li4@oakland.edu hyperspherical harmonics, shape descriptor INTRODUCTION The advancement of 3D scanning device technology and the everyday use of the World Wide Web have made the index of available 3D shapes expand vastly over the years As the demand for computer aided graphics increases, the methods of search and retrieval must become more robust and effective 3D shape retrieval has the potential to greatly improve many fields of research including mechanical engineering, molecular biology, and computer vision 3D shape retrieval has applications in facial recognition, medical image analysis, entertainment, and architectural industries The challenge is to convert 3D objects into compact computational representations called shape descriptors These numerical representations serve as search characteristics during the retrieval process These representations are usually in the form of vectors that are compared by the distance of two points within the space Good shape descriptors must be insensitive to noise, orientation, scale, and translation They must be fast to compute, small in size, and easy to compare A variety of methods have been proposed in the past, however, they all fit into three main categories or are some combination of the three groups [1] This research focuses on the global matching method of 3D spherical harmonics We use the basic idea of harmonics and apply them to hypersphere 4D hyperspherical harmonics require less storage space, provide accurate matching results, and avoid compounding harmonic truncation Hyperspherical harmonics have been used in other applications, but this research is the first step toward using hyperspherical harmonics for shape descriptor representation The organization of this paper follows In section 2, we briefly describe the process of 3D spherical harmonics In section there will be some background information on hyperspherical harmonics and our shape descriptor’s implementation In section 4, we present experimental results and the analysis Section concludes the paper and gives details about future work 2 3D SPHERICAL HARMONICS Spherical harmonic representation as a 3D shape descriptor was presented by Michael Kazhdan This descriptor transforms rotationally dependent shapes into rotationally invariant descriptors Spherical harmonic representation provides better matching results than those obtained by rotation normalization [2] The main goals of this descriptor are to provide better matching results while reducing both the space for storage and the time for comparison Kazhdan reports that normalization via PCA alignment hampers the performance of descriptors Spherical harmonic decomposition is a generalization of Fourier transform, which uses spherical functions obtained at different frequencies The main idea of this approach is to describe a spherical function in terms of the amount of energy it contains at different frequencies [2] These values not change when the function is rotated, hence rotation invariance Besides invariance to rotation, spherical harmonic descriptors offer several other advantages Precision-recall is a method of determining the accuracy of a shape descriptor “Precision” is the fraction of retrieved images which are relevant, while “recall” is the fraction of relevant images which are retrieved A perfect retrieval result would produce a horizontal line along the top of the plot, indicating that all the models within the target object’s class are returned as the top hits To our knowledge, spherical harmonic descriptors currently have the best precision-recall of the available 3D shape descriptors Figure 1: Plot of precision versus recall of spherical harmonics versus other shape matching methods using living room furniture 2.2 Limitation of Spherical Harmonics Although spherical harmonic representation has proven to be successful, there is still room for improvement There is a full dimension of information lost in going from a spherical function to its harmonic representation The descriptors are unchanged if different rotations are applied to different frequency components of a spherical function For each frequency component, the spherical harmonic representation only stores the energy in that component [2] The process of spherical harmonics leads to unique sources of error not seen in other methods The spherical harmonics of a function continue infinitely Much like the rounding of decimals, spherical harmonic descriptors must arbitrarily decide which degree of harmonic decomposition to end analysis at (referred to as harmonic truncation) Because spherical harmonics are applied over a 3D spherical surface, a 3D object must be severed into many spheres with varying radii The number of possible radii cuts is also infinite The error caused by harmonic truncation is compounded over every radii cut, and the error caused by using finitely many radii cuts is obvious These radii cuts also introduce the problem of a descriptor that cannot acknowledge inner rotations Ideally objects would have the same harmonic representation after outer rotation over the entire shape With inner rotations (see Figure 2), the 3D spherical harmonic descriptors for two objects is the same, when it obviously should not be Figure 2: Inner rotation problem that occurs because of radii cuts within spherical harmonics There is the classic problem of storage Spherical harmonic descriptors use three dimensions of storage; that is to say a vector of values as a function of radius and harmonic frequency Our proposed method of using hyperspherical harmonic decomposition as a shape descriptor will solve the problem of radii cuts Thus error due to harmonic truncation is not compounded over radii cuts, the descriptor is sensitive to inner rotations, and the dimensionality of storage is reduced by one The proposed descriptor will be a vector of values as a function of hyperspherical harmonic frequency only 4D HYPERSPHERICAL HARMONICS Typically hyperspherical harmonics are used in the fields of quantum theory and quantum mechanics They have also been applied to problems in molecular, nuclear, and atomic physics Hyperspherical methods have long been a valuable analytical and computational tool for understanding n-body quantum systems [4] Our choice to use hyperspherical harmonics was based on the need to address the inner radius cut issue Harmonics taken over the entire shape would produce better results than those produced by finitely many radii cuts It is known that harmonics may be performed over the n th dimensional sphere, as the idea has been generalized by physicists A test of the theory was performed based on the assumption that if 2D areas can be mapped to the 3D unit sphere (and subsequently spherical analysis may be performed), then a 3D volume may be mapped to the 4D unit hypersphere (and subsequently hyperspherical analysis may be performed) 3.2 Theory Testing Verification of rotational invariance and invertibility were tested using the 2D area-to-3D unit sphere mapping The theory testing was done in MatLab, using Yet Another Wavelet Toolbox [4] to run spherical harmonics and original programming to determine the 3D spherical coordinates from 2D cartesian coordinates Because the mapping was onto the unit sphere, radius may be ignored or assumed to be one; the only values of interest are the longitudinal and latitudinal angles The results of the testing showed that rotation of a 2D shape was represented by a latitudinal phase shift (see figure 3) Figure 3: Original 2D shape (upper left), 60° rotation of original shape (middle left), 90° rotation of original shape (bottom left), and longitudinal and latitudinal angles of 3D spherical coordinates (right) Such a phase shift would yield a rotation in three dimensions; spherical harmonic descriptors that could be applied to the mapped data are invariant to such rotations Inversion (backwards mapping) of the 3D image gave the 2D preimage back (see Figure 4) The mapping is thusly safe to use After having successful results using 2D-to-3D mapping, we were satisfied that 3D volume-to-4D hypersphere mapping was worth pursuing and we were ready to apply hyperspherical analysis Figure 4: Original 2D shape (left), longitudinal and latitudinal angles of 3D spherical coordinates (middle), then it is inverted back to original shape (right) 3.3 Implementation The process begins by running a surface voxelization algorithm over the entire mesh A volumetric voxelization algorithm might yield better results, but only allows for genus zero shapes We chose surface voxelization for more flexibility Once voxelization is performed, each voxel is considered to be a single point in the original continuous space with a value of true or false An angular grid is set up for the 4D unit hypersphere on the range of [0, π], [0, π], [0, 2π] Each entry of the grid is mapped backwards to the 3D continuous coordinate system and takes on the truth value of that mapped 3D point Now we have an angular voxel grid with the 3D object mapped onto the 4D unit hypersphere The entire sample space of the angular voxel grid is converted into the corresponding continuous angles, then harmonics are performed on them (up to an arbitrary upper limit) This gives a complex matrix A containing the harmonic values over the sample space of points The matrix b is filled in with truth values of the corresponding point Coefficients for each harmonic value are found by solving the linear equation Ax=b for x SOLVING FOR LAMBDA HERE RESULTS Figure 5: Prerendered images [m0 (top left), m150 (top right), m1401 (bottom left), m100 (bottom right)] Figure 6: Voxelization results [m0 (top left), m150 (top right), m1401 (bottom left), m100 (bottom right)] no rotation xrotation yrotation zrotation λ=0 0015 λ=1 0330 λ=2 0209 λ =3 0788 λ=4 0795 0024 0309 0177 1065 0788 0013 0291 0115 0868 0786 0014 0291 0173 0778 0734 Figure 7: m0 Rotation Comparison m0 m100 m150 m1401 λ=0 0015 0084 0103 0090 λ=1 0330 0850 0922 0899 λ=2 0209 0264 0310 0299 λ=3 0788 1711 2239 2134 λ=4 0795 1595 2145 1561 Figure 8: Model Comparison x-rotation y-rotation z-rotation m100 m150 m1401 m0 0280 0094 0081 1330 2072 1654 Figure 9: Euclidean Distance between Descriptors CONCLUSION/ FUTURE WORK In this paper we proposed a novel approach for shape descriptor representation using hyperspherical harmonics Hyperspherical harmonics incorporate global features of a shape by initially taking a surface voxelization of the object Hyperspherical harmonics are not limited to certain types of shapes, it can be perform on points clouds (by skipping voxelization), genus-zero, mesh, or polygon-mesh Hyperspherical Harmonics can be normalized for scale and translation The major advantage of hyperspherical harmonics is that it is rotationally invariant This rotational invariance does not come with the cost of compounded harmonic truncation error across radii cuts Hyperspherical harmonic representation is also sensitive to inner rotation Our approach utilizes sensible storage The dimensionality of storage is reduced by one with hyperspherical harmonics as compared to spherical harmonic methods FUTURE WORK HERE ACKNOWLEDGEMENTS We would like to give special thanks to the NSF for their sponsorship, the UnCoRe program for the experience, Dr Jia Li for all her care and attention advising us, Shaojun Liu for his support, and Oakland University for hosting us REFERENCES [1] J Tangelder, and R Veltkamp A survey of content based 3d shape retrieval methods In Shape Modeling International, pages 145-156, 2004 [2] M Kazhdan, T Funkhouser, and S Rusinkiewicz Rotation invariant spherical harmonic representation of 3D shape descriptors In Eurographics/ACM SIGGRAPH Symposium on Geometry Processing (2003) pages 156-164, 2003 [3] K A Mitchell and R G Littlejohn Derivation of planar three-body hyperspherical harmonics from monopole harmonics In Physical Review, volume 56, 1997 [4] J.P Antoine, L Demanet and L Jacques (2006) Yet Another Wavelet Toolbox Retrieved May 29, 2006, from London’s Global University, Web site: http://rhea.tele.ucl.ac.be/yawtb/ [5] J Avery Hyperspherical Harmonics and Generalized Sturmians Dordrecht: Kluwer Academic Publishers, 2000 [6] N D Cornea, et al 3d object retrieval using many-to-many matching of curve skeletons In Shape Modeling and Applications, 2005 [7] D Eberly Spherical Harmonics http://www.geometrictools.com March 2, 1999 [8] T Funkhouser, et al A search engine for 3D models In ACM Transactions on Graphics, pages 83-105, 2003 [9] X Gu and S J Gortler, and H Hoppe Geometry images In Proceedings of SIGGRAPH, pages 355-361, 2002 [10] M Kazhdan Shape Representations and Algorithms for 3D Model Retrieval PhD thesis, Princeton University, 2004 [11] A Matheny, and D B Goldgof The Use of Three- and Four-Dimensional Surface Harmonics for Rigid and Nonrigid Shape Recovery and Representation IEEE Transactions on Pattern Analysis and Machine Intelligence, volume 17, pages 967-981,1995 [12] A V Meremianin Multipole expansions in four-dimensional hyperspherical harmonics Journal of Physics A: Mathematical and General Issue 39, pages 3099-3112 March 8, 2006 [13] C Misner Spherical Harmonic Decomposition on a Cubic Grid Classical and Quantum Gravity, 2004 [14] M Murata, and S Hashimoto Interactive Environment for Intuitive Understanding of 4D Object and Space In Proceedings of International Conference on Multimedia Modeling, pages 383-401, 2000 [15] W Press, S Teukolsky, W Vetterling, B Flannery Numerical Recipes in C: The Art of Scientific Computing (Second Edition) Cambridge University Press, 1992 ... with the 3D object mapped onto the 4D unit hypersphere The entire sample space of the angular voxel grid is converted into the corresponding continuous angles, then harmonics are performed on them... the 4D unit hypersphere on the range of [0, π], [0, π], [0, 2π] Each entry of the grid is mapped backwards to the 3D continuous coordinate system and takes on the truth value of that mapped 3D. .. for shape descriptor representation using hyperspherical harmonics Hyperspherical harmonics incorporate global features of a shape by initially taking a surface voxelization of the object Hyperspherical