Analysis of 3 d maxillofacial image data 4

84 Chapter Analysis of Volume Data 4.1 Introduction In this chapter, we deal with three-dimensional volumetric data sets Such 3-D data are commonly provided by CT or MRI scanners CT is especially suitable for visualizing high-density objects such as bone, while MRI is sensitive to variations in soft tissue Since bone is the object of interest in this thesis, we focus on analysis of CT data sets CT data comprises a series of 2-D images that exhibit axial cross-sections of an object such as a head A CT image is a pixel map of the X-ray attenuation coefficient of the tissue within a cross-sectional plane The attenuation coefficient is linearly converted to a numerical scale named after Godfrey Hounsfield, the inventor of the first CT scanner, as shown in Figure 4.1 The Hounsfield scale fixes the attenuation coefficient of air at −1000H and that of water at 0H Tooth enamel has the highest attenuation coefficient in the human body, about 3000H Thus, the range of the Hounsfield number is slightly over 4000H, and at least 12-bit gray levels, equivalent to 4096 levels, are necessary to quantize CT data without loss of information In general, 16-bit gray levels (65536 levels) are allocated to each pixel for the convenience of computer hardware In this thesis, however, we linearly transform the gray levels to bits (256 levels) to make the data more workable This does not matter because we are not dealing with subtle variations of soft tissue 85 Figure 4.1: Hounsfield numbers The skull has to be extracted from CT data for the applications in craniofacial surgery that are concerned with the anomalies of the head and facial bones [17], [18] The extraction of the skull is traditionally done by thresholding [51], but the appropriate threshold value may vary from one data set to another It will be convenient if the adequate threshold value for a data set at hand is determined automatically In our approach, background, i.e., air, is first excluded (Section 4.2.1), and then soft tissue is removed using a histogram transformation method (Section 4.2.2) The extraction of the mandible is important for maxillofacial surgery (or orthognathic surgery) that is concerned with the correction of a wide range of jaw and facial irregularities [16]−[19] For instance, assessment of mandibular asymmetry is considered to be an appropriate means of determining the effectiveness of maxillofacial and orthodontic treatment [16], [19] Currently, manual operation is necessary for extracting the mandible, which is time-consuming and labor intensive We attempt to make this segmentation process automatic (Section 4.3) The segmentation along the z-axis (i.e., between CT images) is first performed (Sections 4.3.1 and 4.3.2), and the segmentation in the x-y-plane (i.e., within a CT image) is conducted using a double-thresholding technique (Section 4.3.3) The mandible is 86 finally segmented by region growing via connected component labeling (Section 4.3.4) In recent years, CT has been increasingly used in orthodontic treatment as well, because it provides 3-D information of the jaw without geometrical distortion [20]−[25] One of the most successful applications is in dental implantology, in which an artificial root is surgically inserted into the jawbone to provide anchorage for a dental prosthesis Treatment of tooth loss with dental implants is today a routine specialty procedure For successful implant treatment, it is crucial to determine the exact location of vital anatomic parts that need to be avoided during surgery One such internal structure is the inferior alveolar nerve (IAN), which is the branch of the mandibular nerve that innervates the lower teeth, tongue, and lip The IAN passes in company with the inferior alveolar artery (jointly termed the inferior alveolar neurovascular bundle) through a mandibular canal, the inferior alveolar nerve canal (IAC) Violation or damage to the IAC can cause considerable complications [25] Panoramic radiography is one of the most commonly utilized radiographic techniques in dental implantology (Figure 4.2) Panoramic images present a global view of the shape and height of the jawbone and existing dentition for multiple implant placement, and are widely employed for initial treatment planning or screening However, despite its widespread use, panoramic imaging has a number of limitations [26] in that it provides no information about jaw thickness and suffers from a distortion factor of about 25% [25] 87 Figure 4.2: Panoramic radiograph image To overcome these problems, we employ panoramic CT images (or panoramics), which are a series of cross-sectional images along curved planes through the mandible In commercially available programs, panoramics are generated by reformatting a stack of CT images [24]−[26] Unlike panoramic radiography, these reformatted images are free from distortion, magnification errors, and superimposition of other tissues, and permit the accurate assessment of CT data in a manner that exceeds the information gleaned from radiography alone [24] These commercial programs, however, require frequent human interventions for computing the panoramic images In addition, none of them is capable of automatically detecting the IAC This could be due to the fact that the structure of the IAC is not well defined and is often connected to adjacent hollow spaces It is not unlike a tube with many openings at the sides Because of this rather complicated structure, the accurate segmentation of the IAC remains elusive The problem could be exacerbated if a patient has been without teeth for a long time as that could result in bone loss Current software only allows the user to manually trace the IAC for visualization purposes In one prototype application, the IAC is modeled with a 3-D polybezier path based on several control points specified by the user [70] This means that the path between two consecutive control points is not related to the 88 IAC and may deviate from it significantly To make the deviation small, many control points will have to be specified In Section 4.4, we present a computerized method for extracting the IAC in panoramic CT images The panoramic images are generated automatically once a representative CT slice is selected (Section 4.4.1) Hollow canals are then detected by analyzing the voxel intensities and 3-D gradient orientations in the panoramics (Section 4.4.2) Subsequently, we extract the axis of the IAC using a novel 3-D line- tracking technique followed by the merging adjoining voxels to obtain the full extent of the IAC (Section 4.4.3) Finally, the extracted canal is backprojected to the original CT data to provide the clinicians with a visual aid for treatment planning The method is generic and may be used in other applications that require the extraction of tubular structures We also work on the detection of other anatomic features on the jaw surface (Section 4.5) For this, we make use of the panoramic surface images of CT data (Section 4.5.1) The panoramic pseudo-reflectance image is employed for detecting a pair of mental foramens (or foramina) (Section 4.5.2) and the panoramic range image for a pair of mandibular foramina (Section 4.5.3) The foramina are small openings on the mandibular surfaces through which blood vessels and nerves pass 89 4.2 Extraction of the Skull 4.2.1 Exclusion of Background The Hounsfield number of air and that of the human body are well separated (Figure 4.1) This can also be clearly seen in the gray-level histogram of CT data (Figure 4.3) The histogram is bimodal, with the left peak corresponding to air (i.e., background) and the right one the head With the aim of extracting the skull, we first exclude the background by selecting an appropriate threshold value between the two peaks in the histogram There are a number of techniques proposed to determine a proper threshold value [73] We have employed the Otsu method because of its reliability and computational efficiency [71] The threshold value determined by the method for the CT data of Figure 4.3 was 39 (the vertical line in Figure 4.3) Figure 4.4 shows four images of the CT data and Figure 4.5 the corresponding object maps obtained by thresholding The black areas are the background and the white areas the objects including the head The voxels in the dark areas are ignored in subsequent steps 90 Figure 4.3: Gray-level histogram of a CT data set (a) (b) (c) (d) Figure 4.4: Four cross-sectional images from CT data set corresponding to the histogram in Figure 4.3 91 (a) (b) (c) (d) Figure 4.5: Four object maps corresponding to the four CT images of Figure 4.4 (threshold value=39) 4.2.2 Removal of Soft Tissue We next attempt to remove the soft tissue of the head After excluding the background, the remaining voxels form a unimodal gray-level histogram (on the right side of the vertical line in Figure 4.3) Since the skull occupies only a small portion of the CT image compared with the soft tissue, the peak corresponding to the skull is far smaller than that of the soft tissue and there is no clear valley, making it difficult to select the appropriate threshold value Since the pixels in the neighborhood of an edge have larger gradient magnitudes, the gray-level histogram for these pixels should have a single peak at a gray level between the object and the background gray levels [72], [73] This gray level is, therefore, a 92 suitable choice of the threshold value We apply this idea to CT data to determine a proper threshold value for separating the soft tissue from the skull Just like their twodimensional counterparts, 3-D edges are also defined as discontinuities in image intensity caused by the transition from one homogeneous 3-D region to another 3-D region of a different mean intensity Therefore, the intensity gradient ∇f ⎛ ∂f ∂f ∂f ⎞ ∇f ( x, y , z ) = ⎜ , ⎜ ∂x ∂y , ∂z ⎟ ⎟ ⎝ ⎠ (4.1) provides information about the existence of an edge The gradient magnitude M is a useful measure of edge strength: M ( x, y , z ) = 2 ⎛ ∂f ⎞ ⎛ ∂f ⎞ ⎛ ∂f ⎞ ⎜ ⎟ +⎜ ⎟ +⎜ ⎟ ⎜ ⎟ ⎝ ∂x ⎠ ⎝ ∂y ⎠ ⎝ ∂z ⎠ (4.2) Using the 3-D Sobel filter for obtaining a discrete approximation of the partial derivatives in x, y, and z directions, we apply Eqs (4.1), (4.2), and select voxels with large gradient magnitude Figure 4.6 illustrates a cumulative histogram of gradient magnitude where the horizontal line denotes the largest 10% By doing this, the selection of an appropriate threshold value for extracting edge voxels is automated The average gray level of the selected voxels serves as the threshold value of the CT data Figure 4.7 shows four skull maps obtained by this technique for the CT data of Figure 4.4 93 Figure 4.6: Cumulative histogram of gradient magnitude (a) (b) (c) (d) Figure 4.7: Four skull maps corresponding to the four CT images of Figure 4.4 (threshold value=96) 120 Figure 4.32: Display of the traced path Blue and green dots denote terminal points, red dots the traced path 4.4.5 Merging of Adjacent Voxels Figure 4.33 shows the 3-D presentation of the traced paths within 31 panoramic CT images of size 90×460 voxels Figure 4.34 shows three cross-sections of the panoramic CT images In these images, the IAC appears as a tiny, dark, and approximately circular region Let f (i, j ) be a cross-sectional image, where x and y are the coordinate axes, and (ia , ja ) the voxel denoting the traced path (yellow dots) We obtain the full extent of the IAC by merging adjoining voxels that satisfy the following two conditions: ⎧ f (i, j ) ≤ 20 , ⎪ ⎨ 2 ⎪ (i − ia ) + ( j − ja ) ≤ voxels ( ≈ 1.5 mm) ⎩ (4.9) 121 The first condition is used to select voxels with very low intensities since the target object is hollow space The second condition ensures that we not merge voxels that are too far from the axis Since the diameter of the IAC ranges from 2.0 to 2.4 mm, a limit of mm is reasonable In Figure 4.34, the region satisfying the distance constraint is enclosed by the light blue lines The voxels marked in green satisfy both conditions and will be merged This step is repeated for every cross-sectional image along the traced path (the detected IAC axis) Figure 4.33: Extracted IACs in a set of panoramic CT images 122 Figure 4.34: Cross-sections of panoramic CT images to illustrate the extraction of the IAC from detected axial points 4.4.6 Results and Discussion For the purpose of visualization, the extracted IACs are backprojected and superimposed on the original CT data The results using two CT data sets are shown in Figures 4.35 and 4.36 The IACs are displayed together with the translucent mandible using the biomedical image processing and visualization software ANALYZE [75], [76] 3-D visualization is of great help in medical applications for replacing the mental reconstruction of objects from cross-sectional images, which is a difficult process and depends heavily on the observer’s training and imagination [77] 123 Figure 4.35: Mandible and extracted IACs in various views (sample 1) Figure 4.36: Mandible and extracted IACs in various views (sample 2) We evaluate the accuracy of IAC determination by measuring the deviation between the extracted and true axes We assume that the sequence of voxels with the lowest intensities in the vicinity of the traced path represents the true axis Using the four 124 IACs of the two data sets of Figures 4.35 and 4.36, the results are given in Table The mean deviation ranges from 0.3 to (voxel unit), which is approximately equal to 0.15 to 0.5 mm (Table 1) The deviation is very small and does not pose any problem in the extraction of the full IAC It is interesting to note that the traced path coincides closely with the axis of the IAC in spite of the fact that we are not tracing the voxels with low intensities but with those with significant discontinuity in gradient orientation Table 4.1: Deviation between the traced path and the actual axis of the IACs (in voxel units) Sample Sample Mean deviation (standard deviation) Mean deviation (standard deviation) Left IAC 1.11 (0.89) 0.33 (0.60) Right IAC 0.58 (0.75) 0.77 (0.82) The results highlight the effectiveness of the techniques that we have introduced 3-D GOA is particularly suitable to the detection of hollow canals and thin bony structures, which are discontinuous in gradient orientation The false responses from the hard tissue are subsequently suppressed by using the fact that the voxel intensities of hollow canals and bone are very different The robustness of 3-D GOA is a significant advantage over the eigenvalue methods that are widely employed for extracting 3-D tubular structures [50], [51] Since the latter require the three eigenvalues to be correctly determined, they are more susceptible to noise and inconsistencies in object structures 125 The tracing of the IAC axis is complicated by the presence of spurious lines that may intersect the axis Techniques such as dynamic programming and the active contour model are well known, but they are computationally intensive and not suitable for interactive or real-time use The tracing of the IAC axis is complicated by the presence of spurious lines that may intersect the axis We overcome these difficulties with a two-pronged approach Tracking is guided to ensure that it is steered towards the designated target point, while tracing the path in two directions decreases the susceptibility to be led astray by spurious lines Once the axis is accurately determined, we are able to employ region-growing to complete the full extraction of the IAC 4.5 Detection of Other Anatomic Features 4.5.1 Panoramic Surface Image of CT Data In Section 4.4, a method for extracting the IAC, an important internal anatomic structure, was discussed Meanwhile, there are also several important anatomic features on the skull surface In this section, we propose the use of the panoramic surface image of CT data for detecting those features on the surface The human jaw, which we are particularly interested in, comprises two complex bony structures: the maxilla and mandible Because of their curved configuration, it is difficult to locate surface features in a single view of CT data Due to the large size of CT data sets, the investigation of the original 3-D data is not a preferred option Provided the panoramic surface view of the jaw is given, the detection of various features on the skull surface will be facilitated and also made more efficient 126 We first compute a panoramic range image using the base curve (Section 4.4.1) as the reference, and then the pseudo-reflectance image from the range image The panoramic range image is computed following the procedure addressed in Chapter The only difference is that the dental arch is replaced with the base curve of the mandible here At each sample point on the base curve, an inspection spoke is set up with the orientation perpendicular to the curve The corresponding surface point is the intersection between the inspection spoke and the jaw surface The distance between a sample point and the corresponding surface point gives the height of the jaw surface from the base curve The computation is repeated with all CT images that contain the jaw, producing a map of distance distribution Once the distance map is scaled to the range of to 255, it can be viewed as the panoramic range image of the jaw h( x, y ) (Figure 4.37) (a) (b) Figure 4.37: Panoramic range images of the jaw (a) Anterior surface of the jaw (b) Internal surface of the jaw 127 Since the bone surface is not specular but rather diffusive, we can compute the pseudoreflectance image of the panoramic range image based on the Lambertian cosine law: the perceived brightness of a surface patch illuminated by a point source varies with the incident angle relative to the surface normal of the patch [78] Suppose a point light source is placed right above jaw surface, h( x, y ) Then the reflectance (i.e., image intensity) i ( x, y ) is given by i ( x, y ) ∝ cos θ = 1 + p2 + q2 , (4.10) where p ( x, y ) = ∂h ( x, y ) ∂x and q ( x, y ) = ∂h( x, y ) ∂y Subsequently, the computed reflectance i ( x, y ) is scaled between and 255 Figure 4.38 shows the pseudoreflectance images generated from the panoramic range images of Figure 4.37 (a) (b) Figure 4.38: Panoramic pseudo-reflectance images of the jaw (a) Anterior surface of the jaw (b) Internal surface of the jaw 128 4.5.2 Mental Foramen The mental foramen is a small opening on the lateral anterior (or outer) surface of the mandible through which blood vessels and nerves pass There is a pair of foramens (or foramina) generally located between the first and second premolar teeth to mm below the apices of the premolars The foramina are vital anatomic landmarks for orthognathic surgery The mental foramens can be seen as a pair of dark spots in Figure 4.38(a) Unless the head is tilted considerably, the two dark spots should be found on the same row (horizontal line) Thus, we find the pixel whose gray level is the lowest at each row in the left half and also in the right half of the panoramic pseudo-reflectance image The image is split by the symmetric axis that corresponds to the apex of the base curve Then we find the row that satisfy the following conditions: A Darkness: The gray levels of the two selected pixels (left and right) are sufficiently low B Uniqueness: There are no other pixels whose gray levels are comparable to the lowest C Size: The two spots are sufficiently small D Symmetry: The two spots are located symmetrically with respect to the symmetry axis E The two spots are not located too close to the symmetry axis F The two spots are not located too close to the boundary of the mandible The cost function of condition A is described as A( r ) = Scale( (255 − imin,left ( r ) ) + (255 − imin,right ( r ) ) ) , (4.11) 129 where imin,left ( r ) and imin,right ( r ) are the lowest gray levels of the left and right halves of the row r, respectively Scale(• ) is a scaling function that linearly transforms input values to the range to This condition is important to find dark spots in the image The cost function of the condition B may be expressed as ( ) B (r ) = Scale i2 nd , left (r ) − imin,left (r ) + i2 nd , right (r ) − imin, right (r ) , (4.12) where i2 nd ,left ( r ) and i2 nd , right ( r ) are the second lowest gray levels of the left and right halves of the row r, respectively These second darkest spots are detected at least pixels off the darkest spots to make sure that they are separate spots This condition is necessary to reject rows that include the interstices between teeth as they produce many dark spots in one row In other words, this condition tests the uniqueness of the selected two spots The cost function of the condition C can be given by ( C ( r ) = Scale ineighbor ,left ( r ) − imin,left (r) + ineighbor ,right ( r ) − imin,right (r) ), (4.13) where ineighbor , left ( r ) and ineighbor , right ( r ) are the gray levels of the pixels that are a few mm away from the left and right darkest spots of the row r, respectively This condition ensures that the darkest spot is of a limited size and rejects rows that capture large dark areas between the upper and lower teeth The cost function of the condition D can be expressed as ( D( r ) = Scale − d left − d right (d left + d right ) ), (4.14) where d left and d right are the distances from the symmetry axis of the left and right darkest spots of the row r, respectively This condition may not be as important as the other conditions because the image itself is symmetric by human nature In addition, 130 when the mandible is significantly asymmetric, the condition may not be effective However, it will be helpful to make the detection robust to image noise The conditions E and F are not essential but help to increase the reliability of the method by moderately limiting the region for detecting the mental foramens Finally, we define the overall cost function as the multiplication of the individual cost functions: Z ( r ) = Scale( A( r ) × B( r ) × C ( r ) × D (r ) ) (4.15) Figure 4.39 shows the individual cost functions, A (darkness), B (uniqueness), C (size), D (symmetry), and also the overall cost function Z (overall) against r, the rows of the panoramic pseudo-reflectance image of Figure 4.38(a) The highest peak of Z (r ) is found at the 65th row from the top, and thus this row is assumed to be traversing the mental foramens The darkest spots on this row are shown in Figure 4.40, and the two red circles successfully capture the two mental foramens while the blue vertical line depicts the asymmetry axis By searching for the darkest pixel in the neighborhood, we can further refine the position of the detected mental foramen 131 Figure 4.39: Cost functions against rows of a panoramic pseudo-reflectance image Figure 4.40: Symmetry axis (blue) and detected mental foramina (red circles) 4.5.3 Mandibular Foramen The mandibular foramens are another pair of openings on the internal surface of the mandible through which vessels and nerves pass The opening of the mandibular foramen is larger than that of the mental foramen and there are no distinct dark spots in the panoramic pseudo-reflectance image (Figure 4.38(b)) Instead, they appear as slight steps in the panoramic range image (Figure 4.37(b)) Thus, we apply an edge detection technique to find the step edges at each row of the panoramic range image 132 first The edge magnitude obtained provides an important cue for locating the mandibular foramens as the pixel intensity does for the mental foramens Therefore, to find the row that traverses the mandibular foramens, we resort to the same cost function as above using the edge magnitude in place of the pixel intensity Hence, we find the row that traverses the mandibular foramens with respect to the edge magnitude, uniqueness, size, and symmetry Figure 4.41 shows the individual cost functions, A (edge magnitude), B (uniqueness), C (size), D (symmetry), and also the overall cost function Z (overall) against r, the rows of the panoramic range image of Figure 4.37(b) The highest peak of Z (r ) is found at the 8th row from the top, and thus this row is assumed to be traversing the mandibular foramens The two pixels with the largest edge magnitude on this row are shown in Figure 4.42, and the two red circles successfully capture the two mandibular foramens Furthermore, we search for the pixel whose edge magnitude is the largest in the neighborhood of each mandibular foramen detected This final step is free from the constraint that the two points have to be on the same row 133 Figure 4.41: Cost functions against rows of a panoramic range image Figure 4.42: Symmetric axis (blue) and detected mandibular foramina (red circles) 4.5.4 Results and Discussion As discussed in Section 4.4.1, when CT images are used for maxillofacial surgery and dental treatment, they should be acquired parallel to the occlusal plane of the teeth so that they capture vital structures effectively Similarly, prior to the detection of the foramen, it is desired that the head is well aligned so that there is little tilt, slant, and rotation In particular, if the tilt of the head is considerable, the direct application of our method would not be successful In this case, we need to introduce a step for 134 aligning the head to a standard orientation In fact, there are several techniques proposed to determine the mid-sagittal plane of the head [80], [81], [83] It will be effective for higher accuracy and reliability to combine those techniques with our method Once the locations of the foramens are determined on the panoramic surface image, we can backproject the coordinates to the original CT data The locations of the mandibular and mental foramens are important for orthognathic surgery since they indicate the terminal points of the IAN The IAN enters the mandible through the mandibular foramen and its larger branch emerges from the mental foramen to innervate the skin of the chin and the lower lip, while the smaller branch supplies the canine and incisor teeth It is also possible to relate the detected foramens with the panoramic CT images by finding the nearest sample point (voxel) used for computing the panoramic images The displacement between the backprojected point and the nearest sample point is limited within a pixel of a CT image and is very small Once the foramens are located in the panoramic CT images, they may be used as terminal points for line tracing However, the current tracing technique was not successful in this task because the IAC has a tight curve near the mental foramen that is beyond the tolerance of the technique to a curved path Further refinement of the tracing technique may be necessary for fully automated extraction of the IAC ... (a) 64 74 84 94 1 04 1 14 1 24 1 34 144 (b) Figure 4. 8: Segmentation results in a pediatric case (a) Rendered surfaces of a CT data segmented at various threshold values (b) Threshold values used for... and detected mandibular foramina (red circles) 4. 5 .4 Results and Discussion As discussed in Section 4. 4.1, when CT images are used for maxillofacial surgery and dental treatment, they should... and the IAC in a volumetric data set 120 Figure 4 .32 : Display of the traced path Blue and green dots denote terminal points, red dots the traced path 4. 4.5 Merging of Adjacent Voxels Figure 4 .33