DITM: “tf1732_c010” — 2004/10/22 — 16:37 — page 211 — #1 CHAPTER 10 Management of DTM Data In Chapter 9, it was discussed that DTM data have become part of an NSDI and one usually produced at the national level with multi-scales. For large countries like Brazil, Canada, China, India, and the United States, the volume of DTM data could be huge. Therefore, efficient management of DTM data in a computerized system is an important task at national or provincial geospatial information centers. Therefore, this chapter is devoted to management of DTM data. 10.1 STRATEGIES FOR MANAGEMENT OF DTM DATA Spatial data, including DTM data, must be managed efficiently and database technol- ogy plays an important role. There are different strategies to deal with the problems in the management of DTM data. 10.1.1 Strategy for Making DTM Data Management Operational To make the management of spatial data operational, spatial data sets are partitioned according to five attributes, horizontal or vertical positions, time, theme, and scale. In the management of DTM data, scale and horizontal positions are used. The use of scale was discussed in Chapter 9 and, therefore, only the use of horizontal position will be described in this section. If the area to be modeled is large such as a nation or a province, one is concerned with the arrangement of the huge volume of DTM data. Questions such as “should distributed or centralized databases be used,” or “how can the data of the whole area be split into small pieces so that they can be managed efficiently” are the concern here. As contour maps have been widely used for DTM production, DTMs at a national scale are usually arranged in a way similar to map sheets. Figure 10.1 shows the arrangement for the 1:1,000,000-scale topographic maps of China. Table 10.1 shows the size of each map sheet at different scales, ranging from 1:1,000,000 to 211 © 2005 by CRC Press DITM: “tf1732_c010” — 2004/10/22 — 16:37 — page 212 — #2 212 DIGITAL TERRAIN MODELING: PRINCIPLES AND METHODOLOGY Figure 10.1 The tiling system of China’s map sheets at 1:1,000,000 scale. (Courtesy of the National Geomatics Center of China.) Table 10.1 The Sizes of China’s Map Sheets From 1:1,000,000 to 1:10,000 Scales Scale 1:1,000,000 1:500,000 1:250,000 1:100,000 1:50,000 1:25,000 1:10,000 Size 6 ◦ ×4 ◦ 3 ◦ ×2 ◦ 1.5 ◦ ×1 ◦ 30 ×20 15 ×10 7.5 ×5 3 45 ×2 30 (long/lat) 1:10,000. Taking the DTM of China at 1:1,000,000 as an example, it is in a grid form and there are a total of 25,000,000 data points (at grid nodes). The heights of these points are divided into tiles, which follow the 1:500,000-scale topographic maps (http://nfgis.nsdi.gov.cn/). In other words, each tile covers an area of 3 ◦ × 2 ◦ (longitude/latitude). This kind of partition is the operational strategy for DTM data management. Such a strategy is equally applicable for any project with a relatively large area to be modeled. 10.1.2 Strategy for Using Databases for DTM Data Management The second strategy is about the use of databases to store DTM data. The traditional database is good atmanagingof event (or attribute) data but it is notgoodfor geometric data. On the other hand, all spatial data, including DTM data, have two components, © 2005 by CRC Press DITM: “tf1732_c010” — 2004/10/22 — 16:37 — page 213 — #3 MANAGEMENT OF DTM DATA 213 geometric and attribute, and therefore are quite different from ordinary event data, which have only one component. Therefore, special arrangements for spatial data must be made according to the characteristics of these two components. Currently, the mainstream practice is to use files to store geometric data and to use relational tables to store attribute data (and relational data if any) in a traditional relational database. The files for geometric data are then managed by a computer system. The geometric and attribute data are then linked by pointers. This is also common for DTM data management. Files are cataloged and indexed so that efficient retrieval is possible. This is helped by metadata, or “data about data.” Metadata contain the information describing the contents, quality, status, and other characteristics. Metadata can also be indexed using files. However, if complicated, metadata can also be managed by databases. In this way, databases do now come into the area of geometric data management, but indirectly. Current development is toward object-relational databases. In such databases, geometric data (mainly the coordinates) are also organized into tables and stored and managed by the relational database management system. This has become popular for the management of large-volume DTM data. In practice, when data volume is not very large, a file system is still commonly used due to its convenience and the high cost of object-relational databases. Purely object-oriented databases have also been under development. However, there is still a long way to go before they will be commonly used. 10.2 MANAGEMENT OF DTM DATA WITH FILES In the previous section, it was discussed that file systems are still commonly used for the management of DTM data. The structure of such files will be discussed in this section. 10.2.1 File Structure for Grid DTM When the DTM is in a grid form, it can be represented by point matrix (Figure 10.2), or raster format. The topology between a grid point and its adjacent grid points is implicitly built in the rows and columns of the matrix. The coordinates of a grid node can be computed based on the coordinates (x 0 , y 0 ) of the origin of the area and the square grid intervals d. Suppose the lower-left corner 62666862 66687064 58635957 566058 … … … …………… … 56 Figure 10.2 Matrix representation of grid DTM data. © 2005 by CRC Press DITM: “tf1732_c010” — 2004/10/22 — 16:37 — page 214 — #4 214 DIGITAL TERRAIN MODELING: PRINCIPLES AND METHODOLOGY Table 10.2 Typical File Structure for Grid DTM File Components Contents Comments Header Coordinates of the origin; coordinate data type; height range; height data type; grid interval; numbers of rows and columns; order of rows and columns, position (in the file) where the body starts; position (in the file) where the footer starts; use of compression or not; etc. The information in the footer can also be recorded here Body Height values of grid nodes Row by row and column by column in blocks Footer Data describing the general characteristics of DTM, e.g., name, boundary, producer, projection parameters, version, accuracy, date of production, date of revision, linage, etc. Metadata point of a matrix (m, n) is used as the origin, then the coordinates of the grid node at (i, j) are: x i,j = (j − 1) × d +x 0 , j = (0, n −1) y i,j = (i −1) ×d +y 0 , i = (0, m −1) (10.1) In other words, this elevation matrix records the heights at DTM grid nodes. However, some additional information is required to tell users how to read the height information. The location of the origin and the grid interval are necessary for the computation of coordinates, and information about the sequence of the height values is also needed so that each grid node can be assigned a height value. In a typical file of raster data, such additional information is recorded as the header and the matrix is the file body. In the body, heights are recorded row by row and then column by column, or column by column and then row by row, or block by block. Some other relevant information may also be recorded, either in the header or in a footer. Therefore, the typical file structure for a grid DTM is as shown in Table 10.2. 10.2.2 File Structure for TIN DTM The TIN model represents a surface comprising a series of contiguous triangles, hence triangulated. A triangle has three vertices, which can be arbitrarily located, here irregular in shape. This contrasts with the grid model where points are spaced regularly in a lattice. The big difference between the management of TINs and grids is that, for the TIN model, apart from elevation values, the coordinates (x i , y i ) of each vertex (say ith) and the information describing the topological relations among the three vertices need to be recorded. The topological relationship between triangles also needs to be recorded in most cases. © 2005 by CRC Press DITM: “tf1732_c010” — 2004/10/22 — 16:37 — page 215 — #5 MANAGEMENT OF DTM DATA 215 I II III IV 1 2 4 5 3 6 7 V VI Figure 10.3 A triangulated irregular network (TIN). Table 10.3 A List of Coordinates of Points No. XYZ 1 429.1 269.6 57.5 2 437.3 200.3 60.2 3 504.7 234.1 55.3 4 607.2 190.5 56.1 5 555.4 265.8 50.2 6 506.7 280.3 52.5 7 621.2 251.4 53.8 . . . . . . . . . . . . Table 10.4 A List of Triangles No. Vertex 1 Vertex 2 Vertex 3 I123 II135 III 3 4 5 IV243 V156 VI475 . . . The recording ofgeometric information isillustrated in Figure10.3 and Table 10.3 and Table 10.4, that is, a table of points containing all their coordinates and a table of triangles with their corresponding three vertices. Apart from geometric information, the topological information is recorded for efficient retrieval of data. Table 10.5 lists the adjacent relations between these triangles. The file structure for a TIN DTM is simply the list of points with their coordinates, with some metadata also included in the header. The file structure is like that given in Table 10.6. The topological information about these triangles is stored either in a databaseor in a file. Table 10.7 illustrates a possible arrangement of such topological information in a file. © 2005 by CRC Press DITM: “tf1732_c010” — 2004/10/22 — 16:37 — page 216 — #6 216 DIGITAL TERRAIN MODELING: PRINCIPLES AND METHODOLOGY Table 10.5 Adjacent Relations of Triangles Triangle Edge Neighbors I — IV II II I III V III IV VI II IV — III I VII—— VI ——III . . . Table 10.6 Typical File Structure for TIN Point Coordinates File Components Contents Comments Header The coordinates of the points on the boundary (convex hull); ranges of X , Y , and Z coordinates; coordinate data type; data types; numbers of points; position (in the file) where the body starts; position (in the file) where the footer starts; use of compression or not; etc. The information in the footer can also be recorded here Body X , Y, and Z coordinates of points in sequence May also be in blocks Footer Data describing the general characteristics of DTM, e.g., name, producer, projection parameters, version, accuracy, date of production, date of revision, linage, the null points code, etc. Metadata Table 10.7 Typical File Structure for TIN Topology File Components Contents Comments Header Number of triangles, the bytes of data for Table 10.4 or Table 10.5, data types, etc. The information in the footer can also be recorded here Body Information in Table 10.4 or information in Table 10.5 Adjacent triangular topology is not always necessary Footer Other relevant information Metadata 10.2.3 File Structure for Additional Terrain Feature Data As discussed in Chapter 4, a hybrid DTM network may be generated if data from composite sampling (i.e., grid plus feature points and lines) are used. In normal practice, the grid and feature data are stored in separate files. When modeling or interpolation is needed, grids are split into triangles and feature points and lines are added to the regular triangular network to update local triangles (Figure 10.4). © 2005 by CRC Press DITM: “tf1732_c010” — 2004/10/22 — 16:37 — page 217 — #7 MANAGEMENT OF DTM DATA 217 Figure 10.4 Hybrid of regular grid and TIN. ··· ··· ··· ··· 1(line ID), N 1 (No.of points on line 1) 2(line ID), N 2 (No.of points on line 2) X 11 , Y 11 , Z 11 X 21 , Y 21 , Z 21 X 22 , Y 22 , Z 22 X 12 , Y 12 , Z 12 X 1N 1 , Y 1N 1 , Z 1N 1 X 2N 2 , Y 2N 2 , Z 2N 2 Figure 10.5 The body of vector file structure for terrain feature data. Feature data may be stored in one or two files, one for points and the other for lines. The file structure for terrain feature points is similar to that for the points of TINs. However, for lines, it is slightly different. In the header, the number of lines is specified and in the body the data could be organized as shown in Figure 10.5. 10.3 MANAGEMENT OF DTM DATA WITH SPATIAL DATABASES In the previous section, the filestructures for bothgrid and TIN DTMs were discussed. These files are managed using an indexing system, which can be organized into files or into tables and managed by a database if the indexing is rather complex. In this case, an ordinary relational database will serve for the purpose. On the other hand, © 2005 by CRC Press DITM: “tf1732_c010” — 2004/10/22 — 16:37 — page 218 — #8 218 DIGITAL TERRAIN MODELING: PRINCIPLES AND METHODOLOGY the DTM data can also be organized into tables in an object-relational database, in which DTM data are stored in block as a field. 10.3.1 Organization of Tables for Grid DTM Data A large area (e.g., a country) may be divided into a number of smaller regions (e.g., provinces) and each region can be further divided into a number of smaller units called tiles. Each tile may also be further divided into a number of small units. This is a hierarchical structure and can be indexed efficiently for the management of DTM in a grid form. Figure 10.6 shows an indexing system for the hierarchy in three levels, region, tile, and block. It is not necessary to have rectangular shapes for the tiles. For example, the boundaries will be irregular if the DTM data of a nation is managed based on drainage area or administrative region. In some commercial systems, the block is the basic unit for access and retrieval. Each block comprises many rows and columns. Through the structural index for “region–tile–block–row–column,” the height of any location within the database can be uniquely determined. The spatial index formed by the region–block–unit hierarchy ensures fast retrieval of and seamless access to DTM data. The arrange- ment of tables for a regional DTM in an object-relational database is shown in Table 10.8, Table 10.9, and Table 10.10, which are created by the authors for illustration purposes only. The above data organization methodmay also apply to TINs for large areas. As the TIN boundary of each region is irregular, toavoidthe edge-matching problem between adjacent blocks, a certain degree of overlapping is necessary in block partitioning. Suppose each region is organized into a database. There are only four fields in a record. This is illustrated in Table 10.8. In Table 10.8, the field Region-table-name is the name of the table containing DTM data (see Table 10.9); the field Region-DTM-info is an abstract data type using database BLOB field (variable length), that is, a data stream, and has a Standard Block 13 00 01 02 03 10 22 21 20 23 13 12 Standard tile 11 Column 3 Row 5 Grid cell 1*6 Figure 10.6 Hierarchical structure based on region–tile–block (Modified from ESRI, 1992). © 2005 by CRC Press DITM: “tf1732_c010” — 2004/10/22 — 16:37 — page 219 — #9 MANAGEMENT OF DTM DATA 219 Table 10.8 An Index Table for a Regional DTM Region-ID Region-Table-Name Region-DTM-info Range-of-region 1 GridDEM50000_1 GridDTM50000_1_INFO GridDTM50000_1_ENVELOPE 2 GridDEM50000_2 GridDEM50000_2_INFO GridDTM50000_2_ENVELOPE . . . . . . . . . . . . structure that contains the information about the region. For example, the struc- ture GridDTM50000_1_INFO contains all the tile and block information about this region. The following is an example: * GridDTM50000_1_INFO { int XtilesNum; //number of tiles in column direction, e.g., four in Figure 10.6// int YTilesNum; //number of tiles in row direction, e.g., three in Figure 10.6// int XBlocksNum; //number of blocks in each tile, in column direction, e.g., five in Figure 10.6// int YBlocksNum; //number of blocks in each tile, in row direction, e.g., five in Figure 10.6// int BlockRow; //number of rows in each block, e.g., seven in Figure 10.6// int BlockColumn; //number of columns in each block, e.g., eight in Figure 10.6// float BlockCellSize; //interval of DTM cells, e.g., 25.0 for 25.0 m// int Scale; //scale factor of the DTM, e.g., 50,000 for 1 : 50,000// BOOL bOriDataLayer; //whether it is original or updated, e.g., TRUE if original// BOOL bCompressed; //whether or not data compression is used, e.g., FALSE if no compression// }; © 2005 by CRC Press DITM: “tf1732_c010” — 2004/10/22 — 16:37 — page 220 — #10 220 DIGITAL TERRAIN MODELING: PRINCIPLES AND METHODOLOGY In Table 10.8, the field Range-of-region is also the abstract data type BLOB, that is, a pointer to a structure that contains the coordinates of the four corners of the region. For example, the structure GridDTM50000_1_ENVELOPE may contain: GridDTM50000_1_ENVELOPE { float XMin; //the smallest X coordinates, e.g., 850,000 for 850,000 m// float XMax; //the largest X coordinates, e.g., 860,000 for 860,000 m// float YMin; //the smallest Y coordinates, e.g., 810,000 for 810,000 m// float XMax; //the largest Y coordinates, e.g., 830,000 for 830,000 m// }; In Table 10.8, the index table of the DTM at region level is set for the logical structure of the region–block–tile–block hierarchy. The actual heights at grid nodes are arranged in blocks and stored in a table with a name given under the field Region-table-name. In other words, the height data are stored block by block. Three different ways have been used to organize data in blocks, which are shown in Tables 10.9 and Table 10.10. In Table 10.9, the Block-ID is the main key, which is unique to each block. Each Block-ID consists of four numbers. The first two indicate the location of the corresponding tile (which contains this block) in the region, one for the numbering Table 10.9 Organization of DTM Height Data for Region GridDEM50000_1 in Block Block-ID Bytes-of-Block Block-Data 0000 112 h 0,0 h 0,1 h 0,6 h 1,0 h 6,7 (of Block 0000) . . . . . . . . . 1113 112 h 0,0 h 0,1 h 0,6 h 1,0 h 6,7 (of Block 1133) . . . . . . . . . 2344 112 h 0,0 h 0,1 h 0,6 h 1,0 h 6,7 (of Block 2344) Table 10.10 Organization of DTM Height Data for Region GridDEM50000_1 in Tiles FILE-ID DTM-Info DTM-Data 00 DTMINFO00 Heights at tile 00 01 DTMINFO01 Heights at tile 01 . . . . . . . . . 23 ∗ DTMINFO23 Heights at tile 23 © 2005 by CRC Press [...]... “tf1732_c 010 — 2004 /10/ 22 — 16:37 — page 225 — #15 226 DIGITAL TERRAIN MODELING: PRINCIPLES AND METHODOLOGY Table 10. 15 An Example of a Metadata Set in a Table Table-ID Table-Name Institution-Name Product Updating-Date Scale 100 ,000 PubMetadata GeomaticsCenter DTM 1 0-1 2-2 004 50,000 Table 10. 16 Standards of Metadata Name of Standards Institution or Organization Content Standard for Digital. .. data sets, there must be some standards for them, including accuracy © 2005 by CRC Press DITM: “tf1732_c 010 — 2004 /10/ 22 — 16:37 — page 229 — #19 230 DIGITAL TERRAIN MODELING: PRINCIPLES AND METHODOLOGY and format In this chapter, only format is discussed as this chapter deals with data management 10. 5.1 Concepts and Principles of DTM Data Standards Data are important and expensive It is therefore important... the table formats in a spatial database In Table 10. 12, BLOCK-ID is the main key of the data table (i.e., ID), Both Triangle-List and Point-list are data streams (i.e., type BLOB) Triangle-List contains the data for Table 10. 4 and Table 10. 5 The Point-list contains the coordinates of points in Table 10. 2 © 2005 by CRC Press DITM: “tf1732_c 010 — 2004 /10/ 22 — 16:37 — page 222 — #12 MANAGEMENT OF DTM... coordinates of all the points on a line In fact, the terrain features could be points, line, and areas If there is more than one type of terrain feature, an indexing table can be used to manage them Table 10. 14 is an example © 2005 by CRC Press DITM: “tf1732_c 010 — 2004 /10/ 22 — 16:37 — page 223 — #13 224 DIGITAL TERRAIN MODELING: PRINCIPLES AND METHODOLOGY Point Unique ID (OID) Type Serial number... attributes Pointer to edge Pointer to triangle Other attributes Figure 10. 8 Pointer structure of a point in a TIN database Line Unique ID (OID) Extent of 3-D coordinate Coordinate point number 3-D coordinates Other attributes Figure 10. 9 Table 10. 13 Line-ID The Linear Entity Data Table Line-Type Number-of-points Coordinates-of-the-Line 1 3 N1 N2 X11 , Y11 , Z11 X12 , Y12 , Z12 X1N1 , Y1N1 ,... PRINCIPLES AND METHODOLOGY Table 10. 11 Region-ID The TIN Block Indexing Table Region-Info Range-of-Region TIN50000_0 TIN50000_1 TIN50000_K 0 1 K Region-Table-Name TIN50000_0_INFO TIN50000_1_INFO TIN50000_K _INFO TIN50000_0_ENVELOPE TIN50000_1_ENVELOPE TIN50000_M _ENVELOPE The fields of Table 10. 11 are defined as the ID of the region, the name of the region, other information about the region, and. .. shown in Figure 10. 7 Three tables are required for this structure, one for point coordinates, one for the relationship between a triangle and its three vertices, and one for the relationship between a triangle and its three edge neighbors, for example the tables given in Section 10. 2.2, Table 10. 3, Table 10. 4, and Table 10. 5 These three tables can also be stored in a database table Table 10. 12 illustrates... largest frequency value at the top Table 10. 17 213 216 214 212 212 212 213 212 215 213 213 213 A Set of DTM Data in a Grid 213 213 215 213 215 213 212 212 213 213 214 213 212 212 216 214 213 213 213 216 215 214 212 212 © 2005 by CRC Press DITM: “tf1732_c 010 — 2004 /10/ 22 — 16:37 — page 227 — #17 228 DIGITAL TERRAIN MODELING: PRINCIPLES AND METHODOLOGY Table 10. 18 Huffman Coding for Data Compression... 2 Table 10. 14 Region-ID 0 1 K Line structure of linear entity An Indexing Table for Terrain Features Region-table-name Feature-Info Range-of-region Feature50000_0 Feature50000_1 Feature50000_K Feature50000_0_INFO Feature50000_1_INFO Feature50000_K _INFO Feature50000_0_ENVELOPE Feature50000_1_ENVELOPE Feature50000_K _ENVELOPE © 2005 by CRC Press DITM: “tf1732_c 010 — 2004 /10/ 22 — 16:37... the links between a triangle and its three vertices, between a node and its adjacent nodes, and between a triangle and its adjacent triangles, can also be set up in the database using a pointer system for vertices This structure is given in Figure 10. 8 10. 3.3 Organization of Tables for Additional Terrain Feature Data As has been discussed in Section 10. 2.3, if additional terrain feature data are available, . 2005 by CRC Press DITM: “tf1732_c 010 — 2004 /10/ 22 — 16:37 — page 220 — #10 220 DIGITAL TERRAIN MODELING: PRINCIPLES AND METHODOLOGY In Table 10. 8, the field Range-of-region is also the abstract data. “tf1732_c 010 — 2004 /10/ 22 — 16:37 — page 226 — #16 226 DIGITAL TERRAIN MODELING: PRINCIPLES AND METHODOLOGY Table 10. 15 An Example of a Metadata Set in a Table Table-ID Table-Name Institution-Name. … … … …………… … 56 Figure 10. 2 Matrix representation of grid DTM data. © 2005 by CRC Press DITM: “tf1732_c 010 — 2004 /10/ 22 — 16:37 — page 214 — #4 214 DIGITAL TERRAIN MODELING: PRINCIPLES AND METHODOLOGY Table 10. 2