Oracle® Database Data Warehousing Guide 10g Release (10.1) Part No B10736-01 December 2003 Oracle Database Data Warehousing Guide, 10g Release (10.1) Part No B10736-01 Copyright © 2001, 2003 Oracle Corporation All rights reserved Primary Author: Paul Lane Contributing Authors: Viv Schupmann, Ingrid Stuart (Change Data Capture) Contributors: Patrick Amor, Hermann Baer, Mark Bauer, Subhransu Basu, Srikanth Bellamkonda, Randy Bello, Tolga Bozkaya, Lucy Burgess, Rushan Chen, Benoit Dageville, John Haydu, Lilian Hobbs, Hakan Jakobsson, George Lumpkin, Alex Melidis, Valarie Moore, Cetin Ozbutun, Ananth Raghavan, Jack Raitto, Ray Roccaforte, Sankar Subramanian, Gregory Smith, Murali Thiyagarajan, Ashish Thusoo, Thomas Tong, Jean-Francois Verrier, Gary Vincent, Andreas Walter, Andy Witkowski, Min Xiao, Tsae-Feng Yu The Programs (which include both the software and documentation) contain proprietary information of Oracle Corporation; they are provided under a license agreement containing restrictions on use and disclosure and are also protected by copyright, patent and other intellectual and industrial property laws Reverse engineering, disassembly or decompilation of the Programs, except to the extent required to obtain interoperability with other independently created software or as specified by law, is prohibited The information contained in this document is subject to change without notice If you find any problems in the documentation, please report them to us in writing Oracle Corporation does not warrant that this document is error-free Except as may be expressly permitted in your license agreement for these Programs, no part of these Programs may be reproduced or transmitted in any form or by any means, electronic or mechanical, for any purpose, without the express written permission of Oracle Corporation If the Programs are delivered to the U.S Government or anyone licensing or using the programs on behalf of the U.S Government, the following notice is applicable: Restricted Rights Notice Programs delivered subject to the DOD FAR Supplement are "commercial computer software" and use, duplication, and disclosure of the Programs, including documentation, shall be subject to the licensing restrictions set forth in the applicable Oracle license agreement Otherwise, Programs delivered subject to the Federal Acquisition Regulations are "restricted computer software" and use, duplication, and disclosure of the Programs shall be subject to the restrictions in FAR 52.227-19, Commercial Computer Software - Restricted Rights (June, 1987) Oracle Corporation, 500 Oracle Parkway, Redwood City, CA 94065 The Programs are not intended for use in any nuclear, aviation, mass transit, medical, or other inherently dangerous applications It shall be the licensee's responsibility to take all appropriate fail-safe, backup, redundancy, and other measures to ensure the safe use of such applications if the Programs are used for such purposes, and Oracle Corporation disclaims liability for any damages caused by such use of the Programs Oracle is a registered trademark, and Express, Oracle8i, Oracle9i, Oracle Store, PL/SQL, Pro*C, and SQL*Plus are trademarks or registered trademarks of Oracle Corporation Other names may be trademarks of their respective owners Contents Send Us Your Comments xxix Preface xxxi Audience xxxii Organization xxxii Related Documentation xxxiv Conventions xxxv Documentation Accessibility xxxviii What's New in Oracle Database? xxxix Oracle Database 10g Release (10.1) New Features in Data Warehousing xl Volume Part I Concepts Data Warehousing Concepts What is a Data Warehouse? Subject Oriented Integrated Nonvolatile Time Variant Contrasting OLTP and Data Warehousing Environments Data Warehouse Architectures 1-2 1-2 1-2 1-3 1-3 1-3 1-5 iii Data Warehouse Architecture (Basic) 1-5 Data Warehouse Architecture (with a Staging Area) 1-6 Data Warehouse Architecture (with a Staging Area and Data Marts) 1-6 Part II Logical Design Logical Design in Data Warehouses Logical Versus Physical Design in Data Warehouses Creating a Logical Design Data Warehousing Schemas Star Schemas Other Schemas Data Warehousing Objects Fact Tables Creating a New Fact Table Dimension Tables Hierarchies Typical Dimension Hierarchy Unique Identifiers Relationships Example of Data Warehousing Objects and Their Relationships Part III Physical Design Physical Design in Data Warehouses Moving from Logical to Physical Design Physical Design Physical Design Structures Tablespaces Tables and Partitioned Tables Table Compression Views Integrity Constraints Indexes and Partitioned Indexes iv 2-2 2-2 2-3 2-4 2-4 2-5 2-5 2-5 2-6 2-6 2-7 2-7 2-7 2-8 3-2 3-2 3-3 3-4 3-4 3-5 3-5 3-5 3-6 Materialized Views 3-6 Dimensions 3-6 Hardware and I/O Considerations in Data Warehouses Overview of Hardware and I/O Considerations in Data Warehouses Configure I/O for Bandwidth not Capacity Stripe Far and Wide Use Redundancy Test the I/O System Before Building the Database Plan for Growth Storage Management 4-2 4-2 4-3 4-3 4-4 4-4 4-4 Parallelism and Partitioning in Data Warehouses Overview of Parallel Execution When to Implement Parallel Execution Granules of Parallelism Block Range Granules Partition Granules Partitioning Design Considerations Types of Partitioning Partitioning Methods Index Partitioning Performance Issues for Range, List, Hash, and Composite Partitioning Partitioning and Table Compression Table Compression and Bitmap Indexes Example of Table Compression and Partitioning Partition Pruning Pruning Using DATE Columns Avoiding I/O Bottlenecks Partition-Wise Joins Full Partition-Wise Joins Partial Partition-wise Joins Benefits of Partition-Wise Joins Performance Considerations for Parallel Partition-Wise Joins Partitioning and Subpartitioning Columns and Keys 5-2 5-2 5-3 5-3 5-4 5-4 5-4 5-5 5-9 5-9 5-16 5-17 5-18 5-19 5-20 5-20 5-20 5-20 5-26 5-28 5-29 5-30 v Partition Bounds for Range Partitioning Comparing Partitioning Keys with Partition Bounds MAXVALUE Nulls DATE Datatypes Multicolumn Partitioning Keys Implicit Constraints Imposed by Partition Bounds Index Partitioning Local Partitioned Indexes Global Partitioned Indexes Summary of Partitioned Index Types The Importance of Nonprefixed Indexes Performance Implications of Prefixed and Nonprefixed Indexes Guidelines for Partitioning Indexes Physical Attributes of Index Partitions Indexes Using Bitmap Indexes in Data Warehouses Benefits for Data Warehousing Applications Cardinality Bitmap Indexes and Nulls Bitmap Indexes on Partitioned Tables Using Bitmap Join Indexes in Data Warehouses Four Join Models for Bitmap Join Indexes Bitmap Join Index Restrictions and Requirements Using B-Tree Indexes in Data Warehouses Using Index Compression Choosing Between Local Indexes and Global Indexes 6-2 6-2 6-3 6-5 6-6 6-6 6-6 6-9 6-10 6-10 6-11 Integrity Constraints Why Integrity Constraints are Useful in a Data Warehouse Overview of Constraint States Typical Data Warehouse Integrity Constraints UNIQUE Constraints in a Data Warehouse FOREIGN KEY Constraints in a Data Warehouse vi 5-31 5-31 5-31 5-32 5-32 5-33 5-33 5-33 5-34 5-37 5-39 5-40 5-40 5-41 5-42 7-2 7-3 7-3 7-4 7-5 RELY Constraints Integrity Constraints and Parallelism Integrity Constraints and Partitioning View Constraints 7-6 7-6 7-7 7-7 Basic Materialized Views Overview of Data Warehousing with Materialized Views Materialized Views for Data Warehouses Materialized Views for Distributed Computing Materialized Views for Mobile Computing The Need for Materialized Views Components of Summary Management Data Warehousing Terminology Materialized View Schema Design Schemas and Dimension Tables Materialized View Schema Design Guidelines Loading Data into Data Warehouses Overview of Materialized View Management Tasks Types of Materialized Views Materialized Views with Aggregates Requirements for Using Materialized Views with Aggregates Materialized Views Containing Only Joins Materialized Join Views FROM Clause Considerations Nested Materialized Views Why Use Nested Materialized Views? Nesting Materialized Views with Joins and Aggregates Nested Materialized View Usage Guidelines Restrictions When Using Nested Materialized Views Creating Materialized Views Creating Materialized Views with Column Alias Lists Naming Materialized Views Storage And Table Compression Build Methods Enabling Query Rewrite Query Rewrite Restrictions 8-2 8-2 8-3 8-3 8-3 8-5 8-7 8-8 8-8 8-9 8-10 8-11 8-12 8-12 8-15 8-15 8-16 8-17 8-17 8-19 8-19 8-20 8-20 8-21 8-22 8-22 8-23 8-24 8-24 vii Materialized View Restrictions General Query Rewrite Restrictions Refresh Options General Restrictions on Fast Refresh Restrictions on Fast Refresh on Materialized Views with Joins Only Restrictions on Fast Refresh on Materialized Views with Aggregates Restrictions on Fast Refresh on Materialized Views with UNION ALL Achieving Refresh Goals Refreshing Nested Materialized Views ORDER BY Clause Materialized View Logs Using the FORCE Option with Materialized View Logs Using Oracle Enterprise Manager Using Materialized Views with NLS Parameters Adding Comments to Materialized Views Registering Existing Materialized Views Choosing Indexes for Materialized Views Dropping Materialized Views Analyzing Materialized View Capabilities Using the DBMS_MVIEW.EXPLAIN_MVIEW Procedure DBMS_MVIEW.EXPLAIN_MVIEW Declarations Using MV_CAPABILITIES_TABLE MV_CAPABILITIES_TABLE.CAPABILITY_NAME Details MV_CAPABILITIES_TABLE Column Details Advanced Materialized Views Partitioning and Materialized Views Partition Change Tracking Partition Key Join Dependent Expression Partition Marker Partial Rewrite Partitioning a Materialized View Partitioning a Prebuilt Table Benefits of Partitioning a Materialized View viii 8-24 8-25 8-25 8-27 8-27 8-27 8-29 8-30 8-30 8-31 8-31 8-33 8-33 8-33 8-33 8-34 8-36 8-37 8-37 8-37 8-38 8-38 8-40 8-42 9-2 9-2 9-3 9-4 9-5 9-6 9-7 9-7 9-8 Rolling Materialized Views Materialized Views in OLAP Environments OLAP Cubes Partitioning Materialized Views for OLAP Compressing Materialized Views for OLAP Materialized Views with Set Operators Examples of Materialized Views Using UNION ALL Materialized Views and Models Invalidating Materialized Views Security Issues with Materialized Views Querying Materialized Views with Virtual Private Database Using Query Rewrite with Virtual Private Database Restrictions with Materialized Views and Virtual Private Database Altering Materialized Views 10 Dimensions What are Dimensions? Creating Dimensions Dropping and Creating Attributes with Columns Multiple Hierarchies Using Normalized Dimension Tables Viewing Dimensions Using Oracle Enterprise Manager Using the DESCRIBE_DIMENSION Procedure Using Dimensions with Constraints Validating Dimensions Altering Dimensions Deleting Dimensions Part IV 11 9-9 9-9 9-9 9-10 9-11 9-11 9-11 9-13 9-14 9-14 9-15 9-16 9-16 9-17 10-2 10-4 10-8 10-9 10-10 10-11 10-11 10-11 10-12 10-12 10-14 10-14 Managing the Data Warehouse Environment Overview of Extraction, Transformation, and Loading Overview of ETL in Data Warehouses 11-2 ETL Tools for Data Warehouses 11-3 ix Daily Operations in Data Warehouses 11-3 Evolution of the Data Warehouse 11-4 12 Extraction in Data Warehouses Overview of Extraction in Data Warehouses 12-2 Introduction to Extraction Methods in Data Warehouses 12-2 Logical Extraction Methods 12-3 Full Extraction 12-3 Incremental Extraction 12-3 Physical Extraction Methods 12-4 Online Extraction 12-4 Offline Extraction 12-4 Change Data Capture 12-5 Timestamps 12-6 Partitioning 12-6 Triggers 12-6 Data Warehousing Extraction Examples 12-7 Extraction Using Data Files 12-7 Extracting into Flat Files Using SQL*Plus 12-8 Extracting into Flat Files Using OCI or Pro*C Programs 12-9 Exporting into Export Files Using the Export Utility 12-10 Extracting into Export Files Using External Tables 12-10 Extraction Through Distributed Operations 12-11 13 Transportation in Data Warehouses Overview of Transportation in Data Warehouses Introduction to Transportation Mechanisms in Data Warehouses Transportation Using Flat Files Transportation Through Distributed Operations Transportation Using Transportable Tablespaces Transportable Tablespaces Example Other Uses of Transportable Tablespaces x 13-2 13-2 13-2 13-2 13-3 13-3 13-6 ... What''s New in Oracle Database? xxxix Oracle Database 10g Release (10.1) New Features in Data Warehousing xl Volume Part I Concepts Data Warehousing Concepts What is a Data Warehouse?.. .Oracle Database Data Warehousing Guide, 10g Release (10.1) Part No B10736-01 Copyright © 2001, 2003 Oracle Corporation All rights reserved Primary... Materialized Views with Virtual Private Database Using Query Rewrite with Virtual Private Database Restrictions with Materialized Views and Virtual Private Database Altering Materialized