Data Warehouse Architecture and Models
Lesson Data Warehouse Architecture and Models Objectives Differentiate between an enterprise-wide data warehouse and localized data marts Recognize the difference between independent and dependent data marts Identify the data that is stored in a data warehouse Explain the features of each type of data by examining where and why it is used List the data models that may already exist in a company and describe where they may be useful to the warehouse model Explain the two common data warehouse models Warehouse Architectures Enterprise-wide solution Data mart solution Combined solution Enterprise Data Warehouse Solution Large warehouse containing all business data Incremental implementation First increment provides proof-of-concept Funded at corporate level Data Mart Solution Independent Use a consistent approach Avoid disjointed development Consider the big picture Dependent Localization Subsets of summary data Oracle Data Mart Suite for a pre-configured solution Funded departmentally Dependent Independent Create a Project Team Staff with experts Database designers Database administrators Network specialists ETT specialists Project managers Consider training plans Identify the Data Requirements Successful warehouses provide the right information Analyze business users’ needs Interview users Examine data needed Ascertain data availability Determine data frequency Decide the refresh cycle Types of Warehouse Data Fact data - Measures Dimension data - Query drivers Summary data - Pre-calculated data Fact Data and Tables Many fact tables in the warehouse The bulk of the warehouse data Measures (units sold, sales figures, calls) Millions of rows Multi-part primary keys Summaries Normalized data Partitioning Data Breaking tables into smaller units Horizontal or vertical 1994 1991 1992 1993 Vertical Partitioning Vertical partitioning by column Col A Col B Col C Col D • Oracle servers support both types of partitioning Dimension Data • Provides query criteria • Links to fact tables with keys • Needs update strategy Query path Customer Customer Location Location Sales Sales Time Time Product Product Dimension Tables Are determined by user requirements Region Region Vary in number May contain hierarchies District District May share fact tables Customer Customer Sales Sales Summary Summary Office Office Sales Sales Time Time Product Product Time Dimension Important required dimension Contains special dates Provides flexible and accurate analysis by time Days Weeks Months Quarters Fiscal Period Weekdays Weekends Holidays Special Events Promotion Dates Customer Customer Office Office Sales Sales Time Time Product Product Summary Data Aggregated facts Lightly summarized Highly summarized Cumulative or rolling data Key columns to dimensions Totsales Totsales Totunits Totunits Summary Tables Provide immediate answers to a query Improve query performance SALES FACTS Sales$ Region Month 10,000 North Jan 97 12,000 South Feb 97 11,000 North Jan 97 15,000 West Mar 97 18,000 South Feb 97 20,000 North Jan 97 10,000 East Jan 97 2,000 West Mar 97 SALES BY MONTH/REGION Month Region Tot_Sales$ Jan 97 North 41,000 Jan 97 East 10,000 Feb 97 South 40,000 Mar 97 West 17,000 SALES BY MONTH Month Tot_Sales Jan 97 51,000 Feb 97 40,000 Mar 97 17,000 Requirements change and need managing Modeling Define the model Use graphical modeling tools Use a tool capable of prototyping Proof of concept essential New design techniques The Enterprise Model Defines an overall scope A good start point for analysis An information framework for the warehouse A guide to integration Sales Accounts Receivable Marketing HR Accounts Payable Inventory Never use directly for the warehouse The Corporate Data Model Current operational data structures Source of mapping rules for the warehouse Marketing Never use directly for the warehouse A Typical Modeling Approach Analyze the subject area Gather the requirements Develop the models Map the entities, attributes, and relationships Re-engineer the source data Design the database Integrate the model into the warehouse architecture repository Review with client and revise Oracle Data Warehouse Method supports an iterative approach Star Model Physical model Denormalized Store Table Store Table Fast query response Store_id Store_id District_id District_id Flexible design Many query tools Sales Fact Table Sales Fact Table Item_id Item_id Store_id Store_id Sales_dollars Sales_dollars Sales_units Sales_units Time Table Time Table Week_id Week_id Period_id Period_id Year_id Year_id Item Table Item Table Item_id Item_id Dept_id Dept_id Totsales Summary Totsales Summary Month_id Month_id Store_id Store_id Item_id Item_id Total_dollars Total_dollars Snowflake Model Store Table Store Table Store_id Store_id Store_desc Store_desc District_id District_id Sales Fact Table Sales Fact Table Item_id Item_id Store_id Store_id Sales_dollars Sales_dollars Sales_units Sales_units Time Table Time Table Week_id Week_id Period_id Period_id Year_id Year_id Item Table Item Table Item_id Item_id Item_desc Item_desc Dept_id Dept_id District Table District Table District_id District_id District_desc District_desc Totsales Totsales Month_id Month_id Store_id Store_id Item_id Item_id Total_dollars Total_dollars Dept Table Dept Table Dept_id Dept_id Dept_desc Dept_desc Mgr Table Mgr Table Dept_id Dept_id Mgr_id Mgr_id Snowflake Model A normalized star schema Easier to model requirements Flexible dimension structures Readily maps to existing data Used directly by tools Database servers Star queries Star joins VLDB support Which Model Do you Use? Design for simplicity Design for relevance Simple star model Flexible snowflake model Summary Enterprise data warehouses Data marts Independent Dependant Warehouse data Data models Models compared ... an enterprise-wide data warehouse and localized data marts Recognize the difference between independent and dependent data marts Identify the data that is stored in a data warehouse Explain the... of data by examining where and why it is used List the data models that may already exist in a company and describe where they may be useful to the warehouse model Explain the two common data warehouse. .. warehouse models Warehouse Architectures Enterprise-wide solution Data mart solution Combined solution Enterprise Data Warehouse Solution Large warehouse containing all business data