Oracle® Database SecureFiles and Large Objects Developer's Guide 11g Release 2 (11.2) E18294-02 January 2013 Oracle Database SecureFiles and Large Objects Developer's Guide, 11g Release 2 (11.2) E18294-02 Copyright © 1996, 2013, Oracle and/or its affiliates. All rights reserved. Primary Author: Roza Leyderman Contributors: Bharath Aleti, Geeta Arora, Thomas H. Chang, Maria Chien, Subramanyam Chitti, Amit Ganesh, Kevin Jernigan, Vikram Kapoor, Balaji Krishnan, Jean de Lavarene, Geoff Lee, Scott Lynn, Jack Melnick, Atrayee Mullick, Eric Paapanen, Ravi Rajamani, Kam Shergill, Ed Shirk, Srinivas Vemuri This software and related documentation are provided under a license agreement containing restrictions on use and disclosure and are protected by intellectual property laws. Except as expressly permitted in your license agreement or allowed by law, you may not use, copy, reproduce, translate, broadcast, modify, license, transmit, distribute, exhibit, perform, publish, or display any part, in any form, or by any means. Reverse engineering, disassembly, or decompilation of this software, unless required by law for interoperability, is prohibited. The information contained herein is subject to change without notice and is not warranted to be error-free. If you find any errors, please report them to us in writing. If this is software or related documentation that is delivered to the U.S. Government or anyone licensing it on behalf of the U.S. Government, the following notice is applicable: U.S. GOVERNMENT END USERS: Oracle programs, including any operating system, integrated software, any programs installed on the hardware, and/or documentation, delivered to U.S. Government end users are "commercial computer software" pursuant to the applicable Federal Acquisition Regulation and agency-specific supplemental regulations. As such, use, duplication, disclosure, modification, and adaptation of the programs, including any operating system, integrated software, any programs installed on the hardware, and/or documentation, shall be subject to license terms and license restrictions applicable to the programs. No other rights are granted to the U.S. Government. This software or hardware is developed for general use in a variety of information management applications. It is not developed or intended for use in any inherently dangerous applications, including applications that may create a risk of personal injury. If you use this software or hardware in dangerous applications, then you shall be responsible to take all appropriate fail-safe, backup, redundancy, and other measures to ensure its safe use. Oracle Corporation and its affiliates disclaim any liability for any damages caused by use of this software or hardware in dangerous applications. Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective owners. Intel and Intel Xeon are trademarks or registered trademarks of Intel Corporation. All SPARC trademarks are used under license and are trademarks or registered trademarks of SPARC International, Inc. AMD, Opteron, the AMD logo, and the AMD Opteron logo are trademarks or registered trademarks of Advanced Micro Devices. UNIX is a registered trademark of The Open Group. This software or hardware and documentation may provide access to or information on content, products, and services from third parties. Oracle Corporation and its affiliates are not responsible for and expressly disclaim all warranties of any kind with respect to third-party content, products, and services. Oracle Corporation and its affiliates will not be responsible for any loss, costs, or damages incurred due to your access to or use of third-party content, products, or services. v Contents Preface xxiii Audience xxiii Documentation Accessibility xxiii Related Documents xxiii Conventions xxv What's New in Oracle Database SecureFiles and Large Objects Developer's Guide? xxvii LOB Features Introduced in Oracle Database 11g Release 2 xxvii LOB Features Introduced in Oracle Database 11g Release 1 xxvii 1 Introduction to Large Objects What Are Large Objects? 1-1 Why Use Large Objects? 1-1 Using LOBs for Semi-structured Data 1-2 Using LOBs for Unstructured Data 1-2 Why Not Use LONGs? 1-3 Different Kinds of LOBs 1-3 Internal LOBs 1-3 External LOBs and the BFILE Data Type 1-4 Introducing LOB Locators 1-4 Database Semantics for Internal and External LOBs 1-5 Large Object Data Types 1-5 Object Data Types and LOBs 1-5 Storing and Creating Other Data Types with LOBs 1-6 VARRAYs Stored as LOBs 1-6 XMLType Columns Stored as CLOBs 1-6 LOBs Used in Oracle Multimedia 1-6 Part I Getting Started 2 Working with LOBs LOB Column States 2-1 Locking a Row Containing a LOB 2-2 vi Opening and Closing LOBs 2-2 LOB Locator and LOB Value 2-2 Using the Data Interface for LOBs 2-2 Using the LOB Locator to Access and Modify LOB Values 2-3 LOB Locators and BFILE Locators 2-3 Table print_media 2-3 Initializing a LOB Column to Contain a Locator 2-4 Initializing a Persistent LOB Column 2-4 Initializing BFILEs 2-5 Accessing LOBs 2-5 Accessing a LOB Using SQL 2-5 Accessing a LOB Using the Data Interface 2-5 Accessing a LOB Using the Locator Interface 2-6 LOB Rules and Restrictions 2-6 Rules for LOB Columns 2-6 Restrictions for LOB Operations 2-8 3 Managing LOBs: Database Administration Database Utilities for Loading Data into LOBs 3-1 Using SQL*Loader to Load LOBs 3-1 Using SQL*Loader to Populate a BFILE Column 3-2 Using Oracle Data Pump to Transfer LOB Data 3-4 Managing Temporary LOBs 3-4 Managing Temporary Tablespace for Temporary LOBs 3-4 Managing BFILEs 3-4 Rules for Using Directory Objects and BFILEs 3-5 Setting Maximum Number of Open BFILEs 3-5 Changing Tablespace Storage for a LOB 3-5 Part II SecureFiles LOBs 4 Using Oracle SecureFiles LOBs About SecureFiles LOBs 4-1 About Compression 4-2 About Deduplication 4-2 About Encryption 4-2 Using CREATE TABLE with SecureFiles LOBs 4-2 Parameters of CREATE TABLE for SecureFiles LOB 4-6 BASICFILE 4-7 SECUREFILE 4-7 CHUNK 4-7 RETENTION 4-7 MAXSIZE 4-7 FREEPOOLS 4-7 LOGGING, NOLOGGING, or FILESYSTEM_LIKE_LOGGING 4-7 FREELISTS or FREELIST GROUPS 4-8 vii PCTVERSION 4-8 COMPRESS or NOCOMPRESS 4-8 DEDUPLICATE or KEEP_DUPLICATES 4-8 ENCRYPT or DECRYPT 4-9 CREATE TABLE Compression 4-9 Usage Notes for CREATE TABLE Compression 4-9 Examples of CREATE TABLE Compression 4-9 CREATE TABLE Deduplication 4-10 Usage Notes for CREATE TABLE Deduplication 4-10 Examples of CREATE TABLE Deduplication 4-11 CREATE TABLE Encryption 4-12 Usage Notes for CREATE TABLE Encryption 4-12 Examples of CREATE TABLE Encryption 4-12 Using ALTER TABLE with SecureFiles LOBs 4-13 Parameters of CREATE TABLE for SecureFiles LOB 4-14 RETENTION 4-15 COMPRESS or NOCOMPRESS 4-15 DEDUPLICATE or KEEP_DUPLICATES 4-15 ENCRYPT or DECRYPT 4-15 ALTER TABLE Compression 4-15 Usage Notes for ALTER TABLE Compression 4-15 Examples of ALTER TABLE Compression 4-16 ALTER TABLE Deduplication 4-16 Usage Notes for ALTER TABLE Deduplication 4-16 Examples of ALTER TABLE Deduplication 4-16 ALTER TABLE Encryption 4-17 Usage Notes for ALTER TABLE Encryption 4-17 Examples of ALTER TABLE Encryption 4-17 Database File System Links 4-18 Overview of Database File System Links 4-18 Creating Database File System Links 4-19 Copying Database File System Links 4-20 Copying a Linked LOB Between Tables 4-20 Online Redefinition and DBFS Links 4-21 Transparent Read 4-21 Initialization Parameter db_securefile for SecureFiles LOBs 4-21 Compatibility and Upgrading 4-21 Migrating Columns from BasicFiles LOBs to SecureFiles LOBs 4-22 Preventing Generation of REDO Space when Migrating to SecureFiles LOBs 4-22 Online Redefinition for BasicFiles LOBs 4-22 Online Redefinition Advantages 4-22 Online Redefinition Disadvantages 4-22 Using Online Redefinition for Migrating Tables with BasicFiles LOBs 4-22 Parallel Online Redefinition 4-23 PL/SQL Packages for SecureFiles LOBs and DBFS 4-24 DBMS_LOB Package 4-24 GETOPTIONS() 4-26 viii SETOPTIONS() 4-26 ISSECUREFILE() 4-26 MOVE_TO_DBFS_LINK() 4-27 COPY_FROM_DBFS_LINK() 4-27 COPY_DBFS_LINK() 4-27 GET_DBFS_LINK() 4-27 SET_DBFS_LINK() 4-27 GET_DBFS_LINK_STATE() 4-27 DBFS_LINK_GENERATE_PATHNAME() 4-27 SETCONTENTTYPE() 4-28 GETCONTENTTYPE() 4-28 APPEND() 4-28 COMPARE() 4-28 CONVERTTOBLOB() 4-28 CONVERTTOCLOB() 4-28 COPY() 4-28 ERASE() 4-28 FRAGMENT_DELETE() 4-28 FRAGMENT_INSERT() 4-28 FRAGMENT_MOVE() 4-29 FRAGMENT_REPLACE() 4-29 LOADBLOBFROMFILE() 4-29 LOADCLOBFROMFILE() 4-29 LOADFROMFILE() 4-29 READ() 4-29 SUBSTR() 4-29 TRIM() 4-29 WRITE() 4-29 WRITEAPPEND() 4-29 DBMS_SPACE Package 4-30 SPACE_USAGE() 4-30 5 Introducing the Oracle Database File System Why a Database File System? 5-1 What is the Oracle Database File System (DBFS)? 5-1 6 DBFS File System Client Installing DBFS 6-1 DBFS Prerequisites 6-1 Installing FUSE (Linux Only) 6-2 DBFS Installation Home 6-2 Creating a File System 6-2 Dropping a File System 6-3 DBFS Mounting Interface 6-3 Mounting the DBFS Store 6-3 Unmounting a File System in Linux 6-5 Restrictions on Mounted File Systems 6-5 ix Mounting DBFS Through fstab Utility in Linux 6-5 Oracle Database Release 11.2.0.3 with Solaris 11 SRU7 6-6 Installing FUSE on Solaris 11 SRU7 and Later 6-6 Unmounting DBFS 6-6 Mounting the DBFS Store: Solaris-Specific Privileges 6-6 Mounting DBFS Through the vfstab Utility for Solaris 6-7 Using the DBFS Command Interface 6-7 Using DBFS 6-7 Creating a Directory 6-8 Listing a Directory 6-8 Copying Files and Directories 6-8 Removing Files and Directories 6-9 DBFS Administration 6-9 Using Oracle Wallet with DBFS Client 6-9 File System Security Model 6-10 Enabling Shared Root Access 6-10 Enabling DBFS Access Among Multiple RDBMS Users 6-11 Performing DBFS Diagnostics 6-14 Managing DBFS Client Failover 6-15 Sharing and Caching DBFS 6-15 Backing up DBFS 6-15 Backing up DBFS at the Database Level 6-16 Backing up DBFS through a File System Utility 6-16 Improving Small File Performance of DBFS 6-16 Enabling Advanced SecureFiles LOB Features for DBFS 6-16 7 DBFS Content API Overview of DBFS Content API 7-1 Stores and Package DBMS_DBFS_CONTENT 7-1 Getting Started with DBMS_DBFS_CONTENT Package 7-2 DBFS Content API Role 7-2 Path Name Constants and Types 7-2 Content Properties 7-2 Path Name Types 7-3 Store Features 7-4 Lock Types 7-6 Standard Properties 7-6 Optional Properties 7-7 Property Access Flags 7-8 Exceptions 7-9 Property Bundles 7-9 Store Descriptors 7-10 Administrative and Query APIs 7-10 Registering a Content Store 7-10 Unregistering A Content Store 7-10 Mounting a Registered Store 7-11 Unmounting a Previously Mounted Store 7-11 x List all Available Stores and Their Features 7-11 List all Available Mount Points 7-12 Look-up Specific Stores and Their Features 7-12 DBFS Content API Space Usage 7-12 DBFS Content API Session Defaults 7-12 DBFS Content API Interface Versioning 7-13 DBFS Content API Notes on Path Names 7-13 DBFS Content API Creation Operations 7-13 DBFS Content API Deletion Operations 7-14 DBFS Content API Path Get and Put Operations 7-14 DBFS Content API Rename and Move Operations 7-15 Directory Listings 7-15 DBFS Content API Directory Navigation and Search 7-15 DBFS Content API Locking Operations 7-16 DBFS Content API Abstract Operations 7-16 DBFS Content API Access Checks 7-16 DBFS Content API Path Normalization 7-16 DBFS Content API Statistics Support 7-17 DBFS Content API Tracing Support 7-17 Resource and Property Views 7-18 8 DBFS SecureFiles Store The DBFS SecureFiles Store Package, DBMS_DBFS_SFS 8-1 Creating and Registering a New SecureFiles Store 8-1 Initializing or Re-initializing a SecureFiles Store 8-1 Unregister and Drop A SecureFiles Store File System Store 8-2 Using a DBFS SecureFiles Store File System 8-2 Permissions Management 8-2 Creating a SecureFiles File System Store 8-3 Comparing SecureFiles LOBs to BasicFiles LOBs 8-4 Initializing SecureFiles Store File Systems 8-4 Drop SecureFiles Store File Systems 8-5 Working with DBFS Content API 8-5 9 DBFS Hierarchical Store Wallet Management 9-1 Managing Storage with DBMS_DBFS_HS 9-2 Constants for DBMS_DBFS_HS Package 9-2 Methods of DBMS_DBFS_HS Package 9-2 CREATESTORE() 9-3 DROPSTORE() 9-3 RECONFIGCACHE() 9-3 SETSTOREPROPERTY() 9-4 GETSTOREPROPERTY() 9-5 CREATEBUCKET() 9-5 STOREPUSH() 9-5 CLEANUPUNUSEDBACKUPFILES() 9-5 [...]... xxii Preface This guide describes database features that support application development using SecureFiles and Large Object (LOB) datatypes The information in this guide applies to all platforms ,and does not include system-specific information Audience Oracle Database SecureFiles and Large Objects Developer's Guide is intended for programmers who develop new applications that use LOBs, and those who have... and now want to take advantage of new features Efficient and secure storage of multimedia and unstructured data is increasingly important, and this guide is a key resource for this topic within the Oracle Application Developers documentation set Feature Coverage and Availability Oracle Database SecureFiles and Large Objects Developer's Guide contains information that describes the SecureFiles LOB and. .. video, and sound waveforms tends to be large in size A typical employee record may be a few hundred bytes, while even small amounts of multimedia data can be thousands of times larger SQL data types that are ideal for large amounts of unstructured binary data include the BLOB data type (Binary Large Object) and the BFILE data type (Binary File object) 1-2 Oracle Database SecureFiles and Large Objects. .. locators and how LOB values are stored that you should be aware of These details are covered in the context of the discussion where they apply throughout this guide See Also: ■ "LOB Locator and LOB Value" on page 2-2 ■ "LOB Locators and BFILE Locators" on page 2-3 ■ "LOB Storage Parameters" on page 11-4 1-4 Oracle Database SecureFiles and Large Objects Developer's Guide Object Data Types and LOBs Database. .. Chapter 4, "Using Oracle SecureFiles LOBs" OCI LOB prefetching improves performance See "Prefetching of LOB Data, Length, and Chunk Size" on page 13-9 xxviii 1 1 Introduction to Large Objects This chapter introduces Large Objects (LOBs) and discusses how LOB data types are used in application development This chapter contains these topics: ■ What Are Large Objects? ■ Why Use Large Objects? ■ Why Not Use... LOB Locators ■ Database Semantics for Internal and External LOBs ■ Large Object Data Types ■ Object Data Types and LOBs ■ Storing and Creating Other Data Types with LOBs What Are Large Objects? Large Objects (LOBs) are a set of data types that are designed to hold large amounts of data A LOB can hold up to a maximum size ranging from 8 terabytes to 128 terabytes depending on how your database is configured... data and unstructured data Large objects features allow you to store these kinds of data in the database and in operating system files that are accessed from the database With the growth of the internet and content-rich applications, it has become imperative that the database support a data type that: ■ Can store unstructured and semi-structured data in an efficient manner ■ Is optimized for large. .. ORDImage, and ORDVideo Oracle Multimedia uses the database infrastructure to define object types, methods, and LOBs necessary to represent these specialized types of data in the database See Also: ■ ■ Oracle Multimedia User's Guide for more information on using Oracle Multimedia Oracle Multimedia Reference for more information on using Oracle Multimedia data types 1-6 Oracle Database SecureFiles and Large Objects. .. ■ ■ ■ ■ ■ ■ Oracle Database 2 Day Developer's Guide is the entry point to the Oracle Database library for application developers It explains general concepts behind development with Oracle Database, introduces basic features of SQL and PL/SQL, and provides references to in-depth information elsewhere in the Oracle Database library Oracle Database Advanced Application Developer's Guide explains topics... Oracle Database version 7 and earlier, used the LONG or LONG RAW data type to store large amounts of unstructured data With the Oracle8i and later versions of the database, using LOB data types is recommended for storing large amounts of structured and semi-structured data LOB data types have several advantages over LONG and LONG RAW types including: ■ ■ ■ ■ LOB Capacity: LOBs can store much larger . Oracle® Database SecureFiles and Large Objects Developer's Guide 11g Release 2 (11.2) E18294-02 January 2013 Oracle Database SecureFiles and Large. xxv What's New in Oracle Database SecureFiles and Large Objects Developer's Guide? xxvii LOB Features Introduced in Oracle Database 11g Release 2 xxvii LOB