Streaming Data Types

setBFILE(int paramIndex, BFILE file) setBfile(int paramIndex, BFILE file) setBLOB(int paramIndex, BLOB lob) setCHAR(int paramIndex, CHAR ch) setCLOB(int paramIndex, CLOB lob) setCursor(int paramIndex, ResultSet rs) setCustomDatum(int paramIndex, CustomDatum x) setDATE(int paramIndex, DATE date) setExecuteBatch(int batchValue) setFixedCHAR(int paramIndex, String x) setNUMBER(int paramIndex, NUMBER num) setOracleObject(int paramIndex, Datum x) setRAW(int paramIndex, RAW raw) setREF(int paramIndex, REF ref) setRefType(int paramIndex, REF ref) setROWID(int paramIndex, ROWID rowid) setSTRUCT(int paramIndex, STRUCT struct) Now that you have an understanding of how to use a PreparedStatement, we can move on to the next chapter on streaming data types. Chapter 12. Streaming Data Types Most of the time, the 4,000 bytes of storage available with the VARCHAR2 data type under Oracle8 and higher is sufficient for application needs. But occasionally, applications require larger text fields or need to store complex binary data types such as word processing files and photo images in the database. Oracle8's solution to the problem of storing large amounts of data is the binary file (BFILE), binary large object (BLOB), and character large object (CLOB) data types. These large object (LOB) data types ease the storage restriction to 4 GB. The difference between a CLOB and a BLOB is that a CLOB is subject to character set translation as part of Oracle's National Language Support (NLS), whereas a BLOB's data is taken verbatim. Oracle7's solution to the problem of storing large amounts of data is the LONG and LONG RAW data types. A LONG column can hold up to 2 GB of character data, while a LONG RAW can hold up to 2 GB of binary data. However, the truth of the matter is that LONGs exist in Oracle8 and higher only for the purpose of backward compatibility. I recommend you use the BLOB and CLOB data types for all new development when you need to store more than 4,000 bytes of data for a column. Collectively, LOBs are normally transferred between your application and the database using streams instead of the get/set accessor methods used for VARCHAR2 and other types. Consequently, LOBs are also referred to as streaming data types. Throughout this chapter I will refer to large objects, that is both BLOBs and CLOBs, as LOBs. When I use the term LOB, I'm referring to a concept that applies to both types. There are differences in the way that the two client-side drivers, the OCI driver and the Thin driver, actually manipulate LOB data. The OCI driver uses native code in the driver, while the Thin driver uses Oracle's built-in DBMS_LOB package. From your perspective, this difference is apparent only when an attempt is made to use the PreparedStatement interface's methods to write LOB data. A PreparedStatement can write LOB data only when the OCI driver is used. I'll mention this again when it's applicable. You may be wondering why there is such a thing as a streaming data type. Why the need for streams? The answer is that when writing large objects, streams are more efficient than the setXXX( ) methods. There's quite a bit of hearsay about the efficiency of using LOBs. It's common to hear someone say: "Using LOBs is really slow!" The truth of the matter is that for all practical purposes, byte-for-byte, using a large object data type is no slower than writing data to a VARCHAR2. What some folks forget is that writing 1 MB of data takes longer than writing 2 KB. If you need to store large objects in a database, then LOBs are the data types of choice. In this chapter, we'll cover the use of both the streaming methods and the get/set accessor methods for inserting, updating, and selecting the large object, streaming data types. We'll start with a detailed explanation of Oracle8's BLOB data type and then move on to cover the differences involved when using a CLOB. Next, we'll cover the Oracle proprietary type BFILE. Finally, we'll briefly discuss the use of LONG and LONG RAW. Let's begin our journey with a look at binary large objects. 12.1 BLOBs BLOBs can be used to store any type of information you desire, as long as the data is less than 4 GB. Unlike the other data types we have covered so far in this book, BLOB data is accessed using a locator stored in a table. This locator points to the actual data. Since the locator is an internal database pointer, you can't create a locator in your application. Instead, you must create a BLOB locator by either inserting a new row into your database or updating an existing row. Once you create a locator, you then retrieve it using SELECT FOR UPDATE to establish a lock on it. When a BLOB locator is retrieved from the database, an instance of the java.sql.Blob, or oracle.sql.BLOB, class is used to hold the locator in your Java program. These classes hold the BLOB locator, not the actual data. To get the actual data, you must use one of the Blob, or BLOB, methods to read the data from the database as a stream or to get the data into a byte array. While the Blob interface supports getting BLOB data from the database, it does not define any methods for inserting or updating that data. Insert and update functionality is JDBC driver- specific. I hope that this inconsistent behavior in the interface for LOBs -- of using methods from the locator to get data but not having any defined for storing it -- will be corrected in the next version of JDBC. For now, you can use Oracle's proprietary methods to write the contents as a stream, or you can use a set accessor method to set the data as a byte array. The JDBC 2.0 specification states that the PreparedStatement object's setObject( ) and setBinaryStream( ) methods may be used to set a BLOB's value, thus bypassing the locator. However, this functionality is currently supported only by Version 8.1.6 of the OCI driver to an 8.1.6 database. In this chapter, I'll first show you how to use oracle.sql.BLOB to manipulate BLOBs. This approach works for either driver. Then I'll show you how to use java.sql.PreparedStatement, which is supported only by the OCI driver. Let's take a moment to clarify some nomenclature. Since I'm an object-oriented programmer, I believe that using the same name for something in different contexts is a great idea. It helps to autodocument the subject. At the same time, however, it can cause some confusion, as it does when discussing LOBs. For example, in this section, the word "blob" has three definitions: BLOB Refers to the SQL data type for a binary large object BLOB Refers to the oracle.sql.BLOB class used to hold a BLOB's locator in your Java program Blob Refers to the java.sql.Blob interface, which is implemented by the oracle.sql.BLOB class and is used to hold a BLOB's locator in your Java program Please keep these distinctions in mind as you read through this section, or you may become hopelessly confused. Before we get into an explanation of how to manipulate BLOBs, we first need a table with a BLOB column in it for our examples. So let's proceed by creating a LOB table. 12.1.1 An Example LOB Table Before you can insert a BLOB, you must have a table containing a BLOB column. For our examples, we'll expand our HR database with a person_information table. In this table, we'll use person_id as a primary key and as a foreign key that references the person table, a biography column defined as a CLOB to hold a person's biographical information, and a photo column defined as a BLOB to hold a picture of the person in question. The following is the DDL for our person_information table: drop table PERSON_INFORMATION / create table PERSON_INFORMATION ( person_id number not null, biography clob, photo blob ) tablespace USERS pctfree 20 storage (initial 100 K next 100 K pctincrease 0) / alter table PERSON_INFORMATION add constraint PERSON_INFORMATION_PK primary key ( person_id ) using index tablespace USERS pctfree 20 storage (initial 10 K next 10 K pctincrease 0) / Now that we have a table for our examples, we can continue by looking at how you insert a BLOB. 12.1.2 Inserting a BLOB Using oracle.sql.BLOB In earlier chapters, when we discussed how to insert, update, delete, and select a DATE, NUMBER, or VARCHAR2 data type, it was simply a matter of providing a character representation of the data for a Statement object, or using a setXXX( ) method with a PreparedStatement object, and then executing the SQL statement. However, with LOBs, this one-step process does not work. Instead, when working with a LOB you need a three-step process. That's because with a LOB, a locator object, not the actual data, is stored in a table's column. You need to retrieve the locator to insert or update the actual LOB data. The three-step process is: 1. Create a locator by inserting a row into a table. 2. Retrieve the locator from the inserted row using a SELECT statement with the FOR UPDATE clause to manually lock the row. 3. Use the locator to insert the BLOB data into the database. 12.1.2.1 Creating a locator A locator is an object that points to the actual location of the BLOB data in the database. You need a locator to manipulate the BLOB data. Because it points to a location in the database address space, only the database can create a new locator. Therefore, your Java program cannot create a locator. I know that last sentence is redundant, but it's very important that you realize up front that creating a locator is solely the job of the database. To create a new locator for a BLOB, use the empty_blob( ) database function to generate the BLOB column's value in an INSERT statement. For example, to insert a new row into the person_information table and at the same time create a new BLOB locator, use the empty_blob( ) database function: insert into person_information ( person_id, photo ) values ( 1, empty_blob( ) ) In this example, the number 1 is passed as the person_id value, and the result of the empty_blob( ) function is passed as the photo value. When this statement is executed, the database creates a new locator for the person_information table's photo column and stores that locator in the row being inserted. Initially, the locator points to a location that contains no data, for you have not yet used the locator to store any data. If you don't use the empty_blob( ) database function to generate a locator, you'll get a null reference error when you later attempt to retrieve the locator to insert your BLOB data. Now that you know how to create a locator, let's look at how to retrieve that locator from the database. 12.1.2.2 Retrieving a locator To retrieve a locator, you must execute a SELECT statement for the BLOB column using either a Statement or PreparedStatement object. You must include the FOR UPDATE clause, or the FOR UPDATE NOWAIT clause, in the SELECT statement to lock the locator; the locator must be manually locked for you to use it to insert or update BLOB data. For example, to retrieve and lock the locator inserted earlier, use the following SQL statement: select photo from person_information where person_id = 1 for update nowait In your Java program, you get the locator value from a ResultSet object using the getBlob( ) accessor method. Alternatively, you can call the OracleResultSet object's getBLOB( ) accessor method. The locator is then assigned to an oracle.sql.BLOB object in your program. If you use the ResultSet.getBlob( ) method, you'll have to cast the returned java.sql.Blob object to an oracle.sql.BLOB object. For example, you'll use code similar to the following: ResultSet rslt = stmt.executeQuery( "select photo " + "from person_information " + "where person_id = 1 " + "for update nowait"); rslt.next( ); oracle.sql.BLOB photo = (oracle.sql.BLOB)rslt.getBlob(1); Now that you know how to retrieve a locator, let's see how you can use it to actually insert some BLOB data. 12.1.2.3 Using the locator to insert BLOB data Once you've retrieved a valid BLOB locator from the database, you can use it to insert binary data into the database. First, you need to get a binary output stream from the BLOB object using the getBinaryOutputStream( ) method, which has the following signature: OutputStream getBinaryOutputStream( ) Next, you need to get the optimal buffer size when writing the binary data to the database by calling the BLOB object's getBufferSize( ) method, which has the following signature: int getBufferSize( ) You can use the optimal buffer size to allocate a byte array to act as a buffer when you write binary data using the BLOB object's OutputStream object. At this point, all that's left to do is use the output stream's write( ) method to write the binary data to the database. The OutputStream object's write( ) method has the following signature: write(byte[] buffer, int offset, int length) which breaks down as: buffer A byte array containing the BLOB data you desire to write to the database BLOB column offset The offset from the beginning of the array to the point from which you wish to begin writing data length The number of bytes to write to the BLOB column After you're done writing the data, you'll need to call the OutputStream object's close( ) method, or your written data will be lost. For example, given that you have someone's picture in a .gif file, and you want to load it into the database using the locator photo that we created earlier, you'll use code such as the following: try { // Open a gif file for reading File binaryFile = new File("picture.gif"); in = new FileInputStream(binaryFile); // Get the BLOB's output stream out = photo.getBinaryOutputStream( ); // Get the optimal buffer size from the BLOB int optimalSize = photo.getBufferSize( ); // Allocate an optimal buffer byte[] buffer = new byte[optimalSize]; // Read the file input stream, in, and // write it to the the output stream, out // When length = -1, there's no more to read int length = 0; while ((length = fin.read(buffer)) != -1) { out.write(buffer, 0, length); } // You need to close the output stream before // you commit, or the changes are lost! out.close( ); out = null; fin.close( ); fin = null; conn.commit( ); } 12.1.2.4 An example that inserts a BLOB using an output stream Example 12-1 shows a complete, fully functional program that uses the BLOB object's getBinaryOutputStream( ) method to insert a .gif file into the person_information table using an output stream. Example 12-1. Using getBinaryOutputStream( ) to insert a BLOB import java.io.*; import java.sql.*; import java.text.*; // Add these imports for access to the required Oracle classes import oracle.jdbc.driver.*; import oracle.sql.BLOB; public class TestBLOBGetBinaryOutputStream { Connection conn; public TestBLOBGetBinaryOutputStream( ) { try { DriverManager.registerDriver(new oracle.jdbc.driver.OracleDriver( )); conn = DriverManager.getConnection( "jdbc:oracle:thin:@dssw2k01:1521:orcl", "scott", "tiger"); } catch (SQLException e) { System.err.println(e.getMessage( )); e.printStackTrace( ); } } public static void main(String[] args) throws Exception, IOException { new TestBLOBGetBinaryOutputStream().process( ); } public void process( ) throws IOException, SQLException { int rows = 0; FileInputStream fin = null; OutputStream out = null; ResultSet rslt = null; Statement stmt = null; BLOB photo = null; // NOTE: oracle.sql.BLOB!!! long person_id = 0; try { conn.setAutoCommit(false); // Get Tim's person_id stmt = conn.createStatement( ); rslt = stmt.executeQuery( "select person_id " + "from person " + "where last_name = 'O''Reilly' " + "and first_name = 'Tim'"); while (rslt.next( )) { rows++; person_id = rslt.getLong(1); } if (rows > 1) { System.err.println("Too many rows!"); System.exit(1); } else if (rows == 0) { System.err.println("Not found!"); System.exit(1); } rslt.close( ); rslt = null; // Check to see if the row already exists rows = 0; rslt = stmt.executeQuery( "select photo " + "from person_information " + "where person_id = " + Long.toString( person_id ) + " " + "for update nowait"); while (rslt.next( )) { rows++; photo = (BLOB)rslt.getBlob(1); } rslt.close( ); rslt = null; // If it doesn't exist, then insert // a row in the information table // This creates the LOB locators if (rows == 0) { rows = stmt.executeUpdate( "insert into person_information " + "( person_id, biography, photo ) " + "values " + "( " + Long.toString(person_id) + ", empty_clob(), empty_blob( ))"); System.out.println(rows + " rows inserted"); // Retrieve the locator rows = 0; rslt = stmt.executeQuery( "select photo " + "from person_information " + "where person_id = " + Long.toString( person_id ) + " " + "for update nowait"); rslt.next( ); photo = ((OracleResultSet)rslt).getBLOB(1); rslt.close( ); rslt = null; } stmt.close( ); stmt = null; // Now that we have the locator, // lets store the photo File binaryFile = new File("tim.gif"); fin = new FileInputStream(binaryFile); out = photo.getBinaryOutputStream( ); // Get the optimal buffer size from the BLOB byte[] buffer = new byte[photo.getBufferSize( )]; int length = 0; while ((length = fin.read(buffer)) != -1) { out.write(buffer, 0, length); } // You need to close the output stream before // you commit, or the changes are lost! out.close( ); out = null; fin.close( ); fin = null; conn.commit( ); } catch (SQLException e) { System.err.println("SQL Error: " + e.getMessage( )); } catch (IOException e) { System.err.println("IO Error: " + e.getMessage( )); } finally { if (rslt != null) try { rslt.close( ); } catch (SQLException ignore) { } if (stmt != null) try { stmt.close( ); } catch (SQLException ignore) { } if (out != null) try { out.close( ); } catch (IOException ignore) { } if (fin != null) try { fin.close( ); } catch (IOException ignore) { } } } protected void finalize( ) throws Throwable { if (conn != null) try { conn.close( ); } catch (SQLException ignore) { } super.finalize( ); } } Since Example 12-1, TestBLOBGetBinaryOutputStream, utilizes Oracle classes, we've added two additional import statements: import oracle.jdbc.driver.* and import oracle.sql.BLOB. The program starts in its main( ) method by instantiating itself and then executes its process( ) method. The process( ) method begins by allocating the following variables: rows An int value to hold the number of rows retrieved by a SELECT statement fin A FileInputStream used to read a file from the filesystem out An OutputStream object used to write the BLOB data into the database rslt A ResultSet object used to retrieve a BLOB locator from the database stmt A Statement object used to retrieve a BLOB locator from the database and to execute an INSERT statement to create that locator in the first place photo A BLOB object to hold a valid locator from the database person_id A long to hold the primary key for a person row from the person table Next, the program enters a try block where it starts by turning off auto-commit. It then executes a SELECT statement against the person table to get the primary key value for Tim O'Reilly. If the program finds Tim O'Reilly in the person table, it continues by executing a SELECT statement that retrieves and locks the photo column locator for Tim. In the while loop for this SELECT statement, I use the java.sql.ResultSet object's getBlob( ) method to retrieve the locator from the result set. Since this method returns a java.sql.Blob, I cast it to an oracle.sql.BLOB in order to assign it to the photo variable. If the program doesn't find an existing entry for Tim in the person_information table, it proceeds by inserting a new row and uses the empty_blob( ) database function in the INSERT statement to create a new locator. The program then retrieves that newly inserted locator with a lock. In the while loop for this SELECT statement, I take a different approach to the casting problem. Instead of casting the object returned from getBlob( ) to an oracle.sql.BLOB, I cast the ResultSet object, rslt, to an OracleResultSet object and call the OracleResultSet object's getBLOB( ) method. At this point in the program, photo is a valid locator that can be used to insert BLOB data into the database. The program proceeds by creating a File object for a file named tim.gif. It uses the File object as an argument to the constructor of a FileInputStream object. This opens the tim.gif file in the local filesystem for reading. Next, the program creates a byte array to act as a buffer, passing to its constructor the optimal buffer size by calling the getBufferSize( ) method of the BLOB object, photo. Now the program has an input stream and an output stream. It enters a while loop where the contents of the input stream are read and then written to the database. The while loop contains the following elements: fin.read(buffer) Reads as many bytes of data from the input stream as will fit into the byte array named buffer. length = fin.read(buffer) Stores the number of bytes actually read into the variable length. The read( ) method of the input stream returns the number of bytes that are actually read as an int. (length = fin.read(buffer)) Evaluates to the actual number of bytes read, so that value can be used in the while loop's conditional statement. while ((length = fin.read(buffer)) != -1) The conditional phrase for the while loop. When the end of the file is reached for the input stream, the read( ) method returns a value of -1, which ultimately ends the while loop. out.write(buffer, 0, length) Calls the OutputStream object's write( ) method, passing it the byte array (buffer), the starting position in the array from which to write data (always 0), and the number of bytes to write. This one statement is the body of the while loop. The write( ) method reads data from the buffer and writes it to the database. After writing the data to the database using an output stream, effectively inserting the data, the program continues by closing the output stream with a call to its close( ) method. This is a critical step. If you don't close the stream, the data is lost. Also, the output stream must be closed before you commit or, again, the data will be lost. The program finishes up by closing the input stream and committing the data. Example 12-1 has highlighted a very important point about LOBs. LOB data is streamed to the database in chunks rather than sent all at once. This is done for two reasons. First, the amount of memory consumed by a program is conserved. Without streaming, if you had a .jpeg file that was 1 GB, you'd need to consume at least 1 GB of memory to load the .jpeg file's data into memory. With streaming, you can read reasonably small chunks of data into memory. Second, it prevents your data transmission from monopolizing the network's available bandwidth. If you sent 1 GB of data in one transmission to the database, everyone else's transmission would have to wait until yours was finished. This might cause network users to think there was something wrong with the network. By streaming the data in chunks, you release access of the network to other users between each chunk. 12.1.2.5 A nonstreaming alternative for small BLOBs It's an oxymoron: small binary large objects. But there is nothing to prevent you from using BLOBs to store small amounts of binary data in the database. If your binary data is always under 4,000 bytes, then you might consider using the oracle.sql.BLOB object's putBytes( ) method to send the data to the database. The putBytes( ) method works in a manner similar to the setXXX( ) accessor methods and has the following signature: int putBytes(long position, byte[] bytes) which breaks down as: position The starting position, in bytes, within the BLOB in the database. Data is written into the BLOB starting from this point. bytes A byte array that contains the data to write to the BLOB in the database. putBytes Returns an int value with the number of bytes actually written to the BLOB. The putBytes( ) method is actually one of several methods that allow you to directly modify a BLOB in the database. In this chapter, I show you how to use it to insert a BLOB value as one chunk of data. 12.1.2.6 An example that inserts a BLOB using the putBytes( ) method [...]... to read CLOB data from a database: InputStream getAsciiStream( ) This method returns an InputStream object that can read ASCII data from a CLOB in the database using streams Reader getCharacterStream( ) This method returns a Reader object that can read Unicode data from a CLOB in the database using streams String getSubString(long position, int length) This method can read a substring of data from a... you have a program that can read and then immediately write streamed data, or that can work with chunks of binary data, not requiring all the data to be in memory at once, then you should use the getBinaryStream( ) method to retrieve BLOB data 12.1.6.1 Using getBinaryStream( ) to retrieve BLOB data To get a BLOB locator from the database, you must execute a SQL SELECT statement that includes the BLOB... character set of ASCII to and from the database's character set If you use the character methods getCharacterStream( ), setCharacterStream( ), getCharacterOutputStream( ), and putChars( ) the driver translates the client character set of Unicode to the database's character set The database will handle the data as ASCII if the database character set is ASCII; otherwise, the database will perform NLS character... to write CLOB data as ASCII characters to the database using streams Writer getCharacterOutputStream( ) Returns an OutputStream object that can be used to write CLOB data as Unicode characters to the database using streams int putChars(long position, char[] buffer) Writes a char array (buffer) to a CLOB in the database The writing begins at the specified position within the CLOB in the database This... putBytes( ) method to write the data to the database The first argument to the putBytes( ) method is 1 This is the starting position at which to begin writing data to the database BLOB Notice that the starting position is a 1 and not a 0 While arrays start at 0 in Java, they start at 1 in the database The second argument to the method is the byte array, buffer, which contains the data to be written Did you... Therefore, selecting a CLOB's value from a database is a two-step process: 1 Select a CLOB locator from a table 2 Use the java.sql.Clob object's getCharacterStream( ) or getAsciiStream( ) method to access the data Selecting a locator from the database is accomplished by selecting a CLOB column in the database The CLOB column in the database contains a locator, not the actual data In your program, create a java.sql.Clob... locator via an oracle.sql.CLOB object to write the CLOB data Just as you use the empty_blob( ) database function to create a new locator for a BLOB, you use the empty_clob( ) database function to create a new locator for a CLOB Once you have a valid locator, you can use one of the following methods from the oracle.sql.CLOB class for writing CLOB data to a database: OutputStream getAsciiOutputStream( ) Returns... setBytes( ) method, I recommend you do so only for small binary objects The streaming methods are much more efficient Now let's take a look at the second, nonstreaming alternative for inserting BLOB data 12.1.3.3 Using setObject( ) to insert a BLOB You can also use the setObject( ) method to insert binary data into a BLOB column in the database We covered the setObject( ) method in Chapter 11, but just in... 8.1.6 OCI driver connected to an 8.1.6 database Let's take a look at how each of these three methods is used We'll begin with the streaming method, setBinaryStream( ) 12.1.3.1 Using setBinaryStream( ) to insert a BLOB Using the setBinaryStream( ) method makes inserting BLOB data into the database a onestep process Instead of inserting a row using the empty_blob( ) database function to create a locator... CLOB in the database It returns a String object containing the CLOB data If you set position to 1 and length to the length of the CLOB in the database, you can return the entire CLOB as a String The ResultSet interface provides a getClob( ) method, but it's used to get the locator for a CLOB from a result set, not the CLOB's data You must, in turn, use the locator to retrieve the actual data Therefore, . chapter on streaming data types. Chapter 12. Streaming Data Types Most of the time, the 4,000 bytes of storage available with the VARCHAR2 data type under. actual data. To get the actual data, you must use one of the Blob, or BLOB, methods to read the data from the database as a stream or to get the data into