ETL Techniques • Chapter 8 317 In addition to the required arguments, there are a number of options that can be used to control BCP’s behavior. You can get a complete list from the SQL Server 2008 documentation. Table 8.2 lists some of the options we will discuss in this chapter. Pay attention to the uppercase/lowercase nature of the options. They are case sensitive. Table 8.1 Continued. Required BCP Syntax Arguments Syntax Element Description format Use this option to generate a format file. When format is used, no import or export is performed. Use this option with the –f and or –x parameters to specify the path of the generated format file. We will discuss format files later. datafile The path of the data file that you will be importing from or exporting to. Optional arguments There are number of additional arguments that can be used with the BCP utility. They are optional overall, but certain options may be required depending on what kind of operation you are performing. For example, if you use the format argument to generate a format file, you must also use the –f argument to specify the path of the file. See Table 8.2 for a list of some additional common bcp arguments. Te s T Da y Ti p The list of options available for BCP can be overwhelming at first. It is worth your time to be familiar with them, but don’t stress on them. You won’t likely be tested on specific command-line switches. 318 Chapter 8 • ETL Techniques Option Description -S instancename Specifies which instance of SQL Server BCP should connect to. If the –S option is not specified the local default instance is assumed. -T Indicates that you wish BCP to connect using a trusted connection (aka Windows authentication). -U loginname Used to specify a SQL authentication login name when not using –T. -P password Used to specify the password associated with the –U login name. -N Use a Unicode native data file type. -n Use a native data file type (use only when you don’t have Unicode data). -c Use a character data file type. -w Use a Unicode data file type. -t fieldterminator Specifies the field terminator that should be used in delimited data files. If special characters are needed, or a space is part of the delimiter, surround the delimiter in double quotes (“). If no field terminator is specified, the tab character is used. -r rowterminator Specifies the row terminator that should be used in delimited data files. As with the field terminator, if special characters are needed or a space is used in the delimiter, you can enclose the terminator in double quotes (“). If no row terminator is specified, a line feed (or new line) character is used (“\n”). -F firstrownum Specifies the row number of the first row of data. This can be used to skip over header rows. It can also be used with the –L argument to specify a range of rows to process if the same data file is being processed on multiple workstations. If you are using a source file that has the column names in the first row, you need to make sure to use the –F parameter to tell BCP to start processing the file at the second row (-F 2). Otherwise, it will try to read the column names as field values. Table 8.2 Optional BCP Arguments Continued ETLTechniques•Chapter8 319 Option Description -L lastrownum Specifies the row number of the last row of data to process. This can be used if the data file has a footer that you don’t want to process. It can also be used with the –F argument to specify a range of rows to process if the same data file is being processed on multiple workstations. -b batchsize Specifies the number of rows to include in a single batch. Each batch then ends up as a separate commit operation. The entire file is treated as a single batch by default. By using the –b option you can break the load up in multiple batches to help manage the load on the transaction log and possibly restart a load at a certain point if it fails partway through. -f formatfile Specifies the path to the format file to use. -x Used to generate an XML-formatted format file. Use this in conjunction with the “format” operation and the –f option to specify the path of the file. -h Allows you to specify a number of “hints” that mainly affect the performance of the BCP option. We will discuss some of them when we talk about performance. Others Again, there are other options available. You should review the information on BCP in the SQL Server 2008 documentation. Table 8.2 Continued. Optional BCP Arguments So taking the information listed in Tables 8.1 and 8.2, let’s try a quick sample. You will export data from the AdventureWorks2008.Person.Person table to a file named person.txt. You will ask BCP to store the data in SQL Server Unicode native format (-N), and you will connect using a trusted connection (-T). Since no instance is specified (no –S parameter) the default local instance is assumed. So this is what it looks like: bcp AdventureWorks2008.Person.Person out person.txt –N –T 320 Chapter8•ETLTechniques The output from the preceding statement looks something like this: Starting copy… 1000 rows successfully bulk-copied to host-file. Total received: 1000 1000 rows successfully bulk-copied to host-file. Total received: 2000 1000 rows successfully bulk-copied to host-file. Total received: 3000 … 1000 rows successfully bulk-copied to host-file. Total received: 19000 19972 rows copied. Network packet size (bytes) : 4096 Clock Time (ms.) Total : 4807 Average: (4154.77 rows per sec.) If you were to open the resulting person.txt file in notepad, you would likely be able to see some information, but the majority of it would be illegible. The reason for this limited legibility of this information is because you asked BCP to use a native data file type. We’ll talk more about data file types and the formatting of the data later. For now, let’s discuss importing the sample data. To test this data file, create a sample table that you can load the data back into. After opening a query window connected to your SQL Server instance inside SQL Server Management Studio (SSMS), you could run the following statement to create a table with the same structure as the AdventureWorks2008.Person.Person table, but without any data: USE AdventureWorks2008; SELECT TOP 0 * INTO Person.PersonCopy FROM Person.Person; You will be testing with this table in various situations. To make sure the table is ready for a load, you could truncate any data in it by running the following Transact-SQL (T-SQL) statement inside SQL Server Management Studio: TRUNCATE TABLE AdventureWorks2008.Person.PersonCopy; So now that you have a place to load the data back into, try loading the data file you just created back into SQL Server. You should notice that the syntax looks very similar to the command line you used for the export. The only differences is the destination table name, AdventureWorks2008.Person.PersonCopy, and “in” instead of “out,” indicating that you will be importing from the file rather than exporting to it. bcp AdventureWorks2008.Person.PersonCopy in person.txt –N –T ETLTechniques•Chapter8 321 If you were to run the preceding statement, the output would look similar to the following: Starting copy… 1000 rows sent to SQL Server. Total sent: 1000 1000 rows sent to SQL Server. Total sent: 2000 1000 rows sent to SQL Server. Total sent: 3000 … 1000 rows sent to SQL Server. Total sent: 19000 19972 rows copied. Network packet size (bytes) : 4096 Clock Time (ms.) Total : 15252 Average: (1309.47 rows per sec.) If you now go back to your query window in SQL Server Management Studio and run the following query, you will see that the AdventureWorks2008.Person. PersonCopy table now has data in it. SELECT * FROM AdventureWorks2008.Person.PersonCopy; In the previous two examples, you exported data to a file and imported it back in using BCP. By using the Unicode native file type for the data (-N), you increased the likelihood that BCP would be able to parse the data files correctly and that the data would be compatible with SQL Server. Using the Unicode native data file type is great when you are moving data from SQL Server to SQL Server. Other data file types are also available, though. We’ll talk about those next. Using BCP Data File Types Often, you need to import data that you get from sources other than SQL Server. As we discussed in the earlier example, native data files types are great as an intermediate format moving data between to SQL Server databases. When you receive data files from business partners or internal systems other than SQL Server, however, the files will more likely be in a nonnative character or Unicode format. Native data files are best used for moving data between two SQL Server instances. However, two “flavors” of native files are available: regular native files (-n) and Unicode native files (-N). The Unicode native files provide the highest likelihood that the characters in the source database get transferred properly to the target system because the native mode can’t represent all the characters that are possible in nchar and nvarchar fields. BCP also can work with nonnative character and Unicode data files. When the native file type is used, BCP knows how to parse the files correctly, so you don’t . parse the data files correctly and that the data would be compatible with SQL Server. Using the Unicode native data file type is great when you are moving data from SQL Server to SQL Server. Other. loading the data file you just created back into SQL Server. You should notice that the syntax looks very similar to the command line you used for the export. The only differences is the destination. connected to your SQL Server instance inside SQL Server Management Studio (SSMS), you could run the following statement to create a table with the same structure as the AdventureWorks2008.Person.Person