138 Hands-On Microsoft SQL Server 2008 Integration Services Workflow Tasks Workflow tasks communicate with other processes to execute packages, run programs or batch files, send and receive messages between packages, send e-mail messages, read Windows Management Instrumentation (WMI) data, and watch for WMI events. Control Flow Task Description Run child packages—i.e., develop packages as modules of work items and choose a combination of such work item modules using this task to run as under a master package or break down a complex package into separate units of work and run them together. Run a business application or a batch file to perform specific functions that can’t be done easily in SSIS, whose output can then be included in the SSIS package. Queue your messages if the destination is unavailable or busy and deliver such messages later. This task uses Microsoft Message Queuing (MSMQ) to send and receive messages. Send messages from your packages, for example, on the basis of success or failure of the package or on the basis of an event raised during execution of the package. Run WQL (WMI Query Language) queries to get information from WMI about a computer system, enabling your package to decide what to do with the other tasks in the package. Watch for WMI events by running WQL queries to specify the events of interest. SQL Server Tasks These tasks help in accessing and performing functions such as Copy, Insert, Delete, and Modify on SQL Server objects and data. Control Flow Task Description Copy large amounts of data into SQL Server. This is the fastest method for importing data into SQL Server if you don’t have to perform transformations while importing. You will use this task in a Hands-On exercise later in the chapter. Run SQL statements or stored procedures to perform operations such as create, alter, truncate, or drop tables; save a result set of a query or stored procedure into a variable; and so on. You have already used this task in earlier Hands-On exercises. Chapter 5: Integration Services Control Flow Tasks 139 Scripting Tasks These tasks help in extending the package functionality by allowing you to write your own code. Control Flow Task Description Write code to perform functions that otherwise cannot be performed using built-in tasks and transformations. Uses Visual Studio Tools for Applications (VSTA) Environment as its engine, for writing and running scripts with Microsoft Visual Basic 2008 or Microsoft Visual C# 2008. Scripting is covered in detail in Chapter 11. Analysis Services Tasks These tasks allow you to work with Analysis Services objects. These tasks are covered in detail in Chapter 12 while discussing data warehousing practices. Control Flow Task Description Process Analysis Services objects such as cubes, dimensions, and mining models. You can process multiple objects in a batch and choose a processing sequence or can process them in parallel. Create, alter, or drop multidimensional objects (such as cubes and dimensions) or mining models. You create data definition language (DDL) statements in Analysis Services Scripting Language (ASSL) framed in an XML for Analysis Services (XMLA) command. Run prediction queries in Analysis Services using defined data mining models. The prediction query is created using Data Mining Extensions (DMX) language, which provides support for using mining models. Transfer Tasks SQL Server Integration Services enables you to transfer databases, error messages, SQL Server Agent jobs, transfer logins, database objects, and master stored procedures using the built-in transfer tasks. Control Flow Task Description Move or copy databases from one instance to another instance of SQL Server. You can transfer database in an offline (Detach/Attach method) or online mode. Transfer SQL Server error messages from a source SQL Server to a destination SQL Server. 140 Hands-On Microsoft SQL Server 2008 Integration Services Control Flow Task Description Transfer SQL Server Agent jobs from one instance of SQL Server to the other instance. Transfer logins from a source SQL Server to a destination SQL Server. You can choose to transfer all logins on the server, transfer all logins for the selected database, or transfer just the selected logins. Copy database objects between databases on a source SQL Server and a destination SQL Server. You can choose to transfer indexes, primary keys, secondary keys, triggers, users, logins and roles, and so on. Transfer the stored procedures saved to the master database in a source SQL Server to a destination SQL Server. Maintenance Tasks SSIS has built-in tasks for administrative functions for databases, such as backing up and shrinking SQL Server databases, rebuilding and reorganizing indexes, and running SQL Server Agent jobs. Though these tasks are intended for maintaining SQL Server 2000 and SQL Server 2005 databases, they can also be used within SSIS packages. Control Flow Task Description You can perform different types of backups on one or more databases with this task. Check the allocation and structural integrity of all the database objects or indexes in the specified database. This task runs the DBCC CHECKDB statement. Runs the jobs already created within the SQL Server Agent. Allows you to reuse the jobs already created in SQL Server Agent and provides a facility to perform administration from within a SSIS package. Run T-SQL statements from within a SSIS package. This task is similar to the Execute SQL task, except this task supports only Transact-SQL version of SQL. Deletes historical data for the specified time period from MSDB tables related to backup and restore activities, SQL Server Agent jobs, and database maintenance plans. Deletes backup files or maintenance plan text reports based on the specified time period. Sends the notifications messages to SQL Server Agent operators via e-mail, pager, or Net Send communication channels. Chapter 5: Integration Services Control Flow Tasks 141 Control Flow Task Description Rebuilds indexes in SQL Server database tables and views. Reorganize indexes in SQL Server database tables and views with this task. Reduce the size of SQL Server database data file and log file with Shrink Database task. This task runs a DBCC SHRINKDATABASE command. Update information about the distribution of key values for one or more statistics groups in the specified table or indexed view with this task. Backward Compatibility Tasks Data Transformations Services (DTS) has been deprecated though backward compatibility support has still been provided. The following tasks and DTS 2000 will be removed in future versions of SSIS, so you should not be developing any new functionality or software code with these tasks. They have been provided purely to support the packages that are still to be migrated to Integration Services. Control Flow Task Description Run existing DTS 2000 ActiveX code in SSIS packages. This task has been marked deprecated in the current SQL Server version; upgrade your scripts to use more advanced features provided by the Script task. Refer to Chapter 14 for more details on migration options. Run packages that were developed using the Data Transformation Services of SQL Server 2000 and include legacy packages in your Integration Services package. The task is covered in detail in Chapter 14. Custom Tasks The object model of Integration Services provides facilities you can use to extend and program any component or complete package by writing custom code. Whether you want to extend the functionality of a component that will be used only in a package or you want to reuse the developed functionality in various packages throughout your enterprise, you will use different techniques to extend and program Integration Services. The Script task provided in the control flow and the script component provided in the data flow environments allow you to create and program any functionality that is not available in the preconfigured tasks and components of Integration Services using a .NET-compliant programming language such as Visual 142 Hands-On Microsoft SQL Server 2008 Integration Services Basic or C#. However, these components are difficult to reuse in multiple packages. Alternatively, you can create your own custom tasks using any .NET-compliant programming language such as Visual Basic or C#. After writing code for the custom task, you can create and register a user interface for the task in the SSIS Designer and reuse the developed and registered custom task or component in your packages as you would use any other built-in component. The development of custom tasks is detailed in Chapter 11. Control Flow Tasks in Detail In the preceding section, you’ve read brief description of each task; now you’ll work through the Hands-On exercise or read about the options available through the task’s GUI. While working through the details of each task and Hands-On exercise, you will not follow the categorization flow. This is to make it easier to learn and work with the tasks for business scenarios, followed by intuitive and step-by-step methods. Your first project will require that you download, expand, and archive these zipped files, and then import them to an SQL Server table. This is a simple but generic scenario we all encounter in our day-to-day job functions. So, let’s start our journey with the FTP task, which will help us in downloading files from a remote FTP server. FTP Task Running an FTP task makes your computer an FTP client and allows you to connect to an FTP server for transferring files. You can drag and drop this task onto the Control Flow Designer surface and double-click to open the FTP Task Editor to configure it. You can select to perform any of the following tasks from the options listed in the Operation field in the File Transfer page: Send Files c Transfers files from the local computer to the remote FTP server. Receive Files c Gets files from the remote FTP server. Create Local Directory c Creates a folder on the local computer. Create Remote Directory c Creates a folder on the FTP server. Remove Local Directory c Deletes the specified folder on the local computer. Remove Remote Directory c Deletes the specified folder on the FTP server. Delete Local Files c Deletes the specified files on the local computer. Delete Remote Files c Deletes the specified files on the FTP server. Chapter 5: Integration Services Control Flow Tasks 143 It is easy to configure the FTP task as long as you understand how it works. During configuration of the FTP task, you may have to define the RemotePath, the LocalPath, and some other properties for both the local and the remote computers. However, the configuration options available to you change depending upon the type of operation you choose in the Operation field. The FTP Task Editor connects to the FTP server using the FTP Connection Manager Editor. FTP Connection Manager is defined in the Connection Managers area on the Control Flow Panel in BIDS. In FTP Connection Manager Editor, you specify the server name/IP address, the server port number, the credentials for accessing the FTP server, and the options such as number of retries, time out, and so on. You can either specify the path to the folder on the FTP server directly in the RemotePath field or use a variable to pass the path to the FTP task. The FTP task can access or download multiple files from the FTP server using wildcards; however, when you want to send or upload files, the FTP task works slightly differently, depending on how you specify the LocalPath option. If you use a File Connection Manager in the LocalPath field to connect to a local file, the FTP task can access only one file in this case as the File Connection Manager can access only one file at a time. To send multiple files from the local computer to the ftp server, you need to specify the LocalPath using a variable, by first setting the IsLocalPathVariable property to True (shown later, in Figure 5-2). If you are using a variable, you can use wildcard characters such as * or ? for specifying multiple files. The FTP task behaves the same when you want to delete files from a local folder. So, if you want to send multiple files to an FTP server or want to delete multiple files on the local computer, you must use a variable to specify the LocalPath. An alternate approach could be to place the FTP task inside the Foreach Loop Container to enumerate across multiple files. Preparations for the Hands-On Exercises in This Chapter Let’s do a Hands-On exercise with the FTP task to help you understand the various options. In this exercise, you will download files from an FTP server. But before you start, make sure you’ve completed the following steps to prepare yourself for the exercise: 1. Download the software and the files from the McGraw-Hill web site and copy them to C: drive as explained in the Appendix. 2. By now, you should have attached the provided Campaign database to your SQL Server 2005 database server; if you have not attached it, do that now so that you can complete the exercises. Attaching the Campaign database to your SQL Server 2008 is explained in the Appendix. 3. Install an FTP service on a second PC for this exercise, or have access to an FTP server. After that, create a folder called Sales on the FTP server root folder and copy the DealerSales01.zip and DealerSales02.zip files from the local C:\SSIS\ RawFiles folder to the Sales folder on the FTP server. 144 Hands-On Microsoft SQL Server 2008 Integration Services Hands-On: Downloading Zipped Files Dealers of Adventure Works Bikes submit their sales reports in zipped form to an FTP server, and these files need to be downloaded from the FTP server for import into an SQL Server database. You have been tasked with creating a project that downloads these zipped files. Method In this exercise, you will use the FTP task to download two files from the remote folder Sales on the FTP server. These files will be downloaded to the C:\SSIS\downloads folder on your local computer. A couple of points to note: First, if you want to use the Downloading Zipped Files package that has been provided with this book, you will receive an error when opening the package. When you click OK on the pop-up error message, the package will load properly but without the connection string in the FTP task. This is because, by default, the sensitive information (passwords, connection strings, and so on) in the packages get encrypted using the User key (which is my user key in this case), and when another user tries to open the package, it throws an error and subsequently opens the package after removing the sensitive information. However, if you open the Downloading Zipped Files package after you’ve completed the work in this Hands-On, you will not get any such error. Second, this package requires a connection to an FTP server. Many FTP sites are available from which you can download a file with this task—such as FTP sites for antivirus updates—and build the package for this exercise in case you don’t have access to a test FTP server. Exercise (Configure FTP Task) You will be creating a new package here to work with FTP task. This package will be used in a later exercise as a child package. 1. Start BIDS and choose File | New | Project to open a New Project window. In this window, make sure Business Intelligence Projects is selected under Project Types, and then choose Integration Services Project from the Templates pane. Type in the following additional details and click OK to create a new project. Name Control Flow Tasks Location C:\SSIS\Projects 2. When the blank project is created, go to Solution Explorer in the BIDS and right-click the Package.dtsx package under the SSIS Packages folder; choose Chapter 5: Integration Services Control Flow Tasks 145 Rename from the context menu. Type Downloading zipped files.dtsx to rename the package and click Yes in the Visual Studio confirmation box. 3. Drag the FTP task from the Toolbox onto the SSIS Designer. Double-click the FTP task icon to open the FTP Task Editor. On the General page, specify the following details: Name Download Files Description This task downloads sales reports files from the FTP Server. Click in the FtpConnection field to show the down arrow button. Click the down arrow and choose <New Connection…> from the drop-down list. This will open the FTP Connection Manager Editor. In the Server Settings area, type the name of your FTP server (W8KR2, in my case) in the Server Name field and 21 in the Server Port field as shown in Figure 5-1. W8KR2 is the name of the FTP server used in the lab setup for creating the projects used in this book. Figure 5-1 Configuring the FTP Connection Manager 146 Hands-On Microsoft SQL Server 2008 Integration Services In the Credentials area, type the user name and password that you can use to connect to the FTP server to download files. Leave other fields set to the default settings. Click the Test Connection button to test the connectivity to the FTP server. If the test reveals a successful connection, click OK twice to close the message window and the FTP Connection Manager Editor. For more details on the FTP Connection Manager, refer to the discussion in Chapter 3. 4. Go to the File Transfer page from the left pane in the FTP Task Editor. 5. Select Receive Files in the Operation field. Note that the available options change with the choice of operation selected. 6. Select True for the IsTransferAscii field. For the benefit of those who are new to using FTP to download files, FTP uses two modes to transfer files—ASCII and Binary. ASCII mode is used to transfer text files—i.e., files that contain letters, numbers, and simple punctuation characters. Binary files, on the other hand, are structured differently and hence require a different mode of transfer—i.e., Binary mode. Examples of binary files include images, executable files, and algorithmically generated packages. 7. In the Remote Parameters section, click in the RemotePath field and you will see an ellipsis (…) button on the far-right corner. Click this ellipsis button to open Browse For File window. This window shows the directory structure of the remote FTP server. Choose the path where you copied the files earlier while preparing for this exercise; or, if you have copied the files to the Sales folder on the FTP server, choose the /Sales folder and then choose the DealerSales01.zip file and press OK. You will see /Sales/DealerSales01.zip listed in the RemotePath field. You want to select both files listed in the folder, but you can’t do that using the graphical interface. The FTP task allows the use of wildcard characters such as * and ? to specify filenames to enable you to select multiple files. Change the preceding path in the RemotePath field to /Sales/*.zip to select both the files. 8. In the Local Parameters section, click in the LocalPath field, click the down arrow, and choose <New Connection…> from the drop-down list. This will open the File Connection Manager Editor for you to specify an existing folder path where you want to download the files. Click the Browse button next to the Folder field and select the C:\SSIS\downloads folder by browsing to the correct path. Create this folder if this doesn’t exist already. Click OK and you will see downloads listed in the LocalPath field. You should also be able to see a File Connection Manager downloads created under the Connection Managers area (see Figure 5-2). 9. Select True for the OverwriteFileAtDest field, since you will be running this package many times. However, while configuring this task for your production servers, carefully consider using this option. Chapter 5: Integration Services Control Flow Tasks 147 10. Click OK to complete the configuration and close the editor. Press 5 to run the package and you will see that the task turns yellow for a while; once the file downloading is completed, it will turn green. (If your network firewall or local computer’s firewall [for instance, the Windows firewall] is blocking the outbound connection to the FTP site, you may have to allow this connection by changing your firewall settings. See your network administrator for help on changing firewall settings.) 11. Navigate to the C:\SSIS\downloads folder and you’ll notice that the two Zip files have been downloaded to that folder. 12. Press -- to save all the files in this solution and then choose File | Close Project. Figure 5-2 Configuring the FTP task in the Task Editor . destination SQL Server. 140 Hands-On Microsoft SQL Server 2008 Integration Services Control Flow Task Description Transfer SQL Server Agent jobs from one instance of SQL Server to the other instance. Transfer. server. 144 Hands-On Microsoft SQL Server 2008 Integration Services Hands-On: Downloading Zipped Files Dealers of Adventure Works Bikes submit their sales reports in zipped form to an FTP server, . preconfigured tasks and components of Integration Services using a .NET-compliant programming language such as Visual 142 Hands-On Microsoft SQL Server 2008 Integration Services Basic or C#. However,