Chapter 10: SQL Server Integration Services 399 Package Security SSIS packages can contain sensitive authentication information, and saving those packages opens up the possibly of a security exposure. To protect against this possibility, SSIS supports the encryption of sensitive information. SSIS uses the Triple Data Encryption Standard (3DES) cipher algorithm with a key length of 192 bits, and packages are encrypted either when they are created or when they are exported. SSIS package encryption is controlled using the package’s ProtectionLevel property, which supports the following values: ProtectionLevel Value Description DontSaveSensitive Sensitive data is not saved in the package. When the package is opened, the sensitive data will not be present and the user will need to provide the sensitive data. EncryptSensitiveWithUserKey Sensitive data is saved as a part of the package and is encrypted with a key that’s based on the user who created or exported the package. Only that user will be able to run the package. If another user opens the package, the sensitive data will not be available. EncryptSensitiveWithPassword Sensitive data is saved as a part of the package and is encrypted with a user- supplied password. When the package is opened the user must provide a password to access the sensitive data. If the password is not provided, the package will be opened without the sensitive data. EncryptAllWithPassword The entire contents of the package will be encrypted with a user-supplied password. When the package is opened, the user must provide the package’s password. If the password is not provided, the package will not be able to be opened. EncryptAllWithUserKey The entire contents of the package will be encrypted with a key that’s based on the user key for the user who created or exported the package. Only the user who created the package will be able to open it. ServerStorage The package is not encrypted. Instead, the package’s contents are secured according to the database’s object access security. If the ServerStorage value is used, the package must be saved to the sysdtspackages90 table in the msdb database. It cannot be saved to the file system. Deploying Packages SSIS supports the deployment of packages through the use of the package configurations and the ability to easily deploy packages using the package deployment utility. In the next section you’ll see how to create a configuration for an SSIS package as well as how to use the package deployment utility. 400 Microsoft SQL Server 2005 Developer’s Guide Creating Confi gurations Configuration information enables an SSIS package to automatically load external information at run time. You can use configurations to pass in variable values and connection information to an SSIS package at run time. For variables, the Value property is assigned with the value that is passed in when the package is run. Likewise, for connection information the Connection Manager’s properties such as ConnectionString, ServerName, and InitialCatalog can be assigned to dynamically change the server system that will be used by an SSIS package. SSIS configurations are created by using the Package Configuration Organizer, which is started from BIDS. To create a package configuration for an Integration Services project, select the SSIS | Package Configurations option in BIDS. This will start the Package Configurations Organizer tool that is shown in Figure 10-24. You can create multiple configurations for a single package. The configurations are applied to the package in the order that they are displayed in the Package Organizer. You can use the directional arrows shown on the right side of Figure 10-24 to move a configuration up or down in the list. You can also create a single configuration that can be applied to multiple packages. For example, if you want to deploy a package to several systems where the only difference in the package properties is the server name, you could create a configuration that uses an environment variable to supply the server name. Figure 10-24 The Package Configuration Organizer Chapter 10: SQL Server Integration Services 401 To create a configuration, first check the Enable Package Configurations check box and then click Add to start the SSIS Configuration Wizard. The Configuration Wizard steps you through creating a package configuration. Click past the Wizard welcome screen to display the Configuration Type dialog shown in Figure 10-25. The Configuration Type drop-down enables you to select the data source that will be used by the configuration. SSIS package configuration supports the following configuration types: ᭤ XML confi guration fi le ᭤ Environment variable ᭤ Registry entry ᭤ Parent package variable ᭤ SQL Server Figure 10-25 Configuration Type 402 Microsoft SQL Server 2005 Developer’s Guide In Figure 10-25 you can see that the type of Environment variable has been selected, along with the COMPUTERNAME variable. XML file configurations and SQL Server configurations support the selection of multiple properties in a single configuration object. The other configuration types permit only one configurable property per configuration. Click Next to display the dialog shown in Figure 10-26, which shows where you select the package’s properties or variables that will have their values set by the configuration when the package is run. In Figure 10-26 you can see that the properties for the OLE DB Connection Manager named SQL2005-2.AventureWorks1 have been expanded and that the ServerName property has been selected. This will enable the COMPUTERNAME environment variable to be substituted for the OLE DB Connection Manager’s ServerName in the connection string when this package attempts to use the OLE DB connection. Clicking Next displays the configuration summary screen, which allows you to view and confirm your selections. If you need to make changes, you can use the Back button to page back through the Configuration Wizard and make any needed changes. Otherwise, clicking Next adds the configuration to the package and will display the Package Configuration Organizer with your configuration as you can see in Figure 10-27. Figure 10-26 Select Target Property Chapter 10: SQL Server Integration Services 403 Later if you need to modify a configuration, click Edit to rerun the Configuration Wizard and select different objects and different properties. Using the Package Deployment Utility SSIS contains a handy feature called the Package Deployment Utility that allows you to assemble your SSIS packages, package configurations, and supporting files into a deployment folder and build an executable setup file to install your packages. To create the Package Deployment Utility, right-click the project properties in the BIDS Solution Explorer pane and then select the Properties option to display the Property Pages dialog box as shown in Figure 10-28. Set the CreateDeploymentUtility option to True on the project property page. Then build your project by selecting the Build Solution option on the BIDS menu. Building the project creates the DTSDeploymentManifest.xml file and copies the project along with the DTSInstall.exe utility to the bin/Deployment folder or to the location specified in the DeploymentOutputPath property. The DTSDeploymentManifest.xml file lists the packages and the package configurations in the project. The DTSInstall.exe program runs the Package Installer Wizard. Figure 10-27 The completed package configuration 404 Microsoft SQL Server 2005 Developer’s Guide Programming with the SQL Server Integration Services APIs In addition to providing a graphical development environment, SSIS also provides an object model API for both the DTR and the DTP that enables you to programmatically create and execute SSIS packages. Programming the data flow engine enables you to automate the creation and configuration of the SSIS tasks, transformations, and data flow tasks, and to create custom components. The run-time engine is exposed both as a native COM object model and as a fully managed object model. The SSIS data flow engine is written in native code, but it can be controlled programmatically using a managed .NET object model. In this next section you’ll see an example of how you can use the SQL Server Integration Services API in a console application to create and execute a new SSIS package. The SQL Server Integration Services API is located in a number of different assemblies: Microsoft.SqlServer.ManagedDTS.dll, SqlServer.DTSPipelineWrap .dll, and SqlServer.DTSRuntimeWrap.dll. To use these assemblies in your program, you need to add references for each of them in your project. Then you can use Figure 10-28 Package Deployment Utility Chapter 10: SQL Server Integration Services 405 the Integration Services classes to create both SSIS DTP and DTR objects in your application. To add references to your project, select the Project | Add Reference menu option to display the Add Reference dialog. Scroll through the list until you see the Microsoft.SqlServer.DTSPipelineWrap, Microsoft.SqlServer.DTSRuntimeWrap, and Microsoft.SqlServer.ManagedDTS assemblies listed in the Component Name list. Select these assemblies as is illustrated in Figure 10-29. Click OK to add the references to your project. Next, add import directives for the SSIS assembly namespaces to the Declarations section of your project. Using import directives enables you to use the classes in the imported namespaces without having to fully qualify the names. The following code listing shows how to create import directives for the SSIS namespace: Imports Microsoft.SqlServer.Dts.Runtime Imports Microsoft.SqlServer.Dts.Pipeline.Wrapper Imports SSISRuntime = Microsoft.SqlServer.Dts.Runtime.Wrapper NOTE To avoid compile-time errors due to common object names, it’s best to use an alternative import name when importing both Microsoft.SqlServer.DtsPipeline.Wrapper and Microsoft.SqlServer .DtsRuntime.Wrapper. That’s why the third Imports statement uses the name SSISRuntime. Figure 10-29 Adding references to SSIS assemblies 406 Microsoft SQL Server 2005 Developer’s Guide After adding the SSIS references to your project and import directives for the appropriate namespaces, you’re ready to begin using the SSIS APIs in your application. The following code sample shows how you can create a package using the SSIS APIs: Module CreateSSISPackage Sub Main() ' Create the Package Console.WriteLine("Creating the MySSIS Package") Dim myPackage As New Package() myPackage.PackageType = DTSPackageType.DTSDesigner90 myPackage.Name = "MySSISPackage" myPackage.Description = "Created using the SSIS API" myPackage.CreatorComputerName = System.Environment.MachineName myPackage.CreatorName = "Otey" 'Add the OLE DB and Flat File Connection Managers Console.WriteLine("Creating the MyOLEDBConnection") Dim cnOLEDB As ConnectionManager = _ MyPackage.Connections.Add("OLEDB") cnOLEDB.Name = "MyOLEDBConnection" cnOLEDB.ConnectionString = _ "Provider=SQLNCLI;Integrated Security=SSPI;" _ & "Initial Catalog=AdventureWorks;Data Source=SQL2005-2;" Console.WriteLine("Creating the MyFlatFileConnection") Dim cnFile As ConnectionManager = _ myPackage.Connections.Add("FLATFILE") cnFile.Properties("Name").SetValue(cnFile, "MyFlatFileConnection") cnFile.Properties("ConnectionString").SetValue _ (cnFile, "c:\temp\MySSISFileExport.csv") cnFile.Properties("Format").SetValue(cnFile, "Delimited") cnFile.Properties("ColumnNamesInFirstDataRow") _ .SetValue(cnFile, False) cnFile.Properties("DataRowsToSkip").SetValue(cnFile, 0) cnFile.Properties("RowDelimiter").SetValue(cnFile, vbCrLf) cnFile.Properties("TextQualifier").SetValue(cnFile, """") Near the top of this listing you can see where a new SSIS package object named myPackage is created. Next, the package’s properties are assigned values. The most important of these are the PackageType and Name properties, where the values of DTSpackageType.DTSdesigner90 and MySSISPackage are used. After creating the package object, the next step is to create Connection Managers for the package. In this example, the package will be performing a simple export from SQL Server to the file system, which requires two Connection Managers: an OLE DB Chapter 10: SQL Server Integration Services 407 Connection Manager to connect to SQL Server, and a Flat File Connection Manager to write the export file. First, the OLE DB Connection Manager is created and named MyOLEDBConnection. The Connection’s Add method is then used to add the new ConnectionManager object to the package’s Connections collection. Then the OLE DB ConnectionManager’s ConnectionString property is assigned a connection string that will connect to the AdventureWorks database on the server named SQL2005-2 using integrated security. After that, a similar process creates a Flat File ConnectionManager object named MyFlatFileConnection and adds it to the package’s Connections collection. Then the ConnectionString property of the MyFlatFileConnection is assigned the value of c:\temp\MySSISFileExport.csv. The following property assignments set up a delimited file type for the export operation. After the creation of the package and Connection Manager objects, the next step to the creation of a Data Flow task is shown in the following code listing: 'Add a Data Flow Task Console.WriteLine("Adding a Data Flow Task") Dim taskDF As TaskHost = _ TryCast(myPackage.Executables.Add("DTS.Pipeline"), TaskHost) taskDF.Name = "DataFlow" Dim DTP As MainPipe DTP = TryCast(taskDF.InnerObject, MainPipe) ' Add the OLE DB Source Console.WriteLine("Adding an OLEDB Source") Dim DFSource As IDTSComponentMetaData90 DFSource = DTP.ComponentMetaDataCollection.New() DFSource.ComponentClassID = "DTSAdapter.OLEDBSource" DFSource.Name = "OLEDBSource" ' Connect, populate the Input collections and disconnect Dim SourceInst As CManagedComponentWrapper = _ DFSource.Instantiate() SourceInst.ProvideComponentProperties() DFSource.RuntimeConnectionCollection(0).ConnectionManagerID _ = myPackage.Connections("MyOLEDBConnection").ID DFSource.RuntimeConnectionCollection(0).ConnectionManager _ = DtsConvert.ToConnectionManager90 _ (myPackage.Connections("MyOLEDBConnection")) SourceInst.SetComponentProperty("OpenRowset", "[Sales].[Customer]") SourceInst.SetComponentProperty("AccessMode”, 0) SourceInst.AcquireConnections(Nothing) SourceInst.ReinitializeMetaData() SourceInst.ReleaseConnections() 408 Microsoft SQL Server 2005 Developer’s Guide Near the top of this listing you can see where a new Data Flow task named taskDF is created. Here the Data Flow task must be set up to read from the OLE DB Connection Manager that will be the source of the data and then write to the flat file connection that acts as the data destination. After creation of the Data Flow task, an OLE DB data adapter for the source data is created that’s named DFSource. The DFSource object’s ComponentClassID is set to DTSADpater.OLEDBSource, defining it as an OLE DB connection, and the Name of the data flow source is set to OLEDBSource. Next, an instance of the DFSource object is created in order to populate the input connections. This enables the downstream data flow components to see the input metadata. Here the OLE DB connection is set to the Sales.Customer table from the AdventureWorks database. After the input metadata has been collected, the connection is released. The next step is to define the data flow destination as is shown in the following listing: ' Add the Flat File Destination Console.WriteLine("Adding a Flat File Destination") Dim DFDestination As IDTSComponentMetaData90 DFDestination = DTP.ComponentMetaDataCollection.New() DFDestination.ComponentClassID = _ "DTSAdapter.FlatFileDestination" DFDestination.Name = "FlatFileDestination" ' Create an instance of the component Dim DestInst As CManagedComponentWrapper = _ DFDestination.Instantiate() DestInst.ProvideComponentProperties() DFDestination.RuntimeConnectionCollection(0).ConnectionManagerID _ = myPackage.Connections("MyFlatFileConnection").ID DFDestination.RuntimeConnectionCollection(0).ConnectionManager _ = DtsConvert.ToConnectionManager90 _ (myPackage.Connections("MyFlatFileConnection")) ' Map a connection between the source and destination DTP.PathCollection.New().AttachPathAndPropagateNotifications _ (DFSource.OutputCollection(0), DFDestination.InputCollection(0)) ' Add columns to the FlatFileConnectionManager Dim MyFlatFilecn As _ SSISRuntime.IDTSConnectionManagerFlatFile90 = Nothing For Each cm As ConnectionManager In myPackage.Connections If cm.Name = "MyFlatFileConnection" Then MyFlatFilecn = TryCast(cm.InnerObject, _ SSISRuntime.IDTSConnectionManagerFlatFile90) DtsConvert.ToConnectionManager90(cm) End If Next . dialog. Scroll through the list until you see the Microsoft. SqlServer.DTSPipelineWrap, Microsoft. SqlServer.DTSRuntimeWrap, and Microsoft. SqlServer.ManagedDTS assemblies listed in the Component. directives for the SSIS namespace: Imports Microsoft. SqlServer.Dts.Runtime Imports Microsoft. SqlServer.Dts.Pipeline.Wrapper Imports SSISRuntime = Microsoft. SqlServer.Dts.Runtime.Wrapper NOTE To avoid. Services API is located in a number of different assemblies: Microsoft. SqlServer.ManagedDTS.dll, SqlServer.DTSPipelineWrap .dll, and SqlServer.DTSRuntimeWrap.dll. To use these assemblies in your