Microsoft SQL Server 2000 Data Transformation Services- P14

In Figure 30.8, these two purposes are shown as separate primary branches coming from the Application in the object hierarchy. Extending the Power of DTS P ART VI 626 The Application object also has two properties that are used to set characteristics of the DTS Design environment. The JITDebug property determines Just-In-Time Debugging of ActiveX Scripts. The DesignerSettings property determines whether or not multiple phases are shown for the transformation tasks. N OTE Application PackageLogRecord PackageLogRecord StepLogRecord Ta skLogRecord StepLogRecord TaskLogRecord PackageLineage PackageLineages OLEDBProviderInfos ScriptingLanguageInfos TaskInfos TransformationInfos StepLineages PackageInfos StepLineage OLEDBProviderInfo ScriptingLanguageInfo TaskInfo TransformationInfo PackageInfo PackageSQLServer PackageRepository F IGURE 30.8 The DTS Application object hierarchy. The PackageSQLServer object and the PackageRepository objects have one collection ( PackageInfos ) in common. N OTE Chapter 23, “The DTS Package and Its Properties,” describes how to use the PackageSQLServer object to obtain information about packages stored in SQL Server. Chapter 29, “Integrating DTS with Meta Data Services,” describes how to use the PackageRepository object to obtain information about packages stored in Meta Data Services. 37 0672320118 CH30 11/13/00 5:00 PM Page 626 Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark. The four types of system information that can be retrieved using the Application object are • TaskInfos —All the DTS custom tasks that have been registered with the system. • TransformationInfos —All the DTS custom transformations that have been registered with the system. • ScriptingLanguageInfos —All the scripting languages that have been registered with the system. • OLEDBProviderInfos —All the OLE DB providers that have been registered with the system. You can retrieve this information with the VBScript code in Listing 30.1. This code is included on the CD in a file called DTSSystemInfo.vbs. L ISTING 30.1 VBScript Code That Writes DTS System Information to a File Option Explicit Dim app, info, prop, msg Dim txtstr, fso, fld, fil Function Main() Set fso = CreateObject(“Scripting.FileSystemObject”) fso.CreateTextFile “c:\temp\DTSSystemInfo.txt” Set fil = fso.GetFile(“c:\temp\DTSSystemInfo.txt”) set txtstr= fil.OpenAsTextStream(2) txtstr.Write “DTS System Info Report Created “ & Now & vbCrLf Set app = CreateObject(“DTS.Application”) txtstr.Write vbCrLf & “Registered Tasks” & vbCrLf & vbCrLf For Each info in app.TaskInfos Call fctWriteInfo Next txtstr.Write vbCrLf & “Registered Transformations” & vbCrLf & vbCrLf For Each info in app.TransformationInfos Call fctWriteInfo Next txtstr.Write vbCrLf & “Registered Scripting Languages” & _ vbCrLf & vbCrLf For Each info in app.ScriptingLanguageInfos Programming with the DTS Object Model C HAPTER 30 30 P ROGRAMMING WITH THE DTS O BJECT M ODEL 627 37 0672320118 CH30 11/13/00 5:00 PM Page 627 Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark. Call fctWriteInfo Next txtstr.Write vbCrLf & “Registered OLE DB Providers” & vbCrLf & vbCrLf For Each info in app.OLEDBProviderInfos Call fctWriteInfo Next Main = DTSTaskExecResult_Success End Function Function fctWriteInfo For Each prop in info.Properties txtstr.Write prop.Name & vbTab & prop.Value & vbCrLf Next txtstr.WriteLine End Function Documenting the Connections in a DTS Package You can use your knowledge of the DTS object hierarchy to document your DTS package. One particular kind of documentation I find useful is an inventory of connections. It’s easy to see all the connection objects in the Package Designer user interface, but it’s not easy to see how all those connections are being used. Listing 30.2 is an ActiveX Script for cre- ating a text file that inventories how a package’s connections are used. You can find this code on the CD in a file called ConnectionInventory.vbs and in a DTS package called ConnectionInventory.dts. L ISTING 30.2 Code That Documents the Use of a Package’s Connections Option Explicit Dim txtstr, cus Function Main() Dim pkg, con, tsk, lkp, dpta Dim fso, fld, fil Set pkg = DTSGlobalVariables.Parent Set fso = CreateObject(“Scripting.FileSystemObject”) fso.CreateTextFile “c:\temp\ConnectionInfo.txt” Extending the Power of DTS P ART VI 628 L ISTING 30.1 Continued 37 0672320118 CH30 11/13/00 5:00 PM Page 628 Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark. Set fil = fso.GetFile(“c:\temp\ConnectionInfo.txt”) set txtstr= fil.OpenAsTextStream(2) txtstr.Write “Report Created “ & Now & vbcrlf & vbcrlf txtstr.Write “Package Name: “ & pkg.Name & vbcrlf & vbcrlf For Each con in pkg.Connections txtstr.Write “Connection Name: “ & con.Name & vbcrlf txtstr.Write “Description: “ & con.Description & vbcrlf txtstr.Write “Connection ID: “ & con.ID & vbcrlf txtstr.Write “Provider ID: “ & con.ProviderID & vbcrlf txtstr.Write “DataSource: “ & con.DataSource & vbcrlf txtstr.Write “Catalog: “ & con.Catalog & vbcrlf & vbcrlf txtstr.Write “This connection is used in:” & vbcrlf & vbcrlf For Each tsk in pkg.Tasks Set cus = tsk.CustomTask Select Case tsk.CustomTaskID Case “DTSExecuteSQLTask”, “DTSExecuteSQLTask2” If con.ID = cus.ConnectionID Then Call fctOutputTask (“Execute SQL Connection”) End If Case “DTSDataDrivenQueryTask”, “DTSDataDrivenQueryTask2” If con.ID = cus.SourceConnectionID Then Call fctOutputTask (“DDQ Source”) End If If con.ID = cus.DestinationConnectionID Then Call fctOutputTask (“DDQ Destination”) End If For Each lkp in cus.Lookups If con.ID = lkp.ConnectionID Then Call fctOutputTask (“Lookup - “ & lkp.Name) Programming with the DTS Object Model C HAPTER 30 30 P ROGRAMMING WITH THE DTS O BJECT M ODEL 629 L ISTING 30.2 Continued 37 0672320118 CH30 11/13/00 5:00 PM Page 629 Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark. End If Next Case “DTSBulkInsertTask” If con.ID = cus.ConnectionID Then Call fctOutputTask (“Bulk Insert Connection”) End If Case “DTSDataPumpTask”, “DTSDataPumpTask2” If con.ID = cus.SourceConnectionID Then Call fctOutputTask (“Transform Data Source”) End If If con.ID = cus.DestinationConnectionID Then Call fctOutputTask (“Transform Data Destination”) End If For Each lkp in cus.Lookups If con.ID = lkp.ConnectionID Then Call fctOutputTask ( “Lookup - “ & lkp.Name) End If Next Case “DTSParallelDataPumpTask” If con.ID = cus.SourceConnectionID Then Call fctOutputTask (“Parallel Data Pump Source”) End If If con.ID = cus.DestinationConnectionID Then Call fctOutputTask (“Parallel Data Pump Destination”) End If For Each trnset in cus.TransformationSets For Each lkp in trnset.Lookups If con.ID = lkp.ConnectionID Then Call fctOutputTask ( “Lookup - “ & lkp.Name) End If Extending the Power of DTS P ART VI 630 L ISTING 30.2 Continued 37 0672320118 CH30 11/13/00 5:00 PM Page 630 Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark. Next Next Case “DTSDynamicPropertiesTask” For each dpta in cus.Assignments If dpta.SourceType = 1 Then If con.ID = dpta.SourceQueryConnectionID Then Call fctOutputTask (“Dynamic Properties Query”) End If End if Next End Select Next Next txtstr.Close Main = DTSTaskExecResult_Success End Function Function fctOutputTask(sConnectionUse) txtstr.Write vbTab & “Task Name: “ & cus.Name & vbcrlf txtstr.Write vbTab & “Description: “ & cus.Description & vbcrlf txtstr.Write vbTab & “Use: “ & sConnectionUse & vbcrlf & vbcrlf End Function Conclusion You can use the DTS Package object hierarchy to retrieve information, create new objects, and modify objects. You can use the DTS Application object hierarchy to retrieve system informa- tion about DTS objects and information about packages stored in SQL Server and Meta Data Services. The next chapter explains how to create DTS custom tasks with Visual Basic. Programming with the DTS Object Model C HAPTER 30 30 P ROGRAMMING WITH THE DTS O BJECT M ODEL 631 L ISTING 30.2 Continued 37 0672320118 CH30 11/13/00 5:00 PM Page 631 Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark. 37 0672320118 CH30 11/13/00 5:00 PM Page 632 Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark. CHAPTER 31 Creating a Custom Task in VB IN THIS CHAPTER • When You Should Create a New Custom Task 634 • Getting Started 635 •Implementing the Custom Task Interface 637 •Implementing the Custom Task User Interface 645 • Events, Errors, and Logs 650 • Registering the Custom Task 652 • Using a Custom Task in a DTS Package 653 • The FindFileTask 655 • The LocalCubeTask 656 38 0672320118 CH31 11/13/00 4:57 PM Page 633 Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark. Extending the Power of DTS P ART VI 634 You can extend the capabilities of Data Transformation Services by creating your own custom tasks with Visual Basic, Visual C++, or another programming language that supports COM. Once you have created and registered a custom task, it will appear on the task palette in the DTS Designer and you can use it along with all the other types of tasks. This chapter explains how to build a custom task using Visual Basic. Two sample applications are used to illustrate this process: •File Find task—A simple custom task that checks for the existence of a file in a particu- lar directory. This sample application is on the CD in two versions—with and without a custom user interface. • Local Cube task—A complex custom task that automates the creation of one or more local cube files from an Analysis Services cube. A demo version of this application is on the CD. Chapter 32, “Creating a Custom Transformation with VC++,” explains how to use the Active Template Library (ATL) custom transformation template that is provided with the SQL Server 2000 sample code. There are also two ATL templates provided for making custom tasks, which you can use as a starting point for building a custom task in VC++. N OTE When You Should Create a New Custom Task A custom task encapsulates a particular set of functionality in a way that can be reused in many packages. You can include a user interface with a custom task so that it can be adapted to different sets of data. You should consider the following options before building a new custom task: • If one of the existing tasks has the functionality you need, use it. • If a set of tasks combined has the needed functionality, create a DTS template that includes all the tasks and use that template when creating a new package. • If you need to manipulate individual rows of data, consider building a custom transfor- mation (as described in Chapter 32) instead of a custom task. • If you only need this functionality on a one-time basis, consider using code in an ActiveX Script task. • If the functionality you need is already available in an external application, consider calling that application from a DTS package with an Execute Process task. 38 0672320118 CH31 11/13/00 4:57 PM Page 634 Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark. If you’re comparing an external application to a custom task, you should consider the advan- tages of having the custom task integrated into the DTS system: • Uniform error handling. • The ability to set precedence based on success and failure. • The ability to participate in transactions. Here are some situations where a custom task could be useful: • If you have a particular ActiveX Script task that you are using in several packages, you could convert that task into a custom task. •You could build a custom task to automate backups in different database systems. •You could build a custom task that would automate the reporting on the results of a DTS package execution. •You could create a custom task to do an incremental update on a set of cubes. The Analysis Services Processing task allows you to do incremental updates on one cube or a full process on a set of cubes, but you have to create separate instances of the task for each cube incremental update. •You could incorporate the ability to design cube aggregations into a custom task that processed cubes. Getting Started The first step in building a custom task is to open Visual Basic and create a new ActiveX DLL project. Give the project and the class module appropriate names. You also have to add a refer- ence to one code library—the Microsoft DTSPackage Object Library. Creating a Custom Task in VB C HAPTER 31 31 C REATING A C USTOM T ASK IN VB 635 You don’t need to add a reference to the Microsoft DTS Custom Task Objects Library. That’s a library that contains some of the new SQL Server 2000 custom tasks. It doesn’t have anything to do with creating a new custom task. N OTE The Find File task described in this chapter also needs a reference to Microsoft Scripting Runtime because it uses the file system objects from this library. N OTE 38 0672320118 CH31 11/13/00 4:57 PM Page 635 Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark. [...]... and re-create it as a custom transformation However, it won’t be worth the effort unless you need the improved performance or can use the custom task repeatedly The Data Pump Interfaces If you are creating a custom transformation for SQL Server 2000, you have to implement both the IDTSDataPumpTransform interface and the IDTSDataPumpTransform2 interface 32 NOTE The IDTSDataPumpTransform interface has... Preparing the C++ Custom Transformation Development Environment 662 • Creating a Custom Transformation 664 660 Extending the Power of DTS PART VI You can create a custom transformation that can be used in all of the DTS transformation tasks—the Transform Data task, the Data Driven Query task, and the Parallel Data Pump task After you have created and registered your custom transformation, it will be... speed Data transformation can be very time-consuming because it is often necessary to transform very large numbers of rows Data transformation code has to be executed for each individual record The ability to move processing code from an ActiveX script into a Visual C++ transformation can significantly reduce processing time • Creating an efficient development environment—Sometimes the same data transformation. .. Create New Transformation dialog (see Figure 32.1) FIGURE 32.1 The AvgXform custom transformation appears in the Create New Transformation dialog Microsoft has not provided the interfaces that are needed to build a custom transformation in Visual Basic They are defined only in Visual C++ header files Why You Should Create a Custom Transformation There are two primary reasons to create a custom transformation: ... the transformation For those phases that occur for every row in the data source, this method is called for each one threads when the data pump is switching execution CREATING A CUSTOM TRANSFORMATION In addition to these two interfaces, each custom transformation must implement the functions of the COM DLL infrastructure—DLLMain, DLLGetClassObject, DLLCanUnloadNow, DLLRegisterServer, and DLLUnregisterServer... how to create custom transformations so that you can automate the process of transforming each row of data 31 CREATING A CUSTOM TASK IN VB The LocalCubeTask allows you to create local cube files from Analysis Server cubes You can do the following: 657 CHAPTER Creating a Custom Transformation with VC++ 32 IN THIS CHAPTER • Why You Should Create a Custom Transformation 660 • The Data Pump Interfaces... pkg.tasks(“tskSelectAuthors”) ‘An Execute SQL task Set cus = tsk.customtask ‘The following are all valid property assignments cus.SQLStatement = “Select * From Authors” cus.Properties(“SQLStatement”) = “Select * From Authors” tsk.Properties(“SQLStatement”) = “Select * From Authors” ‘The following is not a valid property assignment ‘It will cause an error tsk.SQLStatement = “Select * From Authors” Main... is used for row-byrow data transformation It is not used when the ProcessPhase method is used each time Execute is called Use this method to release variable allocations that are needed until the data is loaded into the destination after all the source rows have been processed so that variables can be released The IDTSDataPumpTransform2 interface has these methods: • GetTransformServerInfo—Returns •... c:\temp\LocalCubeTask.dll • Registering the custom task in the Package Designer Creating a Custom Task in VB CHAPTER 31 • Connect to an Analysis Server, choose from the databases on that server, and choose from the cubes in the database • Create a local cube with all the elements of the Analysis Server cube, or limit the cube to a subset of the measures, dimensions, levels, or members in the source cube • Specify the filename... script into a custom transformation, you can write and test it once and use it repeatedly Here are some ideas for creating custom transformations: • Importing from or exporting to XML • Parsing or recombining name fields • Parsing, recombining, and validating address fields • Performing mathematical functions Creating a Custom Transformation with VC++ CHAPTER 32 661 You can take any transformation with . to use the PackageSQLServer object to obtain information about packages stored in SQL Server. Chapter 29, “Integrating DTS with Meta Data Services,” describes. Custom Transformation with VC++,” explains how to use the Active Template Library (ATL) custom transformation template that is provided with the SQL Server 2000