Hands-On Microsoft SQL Server 2008 Integration Services part 53 ppsx

498 Hands-On Microsoft SQL Server 2008 Integration Services And add the following script into the Main subroutine: 'Update the HandshakeFile.txt with UPDATED message if the processing has started in this run 'Otherwise i.e. for any other status such as UPDATED or ELSE do nothing. If Dts.Variables("HandshakeMessage").Value = "OK" Then My.Computer.FileSystem.WriteAllText("C:\SSIS\RawFiles\HandshakeFile.txt", "UPDATED", False) End If 16. After completing the middle branch, let’s complete the leftmost branch, which is very simple, as there is very little logic to implement in this branch other than that it has to be attached to the final Script task. Click the Determine Workflow Using Handshake Script task and drag the green arrow on to the final Script task labeled as Update HandshakeFile. This will set a constraint on the final Script task to be executed only when the two preceding tasks—the first Script task and the Data Flow task—get executed. We do not want this, as the workflow is mutually exclusive and both tasks will never be executed in the same run. We need an OR constraint; that is, the final Script task should execute when either of the tasks complete. Double-click the newly added green line and change the multiple constraints condition from Logical AND to Logical OR as shown in the Figure 11-6. Also, select Expression in the Evalutation operation field and set Figure 11-6 Setting the Logical OR precedence constraint Chapter 11: Programming Integration Services 499 Expression as follows as we want the workflow to run via this branch only when the HandshakeMessage value is UPDATED. Close the Precedence Constraint Editor Dialog box. @HandshakeMessage = = "UPDATED" 17. Now let’s complete the rightmost branch of the work flow. Add another Script task in the package control flow on the right side and above the Data Flow task and join the first Script task and the new Script task. Open the precedence constraint, select Expression in the Evaluation Operation field, and set Expression as follows. Then click OK to close the editor. @HandshakeMessage = = "ELSE" 18. Configure this Script task as follows: Name Check after delay ScriptLanguage Microsoft Visual Basic 2008 EntryPoint Main ReadWriteVariables User::HandshakeMessage And add the following script into the Main subroutine as shown in Figure 11-7: Dim strFileReader As String 'MsgBox("Started _ " & TimeOfDay()) System.Threading.Thread.Sleep(5000) 'MsgBox("Finished _ " & TimeOfDay()) strFileReader = My.Computer.FileSystem.ReadAllText("C:\SSIS\RawFiles\ HandshakeFile.txt") Dts.Variables("HandshakeMessage").Value = strFileReader If strFileReader = "OK" Then My.Computer.FileSystem.WriteAllText("C:\SSIS\RawFiles\HandshakeFile.txt", "PROCESSING", False) 'Set the HandshakeMessage variable as ELSE if the loading or processing doesn't finish in one hour. 'The message in the HandshakeFile.txt stays the same as it existed in the file. ElseIf (strFileReader = "LOADING" Or strFileReader = "PROCESSING") Then Dts.Variables("HandshakeMessage").Value = "ELSE" End If In this script, first of all, a delay has been added for five seconds (though the logic required is for one hour, but for our test, I’m sure you wouldn’t want to wait for that long!). The delay is using another subroutine, Sleep, that has been declared in a separate statement. After the delay, the handshakeFile.txt file is read and the HandshakeMessage variable is updated with the new value. In case the new value is OK, that is, the running process has completed during the time Script task was waiting, the PROCESSING string is written in the handshakeFile.txt; otherwise, the HandshakeMessage variable value is changed to ELSE. 500 Hands-On Microsoft SQL Server 2008 Integration Services 19. As per the logic, we need to add two branches: one to connect to the Import RawDataTxt Data Flow task and the other to the Update HandshakeFile Script task. Refer to Figure 11-8 to see the final layout of the control flow in the package. Double-click the precedence constraint that joins the Check After Delay Script task and the Import RawDataTxt Data Flow task to open the editor. Change the constraint option to Expression with the following value in the Expression field: @HandshakeMessage = = "OK" Also, change the multiple constraints option to Logical OR, as the Data Flow task now has two mutually exclusive constraints defined on it. Open the editor for second constraint from this Script task to the Update HandshakeFile Script task. This expression should already have Logical OR multiple constraints defined. Change the constraint option to Expression with the following value in the Expression field: @HandshakeMessage = = "ELSE" Now that the logic has been implemented for all the three branches, it’s time to run the package and test for various scenarios. Figure 11-7 Script for the Check After Delay Script task Chapter 11: Programming Integration Services 501 Exercise (Debugging the Script Task) The Script task provides breakpoints that can be applied in the script and allow you to step into the code to see how the values of variables change. We will set a breakpoint in the first Script task to see how the values change. 20. Double-click the Determine Workflow Using Handshake Script task to open the editor and then click Edit Script to open the VSTA IDE environment. Scroll through the code to the section where you’ve added your own code. Locate the third line that sets the HandshakeMessage variable value to strFileReader and click the left side gray bar. This will set a breakpoint on the line and will highlight the line in red as shown in Figure 11-9. Figure 11-8 Control flow for the Extending SSIS with Script task package 502 Hands-On Microsoft SQL Server 2008 Integration Services 21. Close the VSTA IDE and the Script task by clicking OK. Notice that a breakpoint has been set up on the task. Right-click the Script task and note that the breakpoint you set earlier has been shown in this dialog box and the line and character number has also been specified. Click OK to close. 22. Make sure that the HandshakeFile.txt file in the C:\SSIS\RawFiles folder has an OK string before we start debugging. Also, be aware that the strings read from the file are case sensitive. The code has been built using all uppercase strings. Click the Start Debugging button in the BIDS to run the package. As the package execution starts and the first Script task changes to yellow color, VSTA IDE will open in couple of seconds to show you where the script breaks as it highlights the row in yellow. If the Locals window does not open, you can open it from the Debug | Windows | Locals command and press the autohide push pin on the top-right corner of the window to keep it open as you walk through the script. 23. There are a few ways to see the value of a variable—the Locals window, a watch or quick watch window, or directly in the code. Note that the strFileReader variable in the Locals window shows a value of OK. Now hover the mouse over the strFileReader in the breakpoint line; you will be shown the value as a pop-up hint. Sometimes the value could be a long string, in which case you can invoke the Text Visualizer by clicking the zooming lens symbol shown next to the value in the pop-up hint or from the Locals window. You can also open the QuickWatch window from the Debug menu or by pressing - 9 and evaluating any expression or a variable. If you type strFileReader in the Expression field and click Reevaluate, you will be shown the current value. Figure 11-9 Setting a breakpoint within the Script task script Chapter 11: Programming Integration Services 503 24. Press 8 to step into the code and move forward line by line. Press  for three more times to stop at the End If line. Switch to Windows Explorer and open the HandshakeFile.txt. You will see the PROCESSING string in the file that has been written by the code. Close the file and switch back to the VSTA IDE. Complete the processing of the script and the package. Once the package execution has been completed, you can check the HandshakeFile.txt again to see the UPDATED message written in the file. 25. You can rerun the package using different strings in the HandshakeFile.txt file and applying breakpoints in the different Script tasks. Exercise (Raising Events and Logging in the Script Task) You can raise events within your script to report information, warnings, and errors to the package so that the package event handlers can respond to these events. The Events property of the Dts object is used to raise events. The Dts.Events property exposes the following methods to fire the events. Event Method Description FireCustomEvent Fires a user-defined custom event—e.g., you may want to capture certain data-related attributes. FireError Fires an error event. FireInformation Fires an informational message—e.g., information for auditing purposes. FireProgress Fires the on-progress event. FireQueryCancel Fires an event to indicate that the task should be canceled immediately. FireWarning Fires a warning that is less critical than an error—e.g., a warning that a varchar(50) column is being mapped to varchar(10), which will work as long as the data doesn’t exceed ten characters. Similar methods are exposed by the Script component in the Data Flow task that you will study later in this chapter; however, keep in mind that the usability for these methods is different in both the script objects in SSIS. The events raised using any of the previously mentioned methods can be logged to any log provider that has been enabled on the package. To learn more about SSIS logging or log providers, refer back to Chapter 8. The Script task, though, can also log information directly without using the user-defined events. The Dts.Log method is used to log the information to all the log providers enabled on the package. When you add a Script task to your package, a custom log entry ScriptTaskLogEntry is made available in the log events to log information when the code is run inside the Script task. To configure logging in a package for a Script task, you have to select this log entry in the Configure SSIS Logs dialog box. 504 Hands-On Microsoft SQL Server 2008 Integration Services 26. Add the following line of code before the End If line to raise an error and fail the package. For this code to be run, change the value in the HandshakeFile.txt to any other string, say ABC. Else : Dts.Events.FireError(0, "Determine Workflow using Handshake", "UNKNOWN STATUS IN THE HANDSHAKEFILE: " + strFileReader, String.Empty, 0) Now if you run this package and step through the code, you will see that this line of code is executed and Script task turns red, failing the package. You can check out the following message displayed in the Output window or the Progress tab. [Determine Workflow using Handshake] Error: UNKNOWN STATUS IN THE HANDSHAKEFILE: ABC Review You have used the Script task three times with minor variations in this exercise and understand how easily you can develop a custom functionality with this task. You’ve used the Dts object, which provides almost all the interactivity to the Script task, and you’ve used its two properties, Variables and TaskResult, in this example. Don’t forget there are other important properties, such as Connections, ExecutionValue, and Transaction, that can be quite helpful for building custom functionalities, while the properties like Events and Log can be used to raise errors, warnings, and informational messages and log them to any of the log providers that have been enabled in the package. I would encourage you to take time to go through the examples available in the Books Online and on the web for the Script task. At this time, it will be worthwhile to highlight some of the best practices and the ideas that you can use while scripting: Always try to use your code within TRY/CATCH block. e previous examples c have not used TRY/CATCH purposely to keep them simple and understandable. However, it is not difficult to implement that in real life. Try to use package connection managers in the script using the Dts.Connections c property instead of creating connections afresh, as the package connection managers come with the benefits of providing the connection information and the flexibility to change the connections using package configurations. Remember that the TaskResult property can be used to force the return status to c the Integration Services run time. Using this, you can better control the workflow in your package. And if you need, you can return a value or information using the ExecutionValue property. And finally and probably a more powerful feature, don’t forget the ability of the c Script task to use the Microsoft.VisualBasic namespace (and the Microsoft.CSharp namespace in case you’re using C#), the .NET Framework class library, and the Chapter 11: Programming Integration Services 505 custom-managed assemblies, to implement custom functionality. You can also enable your script to access objects and methods from a web service when you use the Add Web Reference command in the VSTA. To add a reference to a managed assembly, you need to perform actions for design-time access and run- time access separately. To allow access at design time, save the managed assembly on your local computer and use the Add Reference command in VSTA (right-click the project or use the Project menu), and in the Add Reference dialog box, go to the Browse tab and there locate and add the managed assembly. For run-time access, you need to sign the managed assembly with a strong name and install the assembly in the GAC (global assembly cache) on the computer on which the package will be run. Script Component The Script component has been mentioned briefly in the previous two chapters, where you learned that this component can be used as a data source, a transformation, or a destination. The basic functionality of this component is to enable you to write custom code and include that custom code as a component inside the package. This custom code will be executed for every row of data, as the Script component is part of the row transformations. The Script component makes it easy to extend the data flow with custom components by writing lot of infrastructure code for you. Just like the Script task, this component also allows you to write custom code using either Microsoft Visual Basic 2008 or Microsoft Visual C # 2008. Using the Script component as a transformation, you can write custom code to call functions that are not available in SSIS—for instance, you can call the .NET assembly to use working code that is available outside SSIS; build transformations that are not available in SSIS; or apply multiple transformations with custom code to enhance the performance of your package. You can, in fact, write custom code to do all the transformations within the Script component. When the Script component is configured to work as a transformation, it provides an input and multiple outputs, but no error output. However, you can direct your error rows to one of the outputs using the ExclusionGroup property and the Row.DirectRowTo<Output Name> method and simulate it as an error output. You will learn about this in detail later in this chapter. You may be wondering how the Script component is different than the Script task and when you should choose one over the other. So, before we go into exploring the Script component more, let’s understand the differences between the two scripting objects provided in the Integration Services. 506 Hands-On Microsoft SQL Server 2008 Integration Services Script Task vs. Script Component While both the objects provide scripting environments using the VSTA engine, yet there are inherent differences in their purpose and functions. This is due to their design architecture and the placement in development environment: the Script task is made available in the Control Flow tab, while the Script component works in the Pipeline or the Data Flow tab. You can use either object to extend your package, but they extend different aspects of the package and shouldn’t be used to perform each other’s function. While the Script task extends the workflow, the Script component is used to create custom functionality within the pipeline or the data flow. You may create some data flow functionality in the Script task, but this is not what it is designed for. You will find it much easier to develop and extend your data flow custom script with the Script component than with the Script task. For example, you can create a source or a transformation or a destination with a Script component quite easily and focus on the code required to develop that particular component. Depending on what you choose to build with a Script component—i.e., a source, a transformation, or a destination—you will be configuring different metadata and properties in the editor; for instance, for a source you will be configuring outputs only, as there are no inputs for a source. The choice of function and the metadata and properties you configure in the editor results in creation of different members of the base classes in the auto-generated code. The Script task does not provide any such facility, as the code generated is always the same. One of the main differences you should know is the way the script is executed by both the objects. The Script task executes the script only once for each execution instance of the task, while the Script component generally executes the script for each row, as it applies the custom transformation on each of the data row it reads. This difference alone makes the Script component a much desirable and powerful option for data-related operations. The abilities of both objects to generate and work with different base classes also differentiate them in the way you can write scripts. While working with the Script task, you realized that most of the interaction of the task with the package is provided by the global Dts property of the ScriptMain class. The Dts property works with only the Script task and is not available in the Script component, which uses typed accessor properties to access certain package features such as variables and connection managers. We will go through various code examples when we work with the Script component, but here just keep in mind that the programming object model for the Script component is different than that of the Script task. Chapter 11: Programming Integration Services 507 Some other minor differences to note are that the Script component does not support breakpoints, and you use alternate methods such as a MessageBox to interrupt the execution of the script or raise events that you can examine in the Progress tab or log events to examine later when the execution has completed. I’ll just mention here that some developers prefer to use trace listeners by using the Trace.Writeline method of the System.Diagnostics namespace. Trace Listeners are objects that capture the tracing information and route it outside your application to the place where you want to store it, for instance, in a text file or an event viewer. While doing that, you have an opportunity to watch the trace results using available utilities. If you want to have more visibility to your debug information, you can use this method. Finally, the Script component does not report the execution results as reported by the Script task using the TaskResult or ExecutionValue properties. Once added in a data flow, the Script component becomes an integral part of the pipeline and the success or failure depends on the execution status of Data Flow task. Hands-On: Extending Data Flow with the Script Component A Script component can be configured as a data source, a transformation, or a destination within the Data Flow task. We will now do one Hands-On exercise to import a nonstandard text file for sales, derive a bonus based on employee title and the sales the employee has achieved, and finally will write those values into a comma-delimited and nonstandard text file. Sometimes even though you can perform some operation using preconfigured components, you may still prefer to use a Script component if you find that it is easier to keep the logic in one place or you’re using too many preconfigured components that can otherwise be built easily using a Script component. In this exercise, though we will be working with text files, the Script component can virtually work with any type of source—in fact, it will be correct to say that it can be configured to read and write from any source or destination that is used in most enterprises. Script Component as a Data Source In this part, let’s explore how the Script component can be configured to work as a data source to import nonstandard structured data from a text file. We will import the Sales .txt file saved in the C:\SSIS\RawFiles folder. Open the file and note the structure of the data it contains. The file contains information of one record in six rows that could otherwise be one database row with some blank rows and some sundry comments in between the records. This is not a standard comma-delimited file that SSIS can easily . differences between the two scripting objects provided in the Integration Services. 506 Hands-On Microsoft SQL Server 2008 Integration Services Script Task vs. Script Component While both the objects. otherwise, the HandshakeMessage variable value is changed to ELSE. 500 Hands-On Microsoft SQL Server 2008 Integration Services 19. As per the logic, we need to add two branches: one to connect. 11-9. Figure 11-8 Control flow for the Extending SSIS with Script task package 502 Hands-On Microsoft SQL Server 2008 Integration Services 21. Close the VSTA IDE and the Script task by clicking OK. Notice