318 Hands-On Microsoft SQL Server 2008 Integration Services Hands-On: Extending the Contacting Opportunities Package with Property Expressions In Chapter 4, you created an Integration Services project called Contacting Opportunities that sends mail to persons who raised a query in October 2009. However, during the package development, you used static values in the To field, Subject field, and Message field. You are now to extend the package so that these values are read from the table and evaluated for each person to create personalized messages. Method You will add the Mailing Opportunities package from the Contacting Opportunities package to a new package to keep the package separate from a learning point of view. The Mailing Opportunities package sends mails to six persons who made enquiries in October 2009 using static values. To make the package dynamic, you will update e-mail addresses in the Prospects table with your e-mail address so that you can see the messages that are being generated and sent out by this package. Then you will create variables that will capture values from the Prospects table to pass on to property expressions at run time. Finally, you will create property expressions and will also learn about the DelayValidation property toward the end of this exercise. Exercise (Build Property Expressions for Mailing Opportunities Package) Having understood the method, follow the steps below to work with property expressions. 1. Open SQL Server Management Studio and run the following query on the local server. UPDATE [Campaign].[dbo].[Prospects] SET email = 'youremailaddress' WHERE ENQUIRYDATE BETWEEN '2009/10/01' AND '2009/10/31' Replace youremailaddress in the preceding query with your e-mail address and then execute this query to update the six records. 2. Open BIDS and create a new Integration Services project with the following details: Name Contacting Opportunities with Property Expressions Location C:\SSIS\Projects Chapter 8: Advanced Features of Integration Services 319 3. When the blank project is created, delete the Package.dtsx package in the SSIS Packages node and then right-click the SSIS Packages node and choose Add Existing Package from the context menu. 4. In the Add Copy Of Existing Package dialog box, select Package Location as the File System from the drop-down list. Type C:\SSIS\Projects\Contacting Opportunities\Mailing Opportunities.dtsx in the Package Path field and click OK to add this package. Once the package has been added, double-click the Mailing Opportunities.dtsx package in the SSIS Packages folder to open it on the Designer. 5. Double-click the Iterating October Opportunities Foreach Loop container to open the editor. This package enumerates over the User::Opportunities variable. Earlier, no value was been picked up and used in the package, so let’s fill the gap now. Go to the Variable Mappings page and click in the Variable column and then click the down arrow and choose <New Variable> from the drop-down list. 6. Leave Mailing Opportunities selected in the Container field in the Add Variable pop-up dialog box. Type Title in the Name field and leave the Namespace as User and Value Type set to String. Variables are case-sensitive, so type the name all in lowercase to avoid any issues later on. Click OK to add this variable. You will see that User::title has been added in the Variable column and assigned an Index value of 0. 7. Much as you did in Step 6, create four more variables and make sure they get the Index values as per the following table: Variable Index Value Type fname 1 String lname 2 String email 3 String enquiry_date 4 DateTime For the variable enquiry_date, you will need to assign a placeholder value in the format 09/09/2009, as this type of variable cannot be defined without a value. The Variable Mappings settings should look similar to Figure 8-10. Click OK to close the editor. 8. Now that you’ve mapped the values from the table to the variables, it is time to make use of them in the package. Double-click the Mailing Opportunities Send Mail task icon to open the editor. Go to the Mail page where you will be using Property Expressions to derive and modify values assigned to various fields at run time. To be absolutely sure that the e-mail address in the To address of the e-mails is being read from the table and not from the value you’ve typed directly in the field, type test@test.com in the To field. 320 Hands-On Microsoft SQL Server 2008 Integration Services 9. Go to the Expressions page and click in the Expressions field. Then click the ellipsis button to open the Property Expressions Editor. 10. Click in the Property column and then click the drop-down arrow and select the ToLine property. Now you can either type an expression directly in the Expression field to evaluate a property or use an Expression Builder to build an expression by clicking the ellipsis button next to the Expression field. For now, click the ellipsis button to open the Expression Builder. In the top-left pane, expand Variables, locate the User::email variable, and drag and drop it into the Expression box. Click OK to return to the Property Expression Editor. 11. Similarly, add the Subject property in the next row in the Property column and then click the ellipsis button. First type “Your enquiry dated ” + in the Figure 8-10 Adding variables to the package Chapter 8: Advanced Features of Integration Services 321 Expression box and then expand the Type Casts node in the top-right pane. Locate the (DT_WSTR, <<length>>) type cast and add it after the plus sign in the expression. Replace <<length>> with 20 and then drag the User:: enquiry_date variable from the top-left Variables node into the expression. Click the Evaluate Expression button to check whether the expression has been built properly. Refer to Figure 8-11 to see how it should look. Click OK. 12. Again, click in the next row in the Property column, then click the drop-down arrow, and this time select the MessageSource property. Click the ellipsis button to open the Expression Builder. Build the following expression in the Expression box: "Dear " + @[User::title] + @[User::lname] + "," + " Thank you for your enquiry. One of our sales representatives will be in touch with you. In the meantime, please go to our web site for more information on our products. Thank you very much for showing interest. Kind regards, Sales Support Team" Figure 8-11 Building an expression for the Subject property 322 Hands-On Microsoft SQL Server 2008 Integration Services When you’re done, click OK to close the Expression Builder. Your Property Expression Editor should now look as shown in Figure 8-12. Click OK twice to close the editor and the Mailing Opportunities Task Editor. 13. Press the 5 key to execute the package—but the package fails to execute and a package validation error appears, as shown in Figure 8-13. 14. The error message “No recipient is specified” indicates that during the validation process of execution, Integration Services found that no value was assigned to the To property. If you open the Mailing Opportunities Task Editor, you will find that the earlier assigned direct value of test@test.com no longer exists in the To field and the To field is blank. This is because when the package execution starts, validation happens before any other operation. At the validation time, Integration Services knows that the To property has to be populated from the ToLine property expression and hence ignores any direct value assigned in the field; it failed to find a value because the property expression further needed a value from User::email variable, which was not available because the package had not been executed yet. This type of error is quite normal when using Property Expressions that use variables at run time that are not yet available. To overcome this situation, Integration Services provides a facility to delay validation for such components of the package until actual run time for the component. Click the Mailing Opportunities task and press 4 to open the Properties window. Locate the DelayValidation property in the Execution section and change its value to True. Figure 8-12 Property expressions built for multiple properties Chapter 8: Advanced Features of Integration Services 323 15. Press 5 to execute the package, and this time the package will run and complete successfully. If you check your mailbox after some time, you will see the six personalized messages. Review You’ve used property expressions in this exercise to evaluate properties from static values and also variables that in turn get their value only at run time. This proves how you can dynamically modify a package at run time based on the values of other attributes in the package. You also understood the use of the DelayValidation property, which is handy to use when you are developing a package that uses lots of property expressions. One more important point to learn is that the Property Expressions can be mapped to properties exposed through the Control Flow. For instance, the Derived Column Transformation, which is a Data Flow component, can expose expressions through its parent Data Flow task. So, when you are developing a custom component, you can tag a property that can be exposed through the Data Flow task so that you can add some dynamic expressions to it. Handling Events at Package Run Time When the packages are executed, variables change values, and data flows from sources to destinations and gets transformed along the way by various tasks. These tasks and processes come across changing data and environments at run time, which sometimes results in alerts or events being raised. Integration Services provides event handlers that can act as your agents and be present at the run time configured to respond to the events and add intelligence to the packages by properly handling the events raised. As containers can have subcontainers, the packages can have event handlers. Event handlers are like subpackages waiting for the events to be raised so that they can come Figure 8-13 Validation error due to ToLine property having blank value 324 Hands-On Microsoft SQL Server 2008 Integration Services to action. These powerful tools can extend package functionality greatly when properly implemented. Some of the tasks that can be done with custom event handlers include precharging a cache, sending you an e-mail, or raising a warning when a task fails, dropping the temporary tables created during the package run, processing log files post-execution, or perhaps following an alternative workflow on an event. Event handlers can be created for the packages, Foreach Loop containers, For Loop containers, Sequence containers, and Control Flow tasks as well. Creating event handlers is like creating a work flow for a package. The Event Handlers tab is used in the SSIS Designer to create event handlers in your package; once created, the event handlers can be explored in the Package Explorer under the Event Handlers node. You can also define connection managers in the Event Handlers tab in case event handlers need to connect to any data source. When the tasks, containers, or packages raise events during run time, event handlers come into play. The raised event is captured by Integration Services and passed to an event handler for further action. If an event has no event handler defined on the container, it is passed on to the parent container. The parent container runs the event handler in response to this event; however, if the parent container doesn’t have an event handler defined, it is flagged up the hierarchy, and so on. This passing on of events up the container hierarchy applies not only at the package level, but can travel up the parent package if the package itself is run as a child package using the Execute Package task. However, before you get carried away using this functionality, you need to bear in mind that every bit of functionality comes at a cost. If you try to handle all the events at the package level, your package will have too much to manage. In addition, it can be detrimental to performance to let events travel up the ladder when they could easily be handled at the task or container level. Alternatively, if you create event handlers at every level you may end up having too many alerts and warning messages to look through. You can strategically use event handlers with different types of events handled at different levels. To avoid the extra load, which can degrade performance, you can strike a balance by handling events at various levels and at the same time filter out unwanted events from traveling up the ladder. When you design your event handling strategy for a package, you don’t need to worry about identifying where the event has been raised irrespective of where it is captured in the hierarchy, because the source of the event is retained and makes it easier for you to identify where the event has been raised. Sometimes you may need to stop the events from bubbling up to the parent container. For instance, you may be running a task within a looping structure and may want to continue looping in case an error happens in a particular iteration. This is the classic case where you may want to stop the bubbling up of error events. You can do this by setting the value of a system variable System::Propagate to false in the event handler. This will leave the event bound in the task and won’t let it propagate up to the Chapter 8: Advanced Features of Integration Services 325 parent container. The way you do this is to attach an event handler to the task and then set the System::Propagate system variable, which resides in the event handler scope, to false. While in the event handler, you can see the system variables by clicking the Show System Variables button in the variables window menu bar. Locate the Propagate variable that will have the default value of True. You can change it here to False, you can write a little script using the Script task to set its value to False at run time, or you can write an expression to evaluate its value to false at run time. The System::Propagate method will work on execution errors; however, validation errors could still occur. You could also use the MaximumErrorCount property on the task to let the package run on errors by increasing the value to a higher number. Let’s do a simple Hands-On exercise to create event handlers for one of the packages created earlier in this chapter. Hands-On: Creating Event Handlers in an SSIS Package As the title suggests, you will be creating event handlers in an Integration Services package to understand their configurations and behavior. Method In this exercise, you will add the Package1.dtsx package you created earlier in the Maintaining Data Integrity with Transactions project and then will create event handlers to respond on OnTaskFailed and OnPostExecute events. Exercise (Work with Event Handlers) In this exercise, you create event handlers for OnTaskFailed and OnPostExecute events and see how various events can be used to create complex event handlers. 1. Open BIDS and create a new Integration Services project with the following details: Name Working with Event Handlers Location C:\SSIS\Projects 2. When the blank project is created, delete the Package.dtsx in the SSIS Packages node. Then right-click the SSIS Packages node and select Add Existing Package from the context menu. 3. Configure the Add Copy Of Existing Package dialog box to add the package1 .dtsx from the file system with the C:\SSIS\Projects\Maintaining data Integrity with Transactions\Package1.dtsx as the package path. Once the package has been added, double-click the Package1.dtsx package to open it in the Designer. 326 Hands-On Microsoft SQL Server 2008 Integration Services 4. On the Designer surface, go to the Event Handlers tab and click the down arrow in the Executable field, which will open a Package Explorer window. Expand Package1 | Executables | Sequence Container 1 | Executables | Loading Vehicle (see Figure 8-14). Note that you can select a package object, a container, or a task from here. Once you’ve selected the object for which you want to build event handlers, click OK. 5. For the selected Loading Vehicle executable, you can specify the type of event handler you want to create. Click the down arrow in the Event Handler field and select OnTaskFailed; this indicates that when the Loading Vehicle task fails, this event handler will be executed. Take a moment to go through the other events available. For more details on each of these event types, refer to Microsoft SQL Server 2008 Books Online. 6. Click the Designer surface to create an OnTaskFailed event handler for the Loading Vehicle task. Now, drag and drop the Execute SQL task from the Toolbox onto the Event Handlers surface. Double-click the Execute SQL Task icon to open the editor. Figure 8-14 Selecting a container for which to create event handlers Chapter 8: Advanced Features of Integration Services 327 7. In the General page, select localhost.Campaign Connection Manager in the Connection field. Type the following SQL statement in the SQLStatement field and click OK to close this editor: INSERT INTO Vehicle (CustomerID, VIN, Series, Model) VALUES ('N501', 'UV123WX456YZ789' ,'X11—Series', 'Saloon') You’ve added an alternative workflow for the package that will run when the Loading Vehicle task fails. 8. Click in the Executable field and select Package1 executable; then click OK to close the drop-down box. 9. Click in the Event Handler field and choose the OnPostExecute event from the list. Click the Designer surface to create this event handler. 10. When the event handler is created, drag and drop the Send Mail task from the Toolbox onto the Event Handler surface. Double-click the icon of this task to open the editor. 11. Go to Mail page, click in the SmtpConnection field, and then click the arrow button and select <New Connection . . .>. Specify the name of your SMTP server in the SMTP Server field. If you’ve configured your local computer to route mails using SMTP service, you can specify localhost in this field. Click OK when you’re done. 12. Type your e-mail address in the From and To fields. In the Subject field, type Working with Event Handlers and in the MessageSource field, type The package execution has completed. Click OK to close the editor. Now you’ve added a Send Mail task to the package, which will run when the executables in the package complete. 13. Press the 5 key to execute the package. You will see that the Control Flow components in the package fail while the event handlers successfully execute. Select the appropriate executable from the Executables field and the applied event handler in the Event handler field to see its status. Press - 5 to switch back to design mode. 14. Run SQL Server Management Studio, connect to Database Engine, and run the following query in the New Query pane: Select * from [Campaign].[dbo].[Vehicle] You will see one record in the result set that has been added in the Vehicle table by the event handler created on the Loading Vehicle task. Check your e-mail and you will see that you’ve received four mails instead of one. This is because of the multiple executables raising the OnPostExecute event to kick off this Send Mail task. This is why you need to be careful to design and configure a strategy for handling events at various levels. . 318 Hands-On Microsoft SQL Server 2008 Integration Services Hands-On: Extending the Contacting Opportunities Package with Property Expressions In Chapter 4, you created an Integration Services. you’ve typed directly in the field, type test@test.com in the To field. 320 Hands-On Microsoft SQL Server 2008 Integration Services 9. Go to the Expressions page and click in the Expressions field Team" Figure 8-11 Building an expression for the Subject property 322 Hands-On Microsoft SQL Server 2008 Integration Services When you’re done, click OK to close the Expression Builder. Your