118 Hands-On Microsoft SQL Server 2008 Integration Services 11. In your package, you will store the result set to a variable and then use this variable in the Foreach Loop Container to let it iterate over the resultant records. The result set records will be held in a variable of the Object data type, as they can hold a dataset and will be read by the Foreach Loop Container one by one and fed into the defined tasks for further processing. 12. From the left pane of the Execute SQL Task Editor, click the Result Set page. 13. In the Result Set page, click Add. NewResultName will appear under the Result Name column. Delete it and type 0 (zero) in this field. Click in the field under the Variable Name column. Click the drop-down arrow to select <New variable…>. In the Add Variable window, in the Name field type Opportunities; leave the Namespace field set at User, and in the Value type field, select Object, as shown in Figure 4-4. Variables are case-sensitive, so when you type to add or to select a variable, pay attention to the case of variable. Click OK to return to the Task Editor window. You will see User::Opportunities added as a user variable under the Variable Name column. 14. Click OK to close Execute SQL Task Editor that will select the prospects at run time. Figure 4-4 Creating a user variable Chapter 4: Integration Services Control Flow Containers 119 Exercise (Adding Foreach Loop Container) Here, you will configure the Foreach Loop Container using the Foreach ADO enumerator that allows you to enumerate over records from a record set as provided by Opportunities variable. 15. From the Toolbox, drag and drop a Foreach Loop Container below the Prospects task. Now click on the Prospects and drag the green arrow appearing below it onto the Foreach Loop Container. This is the precedence control letting the Foreach Loop Container to proceed if the previous Prospects task has completed successfully. 16. Double-click the icon of the Foreach Loop Container to open the editor. In the General page, type in the following: Name Iterating October Opportunities Description This will iterate over the record set obtained from October Opportunities. 17. Click the Collection in the left pane to move on to the next page. Click in the Enumerator field to see the drop-down arrow, click it, and choose Foreach ADO Enumerator from the drop-down list. Note that Enumerator configuration area changes to suit the enumerator you choose. 18. In the Enumerator Configuration box, select User::Opportunities in the ADO Object Source Variable field, as shown in Figure 4-5. By specifying User::Opportunities here, you are telling the Foreach Loop Container to get the records stored in the variable. This is a good example of tasks communicating with each other—here, the Prospects task populates variable with values for the downstream components to use those values. 19. Since only one dataset will be generated in your example, select the Rows in the first table radio button, as shown in Figure 4-5. Click OK to close Foreach Loop Editor window. Exercise (Adding Send Mail Task and Executing the Package) Now configure the Send Mail Task so that you can send e-mails for each of the selected records. 20. From the Toolbox, drag and drop the Send Mail task inside the Foreach Loop Container. 21. Double-click the Send Mail Task icon to open the editor. In the General page, type in the following details: Name Mailing Opportunities Description It will send a mail for each of the records iterated by the Foreach Loop Container. 120 Hands-On Microsoft SQL Server 2008 Integration Services 22. Go to the Mail page. From the drop-down list in SmtpConnection field, select My SMTP Server. 23. To keep your first package simple, you will be sending e-mails to yourself in this exercise. Type your e-mail address in the From and To fields and type Your Enquiry in the Subject field. You can actually read e-mail addresses from a database table for these two fields using property expressions, which you will study in Chapter 8. 24. In the MessageSourceType, leave Direct Input selected, as shown in the Figure 4-6. This field provides you three options: you can type your message directly in the task or get your message from a file or a variable. Figure 4-5 Configuring the Foreach Loop using Foreach ADO Enumerator Chapter 4: Integration Services Control Flow Containers 121 25. Click in the MessageSource field and then click the ellipsis button to start typing your message in the Message Source window. Here is a simple message you can use for this example. Type in the following message and click OK. Dear Enquirer, ank you for your enquiry. One of our sales representatives will be in touch with you. In the mean time, please go to our web site for more information on our products. ank you very much for showing interest. Kind regards, Sales Support Team Figure 4-6 Send Mail task configurations 122 Hands-On Microsoft SQL Server 2008 Integration Services 26. Leave Priority set to Normal, though High, Normal, or Low options are available as well. Leave the Attachments field blank. Click OK to close the dialog box. By now, your package should look like the one shown in the Figure 4-7. 27. Press 5 on the keyboard to start debugging. You will see the tasks changing colors. First, Prospects (Execute SQL Task) will turn yellow and then to green to indicate that the task has completed successfully. Then Iterating October Opportunities (Foreach Loop Container) and Mailing Opportunities (Send Mail Task) will turn from yellow to green quickly in turn for as many times as the number of records in the collection. This is a good visual experience to realize that the Foreach Loop Container iterates over each item in the collection and processes the logic you design for it. Check your Inbox to see the e-mails you have sent. If your mailing system has relay restrictions, you may have to adjust them to complete the execution of the package successfully. 28. Choose File | Save All and then choose File | Close Project to close the Contacting Opportunities project. Finally, exit from BIDS and it’s time for a coffee break, but only after reading the review. Figure 4-7 Mailing Opportunities package Chapter 4: Integration Services Control Flow Containers 123 Review In this exercise, you created a workflow for selecting prospects and sent e-mails using the Execute SQL task, Foreach Loop Container, and Send Mail task. You have now seen most of the configuration settings for these tasks, but there are some interesting things that can still be done in this package—such as picking up e-mail addresses of the recipients from a data set; using a combination of title, first name, or last name in the addressing label; and specifying the reference by mentioning the enquiry date in the subject field. The good news is that all these can be done in Integration Services using property expressions. Property expressions help you dynamically update properties of a task such as the To address in the Send Mail Task. Property expressions are covered in more detail in Chapter 8, where we’ll work with the advanced features of SSIS and extend this package to send personalized mails. For Loop Container The For Loop Container is easier than the Foreach Loop Container to work with. If you have worked with programming languages, you must be aware of popular Do While statements. Generally the syntax of these statements defines a numeric variable, with a starting value, the incrementing step after each iteration, and the ending value, along with statements to be executed repeatedly till the end point is reached. The For Loop Container provides a similar functionality of repeating a task or group of tasks for a defined number of times. The For Loop Container iterates over and over while evaluating an expression with each repeat until the expression evaluates to False. To understand this, consider a Send Mail Task configured inside the For Loop Container. If you set to start the looping with counter value equal to zero, the increment step to one, and the maximum allowed value that can be reached to less than five, then the For Loop Container will iterate the Send Mail task five times, which in turn will send out five messages. The package starts with counter value equal to zero to be increased by one with each iteration. On each iteration, the current value of the counter is checked against the maximum possible value of five. If the current counter value is less than five, it will let the Send Mail task run and will increase the counter value by one before it evaluates again. When the counter value reaches five, it determines that the value has reached the maximum allowed and will not proceed beyond this point; it stops the execution. The evaluation expression at this point returns a value of False. To do these iterations, the For Loop Container uses the following elements: InitExpression c An optional initialization expression used to assign initial values to the loop counter. 124 Hands-On Microsoft SQL Server 2008 Integration Services EvalExpression c A required evaluation expression evaluated at the onset of each iteration to decide whether the loop should continue or stop. is is a Boolean evaluation, and when the expression evaluates to FALSE, the process exits looping. AssignExpression c An optional iteration expression that increments or decrements the loop counter. You must provide an evaluation condition, but initialization and assignment expressions are optional. A For Loop Container can have only one evaluation expression and runs all the control flow tasks included in it the same number of times. You can use either literals or expressions while specifying an evaluation condition, an initialization counter value, or an assignment step value. The property expressions can include variables, which can be dynamically updated at run time. You can use this feature quite effectively to change the number of iterations each time the package is run, depending upon the values of variables. Let’s compare the For Loop Container with Foreach Loop Container task, which you studied in the preceding section. The For Loop Container provides a looping functionality that conditionally iterates a control flow defined within the container, whereas the Foreach Loop Container provides a looping construct that enumerates files and objects in the control flow of a package—that is, it executes the control flow defined within the container for each item in the collection. Neither container provides any functionality; rather they provide only a structure in which you can build a repeatable control flow. Both containers can include a control flow with multiple tasks in addition to other containers. You can in fact build nested loops and implement complex looping in your packages using these containers. Hands-On: Deleting Data Month by Month After Archiving In your data warehouse, you have to keep data for the last five years to meet business needs. So, at the beginning of every year, you archive data to tape that is older than last five years and then delete from the data warehouse tables. However, your DBA does not allow you to delete all one year worth of rows in single SQL statement because it swells the transaction log too much and requires lot of free space. So, you decided to delete data month by month and, to achieve this further, decided to use For Loop Container. Method In this exercise, you will delete data month by month without going into the other details of archiving and managing a server. The Campaign database contains a Sales table that you will use to study the For Loop Container and then use to add iterations in your package. The Sales table in the Campaign database is provided with the Chapter 4: Integration Services Control Flow Containers 125 software for this book. So, before you start, make sure you have attached the Campaign database as mentioned in the Appendix. Our Sales table has 24 rows for year 2005, with 2 rows belonging to each month. This table has been simplified for this exercise and the rows for other years have not been included for the sake of clarity. Run the following query against the Sales table in SQL Server Management Studio to see the record set: SELECT * FROM [dbo].[Sales] In this exercise you will delete all these rows using For Loop Container within the package you develop. You will iterate over the data 12 times, that is, once for each month, And in each iteration you will delete rows attached to that month, in our case 2 rows each time. Exercise (Add a For Loop Container) As a first step let us add a looping functionality in a new package to iterate for 12 times. 1. Start BIDS. Choose File | New | Project. Create a new Integration Services project specifying the following: Name Deleting data month by month Location C:\SSIS\Projects Click OK to create a new project. 2. Rename the Package.dtsx to Deleting data month by month.dtsx by right- clicking Package.dtsx in Solution Explorer. 3. Right-click in the Connection Managers area and choose New OLE DB Connection from the context menu. Select the localhost.Campaign OLE DB connection manager listed in the Data Connections area from the Configure OLE DB Connection Manager dialog box. Click OK to add this connection manager. 4. Right-click anywhere on the designer surface and choose Variables from the context menu. In the Variables window, add a variable with the following details: Name MonthCounter Scope Deleting data month by month Data Type Int32 Value 0 You will set the value equal to 0 here, which will be changed during run time. Click OK to create this variable. 126 Hands-On Microsoft SQL Server 2008 Integration Services 5. Drop the For Loop Container from the Toolbox on to the designer surface. 6. Right-click the For Loop Container and choose Edit from the context menu. Type the following in the For Loop page of the For Loop Container: Name Looping for deleting monthly data Description This container adds a looping structure for deleting monthly data. InitExpression @MonthCounter = 1 EvalExpression @ MonthCounter <= 12 AssignExpression @ MonthCounter = @ MonthCounter + 1 By setting these configurations, you are actually telling the For Loop Container to initialize the MonthCounter variable with a value equal to 1 and increase it by one after each iteration. The MonthCounter variable will be evaluated against the number of month in a year, as you are deleting one year’s worth of data, to decide whether to continue to iterate the For Loop Container. Click OK to close the For Loop Container. The For Loop Container configurations are shown in Figure 4-8. Figure 4-8 Configuration settings of the For Loop Container Chapter 4: Integration Services Control Flow Containers 127 Exercise (Delete Monthly Records) Now you will add an Execute SQL task within the For Loop Container for applying the SQL script to delete records for the month specified in the form of a parameter by For Loop Container. After completing the development work, you will run the package and will see how the rows are deleted using breakpoints. 7. Drag and drop the Execute SQL task from the Toolbox inside the Looping for deleting monthly data For Loop Container. Double-click the new Execute SQL task to open the Execute SQL Task Editor window. Type the following in the General page of the Execute SQL task: Name Delete monthly records Description This task deletes monthly records from the table. 8. Leave all the other options as they are, choose the localhost.Campaign connection manager from the drop-down list in the Connection field as you have done in earlier tasks, and type the following query in the SQLStatement field (by now you know how to do that): DELETE FROM [dbo].[Sales] WHERE YEAR(SalesOrderDate) = 2005 AND MONTH(SalesOrderDate) = ? This SQL query deletes all the records belonging to the month specified by the MonthCounter parameter in a single iteration of the For Loop Container. 9. Go to Parameter Mapping page and click Add. Select User::MonthCounter variable as shown in Figure 4-9. This will pass the value of MonthCounter variable from the For Loop Container to the Execute SQL Task. As the For Loop Container iterates and increases the value of MonthCounter variable one by one, the Execute SQL Task deletes data from the Sales table month by month. Close the Execute SQL Task Editor by clicking OK. 10. Before we execute the package, let’s add some breakpoints to the package to see what happens at run time. Right-click the Delete monthly records Execute SQL task and choose the Edit Breakpoints option from the context menu. You will learn a lot more about the breakpoints later in Chapter 8, but for now simply remember that they help you to see the run-time status of a package by pausing its execution and letting you to see the variable values and such. Choose the very first break condition that breaks the package OnPreExecute event (see Figure 4-10). Setting this breakpoint will pause the execution before each time the Execute SQL Task runs. Click OK to set the breakpoint. . Foreach Loop Container. 120 Hands-On Microsoft SQL Server 2008 Integration Services 22. Go to the Mail page. From the drop-down list in SmtpConnection field, select My SMTP Server. 23. To keep your. 118 Hands-On Microsoft SQL Server 2008 Integration Services 11. In your package, you will store the result set to a variable. regards, Sales Support Team Figure 4-6 Send Mail task configurations 122 Hands-On Microsoft SQL Server 2008 Integration Services 26. Leave Priority set to Normal, though High, Normal, or Low options