Hands-On Microsoft SQL Server 2008 Integration Services part 14 pptx

108 Hands-On Microsoft SQL Server 2008 Integration Services constraints are also enhanced beyond DTS 2000, as they can include expressions to consider along with execution results of the precedence executable. Finally, the most powerful feature of Integration Services is its ability to evaluate the properties dynamically at run time. This is achieved with Property Expressions. Integration Services Expression language is rich in syntax and can be used to create complex expressions using variables, columns, functions, and operators. After having learned about connection managers, system- and user-defined variables, precedence constraints, and the Expressions language, it’s time for you to start building packages using these components. Integration Services Control Flow Containers Chapter 4 In This Chapter c Integration Services Package c Foreach Loop Container c For Loop Container c Sequence Container c Task Host Container c Summary 110 Hands-On Microsoft SQL Server 2008 Integration Services I ntegration Services (SSIS) uses containers to host functional objects that build packages. An SSIS container is an object that represents a unit of work and contains the tasks and precedence constraints. A container can contain other containers and tasks. So, while you are creating an SSIS package, you will be adding and configuring tasks inside a container that will provide necessary services to your tasks to accomplish the work that needs to be done. Functionally, SSIS containers provide repeating logic, the ability to enumerate over a set of items, and the ability to group tasks and child-containers to create an independent unit of work that can communicate with a parent container. In this chapter, you will study the following five Integration Services containers: Integration Services package c Foreach Loop Container c For Loop Container c Sequence Container c Task Host Container c You will study another container available in SSIS, the Event Handler, in Chapter 8. Its primary function is to support a container based on the events happening at run time. Integration Services Package The Integration Services package is an XML document that consists of executables, control flow containers, tasks, data flow transformations, connection strings, variables, event handlers, and other components that all work together. The package object is the most important object and sits at the top of the hierarchy in the Integration Services object model architecture. This means that all other containers and tasks are contained and executed within the context of a package. You can build a package using GUI tools, such as the Import and Export Wizard or Business Intelligence Development Studio (BIDS), or you can build it programmatically using an IDE or even Windows Notepad. However, you may find it easier to use GUI tools, which let you build a complex package in a short time. After a package is created in BIDS, you can save it on the file system as an XML file or to SQL Server in the sysssispackages table. After saving the package, you can connect to Integration Services using SQL Server Management Studio to manage your packages. When you created your first project earlier in the book, you also created a blank package inside the solution. In the same exercise, you learned about various components and parts of a package. You can use other objects such as event handlers, variables, Chapter 4: Integration Services Control Flow Containers 111 package configurations, breakpoints, and log providers that can greatly enhance package functionality and give you complete control over what you want to achieve and how you want to achieve it. By now, you have learned that every package will have a control flow and most will include a data flow. SSIS has separated the control flow and data flow in a package. This separation makes it easy for you to design, develop, debug, and maintain complex packages. When you create a package in the BIDS, you see four tabs on the SSIS designer surface: Control Flow tab c Helps you create package workflow using control flow components such as tasks, connection managers, and precedence constraints. Data Flow tab c Helps you create a data flow using data flow components such as transformations, source adapters, and destination adapters. Event c Handlers tab Helps you create a control flow that runs in response to an event raised by a task or a container inside a package. Package c Explorer tab Shows you a hierarchical view of the elements used in the package. Although an SSIS package sits at the top of the object model hierarchy and all the tasks and containers are executed within it, a package can be included as a child in other packages. Think of the SSIS package as a container that can contain a child-container— likewise, a package can contain a child-package. SSIS includes the Execute Package task that acts as a wrapper for a package, making it a child package. You can include this task (and hence the child package) in the control flow of a parent package. This functionality makes SSIS modular in design, which means that you can create child-packages as modules, which can then be combined to form a more complex enterprise-wide package. This functionality also reduces complexity in design, making it easier for you to debug and maintain smaller packages. You will learn more about the Execute Package task in the next chapter, where the Control Flow tasks are discussed. Foreach Loop Container While designing the workflow of an SSIS package, you may want to perform an operation on the members of a collection. For example, you may want to read the names of the files in a folder and set the values of your variables using those names, or you may want to perform logic on each of the records in a table. The Foreach Loop Container in SSIS provides this functionality by bringing the result set into the workflow; this lets you perform interesting work based on the data in the result set. 112 Hands-On Microsoft SQL Server 2008 Integration Services With a Foreach Loop Container you can create a repeating control flow in a package. This concept has been borrowed from programming languages. SSIS has made repeating work flow logic for each member of a collection a lot easier by providing the Foreach Loop Container that uses various enumerators to enumerate over different sets of collections. Here are some examples for which a Foreach Loop Container can be used: Notifying your customers by sending out e-mails when an insurance policy is due c for renewal Processing prospects that have made enquiries during the last month c Automatically sending out welcome packs to new clients c Processing dunning letters whenever a payment is overdue by a certain number c of days The following table lists the enumerators that SSIS provides: Enumerator Description Foreach ADO Enumerates rows in tables such as an ADO record set. Foreach ADO.NET Schema Rowset Enumerates the schema information about a data source. For example, enumerate all the tables in an SQL Server database. Foreach File Enumerates files in a folder. Can select files from the subfolders if you select Traverse Subfolders option while configuring this enumerator. Foreach From Variable Enumerates the enumerable objects that a variable contains. For example, you may enumerate values of an array. Foreach Item Enumerates the items in a collection such as list of files in an Excel sheet. Foreach Nodelist Enumerates the result set of an XPath expression. For example, the expression /Products/vehicle[@Bodystyle='saloon'] Foreach SMO Enumerates SQL Management Objects (SMO). For example, tables or user collections in an SQL Server database. The Foreach Loop Container repeats the work flow tasks built into the container for each member of the collection defined by the enumerator. For example, if you want to send out an e-mail to customers on the basis of a table that has 100 rows, the Foreach loop will iterate over the Send Mail task 100 times by selecting to a record, sending out the e-mail for that record, and then moving on to the next record. You can implement any business logic that requires repetition using child containers and tasks as a single unit of work within a Foreach Loop Container. Once you have Chapter 4: Integration Services Control Flow Containers 113 chosen the enumerator to implement the repetition logic, the Foreach Loop Container will iterate multiple times over the business logic you want to repeat for the set of values defined by the enumerator. Why you would want to do this? Suppose you have created a contacts table, and you want to send an e-mail to each of the contacts listed in the table. You can use Send Mail task provided in SSIS to send these e-mails; however, the challenge, to fill in the e-mail addresses one by one in the Send Mail task dynamically to be able to send an e-mail to each of the contacts, has to be overcome. This is where a Foreach loop can help: it enumerates each contact, passes the e-mail address of each using a variable to the Send Mail task, waits for the task to send the e-mail, and then moves on to the next contact until all the contacts have been sent e-mail. This functionality, which is necessary for customer relationship management (CRM) projects these days, used to mean that custom code had to be written. Now, the Foreach Loop Container makes it easy. In the following Hands-On exercise, you will configure a Foreach Loop Container to send out e-mails to selected contacts. Hands-On: Contacting Opportunities You are tasked to send out first contact e-mail to all the persons who have made enquiries about the products your company makes in the month of October 2009. Method In this exercise, you will use the prospect table of the Campaign database to select persons who have made enquiries in the month of October 2009. By now, you must attach the Campaign database provided with the software for this book to your SQL Server 2008 database server; if you have not attached it yet, do that first so that you can complete this exercise. Attaching the Campaign database to your SQL Server 2008 is explained in the Appendix. For this exercise, you will be performing the following steps: Add an OLE DB connection manager for connecting to the Campaign database c so that you can get data from the prospects table. Add another connection manager for SMTP server for sending out e-mails. Add an Execute SQL task in the control flow for selecting the prospects. c Add a Foreach Loop Container to include the repetition logic. c Within the Foreach Loop Container, add a Send Mail task for sending out c e-mails. e Foreach Loop Container will iterate on the selected records and execute Send Mail task for each record to send out e-mail. 114 Hands-On Microsoft SQL Server 2008 Integration Services Exercise (Adding the Connection Managers) In the first part of this exercise, you will create a new project and add connection managers to connect to the Campaign database and Simple Mail Transfer Protocol (SMTP) server. These connection managers will allow your package to interface with external objects to access data and send e-mails. 1. Start Business Intelligence Development Studio. Choose File | New | Project to create a new project. In the New Project window, click OK after specifying the following: Template Integration Services Project Name Contacting Opportunities Location C:\SSIS\Projects 2. In the Solution Explorer window, right-click Package.dtsx and choose Rename from the context menu. Rename the package Mailing Opportunities.dtsx and click OK in the pop-up confirmation dialog box. Choose File | Save All to save the newly created package. 3. Right-click in the Connection Managers area and choose New OLE DB Connection from the context menu. In the left pane of the Configure OLE DB Connection Manager dialog box (Figure 4-1), under Data Connections, you will see a list of connections that were created earlier. Choose localhost.Campaign from the list and click OK to close the window. The package gets connected to the Campaign database and the localhost.Campaign connection manager appears under the Connection Managers tab. If you don’t find the localhost.Campaign connection manager listed under Data Connections, refer back to the “Using System Variables to Create Custom Logs” Hands-On exercise in Chapter 3 to learn how to create this connection manager. 4. Right-click in the Connection Managers area and choose New Connection from the context menu. In the Add SSIS Connection Manager window, select the SMTP connection manager type and then click Add. 5. In the SMTP Connection Manager Editor window Name field, type My SMTP Server. In the Description field, type Connection to my SMTP Server. In the SMTP Server field, type in the name of your SMTP server as shown in Figure 4-2. In the figure, you can see that I’m using localhost is used to send e-mails. You can use your corporate SMTP server or install SMTP service component of Internet Information Services (IIS) on the local machine and forward all the messages to the gateway SMTP server. You can install IIS from the Control Panel by clicking the Add or Remove Programs icon and then Add/Remove Windows Components. Chapter 4: Integration Services Control Flow Containers 115 If you are using a Microsoft Exchange Server that is configured not to accept unauthenticated connections, select the Use Windows Authentication check box. You can also choose to encrypt communication using Secure Socket Layer (SSL) when sending e-mails from this server. Whichever way you choose to send e-mails, make sure your SMTP server is prohibited from relaying e-mails for external servers and hence prevents any potential attacks from mass mailers. Click OK to return to the SSIS Designer. Exercise (Configuring Execute SQL Task) In the second part of this exercise, you will configure an Execute SQL Task to select prospects that have enquired in the month of October 2009. While configuring this task, Figure 4-1 Adding the OLE DB Connection Manager 116 Hands-On Microsoft SQL Server 2008 Integration Services you will type an SQL statement to extract the required records and will add a variable to store those extracted records. 6. In the Control Flow tab of the designer, go to the Toolbox window and drag and drop an Execute SQL Task on the control flow surface. Double-click the Execute SQL Task icon to open the Execute SQL Task Editor. Type in the following values in the General tab of the editor window: Name Prospects Description SQL to choose persons who inquired in the month of October 2009. As a best practice you should always add a description to the SSIS tasks and components, as it is a convenient way to self-document the package. In the complex packages where you have multiple similar tasks, the description lets you quickly understand what each component is doing. You don’t even have to open the task—just hover your mouse on the task in the Control Flow surface and the description will be shown to you as a task hint. 7. Click in the ResultSet field and choose Full Result Set from the drop-down list. The choice of the option in this box depends on the result set expected from your SQL statement. In this case, the select statement is going to return more than one row, so you will choose a Full Result Set option here. These result set options are explained in detail in Chapter 5. 8. Leave the ConnectionType set to OLE DB and from the drop-down list in the Connection field choose localhost.Campaign connection manager. Figure 4-2 SMTP connection manager configurations Chapter 4: Integration Services Control Flow Containers 117 9. In the SQLSourceType field, leave Direct Input selected. You can see the list of available options, however, by clicking in the SQLSourceType field. You can either input your SQL statement directly in the task or input from a file or read from a variable. In this exercise, you will be entering your SQL in the task directly. 10. Click in the SQLStatement field and an ellipsis button appears in the right corner of the field. Click the ellipsis button and you will see the Enter SQL Query window. Type in the following SQL query and click OK to return to the Execute SQL Task Editor. SELECT TITLE, FIRSTNAME, LASTNAME, EMAIL, ENQUIRYDATE FROM PROSPECTS WHERE ENQUIRYDATE BETWEEN '2009/10/01' AND '2009/10/31' The Execute SQL Task Editor should look as shown in Figure 4-3. Figure 4-3 Configuring the Execute SQL task . for each record to send out e-mail. 114 Hands-On Microsoft SQL Server 2008 Integration Services Exercise (Adding the Connection Managers) In the first part of this exercise, you will create. Container c Sequence Container c Task Host Container c Summary 110 Hands-On Microsoft SQL Server 2008 Integration Services I ntegration Services (SSIS) uses containers to host functional objects that. task, Figure 4-1 Adding the OLE DB Connection Manager 116 Hands-On Microsoft SQL Server 2008 Integration Services you will type an SQL statement to extract the required records and will add