www.it-ebooks.info www.it-ebooks.info Professional Microsoft SQL Server 2014 Integration Services ® ® www.it-ebooks.info ffirs.indd 3/25/2014 8:00:35 AM www.it-ebooks.info ffirs.indd 3/25/2014 8:00:35 AM Professional Microsoft SQL Server 2014 Integration Services ® ® Brian Knight Devin Knight Jessica M Moss Mike Davis Chris Rock www.it-ebooks.info ffirs.indd 3/25/2014 8:00:36 AM Professional Microsoft® SQL Server® 2014 Integration Services Published by John Wiley & Sons, Inc 10475 Crosspoint Boulevard Indianapolis, IN 46256 www.wiley.com Copyright © 2014 by John Wiley & Sons, Inc., Indianapolis, Indiana Published simultaneously in Canada ISBN: 978-1-118-85087-9 ISBN: 978-1-118-85090-9 (ebk) ISBN: 978-1-118-85085-5 (ebk) Manufactured in the United States of America 10 No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means, Â�electronic, mechanical, photocopying, recording, scanning or otherwise, except as permitted under Sections 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, 222 Rosewood Drive, Danvers, MA 01923, (978) 750-8400, fax (978) 646-8600 Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, (201) 748-6011, fax (201) 748-6008, or online at http://www.wiley.com/go/permissions Limit of Liability/Disclaimer of Warranty: The publisher and the author make no representations or warranties with respect to the accuracy or completeness of the contents of this work and specifically disclaim all warranties, including without limitation warranties of fitness for a particular purpose No warranty may be created or extended by sales or promotional materials The advice and strategies contained herein may not be suitable for every situation This work is sold with the understanding that the publisher is not engaged in rendering legal, accounting, or other professional services If professional assistance is required, the services of a competent professional person should be sought Neither the publisher nor the author shall be liable for damages arising herefrom The fact that an organization or Web site is referred to in this work as a citation and/or a potential source of further information does not mean that the author or the publisher endorses the information the organization or Web site may provide or recommendations it may make Further, readers should be aware that Internet Web sites listed in this work may have changed or disappeared between when this work was written and when it is read For general information on our other products and services please contact our Customer Care Department within the United States at (877) 762-2974, outside the United States at (317) 572-3993 or fax (317) 572-4002 Wiley publishes in a variety of print and electronic formats and by print-on-demand Some material included with standard print versions of this book may not be included in e-books or in print-on-demand If this book refers to media such as a CD or DVD that is not included in the version you purchased, you may download this material at http://booksupport wiley.com For more information about Wiley products, visit www.wiley.com Library of Congress Control Number: 2014930406 Trademarks: Wiley, Wrox, the Wrox logo, Programmer to Programmer, and related trade dress are trademarks or registered trademarks of John Wiley & Sons, Inc and/or its affiliates, in the United States and other countries, and may not be used without written permission Microsoft and SQL Server are registered trademarks of Microsoft Corporation All other trademarks are the property of their respective owners John Wiley & Sons, Inc., is not associated with any product or vendor mentioned in this book www.it-ebooks.info ffirs.indd 3/25/2014 8:00:36 AM To my great team and work family at Pragmatic Works —Brian Knight Proverbs 3:5-6 —Devin Knight For Rich: thank you for your support —Jessica M Moss To my kids, Gabriel and Sydney — they are the light of my life —Mike Davis To the three girls in my life that mean more than anything to me, Tammy, Calista, and Callie —Chris Rock www.it-ebooks.info ffirs.indd 3/25/2014 8:00:36 AM www.it-ebooks.info ffirs.indd 3/25/2014 8:00:36 AM About the Authors Brian Knight,╇ SQL Server MVP, MCITP, MCSE, MCDBA, is the owner and founder of Pragmatic Works He is also the cofounder of BIDN.com, SQLServerCentral.com, and SQLShare.com He runs the local SQL Server users group in Jacksonville (JSSUG) Brian is a contributing columnist at several Â�technical magazines He is the author of a dozen SQL Server books Brian has spoken at conferences Â� like PASS, SQL Connections, and TechEd, SQL Saturdays, Code Camps, and many pyramid scheme motivational sessions His blog can be found at http://www.bidn.com Brian lives in Jacksonville, Florida, where he enjoys his kids and running marathons Devin Knight,╇ SQL Server MVP, is the Training Director at Pragmatic Works This is the sixth SQL Server book that he has authored Devin has spoken at past conferences like PASS Summit, PASS Business Analytics Conference, SQL Saturdays, and Code Camps and is a contributing member to the PASS Business Intelligence Virtual Chapter Making his home in Jacksonville, Florida, Devin is the Vice President of the local users’ group (JSSUG) His blog can be found at http://www.devinknightsql.com Jessica M Moss╇ is a well-known practitioner, author, and speaker of Microsoft SQL Server business intelligence and has received a Microsoft MVP award for the past years She has created numerous data warehouse and business intelligence solutions for companies in different industries and has delivered training courses on Integration Services, Reporting Services, and Analysis Services Jessica has authored technical content for multiple magazines, websites, and books, including the Wrox book Professional Microsoft SQL Server 2012 Integration Services, and has spoken internationally at conferences such as the PASS Community Summit, SharePoint Connections, and the SQLTeach International Conference As a strong proponent of developing user-to-user community relations, Jessica actively participates in local user groups and code camps in central Virginia In addition, Jessica volunteers her time to help educate people through the PASS organization Mike Davis,╇ MCTS, MCITP, is the Managing Project Lead at Pragmatic Works This book is his fifth on the subject of Business Intelligence and specifically Integration Services He has worked with SQL Server for over a decade and has led many successful Business Intelligence projects with his clients Mike is an experienced speaker and has presented at many events such as several SQL Server User Groups, Code Camps, SQL Saturday events, and the PASS Summit Mike is an active member at his local user group (JSSUG) in Jacksonville, Florida In his spare time he likes to play darts and guitar You can also find him on twitter @MikeDavisSQL, and his blog on MikeDavisSQL.com Chris Rock╇ is a software developer and program manager for Pragmatic Works He started developing software using VB6 and SQL Server 6.5 in 1998 and has been using SSIS since its inception Chris has spoken at many local SQL Saturday and Code Camp events in Florida When he’s not writing code, Chris enjoys training cats to speak English He blogs at http://rocksthoughts.com www.it-ebooks.info ffirs.indd 3/25/2014 8:00:37 AM ABOUT THE TECHNICAL EDITOR Michael McIntyrꕇ recently progressed into the role of Consulting Sales Engineer with Pragmatic Works, coordinating solutions and engagements that utilize the Microsoft BI stack for customers with a variety of BI needs Previously with Pragmatic Works, he was a BI Software Support Engineer focusing on ETL within SQL Server Integration Services This transition provided the opportunity to relocate to Salem, New Hampshire, from Jacksonville, Florida Michael spends his free time with his wife and 3-year-old daughter and enjoys keeping up with and “tinkering” with mobile technologies www.it-ebooks.info ffirs.indd 3/25/2014 8:00:37 AM SDLC (software development life cycle) – solutions SDLC (software development life cycle) (continued) MSF Agile and, 579–581 overview of, 563–565 Project Portal, 582 shelving and unshelving, 585–587 Subversion (SVN), 568–575 summary, 590 Team Foundation Server and, 573–576 Team System features, 582–584 Team System version and source control, 585–587 versioning and source code control, 567–568 waterfall approach, 566 security catalog, 806–809 legacy security, 809–811 SEH (structured exception handling), 327–330 SELECT * statements with JOINS, UNIONS and subqueries, 437 performing transformations, 436 problems with, 430–432 sorting data, 437–439 WHERE clause and, 432–433 semi-blocking transformations Data Flow design practices, 550–552 overview of, 536–538 Send Mail Task overview of, 87–88 Sequence Container overview of, 206 in single package, multiple transactions, 509–510 tasks, 41 sequence tasks, 41 serialization, XML object-based, 299–302 service-oriented architectures, Web services and, 56 set-based logic, in data extraction, 444–446 SetComponentProperty method, design-time methods, 637 SetUsageType method design-time methods, 638 Destination Component, 676–677 Source Component, 654–655 Transformation Component, 664–665 shadow tables, SQL Server Agent writing entries to, 370 Shannon, Claude, 566 shared methods, 284 SharePoint Portal Services, 582 shelving/unshelving, source control and, 587–588 Shewhart, Dr Walter, 566 shredding recordsets, Execute SQL Task, 76 signing assemblies, 284–285 Slowly Changing Dimension Transformation See SCD (Slowly Changing Dimension) Transformation SMO (SQL Management Objects) Managed Object Model based on, 717 overview of, 92 SMO administration tasks overview of, 92–93 Transfer Database Task, 93–94 Transfer Error Messages Task, 94 Transfer Job Task, 96 Transfer Logins Task, 94–95 Transfer Master Stored Procedures Task, 95 Transfer SQL Server Objects Task, 96–97 SMTP Connection Manager, 87–88 e-mail messages via, 87–88 values returned by, 641 snapshots, database creating, 446–448 Soft NUMA node, in parallel loading, 527–528 software development life cycle See SDLC (software development life cycle) Solution Explorer components in, 26–27 creating new project, 27 executing packages, 26–27, 37 Solution Framework, Microsoft, 567 solutions creating new project in, 27 868 www.it-ebooks.info bindex.indd 868 22-03-2014 20:31:51 s S S S S sort in database, data extraction and – sources creating projects and aligning with, 24–25 defined, 24 sort in database, data extraction and, 437–439 Sort Transformation asynchronous transformation outputs and, 539 as blocking transformation, 538–539 data flow example using, 166 InfoPath example, 725 loading fact table, 412 overview of, 123–125 presorting data for Data Mining Model Training Destination, 114 presorting data for Merge Join Transformation, 231–234 presorting data for Merge Transformation, 145 sorting data in SQL Server compared with, 437–439 Source adapters debugging, 680–682 installing, 679–680 as integral to Data Flow, 542 overview of, 632–633 Source Assistant accessing heterogenous data in, 454 configuring source in Data Flow with, 101 defining Data Flow for packages, 257 Source Component AcquireConnections method, 650–652 buffer objects and, 660 columns and, 659–660 ComponentMetaData properties, 649– 650 ComponentType property and, 644 Connection Managers and, 651–652 data types, 654–655 debugging source adapter, 680–682 FileUsageType property, 651–652 helper methods, 654 installing source adapter, 679–680 MapInputColumn/MapOutputColumn methods, 657 overview of source adapter, 632–633 ParseTheFileAndAddToBuffer method, 658–659 PrimeOutput method, 657–658 ProvideComponentProperties method, 648–649 ReinitializeMetaData method, 655 SetUsageType method, 654–655 types of pipeline components, 632 Validate method, 652–655 source control branching, 588 iterative development and, 567 labeling (striping) source versions, 589–590 merging, 589 MSF Agile, 581 shelving and unshelving, 587–588 Team System and, 585–587 tools for, 565 versioning and source code control, 567–568 Source type, of Script Component configuring Script Component Editor, 311–312 connecting to data sources, 314 defined, 310 sources ADO.NET Source, 111 CDC Source, 374–376 configuring destination vs., 111–112 connecting in Script Component to, 314 connecting in Script Task to, 293–302 connectivity, 719 in Data Flow, 10–11 ETL development and, 565 Excel Source, 104–105 flat files (See Flat File Source) function of, 100 Import and Export Wizard and, 17–18 mapping to destinations, 20 OLE DB (See OLE DB Source) overview of, 101 permissions (See data profiling) 869 www.it-ebooks.info bindex.indd 869 22-03-2014 20:31:52 sources – SSDT (SQL Server Data Tools) sources (continued) processing data from heterogeneous sources, 800 raw files (See Raw File Source) Transfer Database Task and, 93–94 XML Source, 59–62, 110 space padding, string functions and, 190–191 SPC (statistical process control), 566 special characters, string literals with, 183–184 Spiral, iterative development, 567 SQL (Structured Query Language) capturing multi-row results, 75–78 capturing singleton results, 74–75 creating BankBatch table, 813–814 creating BankBatchDetail table, 814–815 creating corporate ledger data, 815 creating CustomerLookup table, 813 creating ErrorDetail table, 816 creating Invoice table with, 811–812 executing batch of statements, 73–74 executing parameterized statements, 71–73 executing stored procedure, 78–80 Management Objects (See SMO (SQL Management Objects)) retrieving output parameters from stored procedures, 78–80 SQL Profiler log provider for, 622–623 package log provider for, 746 programming to log providers, 749 SQL Server Analysis Services (See SSAS (SQL Server Analysis Services)) authentication, 806 Bulk Insert Task, 66–67 CDC (See CDC (Change Data Capture)) creating central server, 790–792 Data Tools (See SSDT (SQL Server Data Tools)) deploying SQL Server 2012, 725–726 deployment options, 773 destinations, 114 editions, 14–15 Integration Services (See SSIS (SQL Server Integration Services), introduction to) log provider for, 622–623 Management Studio (See Management Studio) operation logs in SQL Server 2012, 749–751 package configuration and, 751–752 package log provider for, 746 programming to log providers, 749 project versioning in SQL Server 2012, 746 single package using native transaction in, 512–513 Transfer SQL Server Objects Task, 96–97 upgrading components for SQL Server 2014, 687 WMI Data Reader Task for gathering operational type data, 90 SQL Server Agent, 369–370, 811–812 SQL Server Business Intelligence Edition, 15 SQL Server Enterprise Edition, 14–15 SQL Server Standard Edition, 15 SQLCMD command, in parallel loading, 527 SQLMOBILE, 72 SQLStatement property, Execute SQL Task, 198–199, 512 SSAS (SQL Server Analysis Services) cube processing with, 382–383, 427–428 Data Mining Query Task, 46–47 Execute SQL Task, 44 Processing Task, 44 SSDT (SQL Server Data Tools) adding components to, 679–680 common task properties, 42 creating deployment utility, 775–776 creating first package, 25–26 creating new project, 732 data taps, 789–790 debugging components, 680 locating and opening, 24 opening Import and Export Wizard, 18 overview of, Properties windows, 28 runtime debugging, 683–686 Solution Explorer window, 26–27 870 www.it-ebooks.info bindex.indd 870 22-03-2014 20:31:52 S S S S S S S s s S s s s s SSIS (SQL Server Integration Services) – SVN (Subversion) r solutions and projects in, 24 Toolbox items, 27–28 SSIS (SQL Server Integration Services) overview, SSIS (SQL Server Integration Services), introduction to architecture of, containers, 8–9 Control Flow, 5–6 Data Flow, 9–10 destinations, 13 error handling and logging, 14 history of and what’s new, Import and Export Wizard, packages, 5–6 parameters, 14 precedence constraints, sources, 10–11 SQL Server editions and, 14–15 summary, 15–16 tasks, 6–7 transformations, 11–12 variables, 13–14 SSIS external management See external management, of SSIS SSIS interaction with external applications See external applications, interaction with SSIS Package Configuration, 794–798 SSIS Package Store, 779–781 SSIS tools See tools, SSIS ssis_admin role, 806 staged data across machines, 520–521 Data Flow design for, 554–555 Data Flow restart, 518–520 scaling out by, 518 Standardize Zip Code Transformation, 264– 265 static methods, 284 statistical process control (SPC), 566 steps (phases), in SDLC, 565 storage catalog for, 768 of files, 806 of packages, 729 stored procedures, 830–831 catalog security and, 809 controlling and managing catalog with, 768–771 in databases, 748 encapsulating common queries in, 439–440 executing, 78–80 in parallel loading, 524 retrieving output parameters, 80–82 Transfer Master Stored Procedures Task, 95 T-SQL for package execution, 781 for working with batches, 816–819 streaming assertion, 442 strings concatenation (+) operator, 181 functions, 189–191 literals, 183–184 striping (labeling) source versions, 589–590 strong names GAC (global assembly cache) and, 644–645 signing assembly with, 692–693, 697– 698 structured exception handling (SEH), 327– 330 Structured Query Language See SQL (Structured Query Language) subqueries, in data extraction, 437 SUBSTRING function, 264 Subversion (SVN) See SVN (Subversion) success values, constraints, suffixes, numeric literal, 182–183 surrogate keys, in data warehousing, 392 SVN (Subversion) configuring, 568–569 connecting project to, 571–573 downloading and installing, 568 locking files for editing and committing changes, 572–573 overview of, 567–568 testing integration with project, 573 for version control for packages, 260 walkthrough exercise, 569–571 871 www.it-ebooks.info bindex.indd 871 22-03-2014 20:31:52 Swap Inputs button, Merge Join Transformation – team builds, MSF Agile Swap Inputs button, Merge Join Transformation, 235 synchronous processes limiting in Data Flow design, 550 reducing in Data Flow design, 550 reducing in data-staging environment, 555 tasks in Control Flow, 533 synchronous transformations identifying, 535, 542 vs asynchronous, 115, 540–542 writing Script components to act as, 324–325 SynchronousInputID property, 542 System Monitor, 819 system variables, 33–36 T tab-delimited files, 105–106 tables creating with Management Studio, 731–732 in databases, 748 enabling CDC for, 370 package configuration and, 751, 817 T-SQL querying tables to set parameter values, 783–784 table-valued parameters, 444–446 Tabular Data Stream (TDS), 74 task editors Bulk Insert Task, 67 data profiling and, 51 Expressions tab, 41 FTP Task Editor, 53–54 overview of, 39 Script Task Editor, 287–288 Task Host Container, 205 task objects, 40 tasks Analysis Services, 44–47 archiving files, 52 Bulk Insert Task, 66–70 comparing Data Flow with Control Flow, 530–532 Data Flow Task, 11, 47 Data Mining Query Task, 46–47 data preparation tasks, 48 Data Profiling Task, 48–50 defined, 39 DQS (Data Quality Services), 352 ETL tasks, 6–7 evaluating, 30 Execute Package Task, 82–83 Execute Process Task, 84–85 Execute SQL Task (See Execute SQL Task) File System Task, 50–51 FTP Task, 53–55 grouping in containers, 31 logging, 622–624 looping, 41 looping and sequence tasks, 41 Message Queue Task, 86–87 opening for editing, 257 overview of, 39 precedence constraints controlling, 593 properties of, 42 RDBMS Server tasks, 66 Script Task, 42–44 Send Mail Task, 87–88 SMO administration tasks, 92–93 Task Editor, 41 Transfer Database Task, 93–94 Transfer Error Messages Task, 94 Transfer Job Task, 96 Transfer Logins Task, 94–95 Transfer Master Stored Procedures Task, 95 Transfer SQL Server Objects Task, 96–97 Web Service Task, 56–59 WMI Data Reader Task, 89–91 WMI Event Watcher Task, 91 work flow tasks, 82 working with multiple precedence constraints, 598–599 XML Task, 62–64 TDS (Tabular Data Stream), 74 team builds, MSF Agile, 581 T t T T T T T t t t T T t t 872 www.it-ebooks.info bindex.indd 872 22-03-2014 20:31:52 Team Foundation Server (Microsoft) – Transfer SQL Server Objects Task Team Foundation Server (Microsoft), 568, 573–576 team preparation, ETL and, 564–565 Team Project, setting up, 574–576 Team System See VSTS (Visual Studio Team System) Term Extraction Transformation Advanced tab, 157–158 in Data Flow, 154–158 Exclusion tab, 156–157 Term Extraction Transformation Editor, 156–157 Term Frequency and Inverse Document Frequency (TFIDF) score, 154–155 Term Lookup Transformation, 158–160 testing data flows during development with Union All Transformation, 229 database snapshot functionality, 448 expressions with Expression Builder, 179 external applications, 720 Immediate window for, 333 UI component, 712 text comma-delimited file requirement, 105–106 Derived Column for advanced data cleansing, 338–340 Term Extraction Transformation, 158–160 Term Lookup Transformation, 158–160 text files data extraction, 440–444 log provider for, 622–623, 746 MERGE operator and reading from, 451 programming to log providers, 749 TFIDF (Term Frequency and Inverse Document Frequency) score, 154–155 The Data Warehouse Toolkit (Kimball and Ross), 392 third-party solutions Change Data Capture, 368 trash destinations for testing purposes, 229 threads monitoring Data Flow execution, 544 optimizing package processing, 555–557 Tibco Rendezvous, 86 time data types, 170 functions, 192–194 Toolbox adding components to, 679–680 working with, 27–28 tools, SSIS annotations, 32 Connection Managers, 32 Control Flow, 29–31 creating first package, 25–26 Data Flow, 34 event handlers, 35–36 executing packages, 37 Import and Export Wizard and, 17–22 Management Studio, 37 overview of, 17 Package Designer, 28 Package Explorer, 36–37 parameters, 35 Properties windows, 28 Solution Explorer window, 26–27 SSDT (SQL Server Data Tools), 24–26 summary, 37 task groups, 31 Toolbox items, 27–28 variables, 33–36 TransactionOption property possible settings for, 507 in single package, multiple transactions, 509–510 in single package, single transaction, 508–509 in two packages, one transaction, 511–512 transactions, package See package transactions Transfer Database Task, 93–94 Transfer Error Messages Task, 94 Transfer Job Task, 96 Transfer Logins Task, 94–95 Transfer Master Stored Procedures Task, 95 Transfer SQL Server Objects Task, 96–97 873 www.it-ebooks.info bindex.indd 873 22-03-2014 20:31:53 Transformation Component – transformations Transformation Component building, 660 ComponentType property and, 644 configuring Script Component Editor, 311–312 debugging, 680–682 defined, 310 error handling, 669–670 input/output verification methods, 665–666 installing, 679–680 overview of, 633 PostExecute method, 671 PreExecute method, 666 PrimeOutput method, 666, 671 ProcessInput method, 666–669, 671 ProvideComponentProperties method, 661–662 ReinitializeMetaData method, 663–664 SetUsageType method, 664–665 types of pipeline components, 632 Validate method, 662–663 transformations Aggregate Transformation (See Aggregate Transformation) asynchronous outputs, 539–540 Audit Transformation, 126–127, 269 blocking, 538–539 Cache Transformation, 121, 249–251 Character Map Transformation (See Character Map Transformation) Conditional Split Transformation (See Conditional Split Transformation) Copy Column Transformation, 128–129 Data Conversion Transformation (See Data Conversion Transformation) in Data Flow, 11–12, 47 Data Flow and Control Flow comparison, 530–532 Data Flow design for correlation and integration, 552–553 Data Flow design for data cleansing, 553 Data Flow restart using, 519 Data Mining Query Transformation, 129 Data Quality Services (DQS) Cleansing Transformation, 130 Derived Column Transformation (See Derived Column Transformation) DTS (Data Transformation Services) (See DTS (Data Transformation Services)) Export Column Transformation (See Export Column Transformation) function of, 100 Fuzzy Grouping Transformation (See Fuzzy Grouping Transformation) Fuzzy Lookup Transformation (See Fuzzy Lookup Transformation) Import Column Transformation (See Import Column Transformation) InfoPath example, 723–726 Lookup Transformation (See Lookup Transformation) Merge Join Transformation (See Merge Join Transformation) Merge Transformation (See Merge Transformation) Multicast Transformation, 146–147, 558–559 non-blocking (streaming and row-based), 535–537 OLE DB Command Transformation (See OLE DB Command Transformation) overview of, 115 Percentage Sampling and Row Sampling Transformations, 149 Pivot Transformation, 149–151 Reverse String Transformation (See Reverse String Transformation) Row Count Transformation, 121–122 Row Number Transformation, 520 SCD (Slowly Changing Dimension) Transformation (See SCD (Slowly Changing Dimension) Transformation) Script Component and, 122–123 semi-blocking, 536–538 Sort Transformation (See Sort Transformation) 874 www.it-ebooks.info bindex.indd 874 22-03-2014 20:31:53 t t t T t T T trash destinations, testing data flow in development with – Union All Transformation Source and Destination adapters, 542 Standardize Zip Code Transformation, 264–265 synchronous outputs, 541–542 synchronous vs, asynchronous, 115 synchronous vs, asynchronous transformations, 316–324 Term Extraction Transformation (See Term Extraction Transformation) Term Lookup Transformation, 158–160 troubleshooting bottlenecks in Data Flow, 559 types of, 535–539 Union All Transformation (See Union All Transformation) Unpivot Transformation, 152–154 when to use during data extraction, 433–436 XSLT (Extensible Stylesheet Language Transformations), 63, 722 trash destinations, testing data flow in development with, 229 triggers, adding for Change Data Capture, 368 troubleshooting performance bottlenecks, 558–559 True Boolean expressions and, 186–187 Boolean literals and, 184 in conditional expressions, 191–192 truncation, during casting, 174 Try/Catch/Finally structure, in Visual Basic or C#, 327–330 T-SQL aggregating data, 115–116 configuring projects to use environments, 787–789 controlling environments with, 745 DMX (Data Mining Extension) to, 46 expression functions vs functions in, 179 managing security, 809 for package execution, 736–741, 781–782 querying tables to set parameter values, 783–784 setting environments with, 785 setting parameter values with, 782–783 y U UI (user interface) adding project to, 691–693 building form for, 699 column display in, 700–704 column properties, 711–712 column selection, 703–704 component-level properties, 707–709 design-time functionality and, 635 Expression Builder, 179 extending, 704 handling errors and warnings, 709–711 implementing IDtsComponentUI interface, 693–697 managing security with GUI, 807–808 modifying form constructor, 699–700 overview of, 689 runtime connections, 704–707 setting UITypeName property, 697–699 steps in building, 690 summary, 712 UITypeName property, 690, 697–699 unchanged output, SCD Transformation, 408 Ungroup command, 207 Unicode conversion issues, 171–173 string functions in, 190–191 UNION in data extraction, 437 set-based logic for extraction, 444–446 Union All Transformation adding to Lookup Transformation, 245–246 as asynchronous transformation, 542 in Data Flow, 125–126 data preparation for complex dimension table, 401 in parallel loading, 526 sending cleansed data back into main data path with, 267 testing data flow in development with, 229 testing data flow with Fuzzy Lookup, 344 testing Lookup Transformation, 237 testing Merge Join Transformation, 235 875 www.it-ebooks.info bindex.indd 875 22-03-2014 20:31:53 Unpivot Transformation, in Data Flow – VSTA (Visual Studio Tools for Applications) Unpivot Transformation, in Data Flow, 152–154 UPDATE statements, 448–442 updates capture instance tables in CDC, 372 complex dimension changes with SCD, 406–408 limitations of SCD, 409 loading simple dimension table, 396 upgrading components, to SQL Server 2014, 687 usability, UI design principles, 712 user interface See UI (user interface) user variables, 33 V Validate method design-time methods, 636 Destination Component, 674–675 Source Component, 652–655 Transformation Component, 662–663 Validate operation, XML Task, 722 validation of data using Script Component, 316–324 staged data in, 555 timeout, 747 of XML file, 63 Variable Mappings tab Foreach ADO Enumerator example, 216–217 Foreach File Enumerator, 212 VariableDispenser object, Script Task, 289 variables accessing in Script Component, 313–314 accessing in Script Task, 289–293 adding to checkpoint file, 506 data types for, 176–177 defining, 174–175 displaying list of, 33 as expressions, 195–197 Immediate window for changing value of, 333 NULL values and, 187–188 options for setting, 13–14 overview of, 166 package configuration and, 752, 771 referencing in expressions, 184–185 retrieving data from database into, 294–296 scope, 33–34 setting variable values in environments, 785–786 types of, 33 Variables collection, Script Task, 289–290 VB (Visual Basic) Hello World example, 279–280 Script Task accessing VB libraries, 41–42 selecting as scripting language, 277–278 using VSTA scripting IDE, 278–279 verification methods, Pipeline Components, 635 version control See also source control packages, 260 project versioning in SQL Server 2012, 746 source code control and, 567–568 Team System and, 585–587 views, 829–830 Visual C#, creating Windows application project, 734–736 Visual Studio 32-bit runtime executables in 64-bit mode, 456–457 64-bit issues, 815 creating Visual C# Windows application project, 734–736 source control and, 568 SSDT (SQL Server Data Tools) and, 4, 24 Team System (See VSTS (Visual Studio Team System)) Tools for Applications (See VSTA (Visual Studio Tools for Applications)) Visual Studio Team Explorer 2013, 574 VSTA (Visual Studio Tools for Applications) accessing with Script Task, 41–42 Hello World example, 279–280 Script Task and Script Component using, 276 using managed assemblies for development purposes, 282 using scripting IDE, 278–279 876 www.it-ebooks.info bindex.indd 876 22-03-2014 20:31:54 V w w w W W W W W W W W W W W VSTS (Visual Studio Team System) – XML Path Language (XPATH) VSTS (Visual Studio Team System) features, 582–584 source control and collaboration and, 564 Team Foundation Server and, 573–576 version and source control, 585–587 W warnings packages and, 268 user interface and, 709–711 watch windows script debugging using, 332 viewing debugging with, 615 waterfall methodology, in SDLC, 566 Web Service Task General tab, 57 Input tab, 57–58 Output tab, 58–59 overview of, 56 retrieving data from XML source, 59–62 Web Services, XML and, 454, 472–486 Web Services Description Language (WSDL), 56 WHERE clause, in data extraction, 432–433 Windows Authentication credentials and, 18 securing catalog and, 806 Windows clusters, 792–794 Windows Forms for displaying user interface, 693 steps in building UI (user interface), 690 Windows Management Instrumentation See WMI (Windows Management Instrumentation) Windows OSs increasing memory in 32-bit OS, 517 log providers for Windows events, 622–623, 747 Winform application, for dynamic property assignment, 731–736 WMI (Windows Management Instrumentation) Connection Managers, 88 overview of, 755 values returned by Connection Manager, 641 WMI Data Reader Task example, 756–762 explained, 755–756 overview of, 89–91 WMI Event Watcher Task example, 763–766 explained, 762–763 overview of, 91 polling a directory for file delivery, 91 work flows Execute Package Task, 82–83 Execute Process Task, 84–85 handling with Control Flow, 533 Message Queue Task, 86–87 overview of, 82 Send Mail Task, 87–88 WMI Data Reader Task, 89–91 WMI Event Watcher Task, 91 WQL queries, 89–90 wrapper classes, user interface and, 699–700 WSDL (Web Services Description Language), 56 Xâ•‚Yâ•‚Z XML (Extensible Markup Language) retrieving data from XML source, 59–62 retrieving XML-based result sets using Web service, 56 sources in Data Flow, 11 validating XML file, 64–66 Web Services and, 454, 472–486 XML Diffgram, 63 XML files log provider for, 622–623, 747 package configuration and, 751–752, 795 programming to log providers, 749 saving data to, 298–299 serializing data to, 299–302 storing log information in, 622 XML Path Language (XPATH), 63, 722 877 www.it-ebooks.info bindex.indd 877 22-03-2014 20:31:54 XML Schema Definition (XSD) – zip codes, handling dirty data XML Schema Definition (XSD), 63, 110 XML Source, in Data Flow, 110 XML Task configuring, 720–721 InfoPath document consumed by, 723–726 operation options, 63 OperationType options, 62–63 overview of, 62 validating XML file, 64–66 XMLA code, 424–425 XPATH (XML Path Language), 63, 722 XSD (XML Schema Definition), 63, 110 XSLT (Extensible Stylesheet Language Transformations), 63, 722 zip codes, handling dirty data, 264 878 www.it-ebooks.info bindex.indd 878 22-03-2014 20:31:54 www.it-ebooks.info badvert.indd 879 22-03-2014 20:47:17 Try Safari Books Online FREE for 15 days and take 15% off for up to Months* Gain unlimited subscription access to thousands of books and videos With Safari Books Online, learn without limits from thousands of technology, digital media and professional development books and videos from hundreds of leading publishers With a monthly or annual unlimited access subscription, you get: • A nytime, anywhere mobile access with Safari To Go apps for iPad, iPhone and Android •H undreds of expert-led instructional videos on today’s hottest topics • S ample code to help accelerate a wide variety of software projects • R obust organizing features including favorites, highlights, tags, notes, mash-ups and more • Rough Cuts pre-published manuscripts START YOUR FREE TRIAL TODAY! Visit: www.safaribooksonline.com/wrox *Discount applies to new Safari Library subscribers only and is valid for the first consecutive monthly billing cycles Safari Library is not available in all countries www.it-ebooks.info badvert.indd 880 22-03-2014 20:47:19 Programmer to Programmer™ Connect with Wrox Participate User Group Program Take an active role online by participating in our P2P forums @ p2p.wrox.com Become a member and take advantage of all the benefits Wrox Blox Wrox on Download short informational pieces and code to keep you up to date and out of trouble Follow @wrox on Twitter and be in the know on the latest news in the world of Wrox Join the Community Wrox on Sign up for our free monthly newsletter at newsletter.wrox.com Join the Wrox Facebook page at facebook.com/wroxpress and get updates on new books and publications as well as upcoming programmer conferences and user group events Wrox.com Browse the vast selection of Wrox titles, e-books, and blogs and find exactly what you need Contact Us We love feedback! Have a book idea? Need community support? Let us know by e-mailing wrox-partnerwithus@wrox.com www.it-ebooks.info www.it-ebooks.info badvert.indd 879 22-03-2014 20:47:17 ...www.it-ebooks.info Professional Microsoft SQL Server 2014 Integration Services ® ® www.it-ebooks.info ffirs.indd 3/25 /2014 8:00:35 AM www.it-ebooks.info ffirs.indd 3/25 /2014 8:00:35 AM Professional Microsoft SQL. .. SQL Server 2014 Integration Services ® ® Brian Knight Devin Knight Jessica M Moss Mike Davis Chris Rock www.it-ebooks.info ffirs.indd 3/25 /2014 8:00:36 AM Professional Microsoft SQL Server 2014. .. —Chris Rock www.it-ebooks.info ffirs.indd 12 3/25 /2014 8:00:37 AM Contents Introduction xxvii Chapter 1: Welcome to SQL Server Integration Services SQL Server SSIS Historical Overview What’s New in