this print for content only—size & color not accurate spine = 0.998" 528 page count BOOKS FOR PROFESSIONALS BY PROFESSIONALS ® US $49.99 Shelve in Microsoft Servers User level: Intermediate–Advanced Dunphy, Metwally BizTalk 2006 THE EXPERT’S VOICE ® George Dunphy and Ahmed Metwally Foreword by Marty Wasznicky, Program Manager, Customer Projects and Services–Microsoft Corporation Pro BizTalk 2006 CYAN MAGENTA YELLOW BLACK PANTONE 123 CV ISBN 1-59059-699-4 9 781590 596999 54999 6 89253 59699 9 Solutions to real-world issues for BizTalk professionals www.apress.com SOURCE CODE ONLINE Companion eBook See last page for details on $10 eBook version forums.apress.com FOR PROFESSIONALS BY PROFESSIONALS ™ Join online discussions: THE APRESS ROADMAP Pro BizTalk 2006 BizTalk 2006 Recipes Pro C# 2005 and the .NET 2.0 Platform, Third Edition Companion eBook Available Pro Pro BizTalk 2006 Dear Reader, Microsoft BizTalk Server is a toolkit enabling you to link software systems together in order to implement business processes. You can link completely different systems within a single enterprise or between enterprises. You can monitor and control messages moving between systems, and you can specify their interactions. Business workflow changes are easily made at the BizTalk level, often without modifying underlying system code. We began writing Pro BizTalk 2006 after experiencing the pains and triumphs of architecting large BizTalk Server implementations. We have worked with teams that have implemented BizTalk smoothly without issue and conversely have been asked to rescue other projects that were on the verge of collapse. Over time, we began to understand why so many software architects express disappointment in the level of “public wisdom” that is available for the product. Our goal is to change all that. The art of architecting a BizTalk solution comes from having the experience, or the guidance, to know which tool within the toolbox is best suited to a given task you need to complete. The goal of this book is to share some of the lessons learned and knowledge we have obtained while working with the product. Our hope is that through this book we can give you the skills and knowledge to implement BizTalk Server 2006 successfully within your organization. The book focuses on the tools that are most common within a BizTalk project—messaging, pipelines, pipeline components, orchestrations, and the Business Rule Engine. The book also gives guidance on BizTalk Server performance tuning, administra- tion, high availability, and disaster recovery—something that no book has attempted to do before. In our experience, these are areas of the product that are often overlooked. Our sincere hope is that this book will give you confidence that the solution you implement is well designed by your architects, well built by developers, and well supported by your system administrators. George Dunphy, Ahmed Metwally, Rob Cameron, Carmai Constant, and Sergei Moukhnitski George Dunphy and Ahmed Metwally Pro BizTalk 2006 6994fmfinal.qxd 10/2/06 12:19 AM Page i P ro BizTalk 2006 Copyright © 2006 by George Dunphy, Ahmed Metwally All rights reserved. No part of this work may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or by any information storage or retrieval system, without the prior written permission of the copyright owner and the publisher. ISBN-13 (pbk): 978-1-59059-699-9 ISBN-10 (pbk): 1-59059-699-4 Printed and bound in the United States of America 9 8 7 6 5 4 3 2 1 Trademarked names may appear in this book. Rather than use a trademark symbol with every occurrence of a trademarked name, we use the names only in an editorial fashion and to the benefit of the trademark owner, with no intention of infringement of the trademark. Lead Editor: Jonathan Gennick Technical Reviewer: Stephen Kaufman, David Stucki Editorial Board: Steve Anglin, Ewan Buckingham, Gary Cornell, Jason Gilmore, Jonathan Gennick, Jonathan Hassell, James Huddleston, Chris Mills, Matthew Moodie, Dominic Shakeshaft, Jim Sumser, Keir Thomas, Matt Wade Project Manager: Beth Christmas Copy Edit Manager: Nicole Flores Copy Editor: Ami Knox Assistant Production Director: Kari Brooks-Copony Senior Production Editor: Laura Cheu Compositor: Linda Weidemann, Wolf Creek Press Proofreader: April Eddy Indexer: John Collin Artist: Kinetic Publishing Services, LLC Cover Designer: Kurt Krames Manufacturing Director: Tom Debolski Distributed to the book trade worldwide by Springer-Verlag New York, Inc., 233 Spring Street, 6th Floor, New York, NY 10013. Phone 1-800-SPRINGER, fax 201-348-4505, e-mail orders-ny@springer-sbm.com, or visit http://www.springeronline.com. For information on translations, please contact Apress directly at 2560 Ninth Street, Suite 219, Berkeley, CA 94710. Phone 510-549-5930, fax 510-549-5939, e-mail info@apress.com, or visit http://www.apress.com. The information in this book is distributed on an “as is” basis, without warranty. Although every precaution has been taken in the preparation of this work, neither the author(s) nor Apress shall have any liability to any person or entity with respect to any loss or damage caused or alleged to be caused directly or indirectly by the information contained in this work. Sections in this book borrow from Microsoft documentation and white papers and are reprinted with permission from Microsoft Corporation. The source code for this book is available to readers at http://www.apress.com in the Source Code/ Download section. You will need to answer questions pertaining to this book in order to successfully download the code. 6994fmfinal.qxd 10/2/06 12:19 AM Page ii For my wife and best friend, Keressa—the one who keeps me going; to my son, Ben—master of LEGO blocks and future hockey-playing, race-car-driving, rock star astronaut; to my parents, who taught me that you can do anything you want in life if you strive to be the best. —George Dunphy Dedicated to my family: my late grandmother, who taught and raised me; dad, my role model in Life and my good friend; mom, who always drove me hard to be one of the best; my wife and inspiration, Lamia, who supports and helps me every step of the way. —Ahmed Metwally To my beautiful wife, Ally, and daughters, Amanda and Anna, who bring so much joy to my life. —Rob Cameron 6994fmfinal.qxd 10/2/06 12:19 AM Page iii 6994fmfinal.qxd 10/2/06 12:19 AM Page iv Contents at a Glance Foreword . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv About the Authors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xix About the Technical Reviewers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxi Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxiii Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxv PART 1 ■■■ Readme.1st ■CHAPTER 1 BizTalk in the Enterprise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 ■CHAPTER 2 Starting a New BizTalk Project . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 PART 2 ■■■ BizTalk Revealed ■CHAPTER 3 Thinking Inside the Box . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 ■CHAPTER 4 Pipelining and Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 ■CHAPTER 5 Pipeline Component Best Practices and Examples . . . . . . . . . . . . . 145 ■CHAPTER 6 BizT alk Design Patterns and Practices . . . . . . . . . . . . . . . . . . . . . . . . 197 ■CHAPTER 7 What the Maestro Needs to Know: Advanced Orchestration Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251 ■CHAPTER 8 Playing By the Rules? Use the Business Rule Engine. . . . . . . . . . . 283 PART 3 ■■■ You Mean You Aren’t a Developer? ■CHAPTER 9 BizT alk Ser ver 2006 Oper ations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323 ■CHAPTER 10 Deplo ying and Managing BizT alk Applications . . . . . . . . . . . . . . . . . 377 ■CHAPTER 11 To Tune or Not to Tune? Nobody Should Ask That Question . . . 421 ■APPENDIX Bibliography. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 457 ■INDEX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 459 v 6994fmfinal.qxd 10/2/06 12:19 AM Page v 6994fmfinal.qxd 10/2/06 12:19 AM Page vi Contents Foreword . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv About the Authors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xix About the Technical Reviewers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxi Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxiii Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxv PART 1 ■ ■ ■ Readme.1st ■CHAPTER 1 BizTalk in the Enterprise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Wha t Is in the Toolkit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Common Enterprise Usa ge . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 New BizTalk Solution Checklist. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 What Are the Teams’ Current Core Skill Sets? What Skill Sets Will the Team Need to Attain? . . . . . . . . . . . . . . . 10 What Are the Project Timelines? Will the Team Be Using BizT alk Exclusively to Decrease Development Time? . . . . . . . . . 12 Is There Enough in the Budget to Implement BizTalk? . . . . . . . . . . . 12 How Many Servers Are Required to Implement a BizT alk Solution? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 Ho w Much Custom Code Are You and Your Team Willing to Create? Would You Rather Use Completely Out-of-the-Box Functionality? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 Is BizT alk Suited for the A pplica tion in Question? . . . . . . . . . . . . . . . 18 Will Every Transaction in the System Need Monitoring? Will the T racked Da ta Need Saving for Archiving Purposes? . . . 19 ■CHAPTER 2 Starting a New BizTalk Project. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 Starting Preliminary Design. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 Creating Your Development Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 Isolated Development Configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . 28 Using Virtual Machines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 Organizing Visual SourceSafe Source Control . . . . . . . . . . . . . . . . . . . 30 vii 6994fmfinal.qxd 10/2/06 12:19 AM Page vii Structuring and Integrating with Visual Studio. . . . . . . . . . . . . . . . . . . . . . . 31 Single Visual Studio Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 Multiple Visual Studio Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 Developer-Independent Solutions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 Organizing Artifacts in BizTalk 2006 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 BizTalk Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 Using BizTalk Explorer to Manage Applications . . . . . . . . . . . . . . . . . 38 BizTalk’s Administration Console . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 Creating a Build-and-Integration Environment. . . . . . . . . . . . . . . . . . . . . . . 40 Five-Step Build Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 Using Test-Driven Development . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 Creating a BizTalk Installation Package. . . . . . . . . . . . . . . . . . . . . . . . 46 BizTalk Assembly Naming and Versioning . . . . . . . . . . . . . . . . . . . . . . . . . . 48 Side-by-Side Deployments. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 Side-by-Side Deployment Strategies . . . . . . . . . . . . . . . . . . . . . . . . . . 50 BizTalk Naming Conventions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 BizTalk Artifact Namespaces. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 PART 2 ■■■ BizTalk Revealed ■CHAPTER 3 Thinking Inside the Box . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 Understanding the Message Bus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68 The Messa gebox . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68 Messa ging Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 Using XML Namespaces. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76 Understanding Property Promotions. . . . . . . . . . . . . . . . . . . . . . . . . . . 76 Using Port Filters and Content-Based Routing . . . . . . . . . . . . . . . . . . 79 Using System Property Schemas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86 Tracking and Message Management. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 Handling F ailed Messa ges and Errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 The BizTalk Management Database. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 ■CONTENTSviii 6994fmfinal.qxd 10/2/06 12:19 AM Page viii ■CHAPTER 4 Pipelining and Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 Getting Started with Pipeline Development . . . . . . . . . . . . . . . . . . . . . . . . 107 Pipeline Stages. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107 Understanding Pipeline Execution. . . . . . . . . . . . . . . . . . . . . . . . . . . . 109 Understanding Interchanges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 Overview of Recoverable Interchange Handling. . . . . . . . . . . . . . . . 112 Configuring Recoverable Interchanges . . . . . . . . . . . . . . . . . . . . . . . 114 Using the Default Pipelines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115 Routing Binary Data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117 Using BizTalk Framework 2.0 Reliable Messaging . . . . . . . . . . . . . . . . . . 118 BizTalk Framework Assembler and Disassembler Pipeline Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119 Working with BizTalk F ramework Properties Within Orchestrations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120 Acknowledgement Verification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120 Custom Components. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121 Component Categories . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122 Component Interfaces. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123 Key BizTalk API Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123 Writing Your First Pipeline Component . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128 Creating More Complex Pipeline Components. . . . . . . . . . . . . . . . . . . . . . 128 Dynamically Promoting Properties and Manipula ting the Message Context . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129 Schema Selection in VS .NET Designer . . . . . . . . . . . . . . . . . . . . . . . 132 Validating and Storing Properties in the Designer . . . . . . . . . . . . . . 135 Custom Disassemblers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139 ■CHAPTER 5 Pipeline Component Best Practices and Examples. . . . . . 145 Crea ting New Documents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145 Using BizTalk Streams . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147 VirtualStream . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148 SeekableReadOnlyStream . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148 XPathReader. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148 Pipeline Component Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152 Dealing with Extremely Large Messages . . . . . . . . . . . . . . . . . . . . . . . . . . 152 Caveats and Gotchas. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154 Large Message Decoding Component (Receive Side). . . . . . . . . . . 155 Large Messa ge Encoding Component (Send Side) . . . . . . . . . . . . . 161 ■CONTENTS ix 6994fmfinal.qxd 10/2/06 12:19 AM Page ix [...]... following: • What BizTalk is and is not designed for • An overview of BizTalk Server’s key features • Resource and estimated budget requirements • Answers to the most common questions concerning implementing BizTalk in a new project scenario 6994ch01final.qxd 10 /2/06 10 :19 AM Page 2 6994ch01final.qxd 10 /2/06 10 :19 AM CHAPTER Page 3 1 sss BizTalk in the Enterprise T he BizTalk Server 2006 product is a group... for a “typical” BizTalk solution Figure 1- 1 illustrates the typical architecture for a BizTalk- based solution 3 6994ch01final.qxd 4 10 /2/06 10 :19 AM Page 4 CHAPTER 1 s BIZTALK IN THE ENTERPRISE Figure 1- 1 Typical BizTalk scenario At its most basic, BizTalk is designed to receive inbound messages, pass them through some form of logical processing, and then deliver the result of that processing to an... software project Our hope is that this book will give that person the tools and know-how to successfully implement a BizTalk solution and feel comfortable that he has designed the best application possible 6994ch01final.qxd 10 /2/06 PART 10 :19 AM Page 1 1 sss Readme.1st The first two chapters of this book are designed to help architects and team leaders perform the most important tasks of any BizTalk project... 228 BizTalk Server 2006 Failed Message Routing As a Blueprint 2 31 Failed Orchestration Routing API for BizTalk 2006 233 Running the EAIProcess 240 Processing and Retrieving Messages and Exceptions from the Fault Message 243 Beyond the Next Horizon 249 6994fmfinal.qxd 10 /2/06 12 :19 ... 1- 2 The stages for the two types of pipeline are as follows: • Send pipeline stages: • Pre-Assemble • Assemble • Encode • Receive pipeline stages: • Decode • Disassemble • Validate • Resolve Party 6994ch01final.qxd 10 /2/06 10 :19 AM Page 7 CHAPTER 1 s BIZTALK IN THE ENTERPRISE Figure 1- 2 BizTalk Send Pipeline Designer Pipeline components are classes that are executed within the various stages of a BizTalk. .. into the BizTalk Management Database Assemblies deployed to this database must also be installed into the Global Assembly Cache and have a strong name The Orchestration Designer is a primarily visual tool It allows you to graphically see the workflow you are creating, as Figure 1- 3 demonstrates Figure 1- 3 BizTalk Orchestration Designer 6994ch01final.qxd 10 /2/06 10 :19 AM Page 9 CHAPTER 1 s BIZTALK IN... adapters provide the logical abstraction for sending and receiving messages to and from BizTalk They allow you to code your application in a generic fashion and not worry about the implementation details of how these messages will be consumed and delivered A port is a logical construct that can receive and send messages to/from the BizTalk 6994ch01final.qxd 10 /2/06 10 :19 AM Page 5 CHAPTER 1 s BIZTALK. .. this thing called a book Special thanks to Marty Wasznicky for his continuous help and all the amazing material that he provided us with Ahmed Metwally xxiii 6994fmfinal.qxd 10 /2/06 12 :19 AM Page xxiv 6994fmfinal.qxd 10 /2/06 12 :19 AM Page xxv Introduction A Tale of Two Products When BizTalk was still in its infancy, there were two teams within Microsoft—the Commerce Server team and the COM+ team The... multifaceted problem that requires smart people BizTalk just happens to be the technology that’s proven over time to be flexible and sophisticated enough to solve it xv 6994fmfinal.qxd xvi 10 /2/06 12 :19 AM Page xvi sFOREWORD Over the years, the BizTalk product team did an amazing job at keeping pace with the evolution of our customers and, more importantly, the issues they face I’ve always been extremely proud...6994fmfinal.qxd x 10 /2/06 12 :19 AM Page x sCONTENTS Dealing with Compressed Files 16 6 Sending Simple Zipped Files 16 7 Sending Multiple Zipped Files 17 4 Receiving Zipped Files 17 4 Using PGP 17 9 PGP Encode Component . APRESS ROADMAP Pro BizTalk 2006 BizTalk 2006 Recipes Pro C# 2005 and the .NET 2.0 Platform, Third Edition Companion eBook Available Pro Pro BizTalk 2006 Dear Reader, Microsoft BizTalk Server. Metwally Pro BizTalk 2006 6994fmfinal.qxd 10 /2/06 12 :19 AM Page i P ro BizTalk 2006 Copyright © 2006 by George Dunphy, Ahmed Metwally All rights reserved. No part of this work may be reproduced. by Marty Wasznicky, Program Manager, Customer Projects and Services–Microsoft Corporation Pro BizTalk 2006 CYAN MAGENTA YELLOW BLACK PANTONE 12 3 CV ISBN 1- 59059-699-4 9 7 815 90 596999 54999 6