CYAN MAGENTA YELLOW BLACK PANTONE 123 CV BOOKS FOR PROFESSIONALS BY PROFESSIONALS ® THE EXPERT’S VOICE ® IN NET Companion eBook Available Dear Reader, This book shows you how to write lightweight but powerful test automation in a NET programming environment By lightweight, I mean short (generally less than two pages of code) and quick (generally less than two hours) If you’ve ever had to perform manual software testing, you probably found the process slow, inefficient, and often just plain boring Using the automation techniques in this book you can test your software systems quickly and efficiently During my years as a software tester at Microsoft and other companies, I discovered that it was fairly easy to find good information about software testing theory, but when it came to finding actual concrete test automation examples, there just wasn’t much information available I set out to put together in one place all the test automation techniques I had discovered, and this book is the result In Part I of this book, I present techniques for API (Application Programming Interface) testing Also called unit testing or module testing, this is the most fundamental type of software testing I also show you how to write automated UI (user interface) tests for Windows form-based applications and how to design test harness structures In Part II of this book, I present techniques to write test automation for Web-based applications These techniques include automated HTTP request-response testing, automated UI testing, and automated Web services testing In Part III of the book, I present test automation techniques that are related to data I show you how to automatically generate combinations and permutations of test case input data I also present techniques for testing SQL stored procedures and ADO.NET (data-based) applications And I give you code to perform a wide range of XML data tasks In short, if you are a software developer, tester, or manager in a NET environment, you’ll find this book a useful addition to your resources RELATED TITLES Dr James D McCaffrey A Tester’s Guide to NET Programming 1-59059-600-5 Join online discussions: Companion eBook forums.apress.com SOURCE CODE ONLINE FOR PROFESSIONALS BY PROFESSIONALS ™ www.apress.com Shelve in Software Development 90000 89253 59663 McCaffrey ISBN 1-59059-663-3 See last page for details on $10 eBook version NET Test Automation Recipes NET Test Automation Recipes: A Problem-Solution Approach NET Test Automation Recipes A Problem-Solution Approach Discover how to write lightweight yet powerful test tools in NET James D McCaffrey 781590 596630 User level: Intermediate–Advanced this print for content only—size & color not accurate 7" x 9-1/4" / CASEBOUND / MALLOY 6633FM.qxd 4/3/06 1:54 PM Page i NET Test Automation Recipes A Problem-Solution Approach James D McCaffrey .NET Test Automation Recipes Copyright © 2012 by James D McCaffrey This work is subject to copyright All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed Exempted from this legal reservation are brief excerpts in connection with reviews or scholarly analysis or material supplied specifically for the purpose of being entered and executed on a computer system, for exclusive use by the purchaser of the work Duplication of this publication or parts thereof is permitted only under the provisions of the Copyright Law of the Publisher's location, in its current version, and permission for use must always be obtained from Springer Permissions for use may be obtained through RightsLink at the Copyright Clearance Center Violations are liable to prosecution under the respective Copyright Law ISBN-13 (pbk): 978-1-4302-5077-7 ISBN-13 (electronic): 978-1-4302-5078-4 Trademarked names, logos, and images may appear in this book Rather than use a trademark symbol with every occurrence of a trademarked name, logo, or image we use the names, logos, and images only in an editorial fashion and to the benefit of the trademark owner, with no intention of infringement of the trademark The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to proprietary rights While the advice and information in this book are believed to be true and accurate at the date of publication, neither the authors nor the editors nor the publisher can accept any legal responsibility for any errors or omissions that may be made The publisher makes no warranty, express or implied, with respect to the material contained herein President and Publisher: Paul Manning Lead Editor: Jonathan Hassell Technical Reviewer: Josh Kelling Editorial Board: Steve Anglin, Mark Beckner, Ewan Buckingham, Gary Cornell, Louise Corrigan, Morgan Ertel, Jonathan Gennick, Jonathan Hassell, Robert Hutchinson, Michelle Lowman, James Markham, Matthew Moodie, Jeff Olson, Jeffrey Pepper, Douglas Pundick, Ben Renow-Clarke, Dominic Shakeshaft, Gwenan Spearing, Matt Wade, Tom Welsh Coordinating Editor: Katie Stence Copy Editor: Julie McNamee Compositor: Lynn L’Heureux Indexer: Becky Hornak Artist: Kurt Krames Cover Designer: Anna Ishchenko Distributed to the book trade worldwide by Springer Science+Business Media New York, 233 Spring Street, 6th Floor, New York, NY 10013 Phone 1-800-SPRINGER, fax (201) 348-4505, e-mail orders-ny@springersbm.com, or visit www.springeronline.com Apress Media, LLC is a California LLC and the sole member (owner) is Springer Science + Business Media Finance Inc (SSBM Finance Inc) SSBM Finance Inc is a Delaware corporation For information on translations, please e-mail rights@apress.com, or visit www.apress.com Apress and friends of ED books may be purchased in bulk for academic, corporate, or promotional use eBook versions and licenses are also available for most titles For more information, reference our Special Bulk Sales–eBook Licensing web page at www.apress.com/bulk-sales Any source code or other supplementary materials referenced by the author in this text is available to readers at www.apress.com For detailed information about how to locate your book’s source code, go to www.apress.com/source-code/ 6633FM.qxd 4/3/06 1:54 PM Page v Contents at a Glance About the Author xiii About the Technical Reviewer xv Acknowledgments xvii Introduction xix ■CHAPTER API Testing ■CHAPTER Reflection-Based UI Testing 33 ■CHAPTER Windows-Based UI Testing 65 ■CHAPTER Test Harness Design Patterns 97 ■CHAPTER Request-Response Testing 135 ■CHAPTER Script-Based Web UI Testing ■CHAPTER Low-Level Web UI Testing 185 ■CHAPTER Web Services Testing ■CHAPTER SQL Stored Procedure Testing 167 207 237 ■CHAPTER 10 Combinations and Permutations 265 ■CHAPTER 11 ADO.NET Testing 301 ■CHAPTER 12 XML Testing 335 ■INDEX 365 v 6633FM.qxd 4/3/06 1:54 PM Page vii Contents About the Author xiii About the Technical Reviewer xv Acknowledgments xvii Introduction xix PART ■■■ ■CHAPTER Windows Application Testing API Testing .3 1.0 Introduction 1.1 Storing Test Case Data 1.2 Reading Test Case Data 1.3 Parsing a Test Case 1.4 Converting Data to an Appropriate Data Type 1.5 Determining a Test Case Result 11 1.6 Logging Test Case Results 13 1.7 Time-Stamping Test Case Results 16 1.8 Calculating Summary Results 17 1.9 Determining a Test Run Total Elapsed Time 19 1.10 Dealing with null Input/null Expected Results 20 1.11 Dealing with Methods that Throw Exceptions 22 1.12 Dealing with Empty String Input Arguments 24 1.13 Programmatically Sending E-mail Alerts on Test Case Failures 26 1.14 Launching a Test Harness Automatically 28 1.15 Example Program: ApiTest 29 vii 6633Index.qxd 4/3/06 2:00 PM Page 379 ■INDEX value determining pass/fail result when stored procedure under test returns no, 256–258 of HTML element manipulating, 194–195 verifying, 195–198 testing stored procedure that does not return, 314–317 that returns, 306–309 Verification, 208, 227, 247 verify() function, 174–175 verifying value of HTML element on Web AUT, 174–175, 195–198 ViewState value, programmatically determining, 152–155 Visual Studio.NET calling Web methods and, 208 Object Browser, 195 proxy mechanism, 212–214 SOAP message and, 216–217, 224 W Web application state, determining, 172–173 Web services testing example program, 232–234 HTTP, using, 220–221 in-memory test case data store saving to, 229–231 using, 226–228 key code from application that calls Web service, 212 overview of, 207–208 proxy mechanism, using, 212–214 sockets, using, 214–219, 223 SQL database, creating and populating, 209–211 TCP, using, 222–225 test run with validation, 208 Web UI testing See low-level Web UI testing; script-based Web UI testing WebApp.aspx application, 168–170 WebClient class, 137–138 WebRequest - WebResponse class, 137 WebRequest object, 139 Weekly Test Runs (WTRs), 126 while loop Combination.Successor() method and, 273–274, 287 FindWindow() function and, 71 iterating through test case file with, 7–8 NetworkStream object, 222–225 Permutation.Successor() method and, 286–287 Socket.Receive() method and, 219 StreamReader.ReadLine() method inside, 141–142 white box testing, 127 Win32 functions C# method alias for, 69 FindWindow() message box, dealing with, 82–86 obtaining handle to main window of AUT, 68–73 FindWindowByIndex(), 75–77 FindWindowEx(), 73–75 GetMenu(), 86–89 GetMenuItem(), 86–89 GetSubMenu(), 86–89 PostMessage(), 80–82 SendMessage() application state, checking, 89–91 menu, dealing with, 86–89 PostMessage() compared to, 80 sending characters to control, 78–80 Windows-based UI testing and, 65 window handle, obtaining to main window of AUT, 68–73 to named control, 73–75 to non-named control, 75–77 Windows Authentication, 241 windows of Internet Explorer, 193 Windows Task Scheduler, launching test harness automatically using, 28–29 Windows-based UI testing application state, checking, 89–91 clicking on control, 80–82 example program, 91–96 launching application under test, 66–67 menu, dealing with, 86–89 message box, dealing with, 82–86 obtaining handle to main window of AUT, 68–73 to named control, 73–75 to non-named control, 75–77 overview of, 65–66 sending characters to control, 78–80 WM CHAR notification message, 78–80 WM_COMMAND notification message, 86–89 WM_GETTEXT notification message, 89–91 WM_LBUTTONDOWN and WM_LBUTTONUP notification messages, 80–82 wrapping test harness code in try-catchfinally block, 102 WriteAttributeString() method, 355–356 WriteEndAttribute() method, 110 WriteEndElement() method, 110 WriteLine() method, 177–178, 249–250, 327–329 WriteStartAttribute() method, 110 WriteStartElement() method, 110, 355–356 WriteString() method, 110, 355–356 379 6633Index.qxd 380 4/3/06 2:00 PM Page 380 ■INDEX writing automated tests, procedural versus object-oriented style, 66 reflection-based UI testing, 34 test case results to external storage, 13–15 test results directly to text file from T-SQL test harness, 249–250 to XML file using XmlTextWriter class, 355–356 WTRs (Weekly Test Runs), 126 X XML auto-documentation, 127 XML file data buffered model test harness, creating for, 113–117 streaming model test harness, creating for, 108–112 XML testing comparing files for canonical equivalence, 359–361 for exact equality, 356–357 for exact equality, except for encoding, 358 example program, 361–362 modifying using XSLT, 353–355 overview of, 335–336 parsing DataSet object, using, 347–350 XmlDocument class, using, 339–341 XmlSerializer class, using, 343–347 XmlTextReader class, using, 337–339 XPathDocument class, using, 341–343 validating DTD file, using, 353 XSD schema, using, 350–352 writing to XML file using XmlTextWriter class, 355–356 XML Web services, 208 XmlDataDocument class, 341 XmlDocument class, parsing XML data using, 339–341 XmlDsigC14NTransform class, 359–361 XmlSerializer class, parsing XML data using, 343–347 XmlTextReader class, parsing XML data using, 337–339 XmlTextReader object, 108–112, 117 XmlTextWriter class, writing to XML file using, 355–356 XmlTextWriter object, 108–112 XmlValidatingReader object, 350–352 XPathDocument class, parsing XML data using, 341–343 XPathNavigator object, 343 XPathNodeIterator object, 341–343 XSD schema, validating XML data using, 350–352 XSLT (Extensible Stylesheet Language Transformations), modifying XML data using, 353–355 XslTransform object, 353–355 ... UI testing, 34 test case results to external storage, 13–15 test results directly to text file from T-SQL test harness, 249–250 to XML file using XmlTextWriter class, 355–356 WTRs (Weekly Test. .. Request-Response Testing 135 ■CHAPTER Script-Based Web UI Testing ■CHAPTER Low-Level Web UI Testing 185 ■CHAPTER Web Services Testing... Application Testing API Testing .3 1.0 Introduction 1.1 Storing Test Case Data 1.2 Reading Test Case