20 Unit Testing WHAT'S IN THIS CHAPTER? Learning how unit tests work Adding unit tests to your Objective - C project Confi guring unit tests for an iPhone app Adding unit tests to your C or C++ project Unit testing is a way of validating the run time behavior of your code at build time. In other words, you can test and verify the functionality of individual modules before you assemble them into a working application. In addition to writing your application ’ s classes, you also write one or more unit tests that exercise those classes and verify that they perform as expected. These tests are not part of your application and live in a separate unit test bundle . You can elect to have these tests performed on your code whenever you build your product, ensuring not only that your code compiles and links but that it behaves correctly as well. Unit testing is a fundamental part of Test Driven Development (TDD), a development philosophy popularized by the Extreme Programming movement. In TDD, you develop the test for your function or class fi rst, and then write your function to meet the expectations of the test. In essence, this is what rigorous designers have done for years. They fi rst develop a detailed specifi cation of exactly what a function should do, write the function, and then verify that the function behaves as designed. The quantum leap provided by unit tests is that the “ specifi cation ” is now an automated test that verifi es the design goals of the code rather than a paper description that must be interpreted by the programmer and verifi ed by a quality assurance engineer. Because the test is automated, it can be run every time the application is built, ensuring that every function still conforms to its specifi cation, or immediately alerting the developer if it does not. ➤ ➤ ➤ ➤ c20.indd 549c20.indd 549 1/22/10 4:16:23 PM1/22/10 4:16:23 PM Download at getcoolebook.com 550 ❘ CHAPTER 20 UNIT TESTING Whether or not you subscribe to the principles of Extreme Programming, unit tests provide a powerful tool for avoiding issues like the so - called “ forgotten assumption ” bug. The typical scenario goes like this: 1. You develop a complex application. 2. You then decide to add a new feature to some core class. 3. You make the change in what you think is a completely transparent manner, only to discover that some other part of your application now fails miserably. This is invariably a result of one of two problems: either you inadvertently introduced a bug into the core class, or you forgot about an assumption made by the client code that uses the class. Unit tests can help avoid both of these pitfalls. Xcode supports unit testing of C/C++ and Objective - C applications using two different technologies. Although the concepts and initial steps are the same, most of the details for creating and using unit tests differ for the two languages. After you get past the basics, skip to either the Objective - C or C++ section, as appropriate, for integrating unit tests into your application. HOW UNIT TESTS WORK Unit tests are little more than code — which you write — that exercises the classes and functions in your project. You are entirely responsible for determining what and how your code is tested. Your tests are compiled into a unit test bundle, which is produced by a unit test target added to your project. The collection of tests in a unit test target is called a test suite . To run the tests, all you do is build the unit test target. The target fi rst compiles your tests. It then runs a special build script phase that loads your test code, runs all of the tests, and reports the results. If any of your tests fail, the build process reports these as errors and stops. Figure 10 - 1 shows the build log from a project with unit tests. FIGURE 20 - 1 c20.indd 550c20.indd 550 1/22/10 4:16:29 PM1/22/10 4:16:29 PM Download at getcoolebook.com Unit test bundles are part of the build process. The code associated with unit testing is compiled into the unit test bundle and should never be included in your fi nal product. GETTING STARTED WITH UNIT TESTS There are four basic steps to adding unit tests to a project: 1. Create a unit test target. 2. Confi gure the target and your application for unit tests. 3. Write some tests. 4. Integrate the tests into your development workfl ow. How you approach each step depends on a number of decisions. The biggest decision is whether to create independent or dependent unit tests. Each has its own advantages and disadvantages. The one you choose will determine how you confi gure your unit test target, your application target, and how your tests can be integrated into your development workfl ow. Don’t confuse dependent unit test with target dependencies. Although a dependent unit test target typically depends on its subject target, the term “dependent” has to do with the fact that unit test bundle is not self-contained. Both dependent and independent unit tests may depend on other targets, or not. Independent Unit Tests Independent unit tests are the simplest to create and use, but they have a couple of drawbacks. An independent unit test bundle includes both the tests and the code to be tested. All are compiled and linked into the unit test bundle. At build time, the bundle is loaded and all of the tests are executed. Advantages: Self - contained No special code required Disadvantages: Doesn ’ t test actual product Code must be compiled twice The advantage to independent unit tests, and where they get their name, is that the target and unit test bundle are entirely self - contained. All of the code to be tested is compiled by the target. That is, the target is independent of any other applications or products that your project produces. In fact, the code doesn ’ t even need to be compiled elsewhere. You could, conceivably, create a project that only tests code and produces no products whatsoever. ➤ ➤ ➤ ➤ ➤ ➤ Getting Started with Unit Tests ❘ 551 c20.indd 551c20.indd 551 1/22/10 4:16:30 PM1/22/10 4:16:30 PM Download at getcoolebook.com 552 ❘ CHAPTER 20 UNIT TESTING The disadvantage is that the code being tested is compiled separately from the same code that gets compiled when your application is built. One consideration is the fact that the code could be compiled differently for the unit test bundle and the application. Build - setting differences between the unit test target and your application ’ s target could easily cause subtle differences in the code the compiler produces, which means that your tests are not actually testing the same code that will run in your application. For most code, this probably won ’ t matter, but a difference in, say, the signedness of character variables, optimization, or the size of enums could cause your tests to miss bugs in your application ’ s code or fail tests that should pass. If you are rigorous, or just paranoid, you ’ ll want to test the actual code that your fi nal application will be executing — not just a reasonable facsimile. The other, potential, disadvantage to recompiling all of the same code is that it takes time. All of the code you intend to test will have to be compiled twice — once for the application and again for the unit test bundle. If your code base is large, or it depends on a lot of other code that must be compiled, then compiling everything twice will slow down your builds. Dependent Unit Tests Dependent unit tests perform their tests on the actual code produced by your product. A dependent unit test bundle contains only the test code. When it comes time to perform your unit tests, the program or library that your project produced is loaded into memory along with the unit test bundle. The references in the unit test bundle are linked to the actual classes and functions in your application and then executed. The unit test bundle depends on another product to accomplish its purpose. Advantages: Tests actual code Code only compiled once Disadvantages: Test environment may be awkward Dependent on other targets As you might guess, there ’ s more than just a little sleight of hand involved here. The unit test framework uses two techniques, depending on what kind of product you ’ re testing. The method used to test libraries, frameworks, and independent unit tests is pretty straightforward: the unit test target executes a testing program that loads the unit test bundle (containing the test code and possibly some code to be tested) along with any dynamic libraries or frameworks that need testing. The tests are executed and the testing utility exits. Testing an application is decidedly more bizarre. The unit test target runs a script that launches the actual executable produced by your project. Before the executable is started, several special environment variables are confi gured. These settings are picked up by the system ’ s dynamic library loader and cause it to alter the normal sequence of binding and framework loading that occurs ➤ ➤ ➤ ➤ ➤ ➤ c20.indd 552c20.indd 552 1/22/10 4:16:44 PM1/22/10 4:16:44 PM Download at getcoolebook.com at run time. The settings instruct the loader to fi rst load a special unit test framework into the application ’ s address space. This process is known as bundle injection . The testing framework causes your unit test bundle to also be loaded into memory. Initialization code in your unit test bundle intercepts the execution of your application, preventing it from running normally. Instead, the unit test bundle ’ s code links directly to the functions defi ned in the application and executes all of the tests. It then forces the application to terminate. However convoluted, the beauty of this process is that your unit tests will test the actual, binary code of your application; the same code that will run when your application launches normally. The disadvantage is that this process is complex and requires a number of concessions from your application. Mostly these are restrictions on how your application is built. In the case of some C/C++ applications, you are also required to add code to your application to support dependent unit testing. iPhone Unit Tests The iPhone SDK supports unit testing too. The techniques are very similar to the Objective - C unit testing under Mac OS X — in fact, they both use the same testing framework — but with the following differences: Independent unit tests are called logic tests in iPhone parlance, and are executed using the iPhone simulator. Dependent unit tests are called application tests in iPhone parlance, and are preformed on an actual iPhone or iPod Touch. Setting up an application test suite for the iPhone is signifi cantly different than setting up a dependent test suite for a Mac OS X application. Except for those confi guration differences, you can follow the guidelines and instructions for writing Objective - C unit tests when developing for the iPhone, substituting the terms “ independent test ” and “ dependent test ” with “ logic test ” and “ application test. ” iPhone unit testing requires iPhone OS 3.0 or later. ADDING A UNIT TEST TARGET The fi rst step in adding unit testing to a project is to create a unit test target. Choose Project ➪ New Target and choose a Unit Test Bundle template. Choose the Unit Test Bundle template from the Carbon group to test a C/C++ product, from the Cocoa group to test a Mac OS X Objective - C product, or from the Cocoa Touch group to create an iPhone unit test. An example is shown in Figure 20 - 2. ➤ ➤ ➤ Adding a Unit Test Target ❘ 553 c20.indd 553c20.indd 553 1/22/10 4:16:44 PM1/22/10 4:16:44 PM Download at getcoolebook.com 554 ❘ CHAPTER 20 UNIT TESTING Some releases of the Xcode Development Tools, particularly those intended for iPhone development, do not include the older Carbon and C++ target templates, so your installation might not have a Carbon Unit Test Bundle template. You can “borrow” one from an older Xcode installation or try installing the Mac OS X Xcode package. Give the target a name and select the project it will be added to. Choose a name that refl ects the subject of the test. For example, if you were writing tests for a target named HelperTool , you might name the unit test target HelperToolTests . FIGURE 20 - 2 Xcode creates a new unit test target and adds it to your project. You now need to confi gure it properly and populate it with tests. How you confi gure your unit test target depends on what kind of unit test it is and what kind of product it tests. You might be anxious to try out your new unit test target, but you can ’ t until it is confi gured and you have added at least one test; a unit test bundle will fail if it doesn ’ t contain any tests. The “ Creating a Unit Test ” section, later in this chapter, tells you how to add tests to your unit test bundle. c20.indd 554c20.indd 554 1/22/10 4:16:50 PM1/22/10 4:16:50 PM Download at getcoolebook.com Unit Test Target Dependencies Unit tests are part of the build process. Target dependencies are used to integrate unit tests into your build. What target dependencies you create (if any) will depend on the kind of unit test you are creating. Independent Unit Test Dependencies Because independent/logic unit tests are self - contained, they do not (technically) need to be dependent on any other targets. All of the code that needs to be tested will be compiled when the target is built. Whenever you want to run your unit tests, simply build your unit test target. One of the main tenets of test driven development is that your unit tests should be performed automatically every time you build your project. To do that, follow these steps: 1. Set the active target to your application target. 2. Make your application target dependent on your unit test target. Now every time you build your application, Xcode will fi rst build and run all of the unit tests. Alternatively, you could make the unit test target dependent on your application target; then you have the choice of just building your application or building your application and running all of your unit tests. You could also leave your application and unit test targets independent of each other and create an aggregate target that builds both. As you can see, independent unit test targets are pretty fl exible. Dependent Unit Test Dependencies Dependent Mac OS X (but not iPhone) unit test targets must depend on the target, or targets, that produce the products they test. Otherwise, there is no guarantee that the tests will be performed on up - to - date code. If you want unit tests run every time you build your product, follow these steps: 1. Set the active target to the unit test target. 2. Set the active executable to the results of the product target. Now every time you build, the application is built followed by a run of all of the unit tests. The build will only be successful if both the build and the unit tests pass muster. Using this arrangement, you can easily ignore unit tests by building just the product target, or making another target dependent on the product target directly. In a project with many product and unit test targets you could, for example, create two aggregate targets: one that depends on all of the product targets for “ quick ” builds and a second that depends on all of their respective unit test targets for “ full ” builds. An iPhone unit test target ’ s dependencies are inverted from those used by dependent unit test targets. The section “ Confi guring an iPhone Application Test ” shows you both how to confi gure the iPhone application unit test target and set up its dependencies. Adding a Unit Test Target ❘ 555 c20.indd 555c20.indd 555 1/22/10 4:17:00 PM1/22/10 4:17:00 PM Download at getcoolebook.com . requires iPhone OS 3. 0 or later. ADDING A UNIT TEST TARGET The fi rst step in adding unit testing to a project is to create a unit test target. Choose Project ➪ New Target and choose a Unit Test. 5 53 c20.indd 553c20.indd 5 53 1/22/10 4:16:44 PM1/22/10 4:16:44 PM Download at getcoolebook.com 554 ❘ CHAPTER 20 UNIT TESTING Some releases of the Xcode Development Tools, particularly those. can “borrow” one from an older Xcode installation or try installing the Mac OS X Xcode package. Give the target a name and select the project it will be added to. Choose a name that refl ects the subject