ptg 13 Streaming Data with Ajax and Comet I n Chapter 12, Abstracting Browser Differences: Ajax, we saw how the XML- HttpRequest object enables web pages to take the role of interactive applications that can both update data on the back-end server by issuing POST requests, as well as incrementally update the page without reloading it using GET requests. In this chapter we will take a look at technologies used to implement live data streaming between the server and client. This concept was first enabled by Netscape’s Server Push in 1995, and is possible to implement in a variety of ways in today’s browsers under umbrella terms such as Comet, Reverse Ajax, and Ajax Push. We will look into two implementations in this chapter; regular polling and so-called long polling. This chapter will add some features to the tddjs.ajax.request interface developed in the previous chapter, add a new interface, and finally integrate with tddjs.util.observable, developed in Chapter 11, The Observer Pattern, enabling us to create a streaming data client that allows JavaScript objects to observe server-side events. The goal of this exercise is twofold: learning more about models for client- server interaction, and of course test-driven development. Important TDD lessons in this chapter includes delving deeper into testing asynchronous interfaces and testing timers. We will continue our discussion of stubbing, and get a glimpse of the workflow and choices presented to us as we develop more than a single interface. 293 From the Library of WoweBook.Com Download from www.eBookTM.com ptg 294 Streaming Data with Ajax and Comet 13.1 Polling for Data Although one-off requests to the server can enable highly dynamic and interesting applications, it doesn’t open up for real live applications. Applications such as Facebook’s and GTalk’s in-browser chats are examples of applications that cannot make sense without a constant data stream. Other features, such as stock tickers, auctions, and Tw itter ’s web interface become significantly more useful with a live data stream. The simplest way to keep a constant data stream to the client is to poll the server on some fixed interval. Polling is as simple as issuing a new request every so many milliseconds. The shorter delay between requests, the more live the applica- tion. We will discuss some ups and downs with polling later, but in order for that discussion to be code-driven we will jump right into test driving development of a poller. 13.1.1 Project Layout As usual we will use JsTestDriver to run tests. The initial project layout can be seen in Listing 13.1 and is available for download from the book’s website. 1 Listing 13.1 Directory layout for the poller project chris@laptop:~/projects/poller $ tree . | jsTestDriver.conf | lib | ` ajax.js | ` fake_xhr.js | ` function.js | ` object.js | ` stub.js | ` tdd.js | ` url_params.js | src | ` poller.js | ` request.js ` test ` poller_test.js ` request_test.js 1. http://tddjs.com From the Library of WoweBook.Com Download from www.eBookTM.com ptg 13.1 Polling for Data 295 In many ways this project is a continuation of the previous one. Most files can be recognized from the previous chapter. The request.js file, and its test case are brought along for further development, and we will add some functionality to them. Note that the final refactoring discussed in Chapter 12, Abstracting Browser Differ- ences: Ajax, in which tdd.ajax.request returns an object representing the re- quest, is not implemented. Doing so would probably be a good idea, but we’ll try not to tie the two interfaces too tightly together, allowing the refactoring to be performed at some later time. Sticking with the code exactly as we developed it in the previous chapter will avoid any surprises, allowing us to focus entirely on new features. The jsTestDriver.conf configuration file needs a slight tweak for this project. The lib directory now contains an ajax.js file that depends on the tddjs object defined in tdd.js; however, it will be loaded before the file it depends on. The solution is to manually specify the tdd.js file first, then load the remaining lib files, as seen in Listing 13.2. Listing 13.2 Ensuring correct load order of test files server: http://localhost:4224 load: - lib/tdd.js - lib/stub.js - lib/*.js - src/*.js - test/*.js 13.1.2 The Poller: tddjs.ajax.poller In Chapter 12, Abstracting Browser Differences: Ajax, we built the request interface by focusing heavily on the simplest use case, calling tddjs.ajax.get or tddjs.ajax.post to make one-off GET or POST requests. In this chapter we are going to flip things around and focus our efforts on building a stateful object, such as the one we realized could be refactored from tddjs.ajax.request. This will show us a different way to work, and, because test-driven development really is about design and specification, a slightly different result. Once the object is useful we will implement a cute one-liner interface on top of it to go along with the get and post methods. From the Library of WoweBook.Com Download from www.eBookTM.com ptg 296 Streaming Data with Ajax and Comet 13.1.2.1 Defining the Object The first thing we expect from the interface is simply that it exists, as Listing 13.3 shows. Listing 13.3 Expecting tddjs.ajax.poller to be an object (function () { var ajax = tddjs.ajax; TestCase("PollerTest", { "test should be object": function () { assertObject(ajax.poller); } }); }()); This test jumps the gun on a few details; we know that we are going to want to shorten the full namespace, and doing so requires the anonymous closure to avoid leaking the shortcut into the global namespace. Implementation is a simple matter of defining an object, as seen in Listing 13.4. Listing 13.4 Defining tddjs.ajax.poller (function () { var ajax = tddjs.namespace("ajax"); ajax.poller = {}; }()); The same initial setup (anonymous closure, local alias for namespace) is done here as well. Our first test passes. 13.1.2.2 Start Polling The bulk of the poller’s work is already covered by the request object, so it is simply going to organize issuing requests periodically. The only extra option the poller needs is the interval length in milliseconds. To start polling, the object should offer a start method. In order to make any requests at all we will need a URL to poll, so the test in Listing 13.5 specifies that the method should throw an exception if no url property is set. From the Library of WoweBook.Com Download from www.eBookTM.com ptg 13.1 Polling for Data 297 Listing 13.5 Expecting start to throw an exception on missing URL "test start should throw exception for missing URL": function () { var poller = Object.create(ajax.poller); assertException(function () { poller.start(); }, "TypeError"); } As usual, we run the test before implementing it. The first run coughs up an error stating that there is no Object.create method. To fix this we fetch it from Chap- ter 7, Objects and Prototypal Inheritance, and stick it in tdd.js. What happens next is interesting; the testpasses.Somehow a TypeError is thrown, yetwe haven’tdone anything other than defining the object. To see what’s happening, we edit the test and remove the assertException call, simply calling poller.start() directly in the test. JsTestDriver should pick up the exception and tell us what’s going on. As you might have guessed, the missing start method triggers a TypeError of its own. This indicates that the test isn’t good enough. To improve the situation we add another test stating that there should be a start method, as seenin Listing 13.6. Listing 13.6 Expecting the poller to define a start method "test should define a start method": function () { assertFunction(ajax.poller.start); } With this test in place, we now get a failure stating that start was expected to be a function, but rather was undefined. The previous test still passes. We will fix the newly added test by simply adding a start method, as in Listing 13.7. Listing 13.7 Adding the start method (function () { var ajax = tddjs.namespace("ajax"); function start() { } ajax.poller = { start: start }; }()); From the Library of WoweBook.Com Download from www.eBookTM.com ptg 298 Streaming Data with Ajax and Comet Running the tests again confirms that the existence test passes, but the original test expecting an exception now fails. This is all good and leads us to the next step, seen in Listing 13.8; throwing an exception for the missing URL. Listing 13.8 Throwing an exception for missing URL function start() { if (!this.url) { throw new TypeError("Must specify URL to poll"); } } Running the tests over confirms that they are successful. 13.1.2.3 Deciding the Stubbing Strategy Once a URL is set, the start method should make its first request. At this point we have a choice to make. We still don’t want to make actual requests to the server in the tests, so we will continue stubbing like we did in the previous chapter. How- ever, at this point we have a choice of where to stub. We could keep stubbing ajax.create and have it return a fake request object, or we could hook in higher up, stubbing the ajax.request method. Both approaches have their pros and cons. Some developers will always prefer stubbing and mocking as many of an inter- face’s dependencies as possible (you might even see the term mockists used about these developers). This approach is common in behavior-driven development. Fol- lowing the mockist way means stubbing (or mocking, but we’ll deal with that in Chapter 16, Mocking and Stubbing) ajax.request and possibly other non-trivial dependencies. The advantage of the mockist approach is that it allows us to freely decide development strategy. For instance, by stubbing all of the poller’s dependen- cies, we could easily have built this object first and then used the stubbed calls as starting points for tests for the request interface when we were done. This strategy is known as top-down—in contrast to the current bottom-up strategy—and it even allows a team to work in parallel on dependent interfaces. The opposite approach is to stub and mock as little as possible; only fake those dependencies that are truly inconvenient, slow, or complicated to setup and/or run through in tests.In a dynamically typed language such as JavaScript, stubs and mocks come with a price; because the interface of a test double cannot be enforced (e.g., by an “implements” keyword or similar) in a practical way, there is a real possibility of using fakes in tests that are incompatible with their production counterparts. Making tests succeed with such fakes will guarantee the resulting code will break when faced with the real implementation in an integration test, or worse, in production. From the Library of WoweBook.Com Download from www.eBookTM.com ptg 13.1 Polling for Data 299 Whereas we had no choice of where to stub while developing ajax.request (it only depended on the XMLHttpRequest object via the ajax.create method), we now have the opportunity to choose if we want to stub ajax.request or ajax.create. We will try a slightly different approach in this chapter by stubbing “lower.” This makes our tests mini integration tests, as discussed in Chapter 1, Automated Testing, with the pros and cons that follow. However, as we have just developed a reasonable test suite for ajax.request, we should be able to trust it for the cases we covered in Chapter 12, Abstracting Browser Differences: Ajax. While developing the poller we will strive to fake as little as possible, but we need to cut off the actual server requests. To do this we will simply keep using the fakeXMLHttpRequest object from Chapter 12, Abstracting Browser Differences: Ajax. 13.1.2.4 The First Request To specify that the start method should start polling, we need to assert somehow that a URL made it across to the XMLHttpRequest object. To do this we assert that its open method was called with the expected URL, as seen in Listing 13.9. Listing 13.9 Expecting the poller to issue a request setUp: function () { this.ajaxCreate = ajax.create; this.xhr = Object.create(fakeXMLHttpRequest); ajax.create = stubFn(this.xhr); }, tearDown: function () { ajax.create = this.ajaxCreate; }, /* */ "test start should make XHR request with URL": function () { var poller = Object.create(ajax.poller); poller.url = "/url"; poller.start(); assert(this.xhr.open.called); assertEquals(poller.url, this.xhr.open.args[1]); } From the Library of WoweBook.Com Download from www.eBookTM.com ptg 300 Streaming Data with Ajax and Comet Again, we use Object.create to create a new fake object, assign it to a prop- erty of the test case, and then stub ajax.create to return it. The implementation should be straightforward, as seen in Listing 13.10. Listing 13.10 Making a request function start() { if (!this.url) { throw new TypeError("Must provide URL property"); } ajax.request(this.url); } Note that the test did not specify specifically to use ajax.request. We could have made the request any way we wanted, so long as we used the transport provided by ajax.create. This means, for instance, that we could carry out the aforemen- tioned refactoring on the request interface without touching the poller tests. Running the tests confirms that they all pass. However, the test is not quite as concise as it could be. Knowing that the open method was called on the transport doesn’t necessarily mean that the request was sent. We’d better add an assertion that checks that send was called as well, as Listing 13.11 shows. Listing 13.11 Expecting request to be sent "test start should make XHR request with URL": function () { var poller = Object.create(ajax.poller); poller.url = "/url"; poller.start(); var expectedArgs = ["GET", poller.url, true]; var actualArgs = [].slice.call(this.xhr.open.args); assert(this.xhr.open.called); assertEquals(expectedArgs, actualArgs); assert(this.xhr.send.called); } 13.1.2.5 The complete Callback How will we issue the requests periodically? A simple solution is to make the request through setInterval. However, doing so may cause severe problems. Issuing new requests without knowing whether or not previous requests completed could From the Library of WoweBook.Com Download from www.eBookTM.com ptg 13.1 Polling for Data 301 lead to multiple simultaneous connections, which is not desired. A better solution is to trigger a delayed request once the previous one finishes. This means that we have to wrap the success and failure callbacks. Rather than adding identical success and failure callbacks (save for which user defined callback they delegate to), we are going to make a small addition to tddjs.ajax.request; the complete callback will be called when a request is complete, regardless of success. Listing 13.12 shows the update needed in the requestWithReadyStateAndStatus helper, as well as three new tests, asserting that the complete callback is called for successful, failed, and local requests. Listing 13.12 Specifying the complete callback function forceStatusAndReadyState(xhr, status, rs) { var success = stubFn(); var failure = stubFn(); var complete = stubFn(); ajax.get("/url", { success: success, failure: failure, complete: complete }); xhr.complete(status, rs); return { success: success.called, failure: failure.called, complete: complete.called }; } TestCase("ReadyStateHandlerTest", { /* */ "test should call complete handler for status 200": function () { var request = forceStatusAndReadyState(this.xhr, 200, 4); assert(request.complete); }, "test should call complete handler for status 400": From the Library of WoweBook.Com Download from www.eBookTM.com ptg 302 Streaming Data with Ajax and Comet function () { var request = forceStatusAndReadyState(this.xhr, 400, 4); assert(request.complete); }, "test should call complete handler for status 0": function () { var request = forceStatusAndReadyState(this.xhr, 0, 4); assert(request.complete); } }); As expected, all three tests fail given that no complete callback is called anywhere. Adding it in is straightforward, as Listing 13.13 illustrates. Listing 13.13 Calling the complete callback function requestComplete(options) { var transport = options.transport; if (isSuccess(transport)) { if (typeof options.success == "function") { options.success(transport); } } else { if (typeof options.failure == "function") { options.failure(transport); } } if (typeof options.complete == "function") { options.complete(transport); } } When a request is completed, the poller should schedule another request. Scheduling ahead of time is done with timers, typically setTimeout for a sin- gle execution such as this. Because the new request will end up calling the same callback that scheduled it, another one will be scheduled, and we have a continu- ous polling scheme, even without setInterval. Before we can implement this feature we need to understand how we can test timers. From the Library of WoweBook.Com Download from www.eBookTM.com . Ensuring correct load order of test files server: http://localhost:4224 load: - lib/tdd.js - lib/stub.js - lib/*.js - src/*.js - test/*.js 13.1.2 The Poller: tddjs.ajax.poller In Chapter 12, Abstracting. client that allows JavaScript objects to observe server-side events. The goal of this exercise is twofold: learning more about models for client- server interaction, and of course test-driven development many of an inter- face’s dependencies as possible (you might even see the term mockists used about these developers). This approach is common in behavior-driven development. Fol- lowing the mockist