ptg 13.4 The Comet Client 333 The test fails as dispatch was not called. To fix this we need to parse the responseText as JSON and call the method from within the success callback of the request. A very naive implementation can be seen in Listing 13.63. Listing 13.63 Naive success callback to the poller function connect() { if (!this.url) { throw new TypeError("Provide client URL"); } if (!this.poller) { this.poller = ajax.poll(this.url, { success: function (xhr) { this.dispatch(JSON.parse(xhr.responseText)); }.bind(this) }); } } At this point I am expecting this test to still fail in at least a few browsers. As we discussed in Chapter 8, ECMAScript 5th Edition, EcmaScript5 specifies a JSON object. However, it is not yet widely implemented, least of all in older browsers such as Internet Explorer 6. Still, the tests pass. What’s happening is that JsTestDriver is already using Douglas Crockford’s JSON parser internally, and because it does not namespace its dependencies in the test runner, our test accidentally works because the environment loads our dependencies for us. Hopefully, this issue with JsTest- Driver will be worked out, but until then, we need to keep this in the back of our heads. The proper solution is of course to add, e.g., json2.js from json.org in lib/. I mentioned that the above implementation was naive. A successful response from the server does not imply valid JSON. What do you suppose happens when the test in Listing 13.64 runs? Listing 13.64 Expecting badly formed data not to be dispatched "test should not dispatch badly formed data": function () { this.client.url = "/my/url"; this.client.dispatch = stubFn(); this.client.connect(); From the Library of WoweBook.Com Download from www.eBookTM.com ptg 334 Streaming Data with Ajax and Comet this.xhr.complete(200, "OK"); assertFalse(this.client.dispatch.called); } Furthermore, if we expect the server to return JSON data, it would probably be a good idea to indicate as much by sending the right Accept header with the request. 13.4.5.1 Separating Concerns The current implementation has a code smell—something that doesn’t feel quite right. JSON parsing doesn’t really belong inside a Comet client; its responsibili- ties are delegating server-side events to client-side observers and publishing client- side events to the server. Ideally the transport would handle correct encoding of data. As I’ve mentioned more than a few times already, the ajax.request should be refactored such that it provides an object that can be extended. This would have allowed us to extend it to provide a custom request object specifi- cally for JSON requests, seeing as that is quite a common case. Using such an API, the connect method could look something like Listing 13.65, which is a lot leaner. Listing 13.65 Using tailored JSON requests function connect() { if (!this.url) { throw new TypeError("Provide client URL"); } if (!this.poller) { this.poller = ajax.json.poll(this.url, { success: function (jsonData) { this.dispatch(jsonData); }.bind(this) }); } } Granted, such a poller could be provided with the current implementation of ajax.request and ajax.poll, but parsing JSON belongs in ajax.poll as little as it does in ajax.cometClient. From the Library of WoweBook.Com Download from www.eBookTM.com ptg 13.4 The Comet Client 335 13.4.6 Tracking Requests and Received Data When polling, we need to know what data to retrieve on each request. With long polling, the client polls the server; the server keeps the connection until new data is available, passes it, and closes. Even if the client immediately makes another request, there is a risk of loosing data between requests. This situation gets even worse with normal polling. How will the server know what data to send back on a given request? To be sure all the data makes it to the client, we need a token to track requests. Ideally, the server should not need to keep track of its clients. When polling a single source of data, such as “tweets” on Twi tt er, a reasonable token could be the unique id of the last tweet received by the client. The client sends the id with each request, instructing the server to respond with any newer tweets. In the case of the Comet client, we expect it to handle all kinds of data streams, and unless the server uses some kind of universally unique id, we cannot rely on the id token. Another possibility is to have the client pass along a timestamp indicating when the previous request finished. In other words, the client asks the server to respond with all data that was created since the last request finished. This approach has a major disadvantage; it assumes that the client and server are in sync, possibly down to millisecond granularity and beyond. Such an approach is so fragile it cannot even be expected to work reliably with clients in the same time zone. An alternative solution is to have the server return a token with each response. The kind of token can be decided by the server, all the client needs to do is to include it in the following request. This model works well with both the id and timestamp approaches as well as others. The client doesn’t even know what the token represents. To include the token in the request, a custom request header or a URL parameter are both good choices. We will make the Comet client pass it along as a request header, called X-Access-Token. The server will respond with data guaranteed to be newer than data represented by the token. Listing 13.66 expects the custom header to be provided. Listing 13.66 Expecting the custom header to be set "test should provide custom header": function () { this.client.connect(); assertNotUndefined(this.xhr.headers["X-Access-Token"]); } This test fails as expected, and the implementation can be seen in Listing 13.67. From the Library of WoweBook.Com Download from www.eBookTM.com ptg 336 Streaming Data with Ajax and Comet Listing 13.67 Adding a custom header function connect() { /* */ if (!this.poller) { this.poller = ajax.poll(this.url, { /* */ headers: { "Content-Type": "application/json", "X-Access-Token": "" } }); } } For the first request the token will be blank. In a more sophisticated imple- mentation the initial token could possibly be set manually, e.g., by reading it from a cookie or local database to allow a user to pick up where she left off. Sending blank tokens on every request doesn’t really help us track requests. The next test, shown in Listing 13.68, expects that the token returned from the server is sent on the following request. Listing 13.68 Expecting the received token to be passed on second request tearDown: function () { /* */ Clock.reset(); }, /* */ "test should pass token on following request": function () { this.client.connect(); var data = { token: 1267482145219 }; this.xhr.complete(200, JSON.stringify(data)); Clock.tick(1000); var headers = this.xhr.headers; assertEquals(data.token, headers["X-Access-Token"]); } From the Library of WoweBook.Com Download from www.eBookTM.com ptg 13.4 The Comet Client 337 This test simulates a successful request with a JSON response that includes only the token. After completing the request, the clock is ticked 1,000 milliseconds ahead to trigger a new request, and for this request we expect the token header to be sent with the received token. The test fails as expected; the token is still the blank string. Note that because we didn’t make it possible to configure the polling interval through the client, we cannot set the polling interval explicitly in the test. This makes the Clock.tick(1000) something of a magical incantation, as it is not obvious why it is ticked exactly 1,000 milliseconds ahead. The client should have a way to set the poller interval, and when it does, this test should be updated for clarity. To pass this test we need a reference to the headers object so we can change it after each request. Listing 13.69 shows the implementation. Listing 13.69 Updating the request header upon request completion function connect() { /* */ var headers = { "Content-Type": "application/json", "X-Access-Token": "" }; if (!this.poller) { this.poller = ajax.poll(this.url, { success: function (xhr) { try { var data = JSON.parse(xhr.responseText); headers["X-Access-Token"] = data.token; this.dispatch(data); } catch (e) {} }.bind(this), headers: headers }); } } With this implementation in place the test passes, yet we are not done. If, for some reason, the server fails to deliver a token in response to a request, we should not blatantly overwrite the token we already have with a blank one, losing track of our progress. Also, we do not need to send the token to the dispatch method. From the Library of WoweBook.Com Download from www.eBookTM.com ptg 338 Streaming Data with Ajax and Comet Are there other cases related to the request token that should be tested? Think it over, write tests, and update the implementation to fit. 13.4.7 Publishing Data The Comet client also needs a notify method. As an exercise, try to use TDD to implement this method according to these requirements: • The signature should be client.notify(topic, data) • The method should POST to client.url • The data should be sent as an object with properties topic and data What Content-Type will you send the request with? Will the choice of Content-Type affect the body of the request? 13.4.8 Feature Tests The cometClient object only depends directly on observable and the poller, so adding feature tests to allow it to fail gracefully is fairly simple, as seen in Listing 13.70. Listing 13.70 Comet client feature tests (function () { if (typeof tddjs == "undefined") { return; } var ajax = tddjs.namespace("ajax"); var util = tddjs.namespace("util"); if (!ajax.poll || !util.observable) { return; } /* */ }()); From the Library of WoweBook.Com Download from www.eBookTM.com ptg 13.5 Summary 339 13.5 Summary In this chapter we have built on top of the ajax methods developed in Chapter 12, Abstracting Browser Differences: Ajax, and implemented polling, the client side of long polling and finally a simple Comet client that leveraged the observable object developed in Chapter 11, The Observer Pattern. The main focus has, as usual, been on the testing and how to properly use the tests to instruct us as we dig deeper and deeper. Still, we have been able to get a cursory look at technologies collectively referred to as Comet, Reverse Ajax, and others. In the previous chapter we introduced and worked closely with stubs. In this chapter we developed the poller slightly differently by not stubbing its immediate dependency. The result yields less implementation specific tests at the cost of making them mini integration tests. This chapter also gave an example on how to stub and test timers and the Date constructor. Having used the Clock object to fake time, we have seen how it would be useful if the Date constructor could somehow be synced with it to more effectively fake time in tests. This chapter concludes our client-side library development for now. The next chapter will use test-driven development to implement the server-side of a long polling application using the node.js framework. From the Library of WoweBook.Com Download from www.eBookTM.com ptg This page intentionally left blank From the Library of WoweBook.Com Download from www.eBookTM.com ptg 14 Server-Side JavaScript with Node.js N etscape pushed JavaScript on the server way back in 1996. Since then, several others have tried to do the same, yet none of these projects have made a big impact on the developer community. That is, until 2009, when Ryan Dahl released the Node.js runtime. At the same time, CommonJS, an attempt at a standard library specification for JavaScript, is rapidly gaining attention and involvement from several server-side JavaScript library authors and users alike. Server-side JavaScript is happening, and it’s going to be big. In this chapter we will use test-driven development to develop a small server-side application using Node. Through this exercise we’ll get to know Node and its con- ventions, work with JavaScript in a more predictable environment than browsers, and draw from our experience with TDD and evented programming from previous chapters to produce the backend of an in-browser chat application that we will finish in the next chapter. 14.1 The Node.js Runtime Node.js—“Evented I/O for V8 JavaScript”—is an evented server-side JavaScript runtime implemented on top of Google’s V8 engine, the same engine that powers Google Chrome. Node uses an event loop and consists almost entirely of asyn- chronous non-blocking API’s, making it a good fit for streaming applications such as those built using Comet or WebSockets. 341 From the Library of WoweBook.Com Download from www.eBookTM.com ptg 342 Server-Side JavaScript with Node.js As we discussed in Chapter13, Streaming Data withAjax and Comet, web servers that allocate one thread per connection, such as Apache httpd, do not scale well in terms of concurrency. Even more so when concurrent connections are long lived. When Node receives a request, it will start listening for certain events, such as data ready from a database, the file system, or a network service. It then goes to sleep. Once the data is ready, events notify the request, which then finishes the connection. This is all seamlessly handled by Node’s event loop. JavaScript developers should feel right at home in Node’s evented world. After all, the browser is evented too, and most JavaScript code is triggered by events. Just take a look at the code we’ve developed throughout this book. In Chapter 10, Feature Detection, we wrote a cross browser way to assign event handlers to DOM elements; in Chapter 11, The Observer Pattern, we wrote a library to observe events on any JavaScript object; and in Chapter 12, Abstracting Browser Differences: Ajax and Chapter 13, Streaming Data with Ajax and Comet, we used callbacks to asyn- chronously fetch data from the server. 14.1.1 Setting up the Environment Setting up Node is pretty straightforward, unless you’re on Windows. Unfortunately, at the time of writing, Node does not run on Windows. It is possible to get it running in Cygwin with some effort, but I think the easiest approach for Windows users is to download and install the free virtualization software VirtualBox 1 and run, e.g., Ubuntu Linux 2 inside it. To install Node, download the source from http://nodejs.org and follow instructions. 14.1.1.1 Directory Structure The project directory structure can be seen in Listing 14.1. Listing 14.1 Initial directory structure chris@laptop:~/projects/chapp$ tree . | deps | lib | ' chapp ' test ' chapp 1. http://www.virtualbox.org/ 2. http://www.ubuntu.com/ From the Library of WoweBook.Com Download from www.eBookTM.com . standard library specification for JavaScript, is rapidly gaining attention and involvement from several server-side JavaScript library authors and users alike. Server-side JavaScript is happening, and it’s. { "Content-Type": "application/json", "X-Access-Token": "" } }); } } For the first request the token will be blank. In a more sophisticated imple- mentation. really belong inside a Comet client; its responsibili- ties are delegating server-side events to client-side observers and publishing client- side events to the server. Ideally the transport would