## Other pages A paper version of Eloquent JavaScript, including a bonus chapter, is being brought out by [No Starch Press](http://www.nostarch.com/ejs2). They also sell an ebook version more polished than the files linked below. * [Code sandbox and exercise solutions](http://eloquentjavascript.net/code) * [Errata for the paper book](http://eloquentjavascript.net/errata.html) * [This book as a single PDF file](http://eloquentjavascript.net/Eloquent_JavaScript.pdf) (& [small version for mobile](http://eloquentjavascript.net/Eloquent_JavaScript_small.pdf)) * [This book as an EPUB file](http://eloquentjavascript.net/Eloquent_JavaScript.epub) * [The first edition of the book](http://eloquentjavascript.net/1st_edition)

## Chapter 21 # Project: Skill-Sharing Website [](http://eloquentjavascript.net/21_skillsharing.html#p_lJLUX5aYTh)A *skill-sharing* meeting is an event where people with a shared interest come together and give small, informal presentations about things they know. At a gardening skill-sharing meeting, someone might explain how to cultivate celery. Or in a programming-oriented skill-sharing group, you could drop by and tell everybody about Node.js. [](http://eloquentjavascript.net/21_skillsharing.html#p_a0XqWGdMH9)Such meetups, also often called *users’ groups* when they are about computers, are a great way to broaden your horizon, learn about new developments, or simply meet people with similar interests. Many large cities have a JavaScript meetup. They are typically free to attend, and I’ve found the ones I’ve visited to be friendly and welcoming. [](http://eloquentjavascript.net/21_skillsharing.html#p_PVL3M81JFR)In this final project chapter, our goal is to set up a website for managing talks given at a skill-sharing meeting. Imagine a small group of people meeting up regularly in a member’s office to talk about unicycling. The problem is that when the previous organizer of the meetings moved to another town, nobody stepped forward to take over this task. We want a system that will let the participants propose and discuss talks among themselves, without a central organizer.  [](http://eloquentjavascript.net/21_skillsharing.html#p_We9gc8FghX)Just like in the [previous chapter](http://eloquentjavascript.net/20_node.html#node), the code in this chapter is written for Node.js, and running it directly in the HTML page that you are looking at is unlikely to work. The full code for the project can be downloaded from[*eloquentjavascript.net/code/skillsharing.zip*](http://eloquentjavascript.net/code/skillsharing.zip). ## [](http://eloquentjavascript.net/21_skillsharing.html#h_WbA1NnIRqT)Design [](http://eloquentjavascript.net/21_skillsharing.html#p_MplsNwNLhr)There is a *server* part to this project, written for Node.js, and a *client* part, written for the browser. The server stores the system’s data and provides it to the client. It also serves the HTML and JavaScript files that implement the client-side system. [](http://eloquentjavascript.net/21_skillsharing.html#p_jXT76qvYdx)The server keeps a list of talks proposed for the next meeting, and the client shows this list. Each talk has a presenter name, a title, a summary, and a list of comments associated with it. The client allows users to propose new talks (adding them to the list), delete talks, and comment on existing talks. Whenever the user makes such a change, the client makes an HTTP request to tell the server about it.  [](http://eloquentjavascript.net/21_skillsharing.html#p_R3HdsDdi40)The application will be set up to show a *live* view of the current proposed talks and their comments. Whenever someone, somewhere, submits a new talk or adds a comment, all people who have the page open in their browsers should immediately see the change. This poses a bit of a challenge since there is no way for a web server to open up a connection to a client, nor is there a good way to know which clients currently are looking at a given website. [](http://eloquentjavascript.net/21_skillsharing.html#p_uVBydEeXpf)A common solution to this problem is called *long polling*, which happens to be one of the motivations for Node’s design. ## [](http://eloquentjavascript.net/21_skillsharing.html#h_Yxu7U155Cs)Long polling [](http://eloquentjavascript.net/21_skillsharing.html#p_UARchLL3as)To be able to immediately notify a client that something changed, we need a connection to that client. Since web browsers do not traditionally accept connections and clients are usually behind devices that would block such connections anyway, having the server initiate this connection is not practical. [](http://eloquentjavascript.net/21_skillsharing.html#p_0nqsjRmsQ7)We can arrange for the client to open the connection and keep it around so that the server can use it to send information when it needs to do so. [](http://eloquentjavascript.net/21_skillsharing.html#p_nPHiUyGtEt)But an HTTP request allows only a simple flow of information, where the client sends a request, the server comes back with a single response, and that is it. There is a technology called *web sockets*, supported by modern browsers, which makes it possible to open connections for arbitrary data exchange. But using them properly is somewhat tricky. [](http://eloquentjavascript.net/21_skillsharing.html#p_h/JHQAq292)In this chapter, we will use a relatively simple technique, long polling, where clients continuously ask the server for new information using regular HTTP requests, and the server simply stalls its answer when it has nothing new to report. [](http://eloquentjavascript.net/21_skillsharing.html#p_jgdXNb8hyw)As long as the client makes sure it constantly has a polling request open, it will receive information from the server immediately. For example, if Alice has our skill-sharing application open in her browser, that browser will have made a request for updates and be waiting for a response to that request. When Bob submits a talk on Extreme Downhill Unicycling, the server will notice that Alice is waiting for updates and send information about the new talk as a response to her pending request. Alice’s browser will receive the data and update the screen to show the talk. [](http://eloquentjavascript.net/21_skillsharing.html#p_SAbyKs9a4Y)To prevent connections from timing out (being aborted because of a lack of activity), long-polling techniques usually set a maximum time for each request, after which the server will respond anyway, even though it has nothing to report, and the client will start a new request. Periodically restarting the request also makes the technique more robust, allowing clients to recover from temporary connection failures or server problems. [](http://eloquentjavascript.net/21_skillsharing.html#p_G3JjyqG0SD)A busy server that is using long polling may have thousands of waiting requests, and thus TCP connections, open. Node, which makes it easy to manage many connections without creating a separate thread of control for each one, is a good fit for such a system. ## [](http://eloquentjavascript.net/21_skillsharing.html#h_a1H+QQ9w0g)HTTP interface [](http://eloquentjavascript.net/21_skillsharing.html#p_pnq2n4wyAS)Before we start fleshing out either the server or the client, let’s think about the point where they touch: the HTTP interface over which they communicate. [](http://eloquentjavascript.net/21_skillsharing.html#p_b+doiaw86m)We will base our interface on JSON, and like in the file server from [Chapter 20](http://eloquentjavascript.net/20_node.html#file_server), we’ll try to make good use of HTTP methods. The interface is centered around the `/talks` path. Paths that do not start with `/talks` will be used for serving static files—the HTML and JavaScript code that implements the client-side system. [](http://eloquentjavascript.net/21_skillsharing.html#p_4DScu49oMT)A `GET` request to `/talks` returns a JSON document like this: ~~~ {"serverTime": 1405438911833, "talks": [{"title": "Unituning", "presenter": "Carlos", "summary": "Modifying your cycle for extra style", "comment": []}]} ~~~ [](http://eloquentjavascript.net/21_skillsharing.html#p_/qoYuRF/YK)The `serverTime` field will be used to make reliable long polling possible. I will return to it [later](http://eloquentjavascript.net/21_skillsharing.html#poll_time). [](http://eloquentjavascript.net/21_skillsharing.html#p_S84lcaMrne)Creating a new talk is done by making a `PUT` request to a URL like`/talks/Unituning`, where the part after the second slash is the title of the talk. The `PUT` request’s body should contain a JSON object that has `presenter`and `summary` properties. [](http://eloquentjavascript.net/21_skillsharing.html#p_KfmhZR585o)Since talk titles may contain spaces and other characters that may not appear normally in a URL, title strings must be encoded with the`encodeURIComponent` function when building up such a URL. ~~~ console.log("/talks/" + encodeURIComponent("How to Idle")); // → /talks/How%20to%20Idle ~~~ [](http://eloquentjavascript.net/21_skillsharing.html#p_lVTXP6zy/G)A request to create a talk about idling might look something like this: ~~~ PUT /talks/How%20to%20Idle HTTP/1.1 Content-Type: application/json Content-Length: 92 {"presenter": "Dana", "summary": "Standing still on a unicycle"} ~~~ [](http://eloquentjavascript.net/21_skillsharing.html#p_MlEec58Mz7)Such URLs also support `GET` requests to retrieve the JSON representation of a talk and `DELETE` requests to delete a talk. [](http://eloquentjavascript.net/21_skillsharing.html#p_cq5rvlZpbo)Adding a comment to a talk is done with a `POST` request to a URL like`/talks/Unituning/comments`, with a JSON object that has `author` and`message` properties as the body of the request. ~~~ POST /talks/Unituning/comments HTTP/1.1 Content-Type: application/json Content-Length: 72 {"author": "Alice", "message": "Will you talk about raising a cycle?"} ~~~ [](http://eloquentjavascript.net/21_skillsharing.html#p_nTbHD0Z9xy)To support long polling, `GET` requests to `/talks` may include a query parameter called `changesSince`, which is used to indicate that the client is interested in updates that happened since a given point in time. When there are such changes, they are immediately returned. When there aren’t, the response is delayed until something happens or until a given time period (we will use 90 seconds) has elapsed. [](http://eloquentjavascript.net/21_skillsharing.html#p_zbkFppI/Py)The time must be indicated as the number of milliseconds elapsed since the start of 1970, the same type of number that is returned by `Date.now()`. To ensure that it receives all updates and doesn’t receive the same update more than once, the client must pass the time at which it last received information from the server. The server’s clock might not be exactly in sync with the client’s clock, and even if it were, it would be impossible for the client to know the precise time at which the server sent a response because transferring data over the network takes time. [](http://eloquentjavascript.net/21_skillsharing.html#p_O0TBchv5t6)This is the reason for the existence of the `serverTime` property in responses sent to `GET` requests to `/talks`. That property tells the client the precise time, from the server’s perspective, at which the data it receives was created. The client can then simply store this time and pass it along in its next polling request to make sure that it receives exactly the updates that it has not seen before. ~~~ GET /talks?changesSince=1405438911833 HTTP/1.1 (time passes) HTTP/1.1 200 OK Content-Type: application/json Content-Length: 95 {"serverTime": 1405438913401, "talks": [{"title": "Unituning", "deleted": true}]} ~~~ [](http://eloquentjavascript.net/21_skillsharing.html#p_IIvPCQ2yb6)When a talk has been changed, has been newly created, or has a comment added, the full representation of the talk is included in the response to the client’s next polling request. When a talk is deleted, only its title and the property `deleted` are included. The client can then add talks with titles it has not seen before to its display, update talks that it was already showing, and remove those that were deleted. [](http://eloquentjavascript.net/21_skillsharing.html#p_7eI6fi/el3)The protocol described in this chapter does not do any access control. Everybody can comment, modify talks, and even delete them. Since the Internet is filled with hooligans, putting such a system online without further protection is likely to end in disaster. [](http://eloquentjavascript.net/21_skillsharing.html#p_dRsn3VsD/D)A simple solution would be to put the system behind a *reverse proxy*, which is an HTTP server that accepts connections from outside the system and forwards them to HTTP servers that are running locally. Such a proxy can be configured to require a username and password, and you could make sure only the participants in the skill-sharing group have this password. ## [](http://eloquentjavascript.net/21_skillsharing.html#h_wTUn7xllPe)The server [](http://eloquentjavascript.net/21_skillsharing.html#p_/iHn83lKOE)Let’s start by writing the server-side part of the program. The code in this section runs on Node.js. ### [](http://eloquentjavascript.net/21_skillsharing.html#h_fRXdG+wuBV)Routing [](http://eloquentjavascript.net/21_skillsharing.html#p_YGBPHzmANZ)Our server will use `http.createServer` to start an HTTP server. In the function that handles a new request, we must distinguish between the various kinds of requests (as determined by the method and the path) that we support. This can be done with a long chain of `if` statements, but there is a nicer way. [](http://eloquentjavascript.net/21_skillsharing.html#p_leW0rMWKfc)A *router* is a component that helps dispatch a request to the function that can handle it. You can tell the router, for example, that `PUT` requests with a path that matches the regular expression `/^\/talks\/([^\/]+)$/` (which matches`/talks/` followed by a talk title) can be handled by a given function. In addition, it can help extract the meaningful parts of the path, in this case the talk title, wrapped in parentheses in the regular expression and pass those to the handler function. [](http://eloquentjavascript.net/21_skillsharing.html#p_l/CAybKBxr)There are a number of good router packages on NPM, but here we will write one ourselves to illustrate the principle. [](http://eloquentjavascript.net/21_skillsharing.html#p_76UeceNiA/)This is `router.js`, which we will later `require` from our server module: ~~~ var Router = module.exports = function() { this.routes = []; }; Router.prototype.add = function(method, url, handler) { this.routes.push({method: method, url: url, handler: handler}); }; Router.prototype.resolve = function(request, response) { var path = require("url").parse(request.url).pathname; return this.routes.some(function(route) { var match = route.url.exec(path); if (!match || route.method != request.method) return false; var urlParts = match.slice(1).map(decodeURIComponent); route.handler.apply(null, [request, response] .concat(urlParts)); return true; }); }; ~~~ [](http://eloquentjavascript.net/21_skillsharing.html#p_4UgvQPflKR)The module exports the `Router` constructor. A router object allows new handlers to be registered with the `add` method and can resolve requests with its `resolve` method. [](http://eloquentjavascript.net/21_skillsharing.html#p_zGaZWr/Iyt)The latter will return a Boolean that indicates whether a handler was found. The `some` method on the array of routes will try the routes one at a time (in the order in which they were defined) and stop, returning `true`, when a matching one is found. [](http://eloquentjavascript.net/21_skillsharing.html#p_OL5KXeAkmE)The handler functions are called with the `request` and `response` objects. When the regular expression that matches the URL contains any groups, the strings they match are passed to the handler as extra arguments. These strings have to be URL-decoded since the raw URL contains `%20`-style codes. ### [](http://eloquentjavascript.net/21_skillsharing.html#h_WmSQ4nnG8k)Serving files [](http://eloquentjavascript.net/21_skillsharing.html#p_G58QvpWZ3S)When a request matches none of the request types defined in our router, the server must interpret it as a request for a file in the `public` directory. It would be possible to use the file server defined in [Chapter 20](http://eloquentjavascript.net/20_node.html#file_server) to serve such files, but we neither need nor want to support `PUT` and `DELETE` requests on files, and we would like to have advanced features such as support for caching. So let’s use a solid, well-tested static file server from NPM instead. [](http://eloquentjavascript.net/21_skillsharing.html#p_tOJqwN8CJl)I opted for `ecstatic`. This isn’t the only such server on NPM, but it works well and fits our purposes. The `ecstatic` module exports a function that can be called with a configuration object to produce a request handler function. We use the `root` option to tell the server where it should look for files. The handler function accepts `request` and `response` parameters and can be passed directly to `createServer` to create a server that serves *only* files. We want to first check for requests that we handle specially, though, so we wrap it in another function. ~~~ var http = require("http"); var Router = require("./router"); var ecstatic = require("ecstatic"); var fileServer = ecstatic({root: "./public"}); var router = new Router(); http.createServer(function(request, response) { if (!router.resolve(request, response)) fileServer(request, response); }).listen(8000); ~~~ [](http://eloquentjavascript.net/21_skillsharing.html#p_aqqI6AlQfN)The `respond` and `respondJSON` helper functions are used throughout the server code to send off responses with a single function call. ~~~ function respond(response, status, data, type) { response.writeHead(status, { "Content-Type": type || "text/plain" }); response.end(data); } function respondJSON(response, status, data) { respond(response, status, JSON.stringify(data), "application/json"); } ~~~ ### [](http://eloquentjavascript.net/21_skillsharing.html#h_soh9hQbmUg)Talks as resources [](http://eloquentjavascript.net/21_skillsharing.html#p_F7tu+Amiqq)The server keeps the talks that have been proposed in an object called `talks`, whose property names are the talk titles. These will be exposed as HTTP resources under `/talks/[title]`, so we need to add handlers to our router that implement the various methods that clients can use to work with them. [](http://eloquentjavascript.net/21_skillsharing.html#p_/JQcLOK+vv)The handler for requests that `GET` a single talk must look up the talk and respond either with the talk’s JSON data or with a 404 error response. ~~~ var talks = Object.create(null); router.add("GET", /^\/talks\/([^\/]+)$/, function(request, response, title) { if (title in talks) respondJSON(response, 200, talks[title]); else respond(response, 404, "No talk '" + title + "' found"); }); ~~~ [](http://eloquentjavascript.net/21_skillsharing.html#p_3zgr/RdzUU)Deleting a talk is done by removing it from the `talks` object. ~~~ router.add("DELETE", /^\/talks\/([^\/]+)$/, function(request, response, title) { if (title in talks) { delete talks[title]; registerChange(title); } respond(response, 204, null); }); ~~~ [](http://eloquentjavascript.net/21_skillsharing.html#p_pyoMvuSjhx)The `registerChange` function, which we will define [later](http://eloquentjavascript.net/21_skillsharing.html#registerChange), notifies waiting long-polling requests about the change. [](http://eloquentjavascript.net/21_skillsharing.html#p_OHqEGZA2TJ)To retrieve the content of JSON-encoded request bodies, we define a function called `readStreamAsJSON`, which reads all content from a stream, parses it as JSON, and then calls a callback function. ~~~ function readStreamAsJSON(stream, callback) { var data = ""; stream.on("data", function(chunk) { data += chunk; }); stream.on("end", function() { var result, error; try { result = JSON.parse(data); } catch (e) { error = e; } callback(error, result); }); stream.on("error", function(error) { callback(error); }); } ~~~ [](http://eloquentjavascript.net/21_skillsharing.html#p_nKzpVkDH6v)One handler that needs to read JSON responses is the `PUT` handler, which is used to create new talks. It has to check whether the data it was given has`presenter` and `summary` properties, which are strings. Any data coming from outside the system might be nonsense, and we don’t want to corrupt our internal data model, or even crash, when bad requests come in. [](http://eloquentjavascript.net/21_skillsharing.html#p_ptAYx9j8oM)If the data looks valid, the handler stores an object that represents the new talk in the `talks` object, possibly overwriting an existing talk with this title, and again calls `registerChange`. ~~~ router.add("PUT", /^\/talks\/([^\/]+)$/, function(request, response, title) { readStreamAsJSON(request, function(error, talk) { if (error) { respond(response, 400, error.toString()); } else if (!talk || typeof talk.presenter != "string" || typeof talk.summary != "string") { respond(response, 400, "Bad talk data"); } else { talks[title] = {title: title, presenter: talk.presenter, summary: talk.summary, comments: []}; registerChange(title); respond(response, 204, null); } }); }); ~~~ [](http://eloquentjavascript.net/21_skillsharing.html#p_5d2itdrhai)Adding a comment to a talk works similarly. We use `readStreamAsJSON` to get the content of the request, validate the resulting data, and store it as a comment when it looks valid. ~~~ router.add("POST", /^\/talks\/([^\/]+)\/comments$/, function(request, response, title) { readStreamAsJSON(request, function(error, comment) { if (error) { respond(response, 400, error.toString()); } else if (!comment || typeof comment.author != "string" || typeof comment.message != "string") { respond(response, 400, "Bad comment data"); } else if (title in talks) { talks[title].comments.push(comment); registerChange(title); respond(response, 204, null); } else { respond(response, 404, "No talk '" + title + "' found"); } }); }); ~~~ [](http://eloquentjavascript.net/21_skillsharing.html#p_uLZ9tBoiGb)Trying to add a comment to a nonexistent talk should return a 404 error, of course. ### [](http://eloquentjavascript.net/21_skillsharing.html#h_kDNUlKj7rv)Long-polling support [](http://eloquentjavascript.net/21_skillsharing.html#p_Q2AlAKJnYq)The most interesting aspect of the server is the part that handles long polling. When a `GET` request comes in for `/talks`, it can be either a simple request for all talks or a request for updates, with a `changesSince` parameter. [](http://eloquentjavascript.net/21_skillsharing.html#p_GhbQTw6SZ+)There will be various situations in which we have to send a list of talks to the client, so we first define a small helper function that attaches the `serverTime`field to such responses. ~~~ function sendTalks(talks, response) { respondJSON(response, 200, { serverTime: Date.now(), talks: talks }); } ~~~ [](http://eloquentjavascript.net/21_skillsharing.html#p_EltkhTQ54Y)The handler itself needs to look at the query parameters in the request’s URL to see whether a `changesSince` parameter is given. If you give the `"url"`module’s `parse` function a second argument of `true`, it will also parse the query part of a URL. The object it returns will have a `query` property, which holds another object that maps parameter names to values. ~~~ router.add("GET", /^\/talks$/, function(request, response) { var query = require("url").parse(request.url, true).query; if (query.changesSince == null) { var list = []; for (var title in talks) list.push(talks[title]); sendTalks(list, response); } else { var since = Number(query.changesSince); if (isNaN(since)) { respond(response, 400, "Invalid parameter"); } else { var changed = getChangedTalks(since); if (changed.length > 0) sendTalks(changed, response); else waitForChanges(since, response); } } }); ~~~ [](http://eloquentjavascript.net/21_skillsharing.html#p_0gOtQ9gRNI)When the `changesSince` parameter is missing, the handler simply builds up a list of all talks and returns that. [](http://eloquentjavascript.net/21_skillsharing.html#p_i4f8K0rlnB)Otherwise, the `changeSince` parameter first has to be checked to make sure that it is a valid number. The `getChangedTalks` function, to be defined shortly, returns an array of changed talks since a given point in time. If it returns an empty array, the server does not yet have anything to send back to the client, so it stores the response object (using `waitForChanges`) to be responded to at a later time. ~~~ var waiting = []; function waitForChanges(since, response) { var waiter = {since: since, response: response}; waiting.push(waiter); setTimeout(function() { var found = waiting.indexOf(waiter); if (found > -1) { waiting.splice(found, 1); sendTalks([], response); } }, 90 * 1000); } ~~~ [](http://eloquentjavascript.net/21_skillsharing.html#p_KJPecJ2C8w)The `splice` method is used to cut a piece out of an array. You give it an index and a number of elements, and it *mutates* the array, removing that many elements after the given index. In this case, we remove a single element, the object that tracks the waiting response, whose index we found by calling`indexOf`. If you pass additional arguments to `splice`, their values will be inserted into the array at the given position, replacing the removed elements. [](http://eloquentjavascript.net/21_skillsharing.html#p_33ov2uS0wd)When a response object is stored in the `waiting` array, a timeout is immediately set. After 90 seconds, this timeout sees whether the request is still waiting and, if it is, sends an empty response and removes it from the `waiting`array. [](http://eloquentjavascript.net/21_skillsharing.html#p_w3fgYDXBML)To be able to find exactly those talks that have been changed since a given point in time, we need to keep track of the history of changes. Registering a change with `registerChange` will remember that change, along with the current time, in an array called `changes`. When a change occurs, that means there is new data, so all waiting requests can be responded to immediately. ~~~ var changes = []; function registerChange(title) { changes.push({title: title, time: Date.now()}); waiting.forEach(function(waiter) { sendTalks(getChangedTalks(waiter.since), waiter.response); }); waiting = []; } ~~~ [](http://eloquentjavascript.net/21_skillsharing.html#p_IokNlBSqeg)Finally, `getChangedTalks` uses the `changes` array to build up an array of changed talks, including objects with a `deleted` property for talks that no longer exist. When building that array, `getChangedTalks` has to ensure that it doesn’t include the same talk twice since there might have been multiple changes to a talk since the given time. ~~~ function getChangedTalks(since) { var found = []; function alreadySeen(title) { return found.some(function(f) {return f.title == title;}); } for (var i = changes.length - 1; i >= 0; i--) { var change = changes[i]; if (change.time <= since) break; else if (alreadySeen(change.title)) continue; else if (change.title in talks) found.push(talks[change.title]); else found.push({title: change.title, deleted: true}); } return found; } ~~~ [](http://eloquentjavascript.net/21_skillsharing.html#p_3rLxFmwzJp)That concludes the server code. Running the program defined so far will get you a server running on port 8000, which serves files from the `public`subdirectory alongside a talk-managing interface under the `/talks` URL. ## [](http://eloquentjavascript.net/21_skillsharing.html#h_+jMNtbJ4U3)The client [](http://eloquentjavascript.net/21_skillsharing.html#p_wVvjBUA/vN)The client-side part of the talk-managing website consists of three files: an HTML page, a style sheet, and a JavaScript file. ### [](http://eloquentjavascript.net/21_skillsharing.html#h_n3OM6EV/KR)HTML [](http://eloquentjavascript.net/21_skillsharing.html#p_jNP5xbaXQN)It is a widely used convention for web servers to try to serve a file named`index.html` when a request is made directly to a path that corresponds to a directory. The file server module we use, `ecstatic`, supports this convention. When a request is made to the path `/`, the server looks for the file`./public/index.html` (`./public` being the root we gave it) and returns that file if found. [](http://eloquentjavascript.net/21_skillsharing.html#p_17VtIBLbQE)Thus, if we want a page to show up when a browser is pointed at our server, we should put it in `public/index.html`. This is how our index file starts: ~~~ <!doctype html> <title>Skill Sharing</title> <link rel="stylesheet" href="skillsharing.css"> <h1>Skill sharing</h1> <p>Your name: <input type="text" id="name"></p> <div id="talks"></div> ~~~ [](http://eloquentjavascript.net/21_skillsharing.html#p_CbpqVP4HI4)It defines the document title and includes a style sheet, which defines a few styles to, among other things, add a border around talks. Then it adds a heading and a name field. The user is expected to put their name in the latter so that it can be attached to talks and comments they submit. [](http://eloquentjavascript.net/21_skillsharing.html#p_AyaILfiByw)The `<div>` element with the ID `"talks"` will contain the current list of talks. The script fills the list in when it receives talks from the server. [](http://eloquentjavascript.net/21_skillsharing.html#p_Y7s6DwG+9R)Next comes the form that is used to create a new talk. ~~~ <form id="newtalk"> <h3>Submit a talk</h3> Title: <input type="text" style="width: 40em" name="title"> <br> Summary: <input type="text" style="width: 40em" name="summary"> <button type="submit">Send</button> </form> ~~~ [](http://eloquentjavascript.net/21_skillsharing.html#p_SY0ZWXkS+z)The script will add a `"submit"` event handler to this form, from which it can make the HTTP request that tells the server about the talk. [](http://eloquentjavascript.net/21_skillsharing.html#p_01rGKtjDzP)Next comes a rather mysterious block, which has its `display` style set to `none`, preventing it from actually showing up on the page. Can you guess what it is for? ~~~ <div id="template" style="display: none"> <div class="talk"> <h2>{{title}}</h2> <div>by <span class="name">{{presenter}}</span></div> <p>{{summary}}</p> <div class="comments"></div> <form> <input type="text" name="comment"> <button type="submit">Add comment</button> <button type="button" class="del">Delete talk</button> </form> </div> <div class="comment"> <span class="name">{{author}}</span>: {{message}} </div> </div> ~~~ [](http://eloquentjavascript.net/21_skillsharing.html#p_nUrh268UuK)Creating complicated DOM structures with JavaScript code produces ugly code. You can make the code slightly better by introducing helper functions like the `elt` function from [Chapter 13](http://eloquentjavascript.net/13_dom.html#elt), but the result will still look worse than HTML, which can be thought of as a domain-specific language for expressing DOM structures. [](http://eloquentjavascript.net/21_skillsharing.html#p_+UxIQCbmjS)To create DOM structures for the talks, our program will define a simple*templating* system, which uses hidden DOM structures included in the document to instantiate new DOM structures, replacing the placeholders between double braces with the values of a specific talk. [](http://eloquentjavascript.net/21_skillsharing.html#p_sC79xfZRJT)Finally, the HTML document includes the script file that contains the client-side code. ~~~ <script src="skillsharing_client.js"></script> ~~~ ### [](http://eloquentjavascript.net/21_skillsharing.html#h_mboAXnxq/u)Starting up [](http://eloquentjavascript.net/21_skillsharing.html#p_ZvRC5GfFJX)The first thing the client has to do when the page is loaded is ask the server for the current set of talks. Since we are going to make a lot of HTTP requests, we will again define a small wrapper around `XMLHttpRequest`, which accepts an object to configure the request as well as a callback to call when the request finishes. ~~~ function request(options, callback) { var req = new XMLHttpRequest(); req.open(options.method || "GET", options.pathname, true); req.addEventListener("load", function() { if (req.status < 400) callback(null, req.responseText); else callback(new Error("Request failed: " + req.statusText)); }); req.addEventListener("error", function() { callback(new Error("Network error")); }); req.send(options.body || null); } ~~~ [](http://eloquentjavascript.net/21_skillsharing.html#p_+tZz/acy91)The initial request displays the talks it receives on the screen and starts the long-polling process by calling `waitForChanges`. ~~~ var lastServerTime = 0; request({pathname: "talks"}, function(error, response) { if (error) { reportError(error); } else { response = JSON.parse(response); displayTalks(response.talks); lastServerTime = response.serverTime; waitForChanges(); } }); ~~~ [](http://eloquentjavascript.net/21_skillsharing.html#p_x9cXc2plZU)The `lastServerTime` variable is used to track the time of the last update that was received from the server. After the initial request, the client’s view of the talks corresponds to the view that the server had when it responded to that request. Thus, the `serverTime` property included in the response provides an appropriate initial value for `lastServerTime`. [](http://eloquentjavascript.net/21_skillsharing.html#p_gwDxefJOAx)When the request fails, we don’t want to have our page just sit there, doing nothing without explanation. So we define a simple function called`reportError`, which at least shows the user a dialog that tells them something went wrong. ~~~ function reportError(error) { if (error) alert(error.toString()); } ~~~ [](http://eloquentjavascript.net/21_skillsharing.html#p_CAD2u101IA)The function checks whether there *is* an actual error, and it alerts only when there is one. That way, we can also directly pass this function to `request` for requests where we can ignore the response. This makes sure that if the request fails, the error is reported to the user. ### [](http://eloquentjavascript.net/21_skillsharing.html#h_KV4ik1gcBy)Displaying talks [](http://eloquentjavascript.net/21_skillsharing.html#p_ABlg+B4JoH)To be able to update the view of the talks when changes come in, the client must keep track of the talks that it is currently showing. That way, when a new version of a talk that is already on the screen comes in, the talk can be replaced (in place) with its updated form. Similarly, when information comes in that a talk is being deleted, the right DOM element can be removed from the document. [](http://eloquentjavascript.net/21_skillsharing.html#p_HX4fJQ/YuG)The function `displayTalks` is used both to build up the initial display and to update it when something changes. It will use the `shownTalks` object, which associates talk titles with DOM nodes, to remember the talks it currently has on the screen. ~~~ var talkDiv = document.querySelector("#talks"); var shownTalks = Object.create(null); function displayTalks(talks) { talks.forEach(function(talk) { var shown = shownTalks[talk.title]; if (talk.deleted) { if (shown) { talkDiv.removeChild(shown); delete shownTalks[talk.title]; } } else { var node = drawTalk(talk); if (shown) talkDiv.replaceChild(node, shown); else talkDiv.appendChild(node); shownTalks[talk.title] = node; } }); } ~~~ [](http://eloquentjavascript.net/21_skillsharing.html#p_eNjQj7FZqE)Building up the DOM structure for talks is done using the templates that were included in the HTML document. First, we must define`instantiateTemplate`, which looks up and fills in a template. [](http://eloquentjavascript.net/21_skillsharing.html#p_QzVkBI26S7)The `name` parameter is the template’s name. To look up the template element, we search for an element whose class name matches the template name, which is a child of the element with ID `"template"`. Using the `querySelector`method makes this easy. There were templates named `"talk"` and `"comment"`in the HTML page. ~~~ function instantiateTemplate(name, values) { function instantiateText(text) { return text.replace(/\{\{(\w+)\}\}/g, function(_, name) { return values[name]; }); } function instantiate(node) { if (node.nodeType == document.ELEMENT_NODE) { var copy = node.cloneNode(); for (var i = 0; i < node.childNodes.length; i++) copy.appendChild(instantiate(node.childNodes[i])); return copy; } else if (node.nodeType == document.TEXT_NODE) { return document.createTextNode( instantiateText(node.nodeValue)); } else { return node; } } var template = document.querySelector("#template ." + name); return instantiate(template); } ~~~ [](http://eloquentjavascript.net/21_skillsharing.html#p_/nQdGhe85J)The `cloneNode` method, which all DOM nodes have, creates a copy of a node. It won’t copy the node’s child nodes unless `true` is given as a first argument. The `instantiate` function recursively builds up a copy of the template, filling in the template as it goes. [](http://eloquentjavascript.net/21_skillsharing.html#p_DrLRp8KMhs)The second argument to `instantiateTemplate` should be an object, whose properties hold the strings that are to be filled into the template. A placeholder like `{{title}}` will be replaced with the value of `values`’ `title` property. [](http://eloquentjavascript.net/21_skillsharing.html#p_QeRff45qDA)This is a crude approach to templating, but it is enough to implement`drawTalk`. ~~~ function drawTalk(talk) { var node = instantiateTemplate("talk", talk); var comments = node.querySelector(".comments"); talk.comments.forEach(function(comment) { comments.appendChild( instantiateTemplate("comment", comment)); }); node.querySelector("button.del").addEventListener( "click", deleteTalk.bind(null, talk.title)); var form = node.querySelector("form"); form.addEventListener("submit", function(event) { event.preventDefault(); addComment(talk.title, form.elements.comment.value); form.reset(); }); return node; } ~~~ [](http://eloquentjavascript.net/21_skillsharing.html#p_wkegcUE+fb)After instantiating the `"talk"` template, there are various things that need to be patched up. First, the comments have to be filled in by repeatedly instantiating the `"comment"` template and appending the results to the node with class `"comments"`. Next, event handlers have to be attached to the button that deletes the task and the form that adds a new comment. ### [](http://eloquentjavascript.net/21_skillsharing.html#h_Fxf3YQJcAP)Updating the server [](http://eloquentjavascript.net/21_skillsharing.html#p_+djWXXL4a7)The event handlers registered by `drawTalk` call the function `deleteTalk` and`addComment` to perform the actual actions required to delete a talk or add a comment. These will need to build up URLs that refer to talks with a given title, for which we define the `talkURL` helper function. ~~~ function talkURL(title) { return "talks/" + encodeURIComponent(title); } ~~~ [](http://eloquentjavascript.net/21_skillsharing.html#p_2F7+HjPRJf)The `deleteTalk` function fires off a `DELETE` request and reports the error when that fails. ~~~ function deleteTalk(title) { request({pathname: talkURL(title), method: "DELETE"}, reportError); } ~~~ [](http://eloquentjavascript.net/21_skillsharing.html#p_lPxu6YmNuw)Adding a comment requires building up a JSON representation of the comment and submitting that as part of a `POST` request. ~~~ function addComment(title, comment) { var comment = {author: nameField.value, message: comment}; request({pathname: talkURL(title) + "/comments", body: JSON.stringify(comment), method: "POST"}, reportError); } ~~~ [](http://eloquentjavascript.net/21_skillsharing.html#p_0lktU89vVm)The `nameField` variable used to set the comment’s `author` property is a reference to the `<input>` field at the top of the page that allows the user to specify their name. We also wire up that field to `localStorage` so that it does not have to be filled in again every time the page is reloaded. ~~~ var nameField = document.querySelector("#name"); nameField.value = localStorage.getItem("name") || ""; nameField.addEventListener("change", function() { localStorage.setItem("name", nameField.value); }); ~~~ [](http://eloquentjavascript.net/21_skillsharing.html#p_UkPG5Nhwu/)The form at the bottom of the page, for proposing a new talk, gets a `"submit"`event handler. This handler prevents the event’s default effect (which would cause a page reload), clears the form, and fires off a `PUT` request to create the talk. ~~~ var talkForm = document.querySelector("#newtalk"); talkForm.addEventListener("submit", function(event) { event.preventDefault(); request({pathname: talkURL(talkForm.elements.title.value), method: "PUT", body: JSON.stringify({ presenter: nameField.value, summary: talkForm.elements.summary.value })}, reportError); talkForm.reset(); }); ~~~ ### [](http://eloquentjavascript.net/21_skillsharing.html#h_2EFD142D/X)Noticing changes [](http://eloquentjavascript.net/21_skillsharing.html#p_iWkpEvrdoJ)I should point out that the various functions that change the state of the application by creating or deleting talks or adding a comment do absolutely nothing to ensure that the changes they make are visible on the screen. They simply tell the server and rely on the long-polling mechanism to trigger the appropriate updates to the page. [](http://eloquentjavascript.net/21_skillsharing.html#p_xdAPc8BwNG)Given the mechanism that we implemented in our server and the way we defined `displayTalks` to handle updates of talks that are already on the page, the actual long polling is surprisingly simple. ~~~ function waitForChanges() { request({pathname: "talks?changesSince=" + lastServerTime}, function(error, response) { if (error) { setTimeout(waitForChanges, 2500); console.error(error.stack); } else { response = JSON.parse(response); displayTalks(response.talks); lastServerTime = response.serverTime; waitForChanges(); } }); } ~~~ [](http://eloquentjavascript.net/21_skillsharing.html#p_pB96H5tQ0A)This function is called once when the program starts up and then keeps calling itself to ensure that a polling request is always active. When the request fails, we don’t call `reportError` since popping up a dialog every time we fail to reach the server would get annoying when the server is down. Instead, the error is written to the console (to ease debugging), and another attempt is made 2.5 seconds later. [](http://eloquentjavascript.net/21_skillsharing.html#p_QNpe4g2kOS)When the request succeeds, the new data is put onto the screen, and`lastServerTime` is updated to reflect the fact that we received data corresponding to this new point in time. The request is immediately restarted to wait for the next update. [](http://eloquentjavascript.net/21_skillsharing.html#p_zfJgnDyZM2)If you run the server and open two browser windows for [*localhost:8000/*](http://localhost:8000/) next to each other, you can see that the actions you perform in one window are immediately visible in the other. ## [](http://eloquentjavascript.net/21_skillsharing.html#h_TcUD2vzyMe)Exercises [](http://eloquentjavascript.net/21_skillsharing.html#p_RTbqFFPYXr)The following exercises will involve modifying the system defined in this chapter. To work on them, make sure you download the code first ([*eloquentjavascript.net/code/skillsharing.zip*](http://eloquentjavascript.net/code/skillsharing.zip)) and have Node installed ([*nodejs.org*](http://nodejs.org/)). ### [](http://eloquentjavascript.net/21_skillsharing.html#h_QcUCZfnLE+)Disk persistence [](http://eloquentjavascript.net/21_skillsharing.html#p_wtP8pzYaoa)The skill-sharing server keeps its data purely in memory. This means that when it crashes or is restarted for any reason, all talks and comments are lost. [](http://eloquentjavascript.net/21_skillsharing.html#p_3H2K0biSmc)Extend the server so that it stores the talk data to disk and automatically reloads the data when it is restarted. Do not worry about efficiency—do the simplest thing that works. ### [](http://eloquentjavascript.net/21_skillsharing.html#h_oMIXw3b5pk)Comment field resets [](http://eloquentjavascript.net/21_skillsharing.html#p_ZLgsWCWv6a)The wholesale redrawing of talks works pretty well because you usually can’t tell the difference between a DOM node and its identical replacement. But there are exceptions. If you start typing something in the comment field for a talk in one browser window and then, in another, add a comment to that talk, the field in the first window will be redrawn, removing both its content and its focus. [](http://eloquentjavascript.net/21_skillsharing.html#p_y5nWDjEiiD)In a heated discussion, where multiple people are adding comments to a single talk, this would be very annoying. Can you come up with a way to avoid it? ### [](http://eloquentjavascript.net/21_skillsharing.html#h_mAO3w3FVBR)Better templates [](http://eloquentjavascript.net/21_skillsharing.html#p_/QOXJknIo1)Most templating systems do more than just fill in some strings. At the very least, they also allow conditional inclusion of parts of the template, analogous to `if` statements, and repetition of parts of a template, similar to a loop. [](http://eloquentjavascript.net/21_skillsharing.html#p_zIsRVUDVAg)If we were able to repeat a piece of template for each element in an array, we would not need the second template (`"comment"`). Rather, we could specify the`"talk"` template to loop over the array held in a talk’s `comments` property and render the nodes that make up a comment for every element in the array. [](http://eloquentjavascript.net/21_skillsharing.html#p_2qPMlaQA9i)It could look like this: ~~~ <div class="comments"> <div class="comment" template-repeat="comments"> <span class="name">{{author}}</span>: {{message}} </div> </div> ~~~ [](http://eloquentjavascript.net/21_skillsharing.html#p_BMAO7dW/5p)The idea is that whenever a node with a `template-repeat` attribute is found during template instantiation, the instantiating code loops over the array held in the property named by that attribute. For each element in the array, it adds an instance of the node. The template’s context (the `values` variable in`instantiateTemplate`) would, during this loop, point at the current element of the array so that `{{author}}` would be looked up in the comment object rather than in the original context (the talk). [](http://eloquentjavascript.net/21_skillsharing.html#p_8WQZs9CwPa)Rewrite `instantiateTemplate` to implement this and then change the templates to use this feature and remove the explicit rendering of comments from the `drawTalk` function. [](http://eloquentjavascript.net/21_skillsharing.html#p_0HE7T7eefT)How would you add conditional instantiation of nodes, making it possible to omit parts of the template when a given value is true or false? ### [](http://eloquentjavascript.net/21_skillsharing.html#h_V+TcFyuz/v)The unscriptables [](http://eloquentjavascript.net/21_skillsharing.html#p_pTLwmwvGb9)When someone visits our website with a browser that has JavaScript disabled or is simply not capable of displaying JavaScript, they will get a completely broken, inoperable page. This is not nice. [](http://eloquentjavascript.net/21_skillsharing.html#p_17cAgFKUum)Some types of web applications really can’t be done without JavaScript. For others, you just don’t have the budget or patience to bother about clients that can’t run scripts. But for pages with a wide audience, it is polite to support scriptless users. [](http://eloquentjavascript.net/21_skillsharing.html#p_dFmv2TkNrB)Try to think of a way the skill-sharing website could be set up to preserve basic functionality when run without JavaScript. The automatic updates will have to go, and people will have to refresh their page the old-fashioned way. But being able to see existing talks, create new ones, and submit comments would be nice. [](http://eloquentjavascript.net/21_skillsharing.html#p_d0wQ5C75Bk)Don’t feel obliged to actually implement this. Outlining a solution is enough. Does the revised approach strike you as more or less elegant than what we did initially? Two central aspects of the approach taken in this chapter—a clean HTTP interface and client-side template rendering—don’t work without JavaScript. Normal HTML forms can send `GET` and `POST` requests but not `PUT` or `DELETE`requests and can send their data only to a fixed URL. [](http://eloquentjavascript.net/21_skillsharing.html#p_4JCtv9/G+Q)Thus, the server would have to be revised to accept comments, new talks, and deleted talks through `POST` requests, whose bodies aren’t JSON but rather use the URL-encoded format that HTML forms use (see [Chapter 17](http://eloquentjavascript.net/18_forms.html#forms)). These requests would have to return the full new page so that users see the new state of the site after they make a change. This would not be too hard to engineer and could be implemented alongside the “clean” HTTP interface. [](http://eloquentjavascript.net/21_skillsharing.html#p_BpHJYWubwA)The code for rendering talks would have to be duplicated on the server. The`index.html` file, rather than being a static file, would have to be generated dynamically by adding a handler for it to the router. That way, it already includes the current talks and comments when it gets served.

## Chapter 20 # Node.js > [](http://eloquentjavascript.net/20_node.html#p_+0XsOMiWpx)A student asked ‘The programmers of old used only simple machines and no programming languages, yet they made beautiful programs. Why do we use complicated machines and programming languages?’. Fu-Tzu replied ‘The builders of old used only sticks and clay, yet they made beautiful huts.’ > > Master Yuan-Ma, The Book of Programming [](http://eloquentjavascript.net/20_node.html#p_f4N5IT0jKt)So far, you have learned the JavaScript language and used it within a single environment: the browser. This chapter and the [next one](http://eloquentjavascript.net/21_skillsharing.html#skillsharing) will briefly introduce you to Node.js, a program that allows you to apply your JavaScript skills outside of the browser. With it, you can build anything from simple command-line tools to dynamic HTTP servers. [](http://eloquentjavascript.net/20_node.html#p_Dt2jHJJhuW)These chapters aim to teach you the important ideas that Node.js builds on and to give you enough information to write some useful programs for it. They do not try to be a complete, or even a thorough, treatment of Node. [](http://eloquentjavascript.net/20_node.html#p_NM5XbgzOCu)Whereas you could run the code in previous chapters directly on these pages, since it was either raw JavaScript or written for the browser, the code samples in this chapter are written for Node and won’t run in the browser. [](http://eloquentjavascript.net/20_node.html#p_Bc2F03Fwki)If you want to follow along and run the code in this chapter, start by going to[*nodejs.org*](http://nodejs.org/) and following the installation instructions for your operating system. Also refer to that website for further documentation about Node and its built-in modules. ## [](http://eloquentjavascript.net/20_node.html#h_ZN1g/hoEn+)Background [](http://eloquentjavascript.net/20_node.html#p_I4yhs5+yBu)One of the more difficult problems with writing systems that communicate over the network is managing input and output—that is, the reading and writing of data to and from the network, the hard drive, and other such devices. Moving data around takes time, and scheduling it cleverly can make a big difference in how quickly a system responds to the user or to network requests. [](http://eloquentjavascript.net/20_node.html#p_w4Myb9Rs0P)The traditional way to handle input and output is to have a function, such as`readFile`, start reading a file and return only when the file has been fully read. This is called *synchronous I/O* (I/O stands for input/output). [](http://eloquentjavascript.net/20_node.html#p_1PH8PTd2Ht)Node was initially conceived for the purpose of making *asynchronous* I/O easy and convenient. We have seen asynchronous interfaces before, such as a browser’s `XMLHttpRequest` object, discussed in [Chapter 17](http://eloquentjavascript.net/17_http.html#xmlhttprequest). An asynchronous interface allows the script to continue running while it does its work and calls a callback function when it’s done. This is the way Node does all its I/O. [](http://eloquentjavascript.net/20_node.html#p_tBCEeUjRdi)JavaScript lends itself well to a system like Node. It is one of the few programming languages that does not have a built-in way to do I/O. Thus, JavaScript could be fit onto Node’s rather eccentric approach to I/O without ending up with two inconsistent interfaces. In 2009, when Node was being designed, people were already doing callback-based I/O in the browser, so the community around the language was used to an asynchronous programming style. ## [](http://eloquentjavascript.net/20_node.html#h_HH3wvnWMnd)Asynchronicity [](http://eloquentjavascript.net/20_node.html#p_8dB4v8+vkP)I’ll try to illustrate synchronous versus asynchronous I/O with a small example, where a program needs to fetch two resources from the Internet and then do some simple processing with the result. [](http://eloquentjavascript.net/20_node.html#p_USoDXq6W5R)In a synchronous environment, the obvious way to perform this task is to make the requests one after the other. This method has the drawback that the second request will be started only when the first has finished. The total time taken will be at least the sum of the two response times. This is not an effective use of the machine, which will be mostly idle when it is transmitting and receiving data over the network. [](http://eloquentjavascript.net/20_node.html#p_GzbDIalJ5Y)The solution to this problem, in a synchronous system, is to start additional threads of control. (Refer to [Chapter 14](http://eloquentjavascript.net/14_event.html#timeline) for a previous discussion of threads.) A second thread could start the second request, and then both threads wait for their results to come back, after which they resynchronize to combine their results. [](http://eloquentjavascript.net/20_node.html#p_EWjYy77pGQ)In the following diagram, the thick lines represent time the program spends running normally, and the thin lines represent time spent waiting for I/O. In the synchronous model, the time taken by I/O is *part* of the timeline for a given thread of control. In the asynchronous model, starting an I/O action conceptually causes a *split* in the timeline. The thread that initiated the I/O continues running, and the I/O itself is done alongside it, finally calling a callback function when it is finished.  [](http://eloquentjavascript.net/20_node.html#p_RmZacjuEJP)Another way to express this difference is that waiting for I/O to finish is*implicit* in the synchronous model, while it is *explicit*, directly under our control, in the asynchronous one. But asynchronicity cuts both ways. It makes expressing programs that do not fit the straight-line model of control easier, but it also makes expressing programs that do follow a straight line more awkward. [](http://eloquentjavascript.net/20_node.html#p_OojnwsVJab)In [Chapter 17](http://eloquentjavascript.net/17_http.html#promises), I already touched on the fact that all those callbacks add quite a lot of noise and indirection to a program. Whether this style of asynchronicity is a good idea in general can be debated. In any case, it takes some getting used to. [](http://eloquentjavascript.net/20_node.html#p_rvRV3gJ7Vm)But for a JavaScript-based system, I would argue that callback-style asynchronicity is a sensible choice. One of the strengths of JavaScript is its simplicity, and trying to add multiple threads of control to it would add a lot of complexity. Though callbacks don’t tend to lead to simple *code*, as a *concept*, they’re pleasantly simple yet powerful enough to write high-performance web servers. ## [](http://eloquentjavascript.net/20_node.html#h_TUzbi7lU/0)The node command [](http://eloquentjavascript.net/20_node.html#p_rE0vPeaAdk)When Node.js is installed on a system, it provides a program called `node`, which is used to run JavaScript files. Say you have a file `hello.js`, containing this code: ~~~ var message = "Hello world"; console.log(message); ~~~ [](http://eloquentjavascript.net/20_node.html#p_AQLpVQwSlg)You can then run `node` from the command line like this to execute the program: ~~~ $ node hello.js Hello world ~~~ [](http://eloquentjavascript.net/20_node.html#p_4itSQ4XQYe)The `console.log` method in Node does something similar to what it does in the browser. It prints out a piece of text. But in Node, the text will go to the process’ standard output stream, rather than to a browser’s JavaScript console. [](http://eloquentjavascript.net/20_node.html#p_0xbG1KLur9)If you run `node` without giving it a file, it provides you with a prompt at which you can type JavaScript code and immediately see the result. ~~~ $ node > 1 + 1 2 > [-1, -2, -3].map(Math.abs) [1, 2, 3] > process.exit(0) $ ~~~ [](http://eloquentjavascript.net/20_node.html#p_HtWWK8Hi6k)The `process` variable, just like the `console` variable, is available globally in Node. It provides various ways to inspect and manipulate the current program. The `exit` method ends the process and can be given an exit status code, which tells the program that started `node` (in this case, the command-line shell) whether the program completed successfully (code zero) or encountered an error (any other code). [](http://eloquentjavascript.net/20_node.html#p_r76dNN6hR1)To find the command-line arguments given to your script, you can read`process.argv`, which is an array of strings. Note that it also includes the name of the `node` commands and your script name, so the actual arguments start at index 2\. If `showargv.js` simply contains the statement`console.log(process.argv)`, you could run it like this: ~~~ $ node showargv.js one --and two ["node", "/home/marijn/showargv.js", "one", "--and", "two"] ~~~ [](http://eloquentjavascript.net/20_node.html#p_sAXeLpUyyx)All the standard JavaScript global variables, such as `Array`, `Math`, and `JSON`, are also present in Node’s environment. Browser-related functionality, such as`document` and `alert`, is absent. [](http://eloquentjavascript.net/20_node.html#p_ZyBIfWzBLo)The global scope object, which is called `window` in the browser, has the more sensible name `global` in Node. ## [](http://eloquentjavascript.net/20_node.html#h_BOlGLA/wK7)Modules [](http://eloquentjavascript.net/20_node.html#p_kjExgCix3i)Beyond the few variables I mentioned, such as `console` and `process`, Node puts little functionality in the global scope. If you want to access other built-in functionality, you have to ask the module system for it. [](http://eloquentjavascript.net/20_node.html#p_hr2Rgijb8q)The CommonJS module system, based on the `require` function, was described in [Chapter 10](http://eloquentjavascript.net/10_modules.html#commonjs). This system is built into Node and is used to load anything from built-in modules to downloaded libraries to files that are part of your own program. [](http://eloquentjavascript.net/20_node.html#p_XAhBxX8ROy)When `require` is called, Node has to resolve the given string to an actual file to load. Pathnames that start with `"/"`, `"./"`, or `"../"` are resolved relative to the current module’s path, where `"./"` stands for the current directory, `"../"`for one directory up, and `"/"` for the root of the file system. So if you ask for`"./world/world"` from the file `/home/marijn/elife/run.js`, Node will try to load the file `/home/marijn/elife/world/world.js`. The `.js` extension may be omitted. [](http://eloquentjavascript.net/20_node.html#p_Jbv0K2DaVF)When a string that does not look like a relative or absolute path is given to`require`, it is assumed to refer to either a built-in module or a module installed in a `node_modules` directory. For example, `require("fs")` will give you Node’s built-in file system module, and `require("elife")` will try to load the library found in `node_modules/elife/`. A common way to install such libraries is by using NPM, which I will discuss in a moment. [](http://eloquentjavascript.net/20_node.html#p_5Ya065Hb/J)To illustrate the use of `require`, let’s set up a simple project consisting of two files. The first one is called `main.js`, which defines a script that can be called from the command line to garble a string. ~~~ var garble = require("./garble"); // Index 2 holds the first actual command-line argument var argument = process.argv[2]; console.log(garble(argument)); ~~~ [](http://eloquentjavascript.net/20_node.html#p_2+5gg4XOlc)The file `garble.js` defines a library for garbling strings, which can be used both by the command-line tool defined earlier and by other scripts that need direct access to a garbling function. ~~~ module.exports = function(string) { return string.split("").map(function(ch) { return String.fromCharCode(ch.charCodeAt(0) + 5); }).join(""); }; ~~~ [](http://eloquentjavascript.net/20_node.html#p_1wgmE+Q4Cz)Remember that replacing `module.exports`, rather than adding properties to it, allows us to export a specific value from a module. In this case, we make the result of requiring our `garble` file the garbling function itself. [](http://eloquentjavascript.net/20_node.html#p_DqwnM9koUR)The function splits the string it is given into single characters by splitting on the empty string and then replaces each character with the character whose code is five points higher. Finally, it joins the result back into a string. [](http://eloquentjavascript.net/20_node.html#p_S/nc4v84fJ)We can now call our tool like this: ~~~ $ node main.js JavaScript Of{fXhwnuy ~~~ ## [](http://eloquentjavascript.net/20_node.html#h_J6hW/SmL/a)Installing with NPM [](http://eloquentjavascript.net/20_node.html#p_hIPO+3mmLK)NPM, which was briefly discussed in [Chapter 10](http://eloquentjavascript.net/10_modules.html#modules_npm), is an online repository of JavaScript modules, many of which are specifically written for Node. When you install Node on your computer, you also get a program called `npm`, which provides a convenient interface to this repository. [](http://eloquentjavascript.net/20_node.html#p_vVKwgEFDXW)For example, one module you will find on NPM is `figlet`, which can convert text into *ASCII art*—drawings made out of text characters. The following transcript shows how to install and use it: ~~~ $ npm install figlet npm GET https://registry.npmjs.org/figlet npm 200 https://registry.npmjs.org/figlet npm GET https://registry.npmjs.org/figlet/-/figlet-1.0.9.tgz npm 200 https://registry.npmjs.org/figlet/-/figlet-1.0.9.tgz figlet@1.0.9 node_modules/figlet $ node > var figlet = require("figlet"); > figlet.text("Hello world!", function(error, data) { if (error) console.error(error); else console.log(data); }); _ _ _ _ _ _ _ | | | | ___| | | ___ __ _____ _ __| | __| | | | |_| |/ _ \ | |/ _ \ \ \ /\ / / _ \| '__| |/ _` | | | _ | __/ | | (_) | \ V V / (_) | | | | (_| |_| |_| |_|\___|_|_|\___/ \_/\_/ \___/|_| |_|\__,_(_) ~~~ [](http://eloquentjavascript.net/20_node.html#p_az5okEiggC)After running `npm install`, NPM will have created a directory called`node_modules`. Inside that directory will be a `figlet` directory, which contains the library. When we run `node` and call `require("figlet")`, this library is loaded, and we can call its `text` method to draw some big letters. [](http://eloquentjavascript.net/20_node.html#p_JoI06XJ/Qi)Somewhat unexpectedly perhaps, instead of simply returning the string that makes up the big letters, `figlet.text` takes a callback function that it passes its result to. It also passes the callback another argument, `error`, which will hold an error object when something goes wrong or null when everything is all right. [](http://eloquentjavascript.net/20_node.html#p_wAmsyRKxO8)This is a common pattern in Node code. Rendering something with `figlet`requires the library to read a file that contains the letter shapes. Reading that file from disk is an asynchronous operation in Node, so `figlet.text` can’t immediately return its result. Asynchronicity is infectious, in a way—every function that calls an asynchronous function must itself become asynchronous. [](http://eloquentjavascript.net/20_node.html#p_a6yOLG4/+I)There is much more to NPM than `npm install`. It reads `package.json` files, which contain JSON-encoded information about a program or library, such as which other libraries it depends on. Doing `npm install` in a directory that contains such a file will automatically install all dependencies, as well as *their*dependencies. The `npm` tool is also used to publish libraries to NPM’s online repository of packages so that other people can find, download, and use them. [](http://eloquentjavascript.net/20_node.html#p_hiqvsm4s1n)This book won’t delve further into the details of NPM usage. Refer to[*npmjs.org*](http://npmjs.org/) for further documentation and for an easy way to search for libraries. ## [](http://eloquentjavascript.net/20_node.html#h_o2abiQU0TD)The file system module [](http://eloquentjavascript.net/20_node.html#p_HumHNRQKJx)One of the most commonly used built-in modules that comes with Node is the`"fs"` module, which stands for *file system*. This module provides functions for working with files and directories. [](http://eloquentjavascript.net/20_node.html#p_OWX791kITh)For example, there is a function called `readFile`, which reads a file and then calls a callback with the file’s contents. ~~~ var fs = require("fs"); fs.readFile("file.txt", "utf8", function(error, text) { if (error) throw error; console.log("The file contained:", text); }); ~~~ [](http://eloquentjavascript.net/20_node.html#p_kiSsXyNElK)The second argument to `readFile` indicates the *character encoding* used to decode the file into a string. There are several ways in which text can be encoded to binary data, but most modern systems use UTF-8 to encode text, so unless you have reasons to believe another encoding is used, passing `"utf8"`when reading a text file is a safe bet. If you do not pass an encoding, Node will assume you are interested in the binary data and will give you a `Buffer` object instead of a string. This is an array-like object that contains numbers representing the bytes in the files. ~~~ var fs = require("fs"); fs.readFile("file.txt", function(error, buffer) { if (error) throw error; console.log("The file contained", buffer.length, "bytes.", "The first byte is:", buffer[0]); }); ~~~ [](http://eloquentjavascript.net/20_node.html#p_2eNWgxdLGx)A similar function, `writeFile`, is used to write a file to disk. ~~~ var fs = require("fs"); fs.writeFile("graffiti.txt", "Node was here", function(err) { if (err) console.log("Failed to write file:", err); else console.log("File written."); }); ~~~ [](http://eloquentjavascript.net/20_node.html#p_rDd4gjDwV6)Here, it was not necessary to specify the encoding since `writeFile` will assume that if it is given a string to write, rather than a `Buffer` object, it should write it out as text using its default character encoding, which is UTF-8. [](http://eloquentjavascript.net/20_node.html#p_wbabMpBtqE)The `"fs"` module contains many other useful functions: `readdir` will return the files in a directory as an array of strings, `stat` will retrieve information about a file, `rename` will rename a file, `unlink` will remove one, and so on. See the documentation at [*nodejs.org*](http://nodejs.org/) for specifics. [](http://eloquentjavascript.net/20_node.html#p_yy7JT9M6b9)Many of the functions in `"fs"` come in both synchronous and asynchronous variants. For example, there is a synchronous version of `readFile` called`readFileSync`. ~~~ var fs = require("fs"); console.log(fs.readFileSync("file.txt", "utf8")); ~~~ [](http://eloquentjavascript.net/20_node.html#p_eKCxvkAVwC)Synchronous functions require less ceremony to use and can be useful in simple scripts, where the extra speed provided by asynchronous I/O is irrelevant. But note that while such a synchronous operation is being performed, your program will be stopped entirely. If it should be responding to the user or to other machines on the network, being stuck on synchronous I/O might produce annoying delays. ## [](http://eloquentjavascript.net/20_node.html#h_3O5dGIJE9F)The HTTP module [](http://eloquentjavascript.net/20_node.html#p_jzBrRTKIgP)Another central module is called `"http"`. It provides functionality for running HTTP servers and making HTTP requests. [](http://eloquentjavascript.net/20_node.html#p_vo1yvZzETN)This is all it takes to start a simple HTTP server: ~~~ var http = require("http"); var server = http.createServer(function(request, response) { response.writeHead(200, {"Content-Type": "text/html"}); response.write("<h1>Hello!</h1><p>You asked for <code>" + request.url + "</code></p>"); response.end(); }); server.listen(8000); ~~~ [](http://eloquentjavascript.net/20_node.html#p_aWcDaGah2b)If you run this script on your own machine, you can point your web browser at[*http://localhost:8000/hello*](http://localhost:8000/hello) to make a request to your server. It will respond with a small HTML page. [](http://eloquentjavascript.net/20_node.html#p_ZiHH2p8YG0)The function passed as an argument to `createServer` is called every time a client tries to connect to the server. The `request` and `response` variables are objects representing the incoming and outgoing data. The first contains information about the request, such as its `url` property, which tells us to what URL the request was made. [](http://eloquentjavascript.net/20_node.html#p_eHNvwd3lFU)To send something back, you call methods on the `response` object. The first,`writeHead`, will write out the response headers (see [Chapter 17](http://eloquentjavascript.net/17_http.html#headers)). You give it the status code (200 for “OK” in this case) and an object that contains header values. Here we tell the client that we will be sending back an HTML document. [](http://eloquentjavascript.net/20_node.html#p_AVzTSVzXSM)Next, the actual response body (the document itself) is sent with`response.write`. You are allowed to call this method multiple times if you want to send the response piece by piece, possibly streaming data to the client as it becomes available. Finally, `response.end` signals the end of the response. [](http://eloquentjavascript.net/20_node.html#p_dETBrQIlAE)The call to `server.listen` causes the server to start waiting for connections on port 8000\. This is the reason you have to connect to *localhost:8000*, rather than just *localhost* (which would use the default port, 80), to speak to this server. [](http://eloquentjavascript.net/20_node.html#p_IwS00NKPKg)To stop running a Node script like this, which doesn’t finish automatically because it is waiting for further events (in this case, network connections), press Ctrl-C. [](http://eloquentjavascript.net/20_node.html#p_7qMlOUUlqm)A real web server usually does more than the one in the previous example—it looks at the request’s method (the `method` property) to see what action the client is trying to perform and at the request’s URL to find out which resource this action is being performed on. You’ll see a more advanced server [later in this chapter](http://eloquentjavascript.net/20_node.html#file_server). [](http://eloquentjavascript.net/20_node.html#p_xP0LenM1Z0)To act as an HTTP *client*, we can use the `request` function in the `"http"`module. ~~~ var http = require("http"); var request = http.request({ hostname: "eloquentjavascript.net", path: "/20_node.html", method: "GET", headers: {Accept: "text/html"} }, function(response) { console.log("Server responded with status code", response.statusCode); }); request.end(); ~~~ [](http://eloquentjavascript.net/20_node.html#p_3osXgsZbhQ)The first argument to `request` configures the request, telling Node what server to talk to, what path to request from that server, which method to use, and so on. The second argument is the function that should be called when a response comes in. It is given an object that allows us to inspect the response, for example to find out its status code. [](http://eloquentjavascript.net/20_node.html#p_9OfV0SuhAH)Just like the `response` object we saw in the server, the object returned by`request` allows us to stream data into the request with the `write` method and finish the request with the `end` method. The example does not use `write`because `GET` requests should not contain data in their request body. [](http://eloquentjavascript.net/20_node.html#p_W+uNSMgm+u)To make requests to secure HTTP (HTTPS) URLs, Node provides a package called `https`, which contains its own `request` function, similar to`http.request`. ## [](http://eloquentjavascript.net/20_node.html#h_dJhdomfGgD)Streams [](http://eloquentjavascript.net/20_node.html#p_hrW+qFyIS0)We have seen two examples of writable streams in the HTTP examples—namely, the response object that the server could write to and the request object that was returned from `http.request`. [](http://eloquentjavascript.net/20_node.html#p_8sFjBmKCWT)Writable streams are a widely used concept in Node interfaces. All writable streams have a `write` method, which can be passed a string or a `Buffer`object. Their `end` method closes the stream and, if given an argument, will also write out a piece of data before it does so. Both of these methods can also be given a callback as an additional argument, which they will call when the writing to or closing of the stream has finished. [](http://eloquentjavascript.net/20_node.html#p_CL0ZXsd83G)It is possible to create a writable stream that points at a file with the`fs.createWriteStream` function. Then you can use the `write` method on the resulting object to write the file one piece at a time, rather than in one shot as with `fs.writeFile`. [](http://eloquentjavascript.net/20_node.html#p_93VQIHGgFt)Readable streams are a little more involved. Both the `request` variable that was passed to the HTTP server’s callback function and the `response` variable passed to the HTTP client are readable streams. (A server reads requests and then writes responses, whereas a client first writes a request and then reads a response.) Reading from a stream is done using event handlers, rather than methods. [](http://eloquentjavascript.net/20_node.html#p_CaPkx6i5f+)Objects that emit events in Node have a method called `on` that is similar to the`addEventListener` method in the browser. You give it an event name and then a function, and it will register that function to be called whenever the given event occurs. [](http://eloquentjavascript.net/20_node.html#p_J1cQlNGHLA)Readable streams have `"data"` and `"end"` events. The first is fired every time some data comes in, and the second is called whenever the stream is at its end. This model is most suited for “streaming” data, which can be immediately processed, even when the whole document isn’t available yet. A file can be read as a readable stream by using the `fs.createReadStream` function. [](http://eloquentjavascript.net/20_node.html#p_/VIRSW4CKl)The following code creates a server that reads request bodies and streams them back to the client as all-uppercase text: ~~~ var http = require("http"); http.createServer(function(request, response) { response.writeHead(200, {"Content-Type": "text/plain"}); request.on("data", function(chunk) { response.write(chunk.toString().toUpperCase()); }); request.on("end", function() { response.end(); }); }).listen(8000); ~~~ [](http://eloquentjavascript.net/20_node.html#p_m3l1M7epAY)The `chunk` variable passed to the data handler will be a binary `Buffer`, which we can convert to a string by calling `toString` on it, which will decode it using the default encoding (UTF-8). [](http://eloquentjavascript.net/20_node.html#p_6S1XXBZo7w)The following piece of code, if run while the uppercasing server is running, will send a request to that server and write out the response it gets: ~~~ var http = require("http"); var request = http.request({ hostname: "localhost", port: 8000, method: "POST" }, function(response) { response.on("data", function(chunk) { process.stdout.write(chunk.toString()); }); }); request.end("Hello server"); ~~~ [](http://eloquentjavascript.net/20_node.html#p_H/08BRdstu)The example writes to `process.stdout` (the process’ standard output, as a writable stream) instead of using `console.log`. We can’t use `console.log`because it adds an extra newline character after each piece of text that it writes, which isn’t appropriate here. ## [](http://eloquentjavascript.net/20_node.html#h_LvXChBt2KP)A simple file server [](http://eloquentjavascript.net/20_node.html#p_TeWTnqNsas)Let’s combine our newfound knowledge about HTTP servers and talking to the file system and create a bridge between them: an HTTP server that allows remote access to a file system. Such a server has many uses. It allows web applications to store and share data or give a group of people shared access to a bunch of files. [](http://eloquentjavascript.net/20_node.html#p_Pu9mO8Vutb)When we treat files as HTTP resources, the HTTP methods `GET`, `PUT`, and`DELETE` can be used to read, write, and delete the files, respectively. We will interpret the path in the request as the path of the file that the request refers to. [](http://eloquentjavascript.net/20_node.html#p_hv9uP+YLeb)We probably don’t want to share our whole file system, so we’ll interpret these paths as starting in the server’s working directory, which is the directory in which it was started. If I ran the server from `/home/marijn/public/` (or`C:\Users\marijn\public\` on Windows), then a request for `/file.txt`should refer to `/home/marijn/public/file.txt` (or`C:\Users\marijn\public\file.txt`). [](http://eloquentjavascript.net/20_node.html#p_laeh/qV5fJ)We’ll build the program piece by piece, using an object called `methods` to store the functions that handle the various HTTP methods. ~~~ var http = require("http"), fs = require("fs"); var methods = Object.create(null); http.createServer(function(request, response) { function respond(code, body, type) { if (!type) type = "text/plain"; response.writeHead(code, {"Content-Type": type}); if (body && body.pipe) body.pipe(response); else response.end(body); } if (request.method in methods) methods[request.method](urlToPath(request.url), respond, request); else respond(405, "Method " + request.method + " not allowed."); }).listen(8000); ~~~ [](http://eloquentjavascript.net/20_node.html#p_znP+YS4c6v)This starts a server that just returns 405 error responses, which is the code used to indicate that a given method isn’t handled by the server. [](http://eloquentjavascript.net/20_node.html#p_5pDM8svy8I)The `respond` function is passed to the functions that handle the various methods and acts as a callback to finish the request. It takes an HTTP status code, a body, and optionally a content type as arguments. If the value passed as the body is a readable stream, it will have a `pipe` method, which is used to forward a readable stream to a writable stream. If not, it is assumed to be either `null` (no body) or a string and is passed directly to the response’s `end`method. [](http://eloquentjavascript.net/20_node.html#p_gSrvCsfWuQ)To get a path from the URL in the request, the `urlToPath` function uses Node’s built-in `"url"` module to parse the URL. It takes its pathname, which will be something like `/file.txt`, decodes that to get rid of the `%20`-style escape codes, and prefixes a single dot to produce a path relative to the current directory. ~~~ function urlToPath(url) { var path = require("url").parse(url).pathname; return "." + decodeURIComponent(path); } ~~~ [](http://eloquentjavascript.net/20_node.html#p_ZsCml7c8nG)If you are worried about the security of the `urlToPath` function, you are right. We will return to that in the exercises. [](http://eloquentjavascript.net/20_node.html#p_hLccVoyVPH)We will set up the `GET` method to return a list of files when reading a directory and to return the file’s content when reading a regular file. [](http://eloquentjavascript.net/20_node.html#p_zj6fKVP9SD)One tricky question is what kind of `Content-Type` header we should add when returning a file’s content. Since these files could be anything, our server can’t simply return the same type for all of them. But NPM can help with that. The`mime` package (content type indicators like `text/plain` are also called *MIME types*) knows the correct type for a huge number of file extensions. [](http://eloquentjavascript.net/20_node.html#p_HzGhiQfi8f)If you run the following `npm` command in the directory where the server script lives, you’ll be able to use `require("mime")` to get access to the library: ~~~ $ npm install mime npm http GET https://registry.npmjs.org/mime npm http 304 https://registry.npmjs.org/mime mime@1.2.11 node_modules/mime ~~~ [](http://eloquentjavascript.net/20_node.html#p_0oNrzailqn)When a requested file does not exist, the correct HTTP error code to return is 404\. We will use `fs.stat`, which looks up information on a file, to find out both whether the file exists and whether it is a directory. ~~~ methods.GET = function(path, respond) { fs.stat(path, function(error, stats) { if (error && error.code == "ENOENT") respond(404, "File not found"); else if (error) respond(500, error.toString()); else if (stats.isDirectory()) fs.readdir(path, function(error, files) { if (error) respond(500, error.toString()); else respond(200, files.join("\n")); }); else respond(200, fs.createReadStream(path), require("mime").lookup(path)); }); }; ~~~ [](http://eloquentjavascript.net/20_node.html#p_mV/qPzXlQf)Because it has to touch the disk and thus might take a while, `fs.stat` is asynchronous. When the file does not exist, `fs.stat` will pass an error object with a `code` property of `"ENOENT"` to its callback. It would be nice if Node defined different subtypes of `Error` for different types of error, but it doesn’t. Instead, it just puts obscure, Unix-inspired codes in there. [](http://eloquentjavascript.net/20_node.html#p_jHOsnGBkYC)We are going to report any errors we didn’t expect with status code 500, which indicates that the problem exists in the server, as opposed to codes starting with 4 (such as 404), which refer to bad requests. There are some situations in which this is not entirely accurate, but for a small example program like this, it will have to be good enough. [](http://eloquentjavascript.net/20_node.html#p_fZyZfbXY6J)The `stats` object returned by `fs.stat` tells us a number of things about a file, such as its size (`size` property) and its modification date (`mtime` property). Here we are interested in the question of whether it is a directory or a regular file, which the `isDirectory` method tells us. [](http://eloquentjavascript.net/20_node.html#p_4F379sRM4r)We use `fs.readdir` to read the list of files in a directory and, in yet another callback, return it to the user. For normal files, we create a readable stream with `fs.createReadStream` and pass it to `respond`, along with the content type that the `"mime"` module gives us for the file’s name. [](http://eloquentjavascript.net/20_node.html#p_3SF21U54yR)The code to handle `DELETE` requests is slightly simpler. ~~~ methods.DELETE = function(path, respond) { fs.stat(path, function(error, stats) { if (error && error.code == "ENOENT") respond(204); else if (error) respond(500, error.toString()); else if (stats.isDirectory()) fs.rmdir(path, respondErrorOrNothing(respond)); else fs.unlink(path, respondErrorOrNothing(respond)); }); }; ~~~ [](http://eloquentjavascript.net/20_node.html#p_nmATyMSIB9)You may be wondering why trying to delete a nonexistent file returns a 204 status, rather than an error. When the file that is being deleted is not there, you could say that the request’s objective is already fulfilled. The HTTP standard encourages people to make requests *idempotent*, which means that applying them multiple times does not produce a different result. ~~~ function respondErrorOrNothing(respond) { return function(error) { if (error) respond(500, error.toString()); else respond(204); }; } ~~~ [](http://eloquentjavascript.net/20_node.html#p_M3IjJtc656)When an HTTP response does not contain any data, the status code 204 (“no content”) can be used to indicate this. Since we need to provide callbacks that either report an error or return a 204 response in a few different situations, I wrote a `respondErrorOrNothing` function that creates such a callback. [](http://eloquentjavascript.net/20_node.html#p_gwRQ8JoXxf)This is the handler for `PUT` requests: ~~~ methods.PUT = function(path, respond, request) { var outStream = fs.createWriteStream(path); outStream.on("error", function(error) { respond(500, error.toString()); }); outStream.on("finish", function() { respond(204); }); request.pipe(outStream); }; ~~~ [](http://eloquentjavascript.net/20_node.html#p_ItuasUF74h)Here, we don’t need to check whether the file exists—if it does, we’ll just overwrite it. We again use `pipe` to move data from a readable stream to a writable one, in this case from the request to the file. If creating the stream fails, an `"error"` event is raised for it, which we report in our response. When the data is transferred successfully, `pipe` will close both streams, which will cause a `"finish"` event to fire on the writable stream. When that happens, we can report success to the client with a 204 response. [](http://eloquentjavascript.net/20_node.html#p_ig7HitKqRz)The full script for the server is available at[*eloquentjavascript.net/code/file_server.js*](http://eloquentjavascript.net/code/file_server.js). You can download that and run it with Node to start your own file server. And of course, you can modify and extend it to solve this chapter’s exercises or to experiment. [](http://eloquentjavascript.net/20_node.html#p_fBFWTJncql)The command-line tool `curl`, widely available on Unix-like systems, can be used to make HTTP requests. The following session briefly tests our server. Note that `-X` is used to set the request’s method and `-d` is used to include a request body. ~~~ $ curl http://localhost:8000/file.txt File not found $ curl -X PUT -d hello http://localhost:8000/file.txt $ curl http://localhost:8000/file.txt hello $ curl -X DELETE http://localhost:8000/file.txt $ curl http://localhost:8000/file.txt File not found ~~~ [](http://eloquentjavascript.net/20_node.html#p_AmMFBJGSkb)The first request for `file.txt` fails since the file does not exist yet. The `PUT`request creates the file, and behold, the next request successfully retrieves it. After deleting it with a `DELETE` request, the file is again missing. ## [](http://eloquentjavascript.net/20_node.html#h_c2MdfwNW/b)Error handling [](http://eloquentjavascript.net/20_node.html#p_AzJTO0+z/M)In the code for the file server, there are *six* places where we are explicitly routing exceptions that we don’t know how to handle into error responses. Because exceptions aren’t automatically propagated to callbacks but rather passed to them as arguments, they have to be handled explicitly every time. This completely defeats the advantage of exception handling, namely, the ability to centralize the handling of failure conditions. [](http://eloquentjavascript.net/20_node.html#p_LZHdJfKLv3)What happens when something actually *throws* an exception in this system? Since we are not using any `try` blocks, the exception will propagate to the top of the call stack. In Node, that aborts the program and writes information about the exception (including a stack trace) to the program’s standard error stream. [](http://eloquentjavascript.net/20_node.html#p_dYLIN1E+o4)This means that our server will crash whenever a problem is encountered in the server’s code itself, as opposed to asynchronous problems, which will be passed as arguments to the callbacks. If we wanted to handle all exceptions raised during the handling of a request, to make sure we send a response, we would have to add `try/catch` blocks to *every* callback. [](http://eloquentjavascript.net/20_node.html#p_lyKlcWotG5)This is not workable. Many Node programs are written to make as little use of exceptions as possible, with the assumption that if an exception is raised, it is not something the program can handle, and crashing is the right response. [](http://eloquentjavascript.net/20_node.html#p_m6CCLIC9PT)Another approach is to use promises, which were introduced in [Chapter 17](http://eloquentjavascript.net/17_http.html#promises). Those catch exceptions raised by callback functions and propagate them as failures. It is possible to load a promise library in Node and use that to manage your asynchronous control. Few Node libraries integrate promises, but it is often trivial to wrap them. The excellent `"promise"` module from NPM contains a function called `denodeify`, which takes an asynchronous function like `fs.readFile` and converts it to a promise-returning function. ~~~ var Promise = require("promise"); var fs = require("fs"); var readFile = Promise.denodeify(fs.readFile); readFile("file.txt", "utf8").then(function(content) { console.log("The file contained: " + content); }, function(error) { console.log("Failed to read file: " + error); }); ~~~ [](http://eloquentjavascript.net/20_node.html#p_XJLTFNkV/a)For comparison, I’ve written another version of the file server based on promises, which you can find at[*eloquentjavascript.net/code/file_server_promises.js*](http://eloquentjavascript.net/code/file_server_promises.js). It is slightly cleaner because functions can now *return* their results, rather than having to call callbacks, and the routing of exceptions is implicit, rather than explicit. [](http://eloquentjavascript.net/20_node.html#p_/tLwA645Vy)I’ll list a few lines from the promise-based file server to illustrate the difference in the style of programming. [](http://eloquentjavascript.net/20_node.html#p_sAe0M1se8f)The `fsp` object that is used by this code contains promise-style variants of a number of `fs` functions, wrapped by `Promise.denodeify`. The object returned from the method handler, with `code` and `body` properties, will become the final result of the chain of promises, and it will be used to determine what kind of response to send to the client. ~~~ methods.GET = function(path) { return inspectPath(path).then(function(stats) { if (!stats) // Does not exist return {code: 404, body: "File not found"}; else if (stats.isDirectory()) return fsp.readdir(path).then(function(files) { return {code: 200, body: files.join("\n")}; }); else return {code: 200, type: require("mime").lookup(path), body: fs.createReadStream(path)}; }); }; function inspectPath(path) { return fsp.stat(path).then(null, function(error) { if (error.code == "ENOENT") return null; else throw error; }); } ~~~ [](http://eloquentjavascript.net/20_node.html#p_LEaRcBuKCq)The `inspectPath` function is a simple wrapper around `fs.stat`, which handles the case where the file is not found. In that case, we replace the failure with a success that yields `null`. All other errors are allowed to propagate. When the promise that is returned from these handlers fails, the HTTP server responds with a 500 status code. ## [](http://eloquentjavascript.net/20_node.html#h_ErccPg/l98)Summary [](http://eloquentjavascript.net/20_node.html#p_iV5lhu5NQo)Node is a nice, straightforward system that lets us run JavaScript in a nonbrowser context. It was originally designed for network tasks to play the role of a *node* in a network. But it lends itself to all kinds of scripting tasks, and if writing JavaScript is something you enjoy, automating everyday tasks with Node works wonderfully. [](http://eloquentjavascript.net/20_node.html#p_kDMJDKzKTZ)NPM provides libraries for everything you can think of (and quite a few things you’d probably never think of), and it allows you to fetch and install those libraries by running a simple command. Node also comes with a number of built-in modules, including the `"fs"` module, for working with the file system, and the `"http"` module, for running HTTP servers and making HTTP requests. [](http://eloquentjavascript.net/20_node.html#p_vfv/Kj/H3N)All input and output in Node is done asynchronously, unless you explicitly use a synchronous variant of a function, such as `fs.readFileSync`. You provide callback functions, and Node will call them at the appropriate time, when the I/O you asked for has finished. ## [](http://eloquentjavascript.net/20_node.html#h_TcUD2vzyMe)Exercises ### [](http://eloquentjavascript.net/20_node.html#h_R/n4kTrgEX)Content negotiation, again [](http://eloquentjavascript.net/20_node.html#p_1sc3cgSDRc)In [Chapter 17](http://eloquentjavascript.net/17_http.html#exercise_accept), the first exercise was to make several requests to[*eloquentjavascript.net/author*](http://eloquentjavascript.net/author), asking for different types of content by passing different `Accept` headers. [](http://eloquentjavascript.net/20_node.html#p_FReC/gLaW+)Do this again, using Node’s `http.request` function. Ask for at least the media types `text/plain`, `text/html`, and `application/json`. Remember that headers to a request can be given as an object, in the `headers` property of`http.request`’s first argument. [](http://eloquentjavascript.net/20_node.html#p_ZG9mFshMgi)Write out the content of the responses to each request. ### [](http://eloquentjavascript.net/20_node.html#h_yunNtKqBKK)Fixing a leak [](http://eloquentjavascript.net/20_node.html#p_VfOcuona0c)For easy remote access to some files, I might get into the habit of having the [file server](http://eloquentjavascript.net/20_node.html#file_server) defined in this chapter running on my machine, in the`/home/marijn/public` directory. Then, one day, I find that someone has gained access to all the passwords I stored in my browser. [](http://eloquentjavascript.net/20_node.html#p_dmHmWR/Zqs)What happened? [](http://eloquentjavascript.net/20_node.html#p_DI3INnE8nA)If it isn’t clear to you yet, think back to the `urlToPath` function, defined like this: ~~~ function urlToPath(url) { var path = require("url").parse(url).pathname; return "." + decodeURIComponent(path); } ~~~ [](http://eloquentjavascript.net/20_node.html#p_mQL5b2p0bF)Now consider the fact that paths passed to the `"fs"` functions can be relative—they may contain `"../"` to go up a directory. What happens when a client sends requests to URLs like the ones shown here? ~~~ http://myhostname:8000/../.config/config/google-chrome/Default/Web%20Data http://myhostname:8000/../.ssh/id_dsa http://myhostname:8000/../../../etc/passwd ~~~ [](http://eloquentjavascript.net/20_node.html#p_kaATEvJoFT)Change `urlToPath` to fix this problem. Take into account the fact that Node on Windows allows both forward slashes and backslashes to separate directories. [](http://eloquentjavascript.net/20_node.html#p_N2cR0sagWP)Also, meditate on the fact that as soon as you expose some half-baked system on the Internet, the bugs in that system might be used to do bad things to your machine. ### [](http://eloquentjavascript.net/20_node.html#h_NHUcwC79fZ)Creating directories [](http://eloquentjavascript.net/20_node.html#p_No30amwBdC)Though the `DELETE` method is wired up to delete directories (using `fs.rmdir`), the file server currently does not provide any way to *create* a directory. [](http://eloquentjavascript.net/20_node.html#p_6UFSFAp8Vj)Add support for a method `MKCOL`, which should create a directory by calling`fs.mkdir`. `MKCOL` is not one of the basic HTTP methods, but it does exist, for this same purpose, in the *WebDAV* standard, which specifies a set of extensions to HTTP, making it suitable for writing resources, not just reading them. ### [](http://eloquentjavascript.net/20_node.html#h_TLRTlwK6ZU)A public space on the web [](http://eloquentjavascript.net/20_node.html#p_45IRHCEqky)Since the file server serves up any kind of file and even includes the right`Content-Type` header, you can use it to serve a website. Since it allows everybody to delete and replace files, it would be an interesting kind of website: one that can be modified, vandalized, and destroyed by everybody who takes the time to create the right HTTP request. Still, it would be a website. [](http://eloquentjavascript.net/20_node.html#p_S5RQ8T3sNn)Write a basic HTML page that includes a simple JavaScript file. Put the files in a directory served by the file server and open them in your browser. [](http://eloquentjavascript.net/20_node.html#p_UerJ/YohV1)Next, as an advanced exercise or even a weekend project, combine all the knowledge you gained from this book to build a more user-friendly interface for modifying the website from *inside* the website. [](http://eloquentjavascript.net/20_node.html#p_/XOhzawYGD)Use an HTML form ([Chapter 18](http://eloquentjavascript.net/18_forms.html#forms)) to edit the content of the files that make up the website, allowing the user to update them on the server by using HTTP requests as described in [Chapter 17](http://eloquentjavascript.net/17_http.html#http). [](http://eloquentjavascript.net/20_node.html#p_hWSA1+odAv)Start by making only a single file editable. Then make it so that the user can select which file to edit. Use the fact that our file server returns lists of files when reading a directory. [](http://eloquentjavascript.net/20_node.html#p_Ph56FG1hyk)Don’t work directly in the code on the file server, since if you make a mistake you are likely to damage the files there. Instead, keep your work outside of the publicly accessible directory and copy it there when testing. [](http://eloquentjavascript.net/20_node.html#p_3aqVNaF+TI)If your computer is directly connected to the Internet, without a firewall, router, or other interfering device in between, you might be able to invite a friend to use your website. To check, go to [*whatismyip.com*](http://www.whatismyip.com/), copy the IP address it gives you into the address bar of your browser, and add `:8000` after it to select the right port. If that brings you to your site, it is online for everybody to see. You can create a `<textarea>` element to hold the content of the file that is being edited. A `GET` request, using `XMLHttpRequest`, can be used to get the current content of the file. You can use relative URLs like *index.html*, instead of[*http://localhost:8000/index.html*](http://localhost:8000/index.html), to refer to files on the same server as the running script. [](http://eloquentjavascript.net/20_node.html#p_AddxfNwTA3)Then, when the user clicks a button (you can use a `<form>` element and`"submit"` event or simply a `"click"` handler), make a `PUT` request to the same URL, with the content of the `<textarea>` as request body, to save the file. [](http://eloquentjavascript.net/20_node.html#p_LI3MHsPkwQ)You can then add a `<select>` element that contains all the files in the server’s root directory by adding `<option>` elements containing the lines returned by a`GET` request to the URL `/`. When the user selects another file (a `"change"`event on the field), the script must fetch and display that file. Also make sure that when saving a file, you use the currently selected filename. [](http://eloquentjavascript.net/20_node.html#p_z/PE8qzDXg)Unfortunately, the server is too simplistic to be able to reliably read files from subdirectories since it does not tell us whether the thing we fetched with a `GET`request is a regular file or a directory. Can you think of a way to extend the server to address this?

# Part 3: Node * * * * * #### Chapter 20. [Node.js](76548) #### Chapter 21. [Project: Skill-Sharing Website](76549)

## Chapter 19 # Project: A Paint Program > [](http://eloquentjavascript.net/19_paint.html#p_YcWG8HTlFz)I look at the many colors before me. I look at my blank canvas. Then, I try to apply colors like words that shape poems, like notes that shape music. > > Joan Miro [](http://eloquentjavascript.net/19_paint.html#p_4VGZtxD2xS)The material from the previous chapters gives you all the elements you need to build a simple web application. In this chapter, we will do just that. [](http://eloquentjavascript.net/19_paint.html#p_1emWaHj7VO)Our application will be a web-based drawing program, along the lines of Microsoft Paint. You can use it to open image files, scribble on them with your mouse, and save them. This is what it will look like:  [](http://eloquentjavascript.net/19_paint.html#p_oDwL9FxPW8)Painting on a computer is great. You don’t need to worry about materials, skill, or talent. You just start smearing. ## [](http://eloquentjavascript.net/19_paint.html#h_h4HWFf13vp)Implementation [](http://eloquentjavascript.net/19_paint.html#p_JoMqYLSjLp)The interface for the paint program shows a big `<canvas>` element on top, with a number of form fields below it. The user draws on the picture by selecting a tool from a `<select>` field and then clicking or dragging across the canvas. There are tools for drawing lines, erasing parts of the picture, adding text, and so on. [](http://eloquentjavascript.net/19_paint.html#p_f2FunmZ+W/)Clicking the canvas will hand off the `"mousedown"` event to the currently selected tool, which can handle it in whichever way it chooses. The line drawing tool, for example, will listen for `"mousemove"` events until the mouse button is released and draw lines along the mouse’s path using the current color and brush size. [](http://eloquentjavascript.net/19_paint.html#p_SidGjlM9I+)Color and brush size are selected with additional form fields. These are hooked up to update the canvas drawing context’s `fillStyle`, `strokeStyle`, and`lineWidth` whenever they are changed. [](http://eloquentjavascript.net/19_paint.html#p_ARTBoP4NDe)You can load an image into the program in two ways. The first uses a file field, where the user can select a file on their own file system. The second asks for a URL and will fetch an image from the Web. [](http://eloquentjavascript.net/19_paint.html#p_7b82pMSl8A)Images are saved in a somewhat atypical way. The save link at the right side points at the current image. It can be followed, shared, or saved. I will explain how this is achieved in a moment. ## [](http://eloquentjavascript.net/19_paint.html#h_9CGBdyKskt)Building the DOM [](http://eloquentjavascript.net/19_paint.html#p_sqg53vjc6x)Our program’s interface is built from more than 30 DOM elements. We need to construct these somehow. [](http://eloquentjavascript.net/19_paint.html#p_BwXcOEgv2v)HTML is the most obvious format for defining complex DOM structures. But separating the program into a piece of HTML and a script is made difficult by the fact that many of the DOM elements need event handlers or have to be touched by the script in some other way. Thus, our script would have to make lots of `querySelector` (or similar) calls in order to find the DOM elements that it needs to act on. [](http://eloquentjavascript.net/19_paint.html#p_hwJVTRpgY3)It would be nice if the DOM structure for each part of our interface is defined close to the JavaScript code that drives it. Thus, I’ve chosen to do all creation of DOM nodes in JavaScript. As we saw in [Chapter 13](http://eloquentjavascript.net/13_dom.html#standard), the built-in interface for building up a DOM structure is horrendously verbose. If we are going to do a lot of DOM construction, we need a helper function. [](http://eloquentjavascript.net/19_paint.html#p_7sINQE+0xU)This helper function is an extended version of the `elt` function from [Chapter 13](http://eloquentjavascript.net/13_dom.html#elt). It creates an element with the given name and attributes and appends all further arguments it gets as child nodes, automatically converting strings to text nodes. ~~~ function elt(name, attributes) { var node = document.createElement(name); if (attributes) { for (var attr in attributes) if (attributes.hasOwnProperty(attr)) node.setAttribute(attr, attributes[attr]); } for (var i = 2; i < arguments.length; i++) { var child = arguments[i]; if (typeof child == "string") child = document.createTextNode(child); node.appendChild(child); } return node; } ~~~ [](http://eloquentjavascript.net/19_paint.html#p_ulZXM8Og9d)This allows us to create elements easily, without making our source code as long and dull as a corporate end-user agreement. ## [](http://eloquentjavascript.net/19_paint.html#h_feJq2F1dmk)The foundation [](http://eloquentjavascript.net/19_paint.html#p_fc9fUQEVHE)The core of our program is the `createPaint` function, which appends the paint interface to the DOM element it is given as an argument. Because we want to build our program piece by piece, we define an object called `controls`, which will hold functions to initialize the various controls below the image. ~~~ var controls = Object.create(null); function createPaint(parent) { var canvas = elt("canvas", {width: 500, height: 300}); var cx = canvas.getContext("2d"); var toolbar = elt("div", {class: "toolbar"}); for (var name in controls) toolbar.appendChild(controls[name](cx)); var panel = elt("div", {class: "picturepanel"}, canvas); parent.appendChild(elt("div", null, panel, toolbar)); } ~~~ [](http://eloquentjavascript.net/19_paint.html#p_vEXvH0x+Tb)Each control has access to the canvas drawing context and, through that context’s `canvas` property, to the `<canvas>` element. Most of the program’s state lives in this canvas—it contains the current picture as well as the selected color (in its `fillStyle` property) and brush size (in its `lineWidth` property). [](http://eloquentjavascript.net/19_paint.html#p_p0cg1HO335)We wrap the canvas and the controls in `<div>` elements with classes so we can add some styling, such as a gray border around the picture. ## [](http://eloquentjavascript.net/19_paint.html#h_dgOsN5kwO8)Tool selection [](http://eloquentjavascript.net/19_paint.html#p_g0nRLu/Vl9)The first control we add is the `<select>` element that allows the user to pick a drawing tool. As with `controls`, we will use an object to collect the various tools so that we do not have to hard-code them all in one place and can add more tools later. This object associates the names of the tools with the function that should be called when they are selected and the canvas is clicked. ~~~ var tools = Object.create(null); controls.tool = function(cx) { var select = elt("select"); for (var name in tools) select.appendChild(elt("option", null, name)); cx.canvas.addEventListener("mousedown", function(event) { if (event.which == 1) { tools[select.value](event, cx); event.preventDefault(); } }); return elt("span", null, "Tool: ", select); }; ~~~ [](http://eloquentjavascript.net/19_paint.html#p_8di1zX3Jlb)The tool field is populated with `<option>` elements for all tools that have been defined, and a `"mousedown"` handler on the canvas element takes care of calling the function for the current tool, passing it both the event object and the drawing context as arguments. It also calls `preventDefault` so that holding the mouse button and dragging does not cause the browser to select parts of the page. [](http://eloquentjavascript.net/19_paint.html#p_C/kPVzLbL1)The most basic tool is the line tool, which allows the user to draw lines with the mouse. To put the line ends in the right place, we need to be able to find the canvas-relative coordinates that a given mouse event corresponds to. The`getBoundingClientRect` method, briefly mentioned in [Chapter 13](http://eloquentjavascript.net/13_dom.html#boundingRect), can help us here. It tells us where an element is shown, relative to the top-left corner of the screen. The `clientX` and `clientY` properties on mouse events are also relative to this corner, so we can subtract the top-left corner of the canvas from them to get a position relative to that corner. ~~~ function relativePos(event, element) { var rect = element.getBoundingClientRect(); return {x: Math.floor(event.clientX - rect.left), y: Math.floor(event.clientY - rect.top)}; } ~~~ [](http://eloquentjavascript.net/19_paint.html#p_GsfGlWhhEu)Several of the drawing tools need to listen for `"mousemove"` events as long as the mouse button is held down. The `trackDrag` function takes care of the event registration and unregistration for such situations. ~~~ function trackDrag(onMove, onEnd) { function end(event) { removeEventListener("mousemove", onMove); removeEventListener("mouseup", end); if (onEnd) onEnd(event); } addEventListener("mousemove", onMove); addEventListener("mouseup", end); } ~~~ [](http://eloquentjavascript.net/19_paint.html#p_wDVSXmBYxk)This function takes two arguments. One is a function to call for each`"mousemove"` event, and the other is a function to call when the mouse button is released. Either argument can be omitted when it is not needed. [](http://eloquentjavascript.net/19_paint.html#p_QF1cI/MERi)The line tool uses these two helpers to do the actual drawing. ~~~ tools.Line = function(event, cx, onEnd) { cx.lineCap = "round"; var pos = relativePos(event, cx.canvas); trackDrag(function(event) { cx.beginPath(); cx.moveTo(pos.x, pos.y); pos = relativePos(event, cx.canvas); cx.lineTo(pos.x, pos.y); cx.stroke(); }, onEnd); }; ~~~ [](http://eloquentjavascript.net/19_paint.html#p_WgxKzx/dih)The function starts by setting the drawing context’s `lineCap` property to`"round"`, which causes both ends of a stroked path to be round rather than the default square form. This is a trick to make sure that multiple separate lines, drawn in response to separate events, look like a single, coherent line. With bigger line widths, you will see gaps at corners if you use the default flat line caps. [](http://eloquentjavascript.net/19_paint.html#p_AQy4O9XOrj)Then, for every `"mousemove"` event that occurs as long as the mouse button is down, a simple line segment is drawn between the mouse’s old and new position, using whatever `strokeStyle` and `lineWidth` happen to be currently set. [](http://eloquentjavascript.net/19_paint.html#p_LdZL7msldK)The `onEnd` argument to `tools.Line` is simply passed through to `trackDrag`. The normal way to run tools won’t pass a third argument, so when using the line tool, that argument will hold `undefined`, and nothing happens at the end of the mouse drag. The argument is there to allow us to implement the erase tool on top of the line tool with very little additional code. ~~~ tools.Erase = function(event, cx) { cx.globalCompositeOperation = "destination-out"; tools.Line(event, cx, function() { cx.globalCompositeOperation = "source-over"; }); }; ~~~ [](http://eloquentjavascript.net/19_paint.html#p_0Apcf0lbn3)The `globalCompositeOperation` property influences the way drawing operations on a canvas change the color of the pixels they touch. By default, the property’s value is `"source-over"`, which means that the drawn color is overlaid on the existing color at that spot. If the color is opaque, it will simply replace the old color, but if it is partially transparent, the two will be mixed. [](http://eloquentjavascript.net/19_paint.html#p_+Ckjs7EFIj)The erase tool sets `globalCompositeOperation` to `"destination-out"`, which has the effect of erasing the pixels we touch, making them transparent again. [](http://eloquentjavascript.net/19_paint.html#p_RRoUPpuAjO)That gives us two tools in our paint program. We can draw black lines a single pixel wide (the default `strokeStyle` and `lineWidth` for a canvas) and erase them again. It is a working, albeit rather limited, paint program. ## [](http://eloquentjavascript.net/19_paint.html#h_lIikGuI/QG)Color and brush size [](http://eloquentjavascript.net/19_paint.html#p_STY3nzP377)Assuming that users will want to draw in colors other than black and use different brush sizes, let’s add controls for those two settings. [](http://eloquentjavascript.net/19_paint.html#p_Iq68Mq2ClK)In [Chapter 18](http://eloquentjavascript.net/18_forms.html#forms), I discussed a number of different form fields. Color fields were not among those. Traditionally, browsers don’t have built-in support for color pickers, but in the past few years, a number of new form field types have been standardized. One of those is `<input type="color">`. Others include `"date"`,`"email"`, `"url"`, and `"number"`. Not all browsers support them yet—at the time of writing, no version of Internet Explorer supports color fields. The default type of an `<input>` tag is `"text"`, and when an unsupported type is used, browsers will treat it as a text field. This means that Internet Explorer users running our paint program will have to type in the name of the color they want, rather than select it from a convenient widget. ~~~ controls.color = function(cx) { var input = elt("input", {type: "color"}); input.addEventListener("change", function() { cx.fillStyle = input.value; cx.strokeStyle = input.value; }); return elt("span", null, "Color: ", input); }; ~~~ [](http://eloquentjavascript.net/19_paint.html#p_nNIuUTzgwz)Whenever the value of the color field changes, the drawing context’s`fillStyle` and `strokeStyle` are updated to hold the new value. [](http://eloquentjavascript.net/19_paint.html#p_C0Hqixv9pV)The field for configuring the brush size works similarly. ~~~ controls.brushSize = function(cx) { var select = elt("select"); var sizes = [1, 2, 3, 5, 8, 12, 25, 35, 50, 75, 100]; sizes.forEach(function(size) { select.appendChild(elt("option", {value: size}, size + " pixels")); }); select.addEventListener("change", function() { cx.lineWidth = select.value; }); return elt("span", null, "Brush size: ", select); }; ~~~ [](http://eloquentjavascript.net/19_paint.html#p_qOGqGx5+Pr)The code generates options from an array of brush sizes, and again ensures that the canvas’ `lineWidth` is updated when a brush size is chosen. ## [](http://eloquentjavascript.net/19_paint.html#h_NpxTTfNE7h)Saving [](http://eloquentjavascript.net/19_paint.html#p_7TUivCYbPO)To explain the implementation of the save link, I must first tell you about *data URLs*. A data URL is a URL with *data:* as its protocol. Unlike regular *http:* and*https:* URLs, data URLs don’t point at a resource but rather contain the entire resource in them. This is a data URL containing a simple HTML document: ~~~ data:text/html,<h1 style="color:red">Hello!</h1> ~~~ [](http://eloquentjavascript.net/19_paint.html#p_eWZrRGYvaj)Data URLs are useful for various tasks, such as including small images directly in a style sheet file. They also allow us to link to files that we created on the client side, in the browser, without first moving them to some server. [](http://eloquentjavascript.net/19_paint.html#p_Wcu43CMIZj)Canvas elements have a convenient method, called `toDataURL`, which will return a data URL that contains the picture on the canvas as an image file. We don’t want to update our save link every time the picture is changed, however. For big pictures, that involves moving quite a lot of data into a link and would be noticeably slow. Instead, we rig the link to update its `href` attribute whenever it is focused with the keyboard or the mouse is moved over it. ~~~ controls.save = function(cx) { var link = elt("a", {href: "/"}, "Save"); function update() { try { link.href = cx.canvas.toDataURL(); } catch (e) { if (e instanceof SecurityError) link.href = "javascript:alert(" + JSON.stringify("Can't save: " + e.toString()) + ")"; else throw e; } } link.addEventListener("mouseover", update); link.addEventListener("focus", update); return link; }; ~~~ [](http://eloquentjavascript.net/19_paint.html#p_FK9LTdpkC3)Thus, the link just quietly sits there, pointing at the wrong thing, but when the user approaches it, it magically updates itself to point at the current picture. [](http://eloquentjavascript.net/19_paint.html#p_fvaHWlCaUS)If you load a big image, some browsers will choke on the giant data URLs that this produces. For small pictures, this approach works without problem. [](http://eloquentjavascript.net/19_paint.html#p_yj2JNpVbGr)But here we once again run into the subtleties of browser sandboxing. When an image is loaded from a URL on another domain, if the server’s response doesn’t include a header that tells the browser the resource may be used from other domains (see [Chapter 17](http://eloquentjavascript.net/17_http.html#http_sandbox)), then the canvas will contain information that the*user* may look at but that the *script* may not. [](http://eloquentjavascript.net/19_paint.html#p_P2fyJXYkeA)We may have requested a picture that contains private information (for example, a graph showing the user’s bank account balance) using the user’s session. If scripts could get information out of that picture, they could snoop on the user in undesirable ways. [](http://eloquentjavascript.net/19_paint.html#p_OHFwilBG6F)To prevent these kinds of information leaks, browsers will mark a canvas as*tainted* when an image that the script may not see is drawn onto it. Pixel data, including data URLs, may not be extracted from a tainted canvas. You can write to it, but you can no longer read it. [](http://eloquentjavascript.net/19_paint.html#p_40OvqfoZJr)This is why we need the `try/catch` statement in the `update` function for the save link. When the canvas has become tainted, calling `toDataURL` will raise an exception that is an instance of `SecurityError`. When that happens, we set the link to point at yet another kind of URL, using the *javascript:* protocol. Such links simply execute the script given after the colon when they are followed so that the link will show an `alert` window informing the user of the problem when it is clicked. ## [](http://eloquentjavascript.net/19_paint.html#h_iQAMmIQaWl)Loading image files [](http://eloquentjavascript.net/19_paint.html#p_3vrUSpPAL9)The final two controls are used to load images from local files and from URLs. We’ll need the following helper function, which tries to load an image file from a URL and replace the contents of the canvas with it: ~~~ function loadImageURL(cx, url) { var image = document.createElement("img"); image.addEventListener("load", function() { var color = cx.fillStyle, size = cx.lineWidth; cx.canvas.width = image.width; cx.canvas.height = image.height; cx.drawImage(image, 0, 0); cx.fillStyle = color; cx.strokeStyle = color; cx.lineWidth = size; }); image.src = url; } ~~~ [](http://eloquentjavascript.net/19_paint.html#p_fxJmZUi/g3)We want to change the size of the canvas to precisely fit the image. For some reason, changing the size of a canvas will cause its drawing context to forget configuration properties such as `fillStyle` and `lineWidth`, so the function saves those and restores them after it has updated the canvas size. [](http://eloquentjavascript.net/19_paint.html#p_BRkDNREP7P)The control for loading a local file uses the `FileReader` technique from[Chapter 18](http://eloquentjavascript.net/18_forms.html#filereader). Apart from the `readAsText` method we used there, such reader objects also have a method called `readAsDataURL`, which is exactly what we need here. We load the file that the user chose as a data URL and pass it to`loadImageURL` to put it into the canvas. ~~~ controls.openFile = function(cx) { var input = elt("input", {type: "file"}); input.addEventListener("change", function() { if (input.files.length == 0) return; var reader = new FileReader(); reader.addEventListener("load", function() { loadImageURL(cx, reader.result); }); reader.readAsDataURL(input.files[0]); }); return elt("div", null, "Open file: ", input); }; ~~~ [](http://eloquentjavascript.net/19_paint.html#p_3cQw/osn6v)Loading a file from a URL is even simpler. But with a text field, it is less clear when the user has finished writing the URL, so we can’t simply listen for`"change"` events. Instead, we will wrap the field in a form and respond when the form is submitted, either because the user pressed Enter or because they clicked the load button. ~~~ controls.openURL = function(cx) { var input = elt("input", {type: "text"}); var form = elt("form", null, "Open URL: ", input, elt("button", {type: "submit"}, "load")); form.addEventListener("submit", function(event) { event.preventDefault(); loadImageURL(cx, input.value); }); return form; }; ~~~ [](http://eloquentjavascript.net/19_paint.html#p_kU0DSJWhbC)We have now defined all the controls that our simple paint program needs, but it could still use a few more tools. ## [](http://eloquentjavascript.net/19_paint.html#h_COOrE3GQwB)Finishing up [](http://eloquentjavascript.net/19_paint.html#p_eRxkqfyNnx)We can easily add a text tool that uses `prompt` to ask the user which string it should draw. ~~~ tools.Text = function(event, cx) { var text = prompt("Text:", ""); if (text) { var pos = relativePos(event, cx.canvas); cx.font = Math.max(7, cx.lineWidth) + "px sans-serif"; cx.fillText(text, pos.x, pos.y); } }; ~~~ [](http://eloquentjavascript.net/19_paint.html#p_L3dnKFwhJy)You could add extra fields for the font size and the font, but for simplicity’s sake, we always use a sans-serif font and base the font size on the current brush size. The minimum size is 7 pixels because text smaller than that is unreadable. [](http://eloquentjavascript.net/19_paint.html#p_rTc/MzJCFC)Another indispensable tool for drawing amateurish computer graphics is the spray paint tool. This one draws dots in random locations under the brush as long as the mouse is held down, creating denser or less dense speckling based on how fast or slow the mouse moves. ~~~ tools.Spray = function(event, cx) { var radius = cx.lineWidth / 2; var area = radius * radius * Math.PI; var dotsPerTick = Math.ceil(area / 30); var currentPos = relativePos(event, cx.canvas); var spray = setInterval(function() { for (var i = 0; i < dotsPerTick; i++) { var offset = randomPointInRadius(radius); cx.fillRect(currentPos.x + offset.x, currentPos.y + offset.y, 1, 1); } }, 25); trackDrag(function(event) { currentPos = relativePos(event, cx.canvas); }, function() { clearInterval(spray); }); }; ~~~ [](http://eloquentjavascript.net/19_paint.html#p_PC1+gUob8b)The spray tool uses `setInterval` to spit out colored dots every 25 milliseconds as long as the mouse button is held down. The `trackDrag` function is used to keep `currentPos` pointing at the current mouse position and to turn off the interval when the mouse button is released. [](http://eloquentjavascript.net/19_paint.html#p_CRD18th69E)To determine how many dots to draw every time the interval fires, the function computes the area of the current brush and divides that by 30\. To find a random position under the brush, the `randomPointInRadius` function is used. ~~~ function randomPointInRadius(radius) { for (;;) { var x = Math.random() * 2 - 1; var y = Math.random() * 2 - 1; if (x * x + y * y <= 1) return {x: x * radius, y: y * radius}; } } ~~~ [](http://eloquentjavascript.net/19_paint.html#p_bpIPMJAN/Q)This function generates points in the square between (-1,-1) and (1,1). Using the Pythagorean theorem, it tests whether the generated point lies within a circle of radius 1\. As soon as the function finds such a point, it returns the point multiplied by the `radius` argument. [](http://eloquentjavascript.net/19_paint.html#p_8ATK7SQccV)The loop is necessary for a uniform distribution of dots. The straightforward way of generating a random point within a circle would be to use a random angle and distance and call `Math.sin` and `Math.cos` to create the corresponding point. But with that method, the dots are more likely to appear near the center of the circle. There are other ways around that, but they’re more complicated than the previous loop. [](http://eloquentjavascript.net/19_paint.html#p_Tj66LpaWji)We now have a functioning paint program. Run the code below to try it. ~~~ <link rel="stylesheet" href="css/paint.css"> <body> <script>createPaint(document.body);</script> </body> ~~~ ## [](http://eloquentjavascript.net/19_paint.html#h_TcUD2vzyMe)Exercises [](http://eloquentjavascript.net/19_paint.html#p_We+KsAI00v)There is still plenty of room for improvement in this program. Let’s add a few more features as exercises. ### [](http://eloquentjavascript.net/19_paint.html#h_AXnJJyTV3X)Rectangles [](http://eloquentjavascript.net/19_paint.html#p_4pYKvTwVL6)Define a tool called `Rectangle` that fills a rectangle (see the `fillRect` method from [Chapter 16](http://eloquentjavascript.net/16_canvas.html#fill_stroke)) with the current color. The rectangle should span from the point where the user pressed the mouse button to the point where they released it. Note that the latter might be above or to the left of the former. [](http://eloquentjavascript.net/19_paint.html#p_FZYD7EtH2K)Once it works, you’ll notice that it is somewhat jarring to not see the rectangle as you are dragging the mouse to select its size. Can you come up with a way to show some kind of rectangle during the dragging, without actually drawing to the canvas until the mouse button is released? [](http://eloquentjavascript.net/19_paint.html#p_XS+oeSzbgP)If nothing comes to mind, think back to the `position: absolute` style discussed in [Chapter 13](http://eloquentjavascript.net/13_dom.html#animation), which can be used to overlay a node on the rest of the document. The `pageX` and `pageY` properties of a mouse event can be used to position an element precisely under the mouse, by setting the `left`, `top`,`width`, and `height` styles to the correct pixel values. ~~~ <script> tools.Rectangle = function(event, cx) { // Your code here. }; </script> <link rel="stylesheet" href="css/paint.css"> <body> <script>createPaint(document.body);</script> </body> ~~~ ### [](http://eloquentjavascript.net/19_paint.html#h_kd7pYrPaOx)Color picker [](http://eloquentjavascript.net/19_paint.html#p_Jg9zx2ZHXs)Another tool that is commonly found in graphics programs is a color picker, which allows the user to click the picture and selects the color under the mouse pointer. Build this. [](http://eloquentjavascript.net/19_paint.html#p_2/ActRBD9m)For this tool, we need a way to access the content of the canvas. The`toDataURL` method more or less did that, but getting pixel information out of such a data URL is hard. Instead, we’ll use the `getImageData` method on the drawing context, which returns a rectangular piece of the image as an object with `width`, `height`, and `data` properties. The `data` property holds an array of numbers from 0 to 255, using four numbers to represent each pixel’s red, green, blue, and alpha (opaqueness) components. [](http://eloquentjavascript.net/19_paint.html#p_D6jTExt5KG)This example retrieves the numbers for a single pixel from a canvas once when the canvas is blank (all pixels are transparent black) and once when the pixel has been colored red. ~~~ function pixelAt(cx, x, y) { var data = cx.getImageData(x, y, 1, 1); console.log(data.data); } var canvas = document.createElement("canvas"); var cx = canvas.getContext("2d"); pixelAt(cx, 10, 10); // → [0, 0, 0, 0] cx.fillStyle = "red"; cx.fillRect(10, 10, 1, 1); pixelAt(cx, 10, 10); // → [255, 0, 0, 255] ~~~ [](http://eloquentjavascript.net/19_paint.html#p_LAAk4BgbLG)The arguments to `getImageData` indicate the starting x- and y-coordinates of the rectangle we want to retrieve, followed by its width and height. [](http://eloquentjavascript.net/19_paint.html#p_C46YeilKO0)Ignore transparency during this exercise and look only at the first three values for a given pixel. Also, do not worry about updating the color field when the user picks a color. Just make sure that the drawing context’s `fillStyle` and`strokeStyle` are set to the color under the mouse cursor. [](http://eloquentjavascript.net/19_paint.html#p_AvBU/YxtZc)Remember that these properties accept any color that CSS understands, which includes the `rgb(R, G, B)` style you saw in [Chapter 15](http://eloquentjavascript.net/15_game.html#game_css). [](http://eloquentjavascript.net/19_paint.html#p_TrX1yewkGx)The `getImageData` method is subject to the same restrictions as `toDataURL`—it will raise an error when the canvas contains pixels that originate from another domain. Use a `try/catch` statement to report such errors with an`alert` dialog. ~~~ <script> tools["Pick color"] = function(event, cx) { // Your code here. }; </script> <link rel="stylesheet" href="css/paint.css"> <body> <script>createPaint(document.body);</script> </body> ~~~ ### [](http://eloquentjavascript.net/19_paint.html#h_0M49T26QUl)Flood fill [](http://eloquentjavascript.net/19_paint.html#p_9+5Lhu8/5P)This is a more advanced exercise than the preceding two, and it will require you to design a nontrivial solution to a tricky problem. Make sure you have plenty of time and patience before starting to work on this exercise, and do not get discouraged by initial failures. [](http://eloquentjavascript.net/19_paint.html#p_OkFYhhueP3)A flood fill tool colors the pixel under the mouse and the surrounding pixels of the same color. For the purpose of this exercise, we will consider such a group to include all pixels that can be reached from our starting pixel by moving in single-pixel horizontal and vertical steps (not diagonal), without ever touching a pixel that has a color different from the starting pixel. [](http://eloquentjavascript.net/19_paint.html#p_6uRENk52k1)The following image illustrates the set of pixels colored when the flood fill tool is used at the marked pixel:  [](http://eloquentjavascript.net/19_paint.html#p_wwglcJ4sDv)The flood fill does not leak through diagonal gaps and does not touch pixels that are not reachable, even if they have the same color as the target pixel. [](http://eloquentjavascript.net/19_paint.html#p_76DZCicyt0)You will once again need `getImageData` to find out the color for each pixel. It is probably a good idea to fetch the whole image in one go and then pick out pixel data from the resulting array. The pixels are organized in this array in a similar way to the grid elements in [Chapter 7](http://eloquentjavascript.net/07_elife.html#grid), one row at a time, except that each pixel is represented by four values. The first value for the pixel at (*x*,*y*) is at position (*x* + *y* × width) × 4. [](http://eloquentjavascript.net/19_paint.html#p_jVploHhiTC)Do include the fourth (alpha) value this time since we want to be able to tell the difference between empty and black pixels. [](http://eloquentjavascript.net/19_paint.html#p_pGlySQoufy)Finding all adjacent pixels with the same color requires you to “walk” over the pixel surface, one pixel up, down, left, or right, as long as new same-colored pixels can be found. But you won’t find all pixels in a group on the first walk. Rather, you have to do something similar to the backtracking done by the regular expression matcher, described in [Chapter 9](http://eloquentjavascript.net/09_regexp.html#backtracking). Whenever more than one possible direction to proceed is seen, you must store all the directions you do not take immediately and look at them later, when you finish your current walk. [](http://eloquentjavascript.net/19_paint.html#p_QgadJ7T7nC)In a normal-sized picture, there are a *lot* of pixels. Thus, you must take care to do the minimal amount of work required or your program will take a very long time to run. For example, every walk must ignore pixels seen by previous walks so that it does not redo work that has already been done. [](http://eloquentjavascript.net/19_paint.html#p_tPAU+iaccT)I recommend calling `fillRect` for individual pixels when a pixel that should be colored is found, and keeping some data structure that tells you about all the pixels that have already been looked at. ~~~ <script> tools["Flood fill"] = function(event, cx) { // Your code here. }; </script> <link rel="stylesheet" href="css/paint.css"> <body> <script>createPaint(document.body);</script> </body> ~~~ Given a pair of starting coordinates and the image data for the whole canvas, this approach should work: 1. Create an array to hold information about already colored coordinates. 2. Create a work list array to hold coordinates that must be looked at. Put the start position in it. 3. When the work list is empty, we are done. 4. Remove one pair of coordinates from the work list. 5. If those coordinates are already in our array of colored pixels, go back to step 3. 6. Color the pixel at the current coordinates and add the coordinates to the array of colored pixels. 7. Add the coordinates of each adjacent pixel whose color is the same as the starting pixel’s original color to the work list. 8. Return to step 3. [](http://eloquentjavascript.net/19_paint.html#p_K10R222PDR)The work list can simply be an array of vector objects. The data structure that tracks colored pixels will be consulted *very* often. Searching through the whole thing every time a new pixel is visited will take a lot of time. You could instead create an array that has a value in it for every pixel, using again the x + y × width scheme for associating positions with pixels. When checking whether a pixel has been colored already, you could directly access the field corresponding to the current pixel. [](http://eloquentjavascript.net/19_paint.html#p_238ysWaiXF)You can compare colors by running over the relevant part of the data array, comparing one field at a time. Or you can “condense” a color to a single number or string and compare those. When doing this, ensure that every color produces a unique value. For example, simply adding the color’s components is not safe since multiple colors will have the same sum. [](http://eloquentjavascript.net/19_paint.html#p_JPRZIpzUnJ)When enumerating the neighbors of a given point, take care to exclude neighbors that are not inside of the canvas or your program might run off into one direction forever.

## Chapter 18 # Forms and Form Fields > [](http://eloquentjavascript.net/18_forms.html#p_HT1/Oht2V3)I shall this very day, at Doctor’s feast, > My bounden service duly pay thee. > But one thing!—For insurance’ sake, I pray thee, > Grant me a line or two, at least. > > Mephistopheles, in Goethe's Faust [](http://eloquentjavascript.net/18_forms.html#p_7R2RnNlXWq)Forms were introduced briefly in the [previous chapter](http://eloquentjavascript.net/17_http.html#http_forms) as a way to *submit*information provided by the user over HTTP. They were designed for a pre-JavaScript Web, assuming that interaction with the server always happens by navigating to a new page. [](http://eloquentjavascript.net/18_forms.html#p_j1nZNHqVd+)But their elements are part of the DOM like the rest of the page, and the DOM elements that represent form fields support a number of properties and events that are not present on other elements. These make it possible to inspect and control such input fields with JavaScript programs and do things such as adding functionality to a traditional form or using forms and fields as building blocks in a JavaScript application. ## [](http://eloquentjavascript.net/18_forms.html#h_6LaFJxI4rf)Fields [](http://eloquentjavascript.net/18_forms.html#p_pfSLM800x5)A web form consists of any number of input fields grouped in a `<form>` tag. HTML allows a number of different styles of fields, ranging from simple on/off checkboxes to drop-down menus and fields for text input. This book won’t try to comprehensively discuss all field types, but we will start with a rough overview. [](http://eloquentjavascript.net/18_forms.html#p_BtwDLFMJGW)A lot of field types use the `<input>` tag. This tag’s `type` attribute is used to select the field’s style. These are some commonly used `<input>` types: | `text` | A single-line text field | | `password` | Same as `text` but hides the text that is typed | | `checkbox` | An on/off switch | | `radio` | (Part of) a multiple-choice field | | `file` | Allows the user to choose a file from their computer | [](http://eloquentjavascript.net/18_forms.html#p_LZvIgLQRq6)Form fields do not necessarily have to appear in a `<form>` tag. You can put them anywhere in a page. Such fields cannot be submitted (only a form as a whole can), but when responding to input with JavaScript, we often do not want to submit our fields normally anyway. ~~~ <p><input type="text" value="abc"> (text)</p> <p><input type="password" value="abc"> (password)</p> <p><input type="checkbox" checked> (checkbox)</p> <p><input type="radio" value="A" name="choice"> <input type="radio" value="B" name="choice" checked> <input type="radio" value="C" name="choice"> (radio)</p> <p><input type="file"> (file)</p> ~~~ [](http://eloquentjavascript.net/18_forms.html#p_F4A/ei02Wb)The JavaScript interface for such elements differs with the type of the element. We’ll go over each of them later in the chapter. [](http://eloquentjavascript.net/18_forms.html#p_dU8xgBYYQy)Multiline text fields have their own tag, `<textarea>`, mostly because using an attribute to specify a multiline starting value would be awkward. The`<textarea>` requires a matching `</textarea>` closing tag and uses the text between those two, instead of using its `value` attribute, as starting text. ~~~ <textarea> one two three </textarea> ~~~ [](http://eloquentjavascript.net/18_forms.html#p_df6Ss3FBHe)Finally, the `<select>` tag is used to create a field that allows the user to select from a number of predefined options. ~~~ <select> <option>Pancakes</option> <option>Pudding</option> <option>Ice cream</option> </select> ~~~ [](http://eloquentjavascript.net/18_forms.html#p_Po+2LjUu/R)Whenever the value of a form field changes, it fires a `"change"` event. ## [](http://eloquentjavascript.net/18_forms.html#h_/n9VuL9o/U)Focus [](http://eloquentjavascript.net/18_forms.html#p_sv9Kp9pqvi)Unlike most elements in an HTML document, form fields can get *keyboard focus*. When clicked—or activated in some other way—they become the currently active element, the main recipient of keyboard input. [](http://eloquentjavascript.net/18_forms.html#p_sPi491t84y)If a document has a text field, text typed will end up in there only when the field is focused. Other fields respond differently to keyboard events. For example, a `<select>` menu tries to move to the option that contains the text the user typed and responds to the arrow keys by moving its selection up and down. [](http://eloquentjavascript.net/18_forms.html#p_bTNSIbb+d4)We can control focus from JavaScript with the `focus` and `blur` methods. The first moves focus to the DOM element it is called on, and the second removes focus. The value in `document.activeElement` corresponds to the currently focused element. ~~~ <input type="text"> <script> document.querySelector("input").focus(); console.log(document.activeElement.tagName); // → INPUT document.querySelector("input").blur(); console.log(document.activeElement.tagName); // → BODY </script> ~~~ [](http://eloquentjavascript.net/18_forms.html#p_5/Idt5wdrL)For some pages, the user is expected to want to interact with a form field immediately. JavaScript can be used to focus this field when the document is loaded, but HTML also provides the `autofocus` attribute, which produces the same effect but lets the browser know what we are trying to achieve. This makes it possible for the browser to disable the behavior when it is not appropriate, such as when the user has focused something else. ~~~ <input type="text" autofocus> ~~~ [](http://eloquentjavascript.net/18_forms.html#p_IgspvfRub5)Browsers traditionally also allow the user to move the focus through the document by pressing the Tab key. We can influence the order in which elements receive focus with the `tabindex` attribute. The following example document will let focus jump from the text input to the OK button, rather than going through the help link first: ~~~ <input type="text" tabindex=1> <a href=".">(help)</a> <button onclick="console.log('ok')" tabindex=2>OK</button> ~~~ [](http://eloquentjavascript.net/18_forms.html#p_CSZoExUC/a)By default, most types of HTML elements cannot be focused. But you can add a`tabindex` attribute to any element, which will make it focusable. ## [](http://eloquentjavascript.net/18_forms.html#h_l2z9T1FDym)Disabled fields [](http://eloquentjavascript.net/18_forms.html#p_yO5FA2RLmN)All form fields can be *disabled* through their `disabled` attribute, which also exists as a property on the element’s DOM object. ~~~ <button>I'm all right</button> <button disabled>I'm out</button> ~~~ [](http://eloquentjavascript.net/18_forms.html#p_GR2MtLESGu)Disabled fields cannot be focused or changed, and unlike active fields, they usually look gray and faded. [](http://eloquentjavascript.net/18_forms.html#p_xuHzLzjgZ1)When a program is in the process of handling an action caused by some button or other control, which might require communication with the server and thus take a while, it can be a good idea to disable the control until the action finishes. That way, when the user gets impatient and clicks it again, they don’t accidentally repeat their action. ## [](http://eloquentjavascript.net/18_forms.html#h_R8SZxHmDt0)The form as a whole [](http://eloquentjavascript.net/18_forms.html#p_G8aitsjxoL)When a field is contained in a `<form>` element, its DOM element will have a property `form` linking back to the form’s DOM element. The `<form>` element, in turn, has a property called `elements` that contains an array-like collection of the fields inside it. [](http://eloquentjavascript.net/18_forms.html#p_7Ds/EaFyX5)The `name` attribute of a form field determines the way its value will be identified when the form is submitted. It can also be used as a property name when accessing the form’s `elements` property, which acts both as an array-like object (accessible by number) and a map (accessible by name). ~~~ <form action="example/submit.html"> Name: <input type="text" name="name"><br> Password: <input type="password" name="password"><br> <button type="submit">Log in</button> </form> <script> var form = document.querySelector("form"); console.log(form.elements[1].type); // → password console.log(form.elements.password.type); // → password console.log(form.elements.name.form == form); // → true </script> ~~~ [](http://eloquentjavascript.net/18_forms.html#p_DOQDGp4ZWr)A button with a `type` attribute of `submit` will, when pressed, cause the form to be submitted. Pressing Enter when a form field is focused has the same effect. [](http://eloquentjavascript.net/18_forms.html#p_MVbboPCoBM)Submitting a form normally means that the browser navigates to the page indicated by the form’s `action` attribute, using either a `GET` or a `POST` request. But before that happens, a `"submit"` event is fired. This event can be handled by JavaScript, and the handler can prevent the default behavior by calling`preventDefault` on the event object. ~~~ <form action="example/submit.html"> Value: <input type="text" name="value"> <button type="submit">Save</button> </form> <script> var form = document.querySelector("form"); form.addEventListener("submit", function(event) { console.log("Saving value", form.elements.value.value); event.preventDefault(); }); </script> ~~~ [](http://eloquentjavascript.net/18_forms.html#p_s3PgVwicoH)Intercepting `"submit"` events in JavaScript has various uses. We can write code to verify that the values the user entered make sense and immediately show an error message instead of submitting the form when they don’t. Or we can disable the regular way of submitting the form entirely, as in the previous example, and have our program handle the input, possibly using`XMLHttpRequest` to send it over to a server without reloading the page. ## [](http://eloquentjavascript.net/18_forms.html#h_+RaBROT23F)Text fields [](http://eloquentjavascript.net/18_forms.html#p_yXEHtldoGS)Fields created by `<input>` tags with a type of `text` or `password`, as well as`textarea` tags, share a common interface. Their DOM elements have a `value`property that holds their current content as a string value. Setting this property to another string changes the field’s content. [](http://eloquentjavascript.net/18_forms.html#p_L8Jf4YksDD)The `selectionStart` and `selectionEnd` properties of text fields give us information about the cursor and selection in the text. When nothing is selected, these two properties hold the same number, indicating the position of the cursor. For example, 0 indicates the start of the text, and 10 indicates the cursor is after the 10th character. When part of the field is selected, the two properties will differ, giving us the start and end of the selected text. Like`value`, these properties may also be written to. [](http://eloquentjavascript.net/18_forms.html#p_xbJja7lj1o)As an example, imagine you are writing an article about Khasekhemwy but have some trouble spelling his name. The following code wires up a`<textarea>` tag with an event handler that, when you press F2, inserts the string “Khasekhemwy” for you. ~~~ <textarea></textarea> <script> var textarea = document.querySelector("textarea"); textarea.addEventListener("keydown", function(event) { // The key code for F2 happens to be 113 if (event.keyCode == 113) { replaceSelection(textarea, "Khasekhemwy"); event.preventDefault(); } }); function replaceSelection(field, word) { var from = field.selectionStart, to = field.selectionEnd; field.value = field.value.slice(0, from) + word + field.value.slice(to); // Put the cursor after the word field.selectionStart = field.selectionEnd = from + word.length; } </script> ~~~ [](http://eloquentjavascript.net/18_forms.html#p_Gwpy8qmRbK)The `replaceSelection` function replaces the currently selected part of a text field’s content with the given word and then moves the cursor after that word so that the user can continue typing. [](http://eloquentjavascript.net/18_forms.html#p_n9HEUfXSP7)The `"change"` event for a text field does not fire every time something is typed. Rather, it fires when the field loses focus after its content was changed. To respond immediately to changes in a text field, you should register a handler for the `"input"` event instead, which fires for every time the user types a character, deletes text, or otherwise manipulates the field’s content. [](http://eloquentjavascript.net/18_forms.html#p_EA90jWvW4M)The following example shows a text field and a counter showing the current length of the text entered: ~~~ <input type="text"> length: <span id="length">0</span> <script> var text = document.querySelector("input"); var output = document.querySelector("#length"); text.addEventListener("input", function() { output.textContent = text.value.length; }); </script> ~~~ ## [](http://eloquentjavascript.net/18_forms.html#h_axUK+XNvTN)Checkboxes and radio buttons [](http://eloquentjavascript.net/18_forms.html#p_DFZwKUx9E+)A checkbox field is a simple binary toggle. Its value can be extracted or changed through its `checked` property, which holds a Boolean value. ~~~ <input type="checkbox" id="purple"> <label for="purple">Make this page purple</label> <script> var checkbox = document.querySelector("#purple"); checkbox.addEventListener("change", function() { document.body.style.background = checkbox.checked ? "mediumpurple" : ""; }); </script> ~~~ [](http://eloquentjavascript.net/18_forms.html#p_AcH0aFeYIj)The `<label>` tag is used to associate a piece of text with an input field. Its `for`attribute should refer to the `id` of the field. Clicking the label will activate the field, which focuses it and toggles its value when it is a checkbox or radio button. [](http://eloquentjavascript.net/18_forms.html#p_vNsiBxZlre)A radio button is similar to a checkbox, but it’s implicitly linked to other radio buttons with the same `name` attribute so that only one of them can be active at any time. ~~~ Color: <input type="radio" name="color" value="mediumpurple"> Purple <input type="radio" name="color" value="lightgreen"> Green <input type="radio" name="color" value="lightblue"> Blue <script> var buttons = document.getElementsByName("color"); function setColor(event) { document.body.style.background = event.target.value; } for (var i = 0; i < buttons.length; i++) buttons[i].addEventListener("change", setColor); </script> ~~~ [](http://eloquentjavascript.net/18_forms.html#p_TLnCgOnKU5)The `document.getElementsByName` method gives us all elements with a given`name` attribute. The example loops over those (with a regular `for` loop, not`forEach`, because the returned collection is not a real array) and registers an event handler for each element. Remember that event objects have a `target`property referring to the element that triggered the event. This is often useful in event handlers like this one, which will be called on different elements and need some way to access the current target. ## [](http://eloquentjavascript.net/18_forms.html#h_eaB6RprDK3)Select fields [](http://eloquentjavascript.net/18_forms.html#p_C43QMBKrcK)Select fields are conceptually similar to radio buttons—they also allow the user to choose from a set of options. But where a radio button puts the layout of the options under our control, the appearance of a `<select>` tag is determined by the browser. [](http://eloquentjavascript.net/18_forms.html#p_NtH8drqnHG)Select fields also have a variant that is more akin to a list of checkboxes, rather than radio boxes. When given the `multiple` attribute, a `<select>` tag will allow the user to select any number of options, rather than just a single option. ~~~ <select multiple> <option>Pancakes</option> <option>Pudding</option> <option>Ice cream</option> </select> ~~~ [](http://eloquentjavascript.net/18_forms.html#p_ZdHBLZYAi8)This will, in most browsers, show up differently than a non-`multiple` select field, which is commonly drawn as a *drop-down* control that shows the options only when you open it. [](http://eloquentjavascript.net/18_forms.html#p_Hyw0pVZgH2)The `size` attribute to the `<select>` tag is used to set the number of options that are visible at the same time, which gives you explicit control over the drop-down’s appearance. For example, setting the `size` attribute to `"3"` will make the field show three lines, whether it has the `multiple` option enabled or not. [](http://eloquentjavascript.net/18_forms.html#p_6B1JfRLbxg)Each `<option>` tag has a value. This value can be defined with a `value`attribute, but when that is not given, the text inside the option will count as the option’s value. The `value` property of a `<select>` element reflects the currently selected option. For a `multiple` field, though, this property doesn’t mean much since it will give the value of only *one* of the currently selected options. [](http://eloquentjavascript.net/18_forms.html#p_Aum9ltngdE)The `<option>` tags for a `<select>` field can be accessed as an array-like object through the field’s `options` property. Each option has a property called`selected`, which indicates whether that option is currently selected. The property can also be written to select or deselect an option. [](http://eloquentjavascript.net/18_forms.html#p_IxfFgaCJNR)The following example extracts the selected values from a `multiple` select field and uses them to compose a binary number from individual bits. Hold Ctrl (or Command on a Mac) to select multiple options. ~~~ <select multiple> <option value="1">0001</option> <option value="2">0010</option> <option value="4">0100</option> <option value="8">1000</option> </select> = <span id="output">0</span> <script> var select = document.querySelector("select"); var output = document.querySelector("#output"); select.addEventListener("change", function() { var number = 0; for (var i = 0; i < select.options.length; i++) { var option = select.options[i]; if (option.selected) number += Number(option.value); } output.textContent = number; }); </script> ~~~ ## [](http://eloquentjavascript.net/18_forms.html#h_tK84z183/8)File fields [](http://eloquentjavascript.net/18_forms.html#p_12DuX873Jk)File fields were originally designed as a way to upload files from the browser’s machine through a form. In modern browsers, they also provide a way to read such files from JavaScript programs. The field acts as a manner of gatekeeper. The script cannot simply start reading private files from the user’s computer, but if the user selects a file in such a field, the browser interprets that action to mean that the script may read the file. [](http://eloquentjavascript.net/18_forms.html#p_poF7nvM5XY)A file field usually looks like a button labeled with something like “choose file” or “browse”, with information about the chosen file next to it. ~~~ <input type="file"> <script> var input = document.querySelector("input"); input.addEventListener("change", function() { if (input.files.length > 0) { var file = input.files[0]; console.log("You chose", file.name); if (file.type) console.log("It has type", file.type); } }); </script> ~~~ [](http://eloquentjavascript.net/18_forms.html#p_J8fD00m75/)The `files` property of a file field element is an array-like object (again, not a real array) containing the files chosen in the field. It is initially empty. The reason there isn’t simply a `file` property is that file fields also support a`multiple` attribute, which makes it possible to select multiple files at the same time. [](http://eloquentjavascript.net/18_forms.html#p_VjLruEMwm5)Objects in the `files` property have properties such as `name` (the filename),`size` (the file’s size in bytes), and `type` (the media type of the file, such as`text/plain` or `image/jpeg`). [](http://eloquentjavascript.net/18_forms.html#p_ezd+X1Rw54)What it does not have is a property that contains the content of the file. Getting at that is a little more involved. Since reading a file from disk can take time, the interface will have to be asynchronous to avoid freezing the document. You can think of the `FileReader` constructor as being similar to `XMLHttpRequest` but for files. ~~~ <input type="file" multiple> <script> var input = document.querySelector("input"); input.addEventListener("change", function() { Array.prototype.forEach.call(input.files, function(file) { var reader = new FileReader(); reader.addEventListener("load", function() { console.log("File", file.name, "starts with", reader.result.slice(0, 20)); }); reader.readAsText(file); }); }); </script> ~~~ [](http://eloquentjavascript.net/18_forms.html#p_ig0flob0QZ)Reading a file is done by creating a `FileReader` object, registering a `"load"`event handler for it, and calling its `readAsText` method, giving it the file we want to read. Once loading finishes, the reader’s `result` property contains the file’s content. [](http://eloquentjavascript.net/18_forms.html#p_mwjAqF7An3)The example uses `Array.prototype.forEach` to iterate over the array since in a normal loop it would be awkward to get the correct `file` and `reader` objects from the event handler. The variables would be shared by all iterations of the loop. [](http://eloquentjavascript.net/18_forms.html#p_f5Kaoq//VK)`FileReader`s also fire an `"error"` event when reading the file fails for any reason. The error object itself will end up in the reader’s `error` property. If you don’t want to remember the details of yet another inconsistent asynchronous interface, you could wrap it in a `Promise` (see [Chapter 17](http://eloquentjavascript.net/17_http.html#promises)) like this: ~~~ function readFile(file) { return new Promise(function(succeed, fail) { var reader = new FileReader(); reader.addEventListener("load", function() { succeed(reader.result); }); reader.addEventListener("error", function() { fail(reader.error); }); reader.readAsText(file); }); } ~~~ [](http://eloquentjavascript.net/18_forms.html#p_IGRea9SXwb)It is possible to read only part of a file by calling `slice` on it and passing the result (a so-called *blob* object) to the file reader. ## [](http://eloquentjavascript.net/18_forms.html#h_xMhbz7W4BY)Storing data client-side [](http://eloquentjavascript.net/18_forms.html#p_JABB1DfeON)Simple HTML pages with a bit of JavaScript can be a great medium for “mini applications”—small helper programs that automate everyday things. By connecting a few form fields with event handlers, you can do anything from converting between degrees Celsius and Fahrenheit to computing passwords from a master password and a website name. [](http://eloquentjavascript.net/18_forms.html#p_jnKM+3wefQ)When such an application needs to remember something between sessions, you cannot use JavaScript variables since those are thrown away every time a page is closed. You could set up a server, connect it to the Internet, and have your application store something there. We will see how to do that in [Chapter 20](http://eloquentjavascript.net/20_node.html#node). But this adds a lot of extra work and complexity. Sometimes it is enough to just keep the data in the browser. But how? [](http://eloquentjavascript.net/18_forms.html#p_Coj/4Qj2Wo)You can store string data in a way that survives page reloads by putting it in the`localStorage` object. This object allows you to file string values under names (also strings), as in this example: ~~~ localStorage.setItem("username", "marijn"); console.log(localStorage.getItem("username")); // → marijn localStorage.removeItem("username"); ~~~ [](http://eloquentjavascript.net/18_forms.html#p_6Jr54na1O5)A value in `localStorage` sticks around until it is overwritten, it is removed with `removeItem`, or the user clears their local data. [](http://eloquentjavascript.net/18_forms.html#p_QTrhARZTb/)Sites from different domains get different storage compartments. That means data stored in `localStorage` by a given website can, in principle, only be read (and overwritten) by scripts on that same site. [](http://eloquentjavascript.net/18_forms.html#p_s4JKCuTdO4)Browsers also enforce a limit on the size of the data a site can store in`localStorage`, typically on the order of a few megabytes. That restriction, along with the fact that filling up people’s hard drives with junk is not really profitable, prevents this feature from eating up too much space. [](http://eloquentjavascript.net/18_forms.html#p_YJCl32s1ep)The following code implements a simple note-taking application. It keeps the user’s notes as an object, associating note titles with content strings. This object is encoded as JSON and stored in `localStorage`. The user can select a note from a `<select>` field and change that note’s text in a `<textarea>`. A note can be added by clicking a button. ~~~ Notes: <select id="list"></select> <button onclick="addNote()">new</button><br> <textarea id="currentnote" style="width: 100%; height: 10em"> </textarea> <script> var list = document.querySelector("#list"); function addToList(name) { var option = document.createElement("option"); option.textContent = name; list.appendChild(option); } // Initialize the list from localStorage var notes = JSON.parse(localStorage.getItem("notes")) || {"shopping list": ""}; for (var name in notes) if (notes.hasOwnProperty(name)) addToList(name); function saveToStorage() { localStorage.setItem("notes", JSON.stringify(notes)); } var current = document.querySelector("#currentnote"); current.value = notes[list.value]; list.addEventListener("change", function() { current.value = notes[list.value]; }); current.addEventListener("change", function() { notes[list.value] = current.value; saveToStorage(); }); function addNote() { var name = prompt("Note name", ""); if (!name) return; if (!notes.hasOwnProperty(name)) { notes[name] = ""; addToList(name); saveToStorage(); } list.value = name; current.value = notes[name]; } </script> ~~~ [](http://eloquentjavascript.net/18_forms.html#p_IXFtJG2tv8)The script initializes the `notes` variable to the value stored in `localStorage`or, if that is missing, to a simple object with only an empty `"shopping list"`note in it. Reading a field that does not exist from `localStorage` will yield`null`. Passing `null` to `JSON.parse` will make it parse the string `"null"` and return `null`. Thus, the `||` operator can be used to provide a default value in a situation like this. [](http://eloquentjavascript.net/18_forms.html#p_JoUCwxZ7ZG)Whenever the note data changes (when a new note is added or an existing note changed), the `saveToStorage` function is called to update the storage field. If this application was intended to handle thousands of notes, rather than a handful, this would be too expensive, and we’d have to come up with a more complicated way to store them, such as giving each note its own storage field. [](http://eloquentjavascript.net/18_forms.html#p_jDISw6dQzZ)When the user adds a new note, the code must update the text field explicitly, even though the `<select>` field has a `"change"` handler that does the same thing. This is necessary because `"change"` events fire only when the *user*changes the field’s value, not when a script does it. [](http://eloquentjavascript.net/18_forms.html#p_McB+3/vKwy)There is another object similar to `localStorage` called `sessionStorage`. The difference between the two is that the content of `sessionStorage` is forgotten at the end of each session, which for most browsers means whenever the browser is closed. ## [](http://eloquentjavascript.net/18_forms.html#h_ErccPg/l98)Summary [](http://eloquentjavascript.net/18_forms.html#p_ZKWYUfmZEa)HTML can express various types of form fields, such as text fields, checkboxes, multiple-choice fields, and file pickers. [](http://eloquentjavascript.net/18_forms.html#p_1zn4gfsnbO)Such fields can be inspected and manipulated with JavaScript. They fire the`"change"` event when changed, the `"input"` event when text is typed, and various keyboard events. These events allow us to notice when the user is interacting with the fields. Properties like `value` (for text and select fields) or`checked` (for checkboxes and radio buttons) are used to read or set the field’s content. [](http://eloquentjavascript.net/18_forms.html#p_ga1YeRsvqD)When a form is submitted, its `"submit"` event fires. A JavaScript handler can call `preventDefault` on that event to prevent the submission from happening. Form field elements do not have to be wrapped in `<form>` tags. [](http://eloquentjavascript.net/18_forms.html#p_4GN43j/R8o)When the user has selected a file from their local file system in a file picker field, the `FileReader` interface can be used to access the content of this file from a JavaScript program. [](http://eloquentjavascript.net/18_forms.html#p_KjH+jrxywQ)The `localStorage` and `sessionStorage` objects can be used to save information in a way that survives page reloads. The first saves the data forever (or until the user decides to clear it), and the second saves it until the browser is closed. ## [](http://eloquentjavascript.net/18_forms.html#h_TcUD2vzyMe)Exercises ### [](http://eloquentjavascript.net/18_forms.html#h_wTXvIH5Wds)A JavaScript workbench [](http://eloquentjavascript.net/18_forms.html#p_EyD8Y/DOXP)Build an interface that allows people to type and run pieces of JavaScript code. [](http://eloquentjavascript.net/18_forms.html#p_eGmPVJLy/G)Put a button next to a `<textarea>` field, which, when pressed, uses the`Function` constructor we saw in [Chapter 10](http://eloquentjavascript.net/10_modules.html#eval) to wrap the text in a function and call it. Convert the return value of the function, or any error it raised, to a string and display it after the text field. ~~~ <textarea id="code">return "hi";</textarea> <button id="button">Run</button> <pre id="output"></pre> <script> // Your code here. </script> ~~~ ### [](http://eloquentjavascript.net/18_forms.html#h_pGljYPYc57)Autocompletion [](http://eloquentjavascript.net/18_forms.html#p_xkMXec3dUE)Extend a text field so that when the user types, a list of suggested values is shown below the field. You have an array of possible values available and should show those that start with the text that was typed. When a suggestion is clicked, replace the text field’s current value with it. ~~~ <input type="text" id="field"> <div id="suggestions" style="cursor: pointer"></div> <script> // Builds up an array with global variable names, like // 'alert', 'document', and 'scrollTo' var terms = []; for (var name in window) terms.push(name); // Your code here. </script> ~~~ ### [](http://eloquentjavascript.net/18_forms.html#h_wZdunPqDkM)Conway’s Game of Life [](http://eloquentjavascript.net/18_forms.html#p_Jvg4FGRdgU)Conway’s Game of Life is a simple simulation that creates artificial “life” on a grid, each cell of which is either live or not. Each generation (turn), the following rules are applied: * [](http://eloquentjavascript.net/18_forms.html#p_H6UexnLxhi) Any live cell with fewer than two or more than three live neighbors dies. * [](http://eloquentjavascript.net/18_forms.html#p_jSyY/y/HMp) Any live cell with two or three live neighbors lives on to the next generation. * [](http://eloquentjavascript.net/18_forms.html#p_y/DnAd+/CJ) Any dead cell with exactly three live neighbors becomes a live cell. [](http://eloquentjavascript.net/18_forms.html#p_iKiryFZsbT)A neighbor is defined as any adjacent cell, including diagonally adjacent ones. [](http://eloquentjavascript.net/18_forms.html#p_9dLS2RK5SZ)Note that these rules are applied to the whole grid at once, not one square at a time. That means the counting of neighbors is based on the situation at the start of the generation, and changes happening to neighbor cells during this generation should not influence the new state of a given cell. [](http://eloquentjavascript.net/18_forms.html#p_W0m8oVpi7M)Implement this game using whichever data structure you find appropriate. Use`Math.random` to populate the grid with a random pattern initially. Display it as a grid of checkbox fields, with a button next to it to advance to the next generation. When the user checks or unchecks the checkboxes, their changes should be included when computing the next generation. ~~~ <div id="grid"></div> <button id="next">Next generation</button> <script> // Your code here. </script> ~~~ To solve the problem of having the changes conceptually happen at the same time, try to see the computation of a generation as a pure function, which takes one grid and produces a new grid that represents the next turn. [](http://eloquentjavascript.net/18_forms.html#p_xuG6OwlRd8)Representing the grid can be done in any of the ways shown in Chapters [7](http://eloquentjavascript.net/07_elife.html#grid) and[15](http://eloquentjavascript.net/15_game.html#level). Counting live neighbors can be done with two nested loops, looping over adjacent coordinates. Take care not to count cells outside of the field and to ignore the cell in the center, whose neighbors we are counting. [](http://eloquentjavascript.net/18_forms.html#p_KHncyuOOIQ)Making changes to checkboxes take effect on the next generation can be done in two ways. An event handler could notice these changes and update the current grid to reflect them, or you could generate a fresh grid from the values in the checkboxes before computing the next turn. [](http://eloquentjavascript.net/18_forms.html#p_wu3Ty9I4d4)If you choose to go with event handlers, you might want to attach attributes that identify the position that each checkbox corresponds to so that it is easy to find out which cell to change. [](http://eloquentjavascript.net/18_forms.html#p_XFp+gOIONt)To draw the grid of checkboxes, you either can use a `<table>` element (see[Chapter 13](http://eloquentjavascript.net/13_dom.html#exercise_table)) or simply put them all in the same element and put `<br>` (line break) elements between the rows.

## Chapter 17 # HTTP > [](http://eloquentjavascript.net/17_http.html#p_QvqpbswJB1)The dream behind the Web is of a common information space in which we communicate by sharing information. Its universality is essential: the fact that a hypertext link can point to anything, be it personal, local or global, be it draft or highly polished. > > Tim Berners-Lee, The World Wide Web: A very short personal history [](http://eloquentjavascript.net/17_http.html#p_C6+RXvCvFu)The *Hypertext Transfer Protocol*, already mentioned in [Chapter 12](http://eloquentjavascript.net/12_browser.html#web), is the mechanism through which data is requested and provided on the World Wide Web. This chapter describes the protocol in more detail and explains the way browser JavaScript has access to it. ## [](http://eloquentjavascript.net/17_http.html#h_vXdZCu/Tty)The protocol [](http://eloquentjavascript.net/17_http.html#p_uAGANEYkBZ)If you type *eloquentjavascript.net/17_http.html* into your browser’s address bar, the browser first looks up the address of the server associated with*eloquentjavascript.net* and tries to open a TCP connection to it on port 80, the default port for HTTP traffic. If the server exists and accepts the connection, the browser sends something like this: ~~~ GET /17_http.html HTTP/1.1 Host: eloquentjavascript.net User-Agent: Your browser's name ~~~ [](http://eloquentjavascript.net/17_http.html#p_cpaXDnG/vq)Then the server responds, through that same connection. ~~~ HTTP/1.1 200 OK Content-Length: 65585 Content-Type: text/html Last-Modified: Wed, 09 Apr 2014 10:48:09 GMT <!doctype html> ... the rest of the document ~~~ [](http://eloquentjavascript.net/17_http.html#p_RRCiB3G5cb)The browser then takes the part of the response after the blank line and displays it as an HTML document. [](http://eloquentjavascript.net/17_http.html#p_EF5P7+qf9n)The information sent by the client is called the *request*. It starts with this line: ~~~ GET /17_http.html HTTP/1.1 ~~~ [](http://eloquentjavascript.net/17_http.html#p_0OgmCKjRs2)The first word is the *method* of the request. `GET` means that we want to *get* the specified resource. Other common methods are `DELETE` to delete a resource,`PUT` to replace it, and `POST` to send information to it. Note that the server is not obliged to carry out every request it gets. If you walk up to a random website and tell it to `DELETE` its main page, it’ll probably refuse. [](http://eloquentjavascript.net/17_http.html#p_dRtHLefq7L)The part after the method name is the path of the resource the request applies to. In the simplest case, a resource is simply a file on the server, but the protocol doesn’t require it to be. A resource may be anything that can be transferred *as if* it is a file. Many servers generate the responses they produce on the fly. For example, if you open [*twitter.com/marijnjh*](http://twitter.com/marijnjh), the server looks in its database for a user named *marijnjh*, and if it finds one, it will generate a profile page for that user. [](http://eloquentjavascript.net/17_http.html#p_M95qaU9kuN)After the resource path, the first line of the request mentions `HTTP/1.1` to indicate the version of the HTTP protocol it is using. [](http://eloquentjavascript.net/17_http.html#p_pIMelD1mZh)The server’s response will start with a version as well, followed by the status of the response, first as a three-digit status code and then as a human-readable string. ~~~ HTTP/1.1 200 OK ~~~ [](http://eloquentjavascript.net/17_http.html#p_I8BGMaErRo)Status codes starting with a 2 indicate that the request succeeded. Codes starting with 4 mean there was something wrong with the request. 404 is probably the most famous HTTP status code—it means that the resource that was requested could not be found. Codes that start with 5 mean an error happened on the server and the request is not to blame. [](http://eloquentjavascript.net/17_http.html#p_CA64oLhdLd)The first line of a request or response may be followed by any number of*headers*. These are lines in the form “name: value” that specify extra information about the request or response. These headers were part of the example response: ~~~ Content-Length: 65585 Content-Type: text/html Last-Modified: Wed, 09 Apr 2014 10:48:09 GMT ~~~ [](http://eloquentjavascript.net/17_http.html#p_04zcUGZ8sT)This tells us the size and type of the response document. In this case, it is an HTML document of 65,585 bytes. It also tells us when that document was last modified. [](http://eloquentjavascript.net/17_http.html#p_dIM0DmAcMY)For the most part, a client or server decides which headers to include in a request or response, though a few headers are required. For example, the `Host`header, which specifies the hostname, should be included in a request because a server might be serving multiple hostnames on a single IP address, and without that header, the server won’t know which host the client is trying to talk to. [](http://eloquentjavascript.net/17_http.html#p_DeF17lBSIn)After the headers, both requests and responses may include a blank line followed by a *body*, which contains the data being sent. `GET` and `DELETE`requests don’t send along any data, but `PUT` and `POST` requests do. Similarly, some response types, such as error responses, do not require a body. ## [](http://eloquentjavascript.net/17_http.html#h_G1xxLfiYeu)Browsers and HTTP [](http://eloquentjavascript.net/17_http.html#p_FaMpawa4Gl)As we saw in the example, a browser will make a request when we enter a URL in its address bar. When the resulting HTML page references other files, such as images and JavaScript files, those are also fetched. [](http://eloquentjavascript.net/17_http.html#p_7zqOoBgyIg)A moderately complicated website can easily include anywhere from 10 to 200 resources. To be able to fetch those quickly, browsers will make several requests simultaneously, rather than waiting for the responses one at a time. Such documents are always fetched using `GET` requests. [](http://eloquentjavascript.net/17_http.html#p_Aub9jfdnAA)HTML pages may include *forms*, which allow the user to fill out information and send it to the server. This is an example of a form: ~~~ <form method="GET" action="example/message.html"> <p>Name: <input type="text" name="name"></p> <p>Message:<br><textarea name="message"></textarea></p> <p><button type="submit">Send</button></p> </form> ~~~ [](http://eloquentjavascript.net/17_http.html#p_7XkP0qerQY)This code describes a form with two fields: a small one asking for a name and a larger one to write a message in. When you click the Send button, the information in those fields will be encoded into a *query string*. When the`<form>` element’s `method` attribute is `GET` (or is omitted), that query string is tacked onto the `action` URL, and the browser makes a `GET` request to that URL. ~~~ GET /example/message.html?name=Jean&message=Yes%3F HTTP/1.1 ~~~ [](http://eloquentjavascript.net/17_http.html#p_ExLzsDAoPr)The start of a query string is indicated by a question mark. After that follow pairs of names and values, corresponding to the `name` attribute on the form field elements and the content of those elements, respectively. An ampersand character (`&`) is used to separate the pairs. [](http://eloquentjavascript.net/17_http.html#p_4Eo/x5v9Qy)The actual message encoded in the previous URL is “Yes?”, even though the question mark is replaced by a strange code. Some characters in query strings must be escaped. The question mark, represented as `%3F`, is one of those. There seems to be an unwritten rule that every format needs its own way of escaping characters. This one, called *URL encoding*, uses a percent sign followed by two hexadecimal digits that encode the character code. In this case, 3F, which is 63 in decimal notation, is the code of a question mark character. JavaScript provides the `encodeURIComponent` and `decodeURIComponent`functions to encode and decode this format. ~~~ console.log(encodeURIComponent("Hello & goodbye")); // → Hello%20%26%20goodbye console.log(decodeURIComponent("Hello%20%26%20goodbye")); // → Hello & goodbye ~~~ [](http://eloquentjavascript.net/17_http.html#p_l15d9HkVlZ)If we change the `method` attribute of the HTML form in the example we saw earlier to `POST`, the HTTP request made to submit the form will use the `POST`method and put the query string in body of the request, rather than adding it to the URL. ~~~ POST /example/message.html HTTP/1.1 Content-length: 24 Content-type: application/x-www-form-urlencoded name=Jean&message=Yes%3F ~~~ [](http://eloquentjavascript.net/17_http.html#p_K5PHhwPNop)By convention, the `GET` method is used for requests that do not have side effects, such as doing a search. Requests that change something on the server, such as creating a new account or posting a message, should be expressed with other methods, such as `POST`. Client-side software, such as a browser, knows that it shouldn’t blindly make `POST` requests but will often implicitly make `GET`requests—for example, to prefetch a resource it believes the user will soon need. [](http://eloquentjavascript.net/17_http.html#p_mOjZBRUuCo)The [next chapter](http://eloquentjavascript.net/18_forms.html#forms) will return to forms and talk about how we can script them with JavaScript. ## [](http://eloquentjavascript.net/17_http.html#h_Gh3HVKEFJQ)XMLHttpRequest [](http://eloquentjavascript.net/17_http.html#p_sEiPS/dTmx)The interface through which browser JavaScript can make HTTP requests is called `XMLHttpRequest` (note the inconsistent capitalization). It was designed by Microsoft, for its Internet Explorer browser, in the late 1990s. During this time, the XML file format was *very* popular in the world of business software—a world where Microsoft has always been at home. In fact, it was so popular that the acronym XML was tacked onto the front of the name of an interface for HTTP, which is in no way tied to XML. [](http://eloquentjavascript.net/17_http.html#p_ED7N2rpOmD)The name isn’t completely nonsensical, though. The interface allows you to parse response documents as XML if you want. Conflating two distinct concepts (making a request and parsing the response) into a single thing is terrible design, of course, but so it goes. [](http://eloquentjavascript.net/17_http.html#p_huklKLNeHt)When the `XMLHttpRequest` interface was added to Internet Explorer, it allowed people to do things with JavaScript that had been very hard before. For example, websites started showing lists of suggestions when the user was typing something into a text field. The script would send the text to the server over HTTP as the user typed. The server, which had some database of possible inputs, would match the database entries against the partial input and send back possible completions to show the user. This was considered spectacular—people were used to waiting for a full page reload for every interaction with a website. [](http://eloquentjavascript.net/17_http.html#p_yNW5lu4ilN)The other significant browser at that time, Mozilla (later Firefox), did not want to be left behind. To allow people to do similarly neat things in *its* browser, Mozilla copied the interface, including the bogus name. The next generation of browsers followed this example, and today `XMLHttpRequest` is a de facto standard interface. ## [](http://eloquentjavascript.net/17_http.html#h_NfPWoSbXzR)Sending a request [](http://eloquentjavascript.net/17_http.html#p_EQKwKXO14E)To make a simple request, we create a request object with the `XMLHttpRequest`constructor and call its `open` and `send` methods. ~~~ var req = new XMLHttpRequest(); req.open("GET", "example/data.txt", false); req.send(null); console.log(req.responseText); // → This is the content of data.txt ~~~ [](http://eloquentjavascript.net/17_http.html#p_wgyJl8Iwi6)The `open` method configures the request. In this case, we choose to make a`GET` request for the *example/data.txt* file. URLs that don’t start with a protocol name (such as *http:*) are relative, which means that they are interpreted relative to the current document. When they start with a slash (/), they replace the current path, which is the part after the server name. When they do not, the part of the current path up to and including its last slash character is put in front of the relative URL. [](http://eloquentjavascript.net/17_http.html#p_AodYq0EUqt)After opening the request, we can send it with the `send` method. The argument to send is the request body. For `GET` requests, we can pass `null`. If the third argument to `open` was `false`, `send` will return only after the response to our request was received. We can read the request object’s `responseText` property to get the response body. [](http://eloquentjavascript.net/17_http.html#p_OWl9lqs7j6)The other information included in the response can also be extracted from this object. The status code is accessible through the `status` property, and the human-readable status text is accessible through `statusText`. Headers can be read with `getResponseHeader`. ~~~ var req = new XMLHttpRequest(); req.open("GET", "example/data.txt", false); req.send(null); console.log(req.status, req.statusText); // → 200 OK console.log(req.getResponseHeader("content-type")); // → text/plain ~~~ [](http://eloquentjavascript.net/17_http.html#p_lJQuRg3OV0)Header names are case-insensitive. They are usually written with a capital letter at the start of each word, such as “Content-Type”, but “content-type” and “cOnTeNt-TyPe” refer to the same header. [](http://eloquentjavascript.net/17_http.html#p_UkSLHXvU3D)The browser will automatically add some request headers, such as “Host” and those needed for the server to figure out the size of the body. But you can add your own headers with the `setRequestHeader` method. This is needed only for advanced uses and requires the cooperation of the server you are talking to—a server is free to ignore headers it does not know how to handle. ## [](http://eloquentjavascript.net/17_http.html#h_B6OtWq8Z7x)Asynchronous Requests [](http://eloquentjavascript.net/17_http.html#p_xK9lUJ4zpm)In the examples we saw, the request has finished when the call to `send` returns. This is convenient because it means properties such as `responseText` are available immediately. But it also mean that our program is suspended as long as the browser and server are communicating. When the connection is bad, the server is slow, or the file is big, that might take quite a while. Worse, because no event handlers can fire while our program is suspended, the whole document will become unresponsive. [](http://eloquentjavascript.net/17_http.html#p_zjXoJQW9df)If we pass `true` as the third argument to `open`, the request is *asynchronous*. This means that when we call `send`, the only thing that happens right away is that the request is scheduled to be sent. Our program can continue, and the browser will take care of the sending and receiving of data in the background. [](http://eloquentjavascript.net/17_http.html#p_4WsIyXtb4Y)But as long as the request is running, we won’t be able to access the response. We need a mechanism that will notify us when the data is available. [](http://eloquentjavascript.net/17_http.html#p_icR3wVhR/6)For this, we must listen for the `"load"` event on the request object. ~~~ var req = new XMLHttpRequest(); req.open("GET", "example/data.txt", true); req.addEventListener("load", function() { console.log("Done:", req.status); }); req.send(null); ~~~ [](http://eloquentjavascript.net/17_http.html#p_CAzrPM0mYP)Just like the use of `requestAnimationFrame` in [Chapter 15](http://eloquentjavascript.net/15_game.html#game), this forces us to use an asynchronous style of programming, wrapping the things that have to be done after the request in a function and arranging for that to be called at the appropriate time. We will come back to this [later](http://eloquentjavascript.net/17_http.html#promises). ## [](http://eloquentjavascript.net/17_http.html#h_FhSkVawGDi)Fetching XML Data [](http://eloquentjavascript.net/17_http.html#p_96cGuKLvjr)When the resource retrieved by an `XMLHttpRequest` object is an XML document, the object’s `responseXML` property will hold a parsed representation of this document. This representation works much like the DOM discussed in [Chapter 13](http://eloquentjavascript.net/13_dom.html#dom), except that it doesn’t have HTML-specific functionality like the `style` property. The object that `responseXML` holds corresponds to the `document` object. Its `documentElement` property refers to the outer tag of the XML document. In the following document (*example/fruit.xml*), that would would be the `<fruits>` tag: ~~~ <fruits> <fruit name="banana" color="yellow"/> <fruit name="lemon" color="yellow"/> <fruit name="cherry" color="red"/> </fruits> ~~~ [](http://eloquentjavascript.net/17_http.html#p_3NjSjUCAEg)We can retrieve such a file like this: ~~~ var req = new XMLHttpRequest(); req.open("GET", "example/fruit.xml", false); req.send(null); console.log(req.responseXML.querySelectorAll("fruit").length); // → 3 ~~~ [](http://eloquentjavascript.net/17_http.html#p_5fmUcEnbjN)XML documents can be used to exchange structured information with the server. Their form—tags nested inside other tags—lends itself well to storing most types of data, or at least better than flat text files. The DOM interface is rather clumsy for extracting information, though, and XML documents tend to be verbose. It is often a better idea to communicate using JSON data, which is easier to read and write, both for programs and for humans. ~~~ var req = new XMLHttpRequest(); req.open("GET", "example/fruit.json", false); req.send(null); console.log(JSON.parse(req.responseText)); // → {banana: "yellow", lemon: "yellow", cherry: "red"} ~~~ ## [](http://eloquentjavascript.net/17_http.html#h_4h3DL+zoLY)HTTP sandboxing [](http://eloquentjavascript.net/17_http.html#p_RMl+jfbkqI)Making HTTP requests in web page scripts once again raises concerns about security. The person who controls the script might not have the same interests as the person on whose computer it is running. More specifically, if I visit*themafia.org*, I do not want its scripts to be able to make a request to*mybank.com*, using identifying information from my browser, with instructions to transfer all my money to some random mafia account. [](http://eloquentjavascript.net/17_http.html#p_qIXFpjWK5N)It is possible for websites to protect themselves against such attacks, but that requires effort, and many websites fail to do it. For this reason, browsers protect us by disallowing scripts to make HTTP requests to other *domains*(names such as *themafia.org* and *mybank.com*). [](http://eloquentjavascript.net/17_http.html#p_UCqQ0Yw7G5)This can be an annoying problem when building systems that want to access several domains for legitimate reasons. Fortunately, servers can include a header like this in their response to explicitly indicate to browsers that it is okay for the request to come from other domains: ~~~ Access-Control-Allow-Origin: * ~~~ ## [](http://eloquentjavascript.net/17_http.html#h_RR7vot6vKN)Abstracting requests [](http://eloquentjavascript.net/17_http.html#p_3R01AAt3PW)In [Chapter 10](http://eloquentjavascript.net/10_modules.html#amd), in our implementation of the AMD module system, we used a hypothetical function called `backgroundReadFile`. It took a filename and a function and called that function with the contents of the file when it had finished fetching it. Here’s a simple implementation of that function: ~~~ function backgroundReadFile(url, callback) { var req = new XMLHttpRequest(); req.open("GET", url, true); req.addEventListener("load", function() { if (req.status < 400) callback(req.responseText); }); req.send(null); } ~~~ [](http://eloquentjavascript.net/17_http.html#p_nNxmesIyV4)This simple abstraction makes it easier to use `XMLHttpRequest` for simple `GET`requests. If you are writing a program that has to make HTTP requests, it is a good idea to use a helper function so that you don’t end up repeating the ugly`XMLHttpRequest` pattern all through your code. [](http://eloquentjavascript.net/17_http.html#p_8Shcg3/WzI)The function argument’s name, `callback`, is a term that is often used to describe functions like this. A callback function is given to other code to provide that code with a way to “call us back” later. [](http://eloquentjavascript.net/17_http.html#p_xKwFWu3zF0)It is not hard to write an HTTP utility function, tailored to what your application is doing. The previous one does only `GET` requests and doesn’t give us control over the headers or the request body. You could write another variant for `POST` requests or a more generic one that supports various kinds of requests. Many JavaScript libraries also provide wrappers for`XMLHttpRequest`. [](http://eloquentjavascript.net/17_http.html#p_2cF6BoTzh8)The main problem with the previous wrapper is its handling of failure. When the request returns a status code that indicates an error (400 and up), it does nothing. This might be okay, in some circumstances, but imagine we put a “loading” indicator on the page to indicate that we are fetching information. If the request fails because the server crashed or the connection is briefly interrupted, the page will just sit there, misleadingly looking like it is doing something. The user will wait for a while, get impatient, and consider the site uselessly flaky. [](http://eloquentjavascript.net/17_http.html#p_GM6g10t1uS)We should also have an option to be notified when the request fails so that we can take appropriate action. For example, we could remove the “loading” message and inform the user that something went wrong. [](http://eloquentjavascript.net/17_http.html#p_qQG5mkGJ5M)Error handling in asynchronous code is even trickier than error handling in synchronous code. Because we often need to defer part of our work, putting it in a callback function, the scope of a `try` block becomes meaningless. In the following code, the exception will *not* be caught because the call to`backgroundReadFile` returns immediately. Control then leaves the `try` block, and the function it was given won’t be called until later. ~~~ try { backgroundReadFile("example/data.txt", function(text) { if (text != "expected") throw new Error("That was unexpected"); }); } catch (e) { console.log("Hello from the catch block"); } ~~~ [](http://eloquentjavascript.net/17_http.html#p_ZP2klx8rqb)To handle failing requests, we have to allow an additional function to be passed to our wrapper and call that when a request goes wrong. Alternatively, we can use the convention that if the request fails, an additional argument describing the problem is passed to the regular callback function. Here’s an example: ~~~ function getURL(url, callback) { var req = new XMLHttpRequest(); req.open("GET", url, true); req.addEventListener("load", function() { if (req.status < 400) callback(req.responseText); else callback(null, new Error("Request failed: " + req.statusText)); }); req.addEventListener("error", function() { callback(null, new Error("Network error")); }); req.send(null); } ~~~ [](http://eloquentjavascript.net/17_http.html#p_OwEcgNZRXQ)We have added a handler for the `"error"` event, which will be signaled when the request fails entirely. We also call the callback function with an error argument when the request completes with a status code that indicates an error. [](http://eloquentjavascript.net/17_http.html#p_nye9N1QULd)Code using `getURL` must then check whether an error was given and, if it finds one, handle it. ~~~ getURL("data/nonsense.txt", function(content, error) { if (error != null) console.log("Failed to fetch nonsense.txt: " + error); else console.log("nonsense.txt: " + content); }); ~~~ [](http://eloquentjavascript.net/17_http.html#p_j4unfsTO1z)This does not help when it comes to exceptions. When chaining several asynchronous actions together, an exception at any point of the chain will still (unless you wrap each handling function in its own `try/catch` block) land at the top level and abort your chain of actions. ## [](http://eloquentjavascript.net/17_http.html#h_sdRy5CTAP/)Promises [](http://eloquentjavascript.net/17_http.html#p_v7YfQvxwuV)For complicated projects, writing asynchronous code in plain callback style is hard to do correctly. It is easy to forget to check for an error or to allow an unexpected exception to cut the program short in a crude way. Additionally, arranging for correct error handling when the error has to flow through multiple callback functions and `catch` blocks is tedious. [](http://eloquentjavascript.net/17_http.html#p_mO96gXoG0q)There have been a lot of attempts to solve this with extra abstractions. One of the more successful ones is called *promises*. Promises wrap an asynchronous action in an object, which can be passed around and told to do certain things when the action finishes or fails. This interface is set to become part of the next version of the JavaScript language but can already be used as a library. [](http://eloquentjavascript.net/17_http.html#p_HgVayjbMGB)The interface for promises isn’t entirely intuitive, but it is powerful. This chapter will only roughly describe it. You can find a more thorough treatment at [*www.promisejs.org*](https://www.promisejs.org/). [](http://eloquentjavascript.net/17_http.html#p_JxpiCLDcQ3)To create a promise object, we call the `Promise` constructor, giving it a function that initializes the asynchronous action. The constructor calls that function, passing it two arguments, which are themselves functions. The first should be called when the action finishes successfully, and the second should be called when it fails. [](http://eloquentjavascript.net/17_http.html#p_WoWIWkMqow)Once again, here is our wrapper for `GET` requests, this time returning a promise. We’ll simply call it `get` this time. ~~~ function get(url) { return new Promise(function(succeed, fail) { var req = new XMLHttpRequest(); req.open("GET", url, true); req.addEventListener("load", function() { if (req.status < 400) succeed(req.responseText); else fail(new Error("Request failed: " + req.statusText)); }); req.addEventListener("error", function() { fail(new Error("Network error")); }); req.send(null); }); } ~~~ [](http://eloquentjavascript.net/17_http.html#p_RE/bEoSIjp)Note that the interface to the function itself is now a lot simpler. You give it a URL, and it returns a promise. That promise acts as a *handle* to the request’s outcome. It has a `then` method that you can call with two functions: one to handle success and one to handle failure. ~~~ get("example/data.txt").then(function(text) { console.log("data.txt: " + text); }, function(error) { console.log("Failed to fetch data.txt: " + error); }); ~~~ [](http://eloquentjavascript.net/17_http.html#p_4O6znqfcpb)So far, this is just another way to express the same thing we already expressed. It is only when you need to chain actions together that promises make a significant difference. [](http://eloquentjavascript.net/17_http.html#p_uiJcYJELKb)Calling `then` produces a new promise, whose result (the value passed to success handlers) depends on the return value of the first function we passed to`then`. This function may return another promise to indicate that more asynchronous work is being done. In this case, the promise returned by `then`itself will wait for the promise returned by the handler function, succeeding or failing with the same value when it is resolved. When the handler function returns a nonpromise value, the promise returned by `then` immediately succeeds with that value as its result. [](http://eloquentjavascript.net/17_http.html#p_wC6ZA4AmWr)This means you can use `then` to transform the result of a promise. For example, this returns a promise whose result is the content of the given URL, parsed as JSON: ~~~ function getJSON(url) { return get(url).then(JSON.parse); } ~~~ [](http://eloquentjavascript.net/17_http.html#p_Cpzqy849cD)That last call to `then` did not specify a failure handler. This is allowed. The error will be passed to the promise returned by `then`, which is exactly what we want—`getJSON` does not know what to do when something goes wrong, but hopefully its caller does. [](http://eloquentjavascript.net/17_http.html#p_xh6XPjbZAz)As an example that shows the use of promises, we will build a program that fetches a number of JSON files from the server and, while it is doing that, shows the word *loading*. The JSON files contain information about people, with links to files that represent other people in properties such as `father`,`mother`, or `spouse`. [](http://eloquentjavascript.net/17_http.html#p_JBcCWCd0HV)We want to get the name of the mother of the spouse of *example/bert.json*. And if something goes wrong, we want to remove the *loading* text and show an error message instead. Here is how that might be done with promises: ~~~ <script> function showMessage(msg) { var elt = document.createElement("div"); elt.textContent = msg; return document.body.appendChild(elt); } var loading = showMessage("Loading..."); getJSON("example/bert.json").then(function(bert) { return getJSON(bert.spouse); }).then(function(spouse) { return getJSON(spouse.mother); }).then(function(mother) { showMessage("The name is " + mother.name); }).catch(function(error) { showMessage(String(error)); }).then(function() { document.body.removeChild(loading); }); </script> ~~~ [](http://eloquentjavascript.net/17_http.html#p_jx/D3dcS2W)The resulting program is relatively compact and readable. The `catch` method is similar to `then`, except that it only expects a failure handler and will pass through the result unmodified in case of success. Much like with the `catch`clause for the `try` statement, control will continue as normal after the failure is caught. That way, the final `then`, which removes the loading message, is always executed, even if something went wrong. [](http://eloquentjavascript.net/17_http.html#p_8DILWnDxzH)You can think of the promise interface as implementing its own language for asynchronous control flow. The extra method calls and function expressions needed to achieve this make the code look somewhat awkward but not remotely as awkward as it would look if we took care of all the error handling ourselves. ## [](http://eloquentjavascript.net/17_http.html#h_OJmHENDG5y)Appreciating HTTP [](http://eloquentjavascript.net/17_http.html#p_dVrq2xCOfJ)When building a system that requires communication between a JavaScript program running in the browser (client-side) and a program on a server (server-side), there are several different ways to model this communication. [](http://eloquentjavascript.net/17_http.html#p_f4x0TdEU2h)A commonly used model is that of *remote procedure calls*. In this model, communication follows the patterns of normal function calls, except that the function is actually running on another machine. Calling it involves making a request to the server that includes the function’s name and arguments. The response to that request contains the returned value. [](http://eloquentjavascript.net/17_http.html#p_7z+OqsDVUu)When thinking in terms of remote procedure calls, HTTP is just a vehicle for communication, and you will most likely write an abstraction layer that hides it entirely. [](http://eloquentjavascript.net/17_http.html#p_dOvEvBngFh)Another approach is to build your communication around the concept of resources and HTTP methods. Instead of a remote procedure called `addUser`, you use a `PUT` request to `/users/larry`. Instead of encoding that user’s properties in function arguments, you define a document format or use an existing format that represents a user. The body of the `PUT` request to create a new resource is then simply such a document. A resource is fetched by making a `GET` request to the resource’s URL (for example, `/user/larry`), which returns the document representing the resource. [](http://eloquentjavascript.net/17_http.html#p_6PHqO+Vd0H)This second approach makes it easier to use some of the features that HTTP provides, such as support for caching resources (keeping a copy on the client side). It can also help the coherence of your interface since resources are easier to reason about than a jumble of functions. ## [](http://eloquentjavascript.net/17_http.html#h_oDqIFugKX4)Security and HTTPS [](http://eloquentjavascript.net/17_http.html#p_YUIH84Ok2d)Data traveling over the Internet tends to follow a long, dangerous road. To get to its destination, it must hop through anything from coffee-shop Wi-Fi networks to networks controlled by various companies and states. At any point along its route it may be inspected or even modified. [](http://eloquentjavascript.net/17_http.html#p_FkVKmUiG/R)If it is important that something remain secret, such as the password to your email account, or that it arrive at its destination unmodified, such as the account number you transfer money to from your bank’s website, plain HTTP is not good enough. [](http://eloquentjavascript.net/17_http.html#p_s1KEZa2j+E)The secure HTTP protocol, whose URLs start with *https://*, wraps HTTP traffic in a way that makes it harder to read and tamper with. First, the client verifies that the server is who it claims to be by requiring that server to prove that it has a cryptographic certificate issued by a certificate authority that the browser recognizes. Next, all data going over the connection is encrypted in a way that should prevent eavesdropping and tampering. [](http://eloquentjavascript.net/17_http.html#p_whG/pc2PK4)Thus, when it works right, HTTPS prevents both the someone impersonating the website you were trying to talk to and the someone snooping on your communication. It is not perfect, and there have been various incidents where HTTPS failed because of forged or stolen certificates and broken software. Still, plain HTTP is trivial to mess with, whereas breaking HTTPS requires the kind of effort that only states or sophisticated criminal organizations can hope to make. ## [](http://eloquentjavascript.net/17_http.html#h_ErccPg/l98)Summary [](http://eloquentjavascript.net/17_http.html#p_9mkfl3ctRV)In this chapter, we saw that HTTP is a protocol for accessing resources over the Internet. A *client* sends a request, which contains a method (usually `GET`) and a path that identifies a resource. The *server* then decides what to do with the request and responds with a status code and a response body. Both requests and responses may contain headers that provide additional information. [](http://eloquentjavascript.net/17_http.html#p_QYlJoeSssL)Browsers make `GET` requests to fetch the resources needed to display a web page. A web page may also contain forms, which allow information entered by the user to be sent along in the request made when the form is submitted. You will learn more about that in the [next chapter](http://eloquentjavascript.net/18_forms.html#forms). [](http://eloquentjavascript.net/17_http.html#p_taYkCAtnud)The interface through which browser JavaScript can make HTTP requests is called `XMLHttpRequest`. You can usually ignore the “XML” part of that name (but you still have to type it). There are two ways in which it can be used—synchronous, which blocks everything until the request finishes, and asynchronous, which requires an event handler to notice that the response came in. In almost all cases, asynchronous is preferable. Making a request looks like this: ~~~ var req = new XMLHttpRequest(); req.open("GET", "example/data.txt", true); req.addEventListener("load", function() { console.log(req.status); }); req.send(null); ~~~ [](http://eloquentjavascript.net/17_http.html#p_+TkoeLUgPt)Asynchronous programming is tricky. *Promises* are an interface that makes it slightly easier by helping route error conditions and exceptions to the right handler and by abstracting away some of the more repetitive and error-prone elements in this style of programming. ## [](http://eloquentjavascript.net/17_http.html#h_TcUD2vzyMe)Exercises ### [](http://eloquentjavascript.net/17_http.html#h_uaWwL8WGXf)Content negotiation [](http://eloquentjavascript.net/17_http.html#p_4RSIQjkFSG)One of the things that HTTP can do, but that we have not discussed in this chapter, is called *content negotiation*. The `Accept` header for a request can be used to tell the server what type of document the client would like to get. Many servers ignore this header, but when a server knows of various ways to encode a resource, it can look at this header and send the one that the client prefers. [](http://eloquentjavascript.net/17_http.html#p_o+GIbRSLtv)The URL [*eloquentjavascript.net/author*](http://eloquentjavascript.net/author) is configured to respond with either plaintext, HTML, or JSON, depending on what the client asks for. These formats are identified by the standardized *media types* `text/plain`,`text/html`, and `application/json`. [](http://eloquentjavascript.net/17_http.html#p_uUUGBBkyGq)Send requests to fetch all three formats of this resource. Use the`setRequestHeader` method of your `XMLHttpRequest` object to set the header named `Accept` to one of the media types given earlier. Make sure you set the header *after* calling `open` but before calling `send`. [](http://eloquentjavascript.net/17_http.html#p_ydN7YyErvr)Finally, try asking for the media type `application/rainbows+unicorns` and see what happens. ~~~ // Your code here. ~~~ ### [](http://eloquentjavascript.net/17_http.html#h_MlI+KUOVBK)Waiting for multiple promises [](http://eloquentjavascript.net/17_http.html#p_bjI0ituzCO)The `Promise` constructor has an `all` method that, given an array of promises, returns a promise that waits for all of the promises in the array to finish. It then succeeds, yielding an array of result values. If any of the promises in the array fail, the promise returned by `all` fails too (with the failure value from the failing promise). [](http://eloquentjavascript.net/17_http.html#p_aWPI8r+IIn)Try to implement something like this yourself as a regular function called `all`. [](http://eloquentjavascript.net/17_http.html#p_n5BsvIT7F5)Note that after a promise is resolved (has succeeded or failed), it can’t succeed or fail again, and further calls to the functions that resolve it are ignored. This can simplify the way you handle failure of your promise. ~~~ function all(promises) { return new Promise(function(success, fail) { // Your code here. }); } // Test code. all([]).then(function(array) { console.log("This should be []:", array); }); function soon(val) { return new Promise(function(success) { setTimeout(function() { success(val); }, Math.random() * 500); }); } all([soon(1), soon(2), soon(3)]).then(function(array) { console.log("This should be [1, 2, 3]:", array); }); function fail() { return new Promise(function(success, fail) { fail(new Error("boom")); }); } all([soon(1), fail(), soon(3)]).then(function(array) { console.log("We should not get here"); }, function(error) { if (error.message != "boom") console.log("Unexpected failure:", error); }); ~~~ The function passed to the `Promise` constructor will have to call `then` on each of the promises in the given array. When one of them succeeds, two things need to happen. The resulting value needs to be stored in the correct position of a result array, and we must check whether this was the last pending promise and finish our own promise if it was. [](http://eloquentjavascript.net/17_http.html#p_gVL8VQtmdx)The latter can be done with a counter, which is initialized to the length of the input array and from which we subtract 1 every time a promise succeeds. When it reaches 0, we are done. Make sure you take the situation where the input array is empty (and thus no promise will ever resolve) into account. [](http://eloquentjavascript.net/17_http.html#p_xLxIRNCHYI)Handling failure requires some thought but turns out to be extremely simple. Just pass the failure function of the wrapping promise to each of the promises in the array so that a failure in one of them triggers the failure of the whole wrapper.

## Chapter 16 # Drawing on Canvas > [](http://eloquentjavascript.net/16_canvas.html#p_ubdp8gf0Gn)Drawing is deception. > > M.C. Escher, cited by Bruno Ernst in The Magic Mirror of M.C. Escher [](http://eloquentjavascript.net/16_canvas.html#p_rMRN8Q3Ilr)Browsers give us several ways to display graphics. The simplest way is to use styles to position and color regular DOM elements. This can get you quite far, as the game in the [previous chapter](http://eloquentjavascript.net/15_game.html#game) showed. By adding partially transparent background images to the nodes, we can make them look exactly the way we want. It is even possible to rotate or skew nodes by using the `transform` style. [](http://eloquentjavascript.net/16_canvas.html#p_4MfzfTYnsB)But we’d be using the DOM for something that it wasn’t originally designed for. Some tasks, such as drawing a line between arbitrary points, are extremely awkward to do with regular HTML elements. [](http://eloquentjavascript.net/16_canvas.html#p_ETEHgjW4XZ)There are two alternatives. The first is DOM-based but utilizes *Scalable Vector Graphics (SVG)*, rather than HTML elements. Think of SVG as a dialect for describing documents that focuses on shapes rather than text. You can embed an SVG document in an HTML document, or you can include it through an`<img>` tag. [](http://eloquentjavascript.net/16_canvas.html#p_wU9UQEILbC)The second alternative is called a *canvas*. A canvas is a single DOM element that encapsulates a picture. It provides a programming interface for drawing shapes onto the space taken up by the node. The main difference between a canvas and an SVG picture is that in SVG the original description of the shapes is preserved so that they can be moved or resized at any time. A canvas, on the other hand, converts the shapes to pixels (colored dots on a raster) as soon as they are drawn and does not remember what these pixels represent. The only way to move a shape on a canvas is to clear the canvas (or the part of the canvas around the shape) and redraw it with the shape in a new position. ## [](http://eloquentjavascript.net/16_canvas.html#h_UPzm0CiZhQ)SVG [](http://eloquentjavascript.net/16_canvas.html#p_gCxXcLPC1N)This book will not go into SVG in detail, but I will briefly explain how it works. At the [end of the chapter](http://eloquentjavascript.net/16_canvas.html#graphics_tradeoffs), I’ll come back to the trade-offs that you must consider when deciding which drawing mechanism is appropriate for a given application. [](http://eloquentjavascript.net/16_canvas.html#p_aF1bihN0fO)This is an HTML document with a simple SVG picture in it: ~~~ <p>Normal HTML here.</p> <svg xmlns="http://www.w3.org/2000/svg"> <circle r="50" cx="50" cy="50" fill="red"/> <rect x="120" y="5" width="90" height="90" stroke="blue" fill="none"/> </svg> ~~~ [](http://eloquentjavascript.net/16_canvas.html#p_sJ+IuoXUNA)The `xmlns` attribute changes an element (and its children) to a different *XML namespace*. This namespace, identified by a URL, specifies the dialect that we are currently speaking. The `<circle>` and `<rect>` tags, which do not exist in HTML, do have a meaning in SVG—they draw shapes using the style and position specified by their attributes. [](http://eloquentjavascript.net/16_canvas.html#p_DcMLpWRkj0)These tags create DOM elements, just like HTML tags. For example, this changes the `<circle>` element to be colored cyan instead: ~~~ var circle = document.querySelector("circle"); circle.setAttribute("fill", "cyan"); ~~~ ## [](http://eloquentjavascript.net/16_canvas.html#h_QXPPgw0nn4)The canvas element [](http://eloquentjavascript.net/16_canvas.html#p_81TEgZtPp7)Canvas graphics can be drawn onto a `<canvas>` element. You can give such an element `width` and `height` attributes to determine its size in pixels. [](http://eloquentjavascript.net/16_canvas.html#p_oQeTfkBH6v)A new canvas is empty, meaning it is entirely transparent and thus shows up simply as empty space in the document. [](http://eloquentjavascript.net/16_canvas.html#p_r+b1t0y8Si)The `<canvas>` tag is intended to support different styles of drawing. To get access to an actual drawing interface, we first need to create a *context*, which is an object whose methods provide the drawing interface. There are currently two widely supported drawing styles: `"2d"` for two-dimensional graphics and`"webgl"` for three-dimensional graphics through the OpenGL interface. [](http://eloquentjavascript.net/16_canvas.html#p_JaC7mTUVQ0)This book won’t discuss WebGL. We stick to two dimensions. But if you are interested in three-dimensional graphics, I do encourage you to look into WebGL. It provides a very direct interface to modern graphics hardware and thus allows you to render even complicated scenes efficiently, using JavaScript. [](http://eloquentjavascript.net/16_canvas.html#p_GM4b1ASHzf)A context is created through the `getContext` method on the `<canvas>`element. ~~~ <p>Before canvas.</p> <canvas width="120" height="60"></canvas> <p>After canvas.</p> <script> var canvas = document.querySelector("canvas"); var context = canvas.getContext("2d"); context.fillStyle = "red"; context.fillRect(10, 10, 100, 50); </script> ~~~ [](http://eloquentjavascript.net/16_canvas.html#p_fv4PQSHT32)After creating the context object, the example draws a red rectangle 100 pixels wide and 50 pixels high, with its top-left corner at coordinates (10,10). [](http://eloquentjavascript.net/16_canvas.html#p_e1SRsTstnD)Just like in HTML (and SVG), the coordinate system that the canvas uses puts (0,0) at the top-left corner, and the positive y-axis goes down from there. So (10,10) is 10 pixels below and to the right of the top-left corner. ## [](http://eloquentjavascript.net/16_canvas.html#h_pcnyueRl1y)Filling and stroking [](http://eloquentjavascript.net/16_canvas.html#p_UvxB22/zji)In the canvas interface, a shape can be *filled*, meaning its area is given a certain color or pattern, or it can be *stroked*, which means a line is drawn along its edge. The same terminology is used by SVG. [](http://eloquentjavascript.net/16_canvas.html#p_jXutP8Kp6D)The `fillRect` method fills a rectangle. It takes first the x- and y-coordinates of the rectangle’s top-left corner, then its width, and then its height. A similar method, `strokeRect`, draws the outline of a rectangle. [](http://eloquentjavascript.net/16_canvas.html#p_a+mohoFrQD)Neither method takes any further parameters. The color of the fill, thickness of the stroke, and so on are not determined by an argument to the method (as you might justly expect) but rather by properties of the context object. [](http://eloquentjavascript.net/16_canvas.html#p_qwtIen1h5e)Setting `fillStyle` changes the way shapes are filled. It can be set to a string that specifies a color, and any color understood by CSS can also be used here. [](http://eloquentjavascript.net/16_canvas.html#p_z2EgIJKmOE)The `strokeStyle` property works similarly but determines the color used for a stroked line. The width of that line is determined by the `lineWidth` property, which may contain any positive number. ~~~ <canvas></canvas> <script> var cx = document.querySelector("canvas").getContext("2d"); cx.strokeStyle = "blue"; cx.strokeRect(5, 5, 50, 50); cx.lineWidth = 5; cx.strokeRect(135, 5, 50, 50); </script> ~~~ [](http://eloquentjavascript.net/16_canvas.html#p_AIygWrCmAA)When no `width` or `height` attribute is specified, as in the previous example, a canvas element gets a default width of 300 pixels and height of 150 pixels. ## [](http://eloquentjavascript.net/16_canvas.html#h_E+fhFyL32D)Paths [](http://eloquentjavascript.net/16_canvas.html#p_EjhhpUs4B/)A path is a sequence of lines. The 2D canvas interface takes a peculiar approach to describing such a path. It is done entirely through side effects. Paths are not values that can be stored and passed around. Instead, if you want to do something with a path, you make a sequence of method calls to describe its shape. ~~~ <canvas></canvas> <script> var cx = document.querySelector("canvas").getContext("2d"); cx.beginPath(); for (var y = 10; y < 100; y += 10) { cx.moveTo(10, y); cx.lineTo(90, y); } cx.stroke(); </script> ~~~ [](http://eloquentjavascript.net/16_canvas.html#p_CWuGebDoSl)This example creates a path with a number of horizontal line segments and then strokes it using the `stroke` method. Each segment created with `lineTo`starts at the path’s *current* position. That position is usually the end of the last segment, unless `moveTo` was called. In that case, the next segment would start at the position passed to `moveTo`. [](http://eloquentjavascript.net/16_canvas.html#p_/C3/H/w2FX)When filling a path (using the `fill` method), each shape is filled separately. A path can contain multiple shapes—each `moveTo` motion starts a new one. But the path needs to be *closed* (meaning its start and end are in the same position) before it can be filled. If the path is not already closed, a line is added from its end to its start, and the shape enclosed by the completed path is filled. ~~~ <canvas></canvas> <script> var cx = document.querySelector("canvas").getContext("2d"); cx.beginPath(); cx.moveTo(50, 10); cx.lineTo(10, 70); cx.lineTo(90, 70); cx.fill(); </script> ~~~ [](http://eloquentjavascript.net/16_canvas.html#p_9unYw61//h)This example draws a filled triangle. Note that only two of the triangle’s sides are explicitly drawn. The third, from the bottom-right corner back to the top, is implied and won’t be there when you stroke the path. [](http://eloquentjavascript.net/16_canvas.html#p_6TyHPpw704)You could also use the `closePath` method to explicitly close a path by adding an actual line segment back to the path’s start. This segment *is* drawn when stroking the path. ## [](http://eloquentjavascript.net/16_canvas.html#h_B8g7k6vws+)Curves [](http://eloquentjavascript.net/16_canvas.html#p_xee0Jzlsic)A path may also contain curved lines. These are, unfortunately, a bit more involved to draw than straight lines. [](http://eloquentjavascript.net/16_canvas.html#p_q2PxOrpPav)The `quadraticCurveTo` method draws a curve to a given point. To determine the curvature of the line, the method is given a control point as well as a destination point. Imagine this control point as *attracting* the line, giving the line its curve. The line won’t go through the control point. Rather, the direction of the line at its start and end points will be such that it aligns with the line from there to the control point. The following example illustrates this: ~~~ <canvas></canvas> <script> var cx = document.querySelector("canvas").getContext("2d"); cx.beginPath(); cx.moveTo(10, 90); // control=(60,10) goal=(90,90) cx.quadraticCurveTo(60, 10, 90, 90); cx.lineTo(60, 10); cx.closePath(); cx.stroke(); </script> ~~~ [](http://eloquentjavascript.net/16_canvas.html#p_OIjzl6Dc9Y)We draw a quadratic curve from the left to the right, with (60,10) as control point, and then draw two line segments going through that control point and back to the start of the line. The result somewhat resembles a *Star Trek*insignia. You can see the effect of the control point: the lines leaving the lower corners start off in the direction of the control point and then curve toward their target. [](http://eloquentjavascript.net/16_canvas.html#p_Nd10AmdUxD)The `bezierCurveTo` method draws a similar kind of curve. Instead of a single control point, this one has two—one for each of the line's endpoints. Here is a similar sketch to illustrate the behavior of such a curve: ~~~ <canvas></canvas> <script> var cx = document.querySelector("canvas").getContext("2d"); cx.beginPath(); cx.moveTo(10, 90); // control1=(10,10) control2=(90,10) goal=(50,90) cx.bezierCurveTo(10, 10, 90, 10, 50, 90); cx.lineTo(90, 10); cx.lineTo(10, 10); cx.closePath(); cx.stroke(); </script> ~~~ [](http://eloquentjavascript.net/16_canvas.html#p_UVt0ID9CaV)The two control points specify the direction at both ends of the curve. The further they are away from their corresponding point, the more the curve will “bulge” in that direction. [](http://eloquentjavascript.net/16_canvas.html#p_tMRY3uv3sU)Such curves can be hard to work with—it’s not always clear how to find the control points that provide the shape you are looking for. Sometimes you can compute them, and sometimes you’ll just have to find a suitable value by trial and error. [](http://eloquentjavascript.net/16_canvas.html#p_A4ndRHX5Et)*Arcs*—fragments of a circle—are easier to reason about. The `arcTo` method takes no less than five arguments. The first four arguments act somewhat like the arguments to `quadraticCurveTo`. The first pair provides a sort of control point, and the second pair gives the line’s destination. The fifth argument provides the radius of the arc. The method will conceptually project a corner—a line going to the control point and then to the destination point—and round the corner’s point so that it forms part of a circle with the given radius. The `arcTo`method then draws the rounded part, as well as a line from the starting position to the start of the rounded part. ~~~ <canvas></canvas> <script> var cx = document.querySelector("canvas").getContext("2d"); cx.beginPath(); cx.moveTo(10, 10); // control=(90,10) goal=(90,90) radius=20 cx.arcTo(90, 10, 90, 90, 20); cx.moveTo(10, 10); // control=(90,10) goal=(90,90) radius=80 cx.arcTo(90, 10, 90, 90, 80); cx.stroke(); </script> ~~~ [](http://eloquentjavascript.net/16_canvas.html#p_+oDzi4kZW+)The `arcTo` method won’t draw the line from the end of the rounded part to the goal position, though the word *to* in its name would suggest it does. You can follow up with a call to `lineTo` with the same goal coordinates to add that part of the line. [](http://eloquentjavascript.net/16_canvas.html#p_amjPveUnsp)To draw a circle, you could use four calls to `arcTo` (each turning 90 degrees). But the `arc` method provides a simpler way. It takes a pair of coordinates for the arc’s center, a radius, and then a start and end angle. [](http://eloquentjavascript.net/16_canvas.html#p_R0hMJ8VOzu)Those last two parameters make it possible to draw only part of circle. The angles are measured in radians, not degrees. This means a full circle has an angle of 2π, or `2 * Math.PI`, which is about 6.28\. The angle starts counting at the point to the right of the circle’s center and goes clockwise from there. You can use a start of 0 and an end bigger than 2π (say, 7) to draw a full circle. ~~~ <canvas></canvas> <script> var cx = document.querySelector("canvas").getContext("2d"); cx.beginPath(); // center=(50,50) radius=40 angle=0 to 7 cx.arc(50, 50, 40, 0, 7); // center=(150,50) radius=40 angle=0 to ½π cx.arc(150, 50, 40, 0, 0.5 * Math.PI); cx.stroke(); </script> ~~~ [](http://eloquentjavascript.net/16_canvas.html#p_NhI3gukqbY)The resulting picture contains a line from the right of the full circle (first call to`arc`) to the right of the quarter-circle (second call). Like other path-drawing methods, a line drawn with `arc` is connected to the previous path segment by default. You’d have to call `moveTo` or start a new path if you want to avoid this. ## [](http://eloquentjavascript.net/16_canvas.html#h_9yOdkmATfT)Drawing a pie chart [](http://eloquentjavascript.net/16_canvas.html#p_Rd++tqzkHn)Imagine you’ve just taken a job at EconomiCorp, Inc., and your first assignment is to draw a pie chart of their customer satisfaction survey results. [](http://eloquentjavascript.net/16_canvas.html#p_NOg95U8NVA)The `results` variable contains an array of objects that represent the survey responses. ~~~ var results = [ {name: "Satisfied", count: 1043, color: "lightblue"}, {name: "Neutral", count: 563, color: "lightgreen"}, {name: "Unsatisfied", count: 510, color: "pink"}, {name: "No comment", count: 175, color: "silver"} ]; ~~~ [](http://eloquentjavascript.net/16_canvas.html#p_P3SJQbINGf)To draw a pie chart, we draw a number of pie slices, each made up of an arc and a pair of lines to the center of that arc. We can compute the angle taken up by each arc by dividing a full circle (2π) by the total number of responses and then multiplying that number (the angle per response) by the number of people who picked a given choice. ~~~ <canvas width="200" height="200"></canvas> <script> var cx = document.querySelector("canvas").getContext("2d"); var total = results.reduce(function(sum, choice) { return sum + choice.count; }, 0); // Start at the top var currentAngle = -0.5 * Math.PI; results.forEach(function(result) { var sliceAngle = (result.count / total) * 2 * Math.PI; cx.beginPath(); // center=100,100, radius=100 // from current angle, clockwise by slice's angle cx.arc(100, 100, 100, currentAngle, currentAngle + sliceAngle); currentAngle += sliceAngle; cx.lineTo(100, 100); cx.fillStyle = result.color; cx.fill(); }); </script> ~~~ [](http://eloquentjavascript.net/16_canvas.html#p_e5+MpAPZp1)But a chart that doesn’t tell us what it means isn’t very helpful. We need a way to draw text to the canvas. ## [](http://eloquentjavascript.net/16_canvas.html#h_wzKMObDin3)Text [](http://eloquentjavascript.net/16_canvas.html#p_/MkzAT+yJW)A 2D canvas drawing context provides the methods `fillText` and`strokeText`. The latter can be useful for outlining letters, but usually`fillText` is what you need. It will fill the given text with the current`fillColor`. ~~~ <canvas></canvas> <script> var cx = document.querySelector("canvas").getContext("2d"); cx.font = "28px Georgia"; cx.fillStyle = "fuchsia"; cx.fillText("I can draw text, too!", 10, 50); </script> ~~~ [](http://eloquentjavascript.net/16_canvas.html#p_n79D894g2i)You can specify the size, style, and font of the text with the `font` property. This example just gives a font size and family name. You can add `italic` or `bold` to the start of the string to select a style. [](http://eloquentjavascript.net/16_canvas.html#p_3LlDU7wt/M)The last two arguments to `fillText` (and `strokeText`) provide the position at which the font is drawn. By default, they indicate the position of the start of the text’s alphabetic baseline, which is the line that letters “stand” on, not counting hanging parts in letters like *j* or *p*. You can change the horizontal position by setting the `textAlign` property to `"end"` or `"center"` and the vertical position by setting `textBaseline` to `"top"`, `"middle"`, or `"bottom"`. [](http://eloquentjavascript.net/16_canvas.html#p_/4NicYFaZd)We will come back to our pie chart, and the problem of labeling the slices, in the [exercises](http://eloquentjavascript.net/16_canvas.html#exercise_pie_chart) at the end of the chapter. ## [](http://eloquentjavascript.net/16_canvas.html#h_CehxyY/vO5)Images [](http://eloquentjavascript.net/16_canvas.html#p_y7aK+8CMcM)In computer graphics, a distinction is often made between *vector* graphics and*bitmap* graphics. The first is what we have been doing so far in this chapter—specifying a picture by giving a logical description of shapes. Bitmap graphics, on the other hand, don’t specify actual shapes but rather work with pixel data (rasters of colored dots). [](http://eloquentjavascript.net/16_canvas.html#p_qlIKv8QV+7)The `drawImage` method allows us to draw pixel data onto a canvas. This pixel data can originate from an `<img>` element or from another canvas, and neither has to be visible in the actual document. The following example creates a detached `<img>` element and loads an image file into it. But it cannot immediately start drawing from this picture because the browser may not have fetched it yet. To deal with this, we register a `"load"` event handler and do the drawing after the image has loaded. ~~~ <canvas></canvas> <script> var cx = document.querySelector("canvas").getContext("2d"); var img = document.createElement("img"); img.src = "img/hat.png"; img.addEventListener("load", function() { for (var x = 10; x < 200; x += 30) cx.drawImage(img, x, 10); }); </script> ~~~ [](http://eloquentjavascript.net/16_canvas.html#p_RxRhSVTbGY)By default, `drawImage` will draw the image at its original size. You can also give it two additional arguments to dictate a different width and height. [](http://eloquentjavascript.net/16_canvas.html#p_/bLIqvw4R4)When `drawImage` is given *nine* arguments, it can be used to draw only a fragment of an image. The second through fifth arguments indicate the rectangle (x, y, width, and height) in the source image that should be copied, and the sixth to ninth arguments give the rectangle (on the canvas) into which it should be copied. [](http://eloquentjavascript.net/16_canvas.html#p_xKlo8wmmgJ)This can be used to pack multiple *sprites* (image elements) into a single image file and then draw only the part you need. For example, we have this picture containing a game character in multiple poses:  [](http://eloquentjavascript.net/16_canvas.html#p_u8W4Vqh39a)By alternating which pose we draw, we can show an animation that looks like a walking character. [](http://eloquentjavascript.net/16_canvas.html#p_vwSjUDbC98)To animate the picture on a canvas, the `clearRect` method is useful. It resembles `fillRect`, but instead of coloring the rectangle, it makes it transparent, removing the previously drawn pixels. [](http://eloquentjavascript.net/16_canvas.html#p_VcrdBA3T8z)We know that each *sprite*, each subpicture, is 24 pixels wide and 30 pixels high. The following code loads the image and then sets up an interval (repeated timer) to draw the next *frame*: ~~~ <canvas></canvas> <script> var cx = document.querySelector("canvas").getContext("2d"); var img = document.createElement("img"); img.src = "img/player.png"; var spriteW = 24, spriteH = 30; img.addEventListener("load", function() { var cycle = 0; setInterval(function() { cx.clearRect(0, 0, spriteW, spriteH); cx.drawImage(img, // source rectangle cycle * spriteW, 0, spriteW, spriteH, // destination rectangle 0, 0, spriteW, spriteH); cycle = (cycle + 1) % 8; }, 120); }); </script> ~~~ [](http://eloquentjavascript.net/16_canvas.html#p_FwoYTH1Pg7)The `cycle` variable tracks our position in the animation. Each frame, it is incremented and then clipped back to the 0 to 7 range by using the remainder operator. This variable is then used to compute the x-coordinate that the sprite for the current pose has in the picture. ## [](http://eloquentjavascript.net/16_canvas.html#h_3BwjEnWhbh)Transformation [](http://eloquentjavascript.net/16_canvas.html#p_b00lbwHiev)But what if we want our character to walk to the left instead of to the right? We could add another set of sprites, of course. But we can also instruct the canvas to draw the picture the other way round. [](http://eloquentjavascript.net/16_canvas.html#p_JnX4LFnuKL)Calling the `scale` method will cause anything drawn after it to be scaled. This method takes two parameters, one to set a horizontal scale and one to set a vertical scale. ~~~ <canvas></canvas> <script> var cx = document.querySelector("canvas").getContext("2d"); cx.scale(3, .5); cx.beginPath(); cx.arc(50, 50, 40, 0, 7); cx.lineWidth = 3; cx.stroke(); </script> ~~~ [](http://eloquentjavascript.net/16_canvas.html#p_u+8/LLl96H)Scaling will cause everything about the drawn image, including the line width, to be stretched out or squeezed together as specified. Scaling by a negative amount will flip the picture around. The flipping happens around point (0,0), which means it will also flip the direction of the coordinate system. When a horizontal scaling of -1 is applied, a shape drawn at x position 100 will end up at what used to be position -100. [](http://eloquentjavascript.net/16_canvas.html#p_bZnJ+aZtFS)So to turn a picture around, we can’t simply add `cx.scale(-1, 1)` before the call to `drawImage` since that would move our picture outside of the canvas, where it won’t be visible. You could adjust the coordinates given to `drawImage`to compensate for this by drawing the image at x position -50 instead of 0\. Another solution, which doesn’t require the code that does the drawing to know about the scale change, is to adjust the axis around which the scaling happens. [](http://eloquentjavascript.net/16_canvas.html#p_LbiF7s9yt4)There are several other methods besides `scale` that influence the coordinate system for a canvas. You can rotate subsequently drawn shapes with the`rotate` method and move them with the `translate` method. The interesting—and confusing—thing is that these transformations *stack*, meaning that each one happens relative to the previous transformations. [](http://eloquentjavascript.net/16_canvas.html#p_mkNUtuNxcG)So if we translate by 10 horizontal pixels twice, everything will be drawn 20 pixels to the right. If we first move the center of the coordinate system to (50,50) and then rotate by 20 degrees (0.1π in radians), that rotation will happen *around* point (50,50).  [](http://eloquentjavascript.net/16_canvas.html#p_TGBwm05/cn)But if we *first* rotate by 20 degrees and *then* translate by (50,50), the translation will happen in the rotated coordinate system and thus produce a different orientation. The order in which transformations are applied matters. [](http://eloquentjavascript.net/16_canvas.html#p_9a1O8aEtUA)To flip a picture around the vertical line at a given x position, we can do the following: ~~~ function flipHorizontally(context, around) { context.translate(around, 0); context.scale(-1, 1); context.translate(-around, 0); } ~~~ [](http://eloquentjavascript.net/16_canvas.html#p_qob4Y7lZ5I)We move the y-axis to where we want our mirror to be, apply the mirroring, and finally move the y-axis back to its proper place in the mirrored universe. The following picture explains why this works:  [](http://eloquentjavascript.net/16_canvas.html#p_AGahdKv9Zv)This shows the coordinate systems before and after mirroring across the central line. If we draw a triangle at a positive x position, it would, by default, be in the place where triangle 1 is. A call to `flipHorizontally` first does a translation to the right, which gets us to triangle 2\. It then scales, flipping the triangle back to position 3\. This is not where it should be, if it were mirrored in the given line. The second `translate` call fixes this—it “cancels” the initial translation and makes triangle 4 appear exactly where it should. [](http://eloquentjavascript.net/16_canvas.html#p_akkrexzSeV)We can now draw a mirrored character at position (100,0) by flipping the world around the character’s vertical center. ~~~ <canvas></canvas> <script> var cx = document.querySelector("canvas").getContext("2d"); var img = document.createElement("img"); img.src = "img/player.png"; var spriteW = 24, spriteH = 30; img.addEventListener("load", function() { flipHorizontally(cx, 100 + spriteW / 2); cx.drawImage(img, 0, 0, spriteW, spriteH, 100, 0, spriteW, spriteH); }); </script> ~~~ ## [](http://eloquentjavascript.net/16_canvas.html#h_Z+iS7LhRr9)Storing and clearing transformations [](http://eloquentjavascript.net/16_canvas.html#p_Az5iI5GbrT)Transformations stick around. Everything else we draw after drawing that mirrored character would also be mirrored. That might be a problem. [](http://eloquentjavascript.net/16_canvas.html#p_4R4LetOdqr)It is possible to save the current transformation, do some drawing and transforming, and then restore the old transformation. This is usually the proper thing to do for a function that needs to temporarily transform the coordinate system. First, we save whatever transformation the code that called the function was using. Then, the function does its thing (on top of the existing transformation), possibly adding more transformations. And finally, we revert to the transformation that we started with. [](http://eloquentjavascript.net/16_canvas.html#p_65OomGfaeV)The `save` and `restore` methods on the 2D canvas context perform this kind of transformation management. They conceptually keep a stack of transformation states. When you call `save`, the current state is pushed onto the stack, and when you call `restore`, the state on top of the stack is taken off and used as the context’s current transformation. [](http://eloquentjavascript.net/16_canvas.html#p_+U4XOiej4f)The `branch` function in the following example illustrates what you can do with a function that changes the transformation and then calls another function (in this case itself), which continues drawing with the given transformation. [](http://eloquentjavascript.net/16_canvas.html#p_2U+6WbJ0Or)This function draws a treelike shape by drawing a line, moving the center of the coordinate system to the end of the line, and calling itself twice—first rotated to the left and then rotated to the right. Every call reduces the length of the branch drawn, and the recursion stops when the length drops below 8. ~~~ <canvas width="600" height="300"></canvas> <script> var cx = document.querySelector("canvas").getContext("2d"); function branch(length, angle, scale) { cx.fillRect(0, 0, 1, length); if (length < 8) return; cx.save(); cx.translate(0, length); cx.rotate(-angle); branch(length * scale, angle, scale); cx.rotate(2 * angle); branch(length * scale, angle, scale); cx.restore(); } cx.translate(300, 0); branch(60, 0.5, 0.8); </script> ~~~ [](http://eloquentjavascript.net/16_canvas.html#p_Fr9r9ZmPHu)If the calls to `save` and `restore` were not there, the second recursive call to`branch` would end up with the position and rotation created by the first call. It wouldn’t be connected to the current branch but rather to the innermost, rightmost branch drawn by the first call. The resulting shape might also be interesting, but it is definitely not a tree. ## [](http://eloquentjavascript.net/16_canvas.html#h_TOqgrv5vzl)Back to the game [](http://eloquentjavascript.net/16_canvas.html#p_v/iifj4dhd)We now know enough about canvas drawing to start working on a canvas-based display system for the game from the [previous chapter](http://eloquentjavascript.net/15_game.html#game). The new display will no longer be showing just colored boxes. Instead, we’ll use `drawImage` to draw pictures that represent the game’s elements. [](http://eloquentjavascript.net/16_canvas.html#p_dy1myCUsaF)We will define an object type `CanvasDisplay`, supporting the same interface as`DOMDisplay` from [Chapter 15](http://eloquentjavascript.net/15_game.html#domdisplay), namely, the methods `drawFrame` and `clear`. [](http://eloquentjavascript.net/16_canvas.html#p_H/XEh6pFg7)This object keeps a little more information than `DOMDisplay`. Rather than using the scroll position of its DOM element, it tracks its own viewport, which tells us what part of the level we are currently looking at. It also tracks time and uses that to decide which animation frame to use. And finally, it keeps a`flipPlayer` property so that even when the player is standing still, it keeps facing the direction it last moved in. ~~~ function CanvasDisplay(parent, level) { this.canvas = document.createElement("canvas"); this.canvas.width = Math.min(600, level.width * scale); this.canvas.height = Math.min(450, level.height * scale); parent.appendChild(this.canvas); this.cx = this.canvas.getContext("2d"); this.level = level; this.animationTime = 0; this.flipPlayer = false; this.viewport = { left: 0, top: 0, width: this.canvas.width / scale, height: this.canvas.height / scale }; this.drawFrame(0); } CanvasDisplay.prototype.clear = function() { this.canvas.parentNode.removeChild(this.canvas); }; ~~~ [](http://eloquentjavascript.net/16_canvas.html#p_blt6VbEw4l)The `animationTime` counter is the reason we passed the step size to`drawFrame` in [Chapter 15](http://eloquentjavascript.net/15_game.html#domdisplay), even though `DOMDisplay` does not use it. Our new`drawFrame` function uses the counter to track time so that it can switch between animation frames based on the current time. ~~~ CanvasDisplay.prototype.drawFrame = function(step) { this.animationTime += step; this.updateViewport(); this.clearDisplay(); this.drawBackground(); this.drawActors(); }; ~~~ [](http://eloquentjavascript.net/16_canvas.html#p_hwAtbz1fs3)Other than tracking time, the method updates the viewport for the current player position, fills the whole canvas with a background color, and draws the background and actors onto that. Note that this is different from the approach in [Chapter 15](http://eloquentjavascript.net/15_game.html#domdisplay), where we drew the background once and scrolled the wrapping DOM element to move it. [](http://eloquentjavascript.net/16_canvas.html#p_sX9ENmCEO9)Because shapes on a canvas are just pixels, after we draw them, there is no way to move them (or remove them). The only way to update the canvas display is to clear it and redraw the scene. [](http://eloquentjavascript.net/16_canvas.html#p_Kn4mJDmduk)The `updateViewport` method is similar to `DOMDisplay`'s`scrollPlayerIntoView` method. It checks whether the player is too close to the edge of the screen and moves the viewport when this is the case. ~~~ CanvasDisplay.prototype.updateViewport = function() { var view = this.viewport, margin = view.width / 3; var player = this.level.player; var center = player.pos.plus(player.size.times(0.5)); if (center.x < view.left + margin) view.left = Math.max(center.x - margin, 0); else if (center.x > view.left + view.width - margin) view.left = Math.min(center.x + margin - view.width, this.level.width - view.width); if (center.y < view.top + margin) view.top = Math.max(center.y - margin, 0); else if (center.y > view.top + view.height - margin) view.top = Math.min(center.y + margin - view.height, this.level.height - view.height); }; ~~~ [](http://eloquentjavascript.net/16_canvas.html#p_qSUMwlAFeW)The calls to `Math.max` and `Math.min` ensure that the viewport does not end up showing space outside of the level. `Math.max(x, 0)` ensures that the resulting number is not less than zero. `Math.min`, similarly, ensures a value stays below a given bound. [](http://eloquentjavascript.net/16_canvas.html#p_uxQ4EAtiQh)When clearing the display, we’ll use a slightly different color depending on whether the game is won (brighter) or lost (darker). ~~~ CanvasDisplay.prototype.clearDisplay = function() { if (this.level.status == "won") this.cx.fillStyle = "rgb(68, 191, 255)"; else if (this.level.status == "lost") this.cx.fillStyle = "rgb(44, 136, 214)"; else this.cx.fillStyle = "rgb(52, 166, 251)"; this.cx.fillRect(0, 0, this.canvas.width, this.canvas.height); }; ~~~ [](http://eloquentjavascript.net/16_canvas.html#p_TjOnlogmqf)To draw the background, we run through the tiles that are visible in the current viewport, using the same trick used in `obstacleAt` in the [previous chapter](http://eloquentjavascript.net/15_game.html#viewport). ~~~ var otherSprites = document.createElement("img"); otherSprites.src = "img/sprites.png"; CanvasDisplay.prototype.drawBackground = function() { var view = this.viewport; var xStart = Math.floor(view.left); var xEnd = Math.ceil(view.left + view.width); var yStart = Math.floor(view.top); var yEnd = Math.ceil(view.top + view.height); for (var y = yStart; y < yEnd; y++) { for (var x = xStart; x < xEnd; x++) { var tile = this.level.grid[y][x]; if (tile == null) continue; var screenX = (x - view.left) * scale; var screenY = (y - view.top) * scale; var tileX = tile == "lava" ? scale : 0; this.cx.drawImage(otherSprites, tileX, 0, scale, scale, screenX, screenY, scale, scale); } } }; ~~~ [](http://eloquentjavascript.net/16_canvas.html#p_exCLyNQQL6)Tiles that are not empty (null) are drawn with `drawImage`. The `otherSprites`image contains the pictures used for elements other than the player. It contains, from left to right, the wall tile, the lava tile, and the sprite for a coin.  [](http://eloquentjavascript.net/16_canvas.html#p_1R4HdSFR8E)Background tiles are 20 by 20 pixels, since we will use the same scale that we used in `DOMDisplay`. Thus, the offset for lava tiles is 20 (the value of the `scale`variable), and the offset for walls is 0. [](http://eloquentjavascript.net/16_canvas.html#p_KIwkr/Frrz)We don’t bother waiting for the sprite image to load. Calling `drawImage` with an image that hasn’t been loaded yet will simply do nothing. Thus, we might fail to draw the game properly for the first few frames, while the image is still loading, but that is not a serious problem. Since we keep updating the screen, the correct scene will appear as soon as the loading finishes. [](http://eloquentjavascript.net/16_canvas.html#p_/1DXjZpAsq)The walking character shown earlier will be used to represent the player. The code that draws it needs to pick the right sprite and direction based on the player’s current motion. The first eight sprites contain a walking animation. When the player is moving along a floor, we cycle through them based on the display’s `animationTime` property. This is measured in seconds, and we want to switch frames 12 times per second, so the time is multiplied by 12 first. When the player is standing still, we draw the ninth sprite. During jumps, which are recognized by the fact that the vertical speed is not zero, we use the tenth, rightmost sprite. [](http://eloquentjavascript.net/16_canvas.html#p_wuDMxn8vbx)Because the sprites are slightly wider than the player object—24 instead of 16 pixels, to allow some space for feet and arms—the method has to adjust the x-coordinate and width by a given amount (`playerXOverlap`). ~~~ var playerSprites = document.createElement("img"); playerSprites.src = "img/player.png"; var playerXOverlap = 4; CanvasDisplay.prototype.drawPlayer = function(x, y, width, height) { var sprite = 8, player = this.level.player; width += playerXOverlap * 2; x -= playerXOverlap; if (player.speed.x != 0) this.flipPlayer = player.speed.x < 0; if (player.speed.y != 0) sprite = 9; else if (player.speed.x != 0) sprite = Math.floor(this.animationTime * 12) % 8; this.cx.save(); if (this.flipPlayer) flipHorizontally(this.cx, x + width / 2); this.cx.drawImage(playerSprites, sprite * width, 0, width, height, x, y, width, height); this.cx.restore(); }; ~~~ [](http://eloquentjavascript.net/16_canvas.html#p_gswQ2kj9pD)The `drawPlayer` method is called by `drawActors`, which is responsible for drawing all the actors in the game. ~~~ CanvasDisplay.prototype.drawActors = function() { this.level.actors.forEach(function(actor) { var width = actor.size.x * scale; var height = actor.size.y * scale; var x = (actor.pos.x - this.viewport.left) * scale; var y = (actor.pos.y - this.viewport.top) * scale; if (actor.type == "player") { this.drawPlayer(x, y, width, height); } else { var tileX = (actor.type == "coin" ? 2 : 1) * scale; this.cx.drawImage(otherSprites, tileX, 0, width, height, x, y, width, height); } }, this); }; ~~~ [](http://eloquentjavascript.net/16_canvas.html#p_PD+LPXwCgk)When drawing something that is not the player, we look at its type to find the offset of the correct sprite. The lava tile is found at offset 20, and the coin sprite is found at 40 (two times `scale`). [](http://eloquentjavascript.net/16_canvas.html#p_e6Z9O4bib+)We have to subtract the viewport’s position when computing the actor’s position since (0,0) on our canvas corresponds to the top left of the viewport, not the top left of the level. We could also have used `translate` for this. Either way works. [](http://eloquentjavascript.net/16_canvas.html#p_4VReSkKraV)The tiny document shown next plugs the new display into `runGame`: ~~~ <body> <script> runGame(GAME_LEVELS, CanvasDisplay); </script> </body> ~~~ ## [](http://eloquentjavascript.net/16_canvas.html#h_T6CfghQSfx)Choosing a graphics interface [](http://eloquentjavascript.net/16_canvas.html#p_hiYOCc2tar)Whenever you need to generate graphics in the browser, you can choose between plain HTML, SVG, and canvas. There is no single *best* approach that works in all situations. Each option has strengths and weaknesses. [](http://eloquentjavascript.net/16_canvas.html#p_O7BQaYbnwn)Plain HTML has the advantage of being simple. It also integrates well with text. Both SVG and canvas allow you to draw text, but they won’t help you position that text or wrap it when it takes up more than one line. In an HTML-based picture, it is easy to include blocks of text. [](http://eloquentjavascript.net/16_canvas.html#p_6uH6l730WJ)SVG can be used to produce crisp graphics that look good at any zoom level. It is more difficult to use than plain HTML but also much more powerful. [](http://eloquentjavascript.net/16_canvas.html#p_J5B7lFXtzE)Both SVG and HTML build up a data structure (the DOM) that represents the picture. This makes it possible to modify elements after they are drawn. If you need to repeatedly change a small part of a big picture in response to what the user is doing or as part of an animation, doing it in a canvas can be needlessly expensive. The DOM also allows us to register mouse event handlers on every element in the picture (even on shapes drawn with SVG). You can’t do that with canvas. [](http://eloquentjavascript.net/16_canvas.html#p_7N4OSlG5eS)But canvas’s pixel-oriented approach can be an advantage when drawing a huge amount of tiny elements. The fact that it does not build up a data structure but only repeatedly draws onto the same pixel surface gives canvas a lower cost per shape. [](http://eloquentjavascript.net/16_canvas.html#p_6P4A05RVAk)There are also effects, such as rendering a scene one pixel at a time (for example, using a ray tracer) or postprocessing an image with JavaScript (blurring or distorting it), that can only be realistically handled by a pixel-based technique. [](http://eloquentjavascript.net/16_canvas.html#p_lecWO5A8ae)In some cases, you may want to combine several of these techniques. For example, you might draw a graph with SVG or canvas but show textual information by positioning an HTML element on top of the picture. [](http://eloquentjavascript.net/16_canvas.html#p_nXWEbGkTBm)For nondemanding applications, it really doesn’t matter much which interface you choose. The [second display](http://eloquentjavascript.net/16_canvas.html#canvasdisplay) we built for our game in this chapter could have been implemented using any of these three graphics technologies since it does not need to draw text, handle mouse interaction, or work with an extraordinarily large amount of elements. ## [](http://eloquentjavascript.net/16_canvas.html#h_ErccPg/l98)Summary [](http://eloquentjavascript.net/16_canvas.html#p_tJH36yWkVt)In this chapter, we discussed techniques for drawing graphics in the browser, focusing on the `<canvas>` element. [](http://eloquentjavascript.net/16_canvas.html#p_zTR1kU1R2F)A canvas node represents an area in a document that our program may draw on. This drawing is done through a drawing context object, created with the`getContext` method. [](http://eloquentjavascript.net/16_canvas.html#p_77n8vdtsoS)The 2D drawing interface allows us to fill and stroke various shapes. The context’s `fillStyle` property determines how shapes are filled. The`strokeStyle` and `lineWidth` properties control the way lines are drawn. [](http://eloquentjavascript.net/16_canvas.html#p_p9lwBrZeDB)Rectangles and pieces of text can be drawn with a single method call. The`fillRect` and `strokeRect` methods draw rectangles, and the `fillText` and`strokeText` methods draw text. To create custom shapes, we must first build up a path. [](http://eloquentjavascript.net/16_canvas.html#p_+WESqzV0sH)Calling `beginPath` starts a new path. A number of other methods add lines and curves to the current path. For example, `lineTo` can add a straight line. When a path is finished, it can be filled with the `fill` method or stroked with the `stroke` method. [](http://eloquentjavascript.net/16_canvas.html#p_DjIq4BWBzl)Moving pixels from an image or another canvas onto our canvas is done with the `drawImage` method. By default, this method draws the whole source image, but by giving it more parameters, you can copy a specific area of the image. We used this for our game by copying individual poses of the game character out of an image that contained many such poses. [](http://eloquentjavascript.net/16_canvas.html#p_EWHXtIfgMN)Transformations allow you to draw a shape in multiple orientations. A 2D drawing context has a current transformation that can be changed with the`translate`, `scale`, and `rotate` methods. These will affect all subsequent drawing operations. A transformation state can be saved with the `save` method and restored with the `restore` method. [](http://eloquentjavascript.net/16_canvas.html#p_CiZHyGysLU)When drawing an animation on a canvas, the `clearRect` method can be used to clear part of the canvas before redrawing it. ## [](http://eloquentjavascript.net/16_canvas.html#h_TcUD2vzyMe)Exercises ### [](http://eloquentjavascript.net/16_canvas.html#h_sZheOHQF5N)Shapes [](http://eloquentjavascript.net/16_canvas.html#p_ML2Sk/PrfT)Write a program that draws the following shapes on a canvas: 1. [](http://eloquentjavascript.net/16_canvas.html#p_jg9BlkxnI6)A trapezoid (a rectangle that is wider on one side) 2. [](http://eloquentjavascript.net/16_canvas.html#p_TsmIPWkWaZ)A red diamond (a rectangle rotated 45 degrees or ¼π radians) 3. [](http://eloquentjavascript.net/16_canvas.html#p_rB8lM2f3nJ)A zigzagging line 4. [](http://eloquentjavascript.net/16_canvas.html#p_giU72/tC1m)A spiral made up of 100 straight line segments 5. [](http://eloquentjavascript.net/16_canvas.html#p_BaFPNufkG9)A yellow star  [](http://eloquentjavascript.net/16_canvas.html#p_rGwDFrV/8d)When drawing the last two, you may want to refer to the explanation of`Math.cos` and `Math.sin` in [Chapter 13](http://eloquentjavascript.net/13_dom.html#sin_cos), which describes how to get coordinates on a circle using these functions. [](http://eloquentjavascript.net/16_canvas.html#p_8n4Hu9tdGs)I recommend creating a function for each shape. Pass the position, and optionally other properties, such as the size or the number of points, as parameters. The alternative, which is to hard-code numbers all over your code, tends to make the code needlessly hard to read and modify. ~~~ <canvas width="600" height="200"></canvas> <script> var cx = document.querySelector("canvas").getContext("2d"); // Your code here. </script> ~~~ ### [](http://eloquentjavascript.net/16_canvas.html#h_bJrtZj5liF)The pie chart [](http://eloquentjavascript.net/16_canvas.html#p_5/ovyOQpmP)[Earlier](http://eloquentjavascript.net/16_canvas.html#pie_chart) in the chapter, we saw an example program that drew a pie chart. Modify this program so that the name of each category is shown next to the slice that represents it. Try to find a pleasing-looking way to automatically position this text, which would work for other data sets as well. You may assume that categories are no smaller than 5 percent (that is, there won’t be a bunch of tiny ones next to each other). [](http://eloquentjavascript.net/16_canvas.html#p_nTwqKbJWIc)You might again need `Math.sin` and `Math.cos`, as described in the previous exercise. ~~~ <canvas width="600" height="300"></canvas> <script> var cx = document.querySelector("canvas").getContext("2d"); var total = results.reduce(function(sum, choice) { return sum + choice.count; }, 0); var currentAngle = -0.5 * Math.PI; var centerX = 300, centerY = 150; // Add code to draw the slice labels in this loop. results.forEach(function(result) { var sliceAngle = (result.count / total) * 2 * Math.PI; cx.beginPath(); cx.arc(centerX, centerY, 100, currentAngle, currentAngle + sliceAngle); currentAngle += sliceAngle; cx.lineTo(centerX, centerY); cx.fillStyle = result.color; cx.fill(); }); </script> ~~~ ### [](http://eloquentjavascript.net/16_canvas.html#h_IoBBN8CiQ5)A bouncing ball [](http://eloquentjavascript.net/16_canvas.html#p_QtPClJ4kuI)Use the `requestAnimationFrame` technique that we saw in [Chapter 13](http://eloquentjavascript.net/13_dom.html#animationFrame) and[Chapter 15](http://eloquentjavascript.net/15_game.html#runAnimation) to draw a box with a bouncing ball in it. The ball moves at a constant speed and bounces off the box’s sides when it hits them. ~~~ <canvas width="400" height="400"></canvas> <script> var cx = document.querySelector("canvas").getContext("2d"); var lastTime = null; function frame(time) { if (lastTime != null) updateAnimation(Math.min(100, time - lastTime) / 1000); lastTime = time; requestAnimationFrame(frame); } requestAnimationFrame(frame); function updateAnimation(step) { // Your code here. } </script> ~~~ ### [](http://eloquentjavascript.net/16_canvas.html#h_3ePcd0S4v0)Precomputed mirroring [](http://eloquentjavascript.net/16_canvas.html#p_onlMAFaFji)One unfortunate thing about transformations is that they slow down drawing of bitmaps. For vector graphics, the effect is less serious since only a few points (for example, the center of a circle) need to be transformed, after which drawing can happen as normal. For a bitmap image, the position of each pixel has to be transformed, and though it is possible that browsers will get more clever about this in the future, this currently causes a measurable increase in the time it takes to draw a bitmap. [](http://eloquentjavascript.net/16_canvas.html#p_b50ouk4znE)In a game like ours, where we are drawing only a single transformed sprite, this is a nonissue. But imagine that we need to draw hundreds of characters or thousands of rotating particles from an explosion. [](http://eloquentjavascript.net/16_canvas.html#p_WgNsgEoaNe)Think of a way to allow us to draw an inverted character without loading additional image files and without having to make transformed `drawImage`calls every frame. The key to the solution is the fact that we can use a canvas element as a source image when using `drawImage`. It is possible to create an extra `<canvas>`element, without adding it to the document, and draw our inverted sprites to it, once. When drawing an actual frame, we just copy the already inverted sprites to the main canvas. [](http://eloquentjavascript.net/16_canvas.html#p_MbtRzmupcp)Some care would be required because images do not load instantly. We do the inverted drawing only once, and if we do it before the image loads, it won’t draw anything. A `"load"` handler on the image can be used to draw the inverted images to the extra canvas. This canvas can be used as a drawing source immediately (it’ll simply be blank until we draw the character onto it).

## Chapter 15 # Project: A Platform Game > [](http://eloquentjavascript.net/15_game.html#p_kUA7+lr6ay)All reality is a game. > > Iain Banks, The Player of Games [](http://eloquentjavascript.net/15_game.html#p_LHKzmIPa3c)My initial fascination with computers, like that of many kids, originated with computer games. I was drawn into the tiny computer-simulated worlds that I could manipulate and in which stories (sort of) unfolded—more, I suppose, because of the way I could project my imagination into them than because of the possibilities they actually offered. [](http://eloquentjavascript.net/15_game.html#p_oGz9+U3nKJ)I wouldn’t wish a career in game programming on anyone. Much like the music industry, the discrepancy between the many eager young people wanting to work in it and the actual demand for such people creates a rather unhealthy environment. But writing games for fun is amusing. [](http://eloquentjavascript.net/15_game.html#p_j3t+HvGbaT)This chapter will walk through the implementation of a simple platform game. Platform games (or “jump and run” games) are games that expect the player to move a figure through a world, which is often two-dimensional and viewed from the side, and do lots of jumping onto and over things. ## [](http://eloquentjavascript.net/15_game.html#h_lMtTRzata0)The game [](http://eloquentjavascript.net/15_game.html#p_C38xTPlNF8)Our game will be roughly based on [Dark Blue](http://www.lessmilk.com/games/10) by Thomas Palef. I chose this game because it is both entertaining and minimalist, and because it can be built without too much code. It looks like this:  [](http://eloquentjavascript.net/15_game.html#p_mIXBfsCnQQ)The dark box represents the player, whose task is to collect the yellow boxes (coins) while avoiding the red stuff (lava?). A level is completed when all coins have been collected. [](http://eloquentjavascript.net/15_game.html#p_g71I1Gdo7w)The player can walk around with the left and right arrow keys and jump with the up arrow. Jumping is a specialty of this game character. It can reach several times its own height and is able to change direction in midair. This may not be entirely realistic, but it helps give the player the feeling of being in direct control of the onscreen avatar. [](http://eloquentjavascript.net/15_game.html#p_or+OtPnSO1)The game consists of a fixed background, laid out like a grid, with the moving elements overlaid on that background. Each field on the grid is either empty, solid, or lava. The moving elements are the player, coins, and certain pieces of lava. Unlike the artificial life simulation from [Chapter 7](http://eloquentjavascript.net/07_elife.html#elife), the positions of these elements are not constrained to the grid—their coordinates may be fractional, allowing smooth motion. ## [](http://eloquentjavascript.net/15_game.html#h_hLFu/U4fE5)The technology [](http://eloquentjavascript.net/15_game.html#p_w6B1L26QOc)We will use the browser DOM to display the game, and we’ll read user input by handling key events. [](http://eloquentjavascript.net/15_game.html#p_wha4Kv9EnE)The screen- and keyboard-related code is only a tiny part of the work we need to do to build this game. Since everything looks like colored boxes, drawing is uncomplicated: we create DOM elements and use styling to give them a background color, size, and position. [](http://eloquentjavascript.net/15_game.html#p_5a/S/bF/ru)We can represent the background as a table since it is an unchanging grid of squares. The free-moving elements can be overlaid on top of that, using absolutely positioned elements. [](http://eloquentjavascript.net/15_game.html#p_kmznnwbiqx)In games and other programs that have to animate graphics and respond to user input without noticeable delay, efficiency is important. Although the DOM was not originally designed for high-performance graphics, it is actually better at this than you would expect. You saw some animations in [Chapter 13](http://eloquentjavascript.net/13_dom.html#animation). On a modern machine, a simple game like this performs well, even if we don’t think about optimization much. [](http://eloquentjavascript.net/15_game.html#p_wN87+E1mRv)In the [next chapter](http://eloquentjavascript.net/16_canvas.html#canvas), we will explore another browser technology, the `<canvas>`tag, which provides a more traditional way to draw graphics, working in terms of shapes and pixels rather than DOM elements. ## [](http://eloquentjavascript.net/15_game.html#h_7UfwmBGLOk)Levels [](http://eloquentjavascript.net/15_game.html#p_qtbcbQFDkK)In [Chapter 7](http://eloquentjavascript.net/07_elife.html#plan) we used arrays of strings to describe a two-dimensional grid. We can do the same here. It will allow us to design levels without first building a level editor. [](http://eloquentjavascript.net/15_game.html#p_Y3dxlqZNOr)A simple level would look like this: ~~~ var simpleLevelPlan = [ " ", " ", " x = x ", " x o o x ", " x @ xxxxx x ", " xxxxx x ", " x!!!!!!!!!!!!x ", " xxxxxxxxxxxxxx ", " " ]; ~~~ [](http://eloquentjavascript.net/15_game.html#p_/+8Bj7dHhI)Both the fixed grid and the moving elements are included in the plan. The `x`characters stand for walls, the space characters for empty space, and the exclamation marks represent fixed, nonmoving lava tiles. [](http://eloquentjavascript.net/15_game.html#p_lnXaKIc8yw)The `@` defines the place where the player starts. Every `o` is a coin, and the equal sign (`=`) stands for a block of lava that moves back and forth horizontally. Note that the grid for these positions will be set to contain empty space, and another data structure is used to track the position of such moving elements. [](http://eloquentjavascript.net/15_game.html#p_3Abhc0fnza)We’ll support two other kinds of moving lava: the pipe character (`|`) for vertically moving blobs, and `v` for *dripping* lava—vertically moving lava that doesn’t bounce back and forth but only moves down, jumping back to its start position when it hits the floor. [](http://eloquentjavascript.net/15_game.html#p_JSlRu3lL/0)A whole game consists of multiple levels that the player must complete. A level is completed when all coins have been collected. If the player touches lava, the current level is restored to its starting position, and the player may try again. ## [](http://eloquentjavascript.net/15_game.html#h_DeVC1tufta)Reading a level [](http://eloquentjavascript.net/15_game.html#p_+Eovnn74MN)The following constructor builds a level object. Its argument should be the array of strings that define the level. ~~~ function Level(plan) { this.width = plan[0].length; this.height = plan.length; this.grid = []; this.actors = []; for (var y = 0; y < this.height; y++) { var line = plan[y], gridLine = []; for (var x = 0; x < this.width; x++) { var ch = line[x], fieldType = null; var Actor = actorChars[ch]; if (Actor) this.actors.push(new Actor(new Vector(x, y), ch)); else if (ch == "x") fieldType = "wall"; else if (ch == "!") fieldType = "lava"; gridLine.push(fieldType); } this.grid.push(gridLine); } this.player = this.actors.filter(function(actor) { return actor.type == "player"; })[0]; this.status = this.finishDelay = null; } ~~~ [](http://eloquentjavascript.net/15_game.html#p_ZIIJavICol)For brevity, the code does not check for malformed input. It assumes that you’ve given it a proper level plan, complete with a player start position and other essentials. [](http://eloquentjavascript.net/15_game.html#p_x/twI7oWfP)A level stores its width and height, along with two arrays—one for the grid and one for the *actors*, which are the dynamic elements. The grid is represented as an array of arrays, where each of the inner arrays represents a horizontal line and each square contains either null, for empty squares, or a string indicating the type of the square—`"wall"` or `"lava"`. [](http://eloquentjavascript.net/15_game.html#p_lk9TyBu+V9)The actors array holds objects that track the current position and state of the dynamic elements in the level. Each of these is expected to have a `pos` property that gives its position (the coordinates of its top-left corner), a `size` property that gives its size, and a `type` property that holds a string identifying the element (`"lava"`, `"coin"`, or `"player"`). [](http://eloquentjavascript.net/15_game.html#p_CHBpIMO/+A)After building the grid, we use the `filter` method to find the player actor object, which we store in a property of the level. The `status` property tracks whether the player has won or lost. When this happens, `finishDelay` is used to keep the level active for a short period of time so that a simple animation can be shown. (Immediately resetting or advancing the level would look cheap.) This method can be used to find out whether a level is finished: ~~~ Level.prototype.isFinished = function() { return this.status != null && this.finishDelay < 0; }; ~~~ ## [](http://eloquentjavascript.net/15_game.html#h_pw0251T7gn)Actors [](http://eloquentjavascript.net/15_game.html#p_E0kVEhfjBl)To store the position and size of an actor, we will return to our trusty `Vector`type, which groups an x-coordinate and a y-coordinate into an object. ~~~ function Vector(x, y) { this.x = x; this.y = y; } Vector.prototype.plus = function(other) { return new Vector(this.x + other.x, this.y + other.y); }; Vector.prototype.times = function(factor) { return new Vector(this.x * factor, this.y * factor); }; ~~~ [](http://eloquentjavascript.net/15_game.html#p_YBJ9X5jC/P)The `times` method scales a vector by a given amount. It will be useful when we need to multiply a speed vector by a time interval to get the distance traveled during that time. [](http://eloquentjavascript.net/15_game.html#p_ZUYAD4nQKB)In the previous section, the `actorChars` object was used by the `Level`constructor to associate characters with constructor functions. The object looks like this: ~~~ var actorChars = { "@": Player, "o": Coin, "=": Lava, "|": Lava, "v": Lava }; ~~~ [](http://eloquentjavascript.net/15_game.html#p_Ezz8Z00Wak)Three characters map to `Lava`. The `Level` constructor passes the actor’s source character as the second argument to the constructor, and the `Lava`constructor uses that to adjust its behavior (bouncing horizontally, bouncing vertically, or dripping). [](http://eloquentjavascript.net/15_game.html#p_9z5NweLFob)The player type is built with the following constructor. It has a property `speed`that stores its current speed, which will help simulate momentum and gravity. ~~~ function Player(pos) { this.pos = pos.plus(new Vector(0, -0.5)); this.size = new Vector(0.8, 1.5); this.speed = new Vector(0, 0); } Player.prototype.type = "player"; ~~~ [](http://eloquentjavascript.net/15_game.html#p_KwI14wjzqB)Because a player is one-and-a-half squares high, its initial position is set to be half a square above the position where the `@` character appeared. This way, its bottom aligns with the bottom of the square it appeared in. [](http://eloquentjavascript.net/15_game.html#p_tERSi4X6Me)When constructing a dynamic `Lava` object, we need to initialize the object differently depending on the character it is based on. Dynamic lava moves along at its given speed until it hits an obstacle. At that point, if it has a`repeatPos` property, it will jump back to its start position (dripping). If it does not, it will invert its speed and continue in the other direction (bouncing). The constructor only sets up the necessary properties. The method that does the actual moving will be written [later](http://eloquentjavascript.net/15_game.html#actors). ~~~ function Lava(pos, ch) { this.pos = pos; this.size = new Vector(1, 1); if (ch == "=") { this.speed = new Vector(2, 0); } else if (ch == "|") { this.speed = new Vector(0, 2); } else if (ch == "v") { this.speed = new Vector(0, 3); this.repeatPos = pos; } } Lava.prototype.type = "lava"; ~~~ [](http://eloquentjavascript.net/15_game.html#p_h6fsluS1sx)`Coin` actors are simple. They mostly just sit in their place. But to liven up the game a little, they are given a “wobble”, a slight vertical motion back and forth. To track this, a coin object stores a base position as well as a `wobble` property that tracks the phase of the bouncing motion. Together, these determine the coin’s actual position (stored in the `pos` property). ~~~ function Coin(pos) { this.basePos = this.pos = pos.plus(new Vector(0.2, 0.1)); this.size = new Vector(0.6, 0.6); this.wobble = Math.random() * Math.PI * 2; } Coin.prototype.type = "coin"; ~~~ [](http://eloquentjavascript.net/15_game.html#p_sinZPrR05b)In [Chapter 13](http://eloquentjavascript.net/13_dom.html#sin_cos), we saw that `Math.sin` gives us the y-coordinate of a point on a circle. That coordinate goes back and forth in a smooth wave form as we move along the circle, which makes the sine function useful for modeling a wavy motion. [](http://eloquentjavascript.net/15_game.html#p_gQCua74XOk)To avoid a situation where all coins move up and down synchronously, the starting phase of each coin is randomized. The *phase* of `Math.sin`'s wave, the width of a wave it produces, is 2π. We multiply the value returned by`Math.random` by that number to give the coin a random starting position on the wave. [](http://eloquentjavascript.net/15_game.html#p_q4rwQ9GT76)We have now written all the parts needed to represent the state of a level. ~~~ var simpleLevel = new Level(simpleLevelPlan); console.log(simpleLevel.width, "by", simpleLevel.height); // → 22 by 9 ~~~ [](http://eloquentjavascript.net/15_game.html#p_lCdOTin0mI)The task ahead is to display such levels on the screen and to model time and motion inside them. ## [](http://eloquentjavascript.net/15_game.html#h_uCRd57RG2L)Encapsulation as a burden [](http://eloquentjavascript.net/15_game.html#p_M65QHGE4qM)Most of the code in this chapter does not worry about encapsulation for two reasons. First, encapsulation takes extra effort. It makes programs bigger and requires additional concepts and interfaces to be introduced. Since there is only so much code you can throw at a reader before their eyes glaze over, I’ve made an effort to keep the program small. [](http://eloquentjavascript.net/15_game.html#p_xjLloHI+Wv)Second, the various elements in this game are so closely tied together that if the behavior of one of them changed, it is unlikely that any of the others would be able to stay the same. Interfaces between the elements would end up encoding a lot of assumptions about the way the game works. This makes them a lot less effective—whenever you change one part of the system, you still have to worry about the way it impacts the other parts because their interfaces wouldn’t cover the new situation. [](http://eloquentjavascript.net/15_game.html#p_JXDPJBULIv)Some *cutting points* in a system lend themselves well to separation through rigorous interfaces, but others don’t. Trying to encapsulate something that isn’t a suitable boundary is a sure way to waste a lot of energy. When you are making this mistake, you’ll usually notice that your interfaces are getting awkwardly large and detailed and that they need to be modified often, as the program evolves. [](http://eloquentjavascript.net/15_game.html#p_Eajw41C5DX)There is one thing that we *will* encapsulate in this chapter, and that is the drawing subsystem. The reason for this is that we will display the same game in a different way in the [next chapter](http://eloquentjavascript.net/16_canvas.html#canvasdisplay). By putting the drawing behind an interface, we can simply load the same game program there and plug in a new display module. ## [](http://eloquentjavascript.net/15_game.html#h_neNgUMdlHQ)Drawing [](http://eloquentjavascript.net/15_game.html#p_bjlUPfTgQP)The encapsulation of the drawing code is done by defining a *display* object, which displays a given level. The display type we define in this chapter is called`DOMDisplay` because it uses simple DOM elements to show the level. [](http://eloquentjavascript.net/15_game.html#p_iqZAywSomZ)We will be using a style sheet to set the actual colors and other fixed properties of the elements that make up the game. It would also be possible to directly assign to the elements’ `style` property when we create them, but that would produce more verbose programs. [](http://eloquentjavascript.net/15_game.html#p_cp0ecVNl6a)The following helper function provides a short way to create an element and give it a class: ~~~ function elt(name, className) { var elt = document.createElement(name); if (className) elt.className = className; return elt; } ~~~ [](http://eloquentjavascript.net/15_game.html#p_Xjpq/reXQf)A display is created by giving it a parent element to which it should append itself and a level object. ~~~ function DOMDisplay(parent, level) { this.wrap = parent.appendChild(elt("div", "game")); this.level = level; this.wrap.appendChild(this.drawBackground()); this.actorLayer = null; this.drawFrame(); } ~~~ [](http://eloquentjavascript.net/15_game.html#p_ma7b253RBp)We used the fact that `appendChild` returns the appended element to create the wrapper element and store it in the `wrap` property in a single statement. [](http://eloquentjavascript.net/15_game.html#p_IMKo7SUQR3)The level’s background, which never changes, is drawn once. The actors are redrawn every time the display is updated. The `actorLayer` property will be used by `drawFrame` to track the element that holds the actors so that they can be easily removed and replaced. [](http://eloquentjavascript.net/15_game.html#p_si3+n3Lijy)Our coordinates and sizes are tracked in units relative to the grid size, where a size or distance of 1 means 1 grid unit. When setting pixel sizes, we will have to scale these coordinates up—everything in the game would be ridiculously small at a single pixel per square. The `scale` variable gives the number of pixels that a single unit takes up on the screen. ~~~ var scale = 20; DOMDisplay.prototype.drawBackground = function() { var table = elt("table", "background"); table.style.width = this.level.width * scale + "px"; this.level.grid.forEach(function(row) { var rowElt = table.appendChild(elt("tr")); rowElt.style.height = scale + "px"; row.forEach(function(type) { rowElt.appendChild(elt("td", type)); }); }); return table; }; ~~~ [](http://eloquentjavascript.net/15_game.html#p_d+zoai4uXa)As mentioned earlier, the background is drawn as a `<table>` element. This nicely corresponds to the structure of the `grid` property in the level—each row of the grid is turned into a table row (`<tr>` element). The strings in the grid are used as class names for the table cell (`<td>`) elements. The following CSS helps the resulting table look like the background we want: ~~~ .background { background: rgb(52, 166, 251); table-layout: fixed; border-spacing: 0; } .background td { padding: 0; } .lava { background: rgb(255, 100, 100); } .wall { background: white; } ~~~ [](http://eloquentjavascript.net/15_game.html#p_Fm4CLmRVL5)Some of these (`table-layout`, `border-spacing`, and `padding`) are simply used to suppress unwanted default behavior. We don’t want the layout of the table to depend upon the contents of its cells, and we don’t want space between the table cells or padding inside them. [](http://eloquentjavascript.net/15_game.html#p_wtigqOBUxT)The `background` rule sets the background color. CSS allows colors to be specified both as words (`white`) and with a format such as `rgb(R, G, B)`, where the red, green, and blue components of the color are separated into three numbers from 0 to 255\. So, in `rgb(52, 166, 251)`, the red component is 52, green is 166, and blue is 251\. Since the blue component is the largest, the resulting color will be bluish. You can see that in the `.lava` rule, the first number (red) is the largest. [](http://eloquentjavascript.net/15_game.html#p_EGE24ax3xh)We draw each actor by creating a DOM element for it and setting that element’s position and size based on the actor’s properties. The values have to be multiplied by `scale` to go from game units to pixels. ~~~ DOMDisplay.prototype.drawActors = function() { var wrap = elt("div"); this.level.actors.forEach(function(actor) { var rect = wrap.appendChild(elt("div", "actor " + actor.type)); rect.style.width = actor.size.x * scale + "px"; rect.style.height = actor.size.y * scale + "px"; rect.style.left = actor.pos.x * scale + "px"; rect.style.top = actor.pos.y * scale + "px"; }); return wrap; }; ~~~ [](http://eloquentjavascript.net/15_game.html#p_N1paxjwHY8)To give an element more than one class, we separate the class names by spaces. In the CSS code shown next, the `actor` class gives the actors their absolute position. Their type name is used as an extra class to give them a color. We don’t have to define the `lava` class again because we reuse the class for the lava grid squares which we defined earlier. ~~~ .actor { position: absolute; } .coin { background: rgb(241, 229, 89); } .player { background: rgb(64, 64, 64); } ~~~ [](http://eloquentjavascript.net/15_game.html#p_ee+EdXi5xG)When it updates the display, the `drawFrame` method first removes the old actor graphics, if any, and then redraws them in their new positions. It may be tempting to try to reuse the DOM elements for actors, but to make that work, we would need a lot of additional information flow between the display code and the simulation code. We’d need to associate actors with DOM elements, and the drawing code must remove elements when their actors vanish. Since there will typically be only a handful of actors in the game, redrawing all of them is not expensive. ~~~ DOMDisplay.prototype.drawFrame = function() { if (this.actorLayer) this.wrap.removeChild(this.actorLayer); this.actorLayer = this.wrap.appendChild(this.drawActors()); this.wrap.className = "game " + (this.level.status || ""); this.scrollPlayerIntoView(); }; ~~~ [](http://eloquentjavascript.net/15_game.html#p_sZEoSNaFbo)By adding the level’s current status as a class name to the wrapper, we can style the player actor slightly differently when the game is won or lost by adding a CSS rule that takes effect only when the player has an ancestor element with a given class. ~~~ .lost .player { background: rgb(160, 64, 64); } .won .player { box-shadow: -4px -7px 8px white, 4px -7px 8px white; } ~~~ [](http://eloquentjavascript.net/15_game.html#p_Ta1z3iQqo1)After touching lava, the player’s color turns dark red, suggesting scorching. When the last coin has been collected, we use two blurred white box shadows, one to the top left and one to the top right, to create a white halo effect. [](http://eloquentjavascript.net/15_game.html#p_hU1RuMZZhL)We can’t assume that levels always fit in the viewport. That is why the`scrollPlayerIntoView` call is needed—it ensures that if the level is protruding outside the viewport, we scroll that viewport to make sure the player is near its center. The following CSS gives the game’s wrapping DOM element a maximum size and ensures that anything that sticks out of the element’s box is not visible. We also give the outer element a relative position so that the actors inside it are positioned relative to the level’s top-left corner. ~~~ .game { overflow: hidden; max-width: 600px; max-height: 450px; position: relative; } ~~~ [](http://eloquentjavascript.net/15_game.html#p_1/wCuAnu9B)In the `scrollPlayerIntoView` method, we find the player’s position and update the wrapping element’s scroll position. We change the scroll position by manipulating that element’s `scrollLeft` and `scrollTop` properties when the player is too close to the edge. ~~~ DOMDisplay.prototype.scrollPlayerIntoView = function() { var width = this.wrap.clientWidth; var height = this.wrap.clientHeight; var margin = width / 3; // The viewport var left = this.wrap.scrollLeft, right = left + width; var top = this.wrap.scrollTop, bottom = top + height; var player = this.level.player; var center = player.pos.plus(player.size.times(0.5)) .times(scale); if (center.x < left + margin) this.wrap.scrollLeft = center.x - margin; else if (center.x > right - margin) this.wrap.scrollLeft = center.x + margin - width; if (center.y < top + margin) this.wrap.scrollTop = center.y - margin; else if (center.y > bottom - margin) this.wrap.scrollTop = center.y + margin - height; }; ~~~ [](http://eloquentjavascript.net/15_game.html#p_3qHzB4KoD+)The way the player’s center is found shows how the methods on our `Vector`type allow computations with objects to be written in a readable way. To find the actor’s center, we add its position (its top-left corner) and half its size. That is the center in level coordinates, but we need it in pixel coordinates, so we then multiply the resulting vector by our display scale. [](http://eloquentjavascript.net/15_game.html#p_nyYhuiyn32)Next, a series of checks verify that the player position isn’t outside of the allowed range. Note that sometimes this will set nonsense scroll coordinates, below zero or beyond the element’s scrollable area. This is okay—the DOM will constrain them to sane values. Setting `scrollLeft` to -10 will cause it to become 0. [](http://eloquentjavascript.net/15_game.html#p_MFibm1pU7d)It would have been slightly simpler to always try to scroll the player to the center of the viewport. But this creates a rather jarring effect. As you are jumping, the view will constantly shift up and down. It is more pleasant to have a “neutral” area in the middle of the screen where you can move around without causing any scrolling. [](http://eloquentjavascript.net/15_game.html#p_5o/NVpkfaU)Finally, we’ll need a way to clear a displayed level, to be used when the game moves to the next level or resets a level. ~~~ DOMDisplay.prototype.clear = function() { this.wrap.parentNode.removeChild(this.wrap); }; ~~~ [](http://eloquentjavascript.net/15_game.html#p_LSD2j1d23Y)We are now able to display our tiny level. ~~~ <link rel="stylesheet" href="css/game.css"> <script> var simpleLevel = new Level(simpleLevelPlan); var display = new DOMDisplay(document.body, simpleLevel); </script> ~~~ [](http://eloquentjavascript.net/15_game.html#p_CLn8fpSTCw)The `<link>` tag, when used with `rel="stylesheet"`, is a way to load a CSS file into a page. The file `game.css` contains the styles necessary for our game. ## [](http://eloquentjavascript.net/15_game.html#h_zX4xC7JBQU)Motion and collision [](http://eloquentjavascript.net/15_game.html#p_VZl40vezkA)Now we’re at the point where we can start adding motion—the most interesting aspect of the game. The basic approach, taken by most games like this, is to split time into small steps and, for each step, move the actors by a distance corresponding to their speed (distance moved per second) multiplied by the size of the time step (in seconds). [](http://eloquentjavascript.net/15_game.html#p_+a3yt6NY8z)That is easy. The difficult part is dealing with the interactions between the elements. When the player hits a wall or floor, they should not simply move through it. The game must notice when a given motion causes an object to hit another object and respond accordingly. For walls, the motion must be stopped. For coins, the coin must be collected, and so on. [](http://eloquentjavascript.net/15_game.html#p_AidQCfGku0)Solving this for the general case is a big task. You can find libraries, usually called *physics engines*, that simulate interaction between physical objects in two or three dimensions. We’ll take a more modest approach in this chapter, handling only collisions between rectangular objects and handling them in a rather simplistic way. [](http://eloquentjavascript.net/15_game.html#p_qnEGluIxK3)Before moving the player or a block of lava, we test whether the motion would take it inside of a nonempty part of the background. If it does, we simply cancel the motion altogether. The response to such a collision depends on the type of actor—the player will stop, whereas a lava block will bounce back. [](http://eloquentjavascript.net/15_game.html#p_Y/iDVnt+Uc)This approach requires our time steps to be rather small since it will cause motion to stop before the objects actually touch. If the time steps (and thus the motion steps) are too big, the player would end up hovering a noticeable distance above the ground. Another approach, arguably better but more complicated, would be to find the exact collision spot and move there. We will take the simple approach and hide its problems by ensuring the animation proceeds in small steps. [](http://eloquentjavascript.net/15_game.html#p_mgrJ3S203s)This method tells us whether a rectangle (specified by a position and a size) overlaps with any nonempty space on the background grid: ~~~ Level.prototype.obstacleAt = function(pos, size) { var xStart = Math.floor(pos.x); var xEnd = Math.ceil(pos.x + size.x); var yStart = Math.floor(pos.y); var yEnd = Math.ceil(pos.y + size.y); if (xStart < 0 || xEnd > this.width || yStart < 0) return "wall"; if (yEnd > this.height) return "lava"; for (var y = yStart; y < yEnd; y++) { for (var x = xStart; x < xEnd; x++) { var fieldType = this.grid[y][x]; if (fieldType) return fieldType; } } }; ~~~ [](http://eloquentjavascript.net/15_game.html#p_PToDD5K9+0)This method computes the set of grid squares that the body overlaps with by using `Math.floor` and `Math.ceil` on the body’s coordinates. Remember that grid squares are 1×1 units in size. By rounding the sides of a box up and down, we get the range of background squares that the box touches.  [](http://eloquentjavascript.net/15_game.html#p_MZ2cPIh/k/)If the body sticks out of the level, we always return `"wall"` for the sides and top and `"lava"` for the bottom. This ensures that the player dies when falling out of the world. When the body is fully inside the grid, we loop over the block of grid squares found by rounding the coordinates and return the content of the first nonempty square we find. [](http://eloquentjavascript.net/15_game.html#p_oAeaIvbuEY)Collisions between the player and other dynamic actors (coins, moving lava) are handled *after* the player moved. When the motion has taken the player into another actor, the appropriate effect—collecting a coin or dying—is activated. [](http://eloquentjavascript.net/15_game.html#p_9zGn/O3Wbz)This method scans the array of actors, looking for an actor that overlaps the one given as an argument: ~~~ Level.prototype.actorAt = function(actor) { for (var i = 0; i < this.actors.length; i++) { var other = this.actors[i]; if (other != actor && actor.pos.x + actor.size.x > other.pos.x && actor.pos.x < other.pos.x + other.size.x && actor.pos.y + actor.size.y > other.pos.y && actor.pos.y < other.pos.y + other.size.y) return other; } }; ~~~ ## [](http://eloquentjavascript.net/15_game.html#h_CK9RMsBUn4)Actors and actions [](http://eloquentjavascript.net/15_game.html#p_HatfNTrDd5)The `animate` method on the `Level` type gives all actors in the level a chance to move. Its `step` argument is the time step in seconds. The `keys` object contains information about the arrow keys the player has pressed. ~~~ var maxStep = 0.05; Level.prototype.animate = function(step, keys) { if (this.status != null) this.finishDelay -= step; while (step > 0) { var thisStep = Math.min(step, maxStep); this.actors.forEach(function(actor) { actor.act(thisStep, this, keys); }, this); step -= thisStep; } }; ~~~ [](http://eloquentjavascript.net/15_game.html#p_3CL3X0Momn)When the level’s `status` property has a non-null value (which is the case when the player has won or lost), we must count down the `finishDelay` property, which tracks the time between the point where winning or losing happens and the point where we want to stop showing the level. [](http://eloquentjavascript.net/15_game.html#p_1RlyY9ZKcW)The `while` loop cuts the time step we are animating into suitably small pieces. It ensures that no step larger than `maxStep` is taken. For example, a `step` of 0.12 second would be cut into two steps of 0.05 seconds and one step of 0.02. [](http://eloquentjavascript.net/15_game.html#p_AiCSiwkX6Y)Actor objects have an `act` method, which takes as arguments the time step, the level object, and the `keys` object. Here is one, for the `Lava` actor type, which ignores the `keys` object: ~~~ Lava.prototype.act = function(step, level) { var newPos = this.pos.plus(this.speed.times(step)); if (!level.obstacleAt(newPos, this.size)) this.pos = newPos; else if (this.repeatPos) this.pos = this.repeatPos; else this.speed = this.speed.times(-1); }; ~~~ [](http://eloquentjavascript.net/15_game.html#p_hQ66AcUb17)It computes a new position by adding the product of the time step and its current speed to its old position. If no obstacle blocks that new position, it moves there. If there is an obstacle, the behavior depends on the type of the lava block—dripping lava has a `repeatPos` property, to which it jumps back when it hits something. Bouncing lava simply inverts its speed (multiplies it by -1) in order to start moving in the other direction. [](http://eloquentjavascript.net/15_game.html#p_1RXQ29sidv)Coins use their `act` method to wobble. They ignore collisions since they are simply wobbling around inside of their own square, and collisions with the player will be handled by the *player*'s `act` method. ~~~ var wobbleSpeed = 8, wobbleDist = 0.07; Coin.prototype.act = function(step) { this.wobble += step * wobbleSpeed; var wobblePos = Math.sin(this.wobble) * wobbleDist; this.pos = this.basePos.plus(new Vector(0, wobblePos)); }; ~~~ [](http://eloquentjavascript.net/15_game.html#p_aXWUL4a4Y+)The `wobble` property is updated to track time and then used as an argument to`Math.sin` to create a wave, which is used to compute a new position. [](http://eloquentjavascript.net/15_game.html#p_7eSdIMS/wV)That leaves the player itself. Player motion is handled separately per axis because hitting the floor should not prevent horizontal motion, and hitting a wall should not stop falling or jumping motion. This method implements the horizontal part: ~~~ var playerXSpeed = 7; Player.prototype.moveX = function(step, level, keys) { this.speed.x = 0; if (keys.left) this.speed.x -= playerXSpeed; if (keys.right) this.speed.x += playerXSpeed; var motion = new Vector(this.speed.x * step, 0); var newPos = this.pos.plus(motion); var obstacle = level.obstacleAt(newPos, this.size); if (obstacle) level.playerTouched(obstacle); else this.pos = newPos; }; ~~~ [](http://eloquentjavascript.net/15_game.html#p_Tdw21cw9Dq)The horizontal motion is computed based on the state of the left and right arrow keys. When a motion causes the player to hit something, the level’s`playerTouched` method, which handles things like dying in lava and collecting coins, is called. Otherwise, the object updates its position. [](http://eloquentjavascript.net/15_game.html#p_du3894nYAV)Vertical motion works in a similar way but has to simulate jumping and gravity. ~~~ var gravity = 30; var jumpSpeed = 17; Player.prototype.moveY = function(step, level, keys) { this.speed.y += step * gravity; var motion = new Vector(0, this.speed.y * step); var newPos = this.pos.plus(motion); var obstacle = level.obstacleAt(newPos, this.size); if (obstacle) { level.playerTouched(obstacle); if (keys.up && this.speed.y > 0) this.speed.y = -jumpSpeed; else this.speed.y = 0; } else { this.pos = newPos; } }; ~~~ [](http://eloquentjavascript.net/15_game.html#p_n3f+aTx4m7)At the start of the method, the player is accelerated vertically to account for gravity. The gravity, jumping speed, and pretty much all other constants in this game have been set by trial and error. I tested various values until I found a combination I liked. [](http://eloquentjavascript.net/15_game.html#p_Vtbh5elQbX)Next, we check for obstacles again. If we hit an obstacle, there are two possible outcomes. When the up arrow is pressed *and* we are moving down (meaning the thing we hit is below us), the speed is set to a relatively large, negative value. This causes the player to jump. If that is not the case, we simply bumped into something, and the speed is reset to zero. [](http://eloquentjavascript.net/15_game.html#p_YCAh/Cx+MG)The actual `act` method looks like this: ~~~ Player.prototype.act = function(step, level, keys) { this.moveX(step, level, keys); this.moveY(step, level, keys); var otherActor = level.actorAt(this); if (otherActor) level.playerTouched(otherActor.type, otherActor); // Losing animation if (level.status == "lost") { this.pos.y += step; this.size.y -= step; } }; ~~~ [](http://eloquentjavascript.net/15_game.html#p_4WJ+6ygCcZ)After moving, the method checks for other actors that the player is colliding with and again calls `playerTouched` when it finds one. This time, it passes the actor object as the second argument because if the other actor is a coin,`playerTouched` needs to know *which* coin is being collected. [](http://eloquentjavascript.net/15_game.html#p_ZvTlzr2mKf)Finally, when the player dies (touches lava), we set up a little animation that causes them to “shrink” or “sink” down by reducing the height of the player object. [](http://eloquentjavascript.net/15_game.html#p_ax6pc274qT)And here is the method that handles collisions between the player and other objects: ~~~ Level.prototype.playerTouched = function(type, actor) { if (type == "lava" && this.status == null) { this.status = "lost"; this.finishDelay = 1; } else if (type == "coin") { this.actors = this.actors.filter(function(other) { return other != actor; }); if (!this.actors.some(function(actor) { return actor.type == "coin"; })) { this.status = "won"; this.finishDelay = 1; } } }; ~~~ [](http://eloquentjavascript.net/15_game.html#p_2TZ51jE0Cq)When lava is touched, the game’s status is set to `"lost"`. When a coin is touched, that coin is removed from the array of actors, and if it was the last one, the game’s status is set to `"won"`. [](http://eloquentjavascript.net/15_game.html#p_Tb0QFBJu6P)This gives us a level that can actually be animated. All that is missing now is the code that *drives* the animation. ## [](http://eloquentjavascript.net/15_game.html#h_zKch6Si/SS)Tracking keys [](http://eloquentjavascript.net/15_game.html#p_6qSDIbEzkh)For a game like this, we do not want keys to take effect once per keypress. Rather, we want their effect (moving the player figure) to continue happening as long as they are pressed. [](http://eloquentjavascript.net/15_game.html#p_kKDop4NY9B)We need to set up a key handler that stores the current state of the left, right, and up arrow keys. We will also want to call `preventDefault` for those keys so that they don’t end up scrolling the page. [](http://eloquentjavascript.net/15_game.html#p_oH4kiTyM1E)The following function, when given an object with key codes as property names and key names as values, will return an object that tracks the current position of those keys. It registers event handlers for `"keydown"` and `"keyup"` events and, when the key code in the event is present in the set of codes that it is tracking, updates the object. ~~~ var arrowCodes = {37: "left", 38: "up", 39: "right"}; function trackKeys(codes) { var pressed = Object.create(null); function handler(event) { if (codes.hasOwnProperty(event.keyCode)) { var down = event.type == "keydown"; pressed[codes[event.keyCode]] = down; event.preventDefault(); } } addEventListener("keydown", handler); addEventListener("keyup", handler); return pressed; } ~~~ [](http://eloquentjavascript.net/15_game.html#p_ijH6rqUGtM)Note how the same handler function is used for both event types. It looks at the event object’s `type` property to determine whether the key state should be updated to true (`"keydown"`) or false (`"keyup"`). ## [](http://eloquentjavascript.net/15_game.html#h_/jwYTlYjAy)Running the game [](http://eloquentjavascript.net/15_game.html#p_3fS5a/SYfz)The `requestAnimationFrame` function, which we saw in [Chapter 13](http://eloquentjavascript.net/13_dom.html#animationFrame), provides a good way to animate a game. But its interface is quite primitive—using it requires us to track the time at which our function was called the last time around and call `requestAnimationFrame` again after every frame. [](http://eloquentjavascript.net/15_game.html#p_6blpyw8yI3)Let’s define a helper function that wraps those boring parts in a convenient interface and allows us to simply call `runAnimation`, giving it a function that expects a time difference as an argument and draws a single frame. When the frame function returns the value `false`, the animation stops. ~~~ function runAnimation(frameFunc) { var lastTime = null; function frame(time) { var stop = false; if (lastTime != null) { var timeStep = Math.min(time - lastTime, 100) / 1000; stop = frameFunc(timeStep) === false; } lastTime = time; if (!stop) requestAnimationFrame(frame); } requestAnimationFrame(frame); } ~~~ [](http://eloquentjavascript.net/15_game.html#p_qaBaJbkLcb)I have set a maximum frame step of 100 milliseconds (one-tenth of a second). When the browser tab or window with our page is hidden,`requestAnimationFrame` calls will be suspended until the tab or window is shown again. In this case, the difference between `lastTime` and `time` will be the entire time in which the page was hidden. Advancing the game by that much in a single step will look silly and might be a lot of work (remember the time-splitting in the [`animate` method](http://eloquentjavascript.net/15_game.html#actors)). [](http://eloquentjavascript.net/15_game.html#p_jKakPLUmwL)The function also converts the time steps to seconds, which are an easier quantity to think about than milliseconds. [](http://eloquentjavascript.net/15_game.html#p_e8mbmQ7w8j)The `runLevel` function takes a `Level` object, a constructor for a display, and, optionally, a function. It displays the level (in `document.body`) and lets the user play through it. When the level is finished (lost or won), `runLevel` clears the display, stops the animation, and, if an `andThen` function was given, calls that function with the level’s status. ~~~ var arrows = trackKeys(arrowCodes); function runLevel(level, Display, andThen) { var display = new Display(document.body, level); runAnimation(function(step) { level.animate(step, arrows); display.drawFrame(step); if (level.isFinished()) { display.clear(); if (andThen) andThen(level.status); return false; } }); } ~~~ [](http://eloquentjavascript.net/15_game.html#p_eyKzVe0sIB)A game is a sequence of levels. Whenever the player dies, the current level is restarted. When a level is completed, we move on to the next level. This can be expressed by the following function, which takes an array of level plans (arrays of strings) and a display constructor: ~~~ function runGame(plans, Display) { function startLevel(n) { runLevel(new Level(plans[n]), Display, function(status) { if (status == "lost") startLevel(n); else if (n < plans.length - 1) startLevel(n + 1); else console.log("You win!"); }); } startLevel(0); } ~~~ [](http://eloquentjavascript.net/15_game.html#p_Btjn/eg/VT)These functions show a peculiar style of programming. Both `runAnimation`and `runLevel` are higher-order functions but are not in the style we saw in[Chapter 5](http://eloquentjavascript.net/05_higher_order.html#higher_order). The function argument is used to arrange things to happen at some time in the future, and neither of the functions returns anything useful. Their task is, in a way, to schedule actions. Wrapping these actions in functions gives us a way to store them as a value so that they can be called at the right moment. [](http://eloquentjavascript.net/15_game.html#p_HC69jX9E3n)This programming style is usually called *asynchronous* programming. Event handling is also an instance of this style, and we will see much more of it when working with tasks that can take an arbitrary amount of time, such as network requests in [Chapter 17](http://eloquentjavascript.net/17_http.html#http) and input and output in general in [Chapter 20](http://eloquentjavascript.net/20_node.html#node). [](http://eloquentjavascript.net/15_game.html#p_/6dLhjN2fB)There is a set of level plans available in the `GAME_LEVELS` variable . This page feeds them to `runGame`, starting an actual game: ~~~ <link rel="stylesheet" href="css/game.css"> <body> <script> runGame(GAME_LEVELS, DOMDisplay); </script> </body> ~~~ [](http://eloquentjavascript.net/15_game.html#p_MkrZ67rFcA)See if you can beat those. I had quite a lot of fun building them. ## [](http://eloquentjavascript.net/15_game.html#h_TcUD2vzyMe)Exercises ### [](http://eloquentjavascript.net/15_game.html#h_tFsh86eaJC)Game over [](http://eloquentjavascript.net/15_game.html#p_JkKGPYeFzA)It’s traditional for platform games to have the player start with a limited number of *lives* and subtract one life each time they die. When the player is out of lives, the game restarts from the beginning. [](http://eloquentjavascript.net/15_game.html#p_oYENd6hUVn)Adjust `runGame` to implement lives. Have the player start with three. ~~~ <link rel="stylesheet" href="css/game.css"> <body> <script> // The old runGame function. Modify it... function runGame(plans, Display) { function startLevel(n) { runLevel(new Level(plans[n]), Display, function(status) { if (status == "lost") startLevel(n); else if (n < plans.length - 1) startLevel(n + 1); else console.log("You win!"); }); } startLevel(0); } runGame(GAME_LEVELS, DOMDisplay); </script> </body> ~~~ ### [](http://eloquentjavascript.net/15_game.html#h_cNfzuXtVqI)Pausing the game [](http://eloquentjavascript.net/15_game.html#p_GkkSB/bDtt)Make it possible to pause (suspend) and unpause the game by pressing the Esc key. [](http://eloquentjavascript.net/15_game.html#p_FpramcVlTZ)This can be done by changing the `runLevel` function to use another keyboard event handler and interrupting or resuming the animation whenever the Esc key is hit. [](http://eloquentjavascript.net/15_game.html#p_vlGSY83Wnl)The `runAnimation` interface may not look like it is suitable for this at first glance, but it is, if you rearrange the way `runLevel` calls it. [](http://eloquentjavascript.net/15_game.html#p_0zF8rh7TLp)When you have that working, there is something else you could try. The way we have been registering keyboard event handlers is somewhat problematic. The`arrows` object is currently a global variable, and its event handlers are kept around even when no game is running. You could say they *leak* out of our system. Extend `trackKeys` to provide a way to unregister its handlers, and then change `runLevel` to register its handlers when it starts and unregister them again when it is finished. ~~~ <link rel="stylesheet" href="css/game.css"> <body> <script> // The old runLevel function. Modify this... function runLevel(level, Display, andThen) { var display = new Display(document.body, level); runAnimation(function(step) { level.animate(step, arrows); display.drawFrame(step); if (level.isFinished()) { display.clear(); if (andThen) andThen(level.status); return false; } }); } runGame(GAME_LEVELS, DOMDisplay); </script> </body> ~~~ An animation can be interrupted by returning `false` from the function given to `runAnimation`. It can be continued by calling `runAnimation` again. [](http://eloquentjavascript.net/15_game.html#p_m7CtgKwkDs)To communicate that the animation should be interrupted to the function passed to `runAnimation` so that it can return `false`, you can use a variable that both the event handler and that function have access to. [](http://eloquentjavascript.net/15_game.html#p_FDlbFLVUQ7)When finding a way to unregister the handlers registered by `trackKeys`, remember that the *exact* same function value that was passed to`addEventListener` must be passed to `removeEventListener` to successfully remove a handler. Thus, the `handler` function value created in `trackKeys`must be available to the code that unregisters the handlers. [](http://eloquentjavascript.net/15_game.html#p_mn/dvab7i4)You can add a property to the object returned by `trackKeys`, containing either that function value or a method that handles the unregistering directly.

## Chapter 14 # Handling Events > [](http://eloquentjavascript.net/14_event.html#p_9vGtY0kynX)You have power over your mind—not outside events. Realize this, and you will find strength. > > Marcus Aurelius, Meditations [](http://eloquentjavascript.net/14_event.html#p_/DC8VWovoV)Some programs work with direct user input, such as mouse and keyboard interaction. The timing and order of such input can’t be predicted in advance. This requires a different approach to control flow than the one we have used so far. ## [](http://eloquentjavascript.net/14_event.html#h_HQoLxG2r2l)Event handlers [](http://eloquentjavascript.net/14_event.html#p_4crjoEGjCE)Imagine an interface where the only way to find out whether a key on the keyboard is being pressed is to read the current state of that key. To be able to react to keypresses, you would have to constantly read the key’s state so that you’d catch it before it’s released again. It would be dangerous to perform other time-intensive computations since you might miss a keypress. [](http://eloquentjavascript.net/14_event.html#p_gL+2BzAZqa)That is how such input was handled on primitive machines. A step up would be for the hardware or operating system to notice the keypress and put it in a queue. A program can then periodically check the queue for new events and react to what it finds there. [](http://eloquentjavascript.net/14_event.html#p_llZbht+m+p)Of course, it has to remember to look at the queue, and to do it often, because any time between the key being pressed and the program noticing the event will cause the software to feel unresponsive. This approach is called *polling*. Most programmers avoid it whenever possible. [](http://eloquentjavascript.net/14_event.html#p_yR0Vf6qqc8)A better mechanism is for the underlying system to give our code a chance to react to events as they occur. Browsers do this by allowing us to register functions as *handlers* for specific events. ~~~ <p>Click this document to activate the handler.</p> <script> addEventListener("click", function() { console.log("You clicked!"); }); </script> ~~~ [](http://eloquentjavascript.net/14_event.html#p_MOjheGCW5Y)The `addEventListener` function registers its second argument to be called whenever the event described by its first argument occurs. ## [](http://eloquentjavascript.net/14_event.html#h_Kx1VwAV7ei)Events and DOM nodes [](http://eloquentjavascript.net/14_event.html#p_OFueY9yZeF)Each browser event handler is registered in a context. When you call`addEventListener` as shown previously, you are calling it as a method on the whole window because in the browser the global scope is equivalent to the`window` object. Every DOM element has its own `addEventListener` method, which allows you to listen specifically on that element. ~~~ <button>Click me</button> <p>No handler here.</p> <script> var button = document.querySelector("button"); button.addEventListener("click", function() { console.log("Button clicked."); }); </script> ~~~ [](http://eloquentjavascript.net/14_event.html#p_dN0xA6sZi2)The example attaches a handler to the button node. Thus, clicks on the button cause that handler to run, whereas clicks on the rest of the document do not. [](http://eloquentjavascript.net/14_event.html#p_baR6w+cVeA)Giving a node an `onclick` attribute has a similar effect. But a node has only one `onclick` attribute, so you can register only one handler per node that way. The `addEventListener` method allows you to add any number of handlers, so you can’t accidentally replace a handler that has already been registered. [](http://eloquentjavascript.net/14_event.html#p_/QIdzEWfHg)The `removeEventListener` method, called with arguments similar to as`addEventListener`, removes a handler. ~~~ <button>Act-once button</button> <script> var button = document.querySelector("button"); function once() { console.log("Done."); button.removeEventListener("click", once); } button.addEventListener("click", once); </script> ~~~ [](http://eloquentjavascript.net/14_event.html#p_Y/CadjfuHR)To be able to unregister a handler function, we give it a name (such as `once`) so that we can pass it to both `addEventListener` and `removeEventListener`. ## [](http://eloquentjavascript.net/14_event.html#h_0d6qd0WrDY)Event objects [](http://eloquentjavascript.net/14_event.html#p_xMV45J9tPM)Though we have ignored it in the previous examples, event handler functions are passed an argument: the *event object*. This object gives us additional information about the event. For example, if we want to know *which* mouse button was pressed, we can look at the event object’s `which` property. ~~~ <button>Click me any way you want</button> <script> var button = document.querySelector("button"); button.addEventListener("mousedown", function(event) { if (event.which == 1) console.log("Left button"); else if (event.which == 2) console.log("Middle button"); else if (event.which == 3) console.log("Right button"); }); </script> ~~~ [](http://eloquentjavascript.net/14_event.html#p_6Vk6va5Rnz)The information stored in an event object differs per type of event. We’ll discuss various types later in this chapter. The object’s `type` property always holds a string identifying the event (for example `"click"` or `"mousedown"`). ## [](http://eloquentjavascript.net/14_event.html#h_NEhx0cDpml)Propagation [](http://eloquentjavascript.net/14_event.html#p_AMEe+Oabfh)Event handlers registered on nodes with children will also receive some events that happen in the children. If a button inside a paragraph is clicked, event handlers on the paragraph will also receive the click event. [](http://eloquentjavascript.net/14_event.html#p_UBfEVzqY/E)But if both the paragraph and the button have a handler, the more specific handler—the one on the button—gets to go first. The event is said to *propagate*outward, from the node where it happened to that node’s parent node and on to the root of the document. Finally, after all handlers registered on a specific node have had their turn, handlers registered on the whole window get a chance to respond to the event. [](http://eloquentjavascript.net/14_event.html#p_mgXPeV1qcP)At any point, an event handler can call the `stopPropagation` method on the event object to prevent handlers “further up” from receiving the event. This can be useful when, for example, you have a button inside another clickable element and you don’t want clicks on the button to activate the outer element’s click behavior. [](http://eloquentjavascript.net/14_event.html#p_qKf3t2phih)The following example registers `"mousedown"` handlers on both a button and the paragraph around it. When clicked with the right mouse button, the handler for the button calls `stopPropagation`, which will prevent the handler on the paragraph from running. When the button is clicked with another mouse button, both handlers will run. ~~~ <p>A paragraph with a <button>button</button>.</p> <script> var para = document.querySelector("p"); var button = document.querySelector("button"); para.addEventListener("mousedown", function() { console.log("Handler for paragraph."); }); button.addEventListener("mousedown", function(event) { console.log("Handler for button."); if (event.which == 3) event.stopPropagation(); }); </script> ~~~ [](http://eloquentjavascript.net/14_event.html#p_NocTrrs2K+)Most event objects have a `target` property that refers to the node where they originated. You can use this property to ensure that you’re not accidentally handling something that propagated up from a node you do not want to handle. [](http://eloquentjavascript.net/14_event.html#p_RrFjk/g1ly)It is also possible to use the `target` property to cast a wide net for a specific type of event. For example, if you have a node containing a long list of buttons, it may be more convenient to register a single click handler on the outer node and have it use the `target` property to figure out whether a button was clicked, rather than register individual handlers on all of the buttons. ~~~ <button>A</button> <button>B</button> <button>C</button> <script> document.body.addEventListener("click", function(event) { if (event.target.nodeName == "BUTTON") console.log("Clicked", event.target.textContent); }); </script> ~~~ ## [](http://eloquentjavascript.net/14_event.html#h_GaHJsztrot)Default actions [](http://eloquentjavascript.net/14_event.html#p_mNZd3hJWtB)Many events have a default action associated with them. If you click a link, you will be taken to the link’s target. If you press the down arrow, the browser will scroll the page down. If you right-click, you’ll get a context menu. And so on. [](http://eloquentjavascript.net/14_event.html#p_QdllRyXgOw)For most types of events, the JavaScript event handlers are called *before* the default behavior is performed. If the handler doesn’t want the normal behavior to happen, typically because it has already taken care of handling the event, it can call the `preventDefault` method on the event object. [](http://eloquentjavascript.net/14_event.html#p_qt5hvvY54m)This can be used to implement your own keyboard shortcuts or context menu. It can also be used to obnoxiously interfere with the behavior that users expect. For example, here is a link that cannot be followed: ~~~ <a href="https://developer.mozilla.org/">MDN</a> <script> var link = document.querySelector("a"); link.addEventListener("click", function(event) { console.log("Nope."); event.preventDefault(); }); </script> ~~~ [](http://eloquentjavascript.net/14_event.html#p_nc872RiHdX)Try not to do such things unless you have a really good reason to. For people using your page, it can be unpleasant when the behavior they expect is broken. [](http://eloquentjavascript.net/14_event.html#p_KMgK6E70da)Depending on the browser, some events can’t be intercepted. On Chrome, for example, keyboard shortcuts to close the current tab (Ctrl-W or Command-W) cannot be handled by JavaScript. ## [](http://eloquentjavascript.net/14_event.html#h_974t15Z9oa)Key events [](http://eloquentjavascript.net/14_event.html#p_dJaRso8RxV)When a key on the keyboard is pressed, your browser fires a `"keydown"` event. When it is released, a `"keyup"` event fires. ~~~ <p>This page turns violet when you hold the V key.</p> <script> addEventListener("keydown", function(event) { if (event.keyCode == 86) document.body.style.background = "violet"; }); addEventListener("keyup", function(event) { if (event.keyCode == 86) document.body.style.background = ""; }); </script> ~~~ [](http://eloquentjavascript.net/14_event.html#p_gQnz+6p78i)Despite its name, `"keydown"` fires not only when the key is physically pushed down. When a key is pressed and held, the event fires again every time the key*repeats*. Sometimes—for example if you want to increase the acceleration of a game character when an arrow key is pressed and decrease it again when the key is released—you have to be careful not to increase it again every time the key repeats or you’d end up with unintentionally huge values. [](http://eloquentjavascript.net/14_event.html#p_YN1AGK2x1E)The previous example looked at the `keyCode` property of the event object. This is how you can identify which key is being pressed or released. Unfortunately, it’s not always obvious how to translate the numeric key code to an actual key. [](http://eloquentjavascript.net/14_event.html#p_y/Pr/lCtnH)For letter and number keys, the associated key code will be the Unicode character code associated with the (uppercase) letter or number printed on the key. The `charCodeAt` method on strings gives us a way to find this code. ~~~ console.log("Violet".charCodeAt(0)); // → 86 console.log("1".charCodeAt(0)); // → 49 ~~~ [](http://eloquentjavascript.net/14_event.html#p_HVPFsxHIiv)Other keys have less predictable key codes. The best way to find the codes you need is usually by experimenting—register a key event handler that logs the key codes it gets and press the key you are interested in. [](http://eloquentjavascript.net/14_event.html#p_4cdQPevWxW)Modifier keys such as Shift, Ctrl, Alt, and Meta (Command on Mac) generate key events just like normal keys. But when looking for key combinations, you can also find out whether these keys are held down by looking at the `shiftKey`,`ctrlKey`, `altKey`, and `metaKey` properties of keyboard and mouse events. ~~~ <p>Press Ctrl-Space to continue.</p> <script> addEventListener("keydown", function(event) { if (event.keyCode == 32 && event.ctrlKey) console.log("Continuing!"); }); </script> ~~~ [](http://eloquentjavascript.net/14_event.html#p_p/+N5nU+rr)The `"keydown"` and `"keyup"` events give you information about the physical key that is being pressed. But what if you are interested in the actual text being typed? Getting that text from key codes is awkward. Instead, there exists another event, `"keypress"`, which fires right after `"keydown"` (and repeated along with `"keydown"` when the key is held) but only for keys that produce character input. The `charCode` property in the event object contains a code that can be interpreted as a Unicode character code. We can use the`String.fromCharCode` function to turn this code into an actual single-character string. ~~~ <p>Focus this page and type something.</p> <script> addEventListener("keypress", function(event) { console.log(String.fromCharCode(event.charCode)); }); </script> ~~~ [](http://eloquentjavascript.net/14_event.html#p_EHGULml9MQ)The DOM node where a key event originates depends on the element that has focus when the key is pressed. Normal nodes cannot have focus (unless you give them a `tabindex` attribute), but things such as links, buttons, and form fields can. We’ll come back to form fields in [Chapter 18](http://eloquentjavascript.net/18_forms.html#forms). When nothing in particular has focus, `document.body` acts as the target node of key events. ## [](http://eloquentjavascript.net/14_event.html#h_D5iwImkmyt)Mouse clicks [](http://eloquentjavascript.net/14_event.html#p_zZHDVhEhYY)Pressing a mouse button also causes a number of events to fire. The`"mousedown"` and `"mouseup"` events are similar to `"keydown"` and `"keyup"`and fire when the button is pressed and released. These will happen on the DOM nodes that are immediately below the mouse pointer when the event occurs. [](http://eloquentjavascript.net/14_event.html#p_vxFLT2fw8e)After the `"mouseup"` event, a `"click"` event fires on the most specific node that contained both the press and the release of the button. For example, if I press down the mouse button on one paragraph and then move the pointer to another paragraph and release the button, the `"click"` event will happen on the element that contains both those paragraphs. [](http://eloquentjavascript.net/14_event.html#p_gzmmLlVcMF)If two clicks happen close together, a `"dblclick"` (double-click) event also fires, after the second click event. [](http://eloquentjavascript.net/14_event.html#p_V3QuKrAk2Z)To get precise information about the place where a mouse event happened, you can look at its `pageX` and `pageY` properties, which contain the event’s coordinates (in pixels) relative to the top-left corner of the document. [](http://eloquentjavascript.net/14_event.html#p_A7YDC3hfu1)The following implements a primitive drawing program. Every time you click the document, it adds a dot under your mouse pointer. See [Chapter 19](http://eloquentjavascript.net/19_paint.html#paint) for a less primitive drawing program. ~~~ <style> body { height: 200px; background: beige; } .dot { height: 8px; width: 8px; border-radius: 4px; /* rounds corners */ background: blue; position: absolute; } </style> <script> addEventListener("click", function(event) { var dot = document.createElement("div"); dot.className = "dot"; dot.style.left = (event.pageX - 4) + "px"; dot.style.top = (event.pageY - 4) + "px"; document.body.appendChild(dot); }); </script> ~~~ [](http://eloquentjavascript.net/14_event.html#p_qAqiJ6l3wp)The `clientX` and `clientY` properties are similar to `pageX` and `pageY` but relative to the part of the document that is currently scrolled into view. These can be useful when comparing mouse coordinates with the coordinates returned by `getBoundingClientRect`, which also returns viewport-relative coordinates. ## [](http://eloquentjavascript.net/14_event.html#h_XojjiOmg7v)Mouse motion [](http://eloquentjavascript.net/14_event.html#p_Yjy/EtP24n)Every time the mouse pointer moves, a `"mousemove"` event fires. This event can be used to track the position of the mouse. A common situation in which this is useful is when implementing some form of mouse-dragging functionality. [](http://eloquentjavascript.net/14_event.html#p_Z1h4BQYT0/)As an example, the following program displays a bar and sets up event handlers so that dragging to the left or right on this bar makes it narrower or wider: ~~~ <p>Drag the bar to change its width:</p> <div style="background: orange; width: 60px; height: 20px"> </div> <script> var lastX; // Tracks the last observed mouse X position var rect = document.querySelector("div"); rect.addEventListener("mousedown", function(event) { if (event.which == 1) { lastX = event.pageX; addEventListener("mousemove", moved); event.preventDefault(); // Prevent selection } }); function buttonPressed(event) { if (event.buttons == null) return event.which != 0; else return event.buttons != 0; } function moved(event) { if (!buttonPressed(event)) { removeEventListener("mousemove", moved); } else { var dist = event.pageX - lastX; var newWidth = Math.max(10, rect.offsetWidth + dist); rect.style.width = newWidth + "px"; lastX = event.pageX; } } </script> ~~~ [](http://eloquentjavascript.net/14_event.html#p_OkAFTS9SkH)Note that the `"mousemove"` handler is registered on the whole window. Even if the mouse goes outside of the bar during resizing, we still want to update its size and stop dragging when the mouse is released. [](http://eloquentjavascript.net/14_event.html#p_fndkFYbayW)We must stop resizing the bar when the mouse button is released. Unfortunately, not all browsers give `"mousemove"` events a meaningful `which`property. There is a standard property called `buttons`, which provides similar information, but that is also not supported on all browsers. Fortunately, all major browsers support either `buttons` or `which`, so the `buttonPressed`function in the example first tries `buttons`, and falls back to `which` when that isn’t available. [](http://eloquentjavascript.net/14_event.html#p_H543iFvHsm)Whenever the mouse pointer enters or leaves a node, a `"mouseover"` or`"mouseout"` event fires. These two events can be used, among other things, to create hover effects, showing or styling something when the mouse is over a given element. [](http://eloquentjavascript.net/14_event.html#p_5IESlahWDS)Unfortunately, creating such an effect is not as simple as starting the effect on`"mouseover"` and ending it on `"mouseout"`. When the mouse moves from a node onto one of its children, `"mouseout"` fires on the parent node, though the mouse did not actually leave the node’s extent. To make things worse, these events propagate just like other events, and thus you will also receive`"mouseout"` events when the mouse leaves one of the child nodes of the node on which the handler is registered. [](http://eloquentjavascript.net/14_event.html#p_WkAAgbH7zo)To work around this problem, we can use the `relatedTarget` property of the event objects created for these events. It tells us, in the case of `"mouseover"`, what element the pointer was over before and, in the case of `"mouseout"`, what element it is going to. We want to change our hover effect only when the`relatedTarget` is outside of our target node. Only in that case does this event actually represent a *crossing over* from outside to inside the node (or the other way around). ~~~ <p>Hover over this <strong>paragraph</strong>.</p> <script> var para = document.querySelector("p"); function isInside(node, target) { for (; node != null; node = node.parentNode) if (node == target) return true; } para.addEventListener("mouseover", function(event) { if (!isInside(event.relatedTarget, para)) para.style.color = "red"; }); para.addEventListener("mouseout", function(event) { if (!isInside(event.relatedTarget, para)) para.style.color = ""; }); </script> ~~~ [](http://eloquentjavascript.net/14_event.html#p_QTE0dpHOFx)The `isInside` function follows the given node’s parent links until it either reaches the top of the document (when `node` becomes null) or finds the parent we are looking for. [](http://eloquentjavascript.net/14_event.html#p_ujj+9rrIE3)I should add that a hover effect like this can be much more easily achieved using the CSS *pseudoselector* `:hover`, as the next example shows. But when your hover effect involves doing something more complicated than changing a style on the target node, you must use the trick with `"mouseover"` and`"mouseout"` events. ~~~ <style> p:hover { color: red } </style> <p>Hover over this <strong>paragraph</strong>.</p> ~~~ ## [](http://eloquentjavascript.net/14_event.html#h_xGSp7W5DAZ)Scroll events [](http://eloquentjavascript.net/14_event.html#p_QhFs2TfUWV)Whenever an element is scrolled, a `"scroll"` event fires on it. This has various uses, such as knowing what the user is currently looking at (for disabling off-screen animations or sending spy reports to your evil headquarters) or showing some indication of progress (by highlighting part of a table of contents or showing a page number). [](http://eloquentjavascript.net/14_event.html#p_koAfRfBrN2)The following example draws a progress bar in the top-right corner of the document and updates it to fill up as you scroll down: ~~~ <style> .progress { border: 1px solid blue; width: 100px; position: fixed; top: 10px; right: 10px; } .progress > div { height: 12px; background: blue; width: 0%; } body { height: 2000px; } </style> <div class="progress"><div></div></div> <p>Scroll me...</p> <script> var bar = document.querySelector(".progress div"); addEventListener("scroll", function() { var max = document.body.scrollHeight - innerHeight; var percent = (pageYOffset / max) * 100; bar.style.width = percent + "%"; }); </script> ~~~ [](http://eloquentjavascript.net/14_event.html#p_J92IjzNbC5)Giving an element a `position` of `fixed` acts much like an `absolute` position but also prevents it from scrolling along with the rest of the document. The effect is to make our progress bar stay in its corner. Inside it is another element, which is resized to indicate the current progress. We use `%`, rather than `px`, as a unit when setting the width so that the element is sized relative to the whole bar. [](http://eloquentjavascript.net/14_event.html#p_Q5ZDL6Y/Rx)The global `innerHeight` variable gives us the height of the window, which we have to subtract from the total scrollable height—you can’t keep scrolling when you hit the bottom of the document. (There’s also an `innerWidth` to go along with `innerHeight`.) By dividing `pageYOffset`, the current scroll position, by the maximum scroll position and multiplying by 100, we get the percentage for the progress bar. [](http://eloquentjavascript.net/14_event.html#p_0tTa0Fp6S+)Calling `preventDefault` on a scroll event does not prevent the scrolling from happening. In fact, the event handler is called only *after* the scrolling takes place. ## [](http://eloquentjavascript.net/14_event.html#h_NoKd+BgJRm)Focus events [](http://eloquentjavascript.net/14_event.html#p_r6SF152VBY)When an element gains focus, the browser fires a `"focus"` event on it. When it loses focus, a `"blur"` event fires. [](http://eloquentjavascript.net/14_event.html#p_rU6XEBvIwF)Unlike the events discussed earlier, these two events do not propagate. A handler on a parent element is not notified when a child element gains or loses focus. [](http://eloquentjavascript.net/14_event.html#p_XbY3zpvUvn)The following example displays help text for the text field that currently has focus: ~~~ <p>Name: <input type="text" data-help="Your full name"></p> <p>Age: <input type="text" data-help="Age in years"></p> <p id="help"></p> <script> var help = document.querySelector("#help"); var fields = document.querySelectorAll("input"); for (var i = 0; i < fields.length; i++) { fields[i].addEventListener("focus", function(event) { var text = event.target.getAttribute("data-help"); help.textContent = text; }); fields[i].addEventListener("blur", function(event) { help.textContent = ""; }); } </script> ~~~ [](http://eloquentjavascript.net/14_event.html#p_5HyY0t9dpU)The window object will receive `"focus"` and `"blur"` events when the user moves from or to the browser tab or window in which the document is shown. ## [](http://eloquentjavascript.net/14_event.html#h_NmV8RP8lpt)Load event [](http://eloquentjavascript.net/14_event.html#p_NarSNA3PKB)When a page finishes loading, the `"load"` event fires on the window and the document body objects. This is often used to schedule initialization actions that require the whole document to have been built. Remember that the content of`<script>` tags is run immediately when the tag is encountered. This is often too soon, such as when the script needs to do something with parts of the document that appear after the `<script>` tag. [](http://eloquentjavascript.net/14_event.html#p_hCgodGiFEt)Elements such as images and script tags that load an external file also have a`"load"` event that indicates the files they reference were loaded. Like the focus-related events, loading events do not propagate. [](http://eloquentjavascript.net/14_event.html#p_nu8/BUQa7r)When a page is closed or navigated away from (for example by following a link), a `"beforeunload"` event fires. The main use of this event is to prevent the user from accidentally losing work by closing a document. Preventing the page from unloading is not, as you might expect, done with the`preventDefault` method. Instead, it is done by returning a string from the handler. The string will be used in a dialog that asks the user if they want to stay on the page or leave it. This mechanism ensures that a user is able to leave the page, even if it is running a malicious script that would prefer to keep them there forever in order to force them to look at dodgy weight loss ads. ## [](http://eloquentjavascript.net/14_event.html#h_cj44kRfk/h)Script execution timeline [](http://eloquentjavascript.net/14_event.html#p_m898wVhAww)There are various things that can cause a script to start executing. Reading a`<script>` tag is one such thing. An event firing is another. [Chapter 13](http://eloquentjavascript.net/13_dom.html#animationFrame)discussed the `requestAnimationFrame` function, which schedules a function to be called before the next page redraw. That is yet another way in which a script can start running. [](http://eloquentjavascript.net/14_event.html#p_3Ya6RuV22D)It is important to understand that even though events can fire at any time, no two scripts in a single document ever run at the same moment. If a script is already running, event handlers and pieces of code scheduled in other ways have to wait for their turn. This is the reason why a document will freeze when a script runs for a long time. The browser cannot react to clicks and other events inside the document because it can’t run event handlers until the current script finishes running. [](http://eloquentjavascript.net/14_event.html#p_qOcAjNuS8p)Some programming environments do allow multiple *threads of execution* to run at the same time. Doing multiple things at the same time can be used to make a program faster. But when you have multiple actors touching the same parts of the system at the same time, thinking about a program becomes at least an order of magnitude harder. [](http://eloquentjavascript.net/14_event.html#p_omeCxjcdIQ)The fact that JavaScript programs do only one thing at a time makes our lives easier. For cases where you *really* do want to do some time-consuming thing in the background without freezing the page, browsers provide something called*web workers*. A worker is an isolated JavaScript environment that runs alongside the main program for a document and can communicate with it only by sending and receiving messages. [](http://eloquentjavascript.net/14_event.html#p_dXLk5WXFTx)Assume we have the following code in a file called `code/squareworker.js`: ~~~ addEventListener("message", function(event) { postMessage(event.data * event.data); }); ~~~ [](http://eloquentjavascript.net/14_event.html#p_3bETV2ub20)Imagine that squaring a number is a heavy, long-running computation that we want to perform in a background thread. This code spawns a worker, sends it a few messages, and outputs the responses. ~~~ var squareWorker = new Worker("code/squareworker.js"); squareWorker.addEventListener("message", function(event) { console.log("The worker responded:", event.data); }); squareWorker.postMessage(10); squareWorker.postMessage(24); ~~~ [](http://eloquentjavascript.net/14_event.html#p_Stp8nkaZHV)The `postMessage` function sends a message, which will cause a `"message"`event to fire in the receiver. The script that created the worker sends and receives messages through the `Worker` object, whereas the worker talks to the script that created it by sending and listening directly on its global scope—which is a *new* global scope, not shared with the original script. ## [](http://eloquentjavascript.net/14_event.html#h_WCvdQCKgu8)Setting timers [](http://eloquentjavascript.net/14_event.html#p_5YmF46Q06c)The `setTimeout` function is similar to `requestAnimationFrame`. It schedules another function to be called later. But instead of calling the function at the next redraw, it waits for a given amount of milliseconds. This page turns from blue to yellow after two seconds: ~~~ <script> document.body.style.background = "blue"; setTimeout(function() { document.body.style.background = "yellow"; }, 2000); </script> ~~~ [](http://eloquentjavascript.net/14_event.html#p_de3VjAL+lV)Sometimes you need to cancel a function you have scheduled. This is done by storing the value returned by `setTimeout` and calling `clearTimeout` on it. ~~~ var bombTimer = setTimeout(function() { console.log("BOOM!"); }, 500); if (Math.random() < 0.5) { // 50% chance console.log("Defused."); clearTimeout(bombTimer); } ~~~ [](http://eloquentjavascript.net/14_event.html#p_c/F+71/NPg)The `cancelAnimationFrame` function works in the same way as`clearTimeout`—calling it on a value returned by `requestAnimationFrame`will cancel that frame (assuming it hasn’t already been called). [](http://eloquentjavascript.net/14_event.html#p_MTqUUpmFIN)A similar set of functions, `setInterval` and `clearInterval` are used to set timers that should repeat every *X* milliseconds. ~~~ var ticks = 0; var clock = setInterval(function() { console.log("tick", ticks++); if (ticks == 10) { clearInterval(clock); console.log("stop."); } }, 200); ~~~ ## [](http://eloquentjavascript.net/14_event.html#h_AOVmaqj10I)Debouncing [](http://eloquentjavascript.net/14_event.html#p_I3Osg2AlyQ)Some types of events have the potential to fire rapidly, many times in a row (the `"mousemove"` and `"scroll"` events, for example). When handling such events, you must be careful not to do anything too time-consuming or your handler will take up so much time that interaction with the document starts to feel slow and choppy. [](http://eloquentjavascript.net/14_event.html#p_CvRYtJPZwz)If you do need to do something nontrivial in such a handler, you can use`setTimeout` to make sure you are not doing it too often. This is usually called*debouncing* the event. There are several slightly different approaches to this. [](http://eloquentjavascript.net/14_event.html#p_8qDwbjQg0l)In the first example, we want to do something when the user has typed something, but we don’t want to do it immediately for every key event. When they are typing quickly, we just want to wait until a pause occurs. Instead of immediately performing an action in the event handler, we set a timeout instead. We also clear the previous timeout (if any) so that when events occur close together (closer than our timeout delay), the timeout from the previous event will be canceled. ~~~ <textarea>Type something here...</textarea> <script> var textarea = document.querySelector("textarea"); var timeout; textarea.addEventListener("keydown", function() { clearTimeout(timeout); timeout = setTimeout(function() { console.log("You stopped typing."); }, 500); }); </script> ~~~ [](http://eloquentjavascript.net/14_event.html#p_55az3iLtsc)Giving an undefined value to `clearTimeout` or calling it on a timeout that has already fired has no effect. Thus, we don’t have to be careful about when to call it, and we simply do so for every event. [](http://eloquentjavascript.net/14_event.html#p_GisH+i+4tv)We can use a slightly different pattern if we want to space responses so that they’re separated by at least a certain length of time but want to fire them*during* a series of events, not just afterward. For example, we might want to respond to `"mousemove"` events by showing the current coordinates of the mouse, but only every 250 milliseconds. ~~~ <script> function displayCoords(event) { document.body.textContent = "Mouse at " + event.pageX + ", " + event.pageY; } var scheduled = false, lastEvent; addEventListener("mousemove", function(event) { lastEvent = event; if (!scheduled) { scheduled = true; setTimeout(function() { scheduled = false; displayCoords(lastEvent); }, 250); } }); </script> ~~~ ## [](http://eloquentjavascript.net/14_event.html#h_ErccPg/l98)Summary [](http://eloquentjavascript.net/14_event.html#p_bKLVanKSm7)Event handlers make it possible to detect and react to events we have no direct control over. The `addEventListener` method is used to register such a handler. [](http://eloquentjavascript.net/14_event.html#p_pT/A7mXxlh)Each event has a type (`"keydown"`, `"focus"`, and so on) that identifies it. Most events are called on a specific DOM element and then *propagate* to that element’s ancestors, allowing handlers associated with those elements to handle them. [](http://eloquentjavascript.net/14_event.html#p_jK5bk5CuMw)When an event handler is called, it is passed an event object with additional information about the event. This object also has methods that allow us to stop further propagation (`stopPropagation`) and prevent the browser’s default handling of the event (`preventDefault`). [](http://eloquentjavascript.net/14_event.html#p_8R4tiRRH3a)Pressing a key fires `"keydown"`, `"keypress"`, and `"keyup"` events. Pressing a mouse button fires `"mousedown"`, `"mouseup"`, and `"click"` events. Moving the mouse fires `"mousemove"` and possibly `"mouseenter"` and `"mouseout"`events. [](http://eloquentjavascript.net/14_event.html#p_zwo0vgzm7O)Scrolling can be detected with the `"scroll"` event, and focus changes can be detected with the `"focus"` and `"blur"` events. When the document finishes loading, a `"load"` event fires on the window. [](http://eloquentjavascript.net/14_event.html#p_0zJSMPD1MF)Only one piece of JavaScript program can run at a time. Thus, event handlers and other scheduled scripts have to wait until other scripts finish before they get their turn. ## [](http://eloquentjavascript.net/14_event.html#h_TcUD2vzyMe)Exercises ### [](http://eloquentjavascript.net/14_event.html#h_Lhbs8f7VGD)Censored keyboard [](http://eloquentjavascript.net/14_event.html#p_EvVPidig6g)Between 1928 and 2013, Turkish law forbade the use of the letters *Q*, *W*, and *X*in official documents. This was part of a wider initiative to stifle Kurdish culture—those letters occur in the language used by Kurdish people but not in Istanbul Turkish. [](http://eloquentjavascript.net/14_event.html#p_TZAfuJye+C)As an exercise in doing ridiculous things with technology, I’m asking you to program a text field (an `<input type="text">` tag) that these letters cannot be typed into. [](http://eloquentjavascript.net/14_event.html#p_cs0DibmEsI)(Do not worry about copy and paste and other such loopholes.) ~~~ <input type="text"> <script> var field = document.querySelector("input"); // Your code here. </script> ~~~ ### [](http://eloquentjavascript.net/14_event.html#h_NOgRH0Y9st)Mouse trail [](http://eloquentjavascript.net/14_event.html#p_j4+NSbR+hs)In JavaScript’s early days, which was the high time of gaudy home pages with lots of animated images, people came up with some truly inspiring ways to use the language. [](http://eloquentjavascript.net/14_event.html#p_ZseDCwl6/C)One of these was the “mouse trail”—a series of images that would follow the mouse pointer as you moved it across the page. [](http://eloquentjavascript.net/14_event.html#p_0yy1fiF5Gm)In this exercise, I want you to implement a mouse trail. Use absolutely positioned `<div>` elements with a fixed size and background color (refer to the[code](http://eloquentjavascript.net/14_event.html#mouse_drawing) in the “Mouse Clicks” section for an example). Create a bunch of such elements and, when the mouse moves, display them in the wake of the mouse pointer. [](http://eloquentjavascript.net/14_event.html#p_5jL8mAUnIQ)There are various possible approaches here. You can make your solution as simple or as complex as you want. A simple solution to start with is to keep a fixed number of trail elements and cycle through them, moving the next one to the mouse’s current position every time a `"mousemove"` event occurs. ~~~ <style> .trail { /* className for the trail elements */ position: absolute; height: 6px; width: 6px; border-radius: 3px; background: teal; } body { height: 300px; } </style> <script> // Your code here. </script> ~~~ ### [](http://eloquentjavascript.net/14_event.html#h_Kk1WKx2anJ)Tabs [](http://eloquentjavascript.net/14_event.html#p_Ktj5JZEPdu)A tabbed interface is a common design pattern. It allows you to select an interface panel by choosing from a number of tabs “sticking out” above an element. [](http://eloquentjavascript.net/14_event.html#p_rh5X8kUE8g)In this exercise you’ll implement a simple tabbed interface. Write a function,`asTabs`, that takes a DOM node and creates a tabbed interface showing the child elements of that node. It should insert a list of `<button>` elements at the top of the node, one for each child element, containing text retrieved from the`data-tabname` attribute of the child. All but one of the original children should be hidden (given a `display` style of `none`), and the currently visible node can be selected by clicking the buttons. [](http://eloquentjavascript.net/14_event.html#p_744xb2mKFr)When it works, extend it to also style the currently active button differently. ~~~ <div id="wrapper"> <div data-tabname="one">Tab one</div> <div data-tabname="two">Tab two</div> <div data-tabname="three">Tab three</div> </div> <script> function asTabs(node) { // Your code here. } asTabs(document.querySelector("#wrapper")); </script> ~~~ > One pitfall you’ll probably run into is that you can’t directly use the node’s childNodes property as a collection of tab nodes. For one thing, when you add the buttons, they will also become child nodes and end up in this object because it is live. For another, the text nodes created for the whitespace between the nodes are also in there and should not get their own tabs. > To work around this, start by building up a real array of all the children in the wrapper that have a nodeType of 1. > When registering event handlers on the buttons, the handler functions will need to know which tab element is associated with the button. If they are created in a normal loop, you can access the loop index variable from inside the function, but it won’t give you the correct number because that variable will have been further changed by the loop. > A simple workaround is to use the forEach method and create the handler functions from inside the function passed to forEach. The loop index, which is passed as a second argument to that function, will be a normal local variable there and won’t be overwritten by further iterations.

## Chapter 13 # The Document Object Model [](http://eloquentjavascript.net/13_dom.html#p_O255K9+8+X)When you open a web page in your browser, the browser retrieves the page’s HTML text and parses it, much like the way our parser from [Chapter 11](http://eloquentjavascript.net/11_language.html#parsing) parsed programs. The browser builds up a model of the document’s structure and then uses this model to draw the page on the screen. [](http://eloquentjavascript.net/13_dom.html#p_NsgEPDmO0h)This representation of the document is one of the toys that a JavaScript program has available in its sandbox. You can read from the model and also change it. It acts as a *live* data structure: when it is modified, the page on the screen is updated to reflect the changes. ## [](http://eloquentjavascript.net/13_dom.html#h_XJzHjmX32m)Document structure [](http://eloquentjavascript.net/13_dom.html#p_FuW098VAX5)You can imagine an HTML document as a nested set of boxes. Tags such as`<body>` and `</body>` enclose other tags, which in turn contain other tags or text. Here’s the example document from the [previous chapter](http://eloquentjavascript.net/12_browser.html#browser): ~~~ <!doctype html> <html> <head> <title>My home page</title> </head> <body> <h1>My home page</h1> <p>Hello, I am Marijn and this is my home page.</p> <p>I also wrote a book! Read it <a href="http://eloquentjavascript.net">here</a>.</p> </body> </html> ~~~ [](http://eloquentjavascript.net/13_dom.html#p_rRaTGW2wtE)This page has the following structure:  [](http://eloquentjavascript.net/13_dom.html#p_1mgE45KfoF)The data structure the browser uses to represent the document follows this shape. For each box, there is an object, which we can interact with to find out things such as what HTML tag it represents and which boxes and text it contains. This representation is called the *Document Object Model*, or DOM for short. [](http://eloquentjavascript.net/13_dom.html#p_4H8fF9K7jG)The global variable `document` gives us access to these objects. Its`documentElement` property refers to the object representing the `<html>` tag. It also provides the properties `head` and `body`, which hold the objects for those elements. ## [](http://eloquentjavascript.net/13_dom.html#h_HnCB1zb0Ot)Trees [](http://eloquentjavascript.net/13_dom.html#p_63ES4U9bHr)Think back to the syntax trees from [Chapter 11](http://eloquentjavascript.net/11_language.html#parsing) for a moment. Their structures are strikingly similar to the structure of a browser’s document. Each *node* may refer to other nodes, *children*, which in turn may have their own children. This shape is typical of nested structures where elements can contain sub-elements that are similar to themselves. [](http://eloquentjavascript.net/13_dom.html#p_QVBp4KpREl)We call a data structure a *tree* when it has a branching structure, has no cycles (a node may not contain itself, directly or indirectly), and has a single, well-defined “root”. In the case of the DOM, `document.documentElement` serves as the root. [](http://eloquentjavascript.net/13_dom.html#p_UKoOzv3djs)Trees come up a lot in computer science. In addition to representing recursive structures such as HTML documents or programs, they are often used to maintain sorted sets of data because elements can usually be found or inserted more efficiently in a sorted tree than in a sorted flat array. [](http://eloquentjavascript.net/13_dom.html#p_bH4un2TNOK)A typical tree has different kinds of nodes. The syntax tree for [the Egg language](http://eloquentjavascript.net/11_language.html#language)had variables, values, and application nodes. Application nodes always have children, whereas variables and values are *leaves*, or nodes without children. [](http://eloquentjavascript.net/13_dom.html#p_zV2tKpfume)The same goes for the DOM. Nodes for regular *elements*, which represent HTML tags, determine the structure of the document. These can have child nodes. An example of such a node is `document.body`. Some of these children can be leaf nodes, such as pieces of text or comments (comments are written between `<!--` and `-->` in HTML). [](http://eloquentjavascript.net/13_dom.html#p_wUIvlqSzVD)Each DOM node object has a `nodeType` property, which contains a numeric code that identifies the type of node. Regular elements have the value 1, which is also defined as the constant property `document.ELEMENT_NODE`. Text nodes, representing a section of text in the document, have the value 3 (`document.TEXT_NODE`). Comments have the value 8 (`document.COMMENT_NODE`). [](http://eloquentjavascript.net/13_dom.html#p_9JUz0zH2O2)So another way to visualize our document tree is as follows:  [](http://eloquentjavascript.net/13_dom.html#p_CSpbmWiAKq)The leaves are text nodes, and the arrows indicate parent-child relationships between nodes. ## [](http://eloquentjavascript.net/13_dom.html#h_XgjABY6Ugx)The standard [](http://eloquentjavascript.net/13_dom.html#p_abbuMja9XR)Using cryptic numeric codes to represent node types is not a very JavaScript-like thing to do. Later in this chapter, we’ll see that other parts of the DOM interface also feel cumbersome and alien. The reason for this is that the DOM wasn’t designed for just JavaScript. Rather, it tries to define a language-neutral interface that can be used in other systems as well—not just HTML but also XML, which is a generic data format with an HTML-like syntax. [](http://eloquentjavascript.net/13_dom.html#p_aVhfBj+xLx)This is unfortunate. Standards are often useful. But in this case, the advantage (cross-language consistency) isn’t all that compelling. Having an interface that is properly integrated with the language you are using will save you more time than having a familiar interface across languages. [](http://eloquentjavascript.net/13_dom.html#p_ppPK5iRO6h)As an example of such poor integration, consider the `childNodes` property that element nodes in the DOM have. This property holds an array-like object, with a `length` property and properties labeled by numbers to access the child nodes. But it is an instance of the `NodeList` type, not a real array, so it does not have methods such as `slice` and `forEach`. [](http://eloquentjavascript.net/13_dom.html#p_NYvzy6J64s)Then there are issues that are simply poor design. For example, there is no way to create a new node and immediately add children or attributes to it. Instead, you have to first create it, then add the children one by one, and finally set the attributes one by one, using side effects. Code that interacts heavily with the DOM tends to get long, repetitive, and ugly. [](http://eloquentjavascript.net/13_dom.html#p_1dwhmrGwQQ)But these flaws aren’t fatal. Since JavaScript allows us to create our own abstractions, it is easy to write some helper functions that allow you to express the operations you are performing in a clearer and shorter way. In fact, many libraries intended for browser programming come with such tools. ## [](http://eloquentjavascript.net/13_dom.html#h_ShZPVipWw/)Moving through the tree [](http://eloquentjavascript.net/13_dom.html#p_KU+aLLXhA0)DOM nodes contain a wealth of links to other nearby nodes. The following diagram illustrates these:  [](http://eloquentjavascript.net/13_dom.html#p_oWR8F5E2Yw)Although the diagram shows only one link of each type, every node has a`parentNode` property that points to its containing node. Likewise, every element node (node type 1) has a `childNodes` property that points to an array-like object holding its children. [](http://eloquentjavascript.net/13_dom.html#p_zpX7kgIV1h)In theory, you could move anywhere in the tree using just these parent and child links. But JavaScript also gives you access to a number of additional convenience links. The `firstChild` and `lastChild` properties point to the first and last child elements or have the value `null` for nodes without children. Similarly, `previousSibling` and `nextSibling` point to adjacent nodes, which are nodes with the same parent that appear immediately before or after the node itself. For a first child, `previousSibling` will be null, and for a last child,`nextSibling` will be null. [](http://eloquentjavascript.net/13_dom.html#p_wSmPJ/d3c4)When dealing with a nested data structure like this one, recursive functions are often useful. The following recursive function scans a document for text nodes containing a given string and returns `true` when it has found one: ~~~ function talksAbout(node, string) { if (node.nodeType == document.ELEMENT_NODE) { for (var i = 0; i < node.childNodes.length; i++) { if (talksAbout(node.childNodes[i], string)) return true; } return false; } else if (node.nodeType == document.TEXT_NODE) { return node.nodeValue.indexOf(string) > -1; } } console.log(talksAbout(document.body, "book")); // → true ~~~ [](http://eloquentjavascript.net/13_dom.html#p_hpFcZy3sUo)The `nodeValue` property of a text node refers to the string of text that it represents. ## [](http://eloquentjavascript.net/13_dom.html#h_jS5BEpmLY0)Finding elements [](http://eloquentjavascript.net/13_dom.html#p_saBiuECTgW)Navigating these links among parents, children, and siblings is often useful, as in the previous function, which runs through the whole document. But if we want to find a specific node in the document, reaching it by starting at`document.body` and blindly following a hard-coded path of links is a bad idea. Doing so bakes assumptions into our program about the precise structure of the document—a structure we might want to change later. Another complicating factor is that text nodes are created even for the whitespace between nodes. The example document’s body tag does not have just three children (`<h1>` and two `<p>` elements) but actually has seven: those three, plus the spaces before, after, and between them. [](http://eloquentjavascript.net/13_dom.html#p_St5y6wbhGX)So if we want to get the `href` attribute of the link in that document, we don’t want to say something like “Get the second child of the sixth child of the document body”. It’d be better if we could say “Get the first link in the document”. And we can. ~~~ var link = document.body.getElementsByTagName("a")[0]; console.log(link.href); ~~~ [](http://eloquentjavascript.net/13_dom.html#p_VGWClqmGZz)All element nodes have a `getElementsByTagName` method, which collects all elements with the given tag name that are descendants (direct or indirect children) of the given node and returns them as an array-like object. [](http://eloquentjavascript.net/13_dom.html#p_m0eE7Kg38T)To find a specific *single* node, you can give it an `id` attribute and use`document.getElementById` instead. ~~~ <p>My ostrich Gertrude:</p> <p><img id="gertrude" src="img/ostrich.png"></p> <script> var ostrich = document.getElementById("gertrude"); console.log(ostrich.src); </script> ~~~ [](http://eloquentjavascript.net/13_dom.html#p_jbdzUTyWj8)A third, similar method is `getElementsByClassName`, which, like`getElementsByTagName`, searches through the contents of an element node and retrieves all elements that have the given string in their `class` attribute. ## [](http://eloquentjavascript.net/13_dom.html#h_npiFAJENvT)Changing the document [](http://eloquentjavascript.net/13_dom.html#p_PqYkGnE2Ps)Almost everything about the DOM data structure can be changed. Element nodes have a number of methods that can be used to change their content. The`removeChild` method removes the given child node from the document. To add a child, we can use `appendChild`, which puts it at the end of the list of children, or `insertBefore`, which inserts the node given as the first argument before the node given as the second argument. ~~~ <p>One</p> <p>Two</p> <p>Three</p> <script> var paragraphs = document.body.getElementsByTagName("p"); document.body.insertBefore(paragraphs[2], paragraphs[0]); </script> ~~~ [](http://eloquentjavascript.net/13_dom.html#p_izAyoVA99z)A node can exist in the document in only one place. Thus, inserting paragraph “Three” in front of paragraph “One” will first remove it from the end of the document and then insert it at the front, resulting in “Three/One/Two”. All operations that insert a node somewhere will, as a side effect, cause it to be removed from its current position (if it has one). [](http://eloquentjavascript.net/13_dom.html#p_IOhnyZXeWs)The `replaceChild` method is used to replace a child node with another one. It takes as arguments two nodes: a new node and the node to be replaced. The replaced node must be a child of the element the method is called on. Note that both `replaceChild` and `insertBefore` expect the *new* node as their first argument. ## [](http://eloquentjavascript.net/13_dom.html#h_AlX6HES+2D)Creating nodes [](http://eloquentjavascript.net/13_dom.html#p_EwUe6UWMca)In the following example, we want to write a script that replaces all images (`<img>` tags) in the document with the text held in their `alt` attributes, which specifies an alternative textual representation of the image. [](http://eloquentjavascript.net/13_dom.html#p_HJD0rLoFR3)This involves not only removing the images but adding a new text node to replace them. For this, we use the `document.createTextNode` method. ~~~ <p>The <img src="img/cat.png" alt="Cat"> in the <img src="img/hat.png" alt="Hat">.</p> <p><button onclick="replaceImages()">Replace</button></p> <script> function replaceImages() { var images = document.body.getElementsByTagName("img"); for (var i = images.length - 1; i >= 0; i--) { var image = images[i]; if (image.alt) { var text = document.createTextNode(image.alt); image.parentNode.replaceChild(text, image); } } } </script> ~~~ [](http://eloquentjavascript.net/13_dom.html#p_SZGvV3YwdR)Given a string, `createTextNode` gives us a type 3 DOM node (a text node), which we can insert into the document to make it show up on the screen. [](http://eloquentjavascript.net/13_dom.html#p_6KP6Yw4ww7)The loop that goes over the images starts at the end of the list of nodes. This is necessary because the node list returned by a method like`getElementsByTagName` (or a property like `childNodes`) is *live*. That is, it is updated as the document changes. If we started from the front, removing the first image would cause the list to lose its first element so that the second time the loop repeats, where `i` is 1, it would stop because the length of the collection is now also 1. [](http://eloquentjavascript.net/13_dom.html#p_KIHvSOXCAK)If you want a *solid* collection of nodes, as opposed to a live one, you can convert the collection to a real array by calling the array `slice` method on it. ~~~ var arrayish = {0: "one", 1: "two", length: 2}; var real = Array.prototype.slice.call(arrayish, 0); real.forEach(function(elt) { console.log(elt); }); // → one // two ~~~ [](http://eloquentjavascript.net/13_dom.html#p_s5ww9M09mP)To create regular element nodes (type 1), you can use the`document.createElement` method. This method takes a tag name and returns a new empty node of the given type. [](http://eloquentjavascript.net/13_dom.html#p_bvldRsjLfM)The following example defines a utility `elt`, which creates an element node and treats the rest of its arguments as children to that node. This function is then used to add a simple attribution to a quote. ~~~ <blockquote id="quote"> No book can ever be finished. While working on it we learn just enough to find it immature the moment we turn away from it. </blockquote> <script> function elt(type) { var node = document.createElement(type); for (var i = 1; i < arguments.length; i++) { var child = arguments[i]; if (typeof child == "string") child = document.createTextNode(child); node.appendChild(child); } return node; } document.getElementById("quote").appendChild( elt("footer", "—", elt("strong", "Karl Popper"), ", preface to the second editon of ", elt("em", "The Open Society and Its Enemies"), ", 1950")); </script> ~~~ ## [](http://eloquentjavascript.net/13_dom.html#h_pmUmF/LHme)Attributes [](http://eloquentjavascript.net/13_dom.html#p_HSPdyuAruy)Some element attributes, such as `href` for links, can be accessed through a property of the same name on the element’s DOM object. This is the case for a limited set of commonly used standard attributes. [](http://eloquentjavascript.net/13_dom.html#p_vK0AfBjXCM)But HTML allows you to set any attribute you want on nodes. This can be useful because it allows you to store extra information in a document. If you make up your own attribute names, though, such attributes will not be present as a property on the element’s node. Instead, you’ll have to use the`getAttribute` and `setAttribute` methods to work with them. ~~~ <p data-classified="secret">The launch code is 00000000.</p> <p data-classified="unclassified">I have two feet.</p> <script> var paras = document.body.getElementsByTagName("p"); Array.prototype.forEach.call(paras, function(para) { if (para.getAttribute("data-classified") == "secret") para.parentNode.removeChild(para); }); </script> ~~~ [](http://eloquentjavascript.net/13_dom.html#p_TbxTjtVsz1)I recommended prefixing the names of such made-up attributes with `data-` to ensure they do not conflict with any other attributes. [](http://eloquentjavascript.net/13_dom.html#p_wZUvKB4q4J)As a simple example, we’ll write a “syntax highlighter” that looks for `<pre>`tags (“preformatted”, used for code and similar plaintext) with a `data-language` attribute and crudely tries to highlight the keywords for that language. ~~~ function highlightCode(node, keywords) { var text = node.textContent; node.textContent = ""; // Clear the node var match, pos = 0; while (match = keywords.exec(text)) { var before = text.slice(pos, match.index); node.appendChild(document.createTextNode(before)); var strong = document.createElement("strong"); strong.appendChild(document.createTextNode(match[0])); node.appendChild(strong); pos = keywords.lastIndex; } var after = text.slice(pos); node.appendChild(document.createTextNode(after)); } ~~~ [](http://eloquentjavascript.net/13_dom.html#p_pT03b/PJLa)The function `highlightCode` takes a `<pre>` node and a regular expression (with the “global” option turned on) that matches the keywords of the programming language that the element contains. [](http://eloquentjavascript.net/13_dom.html#p_uu6ujSbhOJ)The `textContent` property is used to get all the text in the node and is then set to an empty string, which has the effect of emptying the node. We loop over all matches of the keyword expression, appending the text *between* them as regular text nodes, and the text matched (the keywords) as text nodes wrapped in `<strong>` (bold) elements. [](http://eloquentjavascript.net/13_dom.html#p_lJjXqP5cl1)We can automatically highlight all programs on the page by looping over all the`<pre>` elements that have a `data-language` attribute and calling`highlightCode` on each one with the correct regular expression for the language. ~~~ var languages = { javascript: /\b(function|return|var)\b/g /* … etc */ }; function highlightAllCode() { var pres = document.body.getElementsByTagName("pre"); for (var i = 0; i < pres.length; i++) { var pre = pres[i]; var lang = pre.getAttribute("data-language"); if (languages.hasOwnProperty(lang)) highlightCode(pre, languages[lang]); } } ~~~ [](http://eloquentjavascript.net/13_dom.html#p_q0YBA4Bx6Q)Here is an example: ~~~ <p>Here it is, the identity function:</p> <pre data-language="javascript"> function id(x) { return x; } </pre> <script>highlightAllCode();</script> ~~~ [](http://eloquentjavascript.net/13_dom.html#p_fBkADkC735)There is one commonly used attribute, `class`, which is a reserved word in the JavaScript language. For historical reasons—some old JavaScript implementations could not handle property names that matched keywords or reserved words—the property used to access this attribute is called `className`. You can also access it under its real name, `"class"`, by using the`getAttribute` and `setAttribute` methods. ## [](http://eloquentjavascript.net/13_dom.html#h_lyrY2KUDl7)Layout [](http://eloquentjavascript.net/13_dom.html#p_bLEGhDS9sd)You might have noticed that different types of elements are laid out differently. Some, such as paragraphs (`<p>`) or headings (`<h1>`), take up the whole width of the document and are rendered on separate lines. These are called *block*elements. Others, such as links (`<a>`) or the `<strong>` element used in the previous example, are rendered on the same line with their surrounding text. Such elements are called *inline* elements. [](http://eloquentjavascript.net/13_dom.html#p_uyQ6hsLw6d)For any given document, browsers are able to compute a layout, which gives each element a size and position based on its type and content. This layout is then used to actually draw the document. [](http://eloquentjavascript.net/13_dom.html#p_mT0v96DVDo)The size and position of an element can be accessed from JavaScript. The`offsetWidth` and `offsetHeight` properties give you the space the element takes up in *pixels*. A pixel is the basic unit of measurement in the browser and typically corresponds to the smallest dot that your screen can display. Similarly, `clientWidth` and `clientHeight` give you the size of the space*inside* the element, ignoring border width. ~~~ <p style="border: 3px solid red"> I'm boxed in </p> <script> var para = document.body.getElementsByTagName("p")[0]; console.log("clientHeight:", para.clientHeight); console.log("offsetHeight:", para.offsetHeight); </script> ~~~ [](http://eloquentjavascript.net/13_dom.html#p_RG9FbUJg15)The most effective way to find the precise position of an element on the screen is the `getBoundingClientRect` method. It returns an object with `top`,`bottom`, `left`, and `right` properties, indicating the pixel positions of the sides of the element relative to the top left of the screen. If you want them relative to the whole document, you must add the current scroll position, found under the global `pageXOffset` and `pageYOffset` variables. [](http://eloquentjavascript.net/13_dom.html#p_nnTb9RktUT)Laying out a document can be quite a lot of work. In the interest of speed, browser engines do not immediately re-layout a document every time it is changed but rather wait as long as they can. When a JavaScript program that changed the document finishes running, the browser will have to compute a new layout in order to display the changed document on the screen. When a program *asks* for the position or size of something by reading properties such as `offsetHeight` or calling `getBoundingClientRect`, providing correct information also requires computing a layout. [](http://eloquentjavascript.net/13_dom.html#p_6id19IfKq+)A program that repeatedly alternates between reading DOM layout information and changing the DOM forces a lot of layouts to happen and will consequently run really slowly. The following code shows an example of this. It contains two different programs that build up a line of *X* characters 2,000 pixels wide and measures the time each one takes. ~~~ <p><span id="one"></span></p> <p><span id="two"></span></p> <script> function time(name, action) { var start = Date.now(); // Current time in milliseconds action(); console.log(name, "took", Date.now() - start, "ms"); } time("naive", function() { var target = document.getElementById("one"); while (target.offsetWidth < 2000) target.appendChild(document.createTextNode("X")); }); // → naive took 32 ms time("clever", function() { var target = document.getElementById("two"); target.appendChild(document.createTextNode("XXXXX")); var total = Math.ceil(2000 / (target.offsetWidth / 5)); for (var i = 5; i < total; i++) target.appendChild(document.createTextNode("X")); }); // → clever took 1 ms </script> ~~~ ## [](http://eloquentjavascript.net/13_dom.html#h_VfKBbtOqcL)Styling [](http://eloquentjavascript.net/13_dom.html#p_sWGHgVTbfa)We have seen that different HTML elements display different behavior. Some are displayed as blocks, others inline. Some add styling, such as `<strong>`making its content bold and `<a>` making it blue and underlining it. [](http://eloquentjavascript.net/13_dom.html#p_CLPG6z9ZaH)The way an `<img>` tag shows an image or an `<a>` tag causes a link to be followed when it is clicked is strongly tied to the element type. But the default styling associated with an element, such as the text color or underline, can be changed by us. Here is an example using the `style` property: ~~~ <p><a href=".">Normal link</a></p> <p><a href="." style="color: green">Green link</a></p> ~~~ [](http://eloquentjavascript.net/13_dom.html#p_6HjJoH2gQy)A style attribute may contain one or more *declarations*, which are a property (such as `color`) followed by a colon and a value (such as `green`). When there is more than one declaration, they must be separated by semicolons, as in`"color: red; border: none"`. [](http://eloquentjavascript.net/13_dom.html#p_oyv0VBpq2N)There are a lot of aspects that can be influenced by styling. For example, the`display` property controls whether an element is displayed as a block or an inline element. ~~~ This text is displayed <strong>inline</strong>, <strong style="display: block">as a block</strong>, and <strong style="display: none">not at all</strong>. ~~~ [](http://eloquentjavascript.net/13_dom.html#p_lJ3ZvKI2rm)The `block` tag will end up on its own line since block elements are not displayed inline with the text around them. The last tag is not displayed at all—`display: none` prevents an element from showing up on the screen. This is a way to hide elements. It is often preferable to removing them from the document entirely because it makes it easy to reveal them again at a later time. [](http://eloquentjavascript.net/13_dom.html#p_0ienMYXo3C)JavaScript code can directly manipulate the style of an element through the node’s `style` property. This property holds an object that has properties for all possible style properties. The values of these properties are strings, which we can write to in order to change a particular aspect of the element’s style. ~~~ <p id="para" style="color: purple"> Pretty text </p> <script> var para = document.getElementById("para"); console.log(para.style.color); para.style.color = "magenta"; </script> ~~~ [](http://eloquentjavascript.net/13_dom.html#p_LVJWvFthG2)Some style property names contain dashes, such as `font-family`. Because such property names are awkward to work with in JavaScript (you’d have to say `style["font-family"]`), the property names in the `style` object for such properties have their dashes removed and the letters that follow them capitalized (`style.fontFamily`). ## [](http://eloquentjavascript.net/13_dom.html#h_7kGsaGnBbD)Cascading styles [](http://eloquentjavascript.net/13_dom.html#p_Phol2kWQBL)The styling system for HTML is called CSS for *Cascading Style Sheets*. A *style sheet* is a set of rules for how to style elements in a document. It can be given inside a `<style>` tag. ~~~ <style> strong { font-style: italic; color: gray; } </style> <p>Now <strong>strong text</strong> is italic and gray.</p> ~~~ [](http://eloquentjavascript.net/13_dom.html#p_m7OFPUb4bV)The *cascading* in the name refers to the fact that multiple such rules are combined to produce the final style for an element. In the previous example, the default styling for `<strong>` tags, which gives them `font-weight: bold`, is overlaid by the rule in the `<style>` tag, which adds `font-style` and `color`. [](http://eloquentjavascript.net/13_dom.html#p_TpdOmgyGAl)When multiple rules define a value for the same property, the most recently read rule gets a higher precedence and wins. So if the rule in the `<style>` tag included `font-weight: normal`, conflicting with the default `font-weight`rule, the text would be normal, *not* bold. Styles in a `style` attribute applied directly to the node have the highest precedence and always win. [](http://eloquentjavascript.net/13_dom.html#p_BgSVZ5fpx+)It is possible to target things other than tag names in CSS rules. A rule for `.abc`applies to all elements with `"abc"` in their class attributes. A rule for `#xyz`applies to the element with an `id` attribute of `"xyz"` (which should be unique within the document). ~~~ .subtle { color: gray; font-size: 80%; } #header { background: blue; color: white; } /* p elements, with classes a and b, and id main */ p.a.b#main { margin-bottom: 20px; } ~~~ [](http://eloquentjavascript.net/13_dom.html#p_7kmWmOHAMA)The precedence rule favoring the most recently defined rule holds true only when the rules have the same *specificity*. A rule’s specificity is a measure of how precisely it describes matching elements, determined by the number and kind (tag, class, or ID) of element aspects it requires. For example, a rule that targets `p.a` is more specific than rules that target `p` or just `.a`, and would thus take precedence over them. [](http://eloquentjavascript.net/13_dom.html#p_2eZ1iteQyg)The notation `p > a {…}` applies the given styles to all `<a>` tags that are direct children of `<p>` tags. Similarly, `p a {…}` applies to all `<a>` tags inside `<p>`tags, whether they are direct or indirect children. ## [](http://eloquentjavascript.net/13_dom.html#h_5ooQzToxht)Query selectors [](http://eloquentjavascript.net/13_dom.html#p_RtMy1VYPVy)We won’t be using style sheets all that much in this book. Although understanding them is crucial to programming in the browser, properly explaining all the properties they support and the interaction among those properties would take two or three books. [](http://eloquentjavascript.net/13_dom.html#p_vgThAtbEjX)The main reason I introduced *selector* syntax���the notation used in style sheets to determine which elements a set of styles apply to—is that we can use this same mini-language as an effective way to find DOM elements. [](http://eloquentjavascript.net/13_dom.html#p_6m9AOTvoRS)The `querySelectorAll` method, which is defined both on the `document`object and on element nodes, takes a selector string and returns an array-like object containing all the elements that it matches. ~~~ <p>And if you go chasing <span class="animal">rabbits</span></p> <p>And you know you're going to fall</p> <p>Tell 'em a <span class="character">hookah smoking <span class="animal">caterpillar</span></span></p> <p>Has given you the call</p> <script> function count(selector) { return document.querySelectorAll(selector).length; } console.log(count("p")); // All <p> elements // → 4 console.log(count(".animal")); // Class animal // → 2 console.log(count("p .animal")); // Animal inside of <p> // → 2 console.log(count("p > .animal")); // Direct child of <p> // → 1 </script> ~~~ [](http://eloquentjavascript.net/13_dom.html#p_EQFXoZVpt7)Unlike methods such as `getElementsByTagName`, the object returned by`querySelectorAll` is *not* live. It won’t change when you change the document. [](http://eloquentjavascript.net/13_dom.html#p_sO/Wz9EvWw)The `querySelector` method (without the `All` part) works in a similar way. This one is useful if you want a specific, single element. It will return only the first matching element or null if no elements match. ## [](http://eloquentjavascript.net/13_dom.html#h_MAsyozbjjZ)Positioning and animating [](http://eloquentjavascript.net/13_dom.html#p_dDOfb5gw5J)The `position` style property influences layout in a powerful way. By default it has a value of `static`, meaning the element sits in its normal place in the document. When it is set to `relative`, the element still takes up space in the document, but now the `top` and `left` style properties can be used to move it relative to its normal place. When `position` is set to `absolute`, the element is removed from the normal document flow—that is, it no longer takes up space and may overlap with other elements. Also, its `top` and `left` properties can be used to absolutely position it relative to the top-left corner of the nearest enclosing element whose `position` property isn’t `static`, or relative to the document if no such enclosing element exists. [](http://eloquentjavascript.net/13_dom.html#p_TEJWNXCk6K)We can use this to create an animation. The following document displays a picture of a cat that floats around in an ellipse: ~~~ <p style="text-align: center"> <img src="img/cat.png" style="position: relative"> </p> <script> var cat = document.querySelector("img"); var angle = 0, lastTime = null; function animate(time) { if (lastTime != null) angle += (time - lastTime) * 0.001; lastTime = time; cat.style.top = (Math.sin(angle) * 20) + "px"; cat.style.left = (Math.cos(angle) * 200) + "px"; requestAnimationFrame(animate); } requestAnimationFrame(animate); </script> ~~~ [](http://eloquentjavascript.net/13_dom.html#p_HrW/6s5nxY)The picture is centered on the page and given a `position` of `relative`. We’ll repeatedly update that picture’s `top` and `left` styles in order to move it. [](http://eloquentjavascript.net/13_dom.html#p_YWJ84IIJ5r)The script uses `requestAnimationFrame` to schedule the `animate` function to run whenever the browser is ready to repaint the screen. The `animate` function itself again calls `requestAnimationFrame` to schedule the next update. When the browser window (or tab) is active, this will cause updates to happen at a rate of about 60 per second, which tends to produce a good-looking animation. [](http://eloquentjavascript.net/13_dom.html#p_V1ZyP92HDo)If we just updated the DOM in a loop, the page would freeze and nothing would show up on the screen. Browsers do not update their display while a JavaScript program is running, nor do they allow any interaction with the page. This is why we need `requestAnimationFrame`—it lets the browser know that we are done for now, and it can go ahead and do the things that browsers do, such as updating the screen and responding to user actions. [](http://eloquentjavascript.net/13_dom.html#p_65aVIJgoMb)Our animation function is passed the current time as an argument, which it compares to the time it saw before (the `lastTime` variable) to ensure the motion of the cat per millisecond is stable, and the animation moves smoothly. If it just moved a fixed amount per step, the motion would stutter if, for example, another heavy task running on the same computer were to prevent the function from running for a fraction of a second. [](http://eloquentjavascript.net/13_dom.html#p_hfdwmxu8Y5)Moving in circles is done using the trigonometry functions `Math.cos` and`Math.sin`. For those of you who aren’t familiar with these, I’ll briefly introduce them since we will occasionally need them in this book. [](http://eloquentjavascript.net/13_dom.html#p_4qsmOl5+5L)`Math.cos` and `Math.sin` are useful for finding points that lie on a circle around point (0,0) with a radius of one unit. Both functions interpret their argument as the position on this circle, with zero denoting the point on the far right of the circle, going clockwise until 2π (about 6.28) has taken us around the whole circle. `Math.cos` tells you the x-coordinate of the point that corresponds to the given position around the circle, while `Math.sin` yields the y-coordinate. Positions (or angles) greater than 2π or less than 0 are valid—the rotation repeats so that *a*+2π refers to the same angle as *a*.  [](http://eloquentjavascript.net/13_dom.html#p_XxeJZtQAlH)The cat animation code keeps a counter, `angle`, for the current angle of the animation and increments it in proportion to the elapsed time every time the`animate` function is called. It can then use this angle to compute the current position of the image element. The `top` style is computed with `Math.sin` and multiplied by 20, which is the vertical radius of our circle. The `left` style is based on `Math.cos` and multiplied by 200 so that the circle is much wider than it is high, resulting in an elliptic motion. [](http://eloquentjavascript.net/13_dom.html#p_/Yly9Ir9QF)Note that styles usually need *units*. In this case, we have to append `"px"` to the number to tell the browser we are counting in pixels (as opposed to centimeters, “ems”, or other units). This is easy to forget. Using numbers without units will result in your style being ignored—unless the number is 0, which always means the same thing, regardless of its unit. ## [](http://eloquentjavascript.net/13_dom.html#h_ErccPg/l98)Summary [](http://eloquentjavascript.net/13_dom.html#p_kLcHWntN1r)JavaScript programs may inspect and interfere with the current document that a browser is displaying through a data structure called the DOM. This data structure represents the browser’s model of the document, and a JavaScript program can modify it to change the visible document. [](http://eloquentjavascript.net/13_dom.html#p_7Ce58FA9bp)The DOM is organized like a tree, in which elements are arranged hierarchically according to the structure of the document. The objects representing elements have properties such as `parentNode` and `childNodes`, which can be used to navigate through this tree. [](http://eloquentjavascript.net/13_dom.html#p_kklmAu08Ik)The way a document is displayed can be influenced by *styling*, both by attaching styles to nodes directly and by defining rules that match certain nodes. There are many different style properties, such as `color` or `display`. JavaScript can manipulate an element’s style directly through its `style`property. ## [](http://eloquentjavascript.net/13_dom.html#h_TcUD2vzyMe)Exercises ### [](http://eloquentjavascript.net/13_dom.html#h_g/5UC3zznV)Build a table [](http://eloquentjavascript.net/13_dom.html#p_1dZtDT5P1x)We built plaintext tables in [Chapter 6](http://eloquentjavascript.net/06_object.html#tables). HTML makes laying out tables quite a bit easier. An HTML table is built with the following tag structure: ~~~ <table> <tr> <th>name</th> <th>height</th> <th>country</th> </tr> <tr> <td>Kilimanjaro</td> <td>5895</td> <td>Tanzania</td> </tr> </table> ~~~ [](http://eloquentjavascript.net/13_dom.html#p_5ddcb3G9fr)For each *row*, the `<table>` tag contains a `<tr>` tag. Inside of these `<tr>` tags, we can put cell elements: either heading cells (`<th>`) or regular cells (`<td>`). [](http://eloquentjavascript.net/13_dom.html#p_BtYdHKzhgN)The same source data that was used in [Chapter 6](http://eloquentjavascript.net/06_object.html#mountains) is again available in the`MOUNTAINS` variable in the sandbox. It can also be [downloaded](http://eloquentjavascript.net/code/mountains.js) from the website. [](http://eloquentjavascript.net/13_dom.html#p_BwmSeFluk3)Write a function `buildTable` that, given an array of objects that all have the same set of properties, builds up a DOM structure representing a table. The table should have a header row with the property names wrapped in `<th>`elements and should have one subsequent row per object in the array, with its property values in `<td>` elements. [](http://eloquentjavascript.net/13_dom.html#p_2dqGRmSxc+)The `Object.keys` function, which returns an array containing the property names that an object has, will probably be helpful here. [](http://eloquentjavascript.net/13_dom.html#p_+D0CFGh4SM)Once you have the basics working, right-align cells containing numbers by setting their `style.textAlign` property to `"right"`. ~~~ <style> /* Defines a cleaner look for tables */ table { border-collapse: collapse; } td, th { border: 1px solid black; padding: 3px 8px; } th { text-align: left; } </style> <script> function buildTable(data) { // Your code here. } document.body.appendChild(buildTable(MOUNTAINS)); </script> ~~~ ### [](http://eloquentjavascript.net/13_dom.html#h_VSftnyRTsV)Elements by tag name [](http://eloquentjavascript.net/13_dom.html#p_8ZJVE0ewRe)The `getElementsByTagName` method returns all child elements with a given tag name. Implement your own version of it as a regular nonmethod function that takes a node and a string (the tag name) as arguments and returns an array containing all descendant element nodes with the given tag name. [](http://eloquentjavascript.net/13_dom.html#p_F3uW/zaQpy)To find the tag name of an element, use its `tagName` property. But note that this will return the tag name in all uppercase. Use the `toLowerCase` or`toUpperCase` string method to compensate for this. ~~~ <h1>Heading with a <span>span</span> element.</h1> <p>A paragraph with <span>one</span>, <span>two</span> spans.</p> <script> function byTagName(node, tagName) { // Your code here. } console.log(byTagName(document.body, "h1").length); // → 1 console.log(byTagName(document.body, "span").length); // → 3 var para = document.querySelector("p"); console.log(byTagName(para, "span").length); // → 2 </script> ~~~ ### [](http://eloquentjavascript.net/13_dom.html#h_ye4Ws9YmoI)The cat’s hat [](http://eloquentjavascript.net/13_dom.html#p_6r1baDVOSE)Extend the cat animation defined [earlier](http://eloquentjavascript.net/13_dom.html#animation) so that both the cat and his hat (`<img src="img/hat.png">`) orbit at opposite sides of the ellipse. [](http://eloquentjavascript.net/13_dom.html#p_2gTA+NB5av)Or make the hat circle around the cat. Or alter the animation in some other interesting way. [](http://eloquentjavascript.net/13_dom.html#p_kC6i5Avza+)To make positioning multiple objects easier, it is probably a good idea to switch to absolute positioning. This means that `top` and `left` are counted relative to the top left of the document. To avoid using negative coordinates, you can simply add a fixed number of pixels to the position values. ~~~ <img src="img/cat.png" id="cat" style="position: absolute"> <img src="img/hat.png" id="hat" style="position: absolute"> <script> var cat = document.querySelector("#cat"); var hat = document.querySelector("#hat"); // Your code here. </script> ~~~

## Chapter 12 # JavaScript and the Browser > [](http://eloquentjavascript.net/12_browser.html#p_oSbDqg89i/)The browser is a really hostile programming environment. > > Douglas Crockford, The JavaScript Programming Language (video lecture) [](http://eloquentjavascript.net/12_browser.html#p_GZ7nJVK3wA)The next part of this book will talk about web browsers. Without web browsers, there would be no JavaScript. And even if there were, no one would ever have paid any attention to it. [](http://eloquentjavascript.net/12_browser.html#p_XT5pNqisw+)Web technology has, from the start, been decentralized, not just technically but also in the way it has evolved. Various browser vendors have added new functionality in ad hoc and sometimes poorly thought out ways, which then sometimes ended up being adopted by others and finally set down as a standard. [](http://eloquentjavascript.net/12_browser.html#p_ePAQfpo7ra)This is both a blessing and a curse. On the one hand, it is empowering to not have a central party control a system but have it be improved by various parties working in loose collaboration (or, occasionally, open hostility). On the other hand, the haphazard way in which the Web was developed means that the resulting system is not exactly a shining example of internal consistency. In fact, some parts of it are downright messy and confusing. ## [](http://eloquentjavascript.net/12_browser.html#h_MYPczIw5xZ)Networks and the Internet [](http://eloquentjavascript.net/12_browser.html#p_hsMIYbAKNv)Computer networks have been around since the 1950s. If you put cables between two or more computers and allow them to send data back and forth through these cables, you can do all kinds of wonderful things. [](http://eloquentjavascript.net/12_browser.html#p_rRsGS85yhE)If connecting two machines in the same building allows us to do wonderful things, connecting machines all over the planet should be even better. The technology to start implementing this vision was developed in the 1980s, and the resulting network is called the *Internet*. It has lived up to its promise. [](http://eloquentjavascript.net/12_browser.html#p_cvrUZPtUxT)A computer can use this network to spew bits at another computer. For any effective communication to arise out of this bit-spewing, the computers at both ends must know what the bits are supposed to represent. The meaning of any given sequence of bits depends entirely on the kind of thing that it is trying to express and on the encoding mechanism used. [](http://eloquentjavascript.net/12_browser.html#p_tlYBPgx73H)A *network protocol* describes a style of communication over a network. There are protocols for sending email, for fetching email, for sharing files, or even for controlling computers that happen to be infected by malicious software. [](http://eloquentjavascript.net/12_browser.html#p_u8lDPIC39z)For example, a simple chat protocol might consist of one computer sending the bits that represent the text “CHAT?” to another machine and the other responding with “OK!” to confirm that it understands the protocol. They can then proceed to send each other strings of text, read the text sent by the other from the network, and display whatever they receive on their screens. [](http://eloquentjavascript.net/12_browser.html#p_WXSVtMixCt)Most protocols are built on top of other protocols. Our example chat protocol treats the network as a streamlike device into which you can put bits and have them arrive at the correct destination in the correct order. Ensuring those things is already a rather difficult technical problem. [](http://eloquentjavascript.net/12_browser.html#p_7IOEk4hhjX)The *Transmission Control Protocol* (TCP) is a protocol that solves this problem. All Internet-connected devices “speak” it, and most communication on the Internet is built on top of it. [](http://eloquentjavascript.net/12_browser.html#p_R1N5wYYFn+)A TCP connection works as follows: one computer must be waiting, or*listening*, for other computers to start talking to it. To be able to listen for different kinds of communication at the same time on a single machine, each listener has a number (called a *port*) associated with it. Most protocols specify which port should be used by default. For example, when we want to send an email using the SMTP protocol, the machine through which we send it is expected to be listening on port 25. [](http://eloquentjavascript.net/12_browser.html#p_C9NFg/yfxC)Another computer can then establish a connection by connecting to the target machine using the correct port number. If the target machine can be reached and is listening on that port, the connection is successfully created. The listening computer is called the *server*, and the connecting computer is called the *client*. [](http://eloquentjavascript.net/12_browser.html#p_g48pTp1IKV)Such a connection acts as a two-way pipe through which bits can flow—the machines on both ends can put data into it. Once the bits are successfully transmitted, they can be read out again by the machine on the other side. This is a convenient model. You could say that TCP provides an abstraction of the network. ## [](http://eloquentjavascript.net/12_browser.html#h_tbGTd9Llzv)The Web [](http://eloquentjavascript.net/12_browser.html#p_w4VQt163Im)The *World Wide Web* (not to be confused with the Internet as a whole) is a set of protocols and formats that allow us to visit web pages in a browser. The “Web” part in the name refers to the fact that such pages can easily link to each other, thus connecting into a huge mesh that users can move through. [](http://eloquentjavascript.net/12_browser.html#p_IX9vl21tDt)To add content to the Web, all you need to do is connect a machine to the Internet, and have it listen on port 80, using the *Hypertext Transfer Protocol*(HTTP). This protocol allows other computers to request documents over the network. [](http://eloquentjavascript.net/12_browser.html#p_KH3H5PXZlk)Each document on the Web is named by a *Uniform Resource Locator* (URL), which looks something like this: ~~~ http://eloquentjavascript.net/12_browser.html | | | | protocol server path ~~~ [](http://eloquentjavascript.net/12_browser.html#p_0F4MKpLU9B)The first part tells us that this URL uses the HTTP protocol (as opposed to, for example, encrypted HTTP, which would be *https://*). Then comes the part that identifies which server we are requesting the document from. Last is a path string that identifies the specific document (or *resource*) we are interested in. [](http://eloquentjavascript.net/12_browser.html#p_/GdEX6bncv)Each machine connected to the Internet gets a unique *IP address*, which looks something like `37.187.37.82`. You can use these directly as the server part of a URL. But lists of more or less random numbers are hard to remember and awkward to type, so you can instead register a *domain name* to point toward a specific machine or set of machines. I registered *eloquentjavascript.net* to point at the IP address of a machine I control and can thus use that domain name to serve web pages. [](http://eloquentjavascript.net/12_browser.html#p_VQIXesk4P9)If you type the previous URL into your browser’s address bar, it will try to retrieve and display the document at that URL. First, your browser has to find out what address *eloquentjavascript.net* refers to. Then, using the HTTP protocol, it makes a connection to the server at that address and asks for the resource */12_browser.html*. [](http://eloquentjavascript.net/12_browser.html#p_pfUoCDr2pV)We will take a closer look at the HTTP protocol in [Chapter 17](http://eloquentjavascript.net/17_http.html#http). ## [](http://eloquentjavascript.net/12_browser.html#h_n3OM6EV/KR)HTML [](http://eloquentjavascript.net/12_browser.html#p_m2H1Cp4ACJ)HTML, which stands for *Hypertext Markup Language*, is the document format used for web pages. An HTML document contains text, as well as *tags*that give structure to the text, describing things such as links, paragraphs, and headings. [](http://eloquentjavascript.net/12_browser.html#p_P2V14LZFIv)A simple HTML document looks like this: ~~~ <!doctype html> <html> <head> <title>My home page</title> </head> <body> <h1>My home page</h1> <p>Hello, I am Marijn and this is my home page.</p> <p>I also wrote a book! Read it <a href="http://eloquentjavascript.net">here</a>.</p> </body> </html> ~~~ [](http://eloquentjavascript.net/12_browser.html#p_zP6SuTzWSr)The tags, wrapped in angle brackets (`<` and `>`), provide information about the structure of the document. The other text is just plain text. [](http://eloquentjavascript.net/12_browser.html#p_7PI7BADgG4)The document starts with `<!doctype html>`, which tells the browser to interpret it as *modern* HTML, as opposed to various dialects that were in use in the past. [](http://eloquentjavascript.net/12_browser.html#p_Et3bT2kUby)HTML documents have a head and a body. The head contains information*about* the document, and the body contains the document itself. In this case, we first declared that the title of this document is “My home page” and then gave a document containing a heading (`<h1>`, meaning “heading 1”—`<h2>` to`<h6>` produce more minor headings) and two paragraphs (`<p>`). [](http://eloquentjavascript.net/12_browser.html#p_twHeUPgGnQ)Tags come in several forms. An element, such as the body, a paragraph, or a link, is started by an *opening tag* like `<p>` and ended by a *closing tag* like`</p>`. Some opening tags, such as the one for the link (`<a>`), contain extra information in the form of `name="value"` pairs. These are called *attributes*. In this case, the destination of the link is indicated with`href="http://eloquentjavascript.net"`, where `href` stands for “hypertext reference”. [](http://eloquentjavascript.net/12_browser.html#p_a6Kx16yoMd)Some kinds of tags do not enclose anything and thus do not need to be closed. An example of this would be `<img src="http://example.com/image.jpg">`, which will display the image found at the given source URL. [](http://eloquentjavascript.net/12_browser.html#p_aDzWCv6pnm)To be able to include angle brackets in the text of a document, even though they have a special meaning in HTML, yet another form of special notation has to be introduced. A plain opening angle bracket is written as `&lt;` (“less than”), and a closing bracket is written as `&gt;` (“greater than”). In HTML, an ampersand (`&`) character followed by a word and a semicolon (`;`) is called an*entity*, and will be replaced by the character it encodes. [](http://eloquentjavascript.net/12_browser.html#p_BVTrBS/gKt)This is analogous to the way backslashes are used in JavaScript strings. Since this mechanism gives ampersand characters a special meaning, too, those need to be escaped as `&amp;`. Inside an attribute, which is wrapped in double quotes,`&quot;` can be used to insert an actual quote character. [](http://eloquentjavascript.net/12_browser.html#p_RK6LQEinS9)HTML is parsed in a remarkably error-tolerant way. When tags that should be there are missing, the browser reconstructs them. The way in which this is done has been standardized, and you can rely on all modern browsers to do it in the same way. [](http://eloquentjavascript.net/12_browser.html#p_hnbBzl0NyQ)The following document will be treated just like the one shown previously: ~~~ <!doctype html> <title>My home page</title> <h1>My home page</h1> <p>Hello, I am Marijn and this is my home page. <p>I also wrote a book! Read it <a href=http://eloquentjavascript.net>here</a>. ~~~ [](http://eloquentjavascript.net/12_browser.html#p_6iDhitY04v)The `<html>`, `<head>`, and `<body>` tags are gone completely. The browser knows that `<title>` belongs in a head, and that `<h1>` in a body. Furthermore, I am no longer explicitly closing the paragraphs since opening a new paragraph or ending the document will close them implicitly. The quotes around the link target are also gone. [](http://eloquentjavascript.net/12_browser.html#p_849RpAZtxv)This book will usually omit the `<html>`, `<head>`, and `<body>` tags from examples to keep them short and free of clutter. But I *will* close tags and include quotes around attributes. [](http://eloquentjavascript.net/12_browser.html#p_YK9xXsU8IZ)I will also usually omit the doctype. This is not to be taken as an encouragement to omit doctype declarations. Browsers will often do ridiculous things when you forget them. You should consider doctypes implicitly present in examples, even when they are not actually shown in the text. ## [](http://eloquentjavascript.net/12_browser.html#h_x9VDt2sTZZ)HTML and JavaScript [](http://eloquentjavascript.net/12_browser.html#p_Eruo+DmcTl)In the context of this book, the most important HTML tag is `<script>`. This tag allows us to include a piece of JavaScript in a document. ~~~ <h1>Testing alert</h1> <script>alert("hello!");</script> ~~~ [](http://eloquentjavascript.net/12_browser.html#p_3pFG6nbfUJ)Such a script will run as soon as its `<script>` tag is encountered as the browser reads the HTML. The page shown earlier will pop up an `alert` dialog when opened. [](http://eloquentjavascript.net/12_browser.html#p_GmDPwY/KfO)Including large programs directly in HTML documents is often impractical. The `<script>` tag can be given an `src` attribute in order to fetch a script file (a text file containing a JavaScript program) from a URL. ~~~ <h1>Testing alert</h1> <script src="code/hello.js"></script> ~~~ [](http://eloquentjavascript.net/12_browser.html#p_Hz/JcRYmOp)The *code/hello.js* file included here contains the same simple program,`alert("hello!")`. When an HTML page references other URLs as part of itself, for example an image file or a script—web browsers will retrieve them immediately and include them in the page. [](http://eloquentjavascript.net/12_browser.html#p_DLaR5M/yDv)A script tag must always be closed with `</script>`, even if it refers to a script file and doesn’t contain any code. If you forget this, the rest of the page will be interpreted as part of the script. [](http://eloquentjavascript.net/12_browser.html#p_ztoAMAZEju)Some attributes can also contain a JavaScript program. The `<button>` tag shown next (which shows up as a button) has an `onclick` attribute, whose content will be run whenever the button is clicked. ~~~ <button onclick="alert('Boom!');">DO NOT PRESS</button> ~~~ [](http://eloquentjavascript.net/12_browser.html#p_hBrXiBnQxx)Note that I had to use single quotes for the string in the `onclick` attribute because double quotes are already used to quote the whole attribute. I could also have used `&quot;`, but that’d make the program harder to read. ## [](http://eloquentjavascript.net/12_browser.html#h_xSthu5StoL)In the sandbox [](http://eloquentjavascript.net/12_browser.html#p_AniKiqdG2y)Running programs downloaded from the Internet is potentially dangerous. You do not know much about the people behind most sites you visit, and they do not necessarily mean well. Running programs by people who do not mean well is how you get your computer infected by viruses, your data stolen, and your accounts hacked. [](http://eloquentjavascript.net/12_browser.html#p_D7H4XfnmGB)Yet the attraction of the Web is that you can surf it without necessarily trusting all the pages you visit. This is why browsers severely limit the things a JavaScript program may do: it can’t look at the files on your computer or modify anything not related to the web page it was embedded in. [](http://eloquentjavascript.net/12_browser.html#p_KW2fU3B9nm)Isolating a programming environment in this way is called *sandboxing*, the idea being that the program is harmlessly playing in a sandbox. But you should imagine this particular kind of sandbox as having a cage of thick steel bars over it, which makes it somewhat different from your typical playground sandbox. [](http://eloquentjavascript.net/12_browser.html#p_7pNt468QvZ)The hard part of sandboxing is allowing the programs enough room to be useful yet at the same time restricting them from doing anything dangerous. Lots of useful functionality, such as communicating with other servers or reading the content of the copy-paste clipboard, can also be used to do problematic, privacy-invading things. [](http://eloquentjavascript.net/12_browser.html#p_0xA5OSkc0i)Every now and then, someone comes up with a new way to circumvent the limitations of a browser and do something harmful, ranging from leaking minor private information to taking over the whole machine that the browser runs on. The browser developers respond by fixing the hole, and all is well again—that is, until the next problem is discovered, and hopefully publicized, rather than secretly exploited by some government or mafia. ## [](http://eloquentjavascript.net/12_browser.html#h_p42hxqLkOm)Compatibility and the browser wars [](http://eloquentjavascript.net/12_browser.html#p_EZKXamx/Cn)In the early stages of the Web, a browser called Mosaic dominated the market. After a few years, the balance had shifted to Netscape, which was then, in turn, largely supplanted by Microsoft’s Internet Explorer. At any point where a single browser was dominant, that browser’s vendor would feel entitled to unilaterally invent new features for the Web. Since most users used the same browser, websites would simply start using those features—never mind the other browsers. [](http://eloquentjavascript.net/12_browser.html#p_VsfVT8QgQV)This was the dark age of compatibility, often called the *browser wars*. Web developers were left with not one unified Web but two or three incompatible platforms. To make things worse, the browsers in use around 2003 were all full of bugs, and of course the bugs were different for each browser. Life was hard for people writing web pages. [](http://eloquentjavascript.net/12_browser.html#p_K5Wu7RrwZS)Mozilla Firefox, a not-for-profit offshoot of Netscape, challenged Internet Explorer’s hegemony in the late 2000s. Because Microsoft was not particularly interested in staying competitive at the time, Firefox took quite a chunk of market share away from it. Around the same time, Google introduced its Chrome browser, and Apple’s Safari browser gained popularity, leading to a situation where there were four major players, rather than one. [](http://eloquentjavascript.net/12_browser.html#p_ztjPRIOS1D)The new players had a more serious attitude toward standards and better engineering practices, leading to less incompatibility and fewer bugs. Microsoft, seeing its market share crumble, came around and adopted these attitudes. If you are starting to learn web development today, consider yourself lucky. The latest versions of the major browsers behave quite uniformly and have relatively few bugs. [](http://eloquentjavascript.net/12_browser.html#p_4EMPHMJP+U)That is not to say that the situation is perfect just yet. Some of the people using the Web are, for reasons of inertia or corporate policy, stuck with very old browsers. Until those browsers die out entirely, writing websites that work for them will require a lot of arcane knowledge about their shortcomings and quirks. This book is not about those quirks. Rather, it aims to present the modern, sane style of web programming.

# Part 2: Browser * * * * * #### Chapter 12. [JavaScript and the Browser](76539) #### Chapter 13. [The Document Object Model](76540) #### Chapter 14. [Handling Events](76541) #### Chapter 15. [Project: A Platform Game](76542) #### Chapter 16. [Drawing on Canvas](76543) #### Chapter 17. [HTTP](76544) #### Chapter 18. [Forms and Form Fields](76545) #### Chapter 19. [Project: A Paint Program](76546)

## Chapter 11 # Project: A Programming Language > [](http://eloquentjavascript.net/11_language.html#p_Mt+zDiEThG)The evaluator, which determines the meaning of expressions in a programming language, is just another program. > > Hal Abelson and Gerald Sussman, Structure and Interpretation of Computer Programs > [](http://eloquentjavascript.net/11_language.html#p_KjZntSSadH)When a student asked the master about the nature of the cycle of Data and Control, Yuan-Ma replied ‘Think of a compiler, compiling itself.’ > > Master Yuan-Ma, The Book of Programming [](http://eloquentjavascript.net/11_language.html#p_sLOpE8A1YZ)Building your own programming language is surprisingly easy (as long as you do not aim too high) and very enlightening. [](http://eloquentjavascript.net/11_language.html#p_XWyxXihLGw)The main thing I want to show in this chapter is that there is no magic involved in building your own language. I’ve often felt that some human inventions were so immensely clever and complicated that I’d never be able to understand them. But with a little reading and tinkering, such things often turn out to be quite mundane. [](http://eloquentjavascript.net/11_language.html#p_99Lxo7iLF8)We will build a programming language called Egg. It will be a tiny, simple language but one that is powerful enough to express any computation you can think of. It will also allow simple abstraction based on functions. ## [](http://eloquentjavascript.net/11_language.html#h_cpTTNxAWkQ)Parsing [](http://eloquentjavascript.net/11_language.html#p_iHS+/Um+4q)The most immediately visible part of a programming language is its *syntax*, or notation. A *parser* is a program that reads a piece of text and produces a data structure that reflects the structure of the program contained in that text. If the text does not form a valid program, the parser should complain and point out the error. [](http://eloquentjavascript.net/11_language.html#p_/qJ2+kk2My)Our language will have a simple and uniform syntax. Everything in Egg is an expression. An expression can be a variable, a number, a string, or an*application*. Applications are used for function calls but also for constructs such as `if` or `while`. [](http://eloquentjavascript.net/11_language.html#p_Us9En1q6+r)To keep the parser simple, strings in Egg do not support anything like backslash escapes. A string is simply a sequence of characters that are not double quotes, wrapped in double quotes. A number is a sequence of digits. Variable names can consist of any character that is not whitespace and does not have a special meaning in the syntax. [](http://eloquentjavascript.net/11_language.html#p_N7llkiWaN/)Applications are written the way they are in JavaScript, by putting parentheses after an expression and having any number of arguments between those parentheses, separated by commas. ~~~ do(define(x, 10), if(>(x, 5), print("large"), print("small"))) ~~~ [](http://eloquentjavascript.net/11_language.html#p_HBJ5fbzauR)The uniformity of the Egg language means that things that are operators in JavaScript (such as `>`) are normal variables in this language, applied just like other functions. And since the syntax has no concept of a block, we need a `do`construct to represent doing multiple things in sequence. [](http://eloquentjavascript.net/11_language.html#p_aL3GdDKR9a)The data structure that the parser will use to describe a program will consist of expression objects, each of which has a `type` property indicating the kind of expression it is and other properties to describe its content. [](http://eloquentjavascript.net/11_language.html#p_1s3I5AYr64)Expressions of type `"value"` represent literal strings or numbers. Their `value`property contains the string or number value that they represent. Expressions of type `"word"` are used for identifiers (names). Such objects have a `name`property that holds the identifier’s name as a string. Finally, `"apply"`expressions represent applications. They have an `operator` property that refers to the expression that is being applied, and they have an `args` property that refers to an array of argument expressions. [](http://eloquentjavascript.net/11_language.html#p_GhgrfUGfKN)The `>(x, 5)` part of the previous program would be represented like this: ~~~ { type: "apply", operator: {type: "word", name: ">"}, args: [ {type: "word", name: "x"}, {type: "value", value: 5} ] } ~~~ [](http://eloquentjavascript.net/11_language.html#p_wZheW4Mu+K)Such a data structure is called a *syntax tree*. If you imagine the objects as dots and the links between them as lines between those dots, it has a treelike shape. The fact that expressions contain other expressions, which in turn might contain more expressions, is similar to the way branches split and split again.  [](http://eloquentjavascript.net/11_language.html#p_TiCtOm6qwQ)Contrast this to the parser we wrote for the configuration file format in [Chapter 9](http://eloquentjavascript.net/09_regexp.html#ini), which had a simple structure: it split the input into lines and handled those lines one at a time. There were only a few simple forms that a line was allowed to have. [](http://eloquentjavascript.net/11_language.html#p_OtoCO51d4l)Here we must find a different approach. Expressions are not separated into lines, and they have a recursive structure. Application expressions *contain*other expressions. [](http://eloquentjavascript.net/11_language.html#p_doWGMjYGVI)Fortunately, this problem can be solved elegantly by writing a parser function that is recursive in a way that reflects the recursive nature of the language. [](http://eloquentjavascript.net/11_language.html#p_vEMw7RQ1iA)We define a function `parseExpression`, which takes a string as input and returns an object containing the data structure for the expression at the start of the string, along with the part of the string left after parsing this expression. When parsing subexpressions (the argument to an application, for example), this function can be called again, yielding the argument expression as well as the text that remains. This text may in turn contain more arguments or may be the closing parenthesis that ends the list of arguments. [](http://eloquentjavascript.net/11_language.html#p_qQZOLuZdMw)This is the first part of the parser: ~~~ function parseExpression(program) { program = skipSpace(program); var match, expr; if (match = /^"([^"]*)"/.exec(program)) expr = {type: "value", value: match[1]}; else if (match = /^\d+\b/.exec(program)) expr = {type: "value", value: Number(match[0])}; else if (match = /^[^\s(),"]+/.exec(program)) expr = {type: "word", name: match[0]}; else throw new SyntaxError("Unexpected syntax: " + program); return parseApply(expr, program.slice(match[0].length)); } function skipSpace(string) { var first = string.search(/\S/); if (first == -1) return ""; return string.slice(first); } ~~~ [](http://eloquentjavascript.net/11_language.html#p_A31RFsnSOb)Because Egg allows any amount of whitespace between its elements, we have to repeatedly cut the whitespace off the start of the program string. This is what the `skipSpace` function helps with. [](http://eloquentjavascript.net/11_language.html#p_FglUPeozx+)After skipping any leading space, `parseExpression` uses three regular expressions to spot the three simple (atomic) elements that Egg supports: strings, numbers, and words. The parser constructs a different kind of data structure depending on which one matches. If the input does not match one of these three forms, it is not a valid expression, and the parser throws an error.`SyntaxError` is a standard error object type, which is raised when an attempt is made to run an invalid JavaScript program. [](http://eloquentjavascript.net/11_language.html#p_SHT28zA+Zh)We can then cut off the part that we matched from the program string and pass that, along with the object for the expression, to `parseApply`, which checks whether the expression is an application. If so, it parses a parenthesized list of arguments. ~~~ function parseApply(expr, program) { program = skipSpace(program); if (program[0] != "(") return {expr: expr, rest: program}; program = skipSpace(program.slice(1)); expr = {type: "apply", operator: expr, args: []}; while (program[0] != ")") { var arg = parseExpression(program); expr.args.push(arg.expr); program = skipSpace(arg.rest); if (program[0] == ",") program = skipSpace(program.slice(1)); else if (program[0] != ")") throw new SyntaxError("Expected ',' or ')'"); } return parseApply(expr, program.slice(1)); } ~~~ [](http://eloquentjavascript.net/11_language.html#p_I71H7ro2MW)If the next character in the program is not an opening parenthesis, this is not an application, and `parseApply` simply returns the expression it was given. [](http://eloquentjavascript.net/11_language.html#p_rdoND4Um+i)Otherwise, it skips the opening parenthesis and creates the syntax tree object for this application expression. It then recursively calls `parseExpression` to parse each argument until a closing parenthesis is found. The recursion is indirect, through `parseApply` and `parseExpression` calling each other. [](http://eloquentjavascript.net/11_language.html#p_U3xcmi1mE5)Because an application expression can itself be applied (such as in`multiplier(2)(1)`), `parseApply` must, after it has parsed an application, call itself again to check whether another pair of parentheses follows. [](http://eloquentjavascript.net/11_language.html#p_I3C1yxD9Oq)This is all we need to parse Egg. We wrap it in a convenient `parse` function that verifies that it has reached the end of the input string after parsing the expression (an Egg program is a single expression), and that gives us the program’s data structure. ~~~ function parse(program) { var result = parseExpression(program); if (skipSpace(result.rest).length > 0) throw new SyntaxError("Unexpected text after program"); return result.expr; } console.log(parse("+(a, 10)")); // → {type: "apply", // operator: {type: "word", name: "+"}, // args: [{type: "word", name: "a"}, // {type: "value", value: 10}]} ~~~ [](http://eloquentjavascript.net/11_language.html#p_sbjERs3L8Y)It works! It doesn’t give us very helpful information when it fails and doesn’t store the line and column on which each expression starts, which might be helpful when reporting errors later, but it’s good enough for our purposes. ## [](http://eloquentjavascript.net/11_language.html#h_HlYkF24q0q)The evaluator [](http://eloquentjavascript.net/11_language.html#p_NVE0gYIhuH)What can we do with the syntax tree for a program? Run it, of course! And that is what the evaluator does. You give it a syntax tree and an environment object that associates names with values, and it will evaluate the expression that the tree represents and return the value that this produces. ~~~ function evaluate(expr, env) { switch(expr.type) { case "value": return expr.value; case "word": if (expr.name in env) return env[expr.name]; else throw new ReferenceError("Undefined variable: " + expr.name); case "apply": if (expr.operator.type == "word" && expr.operator.name in specialForms) return specialForms[expr.operator.name](expr.args, env); var op = evaluate(expr.operator, env); if (typeof op != "function") throw new TypeError("Applying a non-function."); return op.apply(null, expr.args.map(function(arg) { return evaluate(arg, env); })); } } var specialForms = Object.create(null); ~~~ [](http://eloquentjavascript.net/11_language.html#p_BBPjU16P+4)The evaluator has code for each of the expression types. A literal value expression simply produces its value. (For example, the expression `100` just evaluates to the number 100.) For a variable, we must check whether it is actually defined in the environment and, if it is, fetch the variable’s value. [](http://eloquentjavascript.net/11_language.html#p_3O0koXQqgW)Applications are more involved. If they are a special form, like `if`, we do not evaluate anything and simply pass the argument expressions, along with the environment, to the function that handles this form. If it is a normal call, we evaluate the operator, verify that it is a function, and call it with the result of evaluating the arguments. [](http://eloquentjavascript.net/11_language.html#p_pq4tp0rkgU)We will use plain JavaScript function values to represent Egg’s function values. We will come back to this [later](http://eloquentjavascript.net/11_language.html#egg_fun), when the special form called `fun` is defined. [](http://eloquentjavascript.net/11_language.html#p_VAS/FPg1UY)The recursive structure of `evaluate` resembles the similar structure of the parser. Both mirror the structure of the language itself. It would also be possible to integrate the parser with the evaluator and evaluate during parsing, but splitting them up this way makes the program more readable. [](http://eloquentjavascript.net/11_language.html#p_lfbXYqM2w2)This is really all that is needed to interpret Egg. It is that simple. But without defining a few special forms and adding some useful values to the environment, you can’t do anything with this language yet. ## [](http://eloquentjavascript.net/11_language.html#h_JOCrYKZbDr)Special forms [](http://eloquentjavascript.net/11_language.html#p_Vq0TKNCoCk)The `specialForms` object is used to define special syntax in Egg. It associates words with functions that evaluate such special forms. It is currently empty. Let’s add some forms. ~~~ specialForms["if"] = function(args, env) { if (args.length != 3) throw new SyntaxError("Bad number of args to if"); if (evaluate(args[0], env) !== false) return evaluate(args[1], env); else return evaluate(args[2], env); }; ~~~ [](http://eloquentjavascript.net/11_language.html#p_1G1ypvHVnk)Egg’s `if` construct expects exactly three arguments. It will evaluate the first, and if the result isn’t the value `false`, it will evaluate the second. Otherwise, the third gets evaluated. This `if` form is more similar to JavaScript’s ternary`?:` operator than to JavaScript’s `if`. It is an expression, not a statement, and it produces a value, namely, the result of the second or third argument. [](http://eloquentjavascript.net/11_language.html#p_NF+mSlruEr)Egg differs from JavaScript in how it handles the condition value to `if`. It will not treat things like zero or the empty string as false, but only the precise value`false`. [](http://eloquentjavascript.net/11_language.html#p_ARUuidAS4I)The reason we need to represent `if` as a special form, rather than a regular function, is that all arguments to functions are evaluated before the function is called, whereas `if` should evaluate only *either* its second or its third argument, depending on the value of the first. [](http://eloquentjavascript.net/11_language.html#p_Z67vCfZJi8)The `while` form is similar. ~~~ specialForms["while"] = function(args, env) { if (args.length != 2) throw new SyntaxError("Bad number of args to while"); while (evaluate(args[0], env) !== false) evaluate(args[1], env); // Since undefined does not exist in Egg, we return false, // for lack of a meaningful result. return false; }; ~~~ [](http://eloquentjavascript.net/11_language.html#p_jN94KP2C2B)Another basic building block is `do`, which executes all its arguments from top to bottom. Its value is the value produced by the last argument. ~~~ specialForms["do"] = function(args, env) { var value = false; args.forEach(function(arg) { value = evaluate(arg, env); }); return value; }; ~~~ [](http://eloquentjavascript.net/11_language.html#p_C6o3zSjPya)To be able to create variables and give them new values, we also create a form called `define`. It expects a word as its first argument and an expression producing the value to assign to that word as its second argument. Since`define`, like everything, is an expression, it must return a value. We’ll make it return the value that was assigned (just like JavaScript’s `=` operator). ~~~ specialForms["define"] = function(args, env) { if (args.length != 2 || args[0].type != "word") throw new SyntaxError("Bad use of define"); var value = evaluate(args[1], env); env[args[0].name] = value; return value; }; ~~~ ## [](http://eloquentjavascript.net/11_language.html#h_2Tc54fkIgF)The environment [](http://eloquentjavascript.net/11_language.html#p_hpUlK1YNqm)The environment accepted by `evaluate` is an object with properties whose names correspond to variable names and whose values correspond to the values those variables are bound to. Let’s define an environment object to represent the global scope. [](http://eloquentjavascript.net/11_language.html#p_DNbLZKiE8S)To be able to use the `if` construct we just defined, we must have access to Boolean values. Since there are only two Boolean values, we do not need special syntax for them. We simply bind two variables to the values `true` and `false`and use those. ~~~ var topEnv = Object.create(null); topEnv["true"] = true; topEnv["false"] = false; ~~~ [](http://eloquentjavascript.net/11_language.html#p_mLmm584R4a)We can now evaluate a simple expression that negates a Boolean value. ~~~ var prog = parse("if(true, false, true)"); console.log(evaluate(prog, topEnv)); // → false ~~~ [](http://eloquentjavascript.net/11_language.html#p_ngLfcnLh/2)To supply basic arithmetic and comparison operators, we will also add some function values to the environment. In the interest of keeping the code short, we’ll use `new Function` to synthesize a bunch of operator functions in a loop, rather than defining them all individually. ~~~ ["+", "-", "*", "/", "==", "<", ">"].forEach(function(op) { topEnv[op] = new Function("a, b", "return a " + op + " b;"); }); ~~~ [](http://eloquentjavascript.net/11_language.html#p_ZP9ZMJAskW)A way to output values is also very useful, so we’ll wrap `console.log` in a function and call it `print`. ~~~ topEnv["print"] = function(value) { console.log(value); return value; }; ~~~ [](http://eloquentjavascript.net/11_language.html#p_LD0lewrgTF)That gives us enough elementary tools to write simple programs. The following`run` function provides a convenient way to write and run them. It creates a fresh environment and parses and evaluates the strings we give it as a single program. ~~~ function run() { var env = Object.create(topEnv); var program = Array.prototype.slice .call(arguments, 0).join("\n"); return evaluate(parse(program), env); } ~~~ [](http://eloquentjavascript.net/11_language.html#p_L3eaessI8L)The use of `Array.prototype.slice.call` is a trick to turn an array-like object, such as `arguments`, into a real array so that we can call `join` on it. It takes all the arguments given to `run` and treats them as the lines of a program. ~~~ run("do(define(total, 0),", " define(count, 1),", " while(<(count, 11),", " do(define(total, +(total, count)),", " define(count, +(count, 1)))),", " print(total))"); // → 55 ~~~ [](http://eloquentjavascript.net/11_language.html#p_4cQhF2ypgW)This is the program we’ve seen several times before, which computes the sum of the numbers 1 to 10, expressed in Egg. It is clearly uglier than the equivalent JavaScript program but not bad for a language implemented in less than 150 lines of code. ## [](http://eloquentjavascript.net/11_language.html#h_K5Yd6h3Axg)Functions [](http://eloquentjavascript.net/11_language.html#p_FabFt4VUPD)A programming language without functions is a poor programming language indeed. [](http://eloquentjavascript.net/11_language.html#p_wOCI3bG7/f)Fortunately, it is not hard to add a `fun` construct, which treats its last argument as the function’s body and treats all the arguments before that as the names of the function’s arguments. ~~~ specialForms["fun"] = function(args, env) { if (!args.length) throw new SyntaxError("Functions need a body"); function name(expr) { if (expr.type != "word") throw new SyntaxError("Arg names must be words"); return expr.name; } var argNames = args.slice(0, args.length - 1).map(name); var body = args[args.length - 1]; return function() { if (arguments.length != argNames.length) throw new TypeError("Wrong number of arguments"); var localEnv = Object.create(env); for (var i = 0; i < arguments.length; i++) localEnv[argNames[i]] = arguments[i]; return evaluate(body, localEnv); }; }; ~~~ [](http://eloquentjavascript.net/11_language.html#p_iTINb2vdSn)Functions in Egg have their own local environment, just like in JavaScript. We use `Object.create` to make a new object that has access to the variables in the outer environment (its prototype) but that can also contain new variables without modifying that outer scope. [](http://eloquentjavascript.net/11_language.html#p_8s1BSoLJ7h)The function created by the `fun` form creates this local environment and adds the argument variables to it. It then evaluates the function body in this environment and returns the result. ~~~ run("do(define(plusOne, fun(a, +(a, 1))),", " print(plusOne(10)))"); // → 11 run("do(define(pow, fun(base, exp,", " if(==(exp, 0),", " 1,", " *(base, pow(base, -(exp, 1)))))),", " print(pow(2, 10)))"); // → 1024 ~~~ ## [](http://eloquentjavascript.net/11_language.html#h_qtdV3kKVQe)Compilation [](http://eloquentjavascript.net/11_language.html#p_JRcPOXDFEi)What we have built is an interpreter. During evaluation, it acts directly on the representation of the program produced by the parser. [](http://eloquentjavascript.net/11_language.html#p_4qkDFya9A2)*Compilation* is the process of adding another step between the parsing and the running of a program, which transforms the program into something that can be evaluated more efficiently by doing as much work as possible in advance. For example, in well-designed languages it is obvious, for each use of a variable, which variable is being referred to, without actually running the program. This can be used to avoid looking up the variable by name every time it is accessed and to directly fetch it from some predetermined memory location. [](http://eloquentjavascript.net/11_language.html#p_HrQe0PuoCJ)Traditionally, compilation involves converting the program to machine code, the raw format that a computer’s processor can execute. But any process that converts a program to a different representation can be thought of as compilation. [](http://eloquentjavascript.net/11_language.html#p_+HfDnbf6cY)It would be possible to write an alternative evaluation strategy for Egg, one that first converts the program to a JavaScript program, uses `new Function` to invoke the JavaScript compiler on it, and then runs the result. When done right, this would make Egg run very fast while still being quite simple to implement. [](http://eloquentjavascript.net/11_language.html#p_PmaubaTjuT)If you are interested in this topic and willing to spend some time on it, I encourage you to try to implement such a compiler as an exercise. ## [](http://eloquentjavascript.net/11_language.html#h_DmDK0dWdfE)Cheating [](http://eloquentjavascript.net/11_language.html#p_cUrg/8XPK+)When we defined `if` and `while`, you probably noticed that they were more or less trivial wrappers around JavaScript’s own `if` and `while`. Similarly, the values in Egg are just regular old JavaScript values. [](http://eloquentjavascript.net/11_language.html#p_qMvm2xpt6o)If you compare the implementation of Egg, built on top of JavaScript, with the amount of work and complexity required to build a programming language directly on the raw functionality provided by a machine, the difference is huge. Regardless, this example hopefully gave you an impression of the way programming languages work. [](http://eloquentjavascript.net/11_language.html#p_xuQu2Hdpx2)And when it comes to getting something done, cheating is more effective than doing everything yourself. Though the toy language in this chapter doesn’t do anything that couldn’t be done better in JavaScript, there *are* situations where writing small languages helps get real work done. [](http://eloquentjavascript.net/11_language.html#p_WIR6a1g/pa)Such a language does not have to resemble a typical programming language. If JavaScript didn’t come equipped with regular expressions, you could write your own parser and evaluator for such a sublanguage. [](http://eloquentjavascript.net/11_language.html#p_j8SqlNkYB/)Or imagine you are building a giant robotic dinosaur and need to program its behavior. JavaScript might not be the most effective way to do this. You might instead opt for a language that looks like this: ~~~ behavior walk perform when destination ahead actions move left-foot move right-foot behavior attack perform when Godzilla in-view actions fire laser-eyes launch arm-rockets ~~~ [](http://eloquentjavascript.net/11_language.html#p_O9+2Ve51P4)This is what is usually called a *domain-specific language*, a language tailored to express a narrow domain of knowledge. Such a language can be more expressive than a general-purpose language because it is designed to express exactly the things that need expressing in its domain and nothing else. ## [](http://eloquentjavascript.net/11_language.html#h_TcUD2vzyMe)Exercises ### [](http://eloquentjavascript.net/11_language.html#h_uQzJv9I1Z6)Arrays [](http://eloquentjavascript.net/11_language.html#p_4rt40yX+cU)Add support for arrays to Egg by adding the following three functions to the top scope: `array(...)` to construct an array containing the argument values,`length(array)` to get an array’s length, and `element(array, n)` to fetch the nth element from an array. ~~~ // Modify these definitions... topEnv["array"] = "..."; topEnv["length"] = "..."; topEnv["element"] = "..."; run("do(define(sum, fun(array,", " do(define(i, 0),", " define(sum, 0),", " while(<(i, length(array)),", " do(define(sum, +(sum, element(array, i))),", " define(i, +(i, 1)))),", " sum))),", " print(sum(array(1, 2, 3))))"); // → 6 ~~~ ### [](http://eloquentjavascript.net/11_language.html#h_hOd+yVxaku)Closure [](http://eloquentjavascript.net/11_language.html#p_v1H/2L3I5F)The way we have defined `fun` allows functions in Egg to “close over” the surrounding environment, allowing the function’s body to use local values that were visible at the time the function was defined, just like JavaScript functions do. [](http://eloquentjavascript.net/11_language.html#p_kjSkKHf16k)The following program illustrates this: function `f` returns a function that adds its argument to `f`'s argument, meaning that it needs access to the local scope inside `f` to be able to use variable `a`. ~~~ run("do(define(f, fun(a, fun(b, +(a, b)))),", " print(f(4)(5)))"); // → 9 ~~~ [](http://eloquentjavascript.net/11_language.html#p_40Yj7LMkYl)Go back to the definition of the `fun` form and explain which mechanism causes this to work. ### [](http://eloquentjavascript.net/11_language.html#h_/OBuIOX390)Comments [](http://eloquentjavascript.net/11_language.html#p_qV4w3Ov/ee)It would be nice if we could write comments in Egg. For example, whenever we find a hash sign (`#`), we could treat the rest of the line as a comment and ignore it, similar to `//` in JavaScript. [](http://eloquentjavascript.net/11_language.html#p_EVGdU2vHLH)We do not have to make any big changes to the parser to support this. We can simply change `skipSpace` to skip comments like they are whitespace so that all the points where `skipSpace` is called will now also skip comments. Make this change. ~~~ // This is the old skipSpace. Modify it... function skipSpace(string) { var first = string.search(/\S/); if (first == -1) return ""; return string.slice(first); } console.log(parse("# hello\nx")); // → {type: "word", name: "x"} console.log(parse("a # one\n # two\n()")); // → {type: "apply", // operator: {type: "word", name: "a"}, // args: []} ~~~ ### [](http://eloquentjavascript.net/11_language.html#h_Y9ZDMshYCQ)Fixing scope [](http://eloquentjavascript.net/11_language.html#p_wiUendOwjA)Currently, the only way to assign a variable a value is `define`. This construct acts as a way both to define new variables and to give existing ones a new value. [](http://eloquentjavascript.net/11_language.html#p_O06fqfa6CU)This ambiguity causes a problem. When you try to give a nonlocal variable a new value, you will end up defining a local one with the same name instead. (Some languages work like this by design, but I’ve always found it a silly way to handle scope.) [](http://eloquentjavascript.net/11_language.html#p_wG0OsCIoJb)Add a special form `set`, similar to `define`, which gives a variable a new value, updating the variable in an outer scope if it doesn’t already exist in the inner scope. If the variable is not defined at all, throw a `ReferenceError` (which is another standard error type). [](http://eloquentjavascript.net/11_language.html#p_vI0CMB91dl)The technique of representing scopes as simple objects, which has made things convenient so far, will get in your way a little at this point. You might want to use the `Object.getPrototypeOf` function, which returns the prototype of an object. Also remember that scopes do not derive from `Object.prototype`, so if you want to call `hasOwnProperty` on them, you have to use this clumsy expression: ~~~ Object.prototype.hasOwnProperty.call(scope, name); ~~~ [](http://eloquentjavascript.net/11_language.html#p_xdKe490o5g)This fetches the `hasOwnProperty` method from the `Object` prototype and then calls it on a scope object. ~~~ specialForms["set"] = function(args, env) { // Your code here. }; run("do(define(x, 4),", " define(setx, fun(val, set(x, val))),", " setx(50),", " print(x))"); // → 50 run("set(quux, true)"); // → Some kind of ReferenceError ~~~ > You will have to loop through one scope at a time, using Object.getPrototypeOf to go the next outer scope. For each scope, use hasOwnProperty to find out whether the variable, indicated by the name property of the first argument to set, exists in that scope. If it does, set it to the result of evaluating the second argument to set and then return that value. > If the outermost scope is reached (Object.getPrototypeOf returns null) and we haven’t found the variable yet, it doesn’t exist, and an error should be thrown.

## Chapter 10 # Modules > [](http://eloquentjavascript.net/10_modules.html#p_PJiK2g1JXC)A beginning programmer writes her programs like an ant builds her hill, one piece at a time, without thought for the bigger structure. Her programs will be like loose sand. They may stand for a while, but growing too big they fall apart. > > [](http://eloquentjavascript.net/10_modules.html#p_OfZWeuo+oF)Realizing this problem, the programmer will start to spend a lot of time thinking about structure. Her programs will be rigidly structured, like rock sculptures. They are solid, but when they must change, violence must be done to them. > > [](http://eloquentjavascript.net/10_modules.html#p_fFMEGH1dk3)The master programmer knows when to apply structure and when to leave things in their simple form. Her programs are like clay, solid yet malleable. > > Master Yuan-Ma, The Book of Programming [](http://eloquentjavascript.net/10_modules.html#p_1aMUBkVbPd)Every program has a shape. On a small scale, this shape is determined by its division into functions and the blocks inside those functions. Programmers have a lot of freedom in the way they structure their programs. Shape follows more from the taste of the programmer than from the program’s intended functionality. [](http://eloquentjavascript.net/10_modules.html#p_NOETmtvWWx)When looking at a larger program in its entirety, individual functions start to blend into the background. Such a program can be made more readable if we have a larger unit of organization. [](http://eloquentjavascript.net/10_modules.html#p_07zdYJYQ1M)*Modules* divide programs into clusters of code that, by *some* criterion, belong together. This chapter explores some of the benefits that such division provides and shows techniques for building modules in JavaScript. ## [](http://eloquentjavascript.net/10_modules.html#h_0DSW4+9YVh)Why modules help [](http://eloquentjavascript.net/10_modules.html#p_ovuaejexnJ)There are a number of reasons why authors divide their books into chapters and sections. These divisions make it easier for a reader to see how the book is built up and to find specific parts that they are interested in. They also help the*author* by providing a clear focus for every section. [](http://eloquentjavascript.net/10_modules.html#p_QxCtWUR2zJ)The benefits of organizing a program into several files or modules are similar. Structure helps people who aren’t yet familiar with the code find what they are looking for and makes it easier for the programmer to keep things that are related close together. [](http://eloquentjavascript.net/10_modules.html#p_4vnN7lCDZy)Some programs are even organized along the model of a traditional text, with a well-defined order in which the reader is encouraged to go through the program and with lots of prose (comments) providing a coherent description of the code. This makes reading the program a lot less intimidating—reading unknown code is usually intimidating—but has the downside of being more work to set up. It also makes the program more difficult to change because prose tends to be more tightly interconnected than code. This style is called*literate programming*. The “project” chapters of this book can be considered literate programs. [](http://eloquentjavascript.net/10_modules.html#p_Ou/d4KbIy/)As a general rule, structuring things costs energy. In the early stages of a project, when you are not quite sure yet what goes where or what kind of modules the program needs at all, I endorse a minimalist, structureless attitude. Just put everything wherever it is convenient to put it until the code stabilizes. That way, you won’t be wasting time moving pieces of the program back and forth, and you won’t accidentally lock yourself into a structure that does not actually fit your program. ### [](http://eloquentjavascript.net/10_modules.html#h_NitCO6r9Hn)Namespacing [](http://eloquentjavascript.net/10_modules.html#p_8S/+Hon3uA)Most modern programming languages have a scope level between *global*(everyone can see it) and *local* (only this function can see it). JavaScript does not. Thus, by default, everything that needs to be visible outside of the scope of a top-level function is visible *everywhere*. [](http://eloquentjavascript.net/10_modules.html#p_QMB7RIsBay)Namespace pollution, the problem of a lot of unrelated code having to share a single set of global variable names, was mentioned in [Chapter 4](http://eloquentjavascript.net/04_data.html#namespace_pollution), where the`Math` object was given as an example of an object that acts like a module by grouping math-related functionality. [](http://eloquentjavascript.net/10_modules.html#p_bBKNVkrywA)Though JavaScript provides no actual module construct yet, objects can be used to create publicly accessible subnamespaces, and functions can be used to create an isolated, private namespace inside of a module. Later in this chapter, I will discuss a way to build reasonably convenient, namespace-isolating modules on top of the primitive concepts that JavaScript gives us. ### [](http://eloquentjavascript.net/10_modules.html#h_UuxJID5Z2e)Reuse [](http://eloquentjavascript.net/10_modules.html#p_0MwF7DbAP0)In a “flat” project, which isn’t structured as a set of modules, it is not apparent which parts of the code are needed to use a particular function. In my program for spying on my enemies (see [Chapter 9](http://eloquentjavascript.net/09_regexp.html#ini)), I wrote a function for reading configuration files. If I want to use that function in another project, I must go and copy out the parts of the old program that look like they are relevant to the functionality that I need and paste them into my new program. Then, if I find a mistake in that code, I’ll fix it only in whichever program that I’m working with at the time and forget to also fix it in the other program. [](http://eloquentjavascript.net/10_modules.html#p_n5Itkf6SsY)Once you have lots of such shared, duplicated pieces of code, you will find yourself wasting a lot of time and energy on moving them around and keeping them up-to-date. [](http://eloquentjavascript.net/10_modules.html#p_zwgHmRoxk9)Putting pieces of functionality that stand on their own into separate files and modules makes them easier to track, update, and share because all the various pieces of code that want to use the module load it from the same actual file. [](http://eloquentjavascript.net/10_modules.html#p_D2yYhnbA9K)This idea gets even more powerful when the relations between modules—which other modules each module depends on—are explicitly stated. You can then automate the process of installing and upgrading external modules (*libraries*). [](http://eloquentjavascript.net/10_modules.html#p_3ED8vjm59N)Taking this idea even further, imagine an online service that tracks and distributes hundreds of thousands of such libraries, allowing you to search for the functionality you need and, once you find it, set up your project to automatically download it. [](http://eloquentjavascript.net/10_modules.html#p_8hHGZUvmtE)This service exists. It is called NPM ([*npmjs.org*](http://npmjs.org/)). NPM consists of an online database of modules and a tool for downloading and upgrading the modules your program depends on. It grew out of Node.js, the browserless JavaScript environment we will discuss in [Chapter 20](http://eloquentjavascript.net/20_node.html#node), but can also be useful when programming for the browser. ### [](http://eloquentjavascript.net/10_modules.html#h_qJacYNwWWL)Decoupling [](http://eloquentjavascript.net/10_modules.html#p_BIr3LxVodJ)Another important role of modules is isolating pieces of code from each other, in the same way that the object interfaces from [Chapter 6](http://eloquentjavascript.net/06_object.html#interface) do. A well-designed module will provide an interface for external code to use. As the module gets updated with bug fixes and new functionality, the existing interface stays the same (it is *stable*) so that other modules can use the new, improved version without any changes to themselves. [](http://eloquentjavascript.net/10_modules.html#p_B3JCtLjLbo)Note that a stable interface does not mean no new functions, methods, or variables are added. It just means that existing functionality isn’t removed and its meaning is not changed. [](http://eloquentjavascript.net/10_modules.html#p_vcubl5qb7a)A good module interface should allow the module to grow without breaking the old interface. This means exposing as few of the module’s internal concepts as possible while also making the “language” that the interface exposes powerful and flexible enough to be applicable in a wide range of situations. [](http://eloquentjavascript.net/10_modules.html#p_jT10dBy1PO)For interfaces that expose a single, focused concept, such as a configuration file reader, this design comes naturally. For others, such as a text editor, which has many different aspects that external code might need to access (content, styling, user actions, and so on), it requires careful design. ## [](http://eloquentjavascript.net/10_modules.html#h_FIe+oM6Cvt)Using functions as namespaces [](http://eloquentjavascript.net/10_modules.html#p_Qi0v4Dtc05)Functions are the only things in JavaScript that create a new scope. So if we want our modules to have their own scope, we will have to base them on functions. [](http://eloquentjavascript.net/10_modules.html#p_TJhIvzABX+)Consider this trivial module for associating names with day-of-the-week numbers, as returned by a `Date` object’s `getDay` method: ~~~ var names = ["Sunday", "Monday", "Tuesday", "Wednesday", "Thursday", "Friday", "Saturday"]; function dayName(number) { return names[number]; } console.log(dayName(1)); // → Monday ~~~ [](http://eloquentjavascript.net/10_modules.html#p_lljN2eMCck)The `dayName` function is part of the module’s interface, but the `names` variable is not. We would prefer *not* to spill it into the global scope. [](http://eloquentjavascript.net/10_modules.html#p_NKmpNxgsFg)We can do this: ~~~ var dayName = function() { var names = ["Sunday", "Monday", "Tuesday", "Wednesday", "Thursday", "Friday", "Saturday"]; return function(number) { return names[number]; }; }(); console.log(dayName(3)); // → Wednesday ~~~ [](http://eloquentjavascript.net/10_modules.html#p_wFyqiGP0o5)Now `names` is a local variable in an (unnamed) function. This function is created and immediately called, and its return value (the actual `dayName`function) is stored in a variable. We could have pages and pages of code in this function, with 100 local variables, and they would all be internal to our module—visible to the module itself but not to outside code. [](http://eloquentjavascript.net/10_modules.html#p_XMMYXbTqGr)We can use a similar pattern to isolate code from the outside world entirely. The following module logs a value to the console but does not actually provide any values for other modules to use: ~~~ (function() { function square(x) { return x * x; } var hundred = 100; console.log(square(hundred)); })(); // → 10000 ~~~ [](http://eloquentjavascript.net/10_modules.html#p_NhLZMeErEJ)This code simply outputs the square of 100, but in the real world it could be a module that adds a method to some prototype or sets up a widget on a web page. It is wrapped in a function to prevent the variables it uses internally from polluting the global scope. [](http://eloquentjavascript.net/10_modules.html#p_TMPi6xxW7Y)Why did we wrap the namespace function in a pair of parentheses? This has to do with a quirk in JavaScript’s syntax. If an *expression* starts with the keyword`function`, it is a function expression. However, if a *statement* starts with`function`, it is a function *declaration*, which requires a name and, not being an expression, cannot be called by writing parentheses after it. You can think of the extra wrapping parentheses as a trick to force the function to be interpreted as an expression. ## [](http://eloquentjavascript.net/10_modules.html#h_tbrMjRcajd)Objects as interfaces [](http://eloquentjavascript.net/10_modules.html#p_jaxyyZ13ux)Now imagine that we want to add another function to our day-of-the-week module, one that goes from a day name to a number. We can’t simply return the function anymore but must wrap the two functions in an object. ~~~ var weekDay = function() { var names = ["Sunday", "Monday", "Tuesday", "Wednesday", "Thursday", "Friday", "Saturday"]; return { name: function(number) { return names[number]; }, number: function(name) { return names.indexOf(name); } }; }(); console.log(weekDay.name(weekDay.number("Sunday"))); // → Sunday ~~~ [](http://eloquentjavascript.net/10_modules.html#p_67L/Eo8MtL)For bigger modules, gathering all the *exported* values into an object at the end of the function becomes awkward since many of the exported functions are likely to be big and you’d prefer to write them somewhere else, near related internal code. A convenient alternative is to declare an object (conventionally named `exports`) and add properties to that whenever we are defining something that needs to be exported. In the following example, the module function takes its interface object as an argument, allowing code outside of the function to create it and store it in a variable. (Outside of a function, `this`refers to the global scope object.) ~~~ (function(exports) { var names = ["Sunday", "Monday", "Tuesday", "Wednesday", "Thursday", "Friday", "Saturday"]; exports.name = function(number) { return names[number]; }; exports.number = function(name) { return names.indexOf(name); }; })(this.weekDay = {}); console.log(weekDay.name(weekDay.number("Saturday"))); // → Saturday ~~~ ## [](http://eloquentjavascript.net/10_modules.html#h_ryZfYeSnJy)Detaching from the global scope [](http://eloquentjavascript.net/10_modules.html#p_AfCtO+ifwZ)The previous pattern is commonly used by JavaScript modules intended for the browser. The module will claim a single global variable and wrap its code in a function in order to have its own private namespace. But this pattern still causes problems if multiple modules happen to claim the same name or if you want to load two versions of a module alongside each other. [](http://eloquentjavascript.net/10_modules.html#p_Y2ejGOe+u4)With a little plumbing, we can create a system that allows one module to directly ask for the interface object of another module, without going through the global scope. Our goal is a `require` function that, when given a module name, will load that module’s file (from disk or the Web, depending on the platform we are running on) and return the appropriate interface value. [](http://eloquentjavascript.net/10_modules.html#p_6BOb5Vhxc7)This approach solves the problems mentioned previously and has the added benefit of making your program’s dependencies explicit, making it harder to accidentally make use of some module without stating that you need it. [](http://eloquentjavascript.net/10_modules.html#p_qsfTCs+Kiu)For `require` we need two things. First, we want a function `readFile`, which returns the content of a given file as a string. (A single such function is not present in standard JavaScript, but different JavaScript environments, such as the browser and Node.js, provide their own ways of accessing files. For now, let’s just pretend we have this function.) Second, we need to be able to actually execute this string as JavaScript code. ## [](http://eloquentjavascript.net/10_modules.html#h_oeOkEDaadU)Evaluating data as code [](http://eloquentjavascript.net/10_modules.html#p_jvij1au4eP)There are several ways to take data (a string of code) and run it as part of the current program. [](http://eloquentjavascript.net/10_modules.html#p_sQ1ASk7W7E)The most obvious way is the special operator `eval`, which will execute a string of code in the *current* scope. This is usually a bad idea because it breaks some of the sane properties that scopes normally have, such as being isolated from the outside world. ~~~ function evalAndReturnX(code) { eval(code); return x; } console.log(evalAndReturnX("var x = 2")); // → 2 ~~~ [](http://eloquentjavascript.net/10_modules.html#p_dJLBESQexx)A better way of interpreting data as code is to use the `Function` constructor. This takes two arguments: a string containing a comma-separated list of argument names and a string containing the function’s body. ~~~ var plusOne = new Function("n", "return n + 1;"); console.log(plusOne(4)); // → 5 ~~~ [](http://eloquentjavascript.net/10_modules.html#p_8DGWg5jk7v)This is precisely what we need for our modules. We can wrap a module’s code in a function, with that function’s scope becoming our module scope. ## [](http://eloquentjavascript.net/10_modules.html#h_v/XE3QWFpP)Require [](http://eloquentjavascript.net/10_modules.html#p_hpfd8nPIk6)The following is a minimal implementation of `require`: ~~~ function require(name) { var code = new Function("exports", readFile(name)); var exports = {}; code(exports); return exports; } console.log(require("weekDay").name(1)); // → Monday ~~~ [](http://eloquentjavascript.net/10_modules.html#p_JrILpUgcrH)Since the `new Function` constructor wraps the module code in a function, we don’t have to write a wrapping namespace function in the module file itself. And since we make `exports` an argument to the module function, the module does not have to declare it. This removes a lot of clutter from our example module. ~~~ var names = ["Sunday", "Monday", "Tuesday", "Wednesday", "Thursday", "Friday", "Saturday"]; exports.name = function(number) { return names[number]; }; exports.number = function(name) { return names.indexOf(name); }; ~~~ [](http://eloquentjavascript.net/10_modules.html#p_B75dGyOs9V)When using this pattern, a module typically starts with a few variable declarations that load the modules it depends on. ~~~ var weekDay = require("weekDay"); var today = require("today"); console.log(weekDay.name(today.dayNumber())); ~~~ [](http://eloquentjavascript.net/10_modules.html#p_5GfcRhlMLC)The simplistic implementation of `require` given previously has several problems. For one, it will load and run a module every time it is `require`d, so if several modules have the same dependency or a `require` call is put inside a function that will be called multiple times, time and energy will be wasted. [](http://eloquentjavascript.net/10_modules.html#p_zLUwyIBMDV)This can be solved by storing the modules that have already been loaded in an object and simply returning the existing value when one is loaded multiple times. [](http://eloquentjavascript.net/10_modules.html#p_Ylx5bX6l+o)The second problem is that it is not possible for a module to directly export a value other than the `exports` object, such as a function. For example, a module might want to export only the constructor of the object type it defines. Right now, it cannot do that because `require` always uses the `exports` object it creates as the exported value. [](http://eloquentjavascript.net/10_modules.html#p_RwxY5rq2Qj)The traditional solution for this is to provide modules with another variable,`module`, which is an object that has a property `exports`. This property initially points at the empty object created by `require` but can be overwritten with another value in order to export something else. ~~~ function require(name) { if (name in require.cache) return require.cache[name]; var code = new Function("exports, module", readFile(name)); var exports = {}, module = {exports: exports}; code(exports, module); require.cache[name] = module.exports; return module.exports; } require.cache = Object.create(null); ~~~ [](http://eloquentjavascript.net/10_modules.html#p_Dv+TulRay7)We now have a module system that uses a single global variable (`require`) to allow modules to find and use each other without going through the global scope. [](http://eloquentjavascript.net/10_modules.html#p_rvZ9T+1c9y)This style of module system is called *CommonJS modules*, after the pseudo-standard that first specified it. It is built into the Node.js system. Real implementations do a lot more than the example I showed. Most importantly, they have a much more intelligent way of going from a module name to an actual piece of code, allowing both pathnames relative to the current file and module names that point directly to locally installed modules. ## [](http://eloquentjavascript.net/10_modules.html#h_A4LDSlASUo)Slow-loading modules [](http://eloquentjavascript.net/10_modules.html#p_isvKkfyGNv)Though it is possible to use the CommonJS module style when writing JavaScript for the browser, it is somewhat involved. The reason for this is that reading a file (module) from the Web is a lot slower than reading it from the hard disk. While a script is running in the browser, nothing else can happen to the website on which it runs, for reasons that will become clear in [Chapter 14](http://eloquentjavascript.net/14_event.html#timeline). This means that if every `require` call went and fetched something from some faraway web server, the page would freeze for a painfully long time while loading its scripts. [](http://eloquentjavascript.net/10_modules.html#p_Ce7X8+/o7v)One way to work around this problem is to run a program like [*Browserify*](http://browserify.org/) on your code before you serve it on a web page. This will look for calls to `require`, resolve all dependencies, and gather the needed code into a single big file. The website itself can simply load this file to get all the modules it needs. [](http://eloquentjavascript.net/10_modules.html#p_L9zjrOe2Un)Another solution is to wrap the code that makes up your module in a function so that the module loader can first load its dependencies in the background and then call the function, initializing the module, when the dependencies have been loaded. That is what the Asynchronous Module Definition (AMD) module system does. [](http://eloquentjavascript.net/10_modules.html#p_69ZNnLLNhP)Our trivial program with dependencies would look like this in AMD: ~~~ define(["weekDay", "today"], function(weekDay, today) { console.log(weekDay.name(today.dayNumber())); }); ~~~ [](http://eloquentjavascript.net/10_modules.html#p_/aD387r6Uv)The `define` function is central to this approach. It takes first an array of module names and then a function that takes one argument for each dependency. It will load the dependencies (if they haven’t already been loaded) in the background, allowing the page to continue working while the files are being fetched. Once all dependencies are loaded, `define` will call the function it was given, with the interfaces of those dependencies as arguments. [](http://eloquentjavascript.net/10_modules.html#p_w7/Slpc0XV)The modules that are loaded this way must themselves contain a call to`define`. The value used as their interface is whatever was returned by the function passed to `define`. Here is the `weekDay` module again: ~~~ define([], function() { var names = ["Sunday", "Monday", "Tuesday", "Wednesday", "Thursday", "Friday", "Saturday"]; return { name: function(number) { return names[number]; }, number: function(name) { return names.indexOf(name); } }; }); ~~~ [](http://eloquentjavascript.net/10_modules.html#p_GARnQl+u0i)To be able to show a minimal implementation of `define`, we will pretend we have a `backgroundReadFile` function that takes a filename and a function and calls the function with the content of the file as soon as it has finished loading it. ([Chapter 17](http://eloquentjavascript.net/17_http.html#getURL) will explain how to write that function.) [](http://eloquentjavascript.net/10_modules.html#p_xNPCNpRiIX)For the purpose of keeping track of modules while they are being loaded, the implementation of `define` will use objects that describe the state of modules, telling us whether they are available yet and providing their interface when they are. [](http://eloquentjavascript.net/10_modules.html#p_/Unllb03Bv)The `getModule` function, when given a name, will return such an object and ensure that the module is scheduled to be loaded. It uses a cache object to avoid loading the same module twice. ~~~ var defineCache = Object.create(null); var currentMod = null; function getModule(name) { if (name in defineCache) return defineCache[name]; var module = {exports: null, loaded: false, onLoad: []}; defineCache[name] = module; backgroundReadFile(name, function(code) { currentMod = module; new Function("", code)(); }); return module; } ~~~ [](http://eloquentjavascript.net/10_modules.html#p_r9NqByjA7c)We assume the loaded file also contains a (single) call to `define`. The`currentMod` variable is used to tell this call about the module object that is currently being loaded so that it can update this object when it finishes loading. We will come back to this mechanism in a moment. [](http://eloquentjavascript.net/10_modules.html#p_lX37OjlA5W)The `define` function itself uses `getModule` to fetch or create the module objects for the current module’s dependencies. Its task is to schedule the`moduleFunction` (the function that contains the module’s actual code) to be run whenever those dependencies are loaded. For this purpose, it defines a function `whenDepsLoaded` that is added to the `onLoad` array of all dependencies that are not yet loaded. This function immediately returns if there are still unloaded dependencies, so it will do actual work only once, when the last dependency has finished loading. It is also called immediately, from`define` itself, in case there are no dependencies that need to be loaded. ~~~ function define(depNames, moduleFunction) { var myMod = currentMod; var deps = depNames.map(getModule); deps.forEach(function(mod) { if (!mod.loaded) mod.onLoad.push(whenDepsLoaded); }); function whenDepsLoaded() { if (!deps.every(function(m) { return m.loaded; })) return; var args = deps.map(function(m) { return m.exports; }); var exports = moduleFunction.apply(null, args); if (myMod) { myMod.exports = exports; myMod.loaded = true; myMod.onLoad.forEach(function(f) { f(); }); } } whenDepsLoaded(); } ~~~ [](http://eloquentjavascript.net/10_modules.html#p_T+RexzI7A0)When all dependencies are available, `whenDepsLoaded` calls the function that holds the module, giving it the dependencies’ interfaces as arguments. [](http://eloquentjavascript.net/10_modules.html#p_8Neb8ba81Q)The first thing `define` does is store the value that `currentMod` had when it was called in a variable `myMod`. Remember that `getModule`, just before evaluating the code for a module, stored the corresponding module object in`currentMod`. This allows `whenDepsLoaded` to store the return value of the module function in that module’s `exports` property, set the module’s `loaded`property to true, and call all the functions that are waiting for the module to load. [](http://eloquentjavascript.net/10_modules.html#p_KBR+VgwyVz)This code is a lot harder to follow than the `require` function. Its execution does not follow a simple, predictable path. Instead, multiple operations are set up to happen at some unspecified time in the future, which obscures the way the code executes. [](http://eloquentjavascript.net/10_modules.html#p_uewjoUxlIw)A real AMD implementation is, again, quite a lot more clever about resolving module names to actual URLs and generally more robust than the one shown previously. The *RequireJS* ([*requirejs.org*](http://requirejs.org/)) project provides a popular implementation of this style of module loader. ## [](http://eloquentjavascript.net/10_modules.html#h_4266RdeHYR)Interface design [](http://eloquentjavascript.net/10_modules.html#p_IUZ7uTWwwd)Designing interfaces for modules and object types is one of the subtler aspects of programming. Any nontrivial piece of functionality can be modeled in various ways. Finding a way that works well requires insight and foresight. [](http://eloquentjavascript.net/10_modules.html#p_KweUtWCb7f)The best way to learn the value of good interface design is to use lots of interfaces—some good, some bad. Experience will teach you what works and what doesn’t. Never assume that a painful interface is “just the way it is”. Fix it, or wrap it in a new interface that works better for you. ### [](http://eloquentjavascript.net/10_modules.html#h_D19q+FXBVd)Predictability [](http://eloquentjavascript.net/10_modules.html#p_2adqTW0fxP)If programmers can predict the way your interface works, they (or you) won’t get sidetracked as often by the need to look up how to use it. Thus, try to follow conventions. When there is another module or part of the standard JavaScript environment that does something similar to what you are implementing, it might be a good idea to make your interface resemble the existing interface. That way, it’ll feel familiar to people who know the existing interface. [](http://eloquentjavascript.net/10_modules.html#p_6l6v5cakoT)Another area where predictability is important is the actual *behavior* of your code. It can be tempting to make an unnecessarily clever interface with the justification that it’s more convenient to use. For example, you could accept all kinds of different types and combinations of arguments and do the “right thing” for all of them. Or you could provide dozens of specialized convenience functions that provide slightly different flavors of your module’s functionality. These might make code that builds on your interface slightly shorter, but they will also make it much harder for people to build a clear mental model of the module’s behavior. ### [](http://eloquentjavascript.net/10_modules.html#h_+NeFt8aXxf)Composability [](http://eloquentjavascript.net/10_modules.html#p_9Nt/796F0G)In your interfaces, try to use the simplest data structures possible and make functions do a single, clear thing. Whenever practical, make them pure functions (see [Chapter 3](http://eloquentjavascript.net/03_functions.html#pure)). [](http://eloquentjavascript.net/10_modules.html#p_qicKwwcIoO)For example, it is not uncommon for modules to provide their own array-like collection objects, with their own interface for counting and extracting elements. Such objects won’t have `map` or `forEach` methods, and any existing function that expects a real array won’t be able to work with them. This is an example of poor *composability*—the module cannot be easily composed with other code. [](http://eloquentjavascript.net/10_modules.html#p_XJefBK7ici)One example would be a module for spell-checking text, which we might need when we want to write a text editor. The spell-checker could be made to operate directly on whichever complicated data structures the editor uses and directly call internal functions in the editor to have the user choose between spelling suggestions. If we go that way, the module cannot be used with any other programs. On the other hand, if we define the spell-checking interface so that you can pass it a simple string and it will return the position in the string where it found a possible misspelling, along with an array of suggested corrections, then we have an interface that could also be composed with other systems because strings and arrays are always available in JavaScript. ### [](http://eloquentjavascript.net/10_modules.html#h_vguoN2JlWc)Layered interfaces [](http://eloquentjavascript.net/10_modules.html#p_gVRzuTdJxo)When designing an interface for a complex piece of functionality—sending email, for example—you often run into a dilemma. On the one hand, you do not want to overload the user of your interface with details. They shouldn’t have to study your interface for 20 minutes before they can send an email. On the other hand, you do not want to hide all the details either—when people need to do complicated things with your module, they should be able to. [](http://eloquentjavascript.net/10_modules.html#p_UAqdlcM0M6)Often the solution is to provide two interfaces: a detailed *low-level* one for complex situations and a simple *high-level* one for routine use. The second can usually be built easily using the tools provided by the first. In the email module, the high-level interface could just be a function that takes a message, a sender address, and a receiver address and then sends the email. The low-level interface would allow full control over email headers, attachments, HTML mail, and so on. ## [](http://eloquentjavascript.net/10_modules.html#h_ErccPg/l98)Summary [](http://eloquentjavascript.net/10_modules.html#p_hGwRhRocwV)Modules provide structure to bigger programs by separating the code into different files and namespaces. Giving these modules well-defined interfaces makes them easier to use and reuse and makes it possible to continue using them as the module itself evolves. [](http://eloquentjavascript.net/10_modules.html#p_G2usOklg33)Though the JavaScript language is characteristically unhelpful when it comes to modules, the flexible functions and objects it provides make it possible to define rather nice module systems. Function scopes can be used as internal namespaces for the module, and objects can be used to store sets of exported values. [](http://eloquentjavascript.net/10_modules.html#p_0LAk7BPgRr)There are two popular, well-defined approaches to such modules. One is called*CommonJS Modules* and revolves around a `require` function that fetches a module by name and returns its interface. The other is called *AMD* and uses a`define` function that takes an array of module names and a function and, after loading the modules, runs the function with their interfaces as arguments. ## [](http://eloquentjavascript.net/10_modules.html#h_TcUD2vzyMe)Exercises ### [](http://eloquentjavascript.net/10_modules.html#h_Ft63zbY95i)Month names [](http://eloquentjavascript.net/10_modules.html#p_dU0pnoSPR9)Write a simple module similar to the `weekDay` module that can convert month numbers (zero-based, as in the `Date` type) to names and can convert names back to numbers. Give it its own namespace since it will need an internal array of month names, and use plain JavaScript, without any module loader system. ~~~ // Your code here. console.log(month.name(2)); // → March console.log(month.number("November")); // → 10 ~~~ ### [](http://eloquentjavascript.net/10_modules.html#h_3j/KaVGNV2)A return to electronic life [](http://eloquentjavascript.net/10_modules.html#p_+royQ52RxC)Hoping that [Chapter 7](http://eloquentjavascript.net/07_elife.html#elife) is still somewhat fresh in your mind, think back to the system designed in that chapter and come up with a way to separate the code into modules. To refresh your memory, these are the functions and types defined in that chapter, in order of appearance: ~~~ Vector Grid directions directionNames randomElement BouncingCritter elementFromChar World charFromElement Wall View WallFollower dirPlus LifelikeWorld Plant PlantEater SmartPlantEater Tiger ~~~ [](http://eloquentjavascript.net/10_modules.html#p_qH0Xj8v2nH)Don’t exaggerate and create too many modules. A book that starts a new chapter for every page would probably get on your nerves, if only because of all the space wasted on titles. Similarly, having to open 10 files to read a tiny project isn’t helpful. Aim for three to five modules. [](http://eloquentjavascript.net/10_modules.html#p_fDzWqQqyw8)You can choose to have some functions become internal to their module and thus inaccessible to other modules. [](http://eloquentjavascript.net/10_modules.html#p_XYYo0/YDOl)There is no single correct solution here. Module organization is largely a matter of taste. ### [](http://eloquentjavascript.net/10_modules.html#h_E/zWqBFdy8)Circular dependencies [](http://eloquentjavascript.net/10_modules.html#p_/LVNV6+2SH)A tricky subject in dependency management is circular dependencies, where module A depends on B, and B also depends on A. Many module systems simply forbid this. CommonJS modules allow a limited form: it works as long as the modules do not replace their default `exports` object with another value and start accessing each other’s interface only after they finish loading. [](http://eloquentjavascript.net/10_modules.html#p_BFm3QczrDZ)Can you think of a way in which support for this feature could be implemented? Look back to the definition of `require` and consider what the function would have to do to allow this. > The trick is to add the exports object created for a module to require's cache before actually running the module. This means the module will not yet have had a chance to override module.exports, so we do not know whether it may want to export some other value. After loading, the cache object is overridden with module.exports, which may be a different value. > But if in the course of loading the module, a second module is loaded that asks for the first module, its default exports object, which is likely still empty at this point, will be in the cache, and the second module will receive a reference to it. If it doesn’t try to do anything with the object until the first module has finished loading, things will work.

## Chapter 9 # Regular Expressions > [](http://eloquentjavascript.net/09_regexp.html#p_MWUwIAb0uO)Some people, when confronted with a problem, think ‘I know, I’ll use regular expressions.’ Now they have two problems. > > Jamie Zawinski > [](http://eloquentjavascript.net/09_regexp.html#p_icxlw7+18l)Yuan-Ma said, ‘When you cut against the grain of the wood, much strength is needed. When you program against the grain of a problem, much code is needed.’ > > Master Yuan-Ma, The Book of Programming [](http://eloquentjavascript.net/09_regexp.html#p_81b6macd9j)Programming tools and techniques survive and spread in a chaotic, evolutionary way. It’s not always the pretty or brilliant ones that win but rather the ones that function well enough within the right niche—for example, by being integrated with another successful piece of technology. [](http://eloquentjavascript.net/09_regexp.html#p_2ySNBCvXNS)In this chapter, I will discuss one such tool, *regular expressions*. Regular expressions are a way to describe patterns in string data. They form a small, separate language that is part of JavaScript and many other languages and tools. [](http://eloquentjavascript.net/09_regexp.html#p_cxbejyPUGl)Regular expressions are both terribly awkward and extremely useful. Their syntax is cryptic, and the programming interface JavaScript provides for them is clumsy. But they are a powerful tool for inspecting and processing strings. Properly understanding regular expressions will make you a more effective programmer. ## [](http://eloquentjavascript.net/09_regexp.html#h_5w4yGFJRYl)Creating a regular expression [](http://eloquentjavascript.net/09_regexp.html#p_u/9SKAI2Yi)A regular expression is a type of object. It can either be constructed with the`RegExp` constructor or written as a literal value by enclosing the pattern in forward slash (`/`) characters. ~~~ var re1 = new RegExp("abc"); var re2 = /abc/; ~~~ [](http://eloquentjavascript.net/09_regexp.html#p_uNMQxzr01n)Both of these regular expression objects represent the same pattern: an *a*character followed by a *b* followed by a *c*. [](http://eloquentjavascript.net/09_regexp.html#p_qv8UWLVrTv)When using the `RegExp` constructor, the pattern is written as a normal string, so the usual rules apply for backslashes. [](http://eloquentjavascript.net/09_regexp.html#p_xB91PDBGKz)The second notation, where the pattern appears between slash characters, treats backslashes somewhat differently. First, since a forward slash ends the pattern, we need to put a backslash before any forward slash that we want to be*part* of the pattern. In addition, backslashes that aren’t part of special character codes (like `\n`) will be *preserved*, rather than ignored as they are in strings, and change the meaning of the pattern. Some characters, such as question marks and plus signs, have special meanings in regular expressions and must be preceded by a backslash if they are meant to represent the character itself. ~~~ var eighteenPlus = /eighteen\+/; ~~~ [](http://eloquentjavascript.net/09_regexp.html#p_GYUjU5Z/Gb)Knowing precisely what characters to backslash-escape when writing regular expressions requires you to know every character with a special meaning. For the time being, this may not be realistic, so when in doubt, just put a backslash before any character that is not a letter, number, or whitespace. ## [](http://eloquentjavascript.net/09_regexp.html#h_vPyyYjMEtz)Testing for matches [](http://eloquentjavascript.net/09_regexp.html#p_0vbMMWCatE)Regular expression objects have a number of methods. The simplest one is`test`. If you pass it a string, it will return a Boolean telling you whether the string contains a match of the pattern in the expression. ~~~ console.log(/abc/.test("abcde")); // → true console.log(/abc/.test("abxde")); // → false ~~~ [](http://eloquentjavascript.net/09_regexp.html#p_WFokgkkjHh)A regular expression consisting of only nonspecial characters simply represents that sequence of characters. If *abc* occurs anywhere in the string we are testing against (not just at the start), `test` will return `true`. ## [](http://eloquentjavascript.net/09_regexp.html#h_z4ZvUsIpEG)Matching a set of characters [](http://eloquentjavascript.net/09_regexp.html#p_DtXmb/uhsO)Finding out whether a string contains *abc* could just as well be done with a call to `indexOf`. Regular expressions allow us to go beyond that and express more complicated patterns. [](http://eloquentjavascript.net/09_regexp.html#p_i/99SEfu9y)Say we want to match any number. In a regular expression, putting a set of characters between square brackets makes that part of the expression match any of the characters between the brackets. [](http://eloquentjavascript.net/09_regexp.html#p_sC+2E08KnL)Both of the following expressions match all strings that contain a digit: ~~~ console.log(/[0123456789]/.test("in 1992")); // → true console.log(/[0-9]/.test("in 1992")); // → true ~~~ [](http://eloquentjavascript.net/09_regexp.html#p_i0WYLVUede)Within square brackets, a dash (`-`) between two characters can be used to indicate a range of characters, where the ordering is determined by the character’s Unicode number. Characters 0 to 9 sit right next to each other in this ordering (codes 48 to 57), so `[0-9]` covers all of them and matches any digit. [](http://eloquentjavascript.net/09_regexp.html#p_+w4wbahJfo)There are a number of common character groups that have their own built-in shortcuts. Digits are one of them: `\d` means the same thing as `[0-9]`. | `\d` | Any digit character | | `\w` | An alphanumeric character (“word character”) | | `\s` | Any whitespace character (space, tab, newline, and similar) | | `\D` | A character that is *not* a digit | | `\W` | A nonalphanumeric character | | `\S` | A nonwhitespace character | | `.` | Any character except for newline | [](http://eloquentjavascript.net/09_regexp.html#p_yXMUKEYpwG)So you could match a date and time format like 30-01-2003 15:20 with the following expression: ~~~ var dateTime = /\d\d-\d\d-\d\d\d\d \d\d:\d\d/; console.log(dateTime.test("30-01-2003 15:20")); // → true console.log(dateTime.test("30-jan-2003 15:20")); // → false ~~~ [](http://eloquentjavascript.net/09_regexp.html#p_NjgwK/FZRp)That looks completely awful, doesn’t it? It has way too many backslashes, producing background noise that makes it hard to spot the actual pattern expressed. We’ll see a slightly improved version of this expression [later](http://eloquentjavascript.net/09_regexp.html#date_regexp_counted). [](http://eloquentjavascript.net/09_regexp.html#p_P0qAMYu0C/)These backslash codes can also be used inside square brackets. For example,`[\d.]` means any digit or a period character. But note that the period itself, when used between square brackets, loses its special meaning. The same goes for other special characters, such as `+`. [](http://eloquentjavascript.net/09_regexp.html#p_HqQEZsitdl)To *invert* a set of characters—that is, to express that you want to match any character *except* the ones in the set—you can write a caret (`^`) character after the opening bracket. ~~~ var notBinary = /[^01]/; console.log(notBinary.test("1100100010100110")); // → false console.log(notBinary.test("1100100010200110")); // → true ~~~ ## [](http://eloquentjavascript.net/09_regexp.html#h_iFI1qvUwY9)Repeating parts of a pattern [](http://eloquentjavascript.net/09_regexp.html#p_crYiu/oAUM)We now know how to match a single digit. What if we want to match a whole number—a sequence of one or more digits? [](http://eloquentjavascript.net/09_regexp.html#p_B4wupHzbR+)When you put a plus sign (`+`) after something in a regular expression, it indicates that the element may be repeated more than once. Thus, `/\d+/`matches one or more digit characters. ~~~ console.log(/'\d+'/.test("'123'")); // → true console.log(/'\d+'/.test("''")); // → false console.log(/'\d*'/.test("'123'")); // → true console.log(/'\d*'/.test("''")); // → true ~~~ [](http://eloquentjavascript.net/09_regexp.html#p_/oNBIVm41F)The star (`*`) has a similar meaning but also allows the pattern to match zero times. Something with a star after it never prevents a pattern from matching—it’ll just match zero instances if it can’t find any suitable text to match. [](http://eloquentjavascript.net/09_regexp.html#p_rtXbvQjQpG)A question mark makes a part of a pattern “optional”, meaning it may occur zero or one time. In the following example, the *u* character is allowed to occur, but the pattern also matches when it is missing. ~~~ var neighbor = /neighbou?r/; console.log(neighbor.test("neighbour")); // → true console.log(neighbor.test("neighbor")); // → true ~~~ [](http://eloquentjavascript.net/09_regexp.html#p_B4ikd8xN8i)To indicate that a pattern should occur a precise number of times, use curly braces. Putting `{4}` after an element, for example, requires it to occur exactly four times. It is also possible to specify a range this way: `{2,4}` means the element must occur at least twice and at most four times. [](http://eloquentjavascript.net/09_regexp.html#p_awR6bPYROd)Here is another version of the date and time pattern that allows both single- and double-digit days, months, and hours. It is also slightly more readable. ~~~ var dateTime = /\d{1,2}-\d{1,2}-\d{4} \d{1,2}:\d{2}/; console.log(dateTime.test("30-1-2003 8:45")); // → true ~~~ [](http://eloquentjavascript.net/09_regexp.html#p_eVBFhX2vD9)You can also specify open-ended ranges when using curly braces by omitting the number on either side of the comma. So `{,5}` means zero to five times, and`{5,}` means five or more times. ## [](http://eloquentjavascript.net/09_regexp.html#h_uICSDspz1I)Grouping subexpressions [](http://eloquentjavascript.net/09_regexp.html#p_pKTOYUDGIr)To use an operator like `*` or `+` on more than one element at a time, you can use parentheses. A part of a regular expression that is enclosed in parentheses counts as a single element as far as the operators following it are concerned. ~~~ var cartoonCrying = /boo+(hoo+)+/i; console.log(cartoonCrying.test("Boohoooohoohooo")); // → true ~~~ [](http://eloquentjavascript.net/09_regexp.html#p_S5jkv2dMC+)The first and second `+` characters apply only to the second *o* in *boo* and *hoo*, respectively. The third `+` applies to the whole group `(hoo+)`, matching one or more sequences like that. [](http://eloquentjavascript.net/09_regexp.html#p_tFxtn8Pxg1)The `i` at the end of the expression in the previous example makes this regular expression case insensitive, allowing it to match the uppercase *B* in the input string, even though the pattern is itself all lowercase. ## [](http://eloquentjavascript.net/09_regexp.html#h_CV5XL/TADP)Matches and groups [](http://eloquentjavascript.net/09_regexp.html#p_K4AlKPLvyA)The `test` method is the absolute simplest way to match a regular expression. It tells you only whether it matched and nothing else. Regular expressions also have an `exec` (execute) method that will return `null` if no match was found and return an object with information about the match otherwise. ~~~ var match = /\d+/.exec("one two 100"); console.log(match); // → ["100"] console.log(match.index); // → 8 ~~~ [](http://eloquentjavascript.net/09_regexp.html#p_fJSwbQyG6w)An object returned from `exec` has an `index` property that tells us *where* in the string the successful match begins. Other than that, the object looks like (and in fact is) an array of strings, whose first element is the string that was matched—in the previous example, this is the sequence of digits that we were looking for. [](http://eloquentjavascript.net/09_regexp.html#p_VT4fpht7D7)String values have a `match` method that behaves similarly. ~~~ console.log("one two 100".match(/\d+/)); // → ["100"] ~~~ [](http://eloquentjavascript.net/09_regexp.html#p_fFGmBROT0i)When the regular expression contains subexpressions grouped with parentheses, the text that matched those groups will also show up in the array. The whole match is always the first element. The next element is the part matched by the first group (the one whose opening parenthesis comes first in the expression), then the second group, and so on. ~~~ var quotedText = /'([^']*)'/; console.log(quotedText.exec("she said 'hello'")); // → ["'hello'", "hello"] ~~~ [](http://eloquentjavascript.net/09_regexp.html#p_f4bciMASJ1)When a group does not end up being matched at all (for example, when followed by a question mark), its position in the output array will hold`undefined`. Similarly, when a group is matched multiple times, only the last match ends up in the array. ~~~ console.log(/bad(ly)?/.exec("bad")); // → ["bad", undefined] console.log(/(\d)+/.exec("123")); // → ["123", "3"] ~~~ [](http://eloquentjavascript.net/09_regexp.html#p_zGhL5UaDaO)Groups can be useful for extracting parts of a string. If we don’t just want to verify whether a string contains a date but also extract it and construct an object that represents it, we can wrap parentheses around the digit patterns and directly pick the date out of the result of `exec`. [](http://eloquentjavascript.net/09_regexp.html#p_B9SEqDbr+Y)But first, a brief detour, in which we discuss the preferred way to store date and time values in JavaScript. ## [](http://eloquentjavascript.net/09_regexp.html#h_+jaxcLjxpL)The date type [](http://eloquentjavascript.net/09_regexp.html#p_2NeTRvucQq)JavaScript has a standard object type for representing dates—or rather, points in time. It is called `Date`. If you simply create a date object using `new`, you get the current date and time. ~~~ console.log(new Date()); // → Wed Dec 04 2013 14:24:57 GMT+0100 (CET) ~~~ [](http://eloquentjavascript.net/09_regexp.html#p_IcV7kv3B1y)You can also create an object for a specific time. ~~~ console.log(new Date(2009, 11, 9)); // → Wed Dec 09 2009 00:00:00 GMT+0100 (CET) console.log(new Date(2009, 11, 9, 12, 59, 59, 999)); // → Wed Dec 09 2009 12:59:59 GMT+0100 (CET) ~~~ [](http://eloquentjavascript.net/09_regexp.html#p_3P1Nh5Y+3T)JavaScript uses a convention where month numbers start at zero (so December is 11), yet day numbers start at one. This is confusing and silly. Be careful. [](http://eloquentjavascript.net/09_regexp.html#p_gVdQSb0Lv9)The last four arguments (hours, minutes, seconds, and milliseconds) are optional and taken to be zero when not given. [](http://eloquentjavascript.net/09_regexp.html#p_1mIMU5T5MA)Timestamps are stored as the number of milliseconds since the start of 1970, using negative numbers for times before 1970 (following a convention set by “Unix time”, which was invented around that time). The `getTime` method on a date object returns this number. It is big, as you can imagine. ~~~ console.log(new Date(2013, 11, 19).getTime()); // → 1387407600000 console.log(new Date(1387407600000)); // → Thu Dec 19 2013 00:00:00 GMT+0100 (CET) ~~~ [](http://eloquentjavascript.net/09_regexp.html#p_bU5mnU7KSw)If you give the `Date` constructor a single argument, that argument is treated as such a millisecond count. You can get the current millisecond count by creating a new `Date` object and calling `getTime` on it but also by calling the `Date.now`function. [](http://eloquentjavascript.net/09_regexp.html#p_OMuHPQoSyh)Date objects provide methods like `getFullYear`, `getMonth`, `getDate`,`getHours`, `getMinutes`, and `getSeconds` to extract their components. There’s also `getYear`, which gives you a rather useless two-digit year value (such as `93`or `14`). [](http://eloquentjavascript.net/09_regexp.html#p_/RCtQyD3w/)Putting parentheses around the parts of the expression that we are interested in, we can now easily create a date object from a string. ~~~ function findDate(string) { var dateTime = /(\d{1,2})-(\d{1,2})-(\d{4})/; var match = dateTime.exec(string); return new Date(Number(match[3]), Number(match[2]) - 1, Number(match[1])); } console.log(findDate("30-1-2003")); // → Thu Jan 30 2003 00:00:00 GMT+0100 (CET) ~~~ ## [](http://eloquentjavascript.net/09_regexp.html#h_26ixny78VY)Word and string boundaries [](http://eloquentjavascript.net/09_regexp.html#p_xmknsh+RBC)Unfortunately, `findDate` will also happily extract the nonsensical date 00-1-3000 from the string `"100-1-30000"`. A match may happen anywhere in the string, so in this case, it’ll just start at the second character and end at the second-to-last character. [](http://eloquentjavascript.net/09_regexp.html#p_tKA0/BuA/v)If we want to enforce that the match must span the whole string, we can add the markers `^` and `$`. The caret matches the start of the input string, while the dollar sign matches the end. So, `/^\d+$/` matches a string consisting entirely of one or more digits, `/^!/` matches any string that starts with an exclamation mark, and `/x^/` does not match any string (there cannot be an *x* before the start of the string). [](http://eloquentjavascript.net/09_regexp.html#p_dWIgppHuON)If, on the other hand, we just want to make sure the date starts and ends on a word boundary, we can use the marker `\b`. A word boundary can be the start or end of the string or any point in the string that has a word character (as in `\w`) on one side and a nonword character on the other. ~~~ console.log(/cat/.test("concatenate")); // → true console.log(/\bcat\b/.test("concatenate")); // → false ~~~ [](http://eloquentjavascript.net/09_regexp.html#p_btxd6luedx)Note that a boundary marker doesn’t represent an actual character. It just enforces that the regular expression matches only when a certain condition holds at the place where it appears in the pattern. ## [](http://eloquentjavascript.net/09_regexp.html#h_In3b+t6uOO)Choice patterns [](http://eloquentjavascript.net/09_regexp.html#p_G5RTt0AFku)Say we want to know whether a piece of text contains not only a number but a number followed by one of the words *pig*, *cow*, or *chicken*, or any of their plural forms. [](http://eloquentjavascript.net/09_regexp.html#p_GcEbQJT+nS)We could write three regular expressions and test them in turn, but there is a nicer way. The pipe character (`|`) denotes a choice between the pattern to its left and the pattern to its right. So I can say this: ~~~ var animalCount = /\b\d+ (pig|cow|chicken)s?\b/; console.log(animalCount.test("15 pigs")); // → true console.log(animalCount.test("15 pigchickens")); // → false ~~~ [](http://eloquentjavascript.net/09_regexp.html#p_+vG2bBcznO)Parentheses can be used to limit the part of the pattern that the pipe operator applies to, and you can put multiple such operators next to each other to express a choice between more than two patterns. ## [](http://eloquentjavascript.net/09_regexp.html#h_AzxCBCKdvY)The mechanics of matching [](http://eloquentjavascript.net/09_regexp.html#p_fSpOFXMKY0)Regular expressions can be thought of as flow diagrams. This is the diagram for the livestock expression in the previous example:  [](http://eloquentjavascript.net/09_regexp.html#p_SNiUdMyezk)Our expression matches a string if we can find a path from the left side of the diagram to the right side. We keep a current position in the string, and every time we move through a box, we verify that the part of the string after our current position matches that box. [](http://eloquentjavascript.net/09_regexp.html#p_MB99a8uIlE)So if we try to match `"the 3 pigs"` with our regular expression, our progress through the flow chart would look like this: * [](http://eloquentjavascript.net/09_regexp.html#p_uBRT2kCvr2) At position 4, there is a word boundary, so we can move past the first box. * [](http://eloquentjavascript.net/09_regexp.html#p_nO+Bi7hXkR) Still at position 4, we find a digit, so we can also move past the second box. * [](http://eloquentjavascript.net/09_regexp.html#p_fQdWHxKgCF) At position 5, one path loops back to before the second (digit) box, while the other moves forward through the box that holds a single space character. There is a space here, not a digit, so we must take the second path. * [](http://eloquentjavascript.net/09_regexp.html#p_KItk5iNp9m) We are now at position 6 (the start of “pigs”) and at the three-way branch in the diagram. We don’t see “cow” or “chicken” here, but we do see “pig”, so we take that branch. * [](http://eloquentjavascript.net/09_regexp.html#p_SowlGZC6lM) At position 9, after the three-way branch, one path skips the *s* box and goes straight to the final word boundary, while the other path matches an*s*. There is an *s* character here, not a word boundary, so we go through the*s* box. * [](http://eloquentjavascript.net/09_regexp.html#p_jtkHSJd1Eg) We’re at position 10 (the end of the string) and can match only a word boundary. The end of a string counts as a word boundary, so we go through the last box and have successfully matched this string. [](http://eloquentjavascript.net/09_regexp.html#p_dfwI9Viw95)Conceptually, a regular expression engine looks for a match in a string as follows: it starts at the start of the string and tries a match there. In this case, there *is* a word boundary there, so it’d get past the first box—but there is no digit, so it’d fail at the second box. Then it moves on to the second character in the string and tries to begin a new match there... and so on, until it finds a match or reaches the end of the string and decides that there really is no match. ## [](http://eloquentjavascript.net/09_regexp.html#h_NFMtGK0tD3)Backtracking [](http://eloquentjavascript.net/09_regexp.html#p_tCd15MFAty)The regular expression `/\b([01]+b|\d+|[\da-f]+h)\b/` matches either a binary number followed by a *b*, a regular decimal number with no suffix character, or a hexadecimal number (that is, base 16, with the letters *a* to *f*standing for the digits 10 to 15) followed by an *h*. This is the corresponding diagram:  [](http://eloquentjavascript.net/09_regexp.html#p_CkxEfo2iHH)When matching this expression, it will often happen that the top (binary) branch is entered even though the input does not actually contain a binary number. When matching the string `"103"`, for example, it becomes clear only at the 3 that we are in the wrong branch. The string *does* match the expression, just not the branch we are currently in. [](http://eloquentjavascript.net/09_regexp.html#p_SjTCKE9hvf)So the matcher *backtracks*. When entering a branch, it remembers its current position (in this case, at the start of the string, just past the first boundary box in the diagram) so that it can go back and try another branch if the current one does not work out. For the string `"103"`, after encountering the 3 character, it will start trying the branch for decimal numbers. This one matches, so a match is reported after all. [](http://eloquentjavascript.net/09_regexp.html#p_+Hy4DF1l6V)The matcher stops as soon as it finds a full match. This means that if multiple branches could potentially match a string, only the first one (ordered by where the branches appear in the regular expression) is used. [](http://eloquentjavascript.net/09_regexp.html#p_zEBIV8lYeb)Backtracking also happens for repetition operators like + and `*`. If you match`/^.*x/` against `"abcxe"`, the `.*` part will first try to consume the whole string. The engine will then realize that it needs an *x* to match the pattern. Since there is no *x* past the end of the string, the star operator tries to match one character less. But the matcher doesn’t find an *x* after `abcx` either, so it backtracks again, matching the star operator to just `abc`. *Now* it finds an *x* where it needs it and reports a successful match from positions 0 to 4. [](http://eloquentjavascript.net/09_regexp.html#p_VxCrsg7UEp)It is possible to write regular expressions that will do a *lot* of backtracking. This problem occurs when a pattern can match a piece of input in many different ways. For example, if we get confused while writing a binary-number regular expression, we might accidentally write something like `/([01]+)+b/`.  [](http://eloquentjavascript.net/09_regexp.html#p_5cI0Ma3Wy8)If that tries to match some long series of zeros and ones with no trailing *b*character, the matcher will first go through the inner loop until it runs out of digits. Then it notices there is no *b*, so it backtracks one position, goes through the outer loop once, and gives up again, trying to backtrack out of the inner loop once more. It will continue to try every possible route through these two loops. This means the amount of work *doubles* with each additional character. For even just a few dozen characters, the resulting match will take practically forever. ## [](http://eloquentjavascript.net/09_regexp.html#h_k0YuTOu54D)The replace method [](http://eloquentjavascript.net/09_regexp.html#p_HMQv5qrs78)String values have a `replace` method, which can be used to replace part of the string with another string. ~~~ console.log("papa".replace("p", "m")); // → mapa ~~~ [](http://eloquentjavascript.net/09_regexp.html#p_jjBKX9l81o)The first argument can also be a regular expression, in which case the first match of the regular expression is replaced. When a `g` option (for *global*) is added to the regular expression, *all* matches in the string will be replaced, not just the first. ~~~ console.log("Borobudur".replace(/[ou]/, "a")); // → Barobudur console.log("Borobudur".replace(/[ou]/g, "a")); // → Barabadar ~~~ [](http://eloquentjavascript.net/09_regexp.html#p_k90SO6OTUP)It would have been sensible if the choice between replacing one match or all matches was made through an additional argument to `replace` or by providing a different method, `replaceAll`. But for some unfortunate reason, the choice relies on a property of the regular expression instead. [](http://eloquentjavascript.net/09_regexp.html#p_/5YU/Qo2Np)The real power of using regular expressions with `replace` comes from the fact that we can refer back to matched groups in the replacement string. For example, say we have a big string containing the names of people, one name per line, in the format `Lastname, Firstname`. If we want to swap these names and remove the comma to get a simple `Firstname Lastname` format, we can use the following code: ~~~ console.log( "Hopper, Grace\nMcCarthy, John\nRitchie, Dennis" .replace(/([\w ]+), ([\w ]+)/g, "$2 $1")); // → Grace Hopper // John McCarthy // Dennis Ritchie ~~~ [](http://eloquentjavascript.net/09_regexp.html#p_yMNsWvr4Lh)The `$1` and `$2` in the replacement string refer to the parenthesized groups in the pattern. `$1` is replaced by the text that matched against the first group, `$2`by the second, and so on, up to `$9`. The whole match can be referred to with`$&`. [](http://eloquentjavascript.net/09_regexp.html#p_vI+NtinnDZ)It is also possible to pass a function, rather than a string, as the second argument to `replace`. For each replacement, the function will be called with the matched groups (as well as the whole match) as arguments, and its return value will be inserted into the new string. [](http://eloquentjavascript.net/09_regexp.html#p_3oXsRdfp6a)Here’s a simple example: ~~~ var s = "the cia and fbi"; console.log(s.replace(/\b(fbi|cia)\b/g, function(str) { return str.toUpperCase(); })); // → the CIA and FBI ~~~ [](http://eloquentjavascript.net/09_regexp.html#p_EbMXoAagyr)And here’s a more interesting one: ~~~ var stock = "1 lemon, 2 cabbages, and 101 eggs"; function minusOne(match, amount, unit) { amount = Number(amount) - 1; if (amount == 1) // only one left, remove the 's' unit = unit.slice(0, unit.length - 1); else if (amount == 0) amount = "no"; return amount + " " + unit; } console.log(stock.replace(/(\d+) (\w+)/g, minusOne)); // → no lemon, 1 cabbage, and 100 eggs ~~~ [](http://eloquentjavascript.net/09_regexp.html#p_bv4e/DVilz)This takes a string, finds all occurrences of a number followed by an alphanumeric word, and returns a string wherein every such occurrence is decremented by one. [](http://eloquentjavascript.net/09_regexp.html#p_pEPUrWK5UJ)The `(\d+)` group ends up as the `amount` argument to the function, and the`(\w+)` group gets bound to `unit`. The function converts `amount` to a number—which always works, since it matched `\d+`—and makes some adjustments in case there is only one or zero left. ## [](http://eloquentjavascript.net/09_regexp.html#h_kiECehz+i+)Greed [](http://eloquentjavascript.net/09_regexp.html#p_Q5tdrWs0Tm)It isn’t hard to use `replace` to write a function that removes all comments from a piece of JavaScript code. Here is a first attempt: ~~~ function stripComments(code) { return code.replace(/\/\/.*|\/\*[^]*\*\//g, ""); } console.log(stripComments("1 + /* 2 */3")); // → 1 + 3 console.log(stripComments("x = 10;// ten!")); // → x = 10; console.log(stripComments("1 /* a */+/* b */ 1")); // → 1 1 ~~~ [](http://eloquentjavascript.net/09_regexp.html#p_6+ZTC2M5ci)The part before the *or* operator simply matches two slash characters followed by any number of non-newline characters. The part for multiline comments is more involved. We use `[^]` (any character that is not in the empty set of characters) as a way to match any character. We cannot just use a dot here because block comments can continue on a new line, and dots do not match the newline character. [](http://eloquentjavascript.net/09_regexp.html#p_s9E9JYjAYp)But the output of the previous example appears to have gone wrong. Why? [](http://eloquentjavascript.net/09_regexp.html#p_atS1ERkauC)The `[^]*` part of the expression, as I described in the section on backtracking, will first match as much as it can. If that causes the next part of the pattern to fail, the matcher moves back one character and tries again from there. In the example, the matcher first tries to match the whole rest of the string and then moves back from there. It will find an occurrence of `*/` after going back four characters and match that. This is not what we wanted—the intention was to match a single comment, not to go all the way to the end of the code and find the end of the last block comment. [](http://eloquentjavascript.net/09_regexp.html#p_eNtLSVH65f)Because of this behavior, we say the repetition operators (`+`, `*`, `?`, and `{}`) are*greedy*, meaning they match as much as they can and backtrack from there. If you put a question mark after them (`+?`, `*?`, `??`, `{}?`), they become nongreedy and start by matching as little as possible, matching more only when the remaining pattern does not fit the smaller match. [](http://eloquentjavascript.net/09_regexp.html#p_0L47KZXZKa)And that is exactly what we want in this case. By having the star match the smallest stretch of characters that brings us to a `*/`, we consume one block comment and nothing more. ~~~ function stripComments(code) { return code.replace(/\/\/.*|\/\*[^]*?\*\//g, ""); } console.log(stripComments("1 /* a */+/* b */ 1")); // → 1 + 1 ~~~ [](http://eloquentjavascript.net/09_regexp.html#p_DQjgIV+cP6)A lot of bugs in regular expression programs can be traced to unintentionally using a greedy operator where a nongreedy one would work better. When using a repetition operator, consider the nongreedy variant first. ## [](http://eloquentjavascript.net/09_regexp.html#h_Rhu25fogrG)Dynamically creating RegExp objects [](http://eloquentjavascript.net/09_regexp.html#p_2A78iOXDCO)There are cases where you might not know the exact pattern you need to match against when you are writing your code. Say you want to look for the user’s name in a piece of text and enclose it in underscore characters to make it stand out. Since you will know the name only once the program is actually running, you can’t use the slash-based notation. [](http://eloquentjavascript.net/09_regexp.html#p_fuOCNTjCbk)But you can build up a string and use the `RegExp` constructor on that. Here’s an example: ~~~ var name = "harry"; var text = "Harry is a suspicious character."; var regexp = new RegExp("\\b(" + name + ")\\b", "gi"); console.log(text.replace(regexp, "_$1_")); // → _Harry_ is a suspicious character. ~~~ [](http://eloquentjavascript.net/09_regexp.html#p_J6H1NBoQy/)When creating the `\b` boundary markers, we have to use two backslashes because we are writing them in a normal string, not a slash-enclosed regular expression. The second argument to the `RegExp` constructor contains the options for the regular expression—in this case `"gi"` for global and case-insensitive. [](http://eloquentjavascript.net/09_regexp.html#p_W/atxcKoEp)But what if the name is `"dea+hl[]rd"` because our user is a nerdy teenager? That would result in a nonsensical regular expression, which won’t actually match the user’s name. [](http://eloquentjavascript.net/09_regexp.html#p_0Mx4Lf/w4/)To work around this, we can add backslashes before any character that we don’t trust. Adding backslashes before alphabetic characters is a bad idea because things like `\b` and `\n` have a special meaning. But escaping everything that’s not alphanumeric or whitespace is safe. ~~~ var name = "dea+hl[]rd"; var text = "This dea+hl[]rd guy is super annoying."; var escaped = name.replace(/[^\w\s]/g, "\\$&"); var regexp = new RegExp("\\b(" + escaped + ")\\b", "gi"); console.log(text.replace(regexp, "_$1_")); // → This _dea+hl[]rd_ guy is super annoying. ~~~ ## [](http://eloquentjavascript.net/09_regexp.html#h_Txg7z4j/ei)The search method [](http://eloquentjavascript.net/09_regexp.html#p_7vPpkhqhrD)The `indexOf` method on strings cannot be called with a regular expression. But there is another method, `search`, which does expect a regular expression. Like`indexOf`, it returns the first index on which the expression was found, or -1 when it wasn’t found. ~~~ console.log(" word".search(/\S/)); // → 2 console.log(" ".search(/\S/)); // → -1 ~~~ [](http://eloquentjavascript.net/09_regexp.html#p_tqlyvUKoi5)Unfortunately, there is no way to indicate that the match should start at a given offset (like we can with the second argument to `indexOf`), which would often be useful. ## [](http://eloquentjavascript.net/09_regexp.html#h_duFTd2hqd0)The lastIndex property [](http://eloquentjavascript.net/09_regexp.html#p_heIQ0/XXEu)The `exec` method similarly does not provide a convenient way to start searching from a given position in the string. But it does provide an*in*convenient way. [](http://eloquentjavascript.net/09_regexp.html#p_F+JgzwxLtK)Regular expression objects have properties. One such property is `source`, which contains the string that expression was created from. Another property is `lastIndex`, which controls, in some limited circumstances, where the next match will start. [](http://eloquentjavascript.net/09_regexp.html#p_y8GS9OJXkh)Those circumstances are that the regular expression must have the global (`g`) option enabled, and the match must happen through the `exec` method. Again, a more sane solution would have been to just allow an extra argument to be passed to `exec`, but sanity is not a defining characteristic of JavaScript’s regular expression interface. ~~~ var pattern = /y/g; pattern.lastIndex = 3; var match = pattern.exec("xyzzy"); console.log(match.index); // → 4 console.log(pattern.lastIndex); // → 5 ~~~ [](http://eloquentjavascript.net/09_regexp.html#p_hjLQ+57mDd)If the match was successful, the call to `exec` automatically updates the`lastIndex` property to point after the match. If no match was found,`lastIndex` is set back to zero, which is also the value it has in a newly constructed regular expression object. [](http://eloquentjavascript.net/09_regexp.html#p_042bNmzNZK)When using a global regular expression value for multiple `exec` calls, these automatic updates to the `lastIndex` property can cause problems. Your regular expression might be accidentally starting at an index that was left over from a previous call. ~~~ var digit = /\d/g; console.log(digit.exec("here it is: 1")); // → ["1"] console.log(digit.exec("and now: 1")); // → null ~~~ [](http://eloquentjavascript.net/09_regexp.html#p_9l7tQ3SsME)Another interesting effect of the global option is that it changes the way the`match` method on strings works. When called with a global expression, instead of returning an array similar to that returned by `exec`, `match` will find *all*matches of the pattern in the string and return an array containing the matched strings. ~~~ console.log("Banana".match(/an/g)); // → ["an", "an"] ~~~ [](http://eloquentjavascript.net/09_regexp.html#p_zFHO63a2iV)So be cautious with global regular expressions. The cases where they are necessary—calls to `replace` and places where you want to explicitly use`lastIndex`—are typically the only places where you want to use them. ### [](http://eloquentjavascript.net/09_regexp.html#h_m0fs21dHEg)Looping over matches [](http://eloquentjavascript.net/09_regexp.html#p_sFhvrUR1vm)A common pattern is to scan through all occurrences of a pattern in a string, in a way that gives us access to the match object in the loop body, by using`lastIndex` and `exec`. ~~~ var input = "A string with 3 numbers in it... 42 and 88."; var number = /\b(\d+)\b/g; var match; while (match = number.exec(input)) console.log("Found", match[1], "at", match.index); // → Found 3 at 14 // Found 42 at 33 // Found 88 at 40 ~~~ [](http://eloquentjavascript.net/09_regexp.html#p_ZdCI2+edqA)This makes use of the fact that the value of an assignment expression (`=`) is the assigned value. So by using `match = number.exec(input)` as the condition in the `while` statement, we perform the match at the start of each iteration, save its result in a variable, and stop looping when no more matches are found. ## [](http://eloquentjavascript.net/09_regexp.html#h_RGsf6ah1EY)Parsing an INI file [](http://eloquentjavascript.net/09_regexp.html#p_JbrLORqV9r)To conclude the chapter, we’ll look at a problem that calls for regular expressions. Imagine we are writing a program to automatically harvest information about our enemies from the Internet. (We will not actually write that program here, just the part that reads the configuration file. Sorry to disappoint.) The configuration file looks like this: ~~~ searchengine=http://www.google.com/search?q=$1 spitefulness=9.7 ; comments are preceded by a semicolon... ; each section concerns an individual enemy [larry] fullname=Larry Doe type=kindergarten bully website=http://www.geocities.com/CapeCanaveral/11451 [gargamel] fullname=Gargamel type=evil sorcerer outputdir=/home/marijn/enemies/gargamel ~~~ [](http://eloquentjavascript.net/09_regexp.html#p_OgIQS1TJxB)The exact rules for this format (which is actually a widely used format, usually called an *INI* file) are as follows: * [](http://eloquentjavascript.net/09_regexp.html#p_jIewfc/40B) Blank lines and lines starting with semicolons are ignored. * [](http://eloquentjavascript.net/09_regexp.html#p_O/dGCr+aR5) Lines wrapped in `[` and `]` start a new section. * [](http://eloquentjavascript.net/09_regexp.html#p_l2Yjl1fUVB) Lines containing an alphanumeric identifier followed by an `=` character add a setting to the current section. * [](http://eloquentjavascript.net/09_regexp.html#p_bCaQwCXJCi) Anything else is invalid. [](http://eloquentjavascript.net/09_regexp.html#p_LWCnDNCUFW)Our task is to convert a string like this into an array of objects, each with a`name` property and an array of settings. We’ll need one such object for each section and one for the global settings at the top. [](http://eloquentjavascript.net/09_regexp.html#p_8golZvaoZS)Since the format has to be processed line by line, splitting up the file into separate lines is a good start. We used `string.split("\n")` to do this in[Chapter 6](http://eloquentjavascript.net/06_object.html#split). Some operating systems, however, use not just a newline character to separate lines but a carriage return character followed by a newline (`"\r\n"`). Given that the `split` method also allows a regular expression as its argument, we can split on a regular expression like `/\r?\n/` to split in a way that allows both `"\n"` and `"\r\n"` between lines. ~~~ function parseINI(string) { // Start with an object to hold the top-level fields var currentSection = {name: null, fields: []}; var categories = [currentSection]; string.split(/\r?\n/).forEach(function(line) { var match; if (/^\s*(;.*)?$/.test(line)) { return; } else if (match = line.match(/^\[(.*)\]$/)) { currentSection = {name: match[1], fields: []}; categories.push(currentSection); } else if (match = line.match(/^(\w+)=(.*)$/)) { currentSection.fields.push({name: match[1], value: match[2]}); } else { throw new Error("Line '" + line + "' is invalid."); } }); return categories; } ~~~ [](http://eloquentjavascript.net/09_regexp.html#p_aXK65jiU6o)This code goes over every line in the file, updating the “current section” object as it goes along. First, it checks whether the line can be ignored, using the expression `/^\s*(;.*)?$/`. Do you see how it works? The part between the parentheses will match comments, and the `?` will make sure it also matches lines containing only whitespace. [](http://eloquentjavascript.net/09_regexp.html#p_CKB+LQEvXY)If the line is not a comment, the code then checks whether the line starts a new section. If so, it creates a new current section object, to which subsequent settings will be added. [](http://eloquentjavascript.net/09_regexp.html#p_XjKuxzEtw/)The last meaningful possibility is that the line is a normal setting, which the code adds to the current section object. [](http://eloquentjavascript.net/09_regexp.html#p_oU+yccn9dn)If a line matches none of these forms, the function throws an error. [](http://eloquentjavascript.net/09_regexp.html#p_FPzqsloIkT)Note the recurring use of `^` and `$` to make sure the expression matches the whole line, not just part of it. Leaving these out results in code that mostly works but behaves strangely for some input, which can be a difficult bug to track down. [](http://eloquentjavascript.net/09_regexp.html#p_SLY/jPojdT)The pattern `if (match = string.match(...))` is similar to the trick of using an assignment as the condition for `while`. You often aren’t sure that your call to `match` will succeed, so you can access the resulting object only inside an `if`statement that tests for this. To not break the pleasant chain of `if` forms, we assign the result of the match to a variable and immediately use that assignment as the test in the `if` statement. ## [](http://eloquentjavascript.net/09_regexp.html#h_+y54//b0l+)International characters [](http://eloquentjavascript.net/09_regexp.html#p_Gw6mX6NBqt)Because of JavaScript’s initial simplistic implementation and the fact that this simplistic approach was later set in stone as standard behavior, JavaScript’s regular expressions are rather dumb about characters that do not appear in the English language. For example, as far as JavaScript’s regular expressions are concerned, a “word character” is only one of the 26 characters in the Latin alphabet (uppercase or lowercase) and, for some reason, the underscore character. Things like *é* or *β*, which most definitely are word characters, will not match `\w` (and *will* match uppercase `\W`, the nonword category). [](http://eloquentjavascript.net/09_regexp.html#p_hQosIPfRoo)By a strange historical accident, `\s` (whitespace) does not have this problem and matches all characters that the Unicode standard considers whitespace, including things like the nonbreaking space and the Mongolian vowel separator. [](http://eloquentjavascript.net/09_regexp.html#p_ssmpiUydMx)Some regular expression implementations in other programming languages have syntax to match specific Unicode character categories, such as “all uppercase letters”, “all punctuation”, or “control characters”. There are plans to add support for such categories JavaScript, but it unfortunately looks like they won’t be realized in the near future. ## [](http://eloquentjavascript.net/09_regexp.html#h_ErccPg/l98)Summary [](http://eloquentjavascript.net/09_regexp.html#p_/hQX04GtpS)Regular expressions are objects that represent patterns in strings. They use their own syntax to express these patterns. | `/abc/` | A sequence of characters | | `/[abc]/` | Any character from a set of characters | | `/[^abc]/` | Any character *not* in a set of characters | | `/[0-9]/` | Any character in a range of characters | | `/x+/` | One or more occurrences of the pattern `x` | | `/x+?/` | One or more occurrences, nongreedy | | `/x*/` | Zero or more occurrences | | `/x?/` | Zero or one occurrence | | `/x{2,4}/` | Between two and four occurrences | | `/(abc)/` | A group | | `/a|b|c/` | Any one of several patterns | | `/\d/` | Any digit character | | `/\w/` | An alphanumeric character (“word character”) | | `/\s/` | Any whitespace character | | `/./` | Any character except newlines | | `/\b/` | A word boundary | | `/^/` | Start of input | | `/$/` | End of input | [](http://eloquentjavascript.net/09_regexp.html#p_AVY5pFcEyH)A regular expression has a method `test` to test whether a given string matches it. It also has an `exec` method that, when a match is found, returns an array containing all matched groups. Such an array has an `index` property that indicates where the match started. [](http://eloquentjavascript.net/09_regexp.html#p_Ml7hKXO6Ll)Strings have a `match` method to match them against a regular expression and a`search` method to search for one, returning only the starting position of the match. Their `replace` method can replace matches of a pattern with a replacement string. Alternatively, you can pass a function to `replace`, which will be used to build up a replacement string based on the match text and matched groups. [](http://eloquentjavascript.net/09_regexp.html#p_XsUkGFZG+B)Regular expressions can have options, which are written after the closing slash. The `i` option makes the match case insensitive, while the `g` option makes the expression *global*, which, among other things, causes the `replace` method to replace all instances instead of just the first. [](http://eloquentjavascript.net/09_regexp.html#p_CW10Ac/SDj)The `RegExp` constructor can be used to create a regular expression value from a string. [](http://eloquentjavascript.net/09_regexp.html#p_gsM3HikR5c)Regular expressions are a sharp tool with an awkward handle. They simplify some tasks tremendously but can quickly become unmanageable when applied to complex problems. Part of knowing how to use them is resisting the urge to try to shoehorn things that they cannot sanely express into them. ## [](http://eloquentjavascript.net/09_regexp.html#h_TcUD2vzyMe)Exercises [](http://eloquentjavascript.net/09_regexp.html#p_meNfX2B/+s)It is almost unavoidable that, in the course of working on these exercises, you will get confused and frustrated by some regular expression’s inexplicable behavior. Sometimes it helps to enter your expression into an online tool like[*debuggex.com*](https://www.debuggex.com/) to see whether its visualization corresponds to what you intended and to experiment with the way it responds to various input strings. ### [](http://eloquentjavascript.net/09_regexp.html#h_vDM8PzwQWU)Regexp golf [](http://eloquentjavascript.net/09_regexp.html#p_1t8xXpFN7O)*Code golf* is a term used for the game of trying to express a particular program in as few characters as possible. Similarly, *regexp golf* is the practice of writing as tiny a regular expression as possible to match a given pattern, and *only* that pattern. [](http://eloquentjavascript.net/09_regexp.html#p_VGCqgCur6C)For each of the following items, write a regular expression to test whether any of the given substrings occur in a string. The regular expression should match only strings containing one of the substrings described. Do not worry about word boundaries unless explicitly mentioned. When your expression works, see whether you can make it any smaller. 1. [](http://eloquentjavascript.net/09_regexp.html#p_togdFO+/b9)*car* and *cat* 2. [](http://eloquentjavascript.net/09_regexp.html#p_2Q37Tsr9DS)*pop* and *prop* 3. [](http://eloquentjavascript.net/09_regexp.html#p_2Ah4dFikw1)*ferret*, *ferry*, and *ferrari* 4. [](http://eloquentjavascript.net/09_regexp.html#p_ttiBCcePDl)Any word ending in *ious* 5. [](http://eloquentjavascript.net/09_regexp.html#p_XnqTy5SopM)A whitespace character followed by a dot, comma, colon, or semicolon 6. [](http://eloquentjavascript.net/09_regexp.html#p_Ku7hE3qqDn)A word longer than six letters 7. [](http://eloquentjavascript.net/09_regexp.html#p_2Tx4SPp5Wm)A word without the letter *e* [](http://eloquentjavascript.net/09_regexp.html#p_Tzjl1Axr+h)Refer to the table in the [chapter summary](http://eloquentjavascript.net/09_regexp.html#summary_regexp) for help. Test each solution with a few test strings. ~~~ // Fill in the regular expressions verify(/.../, ["my car", "bad cats"], ["camper", "high art"]); verify(/.../, ["pop culture", "mad props"], ["plop"]); verify(/.../, ["ferret", "ferry", "ferrari"], ["ferrum", "transfer A"]); verify(/.../, ["how delicious", "spacious room"], ["ruinous", "consciousness"]); verify(/.../, ["bad punctuation ."], ["escape the dot"]); verify(/.../, ["hottentottententen"], ["no", "hotten totten tenten"]); verify(/.../, ["red platypus", "wobbling nest"], ["earth bed", "learning ape"]); function verify(regexp, yes, no) { // Ignore unfinished exercises if (regexp.source == "...") return; yes.forEach(function(s) { if (!regexp.test(s)) console.log("Failure to match '" + s + "'"); }); no.forEach(function(s) { if (regexp.test(s)) console.log("Unexpected match for '" + s + "'"); }); } ~~~ ### [](http://eloquentjavascript.net/09_regexp.html#h_dTiEW14oG0)Quoting style [](http://eloquentjavascript.net/09_regexp.html#p_Zvl6rHT09m)Imagine you have written a story and used single quotation marks throughout to mark pieces of dialogue. Now you want to replace all the dialogue quotes with double quotes, while keeping the single quotes used in contractions like*aren’t*. [](http://eloquentjavascript.net/09_regexp.html#p_k3Y0NF9w4b)Think of a pattern that distinguishes these two kinds of quote usage and craft a call to the `replace` method that does the proper replacement. ~~~ var text = "'I'm the cook,' he said, 'it's my job.'"; // Change this call. console.log(text.replace(/A/g, "B")); // → "I'm the cook," he said, "it's my job." ~~~ ### [](http://eloquentjavascript.net/09_regexp.html#h_izldJoT3uv)Numbers again [](http://eloquentjavascript.net/09_regexp.html#p_2Z15nSCOWO)A series of digits can be matched by the simple regular expression `/\d+/`. [](http://eloquentjavascript.net/09_regexp.html#p_4ILW7h/ou4)Write an expression that matches only JavaScript-style numbers. It must support an optional minus *or* plus sign in front of the number, the decimal dot, and exponent notation—`5e-3` or `1E10`— again with an optional sign in front of the exponent. Also note that it is not necessary for there to be digits in front of or after the dot, but the number cannot be a dot alone. That is, `.5` and `5.` are valid JavaScript numbers, but a lone dot *isn’t*. ~~~ // Fill in this regular expression. var number = /^...$/; // Tests: ["1", "-1", "+15", "1.55", ".5", "5.", "1.3e2", "1E-4", "1e+12"].forEach(function(s) { if (!number.test(s)) console.log("Failed to match '" + s + "'"); }); ["1a", "+-1", "1.2.3", "1+1", "1e4.5", ".5.", "1f5", "."].forEach(function(s) { if (number.test(s)) console.log("Incorrectly accepted '" + s + "'"); }); ~~~ First, do not forget the backslash in front of the dot. > Matching the optional sign in front of the number, as well as in front of the exponent, can be done with `[+\-]?` or `(\+|-|)` (plus, minus, or nothing). > The more complicated part of the exercise is the problem of matching both`"5."` and `".5"` without also matching `"."`. For this, a good solution is to use the `|` operator to separate the two cases—either one or more digits optionally followed by a dot and zero or more digits *or* a dot followed by one or more digits. > Finally, to make the *e* case-insensitive, either add an `i` option to the regular expression or use `[eE]`.

## Chapter 8 # Bugs and Error Handling > [](http://eloquentjavascript.net/08_error.html#p_tsWlII94Rs)Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it. > > Brian Kernighan and P.J. Plauger, The Elements of Programming Style > [](http://eloquentjavascript.net/08_error.html#p_KRfkqUrtHi)Yuan-Ma had written a small program that used many global variables and shoddy shortcuts. Reading it, a student asked, ‘You warned us against these techniques, yet I find them in your program. How can this be?’ The master said, ‘There is no need to fetch a water hose when the house is not on fire.’ > > Master Yuan-Ma, The Book of Programming [](http://eloquentjavascript.net/08_error.html#p_yOWLCBc/o3)A program is crystallized thought. Sometimes those thoughts are confused. Other times, mistakes are introduced when converting thought into code. Either way, the result is a flawed program. [](http://eloquentjavascript.net/08_error.html#p_+XOm2skE5P)Flaws in a program are usually called bugs. Bugs can be programmer errors or problems in other systems that the program interacts with. Some bugs are immediately apparent, while others are subtle and might remain hidden in a system for years. [](http://eloquentjavascript.net/08_error.html#p_S3De4NHpy4)Often, problems surface only when a program encounters a situation that the programmer didn’t originally consider. Sometimes such situations are unavoidable. When the user is asked to input their age and types *orange*, this puts our program in a difficult position. The situation has to be anticipated and handled somehow. ## [](http://eloquentjavascript.net/08_error.html#h_0dUgeA9t6O)Programmer mistakes [](http://eloquentjavascript.net/08_error.html#p_4nGvSb/+db)When it comes to programmer mistakes, our aim is simple. We want to find them and fix them. Such mistakes can range from simple typos that cause the computer to complain as soon as it lays eyes on our program to subtle mistakes in our understanding of the way the program operates, causing incorrect outcomes only in specific situations. Bugs of the latter type can take weeks to diagnose. [](http://eloquentjavascript.net/08_error.html#p_uZJEWof1D2)The degree to which languages help you find such mistakes varies. Unsurprisingly, JavaScript is at the “hardly helps at all” end of that scale. Some languages want to know the types of all your variables and expressions before even running a program and will tell you right away when a type is used in an inconsistent way. JavaScript considers types only when actually running the program, and even then, it allows you to do some clearly nonsensical things without complaint, such as `x = true * "monkey"`. [](http://eloquentjavascript.net/08_error.html#p_o9bRfOflvH)There are some things that JavaScript does complain about, though. Writing a program that is not syntactically valid will immediately trigger an error. Other things, such as calling something that’s not a function or looking up a property on an undefined value, will cause an error to be reported when the program is running and encounters the nonsensical action. [](http://eloquentjavascript.net/08_error.html#p_M/fMASXkeH)But often, your nonsense computation will simply produce a `NaN` (not a number) or undefined value. And the program happily continues, convinced that it’s doing something meaningful. The mistake will manifest itself only later, after the bogus value has traveled though several functions. It might not trigger an error at all but silently cause the program’s output to be wrong. Finding the source of such problems can be difficult. [](http://eloquentjavascript.net/08_error.html#p_D4KIEMyl44)The process of finding mistakes—bugs—in programs is called *debugging*. ## [](http://eloquentjavascript.net/08_error.html#h_u1jlTq3i42)Strict mode [](http://eloquentjavascript.net/08_error.html#p_NMLSAIO3JD)JavaScript can be made a *little* more strict by enabling *strict mode*. This is done by putting the string `"use strict"` at the top of a file or a function body. Here’s an example: ~~~ function canYouSpotTheProblem() { "use strict"; for (counter = 0; counter < 10; counter++) console.log("Happy happy"); } canYouSpotTheProblem(); // → ReferenceError: counter is not defined ~~~ [](http://eloquentjavascript.net/08_error.html#p_4X4yfdyK1S)Normally, when you forget to put `var` in front of your variable, as with`counter` in the example, JavaScript quietly creates a global variable and uses that. In strict mode, however, an error is reported instead. This is very helpful. It should be noted, though, that this doesn’t work when the variable in question already exists as a global variable, but only when assigning to it would have created it. [](http://eloquentjavascript.net/08_error.html#p_mPx7xB5UKb)Another change in strict mode is that the `this` binding holds the value`undefined` in functions that are not called as methods. When making such a call outside of strict mode, `this` refers to the global scope object. So if you accidentally call a method or constructor incorrectly in strict mode, JavaScript will produce an error as soon as it tries to read something from `this`, rather than happily working with the global object, creating and reading global variables. [](http://eloquentjavascript.net/08_error.html#p_eZSU5aTSD4)For example, consider the following code, which calls a constructor without the`new` keyword so that its `this` will *not* refer to a newly constructed object: ~~~ function Person(name) { this.name = name; } var ferdinand = Person("Ferdinand"); // oops console.log(name); // → Ferdinand ~~~ [](http://eloquentjavascript.net/08_error.html#p_NNFpkpZKdP)So the bogus call to `Person` succeeded but returned an undefined value and created the global variable `name`. In strict mode, the result is different. ~~~ "use strict"; function Person(name) { this.name = name; } // Oops, forgot 'new' var ferdinand = Person("Ferdinand"); // → TypeError: Cannot set property 'name' of undefined ~~~ [](http://eloquentjavascript.net/08_error.html#p_0eYPyoGi1l)We are immediately told that something is wrong. This is helpful. [](http://eloquentjavascript.net/08_error.html#p_qlYTT6nRD4)Strict mode does a few more things. It disallows giving a function multiple parameters with the same name and removes certain problematic language features entirely (such as the `with` statement, which is so misguided it is not further discussed in this book). [](http://eloquentjavascript.net/08_error.html#p_kIUErMqROW)In short, putting a `"use strict"` at the top of your program rarely hurts and might help you spot a problem. ## [](http://eloquentjavascript.net/08_error.html#h_CCCzKyBrc1)Testing [](http://eloquentjavascript.net/08_error.html#p_cfHj2WfM1O)If the language is not going to do much to help us find mistakes, we’ll have to find them the hard way: by running the program and seeing whether it does the right thing. [](http://eloquentjavascript.net/08_error.html#p_21E0ZgauWA)Doing this by hand, again and again, is a sure way to drive yourself insane. Fortunately, it is often possible to write a second program that automates testing your actual program. [](http://eloquentjavascript.net/08_error.html#p_rx8TfcyB0X)As an example, we once again use the `Vector` type. ~~~ function Vector(x, y) { this.x = x; this.y = y; } Vector.prototype.plus = function(other) { return new Vector(this.x + other.x, this.y + other.y); }; ~~~ [](http://eloquentjavascript.net/08_error.html#p_HDs3GnqA5b)We will write a program to check that our implementation of `Vector` works as intended. Then, every time we change the implementation, we follow up by running the test program so that we can be reasonably confident that we didn’t break anything. When we add extra functionality (for example, a new method) to the `Vector` type, we also add tests for the new feature. ~~~ function testVector() { var p1 = new Vector(10, 20); var p2 = new Vector(-10, 5); var p3 = p1.plus(p2); if (p1.x !== 10) return "fail: x property"; if (p1.y !== 20) return "fail: y property"; if (p2.x !== -10) return "fail: negative x property"; if (p3.x !== 0) return "fail: x from plus"; if (p3.y !== 25) return "fail: y from plus"; return "everything ok"; } console.log(testVector()); // → everything ok ~~~ [](http://eloquentjavascript.net/08_error.html#p_ugbmGkfrN+)Writing tests like this tends to produce rather repetitive, awkward code. Fortunately, there exist pieces of software that help you build and run collections of tests (*test suites*) by providing a language (in the form of functions and methods) suited to expressing tests and by outputting informative information when a test fails. These are called *testing frameworks*. ## [](http://eloquentjavascript.net/08_error.html#h_iVsnyIAWUT)Debugging [](http://eloquentjavascript.net/08_error.html#p_5mME8LB07m)Once you notice that there is something wrong with your program because it misbehaves or produces errors, the next step is to figure out *what* the problem is. [](http://eloquentjavascript.net/08_error.html#p_CJKevweHV6)Sometimes it is obvious. The error message will point at a specific line of your program, and if you look at the error description and that line of code, you can often see the problem. [](http://eloquentjavascript.net/08_error.html#p_U2MGlMiUv1)But not always. Sometimes the line that triggered the problem is simply the first place where a bogus value produced elsewhere gets used in an invalid way. And sometimes there is no error message at all—just an invalid result. If you have been solving the exercises in the earlier chapters, you will probably have already experienced such situations. [](http://eloquentjavascript.net/08_error.html#p_rsq5hJyXKX)The following example program tries to convert a whole number to a string in any base (decimal, binary, and so on) by repeatedly picking out the last digit and then dividing the number to get rid of this digit. But the insane output that it currently produces suggests that it has a bug. ~~~ function numberToString(n, base) { var result = "", sign = ""; if (n < 0) { sign = "-"; n = -n; } do { result = String(n % base) + result; n /= base; } while (n > 0); return sign + result; } console.log(numberToString(13, 10)); // → 1.5e-3231.3e-3221.3e-3211.3e-3201.3e-3191.3e-3181.3… ~~~ [](http://eloquentjavascript.net/08_error.html#p_RAeAoBjrC7)Even if you see the problem already, pretend for a moment that you don’t. We know that our program is malfunctioning, and we want to find out why. [](http://eloquentjavascript.net/08_error.html#p_2OJsjnmKOW)This is where you must resist the urge to start making random changes to the code. Instead, *think*. Analyze what is happening and come up with a theory of why it might be happening. Then, make additional observations to test this theory—or, if you don’t yet have a theory, make additional observations that might help you come up with one. [](http://eloquentjavascript.net/08_error.html#p_HuAniqpAd5)Putting a few strategic `console.log` calls into the program is a good way to get additional information about what the program is doing. In this case, we want`n` to take the values `13`, `1`, and then `0`. Let’s write out its value at the start of the loop. ~~~ 13 1.3 0.13 0.013 … 1.5e-323 ~~~ [](http://eloquentjavascript.net/08_error.html#p_I+PNN6aJ0e)*Right*. Dividing 13 by 10 does not produce a whole number. Instead of `n /= base`, what we actually want is `n = Math.floor(n / base)` so that the number is properly “shifted” to the right. [](http://eloquentjavascript.net/08_error.html#p_E0PMYRZC2c)An alternative to using `console.log` is to use the *debugger* capabilities of your browser. Modern browsers come with the ability to set a *breakpoint* on a specific line of your code. This will cause the execution of the program to pause every time the line with the breakpoint is reached and allow you to inspect the values of variables at that point. I won’t go into details here since debuggers differ from browser to browser, but look in your browser’s developer tools and search the Web for more information. Another way to set a breakpoint is to include a `debugger` statement (consisting of simply that keyword) in your program. If the developer tools of your browser are active, the program will pause whenever it reaches that statement, and you will be able to inspect its state. ## [](http://eloquentjavascript.net/08_error.html#h_iwwPbaBjJD)Error propagation [](http://eloquentjavascript.net/08_error.html#p_KL47zNa+D/)Not all problems can be prevented by the programmer, unfortunately. If your program communicates with the outside world in any way, there is a chance that the input it gets will be invalid or that other systems that it tries to talk to are broken or unreachable. [](http://eloquentjavascript.net/08_error.html#p_OUzzwpY/Dj)Simple programs, or programs that run only under your supervision, can afford to just give up when such a problem occurs. You’ll look into the problem and try again. “Real” applications, on the other hand, are expected to not simply crash. Sometimes the right thing to do is take the bad input in stride and continue running. In other cases, it is better to report to the user what went wrong and then give up. But in either situation, the program has to actively do something in response to the problem. [](http://eloquentjavascript.net/08_error.html#p_ZrDi3JVFsJ)Say you have a function `promptInteger` that asks the user for a whole number and returns it. What should it return if the user inputs *orange*? [](http://eloquentjavascript.net/08_error.html#p_tL4eC00B0P)One option is to make it return a special value. Common choices for such values are `null` and `undefined`. ~~~ function promptNumber(question) { var result = Number(prompt(question, "")); if (isNaN(result)) return null; else return result; } console.log(promptNumber("How many trees do you see?")); ~~~ [](http://eloquentjavascript.net/08_error.html#p_kf1B4tj0HJ)This is a sound strategy. Now any code that calls `promptNumber` must check whether an actual number was read and, failing that, must somehow recover—maybe by asking again or by filling in a default value. Or it could again return a special value to *its* caller to indicate that it failed to do what it was asked. [](http://eloquentjavascript.net/08_error.html#p_0FyKlwlFr6)In many situations, mostly when errors are common and the caller should be explicitly taking them into account, returning a special value is a perfectly fine way to indicate an error. It does, however, have its downsides. First, what if the function can already return every possible kind of value? For such a function, it is hard to find a special value that can be distinguished from a valid result. [](http://eloquentjavascript.net/08_error.html#p_+E9fllIVBt)The second issue with returning special values is that it can lead to some very cluttered code. If a piece of code calls `promptNumber` 10 times, it has to check 10 times whether `null` was returned. And if its response to finding `null` is to simply return `null` itself, the caller will in turn have to check for it, and so on. ## [](http://eloquentjavascript.net/08_error.html#h_zT3755/aOp)Exceptions [](http://eloquentjavascript.net/08_error.html#p_ZBsTKhGA4i)When a function cannot proceed normally, what we would *like* to do is just stop what we are doing and immediately jump back to a place that knows how to handle the problem. This is what *exception handling* does. [](http://eloquentjavascript.net/08_error.html#p_kjXcPy8jGf)Exceptions are a mechanism that make it possible for code that runs into a problem to *raise* (or *throw*) an exception, which is simply a value. Raising an exception somewhat resembles a super-charged return from a function: it jumps out of not just the current function but also out of its callers, all the way down to the first call that started the current execution. This is called*unwinding the stack*. You may remember the stack of function calls that was mentioned in [Chapter 3](http://eloquentjavascript.net/03_functions.html#stack). An exception zooms down this stack, throwing away all the call contexts it encounters. [](http://eloquentjavascript.net/08_error.html#p_nQYTvbifx9)If exceptions always zoomed right down to the bottom of the stack, they would not be of much use. They would just provide a novel way to blow up your program. Their power lies in the fact that you can set “obstacles” along the stack to *catch* the exception as it is zooming down. Then you can do something with it, after which the program continues running at the point where the exception was caught. [](http://eloquentjavascript.net/08_error.html#p_8dgVjcjpX6)Here’s an example: ~~~ function promptDirection(question) { var result = prompt(question, ""); if (result.toLowerCase() == "left") return "L"; if (result.toLowerCase() == "right") return "R"; throw new Error("Invalid direction: " + result); } function look() { if (promptDirection("Which way?") == "L") return "a house"; else return "two angry bears"; } try { console.log("You see", look()); } catch (error) { console.log("Something went wrong: " + error); } ~~~ [](http://eloquentjavascript.net/08_error.html#p_3fhf6E/ppr)The `throw` keyword is used to raise an exception. Catching one is done by wrapping a piece of code in a `try` block, followed by the keyword `catch`. When the code in the `try` block causes an exception to be raised, the `catch` block is evaluated. The variable name (in parentheses) after `catch` will be bound to the exception value. After the `catch` block finishes—or if the `try` block finishes without problems—control proceeds beneath the entire `try/catch` statement. [](http://eloquentjavascript.net/08_error.html#p_0m/weWGs2g)In this case, we used the `Error` constructor to create our exception value. This is a standard JavaScript constructor that creates an object with a `message`property. In modern JavaScript environments, instances of this constructor also gather information about the call stack that existed when the exception was created, a so-called *stack trace*. This information is stored in the `stack`property and can be helpful when trying to debug a problem: it tells us the precise function where the problem occurred and which other functions led up to the call that failed. [](http://eloquentjavascript.net/08_error.html#p_BYCPINQ0h5)Note that the function `look` completely ignores the possibility that`promptDirection` might go wrong. This is the big advantage of exceptions—error-handling code is necessary only at the point where the error occurs and at the point where it is handled. The functions in between can forget all about it. [](http://eloquentjavascript.net/08_error.html#p_deFX/IC34y)Well, almost... ## [](http://eloquentjavascript.net/08_error.html#h_cgoP7o2fe9)Cleaning up after exceptions [](http://eloquentjavascript.net/08_error.html#p_qnJkou8oDN)Consider the following situation: a function, `withContext`, wants to make sure that, during its execution, the top-level variable `context` holds a specific context value. After it finishes, it restores this variable to its old value. ~~~ var context = null; function withContext(newContext, body) { var oldContext = context; context = newContext; var result = body(); context = oldContext; return result; } ~~~ [](http://eloquentjavascript.net/08_error.html#p_695isCQEpS)What if `body` raises an exception? In that case, the call to `withContext` will be thrown off the stack by the exception, and `context` will never be set back to its old value. [](http://eloquentjavascript.net/08_error.html#p_tSD0hfdZ/H)There is one more feature that `try` statements have. They may be followed by a`finally` block either instead of or in addition to a `catch` block. A `finally`block means “No matter *what* happens, run this code after trying to run the code in the `try` block”. If a function has to clean something up, the cleanup code should usually be put into a `finally` block. ~~~ function withContext(newContext, body) { var oldContext = context; context = newContext; try { return body(); } finally { context = oldContext; } } ~~~ [](http://eloquentjavascript.net/08_error.html#p_exXm7BGc3g)Note that we no longer have to store the result of `body` (which we want to return) in a variable. Even if we return directly from the `try` block, the`finally` block will be run. Now we can do this and be safe: ~~~ try { withContext(5, function() { if (context < 10) throw new Error("Not enough context!"); }); } catch (e) { console.log("Ignoring: " + e); } // → Ignoring: Error: Not enough context! console.log(context); // → null ~~~ [](http://eloquentjavascript.net/08_error.html#p_Ri/W5NdtT2)Even though the function called from `withContext` exploded, `withContext`itself still properly cleaned up the `context` variable. ## [](http://eloquentjavascript.net/08_error.html#h_vfoJqEDazI)Selective catching [](http://eloquentjavascript.net/08_error.html#p_cXjPLkgsqi)When an exception makes it all the way to the bottom of the stack without being caught, it gets handled by the environment. What this means differs between environments. In browsers, a description of the error typically gets written to the JavaScript console (reachable through the browser’s Tools or Developer menu). [](http://eloquentjavascript.net/08_error.html#p_9JOlpepKZE)For programmer mistakes or problems that the program cannot possibly handle, just letting the error go through is often okay. An unhandled exception is a reasonable way to signal a broken program, and the JavaScript console will, on modern browsers, provide you with some information about which function calls were on the stack when the problem occurred. [](http://eloquentjavascript.net/08_error.html#p_6cLdy7IIUS)For problems that are *expected* to happen during routine use, crashing with an unhandled exception is not a very friendly response. [](http://eloquentjavascript.net/08_error.html#p_SFNa748bl0)Invalid uses of the language, such as referencing a nonexistent variable, looking up a property on `null`, or calling something that’s not a function, will also result in exceptions being raised. Such exceptions can be caught just like your own exceptions. [](http://eloquentjavascript.net/08_error.html#p_hqHJot1Va+)When a `catch` body is entered, all we know is that *something* in our `try` body caused an exception. But we don’t know *what*, or *which* exception it caused. [](http://eloquentjavascript.net/08_error.html#p_4Cpd4Xhmxf)JavaScript (in a rather glaring omission) doesn’t provide direct support for selectively catching exceptions: either you catch them all or you don’t catch any. This makes it very easy to *assume* that the exception you get is the one you were thinking about when you wrote the `catch` block. [](http://eloquentjavascript.net/08_error.html#p_O9bj9nd33p)But it might not be. Some other assumption might be violated, or you might have introduced a bug somewhere that is causing an exception. Here is an example, which *attempts* to keep on calling `promptDirection` until it gets a valid answer: ~~~ for (;;) { try { var dir = promtDirection("Where?"); // ← typo! console.log("You chose ", dir); break; } catch (e) { console.log("Not a valid direction. Try again."); } } ~~~ [](http://eloquentjavascript.net/08_error.html#p_UMh66pw2z8)The `for (;;)` construct is a way to intentionally create a loop that doesn’t terminate on its own. We break out of the loop only when a valid direction is given. *But* we misspelled `promptDirection`, which will result in an “undefined variable” error. Because the `catch` block completely ignores its exception value (`e`), assuming it knows what the problem is, it wrongly treats the variable error as indicating bad input. Not only does this cause an infinite loop, but it also “buries” the useful error message about the misspelled variable. [](http://eloquentjavascript.net/08_error.html#p_hk/lwBIhah)As a general rule, don’t blanket-catch exceptions unless it is for the purpose of “routing” them somewhere—for example, over the network to tell another system that our program crashed. And even then, think carefully about how you might be hiding information. [](http://eloquentjavascript.net/08_error.html#p_FVrgLfvNOk)So we want to catch a *specific* kind of exception. We can do this by checking in the `catch` block whether the exception we got is the one we are interested in and by rethrowing it otherwise. But how do we recognize an exception? [](http://eloquentjavascript.net/08_error.html#p_eWVtoGEmaf)Of course, we could match its `message` property against the error message we happen to expect. But that’s a shaky way to write code—we’d be using information that’s intended for human consumption (the message) to make a programmatic decision. As soon as someone changes (or translates) the message, the code will stop working. [](http://eloquentjavascript.net/08_error.html#p_6OoqTlWc8i)Rather, let’s define a new type of error and use `instanceof` to identify it. ~~~ function InputError(message) { this.message = message; this.stack = (new Error()).stack; } InputError.prototype = Object.create(Error.prototype); InputError.prototype.name = "InputError"; ~~~ [](http://eloquentjavascript.net/08_error.html#p_ckH3IH2NQq)The prototype is made to derive from `Error.prototype` so that `instanceof Error` will also return true for `InputError` objects. It’s also given a `name`property since the standard error types (`Error`, `SyntaxError`,`ReferenceError`, and so on) also have such a property. [](http://eloquentjavascript.net/08_error.html#p_BO+BABkFZf)The assignment to the `stack` property tries to give this object a somewhat useful stack trace, on platforms that support it, by creating a regular error object and then using that object’s `stack` property as its own. [](http://eloquentjavascript.net/08_error.html#p_3NvX0v0g0n)Now `promptDirection` can throw such an error. ~~~ function promptDirection(question) { var result = prompt(question, ""); if (result.toLowerCase() == "left") return "L"; if (result.toLowerCase() == "right") return "R"; throw new InputError("Invalid direction: " + result); } ~~~ [](http://eloquentjavascript.net/08_error.html#p_dtouAk0YL3)And the loop can catch it more carefully. ~~~ for (;;) { try { var dir = promptDirection("Where?"); console.log("You chose ", dir); break; } catch (e) { if (e instanceof InputError) console.log("Not a valid direction. Try again."); else throw e; } } ~~~ [](http://eloquentjavascript.net/08_error.html#p_N4ExnmZrQ/)This will catch only instances of `InputError` and let unrelated exceptions through. If you reintroduce the typo, the undefined variable error will be properly reported. ## [](http://eloquentjavascript.net/08_error.html#h_Sb9V3BEus1)Assertions [](http://eloquentjavascript.net/08_error.html#p_CLUBrkFgp7)*Assertions* are a tool to do basic sanity checking for programmer errors. Consider this helper function, `assert`: ~~~ function AssertionFailed(message) { this.message = message; } AssertionFailed.prototype = Object.create(Error.prototype); function assert(test, message) { if (!test) throw new AssertionFailed(message); } function lastElement(array) { assert(array.length > 0, "empty array in lastElement"); return array[array.length - 1]; } ~~~ [](http://eloquentjavascript.net/08_error.html#p_j1ruFSflzn)This provides a compact way to enforce expectations, helpfully blowing up the program if the stated condition does not hold. For instance, the `lastElement`function, which fetches the last element from an array, would return`undefined` on empty arrays if the assertion was omitted. Fetching the last element from an empty array does not make much sense, so it is almost certainly a programmer error to do so. [](http://eloquentjavascript.net/08_error.html#p_wmVVstENIA)Assertions are a way to make sure mistakes cause failures at the point of the mistake, rather than silently producing nonsense values that may go on to cause trouble in an unrelated part of the system. ## [](http://eloquentjavascript.net/08_error.html#h_ErccPg/l98)Summary [](http://eloquentjavascript.net/08_error.html#p_rD8/ab0/w4)Mistakes and bad input are facts of life. Bugs in programs need to be found and fixed. They can become easier to notice by having automated test suites and adding assertions to your programs. [](http://eloquentjavascript.net/08_error.html#p_6/bb/yZogT)Problems caused by factors outside the program’s control should usually be handled gracefully. Sometimes, when the problem can be handled locally, special return values are a sane way to track them. Otherwise, exceptions are preferable. [](http://eloquentjavascript.net/08_error.html#p_ZJkq9NFh8W)Throwing an exception causes the call stack to be unwound until the next enclosing `try/catch` block or until the bottom of the stack. The exception value will be given to the `catch` block that catches it, which should verify that it is actually the expected kind of exception and then do something with it. To deal with the unpredictable control flow caused by exceptions, `finally` blocks can be used to ensure a piece of code is *always* run when a block finishes. ## [](http://eloquentjavascript.net/08_error.html#h_TcUD2vzyMe)Exercises ### [](http://eloquentjavascript.net/08_error.html#h_n1zYouiAfX)Retry [](http://eloquentjavascript.net/08_error.html#p_oAuWXajIJA)Say you have a function `primitiveMultiply` that, in 50 percent of cases, multiplies two numbers, and in the other 50 percent, raises an exception of type `MultiplicatorUnitFailure`. Write a function that wraps this clunky function and just keeps trying until a call succeeds, after which it returns the result. [](http://eloquentjavascript.net/08_error.html#p_FfNd4pOv0L)Make sure you handle only the exceptions you are trying to handle. ~~~ function MultiplicatorUnitFailure() {} function primitiveMultiply(a, b) { if (Math.random() < 0.5) return a * b; else throw new MultiplicatorUnitFailure(); } function reliableMultiply(a, b) { // Your code here. } console.log(reliableMultiply(8, 8)); // → 64 ~~~ ### [](http://eloquentjavascript.net/08_error.html#h_iGlwnUbkRs)The locked box [](http://eloquentjavascript.net/08_error.html#p_uGznOGuYh8)Consider the following (rather contrived) object: ~~~ var box = { locked: true, unlock: function() { this.locked = false; }, lock: function() { this.locked = true; }, _content: [], get content() { if (this.locked) throw new Error("Locked!"); return this._content; } }; ~~~ [](http://eloquentjavascript.net/08_error.html#p_26mfxczF32)It is a box with a lock. Inside is an array, but you can get at it only when the box is unlocked. Directly accessing the `_content` property is not allowed. [](http://eloquentjavascript.net/08_error.html#p_KI+tJ+amDX)Write a function called `withBoxUnlocked` that takes a function value as argument, unlocks the box, runs the function, and then ensures that the box is locked again before returning, regardless of whether the argument function returned normally or threw an exception. ~~~ function withBoxUnlocked(body) { // Your code here. } withBoxUnlocked(function() { box.content.push("gold piece"); }); try { withBoxUnlocked(function() { throw new Error("Pirates on the horizon! Abort!"); }); } catch (e) { console.log("Error raised:", e); } console.log(box.locked); // → true ~~~ For extra points, make sure that if you call `withBoxUnlocked` when the box is already unlocked, the box stays unlocked. > This exercise calls for a finally block, as you probably guessed. Your function should first unlock the box and then call the argument function from inside a try body. The finally block after it should lock the box again. > To make sure we don’t lock the box when it wasn’t already locked, check its lock at the start of the function and unlock and lock it only when it started out locked.

## Chapter 7 # Project: Electronic Life > [](http://eloquentjavascript.net/07_elife.html#p_uJUZHUrdPa)[...] the question of whether Machines Can Think [...] is about as relevant as the question of whether Submarines Can Swim. > > Edsger Dijkstra, The Threats to Computing Science [](http://eloquentjavascript.net/07_elife.html#p_DNRnU3ddVS)In “project” chapters, I’ll stop pummeling you with new theory for a brief moment and instead work through a program with you. Theory is indispensable when learning to program, but it should be accompanied by reading and understanding nontrivial programs. [](http://eloquentjavascript.net/07_elife.html#p_R5hb9BKRqf)Our project in this chapter is to build a virtual ecosystem, a little world populated with critters that move around and struggle for survival. ## [](http://eloquentjavascript.net/07_elife.html#h_vxvit60Huu)Definition [](http://eloquentjavascript.net/07_elife.html#p_h6wKLP2RRa)To make this task manageable, we will radically simplify the concept of a*world*. Namely, a world will be a two-dimensional grid where each entity takes up one full square of the grid. On every *turn*, the critters all get a chance to take some action. [](http://eloquentjavascript.net/07_elife.html#p_M69iRDzFTU)Thus, we chop both time and space into units with a fixed size: squares for space and turns for time. Of course, this is a somewhat crude and inaccurate approximation. But our simulation is intended to be amusing, not accurate, so we can freely cut such corners. [](http://eloquentjavascript.net/07_elife.html#p_LQyST0HVVt)We can define a world with a *plan*, an array of strings that lays out the world’s grid using one character per square. ~~~ var plan = ["############################", "# # # o ##", "# #", "# ##### #", "## # # ## #", "### ## # #", "# ### # #", "# #### #", "# ## o #", "# o # o ### #", "# # #", "############################"]; ~~~ [](http://eloquentjavascript.net/07_elife.html#p_fLAu6TuC/p)The “#” characters in this plan represent walls and rocks, and the “o” characters represent critters. The spaces, as you might have guessed, are empty space. [](http://eloquentjavascript.net/07_elife.html#p_zsYxlvNAj4)A plan array can be used to create a world object. Such an object keeps track of the size and content of the world. It has a `toString` method, which converts the world back to a printable string (similar to the plan it was based on) so that we can see what’s going on inside. The world object also has a `turn` method, which allows all the critters in it to take one turn and updates the world to reflect their actions. ## [](http://eloquentjavascript.net/07_elife.html#h_Xm5GEUJ7Zs)Representing space [](http://eloquentjavascript.net/07_elife.html#p_oFrPkPfpOk)The grid that models the world has a fixed width and height. Squares are identified by their x- and y-coordinates. We use a simple type, `Vector` (as seen in the exercises for the [previous chapter](http://eloquentjavascript.net/06_object.html#exercise_vector)), to represent these coordinate pairs. ~~~ function Vector(x, y) { this.x = x; this.y = y; } Vector.prototype.plus = function(other) { return new Vector(this.x + other.x, this.y + other.y); }; ~~~ [](http://eloquentjavascript.net/07_elife.html#p_XfoPZurQDA)Next, we need an object type that models the grid itself. A grid is part of a world, but we are making it a separate object (which will be a property of a world object) to keep the world object itself simple. The world should concern itself with world-related things, and the grid should concern itself with grid-related things. [](http://eloquentjavascript.net/07_elife.html#p_wCk8yM/+PK)To store a grid of values, we have several options. We can use an array of row arrays and use two property accesses to get to a specific square, like this: ~~~ var grid = [["top left", "top middle", "top right"], ["bottom left", "bottom middle", "bottom right"]]; console.log(grid[1][2]); // → bottom right ~~~ [](http://eloquentjavascript.net/07_elife.html#p_ltezQzq/yp)Or we can use a single array, with size width × height, and decide that the element at (*x*,*y*) is found at position *x* + (*y* × width) in the array. ~~~ var grid = ["top left", "top middle", "top right", "bottom left", "bottom middle", "bottom right"]; console.log(grid[2 + (1 * 3)]); // → bottom right ~~~ [](http://eloquentjavascript.net/07_elife.html#p_L+hHKfbXDj)Since the actual access to this array will be wrapped in methods on the grid object type, it doesn’t matter to outside code which approach we take. I chose the second representation because it makes it much easier to create the array. When calling the `Array` constructor with a single number as an argument, it creates a new empty array of the given length. [](http://eloquentjavascript.net/07_elife.html#p_WhnH9cyDGI)This code defines the `Grid` object, with some basic methods: ~~~ function Grid(width, height) { this.space = new Array(width * height); this.width = width; this.height = height; } Grid.prototype.isInside = function(vector) { return vector.x >= 0 && vector.x < this.width && vector.y >= 0 && vector.y < this.height; }; Grid.prototype.get = function(vector) { return this.space[vector.x + this.width * vector.y]; }; Grid.prototype.set = function(vector, value) { this.space[vector.x + this.width * vector.y] = value; }; ~~~ [](http://eloquentjavascript.net/07_elife.html#p_Nyp/8drByS)And here is a trivial test: ~~~ var grid = new Grid(5, 5); console.log(grid.get(new Vector(1, 1))); // → undefined grid.set(new Vector(1, 1), "X"); console.log(grid.get(new Vector(1, 1))); // → X ~~~ ## [](http://eloquentjavascript.net/07_elife.html#h_OpRteCjOxs)A critter’s programming interface [](http://eloquentjavascript.net/07_elife.html#p_AVQE8U7Nsn)Before we can start on the `World` constructor, we must get more specific about the critter objects that will be living inside it. I mentioned that the world will ask the critters what actions they want to take. This works as follows: each critter object has an `act` method that, when called, returns an *action*. An action is an object with a `type` property, which names the type of action the critter wants to take, for example `"move"`. The action may also contain extra information, such as the direction the critter wants to move in. [](http://eloquentjavascript.net/07_elife.html#p_K7ngGZ/StH)Critters are terribly myopic and can see only the squares directly around them on the grid. But even this limited vision can be useful when deciding which action to take. When the `act` method is called, it is given a *view* object that allows the critter to inspect its surroundings. We name the eight surrounding squares by their compass directions: `"n"` for north, `"ne"` for northeast, and so on. Here’s the object we will use to map from direction names to coordinate offsets: ~~~ var directions = { "n": new Vector( 0, -1), "ne": new Vector( 1, -1), "e": new Vector( 1, 0), "se": new Vector( 1, 1), "s": new Vector( 0, 1), "sw": new Vector(-1, 1), "w": new Vector(-1, 0), "nw": new Vector(-1, -1) }; ~~~ [](http://eloquentjavascript.net/07_elife.html#p_g3a7FTfLaJ)The view object has a method `look`, which takes a direction and returns a character, for example `"#"` when there is a wall in that direction, or `" "`(space) when there is nothing there. The object also provides the convenient methods `find` and `findAll`. Both take a map character as an argument. The first returns a direction in which the character can be found next to the critter or returns `null` if no such direction exists. The second returns an array containing all directions with that character. For example, a creature sitting left (west) of a wall will get `["ne", "e", "se"]` when calling `findAll` on its view object with the `"#"` character as argument. [](http://eloquentjavascript.net/07_elife.html#p_HNBrqFmcYV)Here is a simple, stupid critter that just follows its nose until it hits an obstacle and then bounces off in a random open direction: ~~~ function randomElement(array) { return array[Math.floor(Math.random() * array.length)]; } var directionNames = "n ne e se s sw w nw".split(" "); function BouncingCritter() { this.direction = randomElement(directionNames); }; BouncingCritter.prototype.act = function(view) { if (view.look(this.direction) != " ") this.direction = view.find(" ") || "s"; return {type: "move", direction: this.direction}; }; ~~~ [](http://eloquentjavascript.net/07_elife.html#p_WzFfJ2mQq9)The `randomElement` helper function simply picks a random element from an array, using `Math.random` plus some arithmetic to get a random index. We’ll use this again later because randomness can be useful in simulations. [](http://eloquentjavascript.net/07_elife.html#p_pwO1hCYROA)To pick a random direction, the `BouncingCritter` constructor calls`randomElement` on an array of direction names. We could also have used`Object.keys` to get this array from the `directions` object we defined [earlier](http://eloquentjavascript.net/07_elife.html#directions), but that provides no guarantees about the order in which the properties are listed. In most situations, modern JavaScript engines will return properties in the order they were defined, but they are not required to. [](http://eloquentjavascript.net/07_elife.html#p_fi4mft+UlD)The “`|| "s"`” in the `act` method is there to prevent `this.direction` from getting the value `null` if the critter is somehow trapped with no empty space around it (for example when crowded into a corner by other critters). ## [](http://eloquentjavascript.net/07_elife.html#h_4ewxuNkubc)The world object [](http://eloquentjavascript.net/07_elife.html#p_ik4sLr54Yk)Now we can start on the `World` object type. The constructor takes a plan (the array of strings representing the world’s grid, described [earlier](http://eloquentjavascript.net/07_elife.html#grid)) and a *legend* as arguments. A legend is an object that tells us what each character in the map means. It contains a constructor for every character—except for the space character, which always refers to `null`, the value we’ll use to represent empty space. ~~~ function elementFromChar(legend, ch) { if (ch == " ") return null; var element = new legend[ch](); element.originChar = ch; return element; } function World(map, legend) { var grid = new Grid(map[0].length, map.length); this.grid = grid; this.legend = legend; map.forEach(function(line, y) { for (var x = 0; x < line.length; x++) grid.set(new Vector(x, y), elementFromChar(legend, line[x])); }); } ~~~ [](http://eloquentjavascript.net/07_elife.html#p_0JVotAPiPQ)In `elementFromChar`, first we create an instance of the right type by looking up the character’s constructor and applying `new` to it. Then we add an`originChar` property to it to make it easy to find out what character the element was originally created from. [](http://eloquentjavascript.net/07_elife.html#p_y7y9KPcPVM)We need this `originChar` property when implementing the world’s `toString`method. This method builds up a maplike string from the world’s current state by performing a two-dimensional loop over the squares on the grid. ~~~ function charFromElement(element) { if (element == null) return " "; else return element.originChar; } World.prototype.toString = function() { var output = ""; for (var y = 0; y < this.grid.height; y++) { for (var x = 0; x < this.grid.width; x++) { var element = this.grid.get(new Vector(x, y)); output += charFromElement(element); } output += "\n"; } return output; }; ~~~ [](http://eloquentjavascript.net/07_elife.html#p_H37YR7T8/r)A wall is a simple object—it is used only for taking up space and has no `act`method. ~~~ function Wall() {} ~~~ [](http://eloquentjavascript.net/07_elife.html#p_OW2cfuBxbV)When we try the `World` object by creating an instance based on the plan from[earlier in the chapter](http://eloquentjavascript.net/07_elife.html#plan) and then calling `toString` on it, we get a string very similar to the plan we put in. ~~~ var world = new World(plan, {"#": Wall, "o": BouncingCritter}); console.log(world.toString()); // → ############################ // # # # o ## // # # // # ##### # // ## # # ## # // ### ## # # // # ### # # // # #### # // # ## o # // # o # o ### # // # # # // ############################ ~~~ ## [](http://eloquentjavascript.net/07_elife.html#h_EoVtgPzygv)this and its scope [](http://eloquentjavascript.net/07_elife.html#p_QcRBGpv5Fi)The `World` constructor contains a call to `forEach`. One interesting thing to note is that inside the function passed to `forEach`, we are no longer directly in the function scope of the constructor. Each function call gets its own `this`binding, so the `this` in the inner function does *not* refer to the newly constructed object that the outer `this` refers to. In fact, when a function isn’t called as a method, `this` will refer to the global object. [](http://eloquentjavascript.net/07_elife.html#p_NZC77y3jw6)This means that we can’t write `this.grid` to access the grid from inside the loop. Instead, the outer function creates a normal local variable, `grid`, through which the inner function gets access to the grid. [](http://eloquentjavascript.net/07_elife.html#p_nCmQW78Pl2)This is a bit of a design blunder in JavaScript. Fortunately, the next version of the language provides a solution for this problem. Meanwhile, there are workarounds. A common pattern is to say `var self = this` and from then on refer to `self`, which is a normal variable and thus visible to inner functions. [](http://eloquentjavascript.net/07_elife.html#p_Rh0gGyd888)Another solution is to use the `bind` method, which allows us to provide an explicit `this` object to bind to. ~~~ var test = { prop: 10, addPropTo: function(array) { return array.map(function(elt) { return this.prop + elt; }.bind(this)); } }; console.log(test.addPropTo([5])); // → [15] ~~~ [](http://eloquentjavascript.net/07_elife.html#p_goPioKvWJ+)The function passed to `map` is the result of the `bind` call and thus has its `this`bound to the first argument given to `bind`—the outer function’s `this` value (which holds the `test` object). [](http://eloquentjavascript.net/07_elife.html#p_6Is4vwcRpS)Most standard higher-order methods on arrays, such as `forEach` and `map`, take an optional second argument that can also be used to provide a `this` for the calls to the iteration function. So you could express the previous example in a slightly simpler way. ~~~ var test = { prop: 10, addPropTo: function(array) { return array.map(function(elt) { return this.prop + elt; }, this); // ← no bind } }; console.log(test.addPropTo([5])); // → [15] ~~~ [](http://eloquentjavascript.net/07_elife.html#p_IJrpC7ALpA)This works only for higher-order functions that support such a *context*parameter. When they don’t, you’ll need to use one of the other approaches. [](http://eloquentjavascript.net/07_elife.html#p_/IbAQkWKaF)In our own higher-order functions, we can support such a context parameter by using the `call` method to call the function given as an argument. For example, here is a `forEach` method for our `Grid` type, which calls a given function for each element in the grid that isn’t null or undefined: ~~~ Grid.prototype.forEach = function(f, context) { for (var y = 0; y < this.height; y++) { for (var x = 0; x < this.width; x++) { var value = this.space[x + y * this.width]; if (value != null) f.call(context, value, new Vector(x, y)); } } }; ~~~ ## [](http://eloquentjavascript.net/07_elife.html#h_6OGIzAd5Tr)Animating life [](http://eloquentjavascript.net/07_elife.html#p_fHIh5+R55Q)The next step is to write a `turn` method for the world object that gives the critters a chance to act. It will go over the grid using the `forEach` method we just defined, looking for objects with an `act` method. When it finds one, `turn`calls that method to get an action object and carries out the action when it is valid. For now, only `"move"` actions are understood. [](http://eloquentjavascript.net/07_elife.html#p_iE5jpow+CQ)There is one potential problem with this approach. Can you spot it? If we let critters move as we come across them, they may move to a square that we haven’t looked at yet, and we’ll allow them to move *again* when we reach that square. Thus, we have to keep an array of critters that have already had their turn and ignore them when we see them again. ~~~ World.prototype.turn = function() { var acted = []; this.grid.forEach(function(critter, vector) { if (critter.act && acted.indexOf(critter) == -1) { acted.push(critter); this.letAct(critter, vector); } }, this); }; ~~~ [](http://eloquentjavascript.net/07_elife.html#p_Aa9iyVeBsc)We use the second parameter to the grid’s `forEach` method to be able to access the correct `this` inside the inner function. The `letAct` method contains the actual logic that allows the critters to move. ~~~ World.prototype.letAct = function(critter, vector) { var action = critter.act(new View(this, vector)); if (action && action.type == "move") { var dest = this.checkDestination(action, vector); if (dest && this.grid.get(dest) == null) { this.grid.set(vector, null); this.grid.set(dest, critter); } } }; World.prototype.checkDestination = function(action, vector) { if (directions.hasOwnProperty(action.direction)) { var dest = vector.plus(directions[action.direction]); if (this.grid.isInside(dest)) return dest; } }; ~~~ [](http://eloquentjavascript.net/07_elife.html#p_UrtcsyHVBc)First, we simply ask the critter to act, passing it a view object that knows about the world and the critter’s current position in that world (we’ll define `View` in a[moment](http://eloquentjavascript.net/07_elife.html#view)). The `act` method returns an action of some kind. [](http://eloquentjavascript.net/07_elife.html#p_WfOP6E9DAW)If the action’s `type` is not `"move"`, it is ignored. If it *is* `"move"`, if it has a`direction` property that refers to a valid direction, *and* if the square in that direction is empty (null), we set the square where the critter used to be to hold null and store the critter in the destination square. [](http://eloquentjavascript.net/07_elife.html#p_k+MCUoR4aQ)Note that `letAct` takes care to ignore nonsense input—it doesn’t assume that the action’s `direction` property is valid or that the `type` property makes sense. This kind of *defensive* programming makes sense in some situations. The main reason for doing it is to validate inputs coming from sources you don’t control (such as user or file input), but it can also be useful to isolate subsystems from each other. In this case, the intention is that the critters themselves can be programmed sloppily—they don’t have to verify if their intended actions make sense. They can just request an action, and the world will figure out whether to allow it. [](http://eloquentjavascript.net/07_elife.html#p_4tMXZtdiIU)These two methods are not part of the external interface of a `World` object. They are an internal detail. Some languages provide ways to explicitly declare certain methods and properties *private* and signal an error when you try to use them from outside the object. JavaScript does not, so you will have to rely on some other form of communication to describe what is part of an object’s interface. Sometimes it can help to use a naming scheme to distinguish between external and internal properties, for example by prefixing all internal ones with an underscore character (_). This will make accidental uses of properties that are not part of an object’s interface easier to spot. [](http://eloquentjavascript.net/07_elife.html#p_CEJTIYaI46)The one missing part, the `View` type, looks like this: ~~~ function View(world, vector) { this.world = world; this.vector = vector; } View.prototype.look = function(dir) { var target = this.vector.plus(directions[dir]); if (this.world.grid.isInside(target)) return charFromElement(this.world.grid.get(target)); else return "#"; }; View.prototype.findAll = function(ch) { var found = []; for (var dir in directions) if (this.look(dir) == ch) found.push(dir); return found; }; View.prototype.find = function(ch) { var found = this.findAll(ch); if (found.length == 0) return null; return randomElement(found); }; ~~~ [](http://eloquentjavascript.net/07_elife.html#p_Yr7JFXwnP2)The `look` method figures out the coordinates that we are trying to look at and, if they are inside the grid, finds the character corresponding to the element that sits there. For coordinates outside the grid, `look` simply pretends that there is a wall there so that if you define a world that isn’t walled in, the critters still won’t be tempted to try to walk off the edges. ## [](http://eloquentjavascript.net/07_elife.html#h_d0UsPgSWRD)It moves [](http://eloquentjavascript.net/07_elife.html#p_BQnR0yHaOY)We instantiated a world object earlier. Now that we’ve added all the necessary methods, it should be possible to actually make the world move. ~~~ for (var i = 0; i < 5; i++) { world.turn(); console.log(world.toString()); } // → … five turns of moving critters ~~~ [](http://eloquentjavascript.net/07_elife.html#p_HwVEJ+g6z5)Simply printing out many copies of the map is a rather unpleasant way to observe a world, though. That’s why the sandbox provides an `animateWorld`function that will run a world as an onscreen animation, moving three turns per second, until you hit the stop button. ~~~ animateWorld(world); // → … life! ~~~ [](http://eloquentjavascript.net/07_elife.html#p_jGPHPftEGJ)The implementation of `animateWorld` will remain a mystery for now, but after you’ve read the [later chapters](http://eloquentjavascript.net/13_dom.html#dom) of this book, which discuss JavaScript integration in web browsers, it won’t look so magical anymore. ## [](http://eloquentjavascript.net/07_elife.html#h_r85xE+rzsD)More life forms [](http://eloquentjavascript.net/07_elife.html#p_aVv117kVSp)The dramatic highlight of our world, if you watch for a bit, is when two critters bounce off each other. Can you think of another interesting form of behavior? [](http://eloquentjavascript.net/07_elife.html#p_ZzTWUs5sdf)The one I came up with is a critter that moves along walls. Conceptually, the critter keeps its left hand (paw, tentacle, whatever) to the wall and follows along. This turns out to be not entirely trivial to implement. [](http://eloquentjavascript.net/07_elife.html#p_+nyDt2anla)We need to be able to “compute” with compass directions. Since directions are modeled by a set of strings, we need to define our own operation (`dirPlus`) to calculate relative directions. So `dirPlus("n", 1)` means one 45-degree turn clockwise from north, giving `"ne"`. Similarly, `dirPlus("s", -2)` means 90 degrees counterclockwise from south, which is east. ~~~ function dirPlus(dir, n) { var index = directionNames.indexOf(dir); return directionNames[(index + n + 8) % 8]; } function WallFollower() { this.dir = "s"; } WallFollower.prototype.act = function(view) { var start = this.dir; if (view.look(dirPlus(this.dir, -3)) != " ") start = this.dir = dirPlus(this.dir, -2); while (view.look(this.dir) != " ") { this.dir = dirPlus(this.dir, 1); if (this.dir == start) break; } return {type: "move", direction: this.dir}; }; ~~~ [](http://eloquentjavascript.net/07_elife.html#p_vLV21F0iBf)The `act` method only has to “scan” the critter’s surroundings, starting from its left side and going clockwise until it finds an empty square. It then moves in the direction of that empty square. [](http://eloquentjavascript.net/07_elife.html#p_KvheizXuRR)What complicates things is that a critter may end up in the middle of empty space, either as its start position or as a result of walking around another critter. If we apply the approach I just described in empty space, the poor critter will just keep on turning left at every step, running in circles. [](http://eloquentjavascript.net/07_elife.html#p_TzIxUwgqju)So there is an extra check (the `if` statement) to start scanning to the left only if it looks like the critter has just passed some kind of obstacle—that is, if the space behind and to the left of the critter is not empty. Otherwise, the critter starts scanning directly ahead, so that it’ll walk straight when in empty space. [](http://eloquentjavascript.net/07_elife.html#p_h7LZenYvtj)And finally, there’s a test comparing `this.dir` to `start` after every pass through the loop to make sure that the loop won’t run forever when the critter is walled in or crowded in by other critters and can’t find an empty square. [](http://eloquentjavascript.net/07_elife.html#p_+rUHxdy6xc)This small world demonstrates the wall-following creatures: ~~~ animateWorld(new World( ["############", "# # #", "# ~ ~ #", "# ## #", "# ## o####", "# #", "############"], {"#": Wall, "~": WallFollower, "o": BouncingCritter} )); ~~~ ## [](http://eloquentjavascript.net/07_elife.html#h_yTrS5ee3KB)A more lifelike simulation [](http://eloquentjavascript.net/07_elife.html#p_WyKE9k0MVM)To make life in our world more interesting, we will add the concepts of food and reproduction. Each living thing in the world gets a new property, `energy`, which is reduced by performing actions and increased by eating things. When the critter has enough energy, it can reproduce, generating a new critter of the same kind. To keep things simple, the critters in our world reproduce asexually, all by themselves. [](http://eloquentjavascript.net/07_elife.html#p_W7zSlnat3V)If critters only move around and eat one another, the world will soon succumb to the law of increasing entropy, run out of energy, and become a lifeless wasteland. To prevent this from happening (too quickly, at least), we add plants to the world. Plants do not move. They just use photosynthesis to grow (that is, increase their energy) and reproduce. [](http://eloquentjavascript.net/07_elife.html#p_94pQzOmOVM)To make this work, we’ll need a world with a different `letAct` method. We could just replace the method of the `World` prototype, but I’ve become very attached to our simulation with the wall-following critters and would hate to break that old world. [](http://eloquentjavascript.net/07_elife.html#p_v0IDjeo0fS)One solution is to use inheritance. We create a new constructor,`LifelikeWorld`, whose prototype is based on the `World` prototype but which overrides the `letAct` method. The new `letAct` method delegates the work of actually performing an action to various functions stored in the `actionTypes`object. ~~~ function LifelikeWorld(map, legend) { World.call(this, map, legend); } LifelikeWorld.prototype = Object.create(World.prototype); var actionTypes = Object.create(null); LifelikeWorld.prototype.letAct = function(critter, vector) { var action = critter.act(new View(this, vector)); var handled = action && action.type in actionTypes && actionTypes[action.type].call(this, critter, vector, action); if (!handled) { critter.energy -= 0.2; if (critter.energy <= 0) this.grid.set(vector, null); } }; ~~~ [](http://eloquentjavascript.net/07_elife.html#p_yUSwDSWWG/)The new `letAct` method first checks whether an action was returned at all, then whether a handler function for this type of action exists, and finally whether that handler returned true, indicating that it successfully handled the action. Note the use of `call` to give the handler access to the world, through its`this` binding. [](http://eloquentjavascript.net/07_elife.html#p_XIDOYUJ2VB)If the action didn’t work for whatever reason, the default action is for the creature to simply wait. It loses one-fifth point of energy, and if its energy level drops to zero or below, the creature dies and is removed from the grid. ## [](http://eloquentjavascript.net/07_elife.html#h_s7VEeFM4T2)Action handlers [](http://eloquentjavascript.net/07_elife.html#p_RE6to1zSFD)The simplest action a creature can perform is `"grow"`, used by plants. When an action object like `{type: "grow"}` is returned, the following handler method will be called: ~~~ actionTypes.grow = function(critter) { critter.energy += 0.5; return true; }; ~~~ [](http://eloquentjavascript.net/07_elife.html#p_6k/ZhdfEll)Growing always succeeds and adds half a point to the plant’s energy level. [](http://eloquentjavascript.net/07_elife.html#p_JTfOBvLWbR)Moving is more involved. ~~~ actionTypes.move = function(critter, vector, action) { var dest = this.checkDestination(action, vector); if (dest == null || critter.energy <= 1 || this.grid.get(dest) != null) return false; critter.energy -= 1; this.grid.set(vector, null); this.grid.set(dest, critter); return true; }; ~~~ [](http://eloquentjavascript.net/07_elife.html#p_OyaLcjtdgw)This action first checks, using the `checkDestination` method defined [earlier](http://eloquentjavascript.net/07_elife.html#checkDestination), whether the action provides a valid destination. If not, or if the destination isn’t empty, or if the critter lacks the required energy, `move` returns false to indicate no action was taken. Otherwise, it moves the critter and subtracts the energy cost. [](http://eloquentjavascript.net/07_elife.html#p_0Wdb2GCOkE)In addition to moving, critters can eat. ~~~ actionTypes.eat = function(critter, vector, action) { var dest = this.checkDestination(action, vector); var atDest = dest != null && this.grid.get(dest); if (!atDest || atDest.energy == null) return false; critter.energy += atDest.energy; this.grid.set(dest, null); return true; }; ~~~ [](http://eloquentjavascript.net/07_elife.html#p_I63DQNJFYJ)Eating another critter also involves providing a valid destination square. This time, the destination must not be empty and must contain something with energy, like a critter (but not a wall—walls are not edible). If so, the energy from the eaten is transferred to the eater, and the victim is removed from the grid. [](http://eloquentjavascript.net/07_elife.html#p_aDjES09ZCB)And finally, we allow our critters to reproduce. ~~~ actionTypes.reproduce = function(critter, vector, action) { var baby = elementFromChar(this.legend, critter.originChar); var dest = this.checkDestination(action, vector); if (dest == null || critter.energy <= 2 * baby.energy || this.grid.get(dest) != null) return false; critter.energy -= 2 * baby.energy; this.grid.set(dest, baby); return true; }; ~~~ [](http://eloquentjavascript.net/07_elife.html#p_iznW2m8B2U)Reproducing costs twice the energy level of the newborn critter. So we first create a (hypothetical) baby using `elementFromChar` on the critter’s own origin character. Once we have a baby, we can find its energy level and test whether the parent has enough energy to successfully bring it into the world. We also require a valid (and empty) destination. [](http://eloquentjavascript.net/07_elife.html#p_stSVDa0fH8)If everything is okay, the baby is put onto the grid (it is now no longer hypothetical), and the energy is spent. ## [](http://eloquentjavascript.net/07_elife.html#h_vziJXy2ZdX)Populating the new world [](http://eloquentjavascript.net/07_elife.html#p_vGmE0fy2RS)We now have a framework to simulate these more lifelike creatures. We could put the critters from the old world into it, but they would just die since they don’t have an energy property. So let’s make new ones. First we’ll write a plant, which is a rather simple life-form. ~~~ function Plant() { this.energy = 3 + Math.random() * 4; } Plant.prototype.act = function(view) { if (this.energy > 15) { var space = view.find(" "); if (space) return {type: "reproduce", direction: space}; } if (this.energy < 20) return {type: "grow"}; }; ~~~ [](http://eloquentjavascript.net/07_elife.html#p_OCJXcCZPZm)Plants start with an energy level between 3 and 7, randomized so that they don’t all reproduce in the same turn. When a plant reaches 15 energy points and there is empty space nearby, it reproduces into that empty space. If a plant can’t reproduce, it simply grows until it reaches energy level 20. [](http://eloquentjavascript.net/07_elife.html#p_xjp5a11XH7)We now define a plant eater. ~~~ function PlantEater() { this.energy = 20; } PlantEater.prototype.act = function(view) { var space = view.find(" "); if (this.energy > 60 && space) return {type: "reproduce", direction: space}; var plant = view.find("*"); if (plant) return {type: "eat", direction: plant}; if (space) return {type: "move", direction: space}; }; ~~~ [](http://eloquentjavascript.net/07_elife.html#p_JYpELDqyyT)We’ll use the `*` character for plants, so that’s what this creature will look for when it searches for food. ## [](http://eloquentjavascript.net/07_elife.html#h_pxU2HNE0gc)Bringing it to life [](http://eloquentjavascript.net/07_elife.html#p_/pimHqjNal)And that gives us enough elements to try our new world. Imagine the following map as a grassy valley with a herd of herbivores in it, some boulders, and lush plant life everywhere. ~~~ var valley = new LifelikeWorld( ["############################", "##### ######", "## *** **##", "# *##** ** O *##", "# *** O ##** *#", "# O ##*** #", "# ##** #", "# O #* #", "#* #** O #", "#*** ##** O **#", "##**** ###*** *###", "############################"], {"#": Wall, "O": PlantEater, "*": Plant} ); ~~~ [](http://eloquentjavascript.net/07_elife.html#p_XZiEmlppan)Let’s see what happens if we run this. ~~~ animateWorld(valley); ~~~ [](http://eloquentjavascript.net/07_elife.html#p_1fhW4rcJRU)Most of the time, the plants multiply and expand quite quickly, but then the abundance of food causes a population explosion of the herbivores, who proceed to wipe out all or nearly all of the plants, resulting in a mass starvation of the critters. Sometimes, the ecosystem recovers and another cycle starts. At other times, one of the species dies out completely. If it’s the herbivores, the whole space will fill with plants. If it’s the plants, the remaining critters starve, and the valley becomes a desolate wasteland. Ah, the cruelty of nature. ## [](http://eloquentjavascript.net/07_elife.html#h_TcUD2vzyMe)Exercises ### [](http://eloquentjavascript.net/07_elife.html#h_lU2HuCXhbH)Artificial stupidity [](http://eloquentjavascript.net/07_elife.html#p_w0CByGW7g5)Having the inhabitants of our world go extinct after a few minutes is kind of depressing. To deal with this, we could try to create a smarter plant eater. [](http://eloquentjavascript.net/07_elife.html#p_G39rqGC9Ql)There are several obvious problems with our herbivores. First, they are terribly greedy, stuffing themselves with every plant they see until they have wiped out the local plant life. Second, their randomized movement (recall that the`view.find` method returns a random direction when multiple directions match) causes them to stumble around ineffectively and starve if there don’t happen to be any plants nearby. And finally, they breed very fast, which makes the cycles between abundance and famine quite intense. [](http://eloquentjavascript.net/07_elife.html#p_RoYABqty7W)Write a new critter type that tries to address one or more of these points and substitute it for the old `PlantEater` type in the valley world. See how it fares. Tweak it some more if necessary. ~~~ // Your code here function SmartPlantEater() {} animateWorld(new LifelikeWorld( ["############################", "##### ######", "## *** **##", "# *##** ** O *##", "# *** O ##** *#", "# O ##*** #", "# ##** #", "# O #* #", "#* #** O #", "#*** ##** O **#", "##**** ###*** *###", "############################"], {"#": Wall, "O": SmartPlantEater, "*": Plant} )); ~~~ ### [](http://eloquentjavascript.net/07_elife.html#h_QUnD2hCBv4)Predators [](http://eloquentjavascript.net/07_elife.html#p_wLAPo3ZZqC)Any serious ecosystem has a food chain longer than a single link. Write another critter that survives by eating the herbivore critter. You’ll notice that stability is even harder to achieve now that there are cycles at multiple levels. Try to find a strategy to make the ecosystem run smoothly for at least a little while. [](http://eloquentjavascript.net/07_elife.html#p_jEi7Cawsdy)One thing that will help is to make the world bigger. This way, local population booms or busts are less likely to wipe out a species entirely, and there is space for the relatively large prey population needed to sustain a small predator population. ~~~ // Your code here function Tiger() {} animateWorld(new LifelikeWorld( ["####################################################", "# #### **** ###", "# * @ ## ######## OO ##", "# * ## O O **** *#", "# ##* ########## *#", "# ##*** * **** **#", "#* ** # * *** ######### **#", "#* ** # * # * **#", "# ## # O # *** ######", "#* @ # # * O # #", "#* # ###### ** #", "### **** *** ** #", "# O @ O #", "# * ## ## ## ## ### * #", "# ** # * ##### O #", "## ** O O # # *** *** ### ** #", "### # ***** ****#", "####################################################"], {"#": Wall, "@": Tiger, "O": SmartPlantEater, // from previous exercise "*": Plant} )); ~~~ Many of the same tricks that worked for the previous exercise also apply here. Making the predators big (lots of energy) and having them reproduce slowly is recommended. That’ll make them less vulnerable to periods of starvation when the herbivores are scarce. Beyond staying alive, keeping its food stock alive is a predator’s main objective. Find some way to make predators hunt more aggressively when there are a lot of herbivores and hunt more slowly (or not at all) when prey is rare. Since plant eaters move around, the simple trick of eating one only when others are nearby is unlikely to work—that’ll happen so rarely that your predator will starve. But you could keep track of observations in previous turns, in some data structure kept on the predator objects, and have it base its behavior on what it has seen recently.

## Chapter 6 # The Secret Life of Objects > [](http://eloquentjavascript.net/06_object.html#p_C9NMk2tmA4)The problem with object-oriented languages is they’ve got all this implicit environment that they carry around with them. You wanted a banana but what you got was a gorilla holding the banana and the entire jungle. > > Joe Armstrong, interviewed in Coders at Work [](http://eloquentjavascript.net/06_object.html#p_fPdVAHtS9s)When a programmer says “object”, this is a loaded term. In my profession, objects are a way of life, the subject of holy wars, and a beloved buzzword that still hasn’t quite lost its power. [](http://eloquentjavascript.net/06_object.html#p_1l/AXS0w/+)To an outsider, this is probably a little confusing. Let’s start with a brief history of objects as a programming construct. ## [](http://eloquentjavascript.net/06_object.html#h_kMzWSXQAtV)History [](http://eloquentjavascript.net/06_object.html#p_7Qth+HqmZl)This story, like most programming stories, starts with the problem of complexity. One philosophy is that complexity can be made manageable by separating it into small compartments that are isolated from each other. These compartments have ended up with the name *objects*. [](http://eloquentjavascript.net/06_object.html#p_UuUhBp47+J)An object is a hard shell that hides the gooey complexity inside it and instead offers us a few knobs and connectors (such as methods) that present an*interface* through which the object is to be used. The idea is that the interface is relatively simple and all the complex things going on *inside* the object can be ignored when working with it.  [](http://eloquentjavascript.net/06_object.html#p_KUiP7xA3eo)As an example, you can imagine an object that provides an interface to an area on your screen. It provides a way to draw shapes or text onto this area but hides all the details of how these shapes are converted to the actual pixels that make up the screen. You’d have a set of methods—for example, `drawCircle`—and those are the only things you need to know in order to use such an object. [](http://eloquentjavascript.net/06_object.html#p_2cwmac7A3z)These ideas were initially worked out in the 1970s and 1980s and, in the 1990s, were carried up by a huge wave of hype—the object-oriented programming revolution. Suddenly, there was a large tribe of people declaring that objects were the *right* way to program—and that anything that did not involve objects was outdated nonsense. [](http://eloquentjavascript.net/06_object.html#p_g9yaxQcUB0)That kind of zealotry always produces a lot of impractical silliness, and there has been a sort of counter-revolution since then. In some circles, objects have a rather bad reputation nowadays. [](http://eloquentjavascript.net/06_object.html#p_yDJcWLiV1q)I prefer to look at the issue from a practical, rather than ideological, angle. There are several useful concepts, most importantly that of *encapsulation*(distinguishing between internal complexity and external interface), that the object-oriented culture has popularized. These are worth studying. [](http://eloquentjavascript.net/06_object.html#p_uZjg5Fd1Wj)This chapter describes JavaScript’s rather eccentric take on objects and the way they relate to some classical object-oriented techniques. ## [](http://eloquentjavascript.net/06_object.html#h_fkrGgDyRWc)Methods [](http://eloquentjavascript.net/06_object.html#p_cbCII9cU5W)Methods are simply properties that hold function values. This is a simple method: ~~~ var rabbit = {}; rabbit.speak = function(line) { console.log("The rabbit says '" + line + "'"); }; rabbit.speak("I'm alive."); // → The rabbit says 'I'm alive.' ~~~ [](http://eloquentjavascript.net/06_object.html#p_N+6e0UGvFo)Usually a method needs to do something with the object it was called on. When a function is called as a method—looked up as a property and immediately called, as in `object.method()`—the special variable `this` in its body will point to the object that it was called on. ~~~ function speak(line) { console.log("The " + this.type + " rabbit says '" + line + "'"); } var whiteRabbit = {type: "white", speak: speak}; var fatRabbit = {type: "fat", speak: speak}; whiteRabbit.speak("Oh my ears and whiskers, " + "how late it's getting!"); // → The white rabbit says 'Oh my ears and whiskers, how // late it's getting!' fatRabbit.speak("I could sure use a carrot right now."); // → The fat rabbit says 'I could sure use a carrot // right now.' ~~~ [](http://eloquentjavascript.net/06_object.html#p_FbPIv/+6x+)The code uses the `this` keyword to output the type of rabbit that is speaking. Recall that the `apply` and `bind` methods both take a first argument that can be used to simulate method calls. This first argument is in fact used to give a value to `this`. [](http://eloquentjavascript.net/06_object.html#p_jHNP07w/lm)There is a method similar to `apply`, called `call`. It also calls the function it is a method of but takes its arguments normally, rather than as an array. Like`apply` and `bind`, `call` can be passed a specific `this` value. ~~~ speak.apply(fatRabbit, ["Burp!"]); // → The fat rabbit says 'Burp!' speak.call({type: "old"}, "Oh my."); // → The old rabbit says 'Oh my.' ~~~ ## [](http://eloquentjavascript.net/06_object.html#h_SumMlRB7yn)Prototypes [](http://eloquentjavascript.net/06_object.html#p_hi1TWnD/2p)Watch closely. ~~~ var empty = {}; console.log(empty.toString); // → function toString(){…} console.log(empty.toString()); // → [object Object] ~~~ [](http://eloquentjavascript.net/06_object.html#p_aX24Mkkmn9)I just pulled a property out of an empty object. Magic! [](http://eloquentjavascript.net/06_object.html#p_ymaGpRwPF5)Well, not really. I have simply been withholding information about the way JavaScript objects work. In addition to their set of properties, almost all objects also have a *prototype*. A prototype is another object that is used as a fallback source of properties. When an object gets a request for a property that it does not have, its prototype will be searched for the property, then the prototype’s prototype, and so on. [](http://eloquentjavascript.net/06_object.html#p_7FLJjsfmk4)So who is the prototype of that empty object? It is the great ancestral prototype, the entity behind almost all objects, `Object.prototype`. ~~~ console.log(Object.getPrototypeOf({}) == Object.prototype); // → true console.log(Object.getPrototypeOf(Object.prototype)); // → null ~~~ [](http://eloquentjavascript.net/06_object.html#p_vHbNlN1h5h)As you might expect, the `Object.getPrototypeOf` function returns the prototype of an object. [](http://eloquentjavascript.net/06_object.html#p_4S0XbM8aBm)The prototype relations of JavaScript objects form a tree-shaped structure, and at the root of this structure sits `Object.prototype`. It provides a few methods that show up in all objects, such as `toString`, which converts an object to a string representation. [](http://eloquentjavascript.net/06_object.html#p_LPg9dSSFAi)Many objects don’t directly have `Object.prototype` as their prototype, but instead have another object, which provides its own default properties. Functions derive from `Function.prototype`, and arrays derive from`Array.prototype`. ~~~ console.log(Object.getPrototypeOf(isNaN) == Function.prototype); // → true console.log(Object.getPrototypeOf([]) == Array.prototype); // → true ~~~ [](http://eloquentjavascript.net/06_object.html#p_QVomcVCyPH)Such a prototype object will itself have a prototype, often `Object.prototype`, so that it still indirectly provides methods like `toString`. [](http://eloquentjavascript.net/06_object.html#p_TzOd94K1HD)The `Object.getPrototypeOf` function obviously returns the prototype of an object. You can use `Object.create` to create an object with a specific prototype. ~~~ var protoRabbit = { speak: function(line) { console.log("The " + this.type + " rabbit says '" + line + "'"); } }; var killerRabbit = Object.create(protoRabbit); killerRabbit.type = "killer"; killerRabbit.speak("SKREEEE!"); // → The killer rabbit says 'SKREEEE!' ~~~ [](http://eloquentjavascript.net/06_object.html#p_y8kzFoQw1W)The “proto” rabbit acts as a container for the properties that are shared by all rabbits. An individual rabbit object, like the killer rabbit, contains properties that apply only to itself—in this case its type—and derives shared properties from its prototype. ## [](http://eloquentjavascript.net/06_object.html#h_YKXJZqcaJA)Constructors [](http://eloquentjavascript.net/06_object.html#p_y83XvnG8ez)A more convenient way to create objects that derive from some shared prototype is to use a *constructor*. In JavaScript, calling a function with the `new`keyword in front of it causes it to be treated as a constructor. The constructor will have its `this` variable bound to a fresh object, and unless it explicitly returns another object value, this new object will be returned from the call. [](http://eloquentjavascript.net/06_object.html#p_uPw4dUu94k)An object created with `new` is said to be an *instance* of its constructor. [](http://eloquentjavascript.net/06_object.html#p_v2jJnPrF/1)Here is a simple constructor for rabbits. It is a convention to capitalize the names of constructors so that they are easily distinguished from other functions. ~~~ function Rabbit(type) { this.type = type; } var killerRabbit = new Rabbit("killer"); var blackRabbit = new Rabbit("black"); console.log(blackRabbit.type); // → black ~~~ [](http://eloquentjavascript.net/06_object.html#p_ZZ4u3YPk72)Constructors (in fact, all functions) automatically get a property named`prototype`, which by default holds a plain, empty object that derives from`Object.prototype`. Every instance created with this constructor will have this object as its prototype. So to add a `speak` method to rabbits created with the`Rabbit` constructor, we can simply do this: ~~~ Rabbit.prototype.speak = function(line) { console.log("The " + this.type + " rabbit says '" + line + "'"); }; blackRabbit.speak("Doom..."); // → The black rabbit says 'Doom...' ~~~ [](http://eloquentjavascript.net/06_object.html#p_4sWuvx6wkg)It is important to note the distinction between the way a prototype is associated with a constructor (through its `prototype` property) and the way objects *have*a prototype (which can be retrieved with `Object.getPrototypeOf`). The actual prototype of a constructor is `Function.prototype` since constructors are functions. Its `prototype` *property* will be the prototype of instances created through it but is not its *own* prototype. ## [](http://eloquentjavascript.net/06_object.html#h_oUlUep3Os8)Overriding derived properties [](http://eloquentjavascript.net/06_object.html#p_TULrs9Y+I2)When you add a property to an object, whether it is present in the prototype or not, the property is added to the object *itself*, which will henceforth have it as its own property. If there *is* a property by the same name in the prototype, this property will no longer affect the object. The prototype itself is not changed. ~~~ Rabbit.prototype.teeth = "small"; console.log(killerRabbit.teeth); // → small killerRabbit.teeth = "long, sharp, and bloody"; console.log(killerRabbit.teeth); // → long, sharp, and bloody console.log(blackRabbit.teeth); // → small console.log(Rabbit.prototype.teeth); // → small ~~~ [](http://eloquentjavascript.net/06_object.html#p_HM5YtS3KgJ)The following diagram sketches the situation after this code has run. The`Rabbit` and `Object` prototypes lie behind `killerRabbit` as a kind of backdrop, where properties that are not found in the object itself can be looked up.  [](http://eloquentjavascript.net/06_object.html#p_or3/lz1DV8)Overriding properties that exist in a prototype is often a useful thing to do. As the rabbit teeth example shows, it can be used to express exceptional properties in instances of a more generic class of objects, while letting the nonexceptional objects simply take a standard value from their prototype. [](http://eloquentjavascript.net/06_object.html#p_O8pyFABp7m)It is also used to give the standard function and array prototypes a different`toString` method than the basic object prototype. ~~~ console.log(Array.prototype.toString == Object.prototype.toString); // → false console.log([1, 2].toString()); // → 1,2 ~~~ [](http://eloquentjavascript.net/06_object.html#p_qmKAzHrRQG)Calling `toString` on an array gives a result similar to calling `.join(",")` on it—it puts commas between the values in the array. Directly calling`Object.prototype.toString` with an array produces a different string. That function doesn’t know about arrays, so it simply puts the word “object” and the name of the type between square brackets. ~~~ console.log(Object.prototype.toString.call([1, 2])); // → [object Array] ~~~ ## [](http://eloquentjavascript.net/06_object.html#h_QIL5obVMhZ)Prototype interference [](http://eloquentjavascript.net/06_object.html#p_N34jQ1q+jC)A prototype can be used at any time to add new properties and methods to all objects based on it. For example, it might become necessary for our rabbits to dance. ~~~ Rabbit.prototype.dance = function() { console.log("The " + this.type + " rabbit dances a jig."); }; killerRabbit.dance(); // → The killer rabbit dances a jig. ~~~ [](http://eloquentjavascript.net/06_object.html#p_U5nebQZlJ4)That’s convenient. But there are situations where it causes problems. In previous chapters, we used an object as a way to associate values with names by creating properties for the names and giving them the corresponding value as their value. Here’s an example from [Chapter 4](http://eloquentjavascript.net/04_data.html#object_map): ~~~ var map = {}; function storePhi(event, phi) { map[event] = phi; } storePhi("pizza", 0.069); storePhi("touched tree", -0.081); ~~~ [](http://eloquentjavascript.net/06_object.html#p_FtECKQ2UPV)We can iterate over all phi values in the object using a `for`/`in` loop and test whether a name is in there using the regular `in` operator. But unfortunately, the object’s prototype gets in the way. ~~~ Object.prototype.nonsense = "hi"; for (var name in map) console.log(name); // → pizza // → touched tree // → nonsense console.log("nonsense" in map); // → true console.log("toString" in map); // → true // Delete the problematic property again delete Object.prototype.nonsense; ~~~ [](http://eloquentjavascript.net/06_object.html#p_xoSFOumlvj)That’s all wrong. There is no event called “nonsense” in our data set. And there*definitely* is no event called “toString”. [](http://eloquentjavascript.net/06_object.html#p_NGyDSRSyjr)Oddly, `toString` did not show up in the `for`/`in` loop, but the `in` operator did return true for it. This is because JavaScript distinguishes between *enumerable*and *nonenumerable* properties. [](http://eloquentjavascript.net/06_object.html#p_xfPTy1q4XS)All properties that we create by simply assigning to them are enumerable. The standard properties in `Object.prototype` are all nonenumerable, which is why they do not show up in such a `for`/`in` loop. [](http://eloquentjavascript.net/06_object.html#p_sgLvQxcvMV)It is possible to define our own nonenumerable properties by using the`Object.defineProperty` function, which allows us to control the type of property we are creating. ~~~ Object.defineProperty(Object.prototype, "hiddenNonsense", {enumerable: false, value: "hi"}); for (var name in map) console.log(name); // → pizza // → touched tree console.log(map.hiddenNonsense); // → hi ~~~ [](http://eloquentjavascript.net/06_object.html#p_wMLrx6TIjh)So now the property is there, but it won’t show up in a loop. That’s good. But we still have the problem with the regular `in` operator claiming that the`Object.prototype` properties exist in our object. For that, we can use the object’s `hasOwnProperty` method. ~~~ console.log(map.hasOwnProperty("toString")); // → false ~~~ [](http://eloquentjavascript.net/06_object.html#p_1/lgWgImJc)This method tells us whether the object *itself* has the property, without looking at its prototypes. This is often a more useful piece of information than what the`in` operator gives us. [](http://eloquentjavascript.net/06_object.html#p_BFvrJTamlf)When you are worried that someone (some other code you loaded into your program) might have messed with the base object prototype, I recommend you write your `for`/`in` loops like this: ~~~ for (var name in map) { if (map.hasOwnProperty(name)) { // ... this is an own property } } ~~~ ## [](http://eloquentjavascript.net/06_object.html#h_1vS5Ik5kV5)Prototype-less objects [](http://eloquentjavascript.net/06_object.html#p_kSmosu1+EH)But the rabbit hole doesn’t end there. What if someone registered the name`hasOwnProperty` in our `map` object and set it to the value 42? Now the call to`map.hasOwnProperty` will try to call the local property, which holds a number, not a function. [](http://eloquentjavascript.net/06_object.html#p_tjlKDpfc75)In such a case, prototypes just get in the way, and we would actually prefer to have objects without prototypes. We saw the `Object.create` function, which allows us to create an object with a specific prototype. You are allowed to pass`null` as the prototype to create a fresh object with no prototype. For objects like `map`, where the properties could be anything, this is exactly what we want. ~~~ var map = Object.create(null); map["pizza"] = 0.069; console.log("toString" in map); // → false console.log("pizza" in map); // → true ~~~ [](http://eloquentjavascript.net/06_object.html#p_jKSTfVS3KH)Much better! We no longer need the `hasOwnProperty` kludge because all the properties the object has are its own properties. Now we can safely use `for`/`in`loops, no matter what people have been doing to `Object.prototype`. ## [](http://eloquentjavascript.net/06_object.html#h_mJ/JHQRHg9)Polymorphism [](http://eloquentjavascript.net/06_object.html#p_+2Ates4jdf)When you call the `String` function, which converts a value to a string, on an object, it will call the `toString` method on that object to try to create a meaningful string to return. I mentioned that some of the standard prototypes define their own version of `toString` so they can create a string that contains more useful information than `"[object Object]"`. [](http://eloquentjavascript.net/06_object.html#p_HgUFdFB6vM)This is a simple instance of a powerful idea. When a piece of code is written to work with objects that have a certain interface—in this case, a `toString`method—any kind of object that happens to support this interface can be plugged into the code, and it will just work. [](http://eloquentjavascript.net/06_object.html#p_phZ92zNL98)This technique is called *polymorphism*—though no actual shape-shifting is involved. Polymorphic code can work with values of different shapes, as long as they support the interface it expects. ## [](http://eloquentjavascript.net/06_object.html#h_36C2FHHi44)Laying out a table [](http://eloquentjavascript.net/06_object.html#p_nOQLqBWk1x)I am going to work through a slightly more involved example in an attempt to give you a better idea what polymorphism, as well as object-oriented programming in general, looks like. The project is this: we will write a program that, given an array of arrays of table cells, builds up a string that contains a nicely laid out table—meaning that the columns are straight and the rows are aligned. Something like this: ~~~ name height country ------------ ------ ------------- Kilimanjaro 5895 Tanzania Everest 8848 Nepal Mount Fuji 3776 Japan Mont Blanc 4808 Italy/France Vaalserberg 323 Netherlands Denali 6168 United States Popocatepetl 5465 Mexico ~~~ [](http://eloquentjavascript.net/06_object.html#p_qmL/8YKol6)The way our table-building system will work is that the builder function will ask each cell how wide and high it wants to be and then use this information to determine the width of the columns and the height of the rows. The builder function will then ask the cells to draw themselves at the correct size and assemble the results into a single string. [](http://eloquentjavascript.net/06_object.html#p_AbogVAH0Wj)The layout program will communicate with the cell objects through a well-defined interface. That way, the types of cells that the program supports is not fixed in advance. We can add new cell styles later—for example, underlined cells for table headers—and if they support our interface, they will just work, without requiring changes to the layout program. [](http://eloquentjavascript.net/06_object.html#p_Oe6ruNqozx)This is the interface: * [](http://eloquentjavascript.net/06_object.html#p_/UA7VFmSpc) `minHeight()` returns a number indicating the minimum height this cell requires (in lines). * [](http://eloquentjavascript.net/06_object.html#p_92QZ/amzwk) `minWidth()` returns a number indicating this cell’s minimum width (in characters). * [](http://eloquentjavascript.net/06_object.html#p_yx5zfEFlc0) `draw(width, height)` returns an array of length `height`, which contains a series of strings that are each `width` characters wide. This represents the content of the cell. [](http://eloquentjavascript.net/06_object.html#p_26lYAjlPjb)I’m going to make heavy use of higher-order array methods in this example since it lends itself well to that approach. [](http://eloquentjavascript.net/06_object.html#p_GckWQ2f1q/)The first part of the program computes arrays of minimum column widths and row heights for a grid of cells. The `rows` variable will hold an array of arrays, with each inner array representing a row of cells. ~~~ function rowHeights(rows) { return rows.map(function(row) { return row.reduce(function(max, cell) { return Math.max(max, cell.minHeight()); }, 0); }); } function colWidths(rows) { return rows[0].map(function(_, i) { return rows.reduce(function(max, row) { return Math.max(max, row[i].minWidth()); }, 0); }); } ~~~ [](http://eloquentjavascript.net/06_object.html#p_tFp8Lj1YMr)Using a variable name starting with an underscore (_) or consisting entirely of a single underscore is a way to indicate (to human readers) that this argument is not going to be used. [](http://eloquentjavascript.net/06_object.html#p_LkGlygwwP+)The `rowHeights` function shouldn’t be too hard to follow. It uses `reduce` to compute the maximum height of an array of cells and wraps that in `map` in order to do it for all rows in the `rows` array. [](http://eloquentjavascript.net/06_object.html#p_aeXoDNewFW)Things are slightly harder for the `colWidths` function because the outer array is an array of rows, not of columns. I have failed to mention so far that `map` (as well as `forEach`, `filter`, and similar array methods) passes a second argument to the function it is given: the index of the current element. By mapping over the elements of the first row and only using the mapping function’s second argument, `colWidths` builds up an array with one element for every column index. The call to `reduce` runs over the outer `rows` array for each index and picks out the width of the widest cell at that index. [](http://eloquentjavascript.net/06_object.html#p_/Tbz5eCViE)Here’s the code to draw a table: ~~~ function drawTable(rows) { var heights = rowHeights(rows); var widths = colWidths(rows); function drawLine(blocks, lineNo) { return blocks.map(function(block) { return block[lineNo]; }).join(" "); } function drawRow(row, rowNum) { var blocks = row.map(function(cell, colNum) { return cell.draw(widths[colNum], heights[rowNum]); }); return blocks[0].map(function(_, lineNo) { return drawLine(blocks, lineNo); }).join("\n"); } return rows.map(drawRow).join("\n"); } ~~~ [](http://eloquentjavascript.net/06_object.html#p_QSCDyHczZF)The `drawTable` function uses the internal helper function `drawRow` to draw all rows and then joins them together with newline characters. [](http://eloquentjavascript.net/06_object.html#p_b8wKD7NyzT)The `drawRow` function itself first converts the cell objects in the row to *blocks*, which are arrays of strings representing the content of the cells, split by line. A single cell containing simply the number 3776 might be represented by a single-element array like `["3776"]`, whereas an underlined cell might take up two lines and be represented by the array `["name", "----"]`. [](http://eloquentjavascript.net/06_object.html#p_uYZRAIhAHn)The blocks for a row, which all have the same height, should appear next to each other in the final output. The second call to `map` in `drawRow` builds up this output line by line by mapping over the lines in the leftmost block and, for each of those, collecting a line that spans the full width of the table. These lines are then joined with newline characters to provide the whole row as `drawRow`’s return value. [](http://eloquentjavascript.net/06_object.html#p_1AKETNclQn)The function `drawLine` extracts lines that should appear next to each other from an array of blocks and joins them with a space character to create a one-character gap between the table’s columns. [](http://eloquentjavascript.net/06_object.html#p_bbKpGggVOW)Now let’s write a constructor for cells that contain text, which implements the interface for table cells. The constructor splits a string into an array of lines using the string method `split`, which cuts up a string at every occurrence of its argument and returns an array of the pieces. The `minWidth` method finds the maximum line width in this array. ~~~ function repeat(string, times) { var result = ""; for (var i = 0; i < times; i++) result += string; return result; } function TextCell(text) { this.text = text.split("\n"); } TextCell.prototype.minWidth = function() { return this.text.reduce(function(width, line) { return Math.max(width, line.length); }, 0); }; TextCell.prototype.minHeight = function() { return this.text.length; }; TextCell.prototype.draw = function(width, height) { var result = []; for (var i = 0; i < height; i++) { var line = this.text[i] || ""; result.push(line + repeat(" ", width - line.length)); } return result; }; ~~~ [](http://eloquentjavascript.net/06_object.html#p_XqC4cWwgYI)The code uses a helper function called `repeat`, which builds a string whose value is the `string` argument repeated `times` number of times. The `draw`method uses it to add “padding” to lines so that they all have the required length. [](http://eloquentjavascript.net/06_object.html#p_4T1Kk1qpd9)Let’s try everything we’ve written so far by building up a 5 × 5 checkerboard. ~~~ var rows = []; for (var i = 0; i < 5; i++) { var row = []; for (var j = 0; j < 5; j++) { if ((j + i) % 2 == 0) row.push(new TextCell("##")); else row.push(new TextCell(" ")); } rows.push(row); } console.log(drawTable(rows)); // → ## ## ## // ## ## // ## ## ## // ## ## // ## ## ## ~~~ [](http://eloquentjavascript.net/06_object.html#p_ZlgwbD41JK)It works! But since all cells have the same size, the table-layout code doesn’t really do anything interesting. [](http://eloquentjavascript.net/06_object.html#p_ziLMS9iu5Y)The source data for the table of mountains that we are trying to build is available in the `MOUNTAINS` variable in the sandbox and also [downloadable](http://eloquentjavascript.net/code/mountains.js)from the website. [](http://eloquentjavascript.net/06_object.html#p_UiCC/VFl9F)We will want to highlight the top row, which contains the column names, by underlining the cells with a series of dash characters. No problem—we simply write a cell type that handles underlining. ~~~ function UnderlinedCell(inner) { this.inner = inner; } UnderlinedCell.prototype.minWidth = function() { return this.inner.minWidth(); }; UnderlinedCell.prototype.minHeight = function() { return this.inner.minHeight() + 1; }; UnderlinedCell.prototype.draw = function(width, height) { return this.inner.draw(width, height - 1) .concat([repeat("-", width)]); }; ~~~ [](http://eloquentjavascript.net/06_object.html#p_1VGPKHtYoJ)An underlined cell *contains* another cell. It reports its minimum size as being the same as that of its inner cell (by calling through to that cell’s `minWidth` and`minHeight` methods) but adds one to the height to account for the space taken up by the underline. [](http://eloquentjavascript.net/06_object.html#p_9oFg/rPQCi)Drawing such a cell is quite simple—we take the content of the inner cell and concatenate a single line full of dashes to it. [](http://eloquentjavascript.net/06_object.html#p_e8CuToYm1C)Having an underlining mechanism, we can now write a function that builds up a grid of cells from our data set. ~~~ function dataTable(data) { var keys = Object.keys(data[0]); var headers = keys.map(function(name) { return new UnderlinedCell(new TextCell(name)); }); var body = data.map(function(row) { return keys.map(function(name) { return new TextCell(String(row[name])); }); }); return [headers].concat(body); } console.log(drawTable(dataTable(MOUNTAINS))); // → name height country // ------------ ------ ------------- // Kilimanjaro 5895 Tanzania // … etcetera ~~~ [](http://eloquentjavascript.net/06_object.html#p_cv0EdFlmx8)The standard `Object.keys` function returns an array of property names in an object. The top row of the table must contain underlined cells that give the names of the columns. Below that, the values of all the objects in the data set appear as normal cells—we extract them by mapping over the `keys` array so that we are sure that the order of the cells is the same in every row. [](http://eloquentjavascript.net/06_object.html#p_8Kn9AmEQyS)The resulting table resembles the example shown before, except that it does not right-align the numbers in the `height` column. We will get to that in a moment. ## [](http://eloquentjavascript.net/06_object.html#h_Kd3nnpSvTd)Getters and setters [](http://eloquentjavascript.net/06_object.html#p_nkLul78hk8)When specifying an interface, it is possible to include properties that are not methods. We could have defined `minHeight` and `minWidth` to simply hold numbers. But that’d have required us to compute them in the constructor, which adds code there that isn’t strictly relevant to *constructing* the object. It would cause problems if, for example, the inner cell of an underlined cell was changed, at which point the size of the underlined cell should also change. [](http://eloquentjavascript.net/06_object.html#p_U+A3uiYe5T)This has led some people to adopt a principle of never including nonmethod properties in interfaces. Rather than directly access a simple value property, they’d use `getSomething` and `setSomething` methods to read and write the property. This approach has the downside that you will end up writing—and reading—a lot of additional methods. [](http://eloquentjavascript.net/06_object.html#p_SkpjeeWsdC)Fortunately, JavaScript provides a technique that gets us the best of both worlds. We can specify properties that, from the outside, look like normal properties but secretly have methods associated with them. ~~~ var pile = { elements: ["eggshell", "orange peel", "worm"], get height() { return this.elements.length; }, set height(value) { console.log("Ignoring attempt to set height to", value); } }; console.log(pile.height); // → 3 pile.height = 100; // → Ignoring attempt to set height to 100 ~~~ [](http://eloquentjavascript.net/06_object.html#p_av+X6VQ/sQ)In object literal, the `get` or `set` notation for properties allows you to specify a function to be run when the property is read or written. You can also add such a property to an existing object, for example a prototype, using the`Object.defineProperty` function (which we previously used to create nonenumerable properties). ~~~ Object.defineProperty(TextCell.prototype, "heightProp", { get: function() { return this.text.length; } }); var cell = new TextCell("no\nway"); console.log(cell.heightProp); // → 2 cell.heightProp = 100; console.log(cell.heightProp); // → 2 ~~~ [](http://eloquentjavascript.net/06_object.html#p_g533VU/mhK)You can use a similar `set` property, in the object passed to `defineProperty`, to specify a setter method. When a getter but no setter is defined, writing to the property is simply ignored. ## [](http://eloquentjavascript.net/06_object.html#h_/a3bnONnws)Inheritance [](http://eloquentjavascript.net/06_object.html#p_pcw8AFLxa1)We are not quite done yet with our table layout exercise. It helps readability to right-align columns of numbers. We should create another cell type that is like`TextCell`, but rather than padding the lines on the right side, it pads them on the left side so that they align to the right. [](http://eloquentjavascript.net/06_object.html#p_ozN/QrVH/c)We could simply write a whole new constructor with all three methods in its prototype. But prototypes may themselves have prototypes, and this allows us to do something clever. ~~~ function RTextCell(text) { TextCell.call(this, text); } RTextCell.prototype = Object.create(TextCell.prototype); RTextCell.prototype.draw = function(width, height) { var result = []; for (var i = 0; i < height; i++) { var line = this.text[i] || ""; result.push(repeat(" ", width - line.length) + line); } return result; }; ~~~ [](http://eloquentjavascript.net/06_object.html#p_+8Q1+4H5LI)We reuse the constructor and the `minHeight` and `minWidth` methods from the regular `TextCell`. An `RTextCell` is now basically equivalent to a `TextCell`, except that its `draw` method contains a different function. [](http://eloquentjavascript.net/06_object.html#p_hncaW/95ov)This pattern is called *inheritance*. It allows us to build slightly different data types from existing data types with relatively little work. Typically, the new constructor will call the old constructor (using the `call` method in order to be able to give it the new object as its `this` value). Once this constructor has been called, we can assume that all the fields that the old object type is supposed to contain have been added. We arrange for the constructor’s prototype to derive from the old prototype so that instances of this type will also have access to the properties in that prototype. Finally, we can override some of these properties by adding them to our new prototype. [](http://eloquentjavascript.net/06_object.html#p_EcSCrpVw4N)Now, if we slightly adjust the `dataTable` function to use `RTextCell`s for cells whose value is a number, we get the table we were aiming for. ~~~ function dataTable(data) { var keys = Object.keys(data[0]); var headers = keys.map(function(name) { return new UnderlinedCell(new TextCell(name)); }); var body = data.map(function(row) { return keys.map(function(name) { var value = row[name]; // This was changed: if (typeof value == "number") return new RTextCell(String(value)); else return new TextCell(String(value)); }); }); return [headers].concat(body); } console.log(drawTable(dataTable(MOUNTAINS))); // → … beautifully aligned table ~~~ [](http://eloquentjavascript.net/06_object.html#p_XiWor+mk9j)Inheritance is a fundamental part of the object-oriented tradition, alongside encapsulation and polymorphism. But while the latter two are now generally regarded as wonderful ideas, inheritance is somewhat controversial. [](http://eloquentjavascript.net/06_object.html#p_Q82/17hIfZ)The main reason for this is that it is often confused with polymorphism, sold as a more powerful tool than it really is, and subsequently overused in all kinds of ugly ways. Whereas encapsulation and polymorphism can be used to *separate*pieces of code from each other, reducing the tangledness of the overall program, inheritance fundamentally ties types together, creating *more* tangle. [](http://eloquentjavascript.net/06_object.html#p_jis0SWYeNY)You can have polymorphism without inheritance, as we saw. I am not going to tell you to avoid inheritance entirely—I use it regularly in my own programs. But you should see it as a slightly dodgy trick that can help you define new types with little code, not as a grand principle of code organization. A preferable way to extend types is through composition, such as how`UnderlinedCell` builds on another cell object by simply storing it in a property and forwarding method calls to it in its own methods. ## [](http://eloquentjavascript.net/06_object.html#h_Fdk67dJHwg)The instanceof operator [](http://eloquentjavascript.net/06_object.html#p_j4pk5Qzlwd)It is occasionally useful to know whether an object was derived from a specific constructor. For this, JavaScript provides a binary operator called`instanceof`. ~~~ console.log(new RTextCell("A") instanceof RTextCell); // → true console.log(new RTextCell("A") instanceof TextCell); // → true console.log(new TextCell("A") instanceof RTextCell); // → false console.log([1] instanceof Array); // → true ~~~ [](http://eloquentjavascript.net/06_object.html#p_o+ZFEn1oZl)The operator will see through inherited types. An `RTextCell` is an instance of`TextCell` because `RTextCell.prototype` derives from`TextCell.prototype`. The operator can be applied to standard constructors like `Array`. Almost every object is an instance of `Object`. ## [](http://eloquentjavascript.net/06_object.html#h_ErccPg/l98)Summary [](http://eloquentjavascript.net/06_object.html#p_StNbsy1mmY)So objects are more complicated than I initially portrayed them. They have prototypes, which are other objects, and will act as if they have properties they don’t have as long as the prototype has that property. Simple objects have`Object.prototype` as their prototype. [](http://eloquentjavascript.net/06_object.html#p_qMN9iu/ELN)Constructors, which are functions whose names usually start with a capital letter, can be used with the `new` operator to create new objects. The new object’s prototype will be the object found in the `prototype` property of the constructor function. You can make good use of this by putting the properties that all values of a given type share into their prototype. The `instanceof`operator can, given an object and a constructor, tell you whether that object is an instance of that constructor. [](http://eloquentjavascript.net/06_object.html#p_zzr1L1PppY)One useful thing to do with objects is to specify an interface for them and tell everybody that they are supposed to talk to your object only through that interface. The rest of the details that make up your object are now*encapsulated*, hidden behind the interface. [](http://eloquentjavascript.net/06_object.html#p_5POqDFEqYS)Once you are talking in terms of interfaces, who says that only one kind of object may implement this interface? Having different objects expose the same interface and then writing code that works on any object with the interface is called *polymorphism*. It is very useful. [](http://eloquentjavascript.net/06_object.html#p_9dzDvJ4kxT)When implementing multiple types that differ in only some details, it can be helpful to simply make the prototype of your new type derive from the prototype of your old type and have your new constructor call the old one. This gives you an object type similar to the old type but for which you can add and override properties as you see fit. ## [](http://eloquentjavascript.net/06_object.html#h_TcUD2vzyMe)Exercises ### [](http://eloquentjavascript.net/06_object.html#h_zO8FRQBMAy)A vector type [](http://eloquentjavascript.net/06_object.html#p_/6RUkCsOAv)Write a constructor `Vector` that represents a vector in two-dimensional space. It takes `x` and `y` parameters (numbers), which it should save to properties of the same name. [](http://eloquentjavascript.net/06_object.html#p_smri4stKuh)Give the `Vector` prototype two methods, `plus` and `minus`, that take another vector as a parameter and return a new vector that has the sum or difference of the two vectors’ (the one in `this` and the parameter) *x* and *y* values. [](http://eloquentjavascript.net/06_object.html#p_F5nP+jpza3)Add a getter property `length` to the prototype that computes the length of the vector—that is, the distance of the point (*x*, *y*) from the origin (0, 0). ~~~ // Your code here. console.log(new Vector(1, 2).plus(new Vector(2, 3))); // → Vector{x: 3, y: 5} console.log(new Vector(1, 2).minus(new Vector(2, 3))); // → Vector{x: -1, y: -1} console.log(new Vector(3, 4).length); // → 5 ~~~ ### [](http://eloquentjavascript.net/06_object.html#h_nLNNevzcF7)Another cell [](http://eloquentjavascript.net/06_object.html#p_fkvt0g2UPk)Implement a cell type named `StretchCell(inner, width, height)` that conforms to the [table cell interface](http://eloquentjavascript.net/06_object.html#table_interface) described earlier in the chapter. It should wrap another cell (like `UnderlinedCell` does) and ensure that the resulting cell has at least the given `width` and `height`, even if the inner cell would naturally be smaller. ~~~ // Your code here. var sc = new StretchCell(new TextCell("abc"), 1, 2); console.log(sc.minWidth()); // → 3 console.log(sc.minHeight()); // → 2 console.log(sc.draw(3, 2)); // → ["abc", " "] ~~~ ### [](http://eloquentjavascript.net/06_object.html#h_a0w19Kx5iq)Sequence interface [](http://eloquentjavascript.net/06_object.html#p_zIiyP1YArp)Design an *interface* that abstracts iteration over a collection of values. An object that provides this interface represents a sequence, and the interface must somehow make it possible for code that uses such an object to iterate over the sequence, looking at the element values it is made up of and having some way to find out when the end of the sequence is reached. [](http://eloquentjavascript.net/06_object.html#p_n9mbvCSpi+)When you have specified your interface, try to write a function `logFive` that takes a sequence object and calls `console.log` on its first five elements—or fewer, if the sequence has fewer than five elements. [](http://eloquentjavascript.net/06_object.html#p_8fIkJ9NfIu)Then implement an object type `ArraySeq` that wraps an array and allows iteration over the array using the interface you designed. Implement another object type `RangeSeq` that iterates over a range of integers (taking `from` and`to` arguments to its constructor) instead. ~~~ // Your code here. logFive(new ArraySeq([1, 2])); // → 1 // → 2 logFive(new RangeSeq(100, 1000)); // → 100 // → 101 // → 102 // → 103 // → 104 ~~~ One way to solve this is to give the sequence objects state, meaning their properties are changed in the process of using them. You could store a counter that indicates how far the sequence object has advanced. Your interface will need to expose at least a way to get the next element and to find out whether the iteration has reached the end of the sequence yet. It is tempting to roll these into one method, next, which returns null or undefined when the sequence is at its end. But now you have a problem when a sequence actually contains null. So a separate method (or getter property) to find out whether the end has been reached is probably preferable. Another solution is to avoid changing state in the object. You can expose a method for getting the current element (without advancing any counter) and another for getting a new sequence that represents the remaining elements after the current one (or a special value if the end of the sequence is reached). This is quite elegant—a sequence value will “stay itself” even after it is used and can thus be shared with other code without worrying about what might happen to it. It is, unfortunately, also somewhat inefficient in a language like JavaScript because it involves creating a lot of objects during iteration.

## Chapter 5 # Higher-Order Functions > [](http://eloquentjavascript.net/05_higher_order.html#p_18CYJsdOxo)Tzu-li and Tzu-ssu were boasting about the size of their latest programs. ‘Two-hundred thousand lines,’ said Tzu-li, ‘not counting comments!’ Tzu-ssu responded, ‘Pssh, mine is almost a **million** lines already.’ Master Yuan-Ma said, ‘My best program has five hundred lines.’ Hearing this, Tzu-li and Tzu-ssu were enlightened. > > Master Yuan-Ma, The Book of Programming > [](http://eloquentjavascript.net/05_higher_order.html#p_MZD8y19lPz)There are two ways of constructing a software design: One way is to make it so simple that there are obviously no deficiencies, and the other way is to make it so complicated that there are no obvious deficiencies. > > C.A.R. Hoare, 1980 ACM Turing Award Lecture [](http://eloquentjavascript.net/05_higher_order.html#p_1SpKr5NIrc)A large program is a costly program, and not just because of the time it takes to build. Size almost always involves complexity, and complexity confuses programmers. Confused programmers, in turn, tend to introduce mistakes (*bugs*) into programs. A large program also provides a lot of space for these bugs to hide, making them hard to find. [](http://eloquentjavascript.net/05_higher_order.html#p_ethH+CXxsM)Let’s briefly go back to the final two example programs in the introduction. The first is self-contained and six lines long. ~~~ var total = 0, count = 1; while (count <= 10) { total += count; count += 1; } console.log(total); ~~~ [](http://eloquentjavascript.net/05_higher_order.html#p_Sy8N6Qcb09)The second relies on two external functions and is one line long. ~~~ console.log(sum(range(1, 10))); ~~~ [](http://eloquentjavascript.net/05_higher_order.html#p_srOi7846QY)Which one is more likely to contain a bug? [](http://eloquentjavascript.net/05_higher_order.html#p_G+ApGqjOk4)If we count the size of the definitions of `sum` and `range`, the second program is also big—even bigger than the first. But still, I’d argue that it is more likely to be correct. [](http://eloquentjavascript.net/05_higher_order.html#p_pkQRvZqQ0u)It is more likely to be correct because the solution is expressed in a vocabulary that corresponds to the problem being solved. Summing a range of numbers isn’t about loops and counters. It is about ranges and sums. [](http://eloquentjavascript.net/05_higher_order.html#p_VwxmDNivhe)The definitions of this vocabulary (the functions `sum` and `range`) will still involve loops, counters, and other incidental details. But because they are expressing simpler concepts than the program as a whole, they are easier to get right. ## [](http://eloquentjavascript.net/05_higher_order.html#h_j9ps8qrlyo)Abstraction [](http://eloquentjavascript.net/05_higher_order.html#p_ZPK6t/LrMG)In the context of programming, these kinds of vocabularies are usually called*abstractions*. Abstractions hide details and give us the ability to talk about problems at a higher (or more abstract) level. [](http://eloquentjavascript.net/05_higher_order.html#p_XMbJPaIiI4)As an analogy, compare these two recipes for pea soup: > [](http://eloquentjavascript.net/05_higher_order.html#p_F+PunfZCXq)Put 1 cup of dried peas per person into a container. Add water until the peas are well covered. Leave the peas in water for at least 12 hours. Take the peas out of the water and put them in a cooking pan. Add 4 cups of water per person. Cover the pan and keep the peas simmering for two hours. Take half an onion per person. Cut it into pieces with a knife. Add it to the peas. Take a stalk of celery per person. Cut it into pieces with a knife. Add it to the peas. Take a carrot per person. Cut it into pieces. With a knife! Add it to the peas. Cook for 10 more minutes. [](http://eloquentjavascript.net/05_higher_order.html#p_jOZH7oGiYb)And the second recipe: > [](http://eloquentjavascript.net/05_higher_order.html#p_iNBZReprTd)Per person: 1 cup dried split peas, half a chopped onion, a stalk of celery, and a carrot. > > [](http://eloquentjavascript.net/05_higher_order.html#p_k5rI5P5p5u)Soak peas for 12 hours. Simmer for 2 hours in 4 cups of water (per person). Chop and add vegetables. Cook for 10 more minutes. [](http://eloquentjavascript.net/05_higher_order.html#p_cSeY164LbX)The second is shorter and easier to interpret. But you do need to understand a few more cooking-related words—*soak*, *simmer*, *chop*, and, I guess, *vegetable*. [](http://eloquentjavascript.net/05_higher_order.html#p_g/g3l7uyEG)When programming, we can’t rely on all the words we need to be waiting for us in the dictionary. Thus, you might fall into the pattern of the first recipe—work out the precise steps the computer has to perform, one by one, blind to the higher-level concepts that they express. [](http://eloquentjavascript.net/05_higher_order.html#p_SfvOkDsRNZ)It has to become second nature, for a programmer, to notice when a concept is begging to be abstracted into a new word. ## [](http://eloquentjavascript.net/05_higher_order.html#h_inPZJ813r4)Abstracting array traversal [](http://eloquentjavascript.net/05_higher_order.html#p_kYV0l7wLUe)Plain functions, as we’ve seen them so far, are a good way to build abstractions. But sometimes they fall short. [](http://eloquentjavascript.net/05_higher_order.html#p_zubh2XbQ26)In the [previous chapter](http://eloquentjavascript.net/04_data.html#data), this type of `for` loop made several appearances: ~~~ var array = [1, 2, 3]; for (var i = 0; i < array.length; i++) { var current = array[i]; console.log(current); } ~~~ [](http://eloquentjavascript.net/05_higher_order.html#p_NFtmq7O3su)It’s trying to say, “For each element in the array, log it to the console”. But it uses a roundabout way that involves a counter variable `i`, a check against the array’s length, and an extra variable declaration to pick out the current element. Apart from being a bit of an eyesore, this provides a lot of space for potential mistakes. We might accidentally reuse the `i` variable, misspell`length` as `lenght`, confuse the `i` and `current` variables, and so on. [](http://eloquentjavascript.net/05_higher_order.html#p_Ij8hUSNURG)So let’s try to abstract this into a function. Can you think of a way? [](http://eloquentjavascript.net/05_higher_order.html#p_QZzSQpGUXs)Well, it’s easy to write a function that goes over an array and calls`console.log` on every element. ~~~ function logEach(array) { for (var i = 0; i < array.length; i++) console.log(array[i]); } ~~~ [](http://eloquentjavascript.net/05_higher_order.html#p_q8hNTrfIYb)But what if we want to do something other than logging the elements? Since “doing something” can be represented as a function and functions are just values, we can pass our action as a function value. ~~~ function forEach(array, action) { for (var i = 0; i < array.length; i++) action(array[i]); } forEach(["Wampeter", "Foma", "Granfalloon"], console.log); // → Wampeter // → Foma // → Granfalloon ~~~ [](http://eloquentjavascript.net/05_higher_order.html#p_gLkhZ+aBcQ)(In some browsers, calling `console.log` in this way does not work. You can use `alert` instead of `console.log` if this example fails to work.) [](http://eloquentjavascript.net/05_higher_order.html#p_TXDEyQXjy/)Often, you don’t pass a predefined function to `forEach` but create a function value on the spot instead. ~~~ var numbers = [1, 2, 3, 4, 5], sum = 0; forEach(numbers, function(number) { sum += number; }); console.log(sum); // → 15 ~~~ [](http://eloquentjavascript.net/05_higher_order.html#p_v2REJFVWCD)This looks quite a lot like the classical `for` loop, with its body written as a block below it. However, now the body is inside the function value, as well as inside the parentheses of the call to `forEach`. This is why it has to be closed with the closing brace *and* closing parenthesis. [](http://eloquentjavascript.net/05_higher_order.html#p_u1F6T5TuZI)Using this pattern, we can specify a variable name for the current element (`number`), rather than having to pick it out of the array manually. [](http://eloquentjavascript.net/05_higher_order.html#p_w5leJJdYGK)In fact, we don’t need to write `forEach` ourselves. It is available as a standard method on arrays. Since the array is already provided as the thing the method acts on, `forEach` takes only one required argument: the function to be executed for each element. [](http://eloquentjavascript.net/05_higher_order.html#p_BbxEeZ0Pmw)To illustrate how helpful this is, let’s look back at a function from [the previous chapter](http://eloquentjavascript.net/04_data.html#analysis). It contains two array-traversing loops. ~~~ function gatherCorrelations(journal) { var phis = {}; for (var entry = 0; entry < journal.length; entry++) { var events = journal[entry].events; for (var i = 0; i < events.length; i++) { var event = events[i]; if (!(event in phis)) phis[event] = phi(tableFor(event, journal)); } } return phis; } ~~~ [](http://eloquentjavascript.net/05_higher_order.html#p_CGf69rc/Yq)Working with `forEach` makes it slightly shorter and quite a bit cleaner. ~~~ function gatherCorrelations(journal) { var phis = {}; journal.forEach(function(entry) { entry.events.forEach(function(event) { if (!(event in phis)) phis[event] = phi(tableFor(event, journal)); }); }); return phis; } ~~~ ## [](http://eloquentjavascript.net/05_higher_order.html#h_xxCc98lOBK)Higher-order functions [](http://eloquentjavascript.net/05_higher_order.html#p_cao2fH68Tj)Functions that operate on other functions, either by taking them as arguments or by returning them, are called *higher-order functions*. If you have already accepted the fact that functions are regular values, there is nothing particularly remarkable about the fact that such functions exist. The term comes from mathematics, where the distinction between functions and other values is taken more seriously. [](http://eloquentjavascript.net/05_higher_order.html#p_+cgNTV2i2y)Higher-order functions allow us to abstract over *actions*, not just values. They come in several forms. For example, you can have functions that create new functions. ~~~ function greaterThan(n) { return function(m) { return m > n; }; } var greaterThan10 = greaterThan(10); console.log(greaterThan10(11)); // → true ~~~ [](http://eloquentjavascript.net/05_higher_order.html#p_MKDSIQ0X9D)And you can have functions that change other functions. ~~~ function noisy(f) { return function(arg) { console.log("calling with", arg); var val = f(arg); console.log("called with", arg, "- got", val); return val; }; } noisy(Boolean)(0); // → calling with 0 // → called with 0 - got false ~~~ [](http://eloquentjavascript.net/05_higher_order.html#p_2qBy/XBHVa)You can even write functions that provide new types of control flow. ~~~ function unless(test, then) { if (!test) then(); } function repeat(times, body) { for (var i = 0; i < times; i++) body(i); } repeat(3, function(n) { unless(n % 2, function() { console.log(n, "is even"); }); }); // → 0 is even // → 2 is even ~~~ [](http://eloquentjavascript.net/05_higher_order.html#p_i6rGSdWQvz)The lexical scoping rules that we discussed in [Chapter 3](http://eloquentjavascript.net/03_functions.html#scoping) work to our advantage when using functions in this way. In the previous example, the `n` variable is a parameter to the outer function. Because the inner function lives inside the environment of the outer one, it can use `n`. The bodies of such inner functions can access the variables around them. They can play a role similar to the `{}`blocks used in regular loops and conditional statements. An important difference is that variables declared inside inner functions do not end up in the environment of the outer function. And that is usually a good thing. ## [](http://eloquentjavascript.net/05_higher_order.html#h_7/X8BSjdvi)Passing along arguments [](http://eloquentjavascript.net/05_higher_order.html#p_IYAO9/PwTt)The `noisy` function defined earlier, which wraps its argument in another function, has a rather serious deficit. ~~~ function noisy(f) { return function(arg) { console.log("calling with", arg); var val = f(arg); console.log("called with", arg, "- got", val); return val; }; } ~~~ [](http://eloquentjavascript.net/05_higher_order.html#p_xa/8T14jAc)If `f` takes more than one parameter, it gets only the first one. We could add a bunch of arguments to the inner function (`arg1`, `arg2`, and so on) and pass them all to `f`, but it is not clear how many would be enough. This solution would also deprive `f` of the information in `arguments.length`. Since we’d always pass the same number of arguments, it wouldn’t know how many arguments were originally given. [](http://eloquentjavascript.net/05_higher_order.html#p_d8oC4azbX1)For these kinds of situations, JavaScript functions have an `apply` method. You pass it an array (or array-like object) of arguments, and it will call the function with those arguments. ~~~ function transparentWrapping(f) { return function() { return f.apply(null, arguments); }; } ~~~ [](http://eloquentjavascript.net/05_higher_order.html#p_DzgFx1ko9R)That’s a useless function, but it shows the pattern we are interested in—the function it returns passes all of the given arguments, and only those arguments, to `f`. It does this by passing its own `arguments` object to `apply`. The first argument to `apply`, for which we are passing `null` here, can be used to simulate a method call. We will come back to that in the [next chapter](http://eloquentjavascript.net/06_object.html#call_method). ## [](http://eloquentjavascript.net/05_higher_order.html#h_AxpOdvCznQ)JSON [](http://eloquentjavascript.net/05_higher_order.html#p_A2VaNOTR7R)Higher-order functions that somehow apply a function to the elements of an array are widely used in JavaScript. The `forEach` method is the most primitive such function. There are a number of other variants available as methods on arrays. To familiarize ourselves with them, let’s play around with another data set. [](http://eloquentjavascript.net/05_higher_order.html#p_U8cxfiInkI)A few years ago, someone crawled through a lot of archives and put together a book on the history of my family name (Haverbeke—meaning Oatbrook). I opened it hoping to find knights, pirates, and alchemists ... but the book turns out to be mostly full of Flemish farmers. For my amusement, I extracted the information on my direct ancestors and put it into a computer-readable format. [](http://eloquentjavascript.net/05_higher_order.html#p_kyOqr/Ovkw)The file I created looks something like this: ~~~ [ {"name": "Emma de Milliano", "sex": "f", "born": 1876, "died": 1956, "father": "Petrus de Milliano", "mother": "Sophia van Damme"}, {"name": "Carolus Haverbeke", "sex": "m", "born": 1832, "died": 1905, "father": "Carel Haverbeke", "mother": "Maria van Brussel"}, … and so on ] ~~~ [](http://eloquentjavascript.net/05_higher_order.html#p_TtSCtpw/XN)This format is called JSON (pronounced “Jason”), which stands for JavaScript Object Notation. It is widely used as a data storage and communication format on the Web. [](http://eloquentjavascript.net/05_higher_order.html#p_LkHACkXND1)JSON is similar to JavaScript’s way of writing arrays and objects, with a few restrictions. All property names have to be surrounded by double quotes, and only simple data expressions are allowed—no function calls, variables, or anything that involves actual computation. Comments are not allowed in JSON. [](http://eloquentjavascript.net/05_higher_order.html#p_1IU60Zh2Af)JavaScript gives us functions, `JSON.stringify` and `JSON.parse`, that convert data from and to this format. The first takes a JavaScript value and returns a JSON-encoded string. The second takes such a string and converts it to the value it encodes. ~~~ var string = JSON.stringify({name: "X", born: 1980}); console.log(string); // → {"name":"X","born":1980} console.log(JSON.parse(string).born); // → 1980 ~~~ [](http://eloquentjavascript.net/05_higher_order.html#p_NDrk0od6QR)The variable `ANCESTRY_FILE`, available in the sandbox for this chapter and in [a downloadable file](http://eloquentjavascript.net/code/ancestry.js) on the website, contains the content of my JSON file as a string. Let’s decode it and see how many people it contains. ~~~ var ancestry = JSON.parse(ANCESTRY_FILE); console.log(ancestry.length); // → 39 ~~~ ## [](http://eloquentjavascript.net/05_higher_order.html#h_1BJbwiI0gI)Filtering an array [](http://eloquentjavascript.net/05_higher_order.html#p_Vpf83lHLbL)To find the people in the ancestry data set who were young in 1924, the following function might be helpful. It filters out the elements in an array that don’t pass a test. ~~~ function filter(array, test) { var passed = []; for (var i = 0; i < array.length; i++) { if (test(array[i])) passed.push(array[i]); } return passed; } console.log(filter(ancestry, function(person) { return person.born > 1900 && person.born < 1925; })); // → [{name: "Philibert Haverbeke", …}, …] ~~~ [](http://eloquentjavascript.net/05_higher_order.html#p_F1sdIqvMdQ)This uses the argument named `test`, a function value, to fill in a “gap” in the computation. The `test` function is called for each element, and its return value determines whether an element is included in the returned array. [](http://eloquentjavascript.net/05_higher_order.html#p_l78Pj0p/pD)Three people in the file were alive and young in 1924: my grandfather, grandmother, and great-aunt. [](http://eloquentjavascript.net/05_higher_order.html#p_vcnUQxte/7)Note how the `filter` function, rather than deleting elements from the existing array, builds up a new array with only the elements that pass the test. This function is *pure*. It does not modify the array it is given. [](http://eloquentjavascript.net/05_higher_order.html#p_OduOippCCU)Like `forEach`, `filter` is also a standard method on arrays. The example defined the function only in order to show what it does internally. From now on, we’ll use it like this instead: ~~~ console.log(ancestry.filter(function(person) { return person.father == "Carel Haverbeke"; })); // → [{name: "Carolus Haverbeke", …}] ~~~ ## [](http://eloquentjavascript.net/05_higher_order.html#h_lJEtQ+qjXz)Transforming with map [](http://eloquentjavascript.net/05_higher_order.html#p_iZsa8JAje9)Say we have an array of objects representing people, produced by filtering the`ancestry` array somehow. But we want an array of names, which is easier to read. [](http://eloquentjavascript.net/05_higher_order.html#p_/HoP3C1P28)The `map` method transforms an array by applying a function to all of its elements and building a new array from the returned values. The new array will have the same length as the input array, but its content will have been “mapped” to a new form by the function. ~~~ function map(array, transform) { var mapped = []; for (var i = 0; i < array.length; i++) mapped.push(transform(array[i])); return mapped; } var overNinety = ancestry.filter(function(person) { return person.died - person.born > 90; }); console.log(map(overNinety, function(person) { return person.name; })); // → ["Clara Aernoudts", "Emile Haverbeke", // "Maria Haverbeke"] ~~~ [](http://eloquentjavascript.net/05_higher_order.html#p_Jd3axE7OGU)Interestingly, the people who lived to at least 90 years of age are the same three people who we saw before—the people who were young in the 1920s, which happens to be the most recent generation in my data set. I guess medicine has come a long way. [](http://eloquentjavascript.net/05_higher_order.html#p_kz8fxQbtBm)Like `forEach` and `filter`, `map` is also a standard method on arrays. ## [](http://eloquentjavascript.net/05_higher_order.html#h_fx3e34kT/k)Summarizing with reduce [](http://eloquentjavascript.net/05_higher_order.html#p_jc3nW2aN35)Another common pattern of computation on arrays is computing a single value from them. Our recurring example, summing a collection of numbers, is an instance of this. Another example would be finding the person with the earliest year of birth in the data set. [](http://eloquentjavascript.net/05_higher_order.html#p_AR3Kp1oBC4)The higher-order operation that represents this pattern is called *reduce* (or sometimes *fold*). You can think of it as folding up the array, one element at a time. When summing numbers, you’d start with the number zero and, for each element, combine it with the current sum by adding the two. [](http://eloquentjavascript.net/05_higher_order.html#p_2+GsHstzgE)The parameters to the `reduce` function are, apart from the array, a combining function and a start value. This function is a little less straightforward than`filter` and `map`, so pay careful attention. ~~~ function reduce(array, combine, start) { var current = start; for (var i = 0; i < array.length; i++) current = combine(current, array[i]); return current; } console.log(reduce([1, 2, 3, 4], function(a, b) { return a + b; }, 0)); // → 10 ~~~ [](http://eloquentjavascript.net/05_higher_order.html#p_r9cFmJJTar)The standard array method `reduce`, which of course corresponds to this function, has an added convenience. If your array contains at least one element, you are allowed to leave off the `start` argument. The method will take the first element of the array as its start value and start reducing at the second element. [](http://eloquentjavascript.net/05_higher_order.html#p_I1pmLx5OiJ)To use `reduce` to find my most ancient known ancestor, we can write something like this: ~~~ console.log(ancestry.reduce(function(min, cur) { if (cur.born < min.born) return cur; else return min; })); // → {name: "Pauwels van Haverbeke", born: 1535, …} ~~~ ## [](http://eloquentjavascript.net/05_higher_order.html#h_+NeFt8aXxf)Composability [](http://eloquentjavascript.net/05_higher_order.html#p_UHvCjnmGjM)Consider how we would have written the previous example (finding the person with the earliest year of birth) without higher-order functions. The code is not that much worse. ~~~ var min = ancestry[0]; for (var i = 1; i < ancestry.length; i++) { var cur = ancestry[i]; if (cur.born < min.born) min = cur; } console.log(min); // → {name: "Pauwels van Haverbeke", born: 1535, …} ~~~ [](http://eloquentjavascript.net/05_higher_order.html#p_EbeznYJOvD)There are a few more variables, and the program is two lines longer but still quite easy to understand. [](http://eloquentjavascript.net/05_higher_order.html#p_a0FspsuDHF)Higher-order functions start to shine when you need to *compose* functions. As an example, let’s write code that finds the average age for men and for women in the data set. ~~~ function average(array) { function plus(a, b) { return a + b; } return array.reduce(plus) / array.length; } function age(p) { return p.died - p.born; } function male(p) { return p.sex == "m"; } function female(p) { return p.sex == "f"; } console.log(average(ancestry.filter(male).map(age))); // → 61.67 console.log(average(ancestry.filter(female).map(age))); // → 54.56 ~~~ [](http://eloquentjavascript.net/05_higher_order.html#p_Tnj4SDSgBt)(It’s a bit silly that we have to define `plus` as a function, but operators in JavaScript, unlike functions, are not values, so you can’t pass them as arguments.) [](http://eloquentjavascript.net/05_higher_order.html#p_vIaMNOYUL0)Instead of tangling the logic into a big loop, it is neatly composed into the concepts we are interested in—determining sex, computing age, and averaging numbers. We can apply these one by one to get the result we are looking for. [](http://eloquentjavascript.net/05_higher_order.html#p_ZOp8f8zApw)This is *fabulous* for writing clear code. Unfortunately, this clarity comes at a cost. ## [](http://eloquentjavascript.net/05_higher_order.html#h_KTy9HpX3R4)The cost [](http://eloquentjavascript.net/05_higher_order.html#p_ROxex88LkV)In the happy land of elegant code and pretty rainbows, there lives a spoil-sport monster called *inefficiency*. [](http://eloquentjavascript.net/05_higher_order.html#p_KZv3u+lncm)A program that processes an array is most elegantly expressed as a sequence of cleanly separated steps that each do something with the array and produce a new array. But building up all those intermediate arrays is somewhat expensive. [](http://eloquentjavascript.net/05_higher_order.html#p_fwEhp/GEHu)Likewise, passing a function to `forEach` and letting that method handle the array iteration for us is convenient and easy to read. But function calls in JavaScript are costly compared to simple loop bodies. [](http://eloquentjavascript.net/05_higher_order.html#p_QCCz0cq1ti)And so it goes with a lot of techniques that help improve the clarity of a program. Abstractions add layers between the raw things the computer is doing and the concepts we are working with and thus cause the machine to perform more work. This is not an iron law—there are programming languages that have better support for building abstractions without adding inefficiencies, and even in JavaScript, an experienced programmer can find ways to write abstract code that is still fast. But it is a problem that comes up a lot. [](http://eloquentjavascript.net/05_higher_order.html#p_8YtdVmu3Oc)Fortunately, most computers are insanely fast. If you are processing a modest set of data or doing something that has to happen only on a human time scale (say, every time the user clicks a button), then it *does not matter* whether you wrote a pretty solution that takes half a millisecond or a super-optimized solution that takes a tenth of a millisecond. [](http://eloquentjavascript.net/05_higher_order.html#p_s9R2GjExxl)It is helpful to roughly keep track of how often a piece of your program is going to run. If you have a loop inside a loop (either directly or through the outer loop calling a function that ends up performing the inner loop), the code inside the inner loop will end up running *N*×*M* times, where *N* is the number of times the outer loop repeats and *M* is the number of times the inner loop repeats within each iteration of the outer loop. If that inner loop contains another loop that makes *P* rounds, its body will run *M*×*N*×*P* times, and so on. This can add up to large numbers, and when a program is slow, the problem can often be traced to only a small part of the code, which sits inside an inner loop. ## [](http://eloquentjavascript.net/05_higher_order.html#h_XnMZS4zMW6)Great-great-great-great-... [](http://eloquentjavascript.net/05_higher_order.html#p_Typ91CWfYn)My grandfather, Philibert Haverbeke, is included in the data file. By starting with him, I can trace my lineage to find out whether the most ancient person in the data, Pauwels van Haverbeke, is my direct ancestor. And if he is, I would like to know how much DNA I theoretically share with him. [](http://eloquentjavascript.net/05_higher_order.html#p_pwQeT587/V)To be able to go from a parent’s name to the actual object that represents this person, we first build up an object that associates names with people. ~~~ var byName = {}; ancestry.forEach(function(person) { byName[person.name] = person; }); console.log(byName["Philibert Haverbeke"]); // → {name: "Philibert Haverbeke", …} ~~~ [](http://eloquentjavascript.net/05_higher_order.html#p_Yko1+UXeBG)Now, the problem is not entirely as simple as following the `father` properties and counting how many we need to reach Pauwels. There are several cases in the family tree where people married their second cousins (tiny villages and all that). This causes the branches of the family tree to rejoin in a few places, which means I share more than 1/2*G* of my genes with this person, where *G* for the number of generations between Pauwels and me. This formula comes from the idea that each generation splits the gene pool in two. [](http://eloquentjavascript.net/05_higher_order.html#p_qznNjPi9uB)A reasonable way to think about this problem is to look at it as being analogous to `reduce`, which condenses an array to a single value by repeatedly combining values, left to right. In this case, we also want to condense our data structure to a single value but in a way that follows family lines. The *shape* of the data is that of a family tree, rather than a flat list. [](http://eloquentjavascript.net/05_higher_order.html#p_1De08sAcOD)The way we want to reduce this shape is by computing a value for a given person by combining values from their ancestors. This can be done recursively: if we are interested in person *A*, we have to compute the values for *A*’s parents, which in turn requires us to compute the value for *A*’s grandparents, and so on. In principle, that’d require us to look at an infinite number of people, but since our data set is finite, we have to stop somewhere. We’ll allow a default value to be given to our reduction function, which will be used for people who are not in the data. In our case, that value is simply zero, on the assumption that people not in the list don’t share DNA with the ancestor we are looking at. [](http://eloquentjavascript.net/05_higher_order.html#p_B0sEhVOeSP)Given a person, a function to combine values from the two parents of a given person, and a default value, `reduceAncestors` condenses a value from a family tree. ~~~ function reduceAncestors(person, f, defaultValue) { function valueFor(person) { if (person == null) return defaultValue; else return f(person, valueFor(byName[person.mother]), valueFor(byName[person.father])); } return valueFor(person); } ~~~ [](http://eloquentjavascript.net/05_higher_order.html#p_HK0Nr6MiCM)The inner function (`valueFor`) handles a single person. Through the magic of recursion, it can simply call itself to handle the father and the mother of this person. The results, along with the person object itself, are passed to `f`, which returns the actual value for this person. [](http://eloquentjavascript.net/05_higher_order.html#p_2Q4JEt3Fiv)We can then use this to compute the amount of DNA my grandfather shared with Pauwels van Haverbeke and divide that by four. ~~~ function sharedDNA(person, fromMother, fromFather) { if (person.name == "Pauwels van Haverbeke") return 1; else return (fromMother + fromFather) / 2; } var ph = byName["Philibert Haverbeke"]; console.log(reduceAncestors(ph, sharedDNA, 0) / 4); // → 0.00049 ~~~ [](http://eloquentjavascript.net/05_higher_order.html#p_+kx1i84Abe)The person with the name Pauwels van Haverbeke obviously shared 100 percent of his DNA with Pauwels van Haverbeke (there are no people who share names in the data set), so the function returns 1 for him. All other people share the average of the amounts that their parents share. [](http://eloquentjavascript.net/05_higher_order.html#p_sxFDHiagLr)So, statistically speaking, I share about 0.05 percent of my DNA with this 16th-century person. It should be noted that this is only a statistical approximation, not an exact amount. It is a rather small number, but given how much genetic material we carry (about 3 billion base pairs), there’s still probably some aspect in the biological machine that is me that originates with Pauwels. [](http://eloquentjavascript.net/05_higher_order.html#p_9W8GK5HRM6)We could also have computed this number without relying on`reduceAncestors`. But separating the general approach (condensing a family tree) from the specific case (computing shared DNA) can improve the clarity of the code and allows us to reuse the abstract part of the program for other cases. For example, the following code finds the percentage of a person’s known ancestors who lived past 70 (by lineage, so people may be counted multiple times): ~~~ function countAncestors(person, test) { function combine(current, fromMother, fromFather) { var thisOneCounts = current != person && test(current); return fromMother + fromFather + (thisOneCounts ? 1 : 0); } return reduceAncestors(person, combine, 0); } function longLivingPercentage(person) { var all = countAncestors(person, function(person) { return true; }); var longLiving = countAncestors(person, function(person) { return (person.died - person.born) >= 70; }); return longLiving / all; } console.log(longLivingPercentage(byName["Emile Haverbeke"])); // → 0.129 ~~~ [](http://eloquentjavascript.net/05_higher_order.html#p_yX3KPrQkep)Such numbers are not to be taken too seriously, given that our data set contains a rather arbitrary collection of people. But the code illustrates the fact that`reduceAncestors` gives us a useful piece of vocabulary for working with the family tree data structure. ## [](http://eloquentjavascript.net/05_higher_order.html#h_fwBD5oTMLl)Binding [](http://eloquentjavascript.net/05_higher_order.html#p_JR92h9ZvcR)The `bind` method, which all functions have, creates a new function that will call the original function but with some of the arguments already fixed. [](http://eloquentjavascript.net/05_higher_order.html#p_RpFvD2j6a4)The following code shows an example of `bind` in use. It defines a function`isInSet` that tells us whether a person is in a given set of strings. To call`filter` in order to collect those person objects whose names are in a specific set, we can either write a function expression that makes a call to `isInSet` with our set as its first argument or *partially apply* the `isInSet` function. ~~~ var theSet = ["Carel Haverbeke", "Maria van Brussel", "Donald Duck"]; function isInSet(set, person) { return set.indexOf(person.name) > -1; } console.log(ancestry.filter(function(person) { return isInSet(theSet, person); })); // → [{name: "Maria van Brussel", …}, // {name: "Carel Haverbeke", …}] console.log(ancestry.filter(isInSet.bind(null, theSet))); // → … same result ~~~ [](http://eloquentjavascript.net/05_higher_order.html#p_P5qnPOIMMg)The call to `bind` returns a function that will call `isInSet` with `theSet` as first argument, followed by any remaining arguments given to the bound function. [](http://eloquentjavascript.net/05_higher_order.html#p_j9JPuWJlk7)The first argument, where the example passes `null`, is used for method calls, similar to the first argument to `apply`. I’ll describe this in more detail in the[next chapter](http://eloquentjavascript.net/06_object.html#call_method). ## [](http://eloquentjavascript.net/05_higher_order.html#h_ErccPg/l98)Summary [](http://eloquentjavascript.net/05_higher_order.html#p_EmYdYeFPjo)Being able to pass function values to other functions is not just a gimmick but a deeply useful aspect of JavaScript. It allows us to write computations with “gaps” in them as functions and have the code that calls these functions fill in those gaps by providing function values that describe the missing computations. [](http://eloquentjavascript.net/05_higher_order.html#p_YfXho6cgu+)Arrays provide a number of useful higher-order methods—`forEach` to do something with each element in an array, `filter` to build a new array with some elements filtered out, `map` to build a new array where each element has been put through a function, and `reduce` to combine all an array’s elements into a single value. [](http://eloquentjavascript.net/05_higher_order.html#p_3UBuCIfCEM)Functions have an `apply` method that can be used to call them with an array specifying their arguments. They also have a `bind` method, which is used to create a partially applied version of the function. ## [](http://eloquentjavascript.net/05_higher_order.html#h_TcUD2vzyMe)Exercises ### [](http://eloquentjavascript.net/05_higher_order.html#h_aIOczlLyX1)Flattening [](http://eloquentjavascript.net/05_higher_order.html#p_RqAkArolEa)Use the `reduce` method in combination with the `concat` method to “flatten” an array of arrays into a single array that has all the elements of the input arrays. ~~~ var arrays = [[1, 2, 3], [4, 5], [6]]; // Your code here. // → [1, 2, 3, 4, 5, 6] ~~~ ### [](http://eloquentjavascript.net/05_higher_order.html#h_I9XoVSLsTV)Mother-child age difference [](http://eloquentjavascript.net/05_higher_order.html#p_a44KIXR4aT)Using the example data set from this chapter, compute the average age difference between mothers and children (the age of the mother when the child is born). You can use the `average` function defined [earlier](http://eloquentjavascript.net/05_higher_order.html#average_function) in this chapter. [](http://eloquentjavascript.net/05_higher_order.html#p_/E+zqsb0UA)Note that not all the mothers mentioned in the data are themselves present in the array. The `byName` object, which makes it easy to find a person’s object from their name, might be useful here. ~~~ function average(array) { function plus(a, b) { return a + b; } return array.reduce(plus) / array.length; } var byName = {}; ancestry.forEach(function(person) { byName[person.name] = person; }); // Your code here. // → 31.2 ~~~ ### [](http://eloquentjavascript.net/05_higher_order.html#h_FkNn96IrQe)Historical life expectancy [](http://eloquentjavascript.net/05_higher_order.html#p_ENbLeUYppS)When we looked up all the people in our data set that lived more than 90 years, only the latest generation in the data came out. Let’s take a closer look at that phenomenon. [](http://eloquentjavascript.net/05_higher_order.html#p_0JOS/ablSJ)Compute and output the average age of the people in the ancestry data set per century. A person is assigned to a century by taking their year of death, dividing it by 100, and rounding it up, as in `Math.ceil(person.died / 100)`. ~~~ function average(array) { function plus(a, b) { return a + b; } return array.reduce(plus) / array.length; } // Your code here. // → 16: 43.5 // 17: 51.2 // 18: 52.8 // 19: 54.8 // 20: 84.7 // 21: 94 ~~~ [](http://eloquentjavascript.net/05_higher_order.html#p_yzI++go/yy)For bonus points, write a function `groupBy` that abstracts the grouping operation. It should accept as arguments an array and a function that computes the group for an element in the array and returns an object that maps group names to arrays of group members. ### [](http://eloquentjavascript.net/05_higher_order.html#h_jr7hZiuR7+)Every and then some [](http://eloquentjavascript.net/05_higher_order.html#p_BZKBxgsPML)Arrays also come with the standard methods `every` and `some`. Both take a predicate function that, when called with an array element as argument, returns true or false. Just like `&&` returns a true value only when the expressions on both sides are true, `every` returns true only when the predicate returns true for *all* elements of the array. Similarly, `some` returns true as soon as the predicate returns true for *any* of the elements. They do not process more elements than necessary—for example, if `some` finds that the predicate holds for the first element of the array, it will not look at the values after that. [](http://eloquentjavascript.net/05_higher_order.html#p_aMPYznovUm)Write two functions, `every` and `some`, that behave like these methods, except that they take the array as their first argument rather than being a method. ~~~ // Your code here. console.log(every([NaN, NaN, NaN], isNaN)); // → true console.log(every([NaN, NaN, 4], isNaN)); // → false console.log(some([NaN, 3, 4], isNaN)); // → true console.log(some([2, 3, 4], isNaN)); // → false ~~~ The functions can follow a similar pattern to the definition of forEach at the start of the chapter, except that they must return immediately (with the right value) when the predicate function returns false—or true. Don’t forget to put another return statement after the loop so that the function also returns the correct value when it reaches the end of the array.