a tornado of razorblades

REST appealed to me right from the get-go. But it’s taken me a surprisingly long time to wrap my head around all of its implications. Some of my recent projects - building the fully RESTful Heroku API, working with Mike Clark on the Nested Resources recipe for his upcoming book, and writing rest-client - have allowed me to finally get a handle on how all the pricinples of REST fit together into a unified whole.

If you’re a longtime member of the Church of REST, what I’m posting here may seem obvious. But if you’re like I was six months ago - that is, a little confused by how you could make a useful application with only four verbs - then read on.

Raw HTTP

REST is hard to get precisely because it is so simple. It just seems like there isn’t much to it. But it turns out that there are some subtle implications hidden within its seemingly straightforward principles.

First, let’s get the easy one out of the way. REST is about URLs and HTTP. This part makes sense to most people right away. Hitting a URL is something that can easily be done from almost any language or programming environment; it’s extremely transparent (and thereby discoverable and easy to debug) due to the simple nature of URLs and the fact that you can use a web browser to do simple testing; and using HTTP as the transport gives benefits like virtual hosts, easy proxying, and doesn’t require running a separate daemon besides your webserver. In short, REST is the web, and this makes it vastly easier to use than the various RPC protocols whose corpses now litter the highway of software history.

But don’t get tripped up on this - there’s much more to REST than just HTTP. This part of it doesn’t really have a name, but it should. “Raw HTTP,” maybe? Raw HTTP does beat the heck out of SOAP, so it’s a step in the right direction. But true REST devotees make fun of APIs which use only one HTTP verb (GET) and have their own custom verb embedded in the URL. Amazon is (in)famous for this, yielding SimpleDB calls that look like this:

GET https://sdb.amazonaws.com/?Action=PutAttributes&DomainName=MyDomain&ItemName=Item123&Attribute.1.Name=Color&Attribute.1.Value=Blue

A true REST API would look more like this:

PUT https://sdb.amazonaws.com/MyDomain/Item123

…with an input body containing an XML (or even CGI) representation of the data being updated. (Subbu Allamaraju explores what's wrong with the SimpleDB API in detail.)

Headers, Verb, Resource, Payload

So true REST calls divide up the HTTP protocol space into four sections: the headers, the verb, the resource, and the body. (I like to call the input body the “payload,” which makes its purpose much clearer to me.) When you’re designing a REST API, you have to make sure that each piece of data goes into the correct section, or else things turn into a mess. Let’s look closer at each.

The headers are meta information about the request. Most are used by the webserver and don’t concern the web app or its API, but two (Content-type and Accept) are important, since they describe the nature of the input body and output body.

The verb is the HTTP method: GET, POST, PUT, or DELETE. These map exactly to the standard CRUD operations: GET = read, POST = create, PUT = update, DELETE = delete. These are the only verbs you can use, and that’s all there is to it. This rigid uniformity is a plus, but it takes a while to learn to adapt to it. More on that later.

The resource is what is described by the URL. It’s a noun, and by itself does not describe any action. You can do things to a resource, but a resource itself doesn’t do anything. (Hence the clever term “RESTful resource” - the resource is at rest, not moving.) Amazon and others who write raw HTTP APIs get flack for calling their APIs REST when in fact everything goes into the URL, ignoring the other three sections.

The payload is the input body of a POST or a PUT. (The other two verbs don’t have payloads.) This point turned out to be key for my understanding of how to use REST in the real world. The payload is data you are sending to the server, for use in creating or updating a resource. The simplest example is a form full of data, but its uses are actually much more diverse and flexible than just forms.

The flexibility of payloads comes from the Content-Type header. With a standard web form, your content type will usually be application/x-www-urlencoded. This basically makes the payload of the POST be a bunch of CGI parameters. For years I thought of POST and its ability to stuff the parameters into the payload as nothing more than a way to avoid cluttering the URL. But once you start using different content types with your payloads for POST (and PUT), a whole new world of possibilities opens up.

For example, to upload a photo into a photo collection, you might do:

POST http://myphotos.example.com/users/adam/photos

…with the content-type set to image/jpg and the payload containing the binary data of the image.

How about importing an archive of images? Use the exact same resource! Just set the content-type to application/x-gtar and pass a payload containing the binary data of a tarball.

Updating an existing image could be:

PUT http://myphotos.example.com/users/adam/photos/12345

…again with a content-type of image/jpg and the binary data of the image. But what about updating the metadata on the image, like a caption? Same URL, but switch the content-type, perhaps to application/xml (or perhaps trusty old application/x-www-urlencoded).

How about fetching different types of data for a resource? Just mirror the process: it’s now a GET, and we’ll use Accept to ask for a different type of content in the returned body. For example, you could download the image by setting the Accept header to image/jpg; or download a collection of images by setting Accept to application/x-gtar.

This distinction between the verb, the resource, and the input and output bodies (each with a content type) is the core of what makes REST both useful and usable.

Status Codes

There’s one other piece of the HTTP puzzle here, which is the status code returned by a request. In practice I’ve found there are three codes you really care about in designing or accessing an API: 200, 401, and 422.

200 means success, everything went fine.

401 is unauthorized, meaning that you failed to supply the proper credentials. Typically this means you need to check your http basic auth username and password.

422 is unprocessable entity. In a Rails app, this usually means that the attempt to create or save the record failed, and the returned body contains XML with the error messages.

Status codes can be treated something like the return value on a shell command; if it’s bad, you should treat the result differently than if it was normal. Check out the process_result method in rest_client.rb for an example.

Putting It All Together

Now, the big question - how do you make do with only four verbs? The answer is: more resources. In particular, nested resources, but not necessarily full model resources. Rather, think about dividing the domain space of your individual objects into subresources - whether or not this is represented by a separate record in your database.

I’ll give you a particularly gnarly example straight from my own real-world experience: Heroku apps. We have a model, App, which maps to the apps table in the main database. This is our central model: everything revolves around the app. As a result, our first pass at the internal API on our backend ended up with an AppsController that had 20+ actions.

These actions make perfect sense: they’re the verbs that we need for the App object. Some examples: rename, import, export, purge_database, restart_server. How in the world can we represent these with just CRUD operations?

The answer lies in a proliferation of subresources. To quote Mike Clark, the question that REST is always asking is: What’s the resource? And if you let yourself break from a one-to-one mapping between resources and models, things get a lot more flexible. So let’s map those actions I mentioned to REST, keeping in mind that we also have payloads with different content types available.

Check out the subresources on the bottom three. These aren’t necessarily ActiveRecord models, though they could be. But conceptually, they represent a particular area of action. There may or may not be a Database object belonging to the App, but it makes sense to subdivide the problem space by implying a has-a relationship between app and database.

What’s more, you may find that a designing a clean URL scheme gives a great deal of insight into the design for your internal architecture. If you create a REST subresource that isn’t a full object, perhaps you should stop and think about making it one.