An example might help...

I got a lot of great comments on last nights post, including a couple about REST being no different from xml-based RPC. I used to think so too, which is why my recent epiphany was so eye opening.

Consider a protocol for finding and reserving a flight between two cities. The client is in one of these states:

<ready>
- searched
- retrieved details
- reserved

These states map to URIs:

<none>
- http://quuxTravel.com/searched
- ??? depends on previous state
- ??? depends on previous state

A client begins by navigating to the searched state by GETting http://quuxTravel.com/searched?src=London&dest=NYC. The client gets back some XML like this:

<itineraries>
  <itinerary src=“London“ dest=“NYC“ price=“400.03“>
    <getDetails uri=“http://quuxTravel.com/details?itinerary=402“ />
    <reserve uri=“http://reservations.bookingsunlimited.com/quuxTravel?itinerary=402“ />
  </itinerary>
  <itinerary src=“London“ dest=“NYC“ price=“109.88“>
    <getDetails uri=“http://quuxTravel.com/details?itinerary=219“ />
    <reserve uri=“http://reservations.bookingsunlimited.com/quuxTravel?itinerary=219“ />
  </itinerary>
</itineraries>

The client is now in the searched state. It scans the list of itineraries to find the one with the lowest price. If the client wanted some other criteria that isn't surfaced in this state, e.g., total flight time, it could transition to the retrieved details state by GETting the URI stored in the itinerary's getDetails/@uri attribute. It would then return to the searched state (either by an explicit back-link or a history a la' the browser). The system would return an XML representation of that state that contained flight info.

When the client has chosen a flight, it transitions to the reserved state by POSTing to the URI stored in the itinerary's reserve/@uri attribute. It gets back an XML document confirming the reservation. At this point the protocol is complete. The client can begin again if desired, or go do something else.

Now, why is this different from RPC? Imagine the following interface for implementing this same protocol:

interface IFlightSystem
{
    Itineraries Search(string src, string dest);
    Details GetDetails(int itineraryId);
    Confirmation Reserve(itineraryId);
}

This interface exposes the same protocol, with more or less the same requirements on the client to know what the data being sent and received means. The difference is that in this case, the client is talking to one endpoint and mapping request/response payloads to call stacks. In the previous case neither of those things were true.

The REST model opens the door for the protocol to be implemented across different endpoints. This is useful for scalability, partitioning and data-directed routing, integration with external systems (note that the transition to the reserved state uses a URI at a partner company). In other word, it's actually a web of endpoints. Further, those URIs are dynamically constructed, so you can change them based on user, time of day, the data they're interested in, locale they're from, state of your data center, or whatever. That is hugely powerful. Because the documents being sent around are not mapped to call stacks, and may not even be mapped to objects, it's easier to stream data, add extra stuff over time, etc.

In one of his comments, Ittay asked what the REST model for the a method “string Foo(string, int, bool)” would be. In response, I described a simple protocol with one state, “FooInvoked”. To get to that state, you'd access a URI for /foo, passing a string, int and bool as query string parameters or in the request body. You'd get back a result state that contained a string. He countered that that felt just like a function call, which isn't surprising because that's where we started. The key, for me, is to look at problems not from the perspective of methods, as Ittay did, or entities, as Joe describes, but as states and the transitions between them. Then it really starts to make sense. And the power of it is real.


Posted Apr 27 2007, 02:56 PM by tim-ewald

Comments

Scott Seely wrote re: An example might help...
on 04-27-2007 1:18 PM
OK, now let's push on this a little further. Imagine that you have a combination of some sort of endpoint publishing magic (WCF) and some workflow that knows about state transition (WF). What could you do if you pulled these two together with your set of ideas? I think you start getting really close to a practical implementation of a stateful service that understands the right set of transitions.
Mike Marshall wrote re: An example might help...
on 04-27-2007 2:18 PM
Truly thought provoking posts. I know there are some people who want to continue to hope that web services would be just a simple. But Tim adds a dynamic that can really put a damper on things - routing via URI to a third party. In his example, all the third-party has to do is consume a query string and go.

Now look at it from the web services side. The third-party has to conform to an agreed upon RPC-ish XML interface (e.g. WSDL), and has to build a back end to support it (e.g. WSDL import of some kind). No to mention some of the idiosyncrasies betweeb web services plaforms.

In the simplest case with URIs, all I have to do is write an ASPX file with some embedded C# (no codebehinde required) and respond to the GET or POST.

I suppose the hope is that WSDL will someday be as comfortable to devs as URIs, but I wouldn't hold your breath waiting for that day.
Erik Johnson wrote re: An example might help...
on 04-27-2007 3:51 PM
Designing a resource-oriented approach for an ERP system has been quite fun and more liberating than I expected. We replaced the classic API design discipline with a URI composition strategy. It’s a much more leveraged effort, especially if you can derive the URI space from metadata. There have been some other “ah ha” moments we hit along the way. One was that our data model has lots of relationships defined and that our URI strategy should let people exploit them. This has led to URIs that read a lot like the names of “Friends” episodes (“the resource about the product code from the part on the order line”).
Another realization was that a URI can bridge information spaces. For example, message payloads often have formats that are some derivation (or subset) of the system’s information scheme. So, a URI can point to information on the system rooted by information in the message payload. Think of a rules processor that disallows new orders for customers on credit hold. A new order in a message has a customer ID but (likely) not the credit status. A URI that effectively says “the credit status of that customer on the order in this message” is pretty handy thing.
One reason I haven’t done much in the way of REST blogging is that I don’t know if we’re doing it right. I mean, I we eliminated SOAP, use URIs and payload formats to express the intent, and use HTTP (despite WCF). What we’ve done works great, but I don’t want to call it “REST” if it confuses the intentions of Roy Fielding’s work or associates us with another wave of WS/SOA/Services on the Web/whatever hype. In any case, the liberating part of our transition (pun intended) to this thinking is that it seems like there are much fewer wrong ways to expose a system. I also get to retire my sordid collection of proxy generators.
Christopher Steen wrote Link Listing - April 27, 2007
on 04-27-2007 6:51 PM
When you think things are not possible: WCF duplex callbacks through NATs and firewalls - safe and secure...
Craig wrote re: An example might help...
on 04-28-2007 3:30 AM
Erik: What's funny is that you're almost certainly more "right" than Fielding. Because you're building systems and finding that they work for you in many good ways. People need to stop worrying about the definition of "REST" - it's a total distraction. Rather, they need to concern themselves with what tradeoffs technology decisions entail.

I was annoyed reading the comments on the original posts by readers who essentially said, "No, no, that's not what REST is." Who gives a rat's ass what REST *is*? What does your approach *do*, and what does it make it *hard to do*? Because no matter how many epiphanies even smart guys like Tim have, at the end of the day it's still "right" to use the other choice some of the time (as Tim calls out in his original post).
Tim wrote re: An example might help...
on 04-28-2007 12:07 PM
Scott,

IIS and ASP.NET already give me plenty of endpoint magic. The Web itself is a state machine for workflows. WCF and WF could be put together in a form that does the same thing, but why wouldn't I just use the Web.
Tim wrote re: An example might help...
on 04-28-2007 12:10 PM
Chris,

Yep, you're right, there are things that HTTP-based REST solutions can't do. To be fair, I think if I wanted services to callback into a client behind NAT, where the client connects and the service initiaties messaging, I would look at IM-style solutions, e.g., XMPP. Yes, I could use WCF duplex channels, but I feel the weight of RPC upon me. Alternatively, I could just avoid the problem by polling (works for RSS readers!) or async messageing (SMTP?). There are lots of Webby ways to solve that problem, otherwise we wouldn't be happy living behind those little boxes.

Tim-
Tim wrote re: An example might help...
on 04-28-2007 4:25 PM
Erik,

To me, the RESTy-ness comes from (a) thinking about a state machine and (b) the use of links inside payloads. As soon as you have to know through some out of band means that you take this id from this payload and use it to cobble together a URI that follows a particular format, it becomes less useful. That said, there are loads of situations where you'll be mixing protocols together and may need to do that. But getting the transition URIs from the payload of the current state is key.

Tim-

ps- And yes, there is a please lack of extra tooling required. :-)
Erik Johnson wrote re: An example might help...
on 04-28-2007 11:26 PM
Craig,

Thanks for the encouragement. It's crazy the number of things I've worried about. Would the Captains of the Internet smite me for using parentheses in a URI? It's not easy coming out of the REST closet.

-Erik
Erik Johnson wrote re: An example might help...
on 04-28-2007 11:26 PM
Tim,

Yep, I've thought about that and haven't completely made up my mind. Your example with flight itineraries is well conceived, but what if the list of available URIs in the “searched” state numbered in the hundreds? In our work, the URI “grammar space” (sorry for the jargon) is a projection of an abstract data model. So, the list of URIs embedded in resource instance would be a double-pushout (-like) transform of all the links between each record, all other related views, and the successive links beyond those views. So, if a complete collection of links in a resource is unreasonable to achieve, is it worth putting any in at all?

More to your point, having an explicit URI list within a message didn’t really eliminate the need for “out of band” knowledge. The act of resolving the links within the message depends on out-of-band knowledge, does it not? How would the client know to GET the “getDetails” URI to find flight time data? Maybe you could have a URI describing the “getDetails” resource. But the knowledge of that description URI is also out-of-band.

My thinking was to (a) create URIs for the data model and (b) make all of the application URIs aspect-driven from that data model. Out-of-band knowledge is required to determine the application URIs – but (for what it’s worth) I tried to keep it simple, complete, and deterministic.
Bill de hOra wrote re: An example might help...
on 04-29-2007 5:18 AM
A state machine for reliable messaging:

http://www.dehora.net/doc/httplr/draft-httplr-01.html

rm being one of the things WS-* types seem to care so much about.
Tim wrote re: An example might help...
on 04-29-2007 10:46 AM
Erik,

One way you can solve the "hundreds of links" problem with chunked-based retrieval, where each chunk is a new state. That's how how paginated results work in browsers. It's also a typical flow-control solution for retrieving lots of stuff in RPC systems. Another option would be stream-based processing, which isn't so common in RPC-based systems.

Yes, you are right that you need OOB info to know where links appear in a payload and what each link means. For instance you have to know that the getDetails element's uri attribute value is the URI to get to that state and that you invoke it by GET. But you don't have to know the structure of the link itself, and that's what gives you the freedom to rework your server-side infrastructure to change where state transitions are routed without breaking the client.

The alternative approach of documenting an URL structure and constructing them from values you cull from a state don't leave you that flexibility.

Tim-
Tim wrote re: An example might help...
on 04-29-2007 10:52 AM
Bill,

What's so weird about the fascination with WS-RM is that the spec only defines TCP-level semantics. The reliability doesn't cover your processing, unless you use queues and then do queue/database transactional stuff. Without that, if your process tanks right after receiving a reliably delivered message, your hosed. Some of the WS stacks providing reliability via queuing, but all in proprietary ways.

One of the nice things about sticking with HTTP is that you don't need to reinvent TCP-level semantics as you do with a model that assumes you want to create meta-connections across multiple nodes using a range of protocols. One side effect of that, btw, is that WS-RM is hard to implement without pinning to a server that can maintain protocol state for the duration of the meta-connection.

My solution is typically to design my application protocol carefully so that everything is idempotent and that recovery when something goes wrong is as inexpensive as possible. I don't think that will always work, but it has so far. :-)

Tim-

Add a Comment

(required)  
(optional)
(required)  
Remember Me?