Happy New Year to all my fellow readers! I hope you had a good party!

In this post I wanna clarify why representers in Roar are much more versatile than one might think at first sight.

Model != Resource

I know, Rails teaches us that every resource is a model with exactly one representation. This is wrong. Not only can a resource be just anything, ranging from simple 1-to-1 mappings of database rows to virtual models like a workflow but also can your business models be represented in completely different contexts.

Let’s get back to our fruity example to learn more about this.

orange = Fruit.find_by_title("Orange")
orange.extend(FruitRepresenter).to_json
#=> {"title":"Orange","colors":["orange"]}

In the last discussion we had a Fruit and a Bowl model which were represented in their respective resources. While the fruit just contains some properties the bowl was a composition thereof. This is pretty “rails-conform” (I use this word with a negative undertone) and can simply be abstracted to hundreds of projects where people expose users, blog posts or events in their resources. This is what we call a 1-to-1 mapping of model to resource.

Let’s have some Booze!

We wrote a simple FruitRepresenter for mapping apples, lemons or oranges to documents, and we had the BowlRepresenter to provide documents for fruit collections. That was nice. But you can do more with fruits. Why not open a distillery and make some good booze from pears or grapes?

What that means for our REST API is that we have to represent fruits as ingredients. We also want to document what fruits are contained in a bottle of schnaps.

module IngredientRepresenter
  include Roar::Representer::JSON
 
  property :title
  collection :tastes_like
end

The new IngredientRepresenter looks a bit similar to our FruitRepresenter, however, it represents a fruit in another context.

pear = Fruit.find_by_title("Pear")
pear.extend(IngredientRepresenter).to_json
#=> undefined method `tastes_like'

An exception. While the pear exposes a #title method it does not have a #tastes_like method. Why did I do that? I wanted to emphasize that the representer doesn’t guess anything from the represented object (the pear). It is up to the fruit to provide decent accessors like the #tastes_like reader. No magic or implicit semantics in Roar. And this is a good thing.

Another Context, Another Module.

We could easily push the missing methods into the IngredientRepresenter itself, but we will learn in an upcoming post that it is better to provide additional accessors in a separate module.

module Ingredient
  def tastes_like
    TastesLike.query(title) # returns array.
  end
 
  def tastes_like=(*)
  end
end

In the reader, we query the world-famous TastesLike™ web service. Since we don’t need a writer, the second method ain’t doing nothing.

pear.extend(Ingredient, IngredientRepresenter).to_json
#=> {"title":"Pear","tastes_like":["pear","mango"]}

Cool, this is a fruit represented in a completely different context. What about the parsing, does that work, too?

apple = Fruit.new
apple.extend(Ingredient, IngredientRepresenter)
apple.from_json("{\"title\":\"Apple\",
  \"tastes_like\":[\"Pear\",\"Apple\",\"Mango\"]}")
#=> #<Fruit title="Apple", colors=[]>

Yepp, parsing an ingredient and mapping it to a fruit also works fine.

Fruits in a Bottle.

Let’s see how collections can be represented in different contexts.

module BottleRepresenter
  include Roar::Representer::JSON
 
  collection :fruits, :class => Fruit, 
    :extend => [Ingredient, IngredientRepresenter]
end

The bottle representer knows that it’s composed of Fruit instances. It does use the IngredientRepresenter for parsing and rendering, though. BTW, if you don’t like the long :extend assignment you are free to “hard-include” the modules directly in your models.

Summary

So what we’re doing here is having one model used in two completely different contexts: the conventional “fruit” context and the “ingredient” context. Imagine you’d be doing this with #as_json and different hash-parsing algorithms in your controllers – good luck.

We talked about Representers earlier last week. Today, I wanna describe DCI and how it fits perfectly together with representers in Roar.

What is DCI?

For years we’ve been taught that “Fat Models, Skinny Controllers” is the way to organize code. Push all your domain code into your ActiveRecord classes. Many people, including me, agree that this is bullshit. This leads to bloated classes, usually combined with one or two object instances taking care of an entire workflow (aka request).

DCI is an architectural pattern that makes the model “fat-on-demand”: Data objects are enhanced with roles at runtime, but only when needed.

The DCI pattern consists of three concepts.

1. Data objects are supposed to contain application data. Nothing else. This would be your ActiveRecord row instance, but without any predefined behaviour. Just data.
2. Context objects define what is done in the current workflow. In Rails, a controller running an action could be a context object.
3. Roles are behaviour for data objects in a particular context. In Ruby, a Rails controller could mixin a module (a “role”) into an empty model row and thus let it process something.

Hooray for DCI – But Why?

Now the thing is, I absolutely hate to hype something (except stuff I did, of course). However, DCI makes code better maintainable, easier to follow and way easier to test. Period.

BTW, did you already check out my new Roar gem? It’s hip!

Think of a classical model found in many Rails applications with maybe one hundred methods. Or fifty. Fifty methods to interfere with each other. In a DCI setup, an object has a limited behaviour scope. The limited knowledge makes it easier to debug and predictable.

One Model, Multiple Faces

Since representers are modules they fit perfectly into the DCI approach, making them roles. Let’s consider the fruit representer from last week, again.

require 'roar/representer/json'
 
module FruitRepresenter
  include Roar::Representer::JSON
 
  property :title
  collection :colors
end

You still can include this representer into its “host” class directly.

class Fruit
  include Roar::Representer
  include FruitRepresenter
end

The problem now is that the Fruit class is limited to one representation, only. Forever and a day you may only render one particular representation. The key about representations, however, is that one model may be represented by multiple, yeah, representations.

Now, back to the start, let’s assume the Fruit class is some empty ActiveRecord class without any prior knowledge of representations at all.

class Fruit < ActiveRecord::Base
end
 
lemon = Fruit.from_json("{\"title\":\"Lemon\"}")
#=> undefined method `from_json' for Fruit:Class

Yeah, there is no #from_json class method.

Representers On DCI

In order to use the DCI approach, we first need an instance. Here’s how the lemon instance, and only this instance, can be extended with the fruit representer.

lemon = Fruit.new.extend(FruitRepresenter)
lemon.from_json("{\"title\":\"Lemon\"}")
lemon.title #=> "Lemon"

Awesome! The built-in Ruby #extend method makes the lemon instance aware of its representation and mixes in #from_json and friends.

Naturally, this works in both ways.

orange = Fruit.find_by_title("Orange")
orange.extend(FruitRepresenter).to_json
#=> {"title":"Apple","colors":["green"]}

The mixed-in representer also lets us render the JSON document.

Gimme Complexity!

On a flat level with single objects, this works out-of-the-box with recent Roar/representable versions. Problems arise when we have nested setups.

module BowlRepresenter
  include Roar::Representer::JSON
  include Roar::Representer::Feature::Hypermedia
 
  property :location
  collection :items, :as => Fruit,
    :extend => FruitRepresenter
 
  link :self do
    "http://bowl/#{location}"
  end
end

Using the :extend option we can give representable a hint about the representer to use when rendering/parsing contained objects.

And, hey, this works.

empty_bowl = Bowl.new
empty_bowl.extend(BowlRepresenter)
emtpy_bowl.from_json("{\"location\":\"kitchen\", \
  \"items\":[{\"title\":\"Lemon\",\"colors\":[]}]}")
 
puts empty_bowl.items.first.title #=> "Lemon"

Discussion Needed!

Having representers on an object level makes the whole thing really clean and versatile. You can test document rendering and parsing without polluting classes. And you can have a FruitRepresenter, a RottenVegetableRepresenter and what else you like for your orange instance. BTW, oranges ain’t no vegetables.

We still discuss about the API and need your opinion. Right now, we got this

collection :items, :as => Fruit,
    :extend => FruitRepresenter

Here, :as describes the class that encapsulates the contained data. The :extend option helps representable to find the correct module for the nested item. Now, what about this?

collection :items, :class => Fruit,
    :module => FruitRepresenter

Does that make sense? These options really just describe what’s on the right side of the hash rocket. Thanks to Scott for discussion here. Now, tell us your story!

This post introduces the new Roar gem, a framework-agnostic REST framework for Ruby that puts focus on object-oriented REST documents.

More object-orientation?

The calls for more object-orientation in Ruby frameworks are growing ever louder. Most web frameworks, like Rails, come with a nice database abstraction, a Rack-like HTTP endpoint, routing, and a template engine. It’s the lack of additional abstraction layers that make many programmers ask for “more OOP”.

Already yawning? Yeah, I wrote about that earlier this year. Let’s talk about working solutions now. Let’s talk about REST.

REST is about Documents!

REST is not only about pretty URLs. The problem ain’t URLs at all. Most frameworks have a decent routing layer to connect resource logic (aka controllers) to URLs. However, the key of REST is pushing documents between clients and servers, so why don’t we finally focus on documents?

It’s the document parsing, processing and rendering that makes our heads spinning.

Note: I here use the terms representation and document interchangable just because I can!

Parsing and Processing

Both clients and services have to parse incoming documents. Consider this JSON document.

{"title":  "Lemon",
 "colors": ["green"]}

This could be sent to a resource to create a new fruit, or be retrieved by some client which issued a GET request.

In Rails, and many HTTP gems, this is automatically parsed using a JSON gem.

params = JSON['{"title":  "Lemon",
                "colors": ["green"]}']
 
params["title"] #=> "Lemon"
params["colors"][0] #=> "green"

We’re using a hash access to read properties from that document. This is handy! Now, assume the sender forgot to include the colors.

params = JSON['{"title":  "Lemon"}']
 
params["colors"][0]
#=>NoMethodError: undefined method `[]' for nil:NilClass

An exception, shit! While processing the parsed document (aka hash), we have to take care of missing keys, wrong types etc. We as the consumer have to know about all the pitfalls with this particular document.

Rendering

Now that our parsing and processing code is finished, how do we render outgoing representations? We could use #to_json.

@lemon.to_json(:only => [:title, :colors])

Being a quick solution for rendering small models this approach hits the wall when we have nested models but still want a plain JSON representation, or when we want to embed hypermedia. Also, we put “REST” logic into our database layer. It’s up to you to judge.

What about template languages? Let’s try jbuilder for JSON rendering.

Jbuilder.encode do |json|
  json.title  @lemon.title
  json.colors @lemon.colors
end

Nice! But, wait. How do I test this? Can I test this separately? What if I want different representations for the same model? Do I need different templates? And… is it cool to have format knowledge both in our parsing layer and in our rendering layer? I mean, both need to know about the title property and the colors array.

Representers: OOP to our Documents!

I won’t further discuss the shown concepts here. Let me just criticize that they are not object-oriented and encourage the distribution of document internals about the entire framework.

Here’s how representers can handle all that and make you a happy man at the same time!.

require 'roar/representer/json'
 
module FruitRepresenter
  include Roar::Representer::Base
 
  property :title
  collection :colors
end

This is all we need to declare the basic syntax of the fruit document. Note that it resides in a module which can be used anywhere in your app.

Rendering a Lemon

Now let’s say our database layer provides us a class Fruit with ActiveRecord-like behaviour.

lemon = Fruit.new(:title => "Lemon")
puts lemon.title #=> "Lemon"

Since the representer knows everything about the document format it can be used for rendering JSON. We just have to include the FruitRepresenter module into the class and new methods will be available.

Fruit.class_eval do # DB layer.
  include Roar::Representer::JSON
  include FruitRepresenter
end
 
lemon.to_json
#=> "{\"title\":\"Lemon\",\"colors\":[]}"

Awesome, the FruitRepresenter module gives us a #to_json method. Internally, this method compiles the JSON document by asking the represented object (aka lemon) for its properties and values. Remember, the representer knows about those properties, since we declared them. It does not know anything about ActiveRecord and the like.

Now, what about parsing?

Parsing a Lemon

The great thing about representers is: they work bidirectional. That alone is one of the main advantages to many other gems out there. Bidirectional? Let’s just see what that is.

orange = Fruit.from_json("{\"title\":\"Orange\"}")
puts orange.inspect
#=> #<Fruit:0x9df73a8 @title="Orange", @colors=[]>

We can parse representations and create objects from it! That’s bidirectional: rendering and parsing is combined in one representer.

Again, the representer does not know anything about the underlying database – it just creates an object it was mixed into, reads from the document and assigns properties using polite setters. Nothing more, bro.

Don’t like the class method? If you already have an instance, you can use #from_json on the instance.

orange.from_json("{\"title\":\"Yummy Orange\"}")
orange.title #=> "Yummy Orange"

Nesting Representers

Now that we have fruits, lets put ‘em in a bowl. A fruit bowl contains a list of fruits.

bowl = Bowl.new
bowl.items << lemon
bowl.items << orange

In order to bind the fruit bowl to a REST resource we write another representer module.

module BowlRepresenter
  include Roar::Representer::Base
 
  property :location
  collection :items, :as => Fruit
end
 
Bowl.class_eval do # DB layer.
  include Roar::Representer::JSON
  include BowlRepresenter
end

Properties can be typed using the :as option. The target class must be a representer, too. That’s the only requirement.

bowl.to_json
#=> {"items":[
  {"title":"Lemon","colors":[]},
  {"title":"Orange","colors":[]}
]}

Typed properties allow us to build more complex representations without breaking with OOP. The collection assignment makes building lists as easy as possible. And if you still need to tweak things, there are some helpful configuration options around – more on that in one of the next posts.

Representers love HATEOAS

HATEOAS? Check out this talk I gave if you want to learn about Hypermedia and HATEOAS.

Hypermedia helps clients to operate the API without having to compute URLs. We want hypermedia in our documents.

module BowlRepresenter
  include Roar::Representer::Feature::Hypermedia
 
  link :self do
    "http://bowls/#{location}"
  end
end

Including the Hypermedia feature dynamically adds a new method #link to our representer module. What does it do?

bowl.location = :desk
bowl.to_json
#=> {
"items":[
  {"title":"Lemon","colors":[]},
  {"title":"Orange","colors":[]}],
"links":[{"rel":"self","href":"http://bowls/desk"}]
}

Seems like an easy way to embed links in our documents. In roar-rails the Rails URL helpers work like a charm within the link blocks – making it super simple to provide hypermedia to consumers and still being railsy.

Representers on the Client Side

Ok, representers basically are plain modules. Why not use ‘em on the client side as well? The client doesn’t have access to the model layer – if it does, something’s wrong! However, it’s absolutely ok to share the representer modules.

class Bowl
  include Roar::Representer::JSON
  include Roar::Representer::Feature::HttpVerbs
  include BowlRepresenter # from a gem.
end

See how we simply use an empty class with the mixed-in representer on the client side? Remember, we don’t want to have access to the database here.

The next step is creating a new fruit bowl.

bowl = Bowl.new
bowl.location = "kitchen"
bowl.post!("http://bowls")

This code is enough to send a POST request to the specified URL. The bowl instance will serialize a JSON document, send it to the URL using the HttpVerbs feature, receive the new bowl, deserialize it and update its properties accordingly. You may know this behaviour from REST client gems like ActiveResource – here, with less code and manageable complexity.

Consume Hypermedia, it’s gooood.

The representer also makes it easy to extract hypermedia links from documents.

bowl.links[:self]
#=> "http://bowls/kitchen"

No parsing needed, the representer knows where to find your hypermedia. How cool is that?

XML is not a Crime!

Roar comes with both JSON and XML representers.

module BowlRepresenter
  include Roar::Representer::XML
end
 
bowl.to_xml
#=> '<bowl>
#  <link rel="self" href="http://bowl/kitchen"/>
#</bowl>'

Now choose if you want JSON or XML – it’s up to you! I still don’t understand the XML hating – did I miss something or is it just “too java”?

Roar – and more?

The Roar project is still evolving. However, the current API is almost stable. New features, new concepts, testing helpers, etc will be added over time just like in every framework. Roar is framework-agnostic and can be used anywhere, as long as it’s Ruby.

Its modulare feature architecture makes it easy to extend your documents. Currently, we have some exciting on-going brainstorming and hacking on features like client-side caching in the representer instance, generic error handling, validations, DSLs, … We need you to try it out, complain, propose, help, and spread the word. Thank you for reading until here, you rock!

We were discussing the design of search resources in a REST architecture after the REST talk at the EuRuKo. Usually, search resources process GET requests, compute a subset of searched items and return a representation of the respective items.

The following GET request illustrates this.

http://articles/search?type=shirt&color=red&size=s

It’s absolutely common to allow more parameters.

http://articles/search?type=shirt&color=red&size=s& \
  gender=female&limit=10&amount=20&

Where’s the search resource?

Now, let me state two facts about these URLs.

• From a REST perspective, those two search queries refer to two different resources. Remember: Resources are addressed with unique URLs, and when URLs differ you’re addressing differing resources.

• Within Rails, this doesn’t matter that much since both URLs are processed in the same controller. This is cool. A controller per definition is meant to expose a set of resources.

Long URLs and REST

After having discussed the meaning of resources the question that came up was

“Isn’t such a long URL with lots of query parameters comparable to clumsy XML-RPC arguments?”

Well, I’d say: No way.

• Dynamic resources have to receive input in a certain manner – this makes them dynamic. Query parameters were designed to let users specify parameters. There’s nothing wrong with allowing input.

• However, if you decide your URLs are getting too long you have to redesign your search resource architecture!

Exposing named queries

The REST cookbook – a fantastic write-up which I do strongly recommend – suggests to identify commonly used search criterias and expose those queries as separate resources.

Say we’re annoyed with the long “search red female shirts” URLs. Why not donate a dedicated resource?

http://articles/shirts/female?color=red&size=s

While this is just an example, it demonstrates how commonly used parameters were consolidated in a new URL base. Again, from the REST point-of-view, this points to a completely separate resource.

However, in Rails, this can simply be mapped to the search controller while providing the remaining parameters in the route definition.

match 'articles/shirts/female' => 'articles#search', 
  :defaults => { :gender => 'female',
                   :type => 'shirt' }

Controllers and Resources

Don’t give me a hard time about the routing – what I wanted to discuss is the way of thinking about resources: You’re free to map different URLs to one controller. One of main architectural concepts in REST is that it’s up to you how you design your resources and how to expose them.

Query parameters don’t kill your mother and choosing named URLs is free – it’s absolutey “RESTful”.

The representable gem helps to keep parsing and rendering representations in one place. Once defined, it gives your model #to_xml, #from_xml and more. The initial version was released today, let’s see how it helps cleaning up your representation code.

Let’s write a Blog app!

Since your imaginery blog app has a REST API you want to provide a post resource serving and consuming article representations. Here’s a sample XML representation of an arbitrary blog post.

<post>
  <id>1</id>
  <posted_at>2011-05-09</posted_at>
 
  <comment id="1">
    <text>This is stupid!</text>
  </comment>
  <comment id="3">
    <text>No, awesome post!</text>
  </comment>
</post>

Basically, a post representation contains comments in addition to some meta data.

The Problem: Two places need to know about your Representation.

In, say, a Rails app, you’d have to tweak the models’ #to_xml method in order to render an XML representation. Alternatively, you’d use a Builder template.

If, and only if, you stick to the Rails conventions, #from_xml will work out-of-the-box. Otherwise, you have to write a manual parser as discussed earlier. Note that the <comment> tag contains an attribute not compliant with Rails.

You usually end up with some model code, maybe even a template and a hand-made parser. This distributes representation knowledge over the entire MVC framework. While this won’t kill your mother it still makes things more work to manage.

Defining a Representation

The representable gem is designed to prevent you from doing that. You simply define the representation in one place and it does the REST (I still love the word-play).

1
2
3
4
5
6
7
8

require 'representable/xml'
 
class Comment # < ...
  include Representable::XML
 
  representable_property :text
  representable_property :id, :from => "@id"
end

The Representable::XML module allows to define plain properties (line 6) and attributes (line 7).

Nesting Representations

The post representation uses the compositing feature in representable in order to embed comments.

1
2
3
4
5
6
7
8

class Post # < ...
  include Representable::XML
 
  representable_property :id
  representable_property :posted_at
  representable_collection :comments, 
    :tag => :comment, :as => Comment
end

Wow, you can define nested compositions with representable (line 6). This is easy, so how do we work with it?

Rendering Documents

You want to render a document representing your post now.

Post.find(1).to_xml 
  # => "<post><id>1</id>comment id=\"1\"..."

Representable takes care of rendering. No magic included.

Parsing Representations

What if you allow people to POST an XML document to the blog post resource?

@post = Post.from_xml(request.body)
@post.comments
# => "[#<Comment:0x8545a08 @text=\"Your blog sucks.\">]"

Again, representable deserializes the incoming representation and creates the respective objects for you. Representable doesn’t know anything about databases or your underlying data layout, though. It just creates fresh instances for compositions making it your job to process it.

Other media types?

Representable comes with XML and JSON support, you just have to mix in the respective module.

@comment.to_json
  # => "\"{text => ..."

How does it help?

Representable is a completely abstract module for rendering and parsing representations and mapping document fragments to object attributes. Nothing more.

It makes working with representations object-oriented where you had view templates and parsers before. While representable makes that workflow as simple as possible it also allows distributing your representation code, since it can be pushed into Ruby modules. Maybe it makes sense to use it on both client and server, in Rails and Sinatra?

Representable is also used in Roar to make life easier when working with REST representations and embedding or consuming hypermedia. We will discuss how representations and HATEOAS work in the next post. Cheers.

The REST architectural style is still intriguing me. After having discussed how PUT and POST do differ let’s move forward and learn a bit about representations, one of the key principles in REST.

What’s a Representation?

It is inevitable to understand that when accessing a REST service, say http://users.example.com/apotonick, you get a representation of the resource in its current state, and not the resource itself.

By specifying the Content-type header we can even instruct the representation type. While a JPG picture is a decent representation we usually want a XML document keeping all the important attributes of the resource (a representation can be just anything, but I’d like to focus on XML for this post).

To illustrate the requirements for proper representation handling I will use an imaginary order resource for processing orders (orders contain items).

Let the following snippet be subject to further discussion.

<order>
  <id>1</id>  
  <item>
    <link rel="article" href="http://articles/1" />
    <amount>1</amount>
  </item>
  <item>
    <link rel="article" href="http://articles/2" />
    <amount>3</amount>
  </item>
 
  <link rel="self"     href="http://orders/1" />
  <link rel="checkout" href="http://orders/checkout/1"/>
</order>

Notice the contained <item> blocks and the embedded hyperlinks in terms of good HATEOAS.

Representations can be pain.

Now what’s the problem with representations? Why not start with the server, the REST service. It has two duties to handle.

• Serializing the order with its items is one task. This might lead to an XML document that is sent back to the client in a GET request.

• Deserializing a representation of an order is the other tedious task. When a client POSTs/PUTs we usually need to process the sent representation and map it to our data models in order to save the changed world in the database.

Both tasks can cause headache. Rails has a couple of built-in – but limited – mechanics to help you with this work.

Generating XML representations in Rails

ActiveRecord makes serializing a model to XML very easy (at first sight) – you can simply call #to_xml.

order = Order.find(1)
puts order.to_xml

This returns an opinionated XML document.

<?xml version="1.0" encoding="UTF-8"?>
<order>
  <client-id type="integer">1</client-id>
  <created-at type="datetime">2011-04-03</created-at>
  <id type="integer">1</id>
  <updated-at type="datetime">2011-04-03</updated-at>
  ...
</order>

Additionally, #to_xml allows some tweaking by passing in option parameters. However, this approach is pretty limited. For instance, it is quite difficult to render <link rel="..."> tags with attributes.

UPDATE: Also, it is impossible to render the <item> list without a wrapping <items> tag.

Using builder for rendering

Another possibility is using an XML template with builder, in our case this is in app/views/orders/show.xml.

xml.instruct!
xml.order do
  xml.id @order.id
  xml.item do
    xml.link :rel => :article, :href => item_url(item)
#...

This provides a decent way for rendering complex XML documents that represent our resources.

Using Restfulie’s tokamak

The Restfulie gem ships with its own template language for rendering representations. This is called tokamak and works similar to a builder view.

member(@order) do
  id @order.id
  collection(@order.items) do |m, item|
    link :rel => :article, :href => item_url(item)
    amount item.amount
  end
 
  link :rel => self, :href => order_url(@order)
end

The cool thing here is that tokamak allows rendering this abstract representation to multiple targets, like XML or JSON.

Working with representations

Now that we covered the outbound part of rendering representations let’s see how we can work with incoming representations. This usually happens in POST and PUT processing actions in your controller.

Working with a representation means parsing – or deserializing – it and then accessing its attributes to map those to your data models.

How does a real POST look like?

In Rails you typically use HTML forms, POST them to a controller action, and then process the incoming data.

For a long time Rails made me think that POSTs are all about “POST parameters” – incoming data is made available in a nested params[..] data structure.

However, this is something I call a Rails misapprehension. A POST might also have a request body containing the resource representation you want to create.

A REST client written with Restfulie as HTTP helper might do the following to create a new order.

res = Restfulie.at("http://localhost:3000/photos/party")
 
res.as("application/jpg").post(%{
<order>
  <item>
    <link rel="article" href="http://articles/3" />
    <amount>1</amount>
  </item>
</order>
})

Note that we’re sending a document directly, without any form usage.

Assuming our resource controller does understand that we will create a new order. Wow.

What’s wrong with form POSTs?

Nothing! Using the #form_for helper and Rails params is absolutely ok. However, I want to point out that POSTs are not limited to HTML forms – they can also send other representations.

So, what Rails does behind the scenes is

• parsing the incoming application/x-www-form-urlencoded representation of your resource

• and make the deserialized representation available in #params.

This automatical behaviour is all possible if you stick to Rails’ conventions, meaning very limited XML formats and thus representations.

Working with parsed representations

In the simplest case it looks as if the Rails approach would strike.

class OrdersController < ApplicationController
  def create
    @order = Order.create(params[:order])
    # ...
  end

However, in our case, this won’t work as we have “non-standard” <link> tags in our incoming representation. The internal parser in Rails doesn’t recognize that and things will explode.

The internal parser?

Yeah, Rails has a hard-wired internal parser to deserialize a couple of mime- types (“POST parameters”, XML and JSON). This code is found in ParamsParser here.

I’m sorry to say that but that middleware is a debatable concept in Rails. While it blindly parses incoming request data it also urges you to use Rails’ limited representation convention.

If you don’t go the Rails way and use very simple XML representations, it will simply not work.

Parsing D.I.Y.

Now that Rails doesn’t give us what we need we have to do it ourself.

class OrdersController < ApplicationController
  def create
    item_urls = Nokogiri::XML(request.raw_post).
      xpath("//item/link").
      collect { |i| i[:href] }
 
    @order = Order.new
    # ...
  end

Looks horrible.

The dilemma – where is my representation?

A severe problem emerges – where is my representation? Or, in other words: where are transformation rules stored?

• Rendering the representation involves either an XML template or an adjusted #to_xml which knows a lot about the structure and semantics of our representation.

• Parsing the representation implies a hand-made XML parser that itself knows about the very structure and semantics of the representation, again.

The dilemma: Knowledge about the representation is cluttered over the entire MVC-framework.

A new abstraction layer: Representers

What might sound over-engineered to many developers used to the Rails convention is my new approach which abstracts “working with representations” to a new layer.

A so called Representer handles both the in- and outs of representations, not sharing the internal syntax and semantics with the outer world (the best we can).

The Item representer looks simple.

class ItemXmlRepresenter < Roar::Representer::Xml
  name :item
  link
 
  property :amount
end

The Order representer already contains a composition – the ordered items.

class OrderXmlRepresenter < Roar::Representer::Xml
  name :order
 
  property :id
  property :items, :tag => :item, 
    :as => [ItemXmlRepresenter]
 
  link do |controller|
    :rel => :self, :href => controller.order_url(self)
  end
 
  link do |controller|
    :rel => :checkout, 
    :href => controller.order_checkout_url(self)
  end
end

This is a part of my REST framework roar which tries to help with writing real RESTful apps in Rails, Sinatra and Padrino.

How does roar help me?

Now that we defined the representers we can use them in controllers.

class OrdersController < ApplicationController
  def show
    @order = Order.find(params[:id])
 
    xml = OrderXmlRepresenter.serialize_model(@order)
    # ...

The call to #serialize_model will render the XML representation for the @order object using the proper representers, including real HATEOAS hyperlinks, nested items, and so on.

What comes out is the XML document I introduced in the beginning paragraph.

Now, how do we process input?

class OrdersController < ApplicationController
  def create
    rep = OrderXmlRepresenter.
      deserialize(request.raw_post)
 
    @order = Order.create(rep.to_nested_attributes)
 
    # ...

When handling incoming data the #to_nested_attributes method from the representer helps transforming the data into a hash complying to nested-attributes requirements used with #update_attributes.

Benefits?

Representers try to improve the architecture. They…

• embody the transformation of a representation in one class while allowing serialization and deserialization.
• allow compositions of representations. Order has_many items.
• make hypermedia support easy (we will learn more about that in the next post)
• are inheritable and extendable and bring back OOP. For instance, you might write convenience accessors into your representer class.
• are usable in REST clients as well, which allows decoupled, solid systems (see next post).

• make testing easier.

Hey, this is a proposal!

The cool thing here is: that stuff is already working. It’s still under heavy development, but it works. I’m maintaining a sample application at github which I use in my upcoming thesis about Ruby, REST architectures and web GUIs.

My goal with roar is to make it easier and cleaner to develop RESTful, decoupled systems in Ruby while using all the great existing gems like Restfulie.

Looking forward to your feedback, now.

In the next post I will write about how Representers help keeping your REST clients clean and dumb.

In the last post I talked about the rest in REST, beyond the four major verbs. Before getting completely wasted at the tonight’s new year party I’d like to quickly illustrate my understanding of two HTTP verbs and how to think about them in a RESTful mind.

While GET and DELETE are pretty obvious, a few points really bothered me with PUT and POST.

• What exactly is the difference between POST and PUT? In CRUD contexts, the first usually seems to be mapped to create, whereas the latter seems to be the update pendant. But that is not the end of the story!
• How do I implement these verbs in my Rails controllers? Is there some standard, best practices, or something?

• They say PUT is idempotent – that’s awesome, however, what the heck does that mean?

Help! I’m idempotent!

Let’s explore POST first, which is not idempotent. I wrote a resource to categorize photos I have on my hard-drive, where I simply can add photos to categories.

Here’s how I get the textual representation of the party category (with Restfulie, the one and only REST engine for Ruby).

res = Restfulie.at("http://localhost:3000/photos/party")
puts res.accepts("text/csv").get!

The category doesn’t contain too much pics, as I’m not partying enough:

IMG_0056.JPG, jpg
Michal_drinking.png, png

To add another picture to the party resource, sorry, party category, I could use POST.

res.as("application/jpg").post!(binary_jpg)

Don’t care about how we POST and which format – just notice that we are POSTting to the party resource. This results in another picture categorized in “party” (a polite controller would redirect us with a 201 to the new photo resource).

Multiple POSTs, duplicate content!

Now if I do that 3 times in a row, what happens?

3.times { res.post!(binary_jpg) }

We can see the non-idempotent effect of POST right away.

IMG_0056.JPG, jpg
Michal_drinking.png, png
Nick-on-fire.png, png
Nick-on-fire_2.png, png
Nick-on-fire_3.png, png

The last embarrassing picture was added three times. Since we repeatedly POSTed, we have to take into account that things might be duplicated.

If POST duplicates, can I simply use PUT?

Now, a common deception is allowing users to PUT single images to the /photos/party resource. In the best case you’d simply obfuscate the RESTfulness of your API. In the worst case you’d destroy your party picture collection (depending how you wrote your serving controller)!

I’d like to show how a correct PUT could look like. After that, we discuss why.

PUTting – done right!

Using the idempotent nature of PUT I should be able to insert the new photo only once, even if I PUT repetitively. Here’s the client code.

res= Restfulie.at("http://localhost:3000/photos/party").
  accepts("application/zip")
 
pics = res.get!
pics.unzip!
pics << binary_jpg # add the new photo
 
# and PUT it back:
res.put!(pics.zip!)

Does that look confusing? I guess it does. What am I doing here?

1. I retrieve all photos categorized in “party” as a ZIP file
2. then I unzip and add the new photo (for simplicity I simply assume there are methods like #zip! and #unzip! around)

1. last step is PUTting back the grown ZIP archive

That’s how PUT is intended to work. Let’s discuss.

It’s all about entities…

I really had a hard time understanding where these semantics origin from until I read this fantastic summary (any cite following from this page).

In RESTful architectures we have to distinguish between entities and container resources.

A typical entity is the photo resource.

• POSTing to /photos/IMG_0056.JPG doesn’t make sense here – should we append another picture to the existing? No, non-sense.

• However, it is absolutely conclusive that a PUT would overwrite the original picture.

…and containers.

Our “party pictures” resource is a stereotypical container resource.

• as “POST’s job is simply to [...] add something to a container” it is obvious that POSTing to /photos/party expects a new image binary (or whatever) which is added to the party category.

• “If you PUT to a container, you are attempting to overwrite the state that was constructed from all the previous POSTs to it” – I couldn’t find better words to describe that. A PUT to /photos/party replaces the existing collection with the one you send.

With that in mind, it should be pain if you allow people to PUT single images to /photos/party – since your controller (should!) expects a complete image collection!

Conclusion

Think of POST as in adding a new entity (often to a container). When you hear PUT, keep in mind that PUT is more like replacing the addressed resource entirely with the new data you send.

How to do that in Rails is worth another post – in 2011. Cheers and Happy New Year!

CRUD is not REST is what I was trying to clarify in my last restful post. There surely is a terrible confusion in the community about that.

• CRUD is implementation where you simply could pull in some module into your controller and get a CRUD interface for free.

• REST is an implementational approach – there simply is no library that after inclusion makes your code “restful”. There are great gems to give you CRUD code and make it available as a REST interface. However, the real work like deciding which and how resources are exposed, is up to you, the software architect.

REST is more than HTTP restrictions. That’s what I found after studying all the great comments in the last post. Thanks for the helpful links and explanations, guys!

This really is just the tip of the iceberg, as Juan stated correctly.

HTTP restrictions?

With CRUD in mind, REST initially seemed to be nothing more than a constraint saying: Put your verbs from the URL to the HTTP header.

Instead of having a URL like /users/new I simply send a POST to the /users/ resource. Being “RESTful”, a POST is defined to create a new resource.

So, great, we got GET, POST, PUT and DELETE that are – more or less – semantically mapped to CRUD operations.

RESTful urls

Browsing some Rails discussion about REST you can often find the term “RESTful url”, which usually refers to something like

/users/4711/comments/9

Following Adam’s explanation this has nothing to do with REST – it’s a friendly URL, nothing more!

So the real point about URLs in REST is: They are network addresses pointing to a unique resource. If it is pretty, so much the better! Thanks to Rails we got friendly URLs, but keep in mind that this alone doesn’t make ‘em RESTful.

Let’s look at the term resource to find out how real REST urls look like.

What are resources?

In CRUD context I said that resources are usually mapped to business objects, like users, comments or any other model with a database table.

Usually here means: Resources could (and should!) be just anything. They don’t need to map to distinct tables. Transient resources – like a blog feed aggregating posts – are the crucial point about REST. That’s where we have to act as architects deciding what to expose.

Taking the feed as example a real resource identification would be

http://blog.example.org/feeds/4711

A REST id is always a complete URI with host, path, etc.

So on the one hand we have primary keys identifying model instances, and on the other we have REST ids. It’s easy to confuse that!

In a RESTful environment you precisely shouldn’t talk about “user 4711” but about the ressource http://example.com/users/4711. REST is about hiding implementation details, why, this being the case, are we talking about primary keys?

Representations

REST usually operates via HTTP, which doesn’t mean that resource controllers need to render HTML only.

Rails 3 already does a pretty good job here. Controllers and models have built-in renderers for different representations of resources.

class Feeds < ApplicationController
  respond_to :html, :xml, :csv
 
  def index
    @feeds = Feed.all
    respond_with(@feeds)
  end

When requesting an XML list of available feeds the controller will detect that, delegate to the model and returns an XML document without having written any user code at all.

Anyway, what’s still missing here is content negotiation.

Content negotiation

Let’s take the ruby way to illustrate that problem. I use restfulie to write a quick REST client.

Restfulie.at("http://localhost:3000/feeds/").
accepts("application/xml").
get!

Restfulie is a wonderful gem. This code will retrieve my feeds in XML. Here’s my server output.

Started GET "/feeds/" for 127.0.0.1 at 2010-12-...
  Processing by FeedsController#index as XML

Obviously, this worked.

Now, content negotiation implies that we can request a list of acceptable representations.

What if I say “I want your feeds in ATOM first. If you don’t speak ATOM, gimme XML!”.

Restfulie.at("http://localhost:3000/feeds/").
accepts("application/atom").
accepts("application/xml").
get!

Restfulie takes care of the content negotiation, which is pretty awesome. Information about which formats to send is handled in the Accept: header flag.

Started GET "/feeds/" for 127.0.0.1 at 2010-12-...
 Processing by FeedsController#index as APPLICATION/ATOM
Completed 406 Not Acceptable

Well, I’d say the content negotiation failed, although we do provide XML!

What happened? Rails didn’t look at the second accepted format. This usually is not a problem, as most people don’t use content negotiation. Restfulie provides a convenient module to enable your controller of real content negotiation. I’ll post about that separately.

The reason for my lack of understanding here is that Rails simply doesn’t support it, yet. This isn’t a severe bug, I just want to point out what else is in REST.

Hypermedia

A couple of comments remarked another principle in REST that is often swept under the table: Hypermedia support.

When I hear hypermedia I think of JPEG images and links. Not sure why.

It first seemed strange to embedd workflow-related links into my resource representation. This basically means the resource contains links to other “application parts“ where browsing to would make sense in the current application state.

HATEOAS!

An example.

Consider we were looking at a XML representation of a feed – while it contains a lot of post snippets, it could also contain a link to actually subscribe.

<feed>
  <posts>...</posts>
 
  <link href="http://blog.de/feeds/4711/subscription" \
    rel="subscription">
</feed>

The feed list proposes what to do from here. Isn’t that great?

While it is the client’s job to extract that links and decide which action to display, the client as-it is not responsible for generating these links. The resource provides us links to change the application state – this is called HATEOAS.

While I am still exploring this field of REST I just started realizing how HATEOAS can help building forward-compatible, modular applications.

Summary

Did I really manage to work out that REST is more than GET, POST, PUT and DELETE?

So, REST has five basic concepts, that are

• Unique resource identification which is sometimes mistaken as RESTful url
• Standard interface for resources like GET, POST and friends. Each request type has its own semantic and may or may not alter the resource.
• Representations and content negotiation to display resources in different apparels.
• Hypermedia to embedd further actions into the resource.

• and stateless communication – why not write about that separately?

While doing some research on Rails and REST, I took stock on two things.

• It is relatively hard to find a clean, basic and printed explanation of how REST works with Rails. I found thousands of screencasts, and this single great guide.
• People do confuse CRUD with REST.

Especially the latter kept annoying me. While coaching a team I found out that clearly separating CRUD and REST helps people understanding what RESTful really means: Nothing more than HTTP restrictions.

I felt like sharing this insights with you.

Tell me, what is CRUD?

Every application has exactly one purpose: Maintaining business objects. This basically means: You got models that need to be created, displayed, updated and deleted using forms, lists and other fancy web UI elements.

If your Rails app would be managing beers, you’d write a BeersController with actions like show, update, create and so on. The actions would mostly display forms and save attributes to models.

In order to access these forms in the browser you’d create routes, like

• /beers/show/:id
• /beers/new
• /beers/delete/:id

and map them to the respective controller actions I already mentioned.

This is what we call a CRUD interface: An application with descriptive routes, forms and usually pretty boring code – having one destiny: To Create, to Retrieve, to Update and to Delete.

Hey, this is CRUD!

Now, what’s REST?

Couple of years ago a new standard emerged, it is called REST and has the following main principles:

• The central concept of REST is the resource, which is a datastructure representing one or more business objects.
• REST also encourages clean, simple, yeah, “RESTful” URLs – like /beers/1
• They use HTTP verbs (GET/PUT/POST/DELETE) to encode what to do in the request, not in the URL.
• Something called content negotiation: We could request the resource in HTML representation, in XML, and more.

That’s what REST recommends, it does not say a word how to do it, just that we should.

What’s the difference?

I was wondering what differentiates REST from CRUD. Strictly speaking, both concepts ask the same two questions:

• Which resource are you interested in?
• What you wanna do with the resource?

Are there more similarities?

• The resource is the business model in CRUD.
• The CRUD route itself contains the verb (like edit or show) whereas REST encodes this information in the HTTP request type.

Again, the real difference is: Our CRUD code already implements this behaviour, whereas REST only recommends a standard for URLs (which I do appreciate!).

REST on Rails

Now, what Rails’ REST code does is nothing more than providing some “RESTful” helpers.

The real logic, the CRUD code, is still up to you (unless you’re using José’s InheritedResources).

How does Rails help me with REST?

• It ships with a routing helper resources that creates numerous named RESTful routes for you.
• Some helpers as form_for automagically link to RESTful URLs for you depending on the model’s/resource’s internal state.

The essential thing about REST

The really cool thing about REST – beside nice, standardized URLs – is: you get a public API for free!

Folks can

• query your models by GETting /beers/1.xml, which usually returns a nicely formatted XML representation of your model if your code provides it.
• write to your database by POSTing to /beers where they simply add POST parameters to fill in attributes in the new resource.

And so on.

They don’t need wiring code or a special binding library, they can just use HTTP to fuck up your database. Isn’t that great?

What’s your problem now, Nick?

Ok, summing things up, our logic is still CRUD, it’s not RESTful.

When speaking about short URLs and about different request types we should refer to REST. The rest… is CRUD (haha, a word-play!). REST is just a thin layer above our CRUD implementation, telling us how to format requests and URLs.

People will never understand the exceptional – the cool thing – about REST if we keep confusing CRUD and REST.