Happy New Year to all my fellow readers! I hope you had a good party!

In this post I wanna clarify why representers in Roar are much more versatile than one might think at first sight.

Model != Resource

I know, Rails teaches us that every resource is a model with exactly one representation. This is wrong. Not only can a resource be just anything, ranging from simple 1-to-1 mappings of database rows to virtual models like a workflow but also can your business models be represented in completely different contexts.

Let’s get back to our fruity example to learn more about this.

orange = Fruit.find_by_title("Orange")
orange.extend(FruitRepresenter).to_json
#=> {"title":"Orange","colors":["orange"]}

In the last discussion we had a Fruit and a Bowl model which were represented in their respective resources. While the fruit just contains some properties the bowl was a composition thereof. This is pretty “rails-conform” (I use this word with a negative undertone) and can simply be abstracted to hundreds of projects where people expose users, blog posts or events in their resources. This is what we call a 1-to-1 mapping of model to resource.

Let’s have some Booze!

We wrote a simple FruitRepresenter for mapping apples, lemons or oranges to documents, and we had the BowlRepresenter to provide documents for fruit collections. That was nice. But you can do more with fruits. Why not open a distillery and make some good booze from pears or grapes?

What that means for our REST API is that we have to represent fruits as ingredients. We also want to document what fruits are contained in a bottle of schnaps.

module IngredientRepresenter
  include Roar::Representer::JSON
 
  property :title
  collection :tastes_like
end

The new IngredientRepresenter looks a bit similar to our FruitRepresenter, however, it represents a fruit in another context.

pear = Fruit.find_by_title("Pear")
pear.extend(IngredientRepresenter).to_json
#=> undefined method `tastes_like'

An exception. While the pear exposes a #title method it does not have a #tastes_like method. Why did I do that? I wanted to emphasize that the representer doesn’t guess anything from the represented object (the pear). It is up to the fruit to provide decent accessors like the #tastes_like reader. No magic or implicit semantics in Roar. And this is a good thing.

Another Context, Another Module.

We could easily push the missing methods into the IngredientRepresenter itself, but we will learn in an upcoming post that it is better to provide additional accessors in a separate module.

module Ingredient
  def tastes_like
    TastesLike.query(title) # returns array.
  end
 
  def tastes_like=(*)
  end
end

In the reader, we query the world-famous TastesLike™ web service. Since we don’t need a writer, the second method ain’t doing nothing.

pear.extend(Ingredient, IngredientRepresenter).to_json
#=> {"title":"Pear","tastes_like":["pear","mango"]}

Cool, this is a fruit represented in a completely different context. What about the parsing, does that work, too?

apple = Fruit.new
apple.extend(Ingredient, IngredientRepresenter)
apple.from_json("{\"title\":\"Apple\",
  \"tastes_like\":[\"Pear\",\"Apple\",\"Mango\"]}")
#=> #<Fruit title="Apple", colors=[]>

Yepp, parsing an ingredient and mapping it to a fruit also works fine.

Fruits in a Bottle.

Let’s see how collections can be represented in different contexts.

module BottleRepresenter
  include Roar::Representer::JSON
 
  collection :fruits, :class => Fruit, 
    :extend => [Ingredient, IngredientRepresenter]
end

The bottle representer knows that it’s composed of Fruit instances. It does use the IngredientRepresenter for parsing and rendering, though. BTW, if you don’t like the long :extend assignment you are free to “hard-include” the modules directly in your models.

Summary

So what we’re doing here is having one model used in two completely different contexts: the conventional “fruit” context and the “ingredient” context. Imagine you’d be doing this with #as_json and different hash-parsing algorithms in your controllers – good luck.

We talked about Representers earlier last week. Today, I wanna describe DCI and how it fits perfectly together with representers in Roar.

What is DCI?

For years we’ve been taught that “Fat Models, Skinny Controllers” is the way to organize code. Push all your domain code into your ActiveRecord classes. Many people, including me, agree that this is bullshit. This leads to bloated classes, usually combined with one or two object instances taking care of an entire workflow (aka request).

DCI is an architectural pattern that makes the model “fat-on-demand”: Data objects are enhanced with roles at runtime, but only when needed.

The DCI pattern consists of three concepts.

1. Data objects are supposed to contain application data. Nothing else. This would be your ActiveRecord row instance, but without any predefined behaviour. Just data.
2. Context objects define what is done in the current workflow. In Rails, a controller running an action could be a context object.
3. Roles are behaviour for data objects in a particular context. In Ruby, a Rails controller could mixin a module (a “role”) into an empty model row and thus let it process something.

Hooray for DCI – But Why?

Now the thing is, I absolutely hate to hype something (except stuff I did, of course). However, DCI makes code better maintainable, easier to follow and way easier to test. Period.

BTW, did you already check out my new Roar gem? It’s hip!

Think of a classical model found in many Rails applications with maybe one hundred methods. Or fifty. Fifty methods to interfere with each other. In a DCI setup, an object has a limited behaviour scope. The limited knowledge makes it easier to debug and predictable.

One Model, Multiple Faces

Since representers are modules they fit perfectly into the DCI approach, making them roles. Let’s consider the fruit representer from last week, again.

require 'roar/representer/json'
 
module FruitRepresenter
  include Roar::Representer::JSON
 
  property :title
  collection :colors
end

You still can include this representer into its “host” class directly.

class Fruit
  include Roar::Representer
  include FruitRepresenter
end

The problem now is that the Fruit class is limited to one representation, only. Forever and a day you may only render one particular representation. The key about representations, however, is that one model may be represented by multiple, yeah, representations.

Now, back to the start, let’s assume the Fruit class is some empty ActiveRecord class without any prior knowledge of representations at all.

class Fruit < ActiveRecord::Base
end
 
lemon = Fruit.from_json("{\"title\":\"Lemon\"}")
#=> undefined method `from_json' for Fruit:Class

Yeah, there is no #from_json class method.

Representers On DCI

In order to use the DCI approach, we first need an instance. Here’s how the lemon instance, and only this instance, can be extended with the fruit representer.

lemon = Fruit.new.extend(FruitRepresenter)
lemon.from_json("{\"title\":\"Lemon\"}")
lemon.title #=> "Lemon"

Awesome! The built-in Ruby #extend method makes the lemon instance aware of its representation and mixes in #from_json and friends.

Naturally, this works in both ways.

orange = Fruit.find_by_title("Orange")
orange.extend(FruitRepresenter).to_json
#=> {"title":"Apple","colors":["green"]}

The mixed-in representer also lets us render the JSON document.

Gimme Complexity!

On a flat level with single objects, this works out-of-the-box with recent Roar/representable versions. Problems arise when we have nested setups.

module BowlRepresenter
  include Roar::Representer::JSON
  include Roar::Representer::Feature::Hypermedia
 
  property :location
  collection :items, :as => Fruit,
    :extend => FruitRepresenter
 
  link :self do
    "http://bowl/#{location}"
  end
end

Using the :extend option we can give representable a hint about the representer to use when rendering/parsing contained objects.

And, hey, this works.

empty_bowl = Bowl.new
empty_bowl.extend(BowlRepresenter)
emtpy_bowl.from_json("{\"location\":\"kitchen\", \
  \"items\":[{\"title\":\"Lemon\",\"colors\":[]}]}")
 
puts empty_bowl.items.first.title #=> "Lemon"

Discussion Needed!

Having representers on an object level makes the whole thing really clean and versatile. You can test document rendering and parsing without polluting classes. And you can have a FruitRepresenter, a RottenVegetableRepresenter and what else you like for your orange instance. BTW, oranges ain’t no vegetables.

We still discuss about the API and need your opinion. Right now, we got this

collection :items, :as => Fruit,
    :extend => FruitRepresenter

Here, :as describes the class that encapsulates the contained data. The :extend option helps representable to find the correct module for the nested item. Now, what about this?

collection :items, :class => Fruit,
    :module => FruitRepresenter

Does that make sense? These options really just describe what’s on the right side of the hash rocket. Thanks to Scott for discussion here. Now, tell us your story!

This post introduces the new Roar gem, a framework-agnostic REST framework for Ruby that puts focus on object-oriented REST documents.

More object-orientation?

The calls for more object-orientation in Ruby frameworks are growing ever louder. Most web frameworks, like Rails, come with a nice database abstraction, a Rack-like HTTP endpoint, routing, and a template engine. It’s the lack of additional abstraction layers that make many programmers ask for “more OOP”.

Already yawning? Yeah, I wrote about that earlier this year. Let’s talk about working solutions now. Let’s talk about REST.

REST is about Documents!

REST is not only about pretty URLs. The problem ain’t URLs at all. Most frameworks have a decent routing layer to connect resource logic (aka controllers) to URLs. However, the key of REST is pushing documents between clients and servers, so why don’t we finally focus on documents?

It’s the document parsing, processing and rendering that makes our heads spinning.

Note: I here use the terms representation and document interchangable just because I can!

Parsing and Processing

Both clients and services have to parse incoming documents. Consider this JSON document.

{"title":  "Lemon",
 "colors": ["green"]}

This could be sent to a resource to create a new fruit, or be retrieved by some client which issued a GET request.

In Rails, and many HTTP gems, this is automatically parsed using a JSON gem.

params = JSON['{"title":  "Lemon",
                "colors": ["green"]}']
 
params["title"] #=> "Lemon"
params["colors"][0] #=> "green"

We’re using a hash access to read properties from that document. This is handy! Now, assume the sender forgot to include the colors.

params = JSON['{"title":  "Lemon"}']
 
params["colors"][0]
#=>NoMethodError: undefined method `[]' for nil:NilClass

An exception, shit! While processing the parsed document (aka hash), we have to take care of missing keys, wrong types etc. We as the consumer have to know about all the pitfalls with this particular document.

Rendering

Now that our parsing and processing code is finished, how do we render outgoing representations? We could use #to_json.

@lemon.to_json(:only => [:title, :colors])

Being a quick solution for rendering small models this approach hits the wall when we have nested models but still want a plain JSON representation, or when we want to embed hypermedia. Also, we put “REST” logic into our database layer. It’s up to you to judge.

What about template languages? Let’s try jbuilder for JSON rendering.

Jbuilder.encode do |json|
  json.title  @lemon.title
  json.colors @lemon.colors
end

Nice! But, wait. How do I test this? Can I test this separately? What if I want different representations for the same model? Do I need different templates? And… is it cool to have format knowledge both in our parsing layer and in our rendering layer? I mean, both need to know about the title property and the colors array.

Representers: OOP to our Documents!

I won’t further discuss the shown concepts here. Let me just criticize that they are not object-oriented and encourage the distribution of document internals about the entire framework.

Here’s how representers can handle all that and make you a happy man at the same time!.

require 'roar/representer/json'
 
module FruitRepresenter
  include Roar::Representer::Base
 
  property :title
  collection :colors
end

This is all we need to declare the basic syntax of the fruit document. Note that it resides in a module which can be used anywhere in your app.

Rendering a Lemon

Now let’s say our database layer provides us a class Fruit with ActiveRecord-like behaviour.

lemon = Fruit.new(:title => "Lemon")
puts lemon.title #=> "Lemon"

Since the representer knows everything about the document format it can be used for rendering JSON. We just have to include the FruitRepresenter module into the class and new methods will be available.

Fruit.class_eval do # DB layer.
  include Roar::Representer::JSON
  include FruitRepresenter
end
 
lemon.to_json
#=> "{\"title\":\"Lemon\",\"colors\":[]}"

Awesome, the FruitRepresenter module gives us a #to_json method. Internally, this method compiles the JSON document by asking the represented object (aka lemon) for its properties and values. Remember, the representer knows about those properties, since we declared them. It does not know anything about ActiveRecord and the like.

Now, what about parsing?

Parsing a Lemon

The great thing about representers is: they work bidirectional. That alone is one of the main advantages to many other gems out there. Bidirectional? Let’s just see what that is.

orange = Fruit.from_json("{\"title\":\"Orange\"}")
puts orange.inspect
#=> #<Fruit:0x9df73a8 @title="Orange", @colors=[]>

We can parse representations and create objects from it! That’s bidirectional: rendering and parsing is combined in one representer.

Again, the representer does not know anything about the underlying database – it just creates an object it was mixed into, reads from the document and assigns properties using polite setters. Nothing more, bro.

Don’t like the class method? If you already have an instance, you can use #from_json on the instance.

orange.from_json("{\"title\":\"Yummy Orange\"}")
orange.title #=> "Yummy Orange"

Nesting Representers

Now that we have fruits, lets put ‘em in a bowl. A fruit bowl contains a list of fruits.

bowl = Bowl.new
bowl.items << lemon
bowl.items << orange

In order to bind the fruit bowl to a REST resource we write another representer module.

module BowlRepresenter
  include Roar::Representer::Base
 
  property :location
  collection :items, :as => Fruit
end
 
Bowl.class_eval do # DB layer.
  include Roar::Representer::JSON
  include BowlRepresenter
end

Properties can be typed using the :as option. The target class must be a representer, too. That’s the only requirement.

bowl.to_json
#=> {"items":[
  {"title":"Lemon","colors":[]},
  {"title":"Orange","colors":[]}
]}

Typed properties allow us to build more complex representations without breaking with OOP. The collection assignment makes building lists as easy as possible. And if you still need to tweak things, there are some helpful configuration options around – more on that in one of the next posts.

Representers love HATEOAS

HATEOAS? Check out this talk I gave if you want to learn about Hypermedia and HATEOAS.

Hypermedia helps clients to operate the API without having to compute URLs. We want hypermedia in our documents.

module BowlRepresenter
  include Roar::Representer::Feature::Hypermedia
 
  link :self do
    "http://bowls/#{location}"
  end
end

Including the Hypermedia feature dynamically adds a new method #link to our representer module. What does it do?

bowl.location = :desk
bowl.to_json
#=> {
"items":[
  {"title":"Lemon","colors":[]},
  {"title":"Orange","colors":[]}],
"links":[{"rel":"self","href":"http://bowls/desk"}]
}

Seems like an easy way to embed links in our documents. In roar-rails the Rails URL helpers work like a charm within the link blocks – making it super simple to provide hypermedia to consumers and still being railsy.

Representers on the Client Side

Ok, representers basically are plain modules. Why not use ‘em on the client side as well? The client doesn’t have access to the model layer – if it does, something’s wrong! However, it’s absolutely ok to share the representer modules.

class Bowl
  include Roar::Representer::JSON
  include Roar::Representer::Feature::HttpVerbs
  include BowlRepresenter # from a gem.
end

See how we simply use an empty class with the mixed-in representer on the client side? Remember, we don’t want to have access to the database here.

The next step is creating a new fruit bowl.

bowl = Bowl.new
bowl.location = "kitchen"
bowl.post!("http://bowls")

This code is enough to send a POST request to the specified URL. The bowl instance will serialize a JSON document, send it to the URL using the HttpVerbs feature, receive the new bowl, deserialize it and update its properties accordingly. You may know this behaviour from REST client gems like ActiveResource – here, with less code and manageable complexity.

Consume Hypermedia, it’s gooood.

The representer also makes it easy to extract hypermedia links from documents.

bowl.links[:self]
#=> "http://bowls/kitchen"

No parsing needed, the representer knows where to find your hypermedia. How cool is that?

XML is not a Crime!

Roar comes with both JSON and XML representers.

module BowlRepresenter
  include Roar::Representer::XML
end
 
bowl.to_xml
#=> '<bowl>
#  <link rel="self" href="http://bowl/kitchen"/>
#</bowl>'

Now choose if you want JSON or XML – it’s up to you! I still don’t understand the XML hating – did I miss something or is it just “too java”?

Roar – and more?

The Roar project is still evolving. However, the current API is almost stable. New features, new concepts, testing helpers, etc will be added over time just like in every framework. Roar is framework-agnostic and can be used anywhere, as long as it’s Ruby.

Its modulare feature architecture makes it easy to extend your documents. Currently, we have some exciting on-going brainstorming and hacking on features like client-side caching in the representer instance, generic error handling, validations, DSLs, … We need you to try it out, complain, propose, help, and spread the word. Thank you for reading until here, you rock!

The REST architectural style is still intriguing me. After having discussed how PUT and POST do differ let’s move forward and learn a bit about representations, one of the key principles in REST.

What’s a Representation?

It is inevitable to understand that when accessing a REST service, say http://users.example.com/apotonick, you get a representation of the resource in its current state, and not the resource itself.

By specifying the Content-type header we can even instruct the representation type. While a JPG picture is a decent representation we usually want a XML document keeping all the important attributes of the resource (a representation can be just anything, but I’d like to focus on XML for this post).

To illustrate the requirements for proper representation handling I will use an imaginary order resource for processing orders (orders contain items).

Let the following snippet be subject to further discussion.

<order>
  <id>1</id>  
  <item>
    <link rel="article" href="http://articles/1" />
    <amount>1</amount>
  </item>
  <item>
    <link rel="article" href="http://articles/2" />
    <amount>3</amount>
  </item>
 
  <link rel="self"     href="http://orders/1" />
  <link rel="checkout" href="http://orders/checkout/1"/>
</order>

Notice the contained <item> blocks and the embedded hyperlinks in terms of good HATEOAS.

Representations can be pain.

Now what’s the problem with representations? Why not start with the server, the REST service. It has two duties to handle.

• Serializing the order with its items is one task. This might lead to an XML document that is sent back to the client in a GET request.

• Deserializing a representation of an order is the other tedious task. When a client POSTs/PUTs we usually need to process the sent representation and map it to our data models in order to save the changed world in the database.

Both tasks can cause headache. Rails has a couple of built-in – but limited – mechanics to help you with this work.

Generating XML representations in Rails

ActiveRecord makes serializing a model to XML very easy (at first sight) – you can simply call #to_xml.

order = Order.find(1)
puts order.to_xml

This returns an opinionated XML document.

<?xml version="1.0" encoding="UTF-8"?>
<order>
  <client-id type="integer">1</client-id>
  <created-at type="datetime">2011-04-03</created-at>
  <id type="integer">1</id>
  <updated-at type="datetime">2011-04-03</updated-at>
  ...
</order>

Additionally, #to_xml allows some tweaking by passing in option parameters. However, this approach is pretty limited. For instance, it is quite difficult to render <link rel="..."> tags with attributes.

UPDATE: Also, it is impossible to render the <item> list without a wrapping <items> tag.

Using builder for rendering

Another possibility is using an XML template with builder, in our case this is in app/views/orders/show.xml.

xml.instruct!
xml.order do
  xml.id @order.id
  xml.item do
    xml.link :rel => :article, :href => item_url(item)
#...

This provides a decent way for rendering complex XML documents that represent our resources.

Using Restfulie’s tokamak

The Restfulie gem ships with its own template language for rendering representations. This is called tokamak and works similar to a builder view.

member(@order) do
  id @order.id
  collection(@order.items) do |m, item|
    link :rel => :article, :href => item_url(item)
    amount item.amount
  end
 
  link :rel => self, :href => order_url(@order)
end

The cool thing here is that tokamak allows rendering this abstract representation to multiple targets, like XML or JSON.

Working with representations

Now that we covered the outbound part of rendering representations let’s see how we can work with incoming representations. This usually happens in POST and PUT processing actions in your controller.

Working with a representation means parsing – or deserializing – it and then accessing its attributes to map those to your data models.

How does a real POST look like?

In Rails you typically use HTML forms, POST them to a controller action, and then process the incoming data.

For a long time Rails made me think that POSTs are all about “POST parameters” – incoming data is made available in a nested params[..] data structure.

However, this is something I call a Rails misapprehension. A POST might also have a request body containing the resource representation you want to create.

A REST client written with Restfulie as HTTP helper might do the following to create a new order.

res = Restfulie.at("http://localhost:3000/photos/party")
 
res.as("application/jpg").post(%{
<order>
  <item>
    <link rel="article" href="http://articles/3" />
    <amount>1</amount>
  </item>
</order>
})

Note that we’re sending a document directly, without any form usage.

Assuming our resource controller does understand that we will create a new order. Wow.

What’s wrong with form POSTs?

Nothing! Using the #form_for helper and Rails params is absolutely ok. However, I want to point out that POSTs are not limited to HTML forms – they can also send other representations.

So, what Rails does behind the scenes is

• parsing the incoming application/x-www-form-urlencoded representation of your resource

• and make the deserialized representation available in #params.

This automatical behaviour is all possible if you stick to Rails’ conventions, meaning very limited XML formats and thus representations.

Working with parsed representations

In the simplest case it looks as if the Rails approach would strike.

class OrdersController < ApplicationController
  def create
    @order = Order.create(params[:order])
    # ...
  end

However, in our case, this won’t work as we have “non-standard” <link> tags in our incoming representation. The internal parser in Rails doesn’t recognize that and things will explode.

The internal parser?

Yeah, Rails has a hard-wired internal parser to deserialize a couple of mime- types (“POST parameters”, XML and JSON). This code is found in ParamsParser here.

I’m sorry to say that but that middleware is a debatable concept in Rails. While it blindly parses incoming request data it also urges you to use Rails’ limited representation convention.

If you don’t go the Rails way and use very simple XML representations, it will simply not work.

Parsing D.I.Y.

Now that Rails doesn’t give us what we need we have to do it ourself.

class OrdersController < ApplicationController
  def create
    item_urls = Nokogiri::XML(request.raw_post).
      xpath("//item/link").
      collect { |i| i[:href] }
 
    @order = Order.new
    # ...
  end

Looks horrible.

The dilemma – where is my representation?

A severe problem emerges – where is my representation? Or, in other words: where are transformation rules stored?

• Rendering the representation involves either an XML template or an adjusted #to_xml which knows a lot about the structure and semantics of our representation.

• Parsing the representation implies a hand-made XML parser that itself knows about the very structure and semantics of the representation, again.

The dilemma: Knowledge about the representation is cluttered over the entire MVC-framework.

A new abstraction layer: Representers

What might sound over-engineered to many developers used to the Rails convention is my new approach which abstracts “working with representations” to a new layer.

A so called Representer handles both the in- and outs of representations, not sharing the internal syntax and semantics with the outer world (the best we can).

The Item representer looks simple.

class ItemXmlRepresenter < Roar::Representer::Xml
  name :item
  link
 
  property :amount
end

The Order representer already contains a composition – the ordered items.

class OrderXmlRepresenter < Roar::Representer::Xml
  name :order
 
  property :id
  property :items, :tag => :item, 
    :as => [ItemXmlRepresenter]
 
  link do |controller|
    :rel => :self, :href => controller.order_url(self)
  end
 
  link do |controller|
    :rel => :checkout, 
    :href => controller.order_checkout_url(self)
  end
end

This is a part of my REST framework roar which tries to help with writing real RESTful apps in Rails, Sinatra and Padrino.

How does roar help me?

Now that we defined the representers we can use them in controllers.

class OrdersController < ApplicationController
  def show
    @order = Order.find(params[:id])
 
    xml = OrderXmlRepresenter.serialize_model(@order)
    # ...

The call to #serialize_model will render the XML representation for the @order object using the proper representers, including real HATEOAS hyperlinks, nested items, and so on.

What comes out is the XML document I introduced in the beginning paragraph.

Now, how do we process input?

class OrdersController < ApplicationController
  def create
    rep = OrderXmlRepresenter.
      deserialize(request.raw_post)
 
    @order = Order.create(rep.to_nested_attributes)
 
    # ...

When handling incoming data the #to_nested_attributes method from the representer helps transforming the data into a hash complying to nested-attributes requirements used with #update_attributes.

Benefits?

Representers try to improve the architecture. They…

• embody the transformation of a representation in one class while allowing serialization and deserialization.
• allow compositions of representations. Order has_many items.
• make hypermedia support easy (we will learn more about that in the next post)
• are inheritable and extendable and bring back OOP. For instance, you might write convenience accessors into your representer class.
• are usable in REST clients as well, which allows decoupled, solid systems (see next post).

• make testing easier.

Hey, this is a proposal!

The cool thing here is: that stuff is already working. It’s still under heavy development, but it works. I’m maintaining a sample application at github which I use in my upcoming thesis about Ruby, REST architectures and web GUIs.

My goal with roar is to make it easier and cleaner to develop RESTful, decoupled systems in Ruby while using all the great existing gems like Restfulie.

Looking forward to your feedback, now.

In the next post I will write about how Representers help keeping your REST clients clean and dumb.