Posts Tagged roar

Ruby On REST 6: Pagination With Roar

Friday, May 18th, 2012

Yo! How’s it folks? Let’s do some more REST today. I’d like to show how easy it is to have paginated REST documents with Roar while using a nifty feature introduced in version 0.10.2.

What’s In That Fruit Salad, Sir?

Since we keep having fruit salads in the last posts I wanna write a service to display the ingredients of a particular fruit bowl in a list. Not just a list of fruits, a paginated list of fruits.

Let’s GET it.

POST http://bowls/1/fruits?page=1

Considering two fruit items per page this document will be returned.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
{"total_entries":5,
 "links":[
  {"rel": "self",
   "href":"http://bowls/1/fruits?page=1"},
  {"rel": "next",
   "href":"http://bowls/1/fruits?page=2"}],
 "items":[
  {"title":"Apple",
   "links":[{"rel":"self",
              "href":"http://fruits/Apple"}]},
  {"title":"Orange",
   "links":[{"rel":"self",
             "href":"http://fruits/Orange"}]}
  ]
}

Ignore the items for now, the interesting part here are the pagination elements in the first lines. What we have here is:

  • The number of total ingredients in the bowl (line 1).
  • The obligatory self link (line 3-4).
  • A link guiding us to the next page (line 5-6).

Wow! Code?

The backing code in this service endpoint could look like the following snippet. As always, the code used to create this example can be found on github.

1
2
3
4
5
6
bowl = Bowl.find(1)
page = bowl.fruits.paginate(:page => 1, 
  :per_page => 2)
 
page.extend(BowlPageRepresenter)
page.to_json(:bowl => @bowl)

First task is to retrieve the viewed bowl (line 1). I use a static id in this example, feel free to replace it with something like params[:id] in your code. Next, I use paginate from the famous will_paginate gem to get a paged subset of the included fruits. Again, the :page parameter should be refering to a variable, whatever (line 2-3).

Then I simply extend the collection (line 5) since it is a valid Ruby object and call to_json to render the list (line 6). Keep in mind that we pass the :bowl instance into the render method as an argument.

The Bowl Representer Can Do Pagination

To fully understand that example we need to look at the representer now.

module BowlPageRepresenter
  include Roar::Representer::JSON
  include Roar::Representer::Feature::Hypermedia
 
  property :total_entries
  collection :items, :extend => FruitRepresenter
 
  link :self do |opts|
    bowl_url(opts[:bowl], :page => current_page)
  end
 
  link :next do |opts|
    bowl_url(opts[:bowl], :page => next_page) \
      if next_page
  end
 
  link :previous do |opts|
    bowl_url(opts[:bowl], :page => previous_page) \
      if previous_page
  end
 
  def items
    self
  end
end

Instead of refering to line number I’ll use code excerpts now. Do you like that better?

  property :total_entries

In a paginated collection we got the total_entries method as described in the will_paginate API docs. To give our REST consumer a hint about the total amount of ingredients we can simply define a property after that.

  collection :items, :extend => FruitRepresenter

To render the actual items in the doc we extend each fruit with the FruitRepresenter as we learned in an older post. Note that Roar will use the items method to retrieve the collection of fruits.

  def items
    self
  end

Since we are already a collection all we have to do is return self – roar will iterate the paginated collection, extend each element with the FruitRepresenter and render the items.

A Cool New Feature!

The links to the next pages are defined using Roar’s hypermedia feature.

  link :next do |opts|
    bowl_url(opts[:bowl], :page => next_page) \
      if next_page
  end

Two things happen here. First, note that this link is conditional. If next_page, another API method for a will_paginate collection, is evaluated to false, this link won’t be rendered.

Second, we use a new feature of Roar here to access variables passed into to_json. Remember how we called the render method?

page.to_json(:bowl => @bowl)

Right, we pass in some values from the outside since we don’t have access to the actual bowl instance within the represented collection. These parameters are accessible in the link block parameters. Isn’t that nice? I like it.

These few lines of code make it easy to render a paginated collection into a valid REST document.

Writing A Generic Pagination Representer

Now that all paginated documents share attributes (total entries, next and previous link, the self link, etc) why not abstract that into a generic representer? Most of the code can be reused.

module PaginationRepresenter
  include Roar::Representer::JSON
  include Roar::Representer::Feature::Hypermedia
 
  property :total_entries
 
  link :self do |opts|
    page_url(opts[:model], :page => current_page)
  end
 
  link :next do |opts|
    page_url(opts[:model], :page => next_page) \
      if next_page
  end
 
  link :previous do |opts|
    page_url(opts[:model], :page => previous_page) \
      if previous_page
  end
 
  def items
    self
  end
 
  def page_url(*)
    raise "Implement me."
  end
end

All I did was calling a generic page_url method that must be implemented by the using representer. Also, I no longer use the :bowl keyword but a more generic :model, ok?

Nothing in this abstract representer module is related to stinky fruit salads anymore. Man, this thing could even represent a sixpack of beers (a domain I do prefer over fruits).

To inherit we just include the abstract into the concrete representer.

module BowlPageRepresenter
  include Roar::Representer::JSON
  include PaginationRepresenter
 
  collection :items, :extend => FruitRepresenter
 
  def page_url(*args)
    bowl_url(*args)
  end
end

That is cool. All we have to do is defining the concrete items collection and how to compute the pagination URLs. Come on guys, that is easy!

Remember, we changed the incoming parameter, so the rendering call must change, too.

page.to_json(:model => @bowl)

Cheers!

It is Friday eve, have a wonderful weekend and let me know how it was!

Coercion Support in Roar with Virtus

Sunday, May 13th, 2012

Have you ever heard of coercion? Me neither. Anyway, people kept asking for coercion support in roar so I found out that coercion means converting strings into ruby types. Tireless I added a new feature with a whopping 3 lines of code – the cool thing is that this module marries roar with a nifty coercion gem called virtus by my good friend Piotr Solnica.

All you have to do is using the latest roar release (0.10.1), use the appropriate feature and the :type option.

class ImmigrantSong
  include Roar::Representer::JSON
  include Roar::Representer::Feature::Coercion
 
  property :composed_at, :type => DateTime, 
    :default => "May 12th, 2012"
end

This will automatically convert the composed_at property to a decent DateTime object when parsing a document. Note the working :default option, too!

document = "{\"composed_at\":\"November 18th, 1983\"}"
song = ImmigrantSong.new.from_json(document)
song.composed_at #=> 1983-11-18T00:00:00+00:00

The underlying virtus gem takes care of all the conversion work. Note that virtus also automatically adds accessors to the class.

First World Problems

A caveat is that this currently works with inline representers in classes, only. This is cause virtus doesn’t work within modules, yet. I guess we can expect a refactored virtus within days, right, solnic? As soon as this is working in virtus, you can savely use the conversion feature in modules, too. I keep you in the loop!

Oh, I forgot to thank Alex Coles who made me aware of the simplicity of combining those two gems!

Roar 0.10 with JSON-HAL Support and Representable 1.1.6 Released

Friday, April 20th, 2012

Just a quick round-up for new things in the Roar world. Here we go!

Roar Speaks JSON-HAL Now!

In the last Ruby on REST post I explained the basics of the hypermedia type HAL which gives your document nesting and linking semantics. This promising format is now supported by Roar.

By including the Representer::JSON::HAL module in your representer it will render and parse documents according to the HAL standard.

module OrderRepresenter
  include Roar::Representer::JSON::HAL
 
  property :id
  collection :items,
    :class => Item, 
    :extend => ItemRepresenter,
    :embedded => true
 
  link :self do "http://orders/#{id}" end
end

Links will now be pushed into the _links property. Also note the new :embedded option which will key the nested property under _embedded. And all that just by including a module – that’s the power of representers.

Better Links in JSON

Some users asked for another link format when not using the HAL representer. Usually links are handled like this.

"links":[
  { "rel":  "self", 
    "href": "http://orders/42"},
  { "rel":  "items", 
    "href": "http://orders/42/items"}
]}

This is just a format I came up with when writing Roar. Now, if you include Representer::JSON::HAL::Links in your representer, links will be rendered and parsed a bit different.

"links": { 
  "self":  {"href": "http://orders/42"}, 
  "items": {"href": "http://orders/42/items"}
}

Here, links is a hash keyed by rel – very handy on the client side.

Also, you can now include more attributes in links.

link :rel => :self, :title => "That's me!"

Just provide a hash listing the attributes.

Representable Got Conditional Properties!

Last but not least, representable, the underlying mapping gem of Roar got a new option in the 1.1.6 release. :if allows you to define blocks that are evaled when parsing or rendering (“representing”, yo!) the object. If the proc returns false, the property is ignored.

property :state, :if => lambda { authorized? }

The block is executed in instance context allowing you to access instance methods and whatever else you need. Go for it!

Ruby on REST 4: Using Representers in Rails

Monday, March 5th, 2012

It has been a while since I blogged about Roar and it’s representers – beautiful South Africa kept me from doing my homework. Nevertheless, in the meantime, the new roar-rails gem emerged bringing together Rails and Roar in a very easy going way. I’d like to describe how this gem works by implementing a fruit and fruit bowl backend service following the examples from the last posts.

Oh, and before I forget mentioning, the Rails example app is available on github where I will keep adding examples from the Ruby On REST blog series.

Setting It Up…

To make things happen it is necessary adding the roar-rails gem to your Gemfile.

gem 'roar-rails'

As we wanna use Rails’ URL helpers in our representers we have to add one line to the environment.

  config.representer.default_url_options = 
    {:host => "yum.my"}

Rendering Fruits, Done Right.

For simplicity let’s assume our Fruit model has the following underlying, highly complex database scheme.

  create_table "fruits" do |t|
    t.string   "title"
  end

Tough stuff, isn’t it?

Clients might want to retrieve fruit representations, that’s why we start with a functional test for handling GET requests on the FruitsController.

class FruitsControllerTest < ActionController::TestCase
  include Roar::Rails::TestCase
 
  test "GET returns fruit document" do
    get :show, :id => 1, :format => :json
    assert_body "{\"title\":\"Apple\",\"links\":[
      {\"rel\":\"self\",
       \"href\":\"http://yum.my/fruits/Apple\"}]}"
  end

Including the Roar::Rails::TestCase module into the test gives us a couple of new or changed methods. One of those is assert_body and all it does it testing equality of the passed string and the last returned document. This test verifies our REST representations are rendered correctly.

Here’s the handling code in the FruitsController.

class FruitsController < ApplicationController
  include Roar::Rails::ControllerAdditions
  respond_to :json
 
  def show
    fruit = Fruit.find_by_id(params[:id])
    respond_with fruit
  end

This is all very familiar code! But, wait… what happens here behind the scenes? The inclusion of the ControllerAdditions module activates a new responder which will take care of the JSON-rendering that is invoked inside respond_with. So, behind the curtain, the following is executed.

fruit.
  extend(FruitRepresenter).
  to_json

The roar-rails gem infers the representer name used for rendering.

The Fruit Representer

Now, where is that representer? It lives in the app/representers directory and its implementation is quite simple.

module FruitRepresenter
  include Roar::Representer::JSON
 
  property :title
 
  link :self do
    fruit_url(title)
  end
end

All it does is defining the title property and the self link. Please note that we can simply use URL helpers here, making it pretty easy to include hypermedia in our representations.

Amazing! This is all the code needed for rendering fruit documents, including hot hypermedia and handling GET requests. Now, what do we need to consume incoming documents, in, let’s say a POST request?

Eat More Fruits!

To create new fruits in our backend service we’d need to accept POST requests. Test it.

class FruitsControllerTest < ActionController::TestCase
  # ...
 
  test "POST creates fruit" do
    post :create, "{\"title\":\"Orange\"}", 
      :format => :json
 
    assert_body "{\"title\":\"Orange\",\"links\":[
      {\"rel\":\"self\",
      \"href\":\"http://yum.my/fruits/Orange\"}]}"
  end

Again, we use assert_body to assure the resource returns the freshly created representation of the orange. Now, check the post: The second argument is the actual JSON document sent by the client. This is an API change. We no longer pass hashes to the “POST”, but use real documents to test our services.

The consuming controller action is hilariously simple as well.

class FruitsController < ApplicationController
  # ...
  def create
    fruit = Fruit.new.
      extend(FruitRepresenter).
      from_json(request.body.string).
      save
 
    respond_with fruit
  end

It creates a new Fruit instance, mixes in the respective representer, parses the incoming JSON document and updates the fruit’s attributes in the #from_json method and then renders the created model.

In the upcoming version of roar-rails, this code can also be written like:

class FruitsController < ApplicationController
  # ...
  def create
    fruit = consume(Fruit).
      save
 
    respond_with fruit
  end

The consume method provided by roar-rails is still under discussion – feel free to add comments to this post if you have a groundbreaking idea how to simplify the consumption steps!

Fruits On This Planet – Unite!

If that was to easy for you, and I bet it was, why not implement the fruit bowl as well. Consider this model class.

class Bowl < ActiveRecord::Base
  has_and_belongs_to_many :fruits
end

Instead of wasting time with the rendering test and parsing code – which is identical to the fruit code – we jump directly into the parsing test.

class BowlsControllerTest < ActionController::TestCase
  include Roar::Rails::TestCase
 
  test "POST creates bowl with fruits" do
    post :create, 
      "{\"fruits\":[{\"title\":\"Orange\"}]}", 
      :format => :json
 
    bowl = Bowl.find(:last)
    assert_body "{\"fruits\":[
      {\"title\":\"Orange\",\"links\":[
        {\"rel\":\"self\",
        \"href\":\"http://yum.my/fruits/Orange\"}]}],
      \"links\":[
        {\"rel\":\"self\",
        \"href\":\"http://bowls/#{bowl.id}\"}]}"
  end

Sometimes I wish my blog was less narrow.

In the test, an initial bowl document is POSTed to the resource. We expect a fresh bowl representation containing fruits as the response.

Let’s look at the controller action to understand how representers work in a nested setup. Speaking of nesting, here’s the representer responsible for rendering and parsing bowls.

module BowlRepresenter
  include Roar::Representer::JSON
 
  property :location
 
  collection :fruits, 
    :class => Fruit, 
    :extend => FruitRepresenter
 
  link :self do
    bowl_url(id)
  end
end

In addition to the location property the collection defines a nesting fruits. The BowlRepresenter now knows that the collection contains Fruit instances that must be represented with the according FruitRepresenter. We already discussed that in one of the last posts.

Here’s the controller action using that representer.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
class BowlsController < ApplicationController
  # ...
 
  def create
    bowl = consume(Bowl)
 
    # puts bowl.fruits.inspect
    bowl.fruits = bowl.fruits.collect do |f|  
      Fruit.find_by_title(f.title)
    end
 
    bowl.save
 
    respond_with bowl
  end

I simply assume the consume call already works. Then, what is that loop? Well, when parsing the incoming document, the bowl representer creates a new Fruit instance for every fruit in the collection. If line 7 would be uncommented, it would output the following.

[#<Fruit id: nil, title: "Orange">]

That is, before we reset the collection manually. The representer doesn’t know anything about the database – it just maps a fruit representation in the document to a fresh Fruit instance. This is why there is no id set, yet. We have to do that ourselves (lines 8-10). What might look clumsy to you is simplicity – it’s up to you how to map objects to database records.

Conclusion – For Now?!

Come on, it is easy to use representers in your Rails backends using the roar-rails gem. It helps you rendering documents and makes it quite easy to have object-oriented access to a parsed incoming representation – under Rails-conditions.

And, as always, this is just a start. Open source lives from discussion and criticism, so feel free to use the comments form below. In the next post I’ll discuss writing paginated collections using representers. Have a nice day!

Ruby On REST 3: One Model, Multiple Representations!

Monday, January 9th, 2012

Happy New Year to all my fellow readers! I hope you had a good party!

In this post I wanna clarify why representers in Roar are much more versatile than one might think at first sight.

Model != Resource

I know, Rails teaches us that every resource is a model with exactly one representation. This is wrong. Not only can a resource be just anything, ranging from simple 1-to-1 mappings of database rows to virtual models like a workflow but also can your business models be represented in completely different contexts.

Let’s get back to our fruity example to learn more about this.

orange = Fruit.find_by_title("Orange")
orange.extend(FruitRepresenter).to_json
#=> {"title":"Orange","colors":["orange"]}

In the last discussion we had a Fruit and a Bowl model which were represented in their respective resources. While the fruit just contains some properties the bowl was a composition thereof. This is pretty “rails-conform” (I use this word with a negative undertone) and can simply be abstracted to hundreds of projects where people expose users, blog posts or events in their resources. This is what we call a 1-to-1 mapping of model to resource.

Let’s have some Booze!

We wrote a simple FruitRepresenter for mapping apples, lemons or oranges to documents, and we had the BowlRepresenter to provide documents for fruit collections. That was nice. But you can do more with fruits. Why not open a distillery and make some good booze from pears or grapes?

What that means for our REST API is that we have to represent fruits as ingredients. We also want to document what fruits are contained in a bottle of schnaps.

module IngredientRepresenter
  include Roar::Representer::JSON
 
  property :title
  collection :tastes_like
end

The new IngredientRepresenter looks a bit similar to our FruitRepresenter, however, it represents a fruit in another context.

pear = Fruit.find_by_title("Pear")
pear.extend(IngredientRepresenter).to_json
#=> undefined method `tastes_like'

An exception. While the pear exposes a #title method it does not have a #tastes_like method. Why did I do that? I wanted to emphasize that the representer doesn’t guess anything from the represented object (the pear). It is up to the fruit to provide decent accessors like the #tastes_like reader. No magic or implicit semantics in Roar. And this is a good thing.

Another Context, Another Module.

We could easily push the missing methods into the IngredientRepresenter itself, but we will learn in an upcoming post that it is better to provide additional accessors in a separate module.

module Ingredient
  def tastes_like
    TastesLike.query(title) # returns array.
  end
 
  def tastes_like=(*)
  end
end

In the reader, we query the world-famous TastesLikeā„¢ web service. Since we don’t need a writer, the second method ain’t doing nothing.

pear.extend(Ingredient, IngredientRepresenter).to_json
#=> {"title":"Pear","tastes_like":["pear","mango"]}

Cool, this is a fruit represented in a completely different context. What about the parsing, does that work, too?

apple = Fruit.new
apple.extend(Ingredient, IngredientRepresenter)
apple.from_json("{\"title\":\"Apple\",
  \"tastes_like\":[\"Pear\",\"Apple\",\"Mango\"]}")
#=> #<Fruit title="Apple", colors=[]>

Yepp, parsing an ingredient and mapping it to a fruit also works fine.

Fruits in a Bottle.

Let’s see how collections can be represented in different contexts.

module BottleRepresenter
  include Roar::Representer::JSON
 
  collection :fruits, :class => Fruit, 
    :extend => [Ingredient, IngredientRepresenter]
end

The bottle representer knows that it’s composed of Fruit instances. It does use the IngredientRepresenter for parsing and rendering, though. BTW, if you don’t like the long :extend assignment you are free to “hard-include” the modules directly in your models.

Summary

So what we’re doing here is having one model used in two completely different contexts: the conventional “fruit” context and the “ingredient” context. Imagine you’d be doing this with #as_json and different hash-parsing algorithms in your controllers – good luck.

Ruby On REST 2: Representers and the DCI Pattern

Friday, December 16th, 2011

We talked about Representers earlier last week. Today, I wanna describe DCI and how it fits perfectly together with representers in Roar.

What is DCI?

For years we’ve been taught that “Fat Models, Skinny Controllers” is the way to organize code. Push all your domain code into your ActiveRecord classes. Many people, including me, agree that this is bullshit. This leads to bloated classes, usually combined with one or two object instances taking care of an entire workflow (aka request).

DCI is an architectural pattern that makes the model “fat-on-demand”: Data objects are enhanced with roles at runtime, but only when needed.

The DCI pattern consists of three concepts.

  1. Data objects are supposed to contain application data. Nothing else. This would be your ActiveRecord row instance, but without any predefined behaviour. Just data.
  2. Context objects define what is done in the current workflow. In Rails, a controller running an action could be a context object.
  3. Roles are behaviour for data objects in a particular context. In Ruby, a Rails controller could mixin a module (a “role”) into an empty model row and thus let it process something.

Hooray for DCI – But Why?

Now the thing is, I absolutely hate to hype something (except stuff I did, of course). However, DCI makes code better maintainable, easier to follow and way easier to test. Period.

BTW, did you already check out my new Roar gem? It’s hip!

Think of a classical model found in many Rails applications with maybe one hundred methods. Or fifty. Fifty methods to interfere with each other. In a DCI setup, an object has a limited behaviour scope. The limited knowledge makes it easier to debug and predictable.

One Model, Multiple Faces

Since representers are modules they fit perfectly into the DCI approach, making them roles. Let’s consider the fruit representer from last week, again.

require 'roar/representer/json'
 
module FruitRepresenter
  include Roar::Representer::JSON
 
  property :title
  collection :colors
end

You still can include this representer into its “host” class directly.

class Fruit
  include Roar::Representer
  include FruitRepresenter
end

The problem now is that the Fruit class is limited to one representation, only. Forever and a day you may only render one particular representation. The key about representations, however, is that one model may be represented by multiple, yeah, representations.

Now, back to the start, let’s assume the Fruit class is some empty ActiveRecord class without any prior knowledge of representations at all.

class Fruit < ActiveRecord::Base
end
 
lemon = Fruit.from_json("{\"title\":\"Lemon\"}")
#=> undefined method `from_json' for Fruit:Class

Yeah, there is no #from_json class method.

Representers On DCI

In order to use the DCI approach, we first need an instance. Here’s how the lemon instance, and only this instance, can be extended with the fruit representer.

lemon = Fruit.new.extend(FruitRepresenter)
lemon.from_json("{\"title\":\"Lemon\"}")
lemon.title #=> "Lemon"

Awesome! The built-in Ruby #extend method makes the lemon instance aware of its representation and mixes in #from_json and friends.

Naturally, this works in both ways.

orange = Fruit.find_by_title("Orange")
orange.extend(FruitRepresenter).to_json
#=> {"title":"Apple","colors":["green"]}

The mixed-in representer also lets us render the JSON document.

Gimme Complexity!

On a flat level with single objects, this works out-of-the-box with recent Roar/representable versions. Problems arise when we have nested setups.

module BowlRepresenter
  include Roar::Representer::JSON
  include Roar::Representer::Feature::Hypermedia
 
  property :location
  collection :items, :as => Fruit,
    :extend => FruitRepresenter
 
  link :self do
    "http://bowl/#{location}"
  end
end

Using the :extend option we can give representable a hint about the representer to use when rendering/parsing contained objects.

And, hey, this works.

empty_bowl = Bowl.new
empty_bowl.extend(BowlRepresenter)
emtpy_bowl.from_json("{\"location\":\"kitchen\", \
  \"items\":[{\"title\":\"Lemon\",\"colors\":[]}]}")
 
puts empty_bowl.items.first.title #=> "Lemon"

Discussion Needed!

Having representers on an object level makes the whole thing really clean and versatile. You can test document rendering and parsing without polluting classes. And you can have a FruitRepresenter, a RottenVegetableRepresenter and what else you like for your orange instance. BTW, oranges ain’t no vegetables.

We still discuss about the API and need your opinion. Right now, we got this

collection :items, :as => Fruit,
    :extend => FruitRepresenter

Here, :as describes the class that encapsulates the contained data. The :extend option helps representable to find the correct module for the nested item. Now, what about this?

collection :items, :class => Fruit,
    :module => FruitRepresenter

Does that make sense? These options really just describe what’s on the right side of the hash rocket. Thanks to Scott for discussion here. Now, tell us your story!

Ruby On REST: Introducing the Representer Pattern

Wednesday, December 7th, 2011

This post introduces the new Roar gem, a framework-agnostic REST framework for Ruby that puts focus on object-oriented REST documents.

More object-orientation?

The calls for more object-orientation in Ruby frameworks are growing ever louder. Most web frameworks, like Rails, come with a nice database abstraction, a Rack-like HTTP endpoint, routing, and a template engine. It’s the lack of additional abstraction layers that make many programmers ask for “more OOP”.

Already yawning? Yeah, I wrote about that earlier this year. Let’s talk about working solutions now. Let’s talk about REST.

REST is about Documents!

REST is not only about pretty URLs. The problem ain’t URLs at all. Most frameworks have a decent routing layer to connect resource logic (aka controllers) to URLs. However, the key of REST is pushing documents between clients and servers, so why don’t we finally focus on documents?

It’s the document parsing, processing and rendering that makes our heads spinning.

Note: I here use the terms representation and document interchangable just because I can!

Parsing and Processing

Both clients and services have to parse incoming documents. Consider this JSON document.

{"title":  "Lemon",
 "colors": ["green"]}

This could be sent to a resource to create a new fruit, or be retrieved by some client which issued a GET request.

In Rails, and many HTTP gems, this is automatically parsed using a JSON gem.

params = JSON['{"title":  "Lemon",
                "colors": ["green"]}']
 
params["title"] #=> "Lemon"
params["colors"][0] #=> "green"

We’re using a hash access to read properties from that document. This is handy! Now, assume the sender forgot to include the colors.

params = JSON['{"title":  "Lemon"}']
 
params["colors"][0]
#=>NoMethodError: undefined method `[]' for nil:NilClass

An exception, shit! While processing the parsed document (aka hash), we have to take care of missing keys, wrong types etc. We as the consumer have to know about all the pitfalls with this particular document.

Rendering

Now that our parsing and processing code is finished, how do we render outgoing representations? We could use #to_json.

@lemon.to_json(:only => [:title, :colors])

Being a quick solution for rendering small models this approach hits the wall when we have nested models but still want a plain JSON representation, or when we want to embed hypermedia. Also, we put “REST” logic into our database layer. It’s up to you to judge.

What about template languages? Let’s try jbuilder for JSON rendering.

Jbuilder.encode do |json|
  json.title  @lemon.title
  json.colors @lemon.colors
end

Nice! But, wait. How do I test this? Can I test this separately? What if I want different representations for the same model? Do I need different templates? And… is it cool to have format knowledge both in our parsing layer and in our rendering layer? I mean, both need to know about the title property and the colors array.

Representers: OOP to our Documents!

I won’t further discuss the shown concepts here. Let me just criticize that they are not object-oriented and encourage the distribution of document internals about the entire framework.

Here’s how representers can handle all that and make you a happy man at the same time!.

require 'roar/representer/json'
 
module FruitRepresenter
  include Roar::Representer::Base
 
  property :title
  collection :colors
end

This is all we need to declare the basic syntax of the fruit document. Note that it resides in a module which can be used anywhere in your app.

Rendering a Lemon

Now let’s say our database layer provides us a class Fruit with ActiveRecord-like behaviour.

lemon = Fruit.new(:title => "Lemon")
puts lemon.title #=> "Lemon"

Since the representer knows everything about the document format it can be used for rendering JSON. We just have to include the FruitRepresenter module into the class and new methods will be available.

Fruit.class_eval do # DB layer.
  include Roar::Representer::JSON
  include FruitRepresenter
end
 
lemon.to_json
#=> "{\"title\":\"Lemon\",\"colors\":[]}"

Awesome, the FruitRepresenter module gives us a #to_json method. Internally, this method compiles the JSON document by asking the represented object (aka lemon) for its properties and values. Remember, the representer knows about those properties, since we declared them. It does not know anything about ActiveRecord and the like.

Now, what about parsing?

Parsing a Lemon

The great thing about representers is: they work bidirectional. That alone is one of the main advantages to many other gems out there. Bidirectional? Let’s just see what that is.

orange = Fruit.from_json("{\"title\":\"Orange\"}")
puts orange.inspect
#=> #<Fruit:0x9df73a8 @title="Orange", @colors=[]>

We can parse representations and create objects from it! That’s bidirectional: rendering and parsing is combined in one representer.

Again, the representer does not know anything about the underlying database – it just creates an object it was mixed into, reads from the document and assigns properties using polite setters. Nothing more, bro.

Don’t like the class method? If you already have an instance, you can use #from_json on the instance.

orange.from_json("{\"title\":\"Yummy Orange\"}")
orange.title #=> "Yummy Orange"

Nesting Representers

Now that we have fruits, lets put ‘em in a bowl. A fruit bowl contains a list of fruits.

bowl = Bowl.new
bowl.items << lemon
bowl.items << orange

In order to bind the fruit bowl to a REST resource we write another representer module.

module BowlRepresenter
  include Roar::Representer::Base
 
  property :location
  collection :items, :as => Fruit
end
 
Bowl.class_eval do # DB layer.
  include Roar::Representer::JSON
  include BowlRepresenter
end

Properties can be typed using the :as option. The target class must be a representer, too. That’s the only requirement.

bowl.to_json
#=> {"items":[
  {"title":"Lemon","colors":[]},
  {"title":"Orange","colors":[]}
]}

Typed properties allow us to build more complex representations without breaking with OOP. The collection assignment makes building lists as easy as possible. And if you still need to tweak things, there are some helpful configuration options around – more on that in one of the next posts.

Representers love HATEOAS

HATEOAS? Check out this talk I gave if you want to learn about Hypermedia and HATEOAS.

Hypermedia helps clients to operate the API without having to compute URLs. We want hypermedia in our documents.

module BowlRepresenter
  include Roar::Representer::Feature::Hypermedia
 
  link :self do
    "http://bowls/#{location}"
  end
end

Including the Hypermedia feature dynamically adds a new method #link to our representer module. What does it do?

bowl.location = :desk
bowl.to_json
#=> {
"items":[
  {"title":"Lemon","colors":[]},
  {"title":"Orange","colors":[]}],
"links":[{"rel":"self","href":"http://bowls/desk"}]
}

Seems like an easy way to embed links in our documents. In roar-rails the Rails URL helpers work like a charm within the link blocks – making it super simple to provide hypermedia to consumers and still being railsy.

Representers on the Client Side

Ok, representers basically are plain modules. Why not use ‘em on the client side as well? The client doesn’t have access to the model layer – if it does, something’s wrong! However, it’s absolutely ok to share the representer modules.

class Bowl
  include Roar::Representer::JSON
  include Roar::Representer::Feature::HttpVerbs
  include BowlRepresenter # from a gem.
end

See how we simply use an empty class with the mixed-in representer on the client side? Remember, we don’t want to have access to the database here.

The next step is creating a new fruit bowl.

bowl = Bowl.new
bowl.location = "kitchen"
bowl.post!("http://bowls")

This code is enough to send a POST request to the specified URL. The bowl instance will serialize a JSON document, send it to the URL using the HttpVerbs feature, receive the new bowl, deserialize it and update its properties accordingly. You may know this behaviour from REST client gems like ActiveResource – here, with less code and manageable complexity.

Consume Hypermedia, it’s gooood.

The representer also makes it easy to extract hypermedia links from documents.

bowl.links[:self]
#=> "http://bowls/kitchen"

No parsing needed, the representer knows where to find your hypermedia. How cool is that?

XML is not a Crime!

Roar comes with both JSON and XML representers.

module BowlRepresenter
  include Roar::Representer::XML
end
 
bowl.to_xml
#=> '<bowl>
#  <link rel="self" href="http://bowl/kitchen"/>
#</bowl>'

Now choose if you want JSON or XML – it’s up to you! I still don’t understand the XML hating – did I miss something or is it just “too java”?

Roar – and more?

The Roar project is still evolving. However, the current API is almost stable. New features, new concepts, testing helpers, etc will be added over time just like in every framework. Roar is framework-agnostic and can be used anywhere, as long as it’s Ruby.

Its modulare feature architecture makes it easy to extend your documents. Currently, we have some exciting on-going brainstorming and hacking on features like client-side caching in the representer instance, generic error handling, validations, DSLs, … We need you to try it out, complain, propose, help, and spread the word. Thank you for reading until here, you rock!

Rails Misapprehensions: The Hardest Thing about REST is Working with Representations!

Sunday, April 3rd, 2011

The REST architectural style is still intriguing me. After having discussed how PUT and POST do differ let’s move forward and learn a bit about representations, one of the key principles in REST.

What’s a Representation?

It is inevitable to understand that when accessing a REST service, say http://users.example.com/apotonick, you get a representation of the resource in its current state, and not the resource itself.

By specifying the Content-type header we can even instruct the representation type. While a JPG picture is a decent representation we usually want a XML document keeping all the important attributes of the resource (a representation can be just anything, but I’d like to focus on XML for this post).

To illustrate the requirements for proper representation handling I will use an imaginary order resource for processing orders (orders contain items).

Let the following snippet be subject to further discussion.

<order>
  <id>1</id>  
  <item>
    <link rel="article" href="http://articles/1" />
    <amount>1</amount>
  </item>
  <item>
    <link rel="article" href="http://articles/2" />
    <amount>3</amount>
  </item>
 
  <link rel="self"     href="http://orders/1" />
  <link rel="checkout" href="http://orders/checkout/1"/>
</order>

Notice the contained <item> blocks and the embedded hyperlinks in terms of good HATEOAS.

Representations can be pain.

Now what’s the problem with representations? Why not start with the server, the REST service. It has two duties to handle.

  • Serializing the order with its items is one task. This might lead to an XML document that is sent back to the client in a GET request.
  • Deserializing a representation of an order is the other tedious task. When a client POSTs/PUTs we usually need to process the sent representation and map it to our data models in order to save the changed world in the database.

Both tasks can cause headache. Rails has a couple of built-in – but limited – mechanics to help you with this work.

Generating XML representations in Rails

ActiveRecord makes serializing a model to XML very easy (at first sight) – you can simply call #to_xml.

order = Order.find(1)
puts order.to_xml

This returns an opinionated XML document.

<?xml version="1.0" encoding="UTF-8"?>
<order>
  <client-id type="integer">1</client-id>
  <created-at type="datetime">2011-04-03</created-at>
  <id type="integer">1</id>
  <updated-at type="datetime">2011-04-03</updated-at>
  ...
</order>

Additionally, #to_xml allows some tweaking by passing in option parameters. However, this approach is pretty limited. For instance, it is quite difficult to render <link rel="..."> tags with attributes.

UPDATE: Also, it is impossible to render the <item> list without a wrapping <items> tag.

Using builder for rendering

Another possibility is using an XML template with builder, in our case this is in app/views/orders/show.xml.

xml.instruct!
xml.order do
  xml.id @order.id
  xml.item do
    xml.link :rel => :article, :href => item_url(item)
#...

This provides a decent way for rendering complex XML documents that represent our resources.

Using Restfulie’s tokamak

The Restfulie gem ships with its own template language for rendering representations. This is called tokamak and works similar to a builder view.

member(@order) do
  id @order.id
  collection(@order.items) do |m, item|
    link :rel => :article, :href => item_url(item)
    amount item.amount
  end
 
  link :rel => self, :href => order_url(@order)
end

The cool thing here is that tokamak allows rendering this abstract representation to multiple targets, like XML or JSON.

Working with representations

Now that we covered the outbound part of rendering representations let’s see how we can work with incoming representations. This usually happens in POST and PUT processing actions in your controller.

Working with a representation means parsing – or deserializing – it and then accessing its attributes to map those to your data models.

How does a real POST look like?

In Rails you typically use HTML forms, POST them to a controller action, and then process the incoming data.

For a long time Rails made me think that POSTs are all about “POST parameters” – incoming data is made available in a nested params[..] data structure.

However, this is something I call a Rails misapprehension. A POST might also have a request body containing the resource representation you want to create.

A REST client written with Restfulie as HTTP helper might do the following to create a new order.

res = Restfulie.at("http://localhost:3000/photos/party")
 
res.as("application/jpg").post(%{
<order>
  <item>
    <link rel="article" href="http://articles/3" />
    <amount>1</amount>
  </item>
</order>
})

Note that we’re sending a document directly, without any form usage.

Assuming our resource controller does understand that we will create a new order. Wow.

What’s wrong with form POSTs?

Nothing! Using the #form_for helper and Rails params is absolutely ok. However, I want to point out that POSTs are not limited to HTML forms – they can also send other representations.

So, what Rails does behind the scenes is

  • parsing the incoming application/x-www-form-urlencoded representation of your resource
  • and make the deserialized representation available in #params.

This automatical behaviour is all possible if you stick to Rails’ conventions, meaning very limited XML formats and thus representations.

Working with parsed representations

In the simplest case it looks as if the Rails approach would strike.

class OrdersController < ApplicationController
  def create
    @order = Order.create(params[:order])
    # ...
  end

However, in our case, this won’t work as we have “non-standard” <link> tags in our incoming representation. The internal parser in Rails doesn’t recognize that and things will explode.

The internal parser?

Yeah, Rails has a hard-wired internal parser to deserialize a couple of mime- types (“POST parameters”, XML and JSON). This code is found in ParamsParser here.

I’m sorry to say that but that middleware is a debatable concept in Rails. While it blindly parses incoming request data it also urges you to use Rails’ limited representation convention.

If you don’t go the Rails way and use very simple XML representations, it will simply not work.

Parsing D.I.Y.

Now that Rails doesn’t give us what we need we have to do it ourself.

class OrdersController < ApplicationController
  def create
    item_urls = Nokogiri::XML(request.raw_post).
      xpath("//item/link").
      collect { |i| i[:href] }
 
    @order = Order.new
    # ...
  end

Looks horrible.

The dilemma – where is my representation?

A severe problem emerges – where is my representation? Or, in other words: where are transformation rules stored?

  • Rendering the representation involves either an XML template or an adjusted #to_xml which knows a lot about the structure and semantics of our representation.
  • Parsing the representation implies a hand-made XML parser that itself knows about the very structure and semantics of the representation, again.

The dilemma: Knowledge about the representation is cluttered over the entire MVC-framework.

A new abstraction layer: Representers

What might sound over-engineered to many developers used to the Rails convention is my new approach which abstracts “working with representations” to a new layer.

A so called Representer handles both the in- and outs of representations, not sharing the internal syntax and semantics with the outer world (the best we can).

The Item representer looks simple.

class ItemXmlRepresenter < Roar::Representer::Xml
  name :item
  link
 
  property :amount
end

The Order representer already contains a composition – the ordered items.

class OrderXmlRepresenter < Roar::Representer::Xml
  name :order
 
  property :id
  property :items, :tag => :item, 
    :as => [ItemXmlRepresenter]
 
  link do |controller|
    :rel => :self, :href => controller.order_url(self)
  end
 
  link do |controller|
    :rel => :checkout, 
    :href => controller.order_checkout_url(self)
  end
end

This is a part of my REST framework roar which tries to help with writing real RESTful apps in Rails, Sinatra and Padrino.

How does roar help me?

Now that we defined the representers we can use them in controllers.

class OrdersController < ApplicationController
  def show
    @order = Order.find(params[:id])
 
    xml = OrderXmlRepresenter.serialize_model(@order)
    # ...

The call to #serialize_model will render the XML representation for the @order object using the proper representers, including real HATEOAS hyperlinks, nested items, and so on.

What comes out is the XML document I introduced in the beginning paragraph.

Now, how do we process input?

class OrdersController < ApplicationController
  def create
    rep = OrderXmlRepresenter.
      deserialize(request.raw_post)
 
    @order = Order.create(rep.to_nested_attributes)
 
    # ...

When handling incoming data the #to_nested_attributes method from the representer helps transforming the data into a hash complying to nested-attributes requirements used with #update_attributes.

Benefits?

Representers try to improve the architecture. They…

  • embody the transformation of a representation in one class while allowing serialization and deserialization.
  • allow compositions of representations. Order has_many items.
  • make hypermedia support easy (we will learn more about that in the next post)
  • are inheritable and extendable and bring back OOP. For instance, you might write convenience accessors into your representer class.
  • are usable in REST clients as well, which allows decoupled, solid systems (see next post).
  • make testing easier.

Hey, this is a proposal!

The cool thing here is: that stuff is already working. It’s still under heavy development, but it works. I’m maintaining a sample application at github which I use in my upcoming thesis about Ruby, REST architectures and web GUIs.

My goal with roar is to make it easier and cleaner to develop RESTful, decoupled systems in Ruby while using all the great existing gems like Restfulie.

Looking forward to your feedback, now.

In the next post I will write about how Representers help keeping your REST clients clean and dumb.