After almost two weeks of intense hacking, I’m happy to announce Representable 2.2. This is a pure performance update. I removed some very private concepts and methods, hence the minor version bump. Anyway, with the 2.2 version you can expect a 50% and more speed-up for both rendering and parsing.

This is, for Representable itself, but also for Roar, Disposable and Reform, that all use Representable internally for data transformations.

The public API hasn’t changed at all, so you’re safe to update.

To get a quick overview about the code changes, have a look here. Right, that’s only a handful lines of code that have changed.

Profiling: How We Did It.

It all started with a benchmark my friend Max was running to render a nested object graph. Here’s the structure of the document.

class FoosRepresentation < Representable::Decorator
  include Representable::JSON
 
  collection :foos, class: Foo do
    property :value
 
    property :bar, class: Bar do
      property :value
    end
  end
end

As you can see, this is a collection of foos. Every foo contains a nested bar object with a value property.

So he set up a profiling test with 10,000 foos containing one bar each. You can find his profiling repository here. For Representable, that basically means “Iterate 10,000 foo objects and 10,000 bar objects and serialize them.”, making it 20,001 objects to represent in total.

Benchmarks, Slow.

With Representable 2.1.8, to serialize this tree took about 3.1 seconds on my work machine.

Total: 3.138545
 
 %self   total   calls    name
 30.17   0.947   20001    Module#name
  7.01   0.220   790039   Representable::Definition#[]
  5.71   0.308   460025   Representable::Binding#[]
  1.68   0.135   100009   Class#new

As you can see, Module#name is called many times, and more than 100,000 objects are instantiated.

Here’s the same benchmark for deserializing a document with 200,001 objects.

Total: 3.045645
 
 %self  calls    name
 31.78   20001   Module#name
  6.07  710029   Representable::Definition#[]
  5.48  480021   Representable::Binding#[]
  2.46  160008   Class#new

It’s interesting to observe that parsing a document into an object graph takes about the same time as rendering it. Anyway, many objects are created and lot of time is wasted

Benchmarks, Faster!

I applied some simple implementation fixes and a structural change, resulting in the following benchmarks. We’ll discuss how I achieved that in a second.

First, we ran the rendering benchmark.

Total: 1.141275
 
 %self   calls   name
  6.83   310100  Representable::Definition#[]
  2.96   50007   Uber::Options::Value#evaluate
  2.78   40001   Repr..::Deserializer::Prepare#prepare!

Wow, that’s not 3.1 but 1.1 seconds to render a deeply nested object graph. No unnecessary classes are instantiated anymore and the time-consuming Module#name call has vanished.

The exact same I could achieve for parsing time.

Total: 1.173969
 
 %self    calls    name
  7.37    330092   Representable::Definition#[]
  3.33    70007    Uber::Options::Value#evaluate
  3.27    100029   Representable::Binding#[]
  2.09    20001    Representable#representation_wrap

What I didn’t measure, yet, is the memory foot-print which should be dramatically (yes, dramatically) smaller as the amount of objects we need to parse or render object graphs has minimized. I bet you want to know now how this 50-75% speedup was possible, and here we go.

Don’t Ask When You’re Not Interested.

The first thing I tackled was to get rid of the Module#name call. This resulted from computing the wrap for every represented object that was being serialized or parsed. Every represented object. Even though most objects don’t need a wrap, and the default case is to not have wrapping.

I moved the name query into an optional block and things got faster.

Now, Module#name is only being called when we actually want to know the wrap.

Too Many Bindings.

However, this was just one step to increasing the performance.

Another issue clearly visible in the ruby-prof outputs was that we created a Binding for every object. Every represented object. Bindings are a concept in Representable that handle rendering and parsing for one property.

In case this was a nested property, this binding would create a representer instance, again, which in turn would create bindings again, for every represented property.

bindings-before-2.2

My beautiful diagram, which makes me really proud, illustrates that: Every Foo instance in the collection will create a representer and a binding instance, even though they are identical.

When writing Representable a few years ago I had a feeling that this might become a bottle-neck one day, but being focused on designing I “forgot” about it, and no one ever complained.

Reuse Bindings.

The solution is dead simple. I introduced Representable::Cached that will save the Bindings for later reuse.

By making the Binding stateless we won’t have any trouble with stale data. The binding used to save a lot of run-time information like the represented object, and more. This now has to be passed from the outside for every run, making it reusable.

I know, you love my diagrams. Check out how the object graph has changed.

bindings-in-2.2

Say we were representing one Foo object, its decorator will now cache the binding for the Bar property and reuse it. This results in a handful of objects needed to be created.

To give you some figures, in the aforementioned benchmark setup, instead of having to instantiate 200,000 bindings all we need to do is to create four! One for the collection of foos, on for value in every foo, one for a Bar object and the last to represent the value in a bar.

Cached: An Optional Feature!

While I fully trust my changes (would be bad if I wouldn’t) I decided to add this as an optional feature in 2.2. You need to activate it manually on the top-level representer.

class FoosRepresentation < Representable::Decorator
  include Representable::JSON
  feature Representable::Cached
 
  collection :foos, class: Foo do
    # and so on

It’s a bit late to mention that, but Cached only works with the Decorator representers. It also works with modules, but it will unnecessarily pollute your models. Please don’t do that.

Reusing Decorators Across Requests

An interesting new option is caching of representers between requests. This will boost up rendering and parsing documents in your API code many times – not to speak about the reduced memory footprint.

Once a representer’s done its job, it can be reused using the #update! method.

decorator = SongRepresenter.new(song)
decorator.to_json # first request.
 
decorator.update!(better_song) # second request.
decorator.to_json

This is enough to reuse a representer, even a deeply nested graph.

Once I find some time, I will implement this in roar-rails. Caching and reusing representers across requests will give a significantly performance boost for many Roar/Representable apps out there.

Using Ruby-Prof For Tests

One last thing I want to mention is how I use the ruby-prof output for tests.

RubyProf.start
  representer.to_hash
# ..
 
# a total of 4 properties in the object graph.
data.must_match "4   Representable::Binding#initialize"

What might look like a weird crazy bullshit is a fantastic way of asserting that your speed-up actually works. Since we cannot test for speed of test runs (every machine and run is different) I simply test for object creations.

In the Cached tests, I setup a simple, but complex enough object graph and render and parse it. I let ruby-prof track object creation and method invocations. Afterwards, I make sure that really only four bindings were created (and other instantiation counts, of course).

And: this works!

No. I’m not gonna use Rspec’s mocking and expectations for that. First of all, I’ve managed not to use mocking in any of my gems’ test suites and want to keep it that way. Second, the more test code I add, the more I will miss and will go wrong.

Letting a super low-level tool like ruby-prof track method calls is a bullet-proof way to test your speed-up. I love it and was surprised by its accuracy.

There Was More.

A few more improvements went into 2.2, and a lot of lookups can still be improved. I feel like I’ve talked enough for today.

Instead of throwing more words at you, I’d like to thank Max Güntner for setting up most of the benchmark code and encouraging me to work on performance. And, special thanks go out to Charlie Somerville who turned out to be a Ruby-internal-monster and was a great help in explaining the depths of the Ruby VM to me and why attr_reader is faster than your own method, and so on.

Incredibly what you still can learn after having hacked Ruby for more than 10 years.

Many people use the Representable gem to render documents from Ruby objects, or to parse incoming documents into an object graph.

This is great for implementing document APIs with JSON or XML. Since Representable does both ways, rendering and deserializing, it gives you a great tool to cover huge parts of your API code.

What many people do not know is that Representable is also useful when transforming objects to other objects. This is particularly marvelous when decorating object graphs or customizing objects.

For example, we do that a lot in Reform, since a form object is mainly data transformations and pushing objects in different representations back and fourth.

Transforming Objects.

In versions before 1.2.6, we used to first transform the source object to a hash, and then apply that hash to the target object.

Consider the following object graph.

source = OpenStruct.new(
  name: "30 Years Live", 
  songs: [
    OpenStruct.new(id: 1, title: "Dear Beloved"), 
    OpenStruct.new(id: 2, title: "Fuck Armageddon")
])

For simplicity, I’m using OpenStructs to implement an album composed of songs. Let’s assume we need to translate this object graph into objects that look like this.

Album = Struct.new(:name, :songs)
Song = Struct.new(:title)

It is needless to say that the target classes could be ActiveRecord derivates or whatever you fancy. Here, Struct will help us to focus on what we need to do: Transform a graph of OpenStructs into Structs.

The Clumsy Way. Oh, and Slow.

In older versions, transformation to a differing object graph worked via an intermediate hash, using a representer.

class AlbumRepresenter < Representable::Decorator
  include Representable::Hash
 
  property :name
  collection :songs, class: Song do
    property :title
  end
end

Here’s the transformation.

target = Album.new
hash   = AlbumRepresenter.new(source).to_hash
AlbumRepresenter.new(target).from_hash(hash)

This will transform the OpenStruct graph into a tree of an Album instance holding two Song instances. Of course, this transformation doesn’t really make sense, but I hope to proof my point: This is incredible clumsy and slow, as this need two representers.

Representable::Object Helps!

I am surprised that I didn’t come up with that solution earlier, but here’s how it works now.

target = Album.new
AlbumRepresenter.new(target).from_object(source)

Just one call to from_object is required. Speaking of requirements: here’s how the representer changes.

require "representable/object"
 
class AlbumRepresenter < Representable::Decorator
  include Representable::Object
 
  # .. the same as above.
end

Note how the Hash engine of Representable got replaced with Object. And now, check out the simple transformation.

When running the representer, the exact same thing as above will happen, resulting in a target object graph as follows.

#<struct Album
 name="30 Years Live",
 songs=
  [#<struct Song title="Dear Beloved">,
   #<struct Song title="Fuck Armageddon">]>

The representer will simply traverse the source object (the OpenStructs), deserialize necessary composite objects and map (“copy”) properties to the target instance.

This Is Not The End.

My example was simple, probably too simple, but please keep in mind that the transformation can use all kinds of options as :instance, renaming properties using :as and allows an unlimited number of nestings. Also, runtime options like :exclude and friends will work as well.

The new Object representer engine’s a great tool and I started using it heavily in Reform and Disposable as it simplifies the code and speeds up about 20%. If you want to play with it, here’s the above code.
:exclude

Happy new year, dear friends! I hope your dreams come true this year, finally! Releasing Roar 1.0 has been a dream of mine and that’s how we kick off 2015.

Shorter Namespaces.

Releasing Roar 1.0 was a good occassion to introduce brief namespaces and constant names in Roar. Unsure about what Roar’s gonna be, I initially started with constant names such as Roar::Representer::Feature::Hypermedia a few years back.

Major version bumps allow you to change things without deprecating it – yay! Since the concept of a representer is found throughout the Roar gem we ditched the Representer namespace. The same happened to Feature. It is nonsense to prefix a feature module – modules are always features.

JSON-API Support.

Roar 1.0 comes with full JSON-API support. This is both, rendering and parsing JSON-API documents. Roar is the only gem presently that does both ways – all other gems are either pure client gems or can only render JSON-API documents, like ActiveModel::Serializer (AMS).

I am mentioning that because Roar constantly gets compared to AMS. And this is simply wrong. AMS is nothing more but an object-oriented rendering engine. Roar is a document framework that uses the same definition to render and to deserialise documents for further processing. This is a bit like comparing Haml with the JSON gem.

A Minimal JSON-API Representer.

Let’s start with the simplest representer for a JSON-API document. In this example, I use a Roar decorator, nevertheless, you are free to use a module representer in case you fancy the extend approach.

class SongDecorator < Roar::Decorator
  include Roar::JSON::JSONAPI
  type :songs
 
  property :id
end

By mixing JSONAPI into the representer you import semantics and DSL for this hypermedia format.

Rendering JSON-API.

Given you had a Song instance at hand, here’s how you render a JSON-API document.

SongDecorator.prepare(song).to_json
#=> "{"songs":{"id":"1"}}"

This is a singular document, representing an individual entity. JSON-API differentiates between singular and collection documents.

Personally, I dislike this decision as it makes it harder for both server and clients to handle documents. They always have to check whether it’s a singular or a collection document.

Anyway, here’s how you would render a collection of songs.

songs = [song, song2]
SongDecorator.for_collection.prepare(songs).to_json
#=> "{"songs":[{"id":"1"},{"id":"2"}]}"

The for_collection class method will return the collection representer. That one only accepts a collection of songs and renders a JSON array.

Parsing JSON-API.

As already noted, the reason I created Roar is to provide a framework to handle both ways of dealing with representational documents. Here’s how to parse a JSON-API document to a Ruby object.

song = Song.new
json = '{"songs":{"id":"1"}}'
 
SongRepresenter.prepare(song).from_json(json)
 
song.id #=> 1

Roar deserialises the properties back to a Ruby object. This happens by using public setter methods on the represented model, only.

The same works with a collection. Here, you need to provide a collection of new (or existing) songs to update, exactly as we did a minute ago.

Simple Attributes.

You can add as many resource attributes as you want using property.

class SongDecorator < Roar::Decorator
  #..
 
  property :id
  property :title
  property :track_number

By defining properties, Roar knows what to render and what to parse from incoming documents.

Relationship Links.

JSONAPI allows to globally link to related resources in a document. In Roar, you use link blocks to specify those relationships.

class SongDecorator < Roar::Decorator
  #..
  link "songs.album" do
    {
      type: "album",
      href: "http://example.com/albums/{songs.album}"
    }
  end

This will render global links into the document.

"songs" => {
  "id" => "1",
},
"links" => {
  "songs.album"=> {
    "href"=>"http://example.com/albums/{songs.album}", 
    "type"=>"album"
  }
},

Note that the DSL is not final, yet, as we’re still collecting user input.

To-One Relationships.

Representing associations for one object is called To-One relation in JSON-API. You can define that per document in Roar.

class SongDecorator < Roar::Decorator
  #..
  has_one :composer
  has_many :listeners

As you can see, Roar’s JSON-API implementation lets you define associations using has_one and has_many.

This will add links section to each represented object in the document.

{
  "songs" => {
  "id" => "1",
  "links" => {
    "composer"=>"10",
    "listeners"=>["8"]
  }
},

Depending on the type of association it renders an ID or an array of IDs. There is no magic to that: Roar simply calls song.composer and collects the IDs from each object.

Compound Documents.

The JSONAPI media format also allows embedding parts of other, associated resources into the document. This is called a compound document.

In Roar, the compound block acts like a sub-representer to specify the nested documents.

class SongDecorator < Roar::Decorator
  #..
 
  compound do
    property :album do
      property :title
    end
 
    collection :musicians do
      property :name
    end
  end

Again, this is pure Roar DSL and works exactly the way you nest representers in Roar/Representable using inline representers.

This renders associated documents into the linked section.

"songs" => {
  "id" => "1",
 
  "linked" => {
    "album"=> [{"title"=>"Eruption"}],
    "musicians"=> [
      {"name"=>"Eddie Van Halen"},
      {"name"=>"Greg Howe"}
     ]
  }
}

The implementation of JSONAPI in Roar is relatively simple and reuses a lot of existing mechanisms.

More Features? Of course!

The JSONAPI module also allows adding meta data and more. Check out the README for the complete DSL.

The way media formats are supported in Roar makes it straight-forward to try out different specifications without too much change in the representer. We support HAL-JSON and JSON-API out-of-the-box.

Please give the new JSON-API implementation a go and let me know what else you need!

Reform 1.2

Today I released Reform 1.2 – the bug-free edition™. I am extremely excited about it. This release doesn’t break any existing code (hence the minor bump) but brings a bunch of new features that I already use all across my apps.

As always, a complete list of the CHANGES can be found in the CHANGES file.

Non-ActiveModel Models

Reform has been supporting non-ActiveRecord objects (“POROs”) ever since: this was one of the reasons we wrote it. However, in Rails apps we automatically included methods to help the Rails form builder infer field types. This didn’t go well if your model wasn’t an ActiveRecord one.

To allow form helpers like simple_form to access your form’s model for type interrogation you need to activate it manually now.

class SongForm < Reform::Form
  include ModelReflections

Now a TEXT column will be displayed as a textarea, and so on.

Skipping Deserialisation

Often a form needs to skip or ignore data from an incoming submission. For example, when all fields of a nested property are empty, you don’t want to process this item. In Rails, this is known as reject_if in the nested-attributes code.

You can do so now in Reform using :skip_if.

class SongForm < Reform::Form
  property :title, 
    skip_if: lambda { |v, *| v == "Bad song" }

Now, consider the following validation.

form.validate(title: "Bad song")

Given that very parameter hash, this ignores the incoming title property as if it wasn’t present in the hash. The title is not updated on the form or model, later.

This works for both properties and nested forms.

To ignore blank nested forms you can use a macro we provide.

class SongForm < Reform::Form
  property :band, skip_if: :all_blank do
    property :name
    property :label
  end

This does exactly what you think it does! And of course, this works with collections as well.

Dirty Tracker!

In older versions, when syncing the form to the model, Reform used to call every setter for every property – regardless whether they’d actually changed or not. Now, Reform tracks what field has changed.

form.validate(title: "Violins")
 
form.changed?(:title) #=> true

In order to only update those fields that have changed you need to include Sync::SkipUnchanged into your form.

class SongForm < Reform::Form
  include Sync::SkipUnchanged
 
  property :title
  property :genre

In #sync, the #title= writer on the model is only called when the title has actually changed. This is extremely helpful for doing advanced form processing with file uploads, etc.

Performance Speed-Up

I could achieve a speed-up of about 85% with an extremely simple trick. Reform internally uses Representable for all kinds of data transformations. It used to create and configure arbitrary representer classes at run-time, which was costly. Those representer classes are now computed once on the form class level resulting in an incredible speed-up and probably less memory footprint.

Dynamic Syncing

The way #sync pushes all attributes back to the model is very generic. Generic code is a good thing. Generic code gets even better when it’s easily extendible. And this is what the new #sync API offers you.

form.sync(
  title: lambda { |v, opts| form.model.title = sanitize(value) }
)

You can now add a lambda per property which is then called when syncing only if that property was changed. As you can see, the block is called in the caller’s context and allows you to access the form itself, the environment, the processed value and more.

This is a great way to dynamically process a property at run-time.

Dynamic Saving

While the dynamic syncing might smash many problems you sometimes need to run code after the model was saved, for instance, to include the model’s ID in your workflow.

No problem with Reform 1.2!

form.save(
  image: lambda { |v, opts| upload!(v, form.model.id) }
)

Opening Reform’s API for those two steps makes the form object a perfect extension for a Trailblazer operation and allows separating form logic from persistence – one of the key concepts of Trailblazer.

The ActiveForm Drama

Before letting you run to try out all those new things, I quickly want to comment on the “ActiveForm drama” that got averted before it even took off.

Rails recently included active_form into their main repository. I got a bit offended by that since ActiveForm clearly started as a clone of Reform and then got “rebranded” or “re-implemented”. I explicitly had to remind Rails core to add an attribution to this project which copies my README almost identically, and also recycles my DSL, API and all concepts like nesting or defining form fields explicitly instead of guessing.

While I’m cool with that in general, I’m not entirely cool when Rails does that. Those who’ve been with me for the last decade might know why.

Anyway, the Rails core team acted exemplary, apologised for the lack of attributions, removed the debatable repository from the core account, and more.

I’m not sure what they’re gonna do but my blood temperature is back to semi-hot and I don’t mind ActiveForm anymore. At least, the concept of “forms” has finally arrived in Rails core!

Also, I am pretty impressed by the Rails community and how this “accident” was handled on both sides. <3

Representable 2.1 has been out for a few weeks now. I haven’t had time to blog due to my book preview release but here we go.

Faster!

I spend days on figuring out how to speed up Representable. Surprisingly, by removing def_delegators we could improve parsing and rendering performance more than 100%! Instead of using the Ruby delegator, we define the methods manually.

An attempt to introduce binding caches turned out to bring overly complex changes to representable without a note-worthy performance improvement. That’s why it didn’t make it into this release.

Skip Parsing.

An extremely cool new feature is that you can skip parsing for properties and items in collections.

property :title, skip_parse: 
  lambda { |fragment, options| fragment.blank? }

Note how both the fragment and options from the #from_json call are passed into the lambda. If the lambda evaluates to true the property won’t be parsed, as if it hadn’t been present in the incoming document.

The same works with collections: here, the lambda is run for every item. The item is not parsed and not deserialised when the filter applies.

As a side note, this is used in Reform 1.2 to implement skip_if: :all_blank semantics which helps ignoring empty forms.

Skip Rendering.

The same also works the other way round.

property :title, skip_render: 
  lambda { |object, options| options[:skip_title] }

Accessing the options hash means you can implement run-time filters when rendering documents. For instance, this could be used to exclude certain properties from a JSON document when access rights are restricted.

song.to_json(skip_title: Permissions.allow?(:title))

I personally can’t wait to play with this in Trailblazer to allow more dynamic representers in combination with authorization gems.

Debugging – Made Easy!

Even though Representable is very lean and structured it can be hard to understand what is happening in a 5-level nested model. We added Representable::Debug to make just that easier.

Usually, you would inject the debugger into a certain object.

song.extend(SongRepresenter).
     extend(Representable::Debug).
     from_json("..")

Representable will then start to output what is going on. Feel free to add messages: the debugging simply happens by overriding the original methods, collecting messages and then returning to the actual method.

I hope this will help you writing awesome stuff!

Paperdragon is another gem for image processing in Ruby. It has a very explicit DSL that gives you full control about what is going on when.

Now, please don’t stop reading. I know exactly what you’re thinking: “We got Paperclip, we got CarrierWave, and there’s also Dragonfly. Why the hell another gem?”.

Alternative To What?

First of all: Paperdragon uses Dragonfly under the hood. It’s a great gem. However, I couldn’t get the model layer to do what I want.

Then, Paperclip is extremely helpful. This gem is perfect if you don’t need any fine tuning. It allows you to quickly declare formats, cropping, etc. with a nice configuration DSL. What then happens when uploading is not your business.

CarrierWave almost does what I wanted. But still, I was missing the ability to take control myself because I’m a control freak.

Also, all the other gems hook into ActiveRecord’s callbacks. I think the concept of AR’s callbacks is one of the worst things I’ve seen with Rails. Not that those gems use callbacks in the wrong way – the way other people use callbacks makes me mad.

Take Control Of Your Images!

Ok control freaks, here we go. Let’s look at how Paperdragon does it.

class User < ActiveRecord::Base
  include Paperdragon::Model
 
  processable :image
  serialize :image_meta_data
end

Oh, paperdragon doesn’t know anything about ActiveRecord or whatever ORM you use. You could use paperdragon in a PORO and it wouldn’t know it.

By calling processable paperdragon will add a reader method #image to your model that allows accessing the processing API – nothing else happens!

A second requirement is to provide accessors for a field named image_meta_data. It uses this field to store, well, meta data like the images’ paths as a hash. I use serialize from ActiveRecord. And of course, I added a TEXT column with the same name in my users table.

We’ll see how these two play together in a minute.

Uploading

Now, what’s happening when you upload an image?

Processing and storing an uploaded image is an explicit step – you have to code it! That’s right. You wanted control, now you got it.

This code usually goes to a separate class or an Operation in Trailblazer, don’t leave it in the controller if you don’t have to.

def create
  file = params.delete(:image)
  user = User.create(params) # this is your code.
 
  # upload code:
  user.image(file) do |v|
    v.process!(:original)                            
    v.process!(:thumb)   { |job| job.thumb!("75x75#") } 
    v.process!(:public)  { |job| job.watermark! } 
  end
 
  user.save
end

Whoooa, slow down. What’s all that?

file = params.delete(:image)
user = User.create(params) # this is your code.

Well, the first lines is Rails code: We simply create a user object. This is where paperclip and friends would hook in an start their magic. Paperdragon hates magic, that’s why I delete the uploaded file from params.

Explicit Processing.

After that setup, we finally get to process images.

user.image(file) do |v|
  v.process!(:original)                            
  v.process!(:thumb)   { |job| job.thumb!("75x75#") } 
  v.process!(:public)  { |job| job.watermark! } 
end

Do I really need to explain that? Ok.

I use the #image method we talked about earlier and pass the uploaded file into it.

In the block, I can create different versions of the image. It is an convention to name the unprocessed file :original – everything else is up to you!

The job objects are actually Dragonfly::Job instances and give you the entire API for processing, converting, cropping, resizing, analysing images. It’s insane and beautiful!

An important thing here is: once you call process! it happens. It’s your decision whether the entire block goes to a background job, if you want to crop the thumb right away, do the rest later, and so on.

Make It Persistent!

After the block is run, and you hopefully created some images, the last line seems odd.

user.save

This is to save the metadata! Remember, we have that field image_meta_data in the User model. And that very field gets populated with data when processing has finished.

Let’s check out this field’s content before we call #save and after the processing block.

user.image(file) do |v|
  v.process! #...
end
 
user.image_meta_data 
#=> {original: {uid:   "original-logo.jpg", 
#               width: 240, height: 800},
#    thumb:    {uid:   "thumb-logo.jpg", 
#               width: 48, height: 48},
#   ..and so on..

After processing, paperdragon automatically updates this field, so it can find the images later.

This is a fundamentally different concept to paperclip, which uses 5 or more columns of the model to generate the address at run-time.

While this might save some disk space, it is a very dangerous concept: If any of these values change by only one byte, you’re gonna lose your image since the computed address will be wrong. Paperdragon goes the explicit way and saves the address in the metadata hash.

Rendering Images!

All this work is rewarded when rendering a beautifully cropped thumbnail.

user.image[:thumb].url

That’s ready to be used in an #img_tag. Not too hard, right?

Reprocessing.

This was uploading. Paperdragon also makes it extremely explicitly simple to reprocess image version. For instance, let’s assume the thumb had been re-cropped via the UI.

user.image do |v|
  v.reprocess!(:thumb, Time.now) do |job| 
    job.thumb!("#{params[:w]}x#{params[:h]}#") 
  end
end

Reprocessing has a similar semantic. Here, you don’t need to pass in a file as we already got it stored. Furthermore, I call #reprocess! which requires a second parameter: a fingerprint!

That’s right, paperdragon automatically adds a fingerprint to the file name when reprocessing (if you want it).

Ehm, And.. What Else?

I am getting bored as I am replicating paperdragon’s README which will tell you about all the other great features such as renaming images, deleting, S3 support, custom UIDs, paperclip compatibility, and more.

Check It Out!

Paperdragon‘s API requires you to do all that work. In turn, it gives you control.

We are very happy with it, our site literally processes thousands of photos every day and paperdragon gives us exactly the amount of control we need, along with an abstraction that is not painful, yet.

BTW: The best Origami paperdragon image in the comments wins a free installation!

The new Reform 1.1 release brings several nice improvements and lots of fixes to the form object gem. This release is just a transitional one, the 2.0 version is gonna move huge chunks of code into a separate gem. Let me talk about this in a minute.

Non-Intrusive Validation.

Reform comes with a mechanism to populate your object graph by deserialising the incoming form data to objects.

class AlbumForm < Reform::Form
  collection :songs, populate_if_empty: Song do
    property :title
  end
end

This would create Song instances where they were missing in the album.

AlbumForm.new(album).validate(
  "songs" => [
    {"title": "Eat Your Words"}
  ]
)

In former versions of Reform, just by calling #validate Reform would already attach those new songs to the album model – by pushing them via songs[]=.

This created confusion. And is different now. The built-in populators do not touch the model at all. This only happens when saving or syncing the form.

Coercion Simplified.

Reform allows you to use virtus coercion.

class AlbumForm < Reform::Form
  property :released_at, type: DateTime
end

From 1.1 onwards, coercion only happens in #validate – any other API method will not trigger coercion. This makes the workflow predictable as we had problems with coercion happening where it shouldn’t.

Manual Coercion.

You can also implement your own filtering by overriding the setter for a property.

class AlbumForm < Reform::Form
  property :released_at
 
  def released_at=(v)
    super DateTime.parse(v)
  end
end

Note that you can use super to call the original setter but still provide your own code for filtering or sanitising or whatever fun stuff you’re after.

Overriding In Nested Forms.

This brings me to the next improvement. The above example only worked for top-level properties. That sucked and is fixed now, allowing you stuff as follows.

class AlbumForm < Reform::Form
  collection :songs do
    property :written_at
 
    def written_at=(v)
      super DateTime.parse(v)
    end
  end
end

Methods defined in nested forms actually extend that nested form class – as it should be!

Forms In Modules.

This is my favourite improvement as it maximises the reusability of forms: you can now define forms along with properties, validations and accessors/helpers in modules!

module SongsForm
  include Reform::Form::Module
 
  collection :songs
    property :title
    validates :title,  presence: true
  end
end

This module can now be included into real forms.

class AlbumForm < Reform::Form
  property :title
 
  include SongsForm
end

That should help to keep forms DRY, as it is a common pattern to have several different forms for the same model with lots of shared functionality.

Inheritance Improved.

Reform 1.1 uses representable 2.0 internally for all kinds of mappings and declarations. This actually allows you to “fine-tune” forms and overriding or extending properties that were inherited.

class AlbumForm < Reform::Form
  property :title
 
  include SongsForm
 
  collection :songs, inherit: true do
    property :artist
  end
end

This will extend the existing, inherited songs form and add the artist property. Read the docs for in-depth information about this or sign up for my upcoming book which discusses this pattern in detail.

Validations From Models.

While Reform/Trailblazer encourages you to have empty models that only configure your persistence layer, Reform now allows copying validations from models. This way you can quickly set up forms without having to write redundant validations.

class SongForm < Reform::Form
  property :title
 
  extend ActiveModel::ModelValidations
  copy_validations_from Song
end

Thanks to Cameron Martin for his excellent work.

Deserialise JSON.

A simple change in Reform now allows forms to deserialise JSON, XML and YAML besides the original support for hashes. While this might sound weird, this little improvement actually allows forms to become part of your HTTP API.

This is an integral part of Trailblazer: Here, every domain action is encapsulated in a so called “Operation” which internally uses a form object to deserialise incoming data, setup the object graph and validate the application state.

By making forms do JSON and friends, too, they can be used for normal web forms, console/model API and HTTP APIs, which is pretty awesome.

Again, this is all in my book but you can have a sneak peek in the Traiblazer example app gemgem – thanks to @GoGoGarrett for that name, BTW :)

class AlbumForm < Reform::Form
  representer_class.send :include, Representable::JSON
 
  def deserialize_method
    :from_json
  end
 
  property :title
 
  include SongsForm
 
  collection :songs, inherit: true do
    property :artist
  end
end

This rather clumsy extension (to be improved!) allows to call #validate with JSON.

AlbumForm.new(album).
  validate('"songs": [{"title": "Eat ..')

The API is not yet finalised, I just wanted to give you an out-sight on where this is going.

Reform 2.0 coming.

Speaking of out-sights: Reform 2.0 is already on its way. A major improvement here will be to move all the heavy model lifting (populating, syncing, saving, etc) into disposable which will be a model abstraction layer in Trailblazer (or without Trb).

Use it, and let me know what you think!

Is it just the crude title or is it true…? What makes Helpers Are Shit my most popular post ever? I felt like I have to write a follow-up now that Cells view models are replacing more and more helpers in many production apps out there.

Right, the last post is from 2011, that is old. What’s bad about this is that the write-up is still 100% accurate about the mess helpers and partials create in Rails apps. Nothing has changed in the Rails view layers for many years.

Helpers Are Shit.

The way helpers are being used in Rails encourages users to write “script-like” code using global variables and procedural functions in an environment that calls itself “object-oriented”.

Whenever I am refactoring complex Rails apps, the view layer is the worst part with hundreds of redundant helper methods accessing instance variables, calling other helpers, passing around models and options, and rendering random partials from a global directory.

Again, this mess is a result from the lack of a view abstraction layer in Rails: Helper invocations and partial rendering all happens in one monolithic ActionView instance – this is failure by design.

View Models.

Cells “view models” allow you to replace helpers entirely and forever. Instead of a global pile of mud you encapsulate parts of your view into a class. A class that can render templates, can inherit or mixin other methods, allows view inheritance, if needed.

This is faster than what you had before: In a view model, no data is copied between controller and view – the view model is the view context. Let me demonstrate that.

I will use the term “view model” and “cell” interchangeably (yepp, I had to look up that word).

Encapsulation Via Classes.

Encapsulating view logic happens in cell classes.

class CommentCell < Cell::ViewModel
  def show
    render
  end
end

Think of a cell as you think of a Rails controller (without all the HTTP overhead, of course). Per design, a cell has one public method #show. This is a convention. You can expose as many methods as you like from that cell. More on that later.

Where Is The Model?

I’ll walk you through that now. First, we need to render that cell somewhere in our app.

.sidebar
  = cell(:comment, Comment.last).show

Suppose this is somewhere in an application layout (it could be any view or in a controller). This will instantiate a CommentCell and execute the show method. This is a bit like calling a helper.

Note that I pass in a Comment model instance as the 2nd argument. This is the cell’s “model”, making the former a genuine view model™.

Encapsulated Rendering.

Let’s have a look again at the cell’s show method.

class CommentCell < Cell::ViewModel
  def show
    render
  end
end

Once the show method is called, it hits render. That will simply invoke the cell’s rendering as known from a Rails controller. With one exception: the view is located in a separate directory. Here’s how a cell directory looks.

app
├── cells
│   ├── comment_cell.rb
│   ├── comment
│   │   ├── show.haml

This is the old layout soon to be superceded by the Trailblazer-style “concept layout”. That doesn’t matter right now.

Good Old Views.

Now, let me show you how that show.haml could look like.

%h1 Comment
 
= model.body
By 
= link_to model.author.name, model.author

Two things.

  1. The cell’s model (the comment itself) is accessible via the #model method. Wow.
  2. You can still use helpers in a view. In this example, I use link_to.

No Code In My View!

We’ve learned that logic in views is bad. And this is exactly where view models start to be fun to work with.

class CommentCell < Cell::ViewModel
  def show
    render
  end
 
private
 
  def body
    model.body
  end
end

Remember: as the view model is the view context itself, any method called in the view will be simply called on the view model. That reduces complexity in the view a bit.

%h1 Comment
 
= body
By 
= link_to model.author.name, model.author

Embracing Helpers.

Still hating the link line. Why not move that into the cell, too?

class CommentCell < Cell::ViewModel
  def show
    render
  end
 
private
 
  def body
    model.body
  end
 
  def author_link
    link_to model.author.name, model.author
  end
end

Now, what? We can call helpers in the view model? Seerriously?

Well, I never said that helpers are a bad thing. They’re just shit when used to “encapsulate”. I personally love link_to, sanitize and form_for (sometimes). They save me a lot of work. I just don’t like the fact they all play together in a big monolithic class.

Here, the link_to can’t mess up anything as it is running in a limited scope – the cell instance.

Logicless Views.

The second refactoring reduces our view to a dumb template, as it should be.

%h1 Comment
 
= body
By 
= author_link

I love it already. What’s next?

Simple Decorating.

The task of delegating methods to the model is so tedious that it is built into cells. We can get rid of the body method.

class CommentCell < Cell::ViewModel
  property :body
  property :author
 
  def show
    render
  end
 
private
 
  def author_link
    link_to author.name, author
  end
end

Declaring body and author as propertys automatically delegates to the model, making it a no-brainer. It also states your dependencies to the model in a very clear way. Hooray to explicit code!

Object-Oriented Helpers.

What if we had to customise the body method, e.g. apply a markdown filter on the content?

class CommentCell < Cell::ViewModel
  include MarkdownHelper  
 
  property :body
  # ...
 
private
 
  def body
    markdown super
  end
end

First, I include the magic markdown helper to import the markdown method. Secondly, I can override the body method and call super to access the original property. Object-orientation back in the view land!

More Helpers!

A cell is not limited to one public method. It’s good to have only one but sometimes it comes in handy to not having to introduce a new class for every “helper”.

class CommentCell < Cell::ViewModel
  # ...
 
  def show
    render
  end
 
  def related
    render
  end
 
private
  # ...

Here, I add a second public method related. This will render another view, the related.haml. Dump it into the cell’s directory and you’re good to go.

app
├── cells
│   ├── comment_cell.rb
│   ├── comment
│   │   ├── show.haml
│   │   ├── related.haml

You can now use this “helper” anywhere you want.

= cell(:comment, Comment.last).related

Isn’t that nice? We encapsulated a huge part of a mess into a clean, reusable component. This feels good.

Don’t Stop Me Now!

I want to talk more about the benefits of cells, like view inheritance, testability, reusing components, hooking them directly to routes, sharing decorators between cells and other layers, but.. I’m writing this blog post at work and feel like I should stop now.

And, did you know? Recent surveys have concluded that 79.3% of all cells users did not regret their choice and can’t do without cells anymore.

Let me know if you have any questions.

The kaminari gem is a project to simplify pagination – often used in Rails apps. Cells provides view components for Rails. Those two just got married and their child is called kaminari-cells. Yay!!!

Kami..what?

Kaminari adds pagination logic to finders – it works with all kinds of ORMs, e.g. in ActiveRecord. It also gives us a bunch of helpers to render pagination links.

It is extremely popular as it doesn’t pollute objects with pagination logic.

Typically, a controller, an operation or a model will accumulate models to display.

@users = User.order(:name).page params[:page]

Later, in a view, after rendering the rows itself you want to display a link to jump to the next or previous page.

= paginate @users

A really simple and cool thing.

Pagination In Cells.

As kaminari uses global partials to render the pagination links, we had to make it work with cells, which usually does not use global partials. Thanks to the fine kaminari team, this was implemented, tested and merged within hours.

All you need to do to make the paginate helper work in a cell view is add the gem to your Gemfile.

gem "kaminari-cells"

And then extend the class.

class CommentsCell < Cell::Rails
  include Kaminari::Cells

This works for both conventional cells and view models.

Representable 2.0 comes with a bunch of cool new features. I’d like to thank the community for coming up with ideas, working with me on features and trying out new stuff. Speaking of new stuff – have you seen my favicon? It’s new, and I’m a bit proud of it.

Let’s quickly go through the changes.

Changes?

Well, if you’ve been a cautious person and were only using representable’s public API, nothing will break! Even when you’re one of the “power users”, you shouldn’t have too much trouble getting up-to-date, again.

Partial Inheritance.

Inheriting properties from other representer has been working fine for a long time, either by including modules into other modules or decorator classes, or by deriving decorators.

It is now possible to partially override inherited properties.

Suppose we have the following base representer (this could be a module or a class, doesn’t matter). For simplicity, I leave away the noisy includes.

module SongRepresenter
  property :title, as: :Title
 
  property :band do
    property :name
  end
end

Very simple. Now imagine a HitRepresenter needed to inherit all the above plus add a property to the inline band block. This is where :inherit enters the stage.

module HitRepresenter
  include SongRepresenter
 
  property :band, inherit: true do
    property :location
  end

This will add the location property to the band’s representer without destroying the original representer.

You can also use this directive with scalar properties that don’t have a block. This will add the options you provide.

property :title, inherit: true, type: String

The coercion option :type will be added to the existing options, preserving :as and whatever else you specified.

Having this new inheritance “fine tuning” makes representable really powerful and maximises reusability of representers across your project.

Modules In Decorators.

One thing I found not working and fixed in representable 2.0 was when including modules with inline representers into decorators – this works all fine now.

class SongDecorator
  include SongRepresenter
 
  property :label
end

This will work as expected and inherit all properties from SongRepresenter into the decorator – and add the label.

Automatic Collections.

Many users have been asking for this for a long time, and @timoschilling was the power user who submitted a draft more than a year ago! Sorry for the delay. I was busy drinking Australian Beer™.

Representable comes with a feature called lonely collection which allows you to represent an array of objects.

module SongsRepresenter
  include Representable::JSON::Collection
 
  items extend: SongRepresenter, class: Song
end

As you can see, every item will be represented (rendered or parsed) with the specified SongRepresenter (which could also be inline, BTW).

[song, song].extend(SongsRepresenter).to_json

Now, this is explicit and needs to be defined for every collection. I personally am a big fan of explicit code, but lots of users don’t want that boilerplate code.

That’s why you can now use ::for_collection to let representable create this lovely lonely collection for you.

module SongRepresenter
  property :title
end

Assuming you use this simple singular representer to render and parse songs, here’s how you’d render collections with it.

[song, song].extend(SongRepresenter.for_collection).
  to_json

Note how I use the singular representer! Representable will internally create a lonely collection representer and render the collection.

For parsing, it needs to know a bit know, e.g. the class to deserialize the songs to. You need to specify that in the singular representer, again.

module SongRepresenter
  property :title
 
  collection_representer class: Song
end

::collection_representer lets you configure the – surprise! – collection representer. Everything you pass to this method will be forwarded to the ::items call when creating the collection representer.

[].extend(SongRepresenter.for_collection).
  from_json("[{title: ..]")

Automaticer Collections.

If that’s still too much code for you, go ::represent fo’ real!!

SongRepresenter.represent(song).to_json       #=> 1
SongRepresenter.represent([song, ..]).to_json #=> n

This method figures out if it’s working with an array or a singular object. It’s a matter of days until this will make its way into Roar and roar-rails and simplify your user code there.

Filters!

If you ever needed to massage a string before it gets rendered, or a fragment after it got parsed… you’re lucky. Representable now gives you :render_filter and :parse_filter.

property :title, render_filter: 
  lambda { |value, doc, *args| value.markdown }

This is run right before the fragment is inserted into the document.

property :title, parse_filter: 
  lambda { |value, doc, *args| Markdown.parse(value) }

And this lambda is invoked just after the fragment got picked from the incoming document when parsing.

In case you need multiple filters: those two options are implemented using Representable::Pipeline, you can add lambdas later, the results will be passed on from filter to filter.

Apparently, this feature already made its way into grape-roar.

Ditch Lambdas For Callable.

I always disliked the verbose, clumsy lambda syntax in my properties. Representable 2.0 lets you specify “callable” objects in favour of lambdas.

property :title, render_filter: Sanitizer.new

Now, how does it know that Sanitizer instance is invokable? Here’s the trick.

class Sanitizer
  include Uber::Callable
 
  def call(representer, fragment, doc, *args)
    fragment.sanitize
  end
end

All the object needs is a #call method that gets – admittedly – a bunch of parameters. In that, you can do whatever you need. Including Uber::Callable marks this instance as a callable one – no magic here.

I Need More Data!

Another new option helps you retrieving all the possible information you might need in your lambdas (or, cooler, callable objects). Let’s say you need to know the property options, whether :extend is set, or not. :pass_options can help.

property :title, 
  pass_options: true,
  parse_filter: 
    lambda { |value, doc, options|
      options.binding[:extend]
      options.user_options
      options.represented
    }

The options object keeps references to all stakeholder objects involved at that point of representing. This :pass_options is definitely an advanced option, but some peeps might find it helpful. I use it all across my gems.

Enjoy your day and keep smiling!