Uncategorized

On Rails 5, Presenters And Form Objects.

Thursday, May 21st, 2015

My original plan to not blog about conceptual problems in Rails for the next months has failed.

Too many discussions about presenters, decorators, object-oriented helpers and “oh-so-awesome-and-new” form objects I had to overhear. With Aaron’s great keynote and many comments on the aforementioned concepts, I feel the urge to clarify what’s a presenter, a view model, and a form object.

Presenters

I completely agree with tenderlove when he says there doesn’t have to be a presenter library. Presenters (or decorators?) are usually composition objects that add presentation logic to attributes.

However, what many people ask for is the ability to map widgets, or fragments, or parts of their UI, to something in Rails. And this something has turned out to be a mix of controller code, before_filter, partials and helpers. And many people are not happy with this, as their widget is not encapsulated and not reusable across their app.

To summarize: what people want in order to implement widgets is

  • A place in the file system for code and templates.
  • An asset where to put that Ruby code.
  • The ability to render partials in order to present their widget.

Especially the latter one is important and is what makes the difference between presenters and widgets: I want to render templates in order to present an arbitrary object in my UI. And I don’t want to hack ActionView in any way to achieve that.

View Models

This was my motivation to write Cells a good while ago. Instead of cluttering widget logic across the entire framework, there’s a new abstraction layer to solve this. It gives you a view model class where you put presentation logic, but it also lets you render templates.

However, these are not global templates but views that sit in an encapsulated directory, just like the view model’s code is isolated in the cell and doesn’t have global access. Likewise, JavaScript and CSS code can be bundled with the cell. This makes a cell reusable across many controllers, or even apps.

app
├── cells
│   ├── comment
│   │   ├── cell.rb
│   │   ├── show.haml
│   │   ├── grid.haml
│   │   ├── comment.coffee

I am not gonna argue any more whether or not you need Cells. Some people like it, some prefer using POROs and hack Rails’ rendering into that object to achieve what Cells does.

My point is that Cells view models give developers a defined structure and standard how to implement view fragments (not to speak about how Cells handles view inheritance, polymorphic views, caching, and more, hahahaha).

So next time you talk about presenters, ask yourself: Am I talking about a strictly attributes-decorating thing? Then that’s a decorator. As soon as this involves rendering of views, you might want to checkout view models.

Form Objects

The other thing I need to clarify is form objects.

We all know that the way validations are handled in Rails models is a mess. It breaks down as soon as you need to use a model in two different contexts, for two different forms. Everyone reading this blog post has felt the pain with accepts_nested_attributes, which is supposed to handle deserialization of nested forms.

And this brings me to the point of this section. One job of a form object is validating an object graph (e.g. an album composed of songs with artists) and collect validation errors in the top object.

The other job is the deserialization of the incoming hash. And this is completely underestimated by Rails core. Deserialization is the actual problem of forms. Validating and bubbling up errors is easy.

How do I parse a hash into an object? Where do I attach this object? Do I create a new object for that hash fragment, or do I need to find an associated object in the database? How do I handle additional semantics like deleting objects and save? And, how do I prevent the persistence layer to get involved until validation is done?

Reform

This is the real issue a form object (the way we expect it) has to solve. Again, I’d like to point you to another gem called Reform. In Reform, a separate class takes care of that. You define validations and properties in a new class.

class AlbumForm < Reform::Form
  property :name
  validates :name, presence: true
 
  collection :songs do
    property :title
    validates :title, presence: true
  end
end

Deserialization and validation are done with separated entities. While a representer internally takes care of deserializing the incoming hash, validation is handled by the form itself. Usually, you don’t have to worry about this as it happens automatically.

The upcoming Reform 2.0 is doing this in a very neat way, where you can use your Roar representer for parsing, and the Reform object for validation, making it reusable for both document APIs and UI forms. It’s possible to completely replace deserialization with your own code without losing the decoupling from the persistence that Reform gives you.

This is the result of years of work, running into problems, taking a step back, reconsidering, collecting feedback from hundreds of use cases, and so on.

Please don’t brand a form object as a validation, only. There’s more to it to solve the actual problem we have.

And, yes, ActiveForm started as a pure copy of Reform, and then got “re-implemented”. Let’s not fight.

Rails 5: Stillstand

One last thing: Rails 5 comes with a new “render anywhere” feature where ApplicationController.render lets you render partials from virtually everywhere. While this might look startling first, this is the exact wrong way to go.

A globally accessable renderer is the lowest-level tool you can give a developer. Instead of providing new, object-oriented, abstraction layers to solve problems, Rails core resists the idea of introducing new concepts for the sake of Basecamp-compati.. sorry, backward-compatibility.

The result will be render calls from models, hundreds of different “presenter” implementations across Rails projects, and confused people who don’t know where to put their code.

Conclusion

I hope I managed to point out what’s a presenter, a view model and a form object.

My message is: There is gems to help you solving a lot of problems that have been around since Rails’ inception. These solutions are mature and used in thousands of production apps. Many people have put a lot of work into them.

The fact that Rails core now, after almost 10 years, slowly starts to pick up ideas like form objects, is a good sign. However, I am skeptical if view models and real form objects will ever make it into Rails core. Luckily, we got gems to fill the gap.

Representable 2.2 Is More Than 50% Faster Than The World!

Monday, May 18th, 2015

After almost two weeks of intense hacking, I’m happy to announce Representable 2.2. This is a pure performance update. I removed some very private concepts and methods, hence the minor version bump. Anyway, with the 2.2 version you can expect a 50% and more speed-up for both rendering and parsing.

This is, for Representable itself, but also for Roar, Disposable and Reform, that all use Representable internally for data transformations.

The public API hasn’t changed at all, so you’re safe to update.

To get a quick overview about the code changes, have a look here. Right, that’s only a handful lines of code that have changed.

Profiling: How We Did It.

It all started with a benchmark my friend Max was running to render a nested object graph. Here’s the structure of the document.

class FoosRepresentation < Representable::Decorator
  include Representable::JSON
 
  collection :foos, class: Foo do
    property :value
 
    property :bar, class: Bar do
      property :value
    end
  end
end

As you can see, this is a collection of foos. Every foo contains a nested bar object with a value property.

So he set up a profiling test with 10,000 foos containing one bar each. You can find his profiling repository here. For Representable, that basically means “Iterate 10,000 foo objects and 10,000 bar objects and serialize them.”, making it 20,001 objects to represent in total.

Benchmarks, Slow.

With Representable 2.1.8, to serialize this tree took about 3.1 seconds on my work machine.

Total: 3.138545
 
 %self   total   calls    name
 30.17   0.947   20001    Module#name
  7.01   0.220   790039   Representable::Definition#[]
  5.71   0.308   460025   Representable::Binding#[]
  1.68   0.135   100009   Class#new

As you can see, Module#name is called many times, and more than 100,000 objects are instantiated.

Here’s the same benchmark for deserializing a document with 200,001 objects.

Total: 3.045645
 
 %self  calls    name
 31.78   20001   Module#name
  6.07  710029   Representable::Definition#[]
  5.48  480021   Representable::Binding#[]
  2.46  160008   Class#new

It’s interesting to observe that parsing a document into an object graph takes about the same time as rendering it. Anyway, many objects are created and lot of time is wasted

Benchmarks, Faster!

I applied some simple implementation fixes and a structural change, resulting in the following benchmarks. We’ll discuss how I achieved that in a second.

First, we ran the rendering benchmark.

Total: 1.141275
 
 %self   calls   name
  6.83   310100  Representable::Definition#[]
  2.96   50007   Uber::Options::Value#evaluate
  2.78   40001   Repr..::Deserializer::Prepare#prepare!

Wow, that’s not 3.1 but 1.1 seconds to render a deeply nested object graph. No unnecessary classes are instantiated anymore and the time-consuming Module#name call has vanished.

The exact same I could achieve for parsing time.

Total: 1.173969
 
 %self    calls    name
  7.37    330092   Representable::Definition#[]
  3.33    70007    Uber::Options::Value#evaluate
  3.27    100029   Representable::Binding#[]
  2.09    20001    Representable#representation_wrap

What I didn’t measure, yet, is the memory foot-print which should be dramatically (yes, dramatically) smaller as the amount of objects we need to parse or render object graphs has minimized. I bet you want to know now how this 50-75% speedup was possible, and here we go.

Don’t Ask When You’re Not Interested.

The first thing I tackled was to get rid of the Module#name call. This resulted from computing the wrap for every represented object that was being serialized or parsed. Every represented object. Even though most objects don’t need a wrap, and the default case is to not have wrapping.

I moved the name query into an optional block and things got faster.

Now, Module#name is only being called when we actually want to know the wrap.

Too Many Bindings.

However, this was just one step to increasing the performance.

Another issue clearly visible in the ruby-prof outputs was that we created a Binding for every object. Every represented object. Bindings are a concept in Representable that handle rendering and parsing for one property.

In case this was a nested property, this binding would create a representer instance, again, which in turn would create bindings again, for every represented property.

bindings-before-2.2

My beautiful diagram, which makes me really proud, illustrates that: Every Foo instance in the collection will create a representer and a binding instance, even though they are identical.

When writing Representable a few years ago I had a feeling that this might become a bottle-neck one day, but being focused on designing I “forgot” about it, and no one ever complained.

Reuse Bindings.

The solution is dead simple. I introduced Representable::Cached that will save the Bindings for later reuse.

By making the Binding stateless we won’t have any trouble with stale data. The binding used to save a lot of run-time information like the represented object, and more. This now has to be passed from the outside for every run, making it reusable.

I know, you love my diagrams. Check out how the object graph has changed.

bindings-in-2.2

Say we were representing one Foo object, its decorator will now cache the binding for the Bar property and reuse it. This results in a handful of objects needed to be created.

To give you some figures, in the aforementioned benchmark setup, instead of having to instantiate 200,000 bindings all we need to do is to create four! One for the collection of foos, on for value in every foo, one for a Bar object and the last to represent the value in a bar.

Cached: An Optional Feature!

While I fully trust my changes (would be bad if I wouldn’t) I decided to add this as an optional feature in 2.2. You need to activate it manually on the top-level representer.

class FoosRepresentation < Representable::Decorator
  include Representable::JSON
  feature Representable::Cached
 
  collection :foos, class: Foo do
    # and so on

It’s a bit late to mention that, but Cached only works with the Decorator representers. It also works with modules, but it will unnecessarily pollute your models. Please don’t do that.

Reusing Decorators Across Requests

An interesting new option is caching of representers between requests. This will boost up rendering and parsing documents in your API code many times – not to speak about the reduced memory footprint.

Once a representer’s done its job, it can be reused using the #update! method.

decorator = SongRepresenter.new(song)
decorator.to_json # first request.
 
decorator.update!(better_song) # second request.
decorator.to_json

This is enough to reuse a representer, even a deeply nested graph.

Once I find some time, I will implement this in roar-rails. Caching and reusing representers across requests will give a significantly performance boost for many Roar/Representable apps out there.

Using Ruby-Prof For Tests

One last thing I want to mention is how I use the ruby-prof output for tests.

RubyProf.start
  representer.to_hash
# ..
 
# a total of 4 properties in the object graph.
data.must_match "4   Representable::Binding#initialize"

What might look like a weird crazy bullshit is a fantastic way of asserting that your speed-up actually works. Since we cannot test for speed of test runs (every machine and run is different) I simply test for object creations.

In the Cached tests, I setup a simple, but complex enough object graph and render and parse it. I let ruby-prof track object creation and method invocations. Afterwards, I make sure that really only four bindings were created (and other instantiation counts, of course).

And: this works!

No. I’m not gonna use Rspec’s mocking and expectations for that. First of all, I’ve managed not to use mocking in any of my gems’ test suites and want to keep it that way. Second, the more test code I add, the more I will miss and will go wrong.

Letting a super low-level tool like ruby-prof track method calls is a bullet-proof way to test your speed-up. I love it and was surprised by its accuracy.

There Was More.

A few more improvements went into 2.2, and a lot of lookups can still be improved. I feel like I’ve talked enough for today.

Instead of throwing more words at you, I’d like to thank Max Güntner for setting up most of the benchmark code and encouraging me to work on performance. And, special thanks go out to Charlie Somerville who turned out to be a Ruby-internal-monster and was a great help in explaining the depths of the Ruby VM to me and why attr_reader is faster than your own method, and so on.

Incredibly what you still can learn after having hacked Ruby for more than 10 years.

Object-to-Object Transformations with Representable

Tuesday, May 5th, 2015

Many people use the Representable gem to render documents from Ruby objects, or to parse incoming documents into an object graph.

This is great for implementing document APIs with JSON or XML. Since Representable does both ways, rendering and deserializing, it gives you a great tool to cover huge parts of your API code.

What many people do not know is that Representable is also useful when transforming objects to other objects. This is particularly marvelous when decorating object graphs or customizing objects.

For example, we do that a lot in Reform, since a form object is mainly data transformations and pushing objects in different representations back and fourth.

Transforming Objects.

In versions before 1.2.6, we used to first transform the source object to a hash, and then apply that hash to the target object.

Consider the following object graph.

source = OpenStruct.new(
  name: "30 Years Live", 
  songs: [
    OpenStruct.new(id: 1, title: "Dear Beloved"), 
    OpenStruct.new(id: 2, title: "Fuck Armageddon")
])

For simplicity, I’m using OpenStructs to implement an album composed of songs. Let’s assume we need to translate this object graph into objects that look like this.

Album = Struct.new(:name, :songs)
Song = Struct.new(:title)

It is needless to say that the target classes could be ActiveRecord derivates or whatever you fancy. Here, Struct will help us to focus on what we need to do: Transform a graph of OpenStructs into Structs.

The Clumsy Way. Oh, and Slow.

In older versions, transformation to a differing object graph worked via an intermediate hash, using a representer.

class AlbumRepresenter < Representable::Decorator
  include Representable::Hash
 
  property :name
  collection :songs, class: Song do
    property :title
  end
end

Here’s the transformation.

target = Album.new
hash   = AlbumRepresenter.new(source).to_hash
AlbumRepresenter.new(target).from_hash(hash)

This will transform the OpenStruct graph into a tree of an Album instance holding two Song instances. Of course, this transformation doesn’t really make sense, but I hope to proof my point: This is incredible clumsy and slow, as this need two representers.

Representable::Object Helps!

I am surprised that I didn’t come up with that solution earlier, but here’s how it works now.

target = Album.new
AlbumRepresenter.new(target).from_object(source)

Just one call to from_object is required. Speaking of requirements: here’s how the representer changes.

require "representable/object"
 
class AlbumRepresenter < Representable::Decorator
  include Representable::Object
 
  # .. the same as above.
end

Note how the Hash engine of Representable got replaced with Object. And now, check out the simple transformation.

When running the representer, the exact same thing as above will happen, resulting in a target object graph as follows.

#<struct Album
 name="30 Years Live",
 songs=
  [#<struct Song title="Dear Beloved">,
   #<struct Song title="Fuck Armageddon">]>

The representer will simply traverse the source object (the OpenStructs), deserialize necessary composite objects and map (“copy”) properties to the target instance.

This Is Not The End.

My example was simple, probably too simple, but please keep in mind that the transformation can use all kinds of options as :instance, renaming properties using :as and allows an unlimited number of nestings. Also, runtime options like :exclude and friends will work as well.

The new Object representer engine’s a great tool and I started using it heavily in Reform and Disposable as it simplifies the code and speeds up about 20%. If you want to play with it, here’s the above code.
:exclude

Roar 1.0 Released With JSON-API Support!

Tuesday, January 6th, 2015

Happy new year, dear friends! I hope your dreams come true this year, finally! Releasing Roar 1.0 has been a dream of mine and that’s how we kick off 2015.

Shorter Namespaces.

Releasing Roar 1.0 was a good occassion to introduce brief namespaces and constant names in Roar. Unsure about what Roar’s gonna be, I initially started with constant names such as Roar::Representer::Feature::Hypermedia a few years back.

Major version bumps allow you to change things without deprecating it – yay! Since the concept of a representer is found throughout the Roar gem we ditched the Representer namespace. The same happened to Feature. It is nonsense to prefix a feature module – modules are always features.

JSON-API Support.

Roar 1.0 comes with full JSON-API support. This is both, rendering and parsing JSON-API documents. Roar is the only gem presently that does both ways – all other gems are either pure client gems or can only render JSON-API documents, like ActiveModel::Serializer (AMS).

I am mentioning that because Roar constantly gets compared to AMS. And this is simply wrong. AMS is nothing more but an object-oriented rendering engine. Roar is a document framework that uses the same definition to render and to deserialise documents for further processing. This is a bit like comparing Haml with the JSON gem.

A Minimal JSON-API Representer.

Let’s start with the simplest representer for a JSON-API document. In this example, I use a Roar decorator, nevertheless, you are free to use a module representer in case you fancy the extend approach.

class SongDecorator < Roar::Decorator
  include Roar::JSON::JSONAPI
  type :songs
 
  property :id
end

By mixing JSONAPI into the representer you import semantics and DSL for this hypermedia format.

Rendering JSON-API.

Given you had a Song instance at hand, here’s how you render a JSON-API document.

SongDecorator.prepare(song).to_json
#=> "{"songs":{"id":"1"}}"

This is a singular document, representing an individual entity. JSON-API differentiates between singular and collection documents.

Personally, I dislike this decision as it makes it harder for both server and clients to handle documents. They always have to check whether it’s a singular or a collection document.

Anyway, here’s how you would render a collection of songs.

songs = [song, song2]
SongDecorator.for_collection.prepare(songs).to_json
#=> "{"songs":[{"id":"1"},{"id":"2"}]}"

The for_collection class method will return the collection representer. That one only accepts a collection of songs and renders a JSON array.

Parsing JSON-API.

As already noted, the reason I created Roar is to provide a framework to handle both ways of dealing with representational documents. Here’s how to parse a JSON-API document to a Ruby object.

song = Song.new
json = '{"songs":{"id":"1"}}'
 
SongRepresenter.prepare(song).from_json(json)
 
song.id #=> 1

Roar deserialises the properties back to a Ruby object. This happens by using public setter methods on the represented model, only.

The same works with a collection. Here, you need to provide a collection of new (or existing) songs to update, exactly as we did a minute ago.

Simple Attributes.

You can add as many resource attributes as you want using property.

class SongDecorator < Roar::Decorator
  #..
 
  property :id
  property :title
  property :track_number

By defining properties, Roar knows what to render and what to parse from incoming documents.

Relationship Links.

JSONAPI allows to globally link to related resources in a document. In Roar, you use link blocks to specify those relationships.

class SongDecorator < Roar::Decorator
  #..
  link "songs.album" do
    {
      type: "album",
      href: "http://example.com/albums/{songs.album}"
    }
  end

This will render global links into the document.

"songs" => {
  "id" => "1",
},
"links" => {
  "songs.album"=> {
    "href"=>"http://example.com/albums/{songs.album}", 
    "type"=>"album"
  }
},

Note that the DSL is not final, yet, as we’re still collecting user input.

To-One Relationships.

Representing associations for one object is called To-One relation in JSON-API. You can define that per document in Roar.

class SongDecorator < Roar::Decorator
  #..
  has_one :composer
  has_many :listeners

As you can see, Roar’s JSON-API implementation lets you define associations using has_one and has_many.

This will add links section to each represented object in the document.

{
  "songs" => {
  "id" => "1",
  "links" => {
    "composer"=>"10",
    "listeners"=>["8"]
  }
},

Depending on the type of association it renders an ID or an array of IDs. There is no magic to that: Roar simply calls song.composer and collects the IDs from each object.

Compound Documents.

The JSONAPI media format also allows embedding parts of other, associated resources into the document. This is called a compound document.

In Roar, the compound block acts like a sub-representer to specify the nested documents.

class SongDecorator < Roar::Decorator
  #..
 
  compound do
    property :album do
      property :title
    end
 
    collection :musicians do
      property :name
    end
  end

Again, this is pure Roar DSL and works exactly the way you nest representers in Roar/Representable using inline representers.

This renders associated documents into the linked section.

"songs" => {
  "id" => "1",
 
  "linked" => {
    "album"=> [{"title"=>"Eruption"}],
    "musicians"=> [
      {"name"=>"Eddie Van Halen"},
      {"name"=>"Greg Howe"}
     ]
  }
}

The implementation of JSONAPI in Roar is relatively simple and reuses a lot of existing mechanisms.

More Features? Of course!

The JSONAPI module also allows adding meta data and more. Check out the README for the complete DSL.

The way media formats are supported in Roar makes it straight-forward to try out different specifications without too much change in the representer. We support HAL-JSON and JSON-API out-of-the-box.

Please give the new JSON-API implementation a go and let me know what else you need!

Reform 1.2

Friday, November 7th, 2014

Today I released Reform 1.2 – the bug-free edition™. I am extremely excited about it. This release doesn’t break any existing code (hence the minor bump) but brings a bunch of new features that I already use all across my apps.

As always, a complete list of the CHANGES can be found in the CHANGES file.

Non-ActiveModel Models

Reform has been supporting non-ActiveRecord objects (“POROs”) ever since: this was one of the reasons we wrote it. However, in Rails apps we automatically included methods to help the Rails form builder infer field types. This didn’t go well if your model wasn’t an ActiveRecord one.

To allow form helpers like simple_form to access your form’s model for type interrogation you need to activate it manually now.

class SongForm < Reform::Form
  include ModelReflections

Now a TEXT column will be displayed as a textarea, and so on.

Skipping Deserialisation

Often a form needs to skip or ignore data from an incoming submission. For example, when all fields of a nested property are empty, you don’t want to process this item. In Rails, this is known as reject_if in the nested-attributes code.

You can do so now in Reform using :skip_if.

class SongForm < Reform::Form
  property :title, 
    skip_if: lambda { |v, *| v == "Bad song" }

Now, consider the following validation.

form.validate(title: "Bad song")

Given that very parameter hash, this ignores the incoming title property as if it wasn’t present in the hash. The title is not updated on the form or model, later.

This works for both properties and nested forms.

To ignore blank nested forms you can use a macro we provide.

class SongForm < Reform::Form
  property :band, skip_if: :all_blank do
    property :name
    property :label
  end

This does exactly what you think it does! And of course, this works with collections as well.

Dirty Tracker!

In older versions, when syncing the form to the model, Reform used to call every setter for every property – regardless whether they’d actually changed or not. Now, Reform tracks what field has changed.

form.validate(title: "Violins")
 
form.changed?(:title) #=> true

In order to only update those fields that have changed you need to include Sync::SkipUnchanged into your form.

class SongForm < Reform::Form
  include Sync::SkipUnchanged
 
  property :title
  property :genre

In #sync, the #title= writer on the model is only called when the title has actually changed. This is extremely helpful for doing advanced form processing with file uploads, etc.

Performance Speed-Up

I could achieve a speed-up of about 85% with an extremely simple trick. Reform internally uses Representable for all kinds of data transformations. It used to create and configure arbitrary representer classes at run-time, which was costly. Those representer classes are now computed once on the form class level resulting in an incredible speed-up and probably less memory footprint.

Dynamic Syncing

The way #sync pushes all attributes back to the model is very generic. Generic code is a good thing. Generic code gets even better when it’s easily extendible. And this is what the new #sync API offers you.

form.sync(
  title: lambda { |v, opts| form.model.title = sanitize(value) }
)

You can now add a lambda per property which is then called when syncing only if that property was changed. As you can see, the block is called in the caller’s context and allows you to access the form itself, the environment, the processed value and more.

This is a great way to dynamically process a property at run-time.

Dynamic Saving

While the dynamic syncing might smash many problems you sometimes need to run code after the model was saved, for instance, to include the model’s ID in your workflow.

No problem with Reform 1.2!

form.save(
  image: lambda { |v, opts| upload!(v, form.model.id) }
)

Opening Reform’s API for those two steps makes the form object a perfect extension for a Trailblazer operation and allows separating form logic from persistence – one of the key concepts of Trailblazer.

The ActiveForm Drama

Before letting you run to try out all those new things, I quickly want to comment on the “ActiveForm drama” that got averted before it even took off.

Rails recently included active_form into their main repository. I got a bit offended by that since ActiveForm clearly started as a clone of Reform and then got “rebranded” or “re-implemented”. I explicitly had to remind Rails core to add an attribution to this project which copies my README almost identically, and also recycles my DSL, API and all concepts like nesting or defining form fields explicitly instead of guessing.

While I’m cool with that in general, I’m not entirely cool when Rails does that. Those who’ve been with me for the last decade might know why.

Anyway, the Rails core team acted exemplary, apologised for the lack of attributions, removed the debatable repository from the core account, and more.

I’m not sure what they’re gonna do but my blood temperature is back to semi-hot and I don’t mind ActiveForm anymore. At least, the concept of “forms” has finally arrived in Rails core!

Also, I am pretty impressed by the Rails community and how this “accident” was handled on both sides. <3

Representable 2.1 Is Faster!

Tuesday, October 28th, 2014

Representable 2.1 has been out for a few weeks now. I haven’t had time to blog due to my book preview release but here we go.

Faster!

I spend days on figuring out how to speed up Representable. Surprisingly, by removing def_delegators we could improve parsing and rendering performance more than 100%! Instead of using the Ruby delegator, we define the methods manually.

An attempt to introduce binding caches turned out to bring overly complex changes to representable without a note-worthy performance improvement. That’s why it didn’t make it into this release.

Skip Parsing.

An extremely cool new feature is that you can skip parsing for properties and items in collections.

property :title, skip_parse: 
  lambda { |fragment, options| fragment.blank? }

Note how both the fragment and options from the #from_json call are passed into the lambda. If the lambda evaluates to true the property won’t be parsed, as if it hadn’t been present in the incoming document.

The same works with collections: here, the lambda is run for every item. The item is not parsed and not deserialised when the filter applies.

As a side note, this is used in Reform 1.2 to implement skip_if: :all_blank semantics which helps ignoring empty forms.

Skip Rendering.

The same also works the other way round.

property :title, skip_render: 
  lambda { |object, options| options[:skip_title] }

Accessing the options hash means you can implement run-time filters when rendering documents. For instance, this could be used to exclude certain properties from a JSON document when access rights are restricted.

song.to_json(skip_title: Permissions.allow?(:title))

I personally can’t wait to play with this in Trailblazer to allow more dynamic representers in combination with authorization gems.

Debugging – Made Easy!

Even though Representable is very lean and structured it can be hard to understand what is happening in a 5-level nested model. We added Representable::Debug to make just that easier.

Usually, you would inject the debugger into a certain object.

song.extend(SongRepresenter).
     extend(Representable::Debug).
     from_json("..")

Representable will then start to output what is going on. Feel free to add messages: the debugging simply happens by overriding the original methods, collecting messages and then returning to the actual method.

I hope this will help you writing awesome stuff!

Paperdragon: Explicit Image Processing

Thursday, August 28th, 2014

Paperdragon is another gem for image processing in Ruby. It has a very explicit DSL that gives you full control about what is going on when.

Now, please don’t stop reading. I know exactly what you’re thinking: “We got Paperclip, we got CarrierWave, and there’s also Dragonfly. Why the hell another gem?”.

Alternative To What?

First of all: Paperdragon uses Dragonfly under the hood. It’s a great gem. However, I couldn’t get the model layer to do what I want.

Then, Paperclip is extremely helpful. This gem is perfect if you don’t need any fine tuning. It allows you to quickly declare formats, cropping, etc. with a nice configuration DSL. What then happens when uploading is not your business.

CarrierWave almost does what I wanted. But still, I was missing the ability to take control myself because I’m a control freak.

Also, all the other gems hook into ActiveRecord’s callbacks. I think the concept of AR’s callbacks is one of the worst things I’ve seen with Rails. Not that those gems use callbacks in the wrong way – the way other people use callbacks makes me mad.

Take Control Of Your Images!

Ok control freaks, here we go. Let’s look at how Paperdragon does it.

class User < ActiveRecord::Base
  include Paperdragon::Model
 
  processable :image
  serialize :image_meta_data
end

Oh, paperdragon doesn’t know anything about ActiveRecord or whatever ORM you use. You could use paperdragon in a PORO and it wouldn’t know it.

By calling processable paperdragon will add a reader method #image to your model that allows accessing the processing API – nothing else happens!

A second requirement is to provide accessors for a field named image_meta_data. It uses this field to store, well, meta data like the images’ paths as a hash. I use serialize from ActiveRecord. And of course, I added a TEXT column with the same name in my users table.

We’ll see how these two play together in a minute.

Uploading

Now, what’s happening when you upload an image?

Processing and storing an uploaded image is an explicit step – you have to code it! That’s right. You wanted control, now you got it.

This code usually goes to a separate class or an Operation in Trailblazer, don’t leave it in the controller if you don’t have to.

def create
  file = params.delete(:image)
  user = User.create(params) # this is your code.
 
  # upload code:
  user.image(file) do |v|
    v.process!(:original)                            
    v.process!(:thumb)   { |job| job.thumb!("75x75#") } 
    v.process!(:public)  { |job| job.watermark! } 
  end
 
  user.save
end

Whoooa, slow down. What’s all that?

file = params.delete(:image)
user = User.create(params) # this is your code.

Well, the first lines is Rails code: We simply create a user object. This is where paperclip and friends would hook in an start their magic. Paperdragon hates magic, that’s why I delete the uploaded file from params.

Explicit Processing.

After that setup, we finally get to process images.

user.image(file) do |v|
  v.process!(:original)                            
  v.process!(:thumb)   { |job| job.thumb!("75x75#") } 
  v.process!(:public)  { |job| job.watermark! } 
end

Do I really need to explain that? Ok.

I use the #image method we talked about earlier and pass the uploaded file into it.

In the block, I can create different versions of the image. It is an convention to name the unprocessed file :original – everything else is up to you!

The job objects are actually Dragonfly::Job instances and give you the entire API for processing, converting, cropping, resizing, analysing images. It’s insane and beautiful!

An important thing here is: once you call process! it happens. It’s your decision whether the entire block goes to a background job, if you want to crop the thumb right away, do the rest later, and so on.

Make It Persistent!

After the block is run, and you hopefully created some images, the last line seems odd.

user.save

This is to save the metadata! Remember, we have that field image_meta_data in the User model. And that very field gets populated with data when processing has finished.

Let’s check out this field’s content before we call #save and after the processing block.

user.image(file) do |v|
  v.process! #...
end
 
user.image_meta_data 
#=> {original: {uid:   "original-logo.jpg", 
#               width: 240, height: 800},
#    thumb:    {uid:   "thumb-logo.jpg", 
#               width: 48, height: 48},
#   ..and so on..

After processing, paperdragon automatically updates this field, so it can find the images later.

This is a fundamentally different concept to paperclip, which uses 5 or more columns of the model to generate the address at run-time.

While this might save some disk space, it is a very dangerous concept: If any of these values change by only one byte, you’re gonna lose your image since the computed address will be wrong. Paperdragon goes the explicit way and saves the address in the metadata hash.

Rendering Images!

All this work is rewarded when rendering a beautifully cropped thumbnail.

user.image[:thumb].url

That’s ready to be used in an #img_tag. Not too hard, right?

Reprocessing.

This was uploading. Paperdragon also makes it extremely explicitly simple to reprocess image version. For instance, let’s assume the thumb had been re-cropped via the UI.

user.image do |v|
  v.reprocess!(:thumb, Time.now) do |job| 
    job.thumb!("#{params[:w]}x#{params[:h]}#") 
  end
end

Reprocessing has a similar semantic. Here, you don’t need to pass in a file as we already got it stored. Furthermore, I call #reprocess! which requires a second parameter: a fingerprint!

That’s right, paperdragon automatically adds a fingerprint to the file name when reprocessing (if you want it).

Ehm, And.. What Else?

I am getting bored as I am replicating paperdragon’s README which will tell you about all the other great features such as renaming images, deleting, S3 support, custom UIDs, paperclip compatibility, and more.

Check It Out!

Paperdragon‘s API requires you to do all that work. In turn, it gives you control.

We are very happy with it, our site literally processes thousands of photos every day and paperdragon gives us exactly the amount of control we need, along with an abstraction that is not painful, yet.

BTW: The best Origami paperdragon image in the comments wins a free installation!

Reform 1.1 Brings More Reusability!

Thursday, August 21st, 2014

The new Reform 1.1 release brings several nice improvements and lots of fixes to the form object gem. This release is just a transitional one, the 2.0 version is gonna move huge chunks of code into a separate gem. Let me talk about this in a minute.

Non-Intrusive Validation.

Reform comes with a mechanism to populate your object graph by deserialising the incoming form data to objects.

class AlbumForm < Reform::Form
  collection :songs, populate_if_empty: Song do
    property :title
  end
end

This would create Song instances where they were missing in the album.

AlbumForm.new(album).validate(
  "songs" => [
    {"title": "Eat Your Words"}
  ]
)

In former versions of Reform, just by calling #validate Reform would already attach those new songs to the album model – by pushing them via songs[]=.

This created confusion. And is different now. The built-in populators do not touch the model at all. This only happens when saving or syncing the form.

Coercion Simplified.

Reform allows you to use virtus coercion.

class AlbumForm < Reform::Form
  property :released_at, type: DateTime
end

From 1.1 onwards, coercion only happens in #validate – any other API method will not trigger coercion. This makes the workflow predictable as we had problems with coercion happening where it shouldn’t.

Manual Coercion.

You can also implement your own filtering by overriding the setter for a property.

class AlbumForm < Reform::Form
  property :released_at
 
  def released_at=(v)
    super DateTime.parse(v)
  end
end

Note that you can use super to call the original setter but still provide your own code for filtering or sanitising or whatever fun stuff you’re after.

Overriding In Nested Forms.

This brings me to the next improvement. The above example only worked for top-level properties. That sucked and is fixed now, allowing you stuff as follows.

class AlbumForm < Reform::Form
  collection :songs do
    property :written_at
 
    def written_at=(v)
      super DateTime.parse(v)
    end
  end
end

Methods defined in nested forms actually extend that nested form class – as it should be!

Forms In Modules.

This is my favourite improvement as it maximises the reusability of forms: you can now define forms along with properties, validations and accessors/helpers in modules!

module SongsForm
  include Reform::Form::Module
 
  collection :songs
    property :title
    validates :title,  presence: true
  end
end

This module can now be included into real forms.

class AlbumForm < Reform::Form
  property :title
 
  include SongsForm
end

That should help to keep forms DRY, as it is a common pattern to have several different forms for the same model with lots of shared functionality.

Inheritance Improved.

Reform 1.1 uses representable 2.0 internally for all kinds of mappings and declarations. This actually allows you to “fine-tune” forms and overriding or extending properties that were inherited.

class AlbumForm < Reform::Form
  property :title
 
  include SongsForm
 
  collection :songs, inherit: true do
    property :artist
  end
end

This will extend the existing, inherited songs form and add the artist property. Read the docs for in-depth information about this or sign up for my upcoming book which discusses this pattern in detail.

Validations From Models.

While Reform/Trailblazer encourages you to have empty models that only configure your persistence layer, Reform now allows copying validations from models. This way you can quickly set up forms without having to write redundant validations.

class SongForm < Reform::Form
  property :title
 
  extend ActiveModel::ModelValidations
  copy_validations_from Song
end

Thanks to Cameron Martin for his excellent work.

Deserialise JSON.

A simple change in Reform now allows forms to deserialise JSON, XML and YAML besides the original support for hashes. While this might sound weird, this little improvement actually allows forms to become part of your HTTP API.

This is an integral part of Trailblazer: Here, every domain action is encapsulated in a so called “Operation” which internally uses a form object to deserialise incoming data, setup the object graph and validate the application state.

By making forms do JSON and friends, too, they can be used for normal web forms, console/model API and HTTP APIs, which is pretty awesome.

Again, this is all in my book but you can have a sneak peek in the Traiblazer example app gemgem – thanks to @GoGoGarrett for that name, BTW :)

class AlbumForm < Reform::Form
  representer_class.send :include, Representable::JSON
 
  def deserialize_method
    :from_json
  end
 
  property :title
 
  include SongsForm
 
  collection :songs, inherit: true do
    property :artist
  end
end

This rather clumsy extension (to be improved!) allows to call #validate with JSON.

AlbumForm.new(album).
  validate('"songs": [{"title": "Eat ..')

The API is not yet finalised, I just wanted to give you an out-sight on where this is going.

Reform 2.0 coming.

Speaking of out-sights: Reform 2.0 is already on its way. A major improvement here will be to move all the heavy model lifting (populating, syncing, saving, etc) into disposable which will be a model abstraction layer in Trailblazer (or without Trb).

Use it, and let me know what you think!

Rails Misapprehensions: Replace Helpers With View Models!

Thursday, August 14th, 2014

Is it just the crude title or is it true…? What makes Helpers Are Shit my most popular post ever? I felt like I have to write a follow-up now that Cells view models are replacing more and more helpers in many production apps out there.

Right, the last post is from 2011, that is old. What’s bad about this is that the write-up is still 100% accurate about the mess helpers and partials create in Rails apps. Nothing has changed in the Rails view layers for many years.

Helpers Are Shit.

The way helpers are being used in Rails encourages users to write “script-like” code using global variables and procedural functions in an environment that calls itself “object-oriented”.

Whenever I am refactoring complex Rails apps, the view layer is the worst part with hundreds of redundant helper methods accessing instance variables, calling other helpers, passing around models and options, and rendering random partials from a global directory.

Again, this mess is a result from the lack of a view abstraction layer in Rails: Helper invocations and partial rendering all happens in one monolithic ActionView instance – this is failure by design.

View Models.

Cells “view models” allow you to replace helpers entirely and forever. Instead of a global pile of mud you encapsulate parts of your view into a class. A class that can render templates, can inherit or mixin other methods, allows view inheritance, if needed.

This is faster than what you had before: In a view model, no data is copied between controller and view – the view model is the view context. Let me demonstrate that.

I will use the term “view model” and “cell” interchangeably (yepp, I had to look up that word).

Encapsulation Via Classes.

Encapsulating view logic happens in cell classes.

class CommentCell < Cell::ViewModel
  def show
    render
  end
end

Think of a cell as you think of a Rails controller (without all the HTTP overhead, of course). Per design, a cell has one public method #show. This is a convention. You can expose as many methods as you like from that cell. More on that later.

Where Is The Model?

I’ll walk you through that now. First, we need to render that cell somewhere in our app.

.sidebar
  = cell(:comment, Comment.last).show

Suppose this is somewhere in an application layout (it could be any view or in a controller). This will instantiate a CommentCell and execute the show method. This is a bit like calling a helper.

Note that I pass in a Comment model instance as the 2nd argument. This is the cell’s “model”, making the former a genuine view model™.

Encapsulated Rendering.

Let’s have a look again at the cell’s show method.

class CommentCell < Cell::ViewModel
  def show
    render
  end
end

Once the show method is called, it hits render. That will simply invoke the cell’s rendering as known from a Rails controller. With one exception: the view is located in a separate directory. Here’s how a cell directory looks.

app
├── cells
│   ├── comment_cell.rb
│   ├── comment
│   │   ├── show.haml

This is the old layout soon to be superceded by the Trailblazer-style “concept layout”. That doesn’t matter right now.

Good Old Views.

Now, let me show you how that show.haml could look like.

%h1 Comment
 
= model.body
By 
= link_to model.author.name, model.author

Two things.

  1. The cell’s model (the comment itself) is accessible via the #model method. Wow.
  2. You can still use helpers in a view. In this example, I use link_to.

No Code In My View!

We’ve learned that logic in views is bad. And this is exactly where view models start to be fun to work with.

class CommentCell < Cell::ViewModel
  def show
    render
  end
 
private
 
  def body
    model.body
  end
end

Remember: as the view model is the view context itself, any method called in the view will be simply called on the view model. That reduces complexity in the view a bit.

%h1 Comment
 
= body
By 
= link_to model.author.name, model.author

Embracing Helpers.

Still hating the link line. Why not move that into the cell, too?

class CommentCell < Cell::ViewModel
  def show
    render
  end
 
private
 
  def body
    model.body
  end
 
  def author_link
    link_to model.author.name, model.author
  end
end

Now, what? We can call helpers in the view model? Seerriously?

Well, I never said that helpers are a bad thing. They’re just shit when used to “encapsulate”. I personally love link_to, sanitize and form_for (sometimes). They save me a lot of work. I just don’t like the fact they all play together in a big monolithic class.

Here, the link_to can’t mess up anything as it is running in a limited scope – the cell instance.

Logicless Views.

The second refactoring reduces our view to a dumb template, as it should be.

%h1 Comment
 
= body
By 
= author_link

I love it already. What’s next?

Simple Decorating.

The task of delegating methods to the model is so tedious that it is built into cells. We can get rid of the body method.

class CommentCell < Cell::ViewModel
  property :body
  property :author
 
  def show
    render
  end
 
private
 
  def author_link
    link_to author.name, author
  end
end

Declaring body and author as propertys automatically delegates to the model, making it a no-brainer. It also states your dependencies to the model in a very clear way. Hooray to explicit code!

Object-Oriented Helpers.

What if we had to customise the body method, e.g. apply a markdown filter on the content?

class CommentCell < Cell::ViewModel
  include MarkdownHelper  
 
  property :body
  # ...
 
private
 
  def body
    markdown super
  end
end

First, I include the magic markdown helper to import the markdown method. Secondly, I can override the body method and call super to access the original property. Object-orientation back in the view land!

More Helpers!

A cell is not limited to one public method. It’s good to have only one but sometimes it comes in handy to not having to introduce a new class for every “helper”.

class CommentCell < Cell::ViewModel
  # ...
 
  def show
    render
  end
 
  def related
    render
  end
 
private
  # ...

Here, I add a second public method related. This will render another view, the related.haml. Dump it into the cell’s directory and you’re good to go.

app
├── cells
│   ├── comment_cell.rb
│   ├── comment
│   │   ├── show.haml
│   │   ├── related.haml

You can now use this “helper” anywhere you want.

= cell(:comment, Comment.last).related

Isn’t that nice? We encapsulated a huge part of a mess into a clean, reusable component. This feels good.

Don’t Stop Me Now!

I want to talk more about the benefits of cells, like view inheritance, testability, reusing components, hooking them directly to routes, sharing decorators between cells and other layers, but.. I’m writing this blog post at work and feel like I should stop now.

And, did you know? Recent surveys have concluded that 79.3% of all cells users did not regret their choice and can’t do without cells anymore.

Let me know if you have any questions.

Kaminari-Cells Is Ready!

Thursday, August 14th, 2014

The kaminari gem is a project to simplify pagination – often used in Rails apps. Cells provides view components for Rails. Those two just got married and their child is called kaminari-cells. Yay!!!

Kami..what?

Kaminari adds pagination logic to finders – it works with all kinds of ORMs, e.g. in ActiveRecord. It also gives us a bunch of helpers to render pagination links.

It is extremely popular as it doesn’t pollute objects with pagination logic.

Typically, a controller, an operation or a model will accumulate models to display.

@users = User.order(:name).page params[:page]

Later, in a view, after rendering the rows itself you want to display a link to jump to the next or previous page.

= paginate @users

A really simple and cool thing.

Pagination In Cells.

As kaminari uses global partials to render the pagination links, we had to make it work with cells, which usually does not use global partials. Thanks to the fine kaminari team, this was implemented, tested and merged within hours.

All you need to do to make the paginate helper work in a cell view is add the gem to your Gemfile.

gem "kaminari-cells"

And then extend the class.

class CommentsCell < Cell::Rails
  include Kaminari::Cells

This works for both conventional cells and view models.