Complexity is everywhere

Wednesday, February 11th, 2009

I’ve been a fan of Basecamp for years.  Ever since I heard about it (all the way back in 2005) I’ve encouraged its use whenever possible.  It has pretty much become the de-facto standard for web-developers across the world.  Part of its appeal is its unstructured nature – it’s basically a series of messages with task lists and dates.  Use what you want, how you want.  Each task item has three variables – title, task list it belongs to and position in the list (and they recently added a comments stream so you can discuss the item).  

However, sometimes its laissez faire approach is too unstructured.  So you may move to a bug-tracker.  The big problem there is, no matter how simple you think things are, how many fields you make optional, there is always an issue about complexity.  What takes precedence?  The issue-priority or the version that it has been assigned to?  Or does something over due date take precedence over both of those?  

You see, adding anything increases the complexity exponentially – going from three variables in Basecamp to eight or nine in a bug-tracker (and that’s a simple bug-tracker, not like the beast that is Bugzilla) means you have to define what each of those fields means to you and how they interact.  

Even the most apparently simple piece of software has this inherent complexity – look at the massive variation in Twitter clients – despite Twitter being nothing than a single 140-character text field.  All of which shows why software development can sometimes be very very hard to get right.

MVC: A brief history of Models, Views and Controllers

Saturday, January 31st, 2009

Any web-developer Rubyist knows about models, views and controllers. The MVC paradigm is embedded in the structure of Rails and Merb and encouraged by Ramaze and Sinatra. If you’re a Mac developer or an iPhone bod then MVC is common practice there as well. Same goes for Sproutcore and even Microsoft is getting in on the act.

But where does MVC come from?

Well, like most things I enjoy in the world of software development, MVC has its roots in Smalltalk. However, things used to look slightly different to the MVC we know and love. In Smalltalk, the basic Object has a feature known as “dependencies” built-in; nowadays, we would call this the observer pattern. Basically, any object can register an interest in any other object and will be notified whenever the target changes.


target addDependent: listener.

Later, when something changes within target, it can decide to notify all its dependents (observers):


self changed: #SomeArbitraryMessage.

and the listener has its update: aSymbol method called.

Building this notification mechanism right into the core of the system means that it can be used everywhere – especially when designing user-interfaces (remember, Smalltalk was the GUI for the Xerox Star, which his Steveness bought and adapted for the Lisa). Models were the system components, the pieces of the system that actually do the work. Views represented portions of the screen and controllers represented the keyboard and the mouse. These parts were arranged as follows:

MVC in Smalltalk

MVC in Smalltalk

The view would add itself as a dependent to the model and display the relevant aspects of the model on-screen. The user would see this, manipulate it via the mouse and keyboard, which triggers the controller. The controller sends messages to the model, which triggers its notifications, prompting the dependent view to redraw itself. In other words, the controller and view are independent objects that know nothing about each other – the only tie between them is the controller’s knowledge of the model and the dependency mechanism.

Contrast this to the MVC we know today:

MVC in Rails

MVC in Rails

Here the user pokes the controller, which in turn sends messages to the model. The model does its thing and then the controller extracts the relevant information and passes it to the view, which is then rendered to the user. In this manner, the controller lives up to its name, orchestrating the entire cycle – and is the centre of all the coupling (as it knows about the view and the model).

So why did things change (first in Smalltalk and then the following MVC frameworks)?

The first problem is that the controller takes input and the view handles output – as separate objects. Which, especially for desktop applications, doesn’t really work; if you click on a text-field you expect different behaviour to clicking on a scrollbar.

Secondly, imagine an array of models being shown in a list box. The list box has the concept of a current selection. So your array model (a model of models) needs to have a current selection, so that the view knows which is which. But what if the same list is rendered in two places at the same time. If you point both views at the same array model, they would both share the same selection properties. So you actually end up with something more like:

Getting Complicated

Getting Complicated

As you can see it gets nasty pretty quickly. And actually the controller is simply passing its messages directly through to the array model which is actually doing the routing of the other messages – the controller is pretty much redundant.

So if you swap things around, merge the controller and array model (into some sort of array controller) and then add multiple array controllers – one per view – to deal with the selection issue, suddenly you have the newer style object layout. Requests go into one of the controllers, it routes it to the model, then parcels up the results and invokes the view.

Evolution in action.

Writing tests for your controllers improves the design of your models

Saturday, December 20th, 2008

I’ve recently been updating some old code – partly written by someone else, partly written by myself. At the time, I thought I had written this code really well; looking back on it now, it looks awful. Fair enough, I’ve learnt a lot – I want to look back on old code and shudder. But also, there is very poor test coverage on this app and the tests that there are are quite unwieldy due to an over-reliance on fixtures.

So I’ve been reworking them all using RSpec, my fork of RSpec-Rails and my Object Factory (which means I can avoid fixtures).

Most of the work involves writing a spec that mimics the current behaviour (by inspecting the code and trying to match all paths through it), then refactoring the code, using the spec to prove that I haven’t broken it.

But some points have some really horrible code (and lots of it) within the controllers. As you probably know, Skinny Controllers is the Rails way – your application logic belongs in your models (as they are your application) – the controller should just find or create the relevant model, ask it to do something and then render the results.

Because of this, I opted to just rewrite the actions in question.

To do this I started by writing a Cucumber feature describing things from the user’s point of view. Actually writing the steps that match the feature was a lot of work; because Cucumber is a full stack test you have to deal with all the dependencies that your individual action has (for example, are you logged in with the correct permissions with all associated objects created and in the right state?).

Then I wrote a controller spec. Controller specs in RSpec should use mock objects; you don’t really want to test the models, you just want to prove that the controller finds or creates the right model, asks it to do something and renders the correct output at the end.

So a typical spec looks something like this (note that this is not RESTful as it was an existing part of the application that I am about to change):


  it "should process an item" do
    @item = mock_model Item, :work_type => :buy_stuff, :quantity => 2

    on_getting :process_item, :id => '1' do
     Item.should_receive(:find).with('1').and_return(@item)
      @stuff = mock_model Stuff
      @item.should_receive(:process).and_return(@stuff)
    end

    assigns[:stuff].should == @stuff
    response.should be_success
    response.should render_template('admin/orders/process_item')
  end

This basically says:

  • We have an item, of type “buy stuff” with a quantity
  • When the process_item action is called, we expect that the controller will try to find the item with the given id
  • Then we should call process on the item and it should give us some stuff
  • The stuff should be stored in an instance variable, called stuff
  • And a page should be successfully rendered using the process_item template

That’s a pretty succinct explanation of what the controller should do – I can’t think of many ways of making that skinnier. It also bears no resemblance to the actual implementation of the action – which currently looks something like this:


def process_work_item
    @item =Item.find(params[:id])
    case @item.product.class.to_s.underscore.to_sym
    when :buy_stuff
      @stuff = Stuff.build_item(@item)
      @stuff.setup_new
      render :action => :process_stuff
    when :update_stuff
      @item.stuff.prepare_for_update
      render :action => :update_stuff
    else
      render :status => 404, :text => "Item product type #{@item.product.class.to_s} unknown."
    end
rescue ActiveRecord::RecordNotFound
  render_not_found
end

Pretty complicated – and as new types of item are added we need to add more and more clauses to that case statement.

First things first – I’ve said that we should call “process” on the item class. So I add a pending spec to the Item specification – this is to remind me that I’ve got some work to implement later on.


  it "should process itself based upon its work type"

Then I rework the controller so that the controller spec passes.


    @item =Item.find(params[:id])
    @stuff = @item.process
    render :template => "admin/orders/process_item"

Pretty simple huh?

What we have done is shifted the logic from the controller into this new “process” method on the Item. We have made it so that the controller knows virtually nothing about the item – all it knows is how to find it and that it has a process method. All the implementation details are now hidden within the Item, out of the way of the outside world.

Through the use of mocking we can ignore actual implementations and concentrate on presenting ourselves as simply, and minimally, as possible to the outside world. This reduces coupling, increases flexibility and makes our code easier to read. Don’t you agree?

Coping with the VAT Change

Tuesday, November 25th, 2008

There seems to be a lot of wailing and gnashing of teeth about the upcoming VAT change. Especially as it is only a 13 month change and the rate will revert to 17.5% in 2010.

However, it ought to be really simple (although I realise that this may be a bit late for some computer systems).

Never hard-code the VAT rate

Have a Tax class that the rest of the system can ask when it needs the tax rate. That way, you’ve only got one place to make the change.

Store the rate against your invoice lines

Add a field to your invoice lines that stores the rate applicable for that line. Populate this when the line is created. That way, an invoice at tax point 30th November 2008 will have lines with a 17.5% stored against them, those created on the 1st December 2008 will have a 15% stored against them. This keeps your most vital financial documents accurate over time.

Calculate the VAT as late as possible

When calculating the VAT on an invoice do not do this:



value_excluding_vat = 0

vat = 0

value_including_vat = 0

for each line in my invoice

  value_excluding_vat += line.value

  vat += line.value * line.tax_rate

  value_including_vat += line.value + (line.value * line.tax_rate)

The problem here is that you are calculating the VAT on a per line basis and then summing the values. This will lead to rounding errors – where your line values are a penny out from your totals.

Instead you need to place each line into a “bucket” for its tax rate, sum all values for a given rate and then calculate the tax on that. That way you are working on the largest numbers possible, reducing the risk of small decimal place discrepancies throwing your totals out.



value_excluding_vat = 0

for each line in my invoice

  bucket = the_bucket_for line.vat

  bucket.value += line.value

  value_excluding_vat += line.value

vat = 0

for each bucket in my collection of buckets

  vat += bucket.value * bucket.rate

value_including_vat = value_excluding_vat + vat

Have a Tax Band and Tax Rate model that deals with changes over time

If you really want to have a “minimal administration” system for dealing with tax rate changes then you should switch your data model to have multiple Tax Band objects (as remember, there are multiple VAT bands – “standard” is 17.5/15%, “low” is 5%, “zero-rated” is 0% and “exempt” is also 0% but needs to be accounted for separately to “zero-rated”).

A Data Model for dealing with variable tax bands and variable tax ratesEach Tax Band object should have a set of Tax Rate objects hanging off it – where each Tax Rate has a percentage rate and a start and end date. That way you can ask your “Standard Rate” tax object “what is the percentage on the 30th November 2008?” and it can give you the correct answer.

That makes dealing with a change like this simple – you simply go to the “Standard Rate” object, find it’s current rate object and set its end date to 30th November 2008. Then you add a new rate object to the “standard rate” of 15%, setting its start date to 1st December 2008, with an end date of 31st December 2009. Lastly, add another 17.5% rate object, with a start date of 1st January 2010 and no end date.

Update the rest of your systems to always ask the relevant tax object for the correct rate and tax rate changes will never be a headache again.

Acceptance Testing in Ruby, Rails, RSpec and Cucumber

Friday, November 21st, 2008

I’ve written up a new post at the Brightbox blog detailing how we are using RSpec and Cucumber to build acceptance tests for the next generation Brightbox systems.