threehv

One of the things that stands out about Ruby and Rails developers is that the vast majority obsess over “behaviour-driven-development” (or its similar predecessor “test-driven-development”).

At first glance, this allows a suite of automated tests to be run against your code – the idea being that you have proof that your system does what it is supposed to. Bugs are caught early and, most importantly, a new feature cannot break existing features without you knowing about it.

But there is a much bigger advantage to behaviour-driven-development.

The very act of writing your tests before writing your code clarifies what you are trying to achieve and results in designs and code that are actually simpler – and hence easier to maintain.

I was reminded of this when trying to build a data importer. My colleague wrote a huge piece of code that seemed to work at first. But upon further investigation, it appeared to get some internal references wrong – and as the import took an hour to run, debugging became a pain.

So I started over. I wrote out a series of names for my specification: it should import a single product, it should import all variations of that product, it should import the images for a product, it should update an existing product.

These bullet points are a simple description for what it was supposed to achieve.

I then implemented the first bullet point in code – I took a copy of the import file, stripped out most of the data till I found a relevant piece, loaded that and then called the importer. My test just checked the results and ensured that the results were what was expected. The first run failed – because I didn’t have any code in my importer. But I implemented the simple “import a product” case and the test passed. Then I implemented the second bullet point in code. The first test passed, the second failed. I added the code to get the new test to pass – two passing tests and a very simple piece of code for the importer. Continue till all the tests were implemented – fantastic. The importer worked. But the most important thing – because it was built incrementally, it was simple. Even better, if it did start getting messy I could rewrite and rearrange the code to simplify it – and still be sure it was working as the tests proved it.

Writing the tests does not take much time – in fact it saves time as you have to explain to yourself what you are doing before you begin. But even more importantly, it results in clean, simple code. Which saves even more time (and money) in the long run, as that becomes cheaper to maintain.

Just don’t ask why I know this. Please. It’s making me cry.

Install Ruby using the One Click Installer.
Install the ODBC module (by copying the SO files into the c:\Ruby\1.8\i386-mingw32 folder).
Install the ActiveRecord ODBC adapter (gem install activerecord-odbc-adapter).
Edit C:\Ruby\lib\ruby\gems\1.8\gems\activerecord-odbc-adapter-2.0\ lib\active_record\connection_adapters\odbc_adapter.rb – look for line 1588.
Change the line from elsif dbmsName =~ /SQLAnywhere/i to elsif dbmsName =~ /SQLAnywhere/i or dbmsName =~ /adaptiveserveranywhere/i

Write some test code – something like:


require 'rubygems'
require 'active_record'

Whatever.all

Run this - hopefully you should get no errors.
Then sit down and have a long think about what the hell you are doing.

attachment_fu may not be the in-thing any more but there are still a lot of sites out there using it. And every now and then you realise, far too late, that you should have used S3 instead of the local filesystem.

Switching is easy – just change the :storage parameter to :s3. But before you can do that, how do you get the existing files onto S3 in the first place?

Well, here’s your answer.

Before you change the storage parameter, create you S3 account and a bucket to store your files.

Then add a new rake task to your application, looking something like this:

  desc "Upload files to S3"
  task :upload_files_to_s3 => :environment do

    AWS::S3::Base.establish_connection!(
      :access_key_id => 'MYACCESSKEY',
      :secret_access_key => 'MYSECRETKEY')

    Model.find(:all).each do | user |
      filename = "/home/user/app/current/public#{model.public_filename}"
      puts "Migrating model #{model.id} - #{filename}"
      if File.exists?(filename)
        AWS::S3::S3Object.store("/models/#{model.id}/#{model.filename}",
          open(filename),
          'my_bucket',
          :access => :public_read)
        puts "...migrated"
      else
        puts "...not found"
      end
    end
  end

In other words;

• connect to S3
• for each “model” instance, find the attachment on the local filesystem
• if found, then push it to S3, using /models/id/filename as the key (where models is your table name)

As soon as you’ve done that, switch the :storage to :s3 and redeploy your app (not forgetting your config/amazon_s3.yml file). And, with a bit of luck, you should see your files being served from Amazon’s servers, instead of your own.

I think Bigwig has become my favourite bit of code that I’ve ever worked on.

Before Bigwig, it was Object Factory, before that it was a Delphi class that I used to create tree-structured data (imaginatively called TNode).

Bigwig’s doesn’t have a test suite and it’s not even my code – it was started by David Smalley and has contributions from Caius Durling and the rest of the Brightbox team.

But there’s just something about it – it’s a daemon (start it running and forget about it); it’s been running in production for months without a single glitch; and probably most importantly, the code is so simple that there’s almost nothing to it.

Github aren’t building new gems at the moment as they finalise their move to Rackspace. So what do you do if you’ve got a gem that you would like to make available?

Gemcutter’s the new kid on the block – it works as a set of plugins to the gem command that mean you can simply go `gem push my.gem` and it becomes available to the world. Nice as this is, gemcutter’s still pretty young and is currently hosted off one guy’s S3 account.

So we went back to good old-fashioned Rubyforge. Except, believe it or not, I’d never published to Rubyforge before. And it wasn’t as straightforward as it could have been.

I was using the echoe library to build my gem – which gave me a simple `rake manifest`, `rake release` workflow. All good. When you `rake release` it asks if you want to publish to rubyforge; you say “yes” and go to bed happy.

Or not in my case.

Firstly, you need to actually configure rubyforge. Install the rubyforge gem, then use `rubyforge setup` to put in your username and password. Then use `rubyforge config` to make them stick.

That done, I `rake release`d. And got a “no group_id configured for mygem” error. Eh? Googling seemed no help whatsoever – it was mentioned in a few places but people seemed to have resolved it without actually posting their solutions (or I was being blind).

What you need to do is tell Rubyforge where to put your gem. In our case we already had a project on Rubyforge that the gem could be attached to – so `rubyforge create_package project_name gem_name` did the trick. Otherwise you need to get a project set up to house your gem.

But it still didn’t work. This time complaining about a processor id. I had no idea what I was looking for this time, until I discovered this pastie by Dr Nic. The autogenerated config file (normally in ~/.rubyforge/auto-config.yml) is missing some vital information – namely codes for different processor architectures. I pasted the list into your config file and then tried the `rake release` again. And this time it worked!

processor_ids:
  IA64: 6000
  AMD-64: 1500
  Any: 8000
  Sparc: 4000
  PPC: 2000
  Other: 9999
  Alpha: 7000
  i386: 1000
  UltraSparc: 5000
  MIPS: 3000

So for the sake of posterity – and anyone else who gets “no group_id configured for mygem” or “no processor_id configured for mygem” messages, I thought I should let you know how I got there.

We just had some customers report a bug. Not good. We didn’t get an exception email. All the tests passed. We couldn’t see anything untoward in the log files. But it was there. We could reproduce it, both in staging and in production. Not good at all.

But the weirdest thing was we couldn’t figure out the cause. Well I could see why the code was failing (after adding some extra log messages). But ‘git blame’ said those lines of code were unchanged in twelve months. Why hadn’t people complained before? Why hadn’t we noticed it?

After much hunting through log files we found the point when the feature last worked. It coincided with a deployment. That deployment was our Rails 2.3.4 forms vulnerability fix. And the bug was in a form – a missing form parameter that earlier versions of Rails ignored, the newer Rails was choking on.

But why didn’t the tests catch it?

After more hunting I saw that the Cucumber test that exercised the form didn’t have a “When I press the Update button” step. And the subsequent tests were passing, even though the update button hadn’t been pressed.

So I added the step in and made the feature pass. Then deployed it as an emergency fix.

However, what are the lessons to learn here (as there are always some)?

• Firstly, testing cannot catch everything.
• Secondly, the cracks in your tests are where the bugs are.
• Thirdly, we probably need some sort of peer review for tests. I feel that this is more important than for code, because once the tests are right you can refactor the code without worry.
• Fourthly, you really need to log everything. Absolutely everything. Don’t worry about your huge log files – that’s what `logrotate` is for. Get it written down so that one time when you have an obscure bug, you’ll be able to find it easily.

Today, Caius made a discovery that shocked me.

He had a class, descending from ActiveRecord::Base, with a custom constructor (initialize method). To debug it, he had the constructor raise an exception. In the console, Thingy.new(params) raised the exception as expected. But wotsit.thingies.find_by_field(value) did not. Even though it was instantiating an instance of Thingy and returning it.

“It must not be calling the constructor” he said.
“Rubbish” said I, “it’s a constructor. Constructors are always called. That’s the point of them”.

But as he dug deeper it certainly looked like the constructor wasn’t being called.

And then he found an article explaining that you should never rely on things being set up on in an Active Record constructor. Mainly because Active Record uses allocate to instantiate associated objects. And what is this mysterious allocate? Why explains it all.

To be honest, I’ve got mixed feelings about this. I can see the use of “allocate” – why’s example of marshalling an object makes sense (I’m slightly less sure about the way that Active Record uses it to load associations). But, to my mind, the definition of a constructor is “the code that is always called when an object is created”. So maybe I should just stop thinking of initialize as a constructor and more as an initialiser.

I always forget how to do this so I’m writing it down (especially as it’s really easy and git, as usual, makes me feel stupid as Linus is so much smarter than me).

Suppose you’ve been working in branch X and you’re about to merge those changes into branch Y …

• work in branch X and make your commits as needed
• switch to branch Y git checkout Y
• compare the differences between X and Y git log Y..X – the order here is important; it is merge-target..merge-source
• merge the changes from X (merge-source) into Y (merge-target) git merge X

Even better, before doing the merge, while you are still in branch X, you can squash multiple commits into a single one. This way, branch Y, when you examine the log, has a single entry “implemented feature X”, instead of thirty-five entries all related to feature X in some way.

• examine the log git log or even better git log --pretty=oneline
• choose the commit that immediately precedes the one that starts your piece of work (probably the last one before you branched to start work on feature X)
• start an interactive rebase session – this is git-fancy-talk for picking some commits and typing a message – git rebase -i THE-COMMIT-ID-OF-THE-COMMIT-YOU-SELECTED-ABOVE
• Your favourite editor (or nano) will open listing the commits for feature X – edit the word “pick” to “squash” for all except the top entry, save and exit
• Your favourite editor will open again – again showing your commit messages – add a line at the top stating something like “implementation of feature X” and leave the list of individual commits on separate lines below it (adding in a Fixes #123 or whatever your issue tracker demands as the last line), save and exit
• Now if you do a git log you will see a single commit with a nice “implementation of feature X” log message
• And you’re all set to merge to your master branch without cluttering its history with a long string of commit messages

A project I’ve been working on was recently moved up to Rails 2.2 (Rails 2.3 migration coming soon, but we wanted to take things one step at a time).

All the tests passed. Poking it on the staging server worked well. On to production and all was good.

Time passed.

Then, the app needed to be moved to a new server. Actually to a cluster. Slightly more nerve-wracking, but some playing around with the ghost gem and DNS settings and the deployment completed.

More time passed.

And then disaster struck. Well, not disaster, but it wasn’t good. Not at all.

The app had a rake task that was cronned to run every night. And it failed silently. We found out the day after and I scanned the production log file looking for evidence of what went wrong. Nowt. It didn’t even look like the rake task had run – but it had, as I had the data in front of me.

And the day after, it failed again. Another scan of the production log revealed nothing. Things were getting serious.

I suspected the cluster deployment. Maybe cron didn’t have permission to write to the log file, even though Passenger did. But everything looked good on that front. Maybe there was something else weird about the cluster setup – all the cron jobs were set to run on one of the app-servers; maybe moving it would make a difference?

To test this, I opened the Rails console and started manually recreating what the rake task was doing, on a different app-server. Which is when I noticed that script/console wasn’t writing to the production log either. What?

After a couple of hours of head scratching, I finally found out that the cluster deployment was not at fault. In fact, it seems to be the move to Rails 2.2 that had happened many weeks before – in particular it appears that the logger object no longer auto-flushes itself after writing.

We added a quick Rails.logger.auto_flushing = 1 to an initialiser and both script/console and the rake tasks wrote to the log as expected. Phew!