Install Ruby using the One Click Installer.
Install the ODBC module (by copying the SO files into the c:\Ruby\1.8\i386-mingw32 folder).
Install the ActiveRecord ODBC adapter (gem install activerecord-odbc-adapter).
Edit C:\Ruby\lib\ruby\gems\1.8\gems\activerecord-odbc-adapter-2.0\ lib\active_record\connection_adapters\odbc_adapter.rb – look for line 1588.
Change the line from elsif dbmsName =~ /SQLAnywhere/i to elsif dbmsName =~ /SQLAnywhere/i or dbmsName =~ /adaptiveserveranywhere/i

Write some test code – something like:


require 'rubygems'
require 'active_record'

Whatever.all

Run this - hopefully you should get no errors.
Then sit down and have a long think about what the hell you are doing.

Switching is easy – just change the :storage parameter to :s3. But before you can do that, how do you get the existing files onto S3 in the first place?

Well, here’s your answer.

Before you change the storage parameter, create you S3 account and a bucket to store your files.

Then add a new rake task to your application, looking something like this:

  desc "Upload files to S3"
  task :upload_files_to_s3 => :environment do

    AWS::S3::Base.establish_connection!(
      :access_key_id => 'MYACCESSKEY',
      :secret_access_key => 'MYSECRETKEY')

    Model.find(:all).each do | user |
      filename = "/home/user/app/current/public#{model.public_filename}"
      puts "Migrating model #{model.id} - #{filename}"
      if File.exists?(filename)
        AWS::S3::S3Object.store("/models/#{model.id}/#{model.filename}",
          open(filename),
          'my_bucket',
          :access => :public_read)
        puts "...migrated"
      else
        puts "...not found"
      end
    end
  end

In other words;

• connect to S3
• for each “model” instance, find the attachment on the local filesystem
• if found, then push it to S3, using /models/id/filename as the key (where models is your table name)

As soon as you’ve done that, switch the :storage to :s3 and redeploy your app (not forgetting your config/amazon_s3.yml file). And, with a bit of luck, you should see your files being served from Amazon’s servers, instead of your own.

Before Bigwig, it was Object Factory, before that it was a Delphi class that I used to create tree-structured data (imaginatively called TNode).

Bigwig’s doesn’t have a test suite and it’s not even my code – it was started by David Smalley and has contributions from Caius Durling and the rest of the Brightbox team.

But there’s just something about it – it’s a daemon (start it running and forget about it); it’s been running in production for months without a single glitch; and probably most importantly, the code is so simple that there’s almost nothing to it.

Gemcutter’s the new kid on the block – it works as a set of plugins to the gem command that mean you can simply go `gem push my.gem` and it becomes available to the world. Nice as this is, gemcutter’s still pretty young and is currently hosted off one guy’s S3 account.

So we went back to good old-fashioned Rubyforge. Except, believe it or not, I’d never published to Rubyforge before. And it wasn’t as straightforward as it could have been.

I was using the echoe library to build my gem – which gave me a simple `rake manifest`, `rake release` workflow. All good. When you `rake release` it asks if you want to publish to rubyforge; you say “yes” and go to bed happy.

Or not in my case.

Firstly, you need to actually configure rubyforge. Install the rubyforge gem, then use `rubyforge setup` to put in your username and password. Then use `rubyforge config` to make them stick.

That done, I `rake release`d. And got a “no group_id configured for mygem” error. Eh? Googling seemed no help whatsoever – it was mentioned in a few places but people seemed to have resolved it without actually posting their solutions (or I was being blind).

What you need to do is tell Rubyforge where to put your gem. In our case we already had a project on Rubyforge that the gem could be attached to – so `rubyforge create_package project_name gem_name` did the trick. Otherwise you need to get a project set up to house your gem.

But it still didn’t work. This time complaining about a processor id. I had no idea what I was looking for this time, until I discovered this pastie by Dr Nic. The autogenerated config file (normally in ~/.rubyforge/auto-config.yml) is missing some vital information – namely codes for different processor architectures. I pasted the list into your config file and then tried the `rake release` again. And this time it worked!

processor_ids:
  IA64: 6000
  AMD-64: 1500
  Any: 8000
  Sparc: 4000
  PPC: 2000
  Other: 9999
  Alpha: 7000
  i386: 1000
  UltraSparc: 5000
  MIPS: 3000

So for the sake of posterity – and anyone else who gets “no group_id configured for mygem” or “no processor_id configured for mygem” messages, I thought I should let you know how I got there.

But the weirdest thing was we couldn’t figure out the cause. Well I could see why the code was failing (after adding some extra log messages). But ‘git blame’ said those lines of code were unchanged in twelve months. Why hadn’t people complained before? Why hadn’t we noticed it?

After much hunting through log files we found the point when the feature last worked. It coincided with a deployment. That deployment was our Rails 2.3.4 forms vulnerability fix. And the bug was in a form – a missing form parameter that earlier versions of Rails ignored, the newer Rails was choking on.

But why didn’t the tests catch it?

After more hunting I saw that the Cucumber test that exercised the form didn’t have a “When I press the Update button” step. And the subsequent tests were passing, even though the update button hadn’t been pressed.

So I added the step in and made the feature pass. Then deployed it as an emergency fix.

However, what are the lessons to learn here (as there are always some)?

• Firstly, testing cannot catch everything.
• Secondly, the cracks in your tests are where the bugs are.
• Thirdly, we probably need some sort of peer review for tests. I feel that this is more important than for code, because once the tests are right you can refactor the code without worry.
• Fourthly, you really need to log everything. Absolutely everything. Don’t worry about your huge log files – that’s what `logrotate` is for. Get it written down so that one time when you have an obscure bug, you’ll be able to find it easily.

He had a class, descending from ActiveRecord::Base, with a custom constructor (initialize method). To debug it, he had the constructor raise an exception. In the console, Thingy.new(params) raised the exception as expected. But wotsit.thingies.find_by_field(value) did not. Even though it was instantiating an instance of Thingy and returning it.

“It must not be calling the constructor” he said.
“Rubbish” said I, “it’s a constructor. Constructors are always called. That’s the point of them”.

But as he dug deeper it certainly looked like the constructor wasn’t being called.

And then he found an article explaining that you should never rely on things being set up on in an Active Record constructor. Mainly because Active Record uses allocate to instantiate associated objects. And what is this mysterious allocate? Why explains it all.

To be honest, I’ve got mixed feelings about this. I can see the use of “allocate” – why’s example of marshalling an object makes sense (I’m slightly less sure about the way that Active Record uses it to load associations). But, to my mind, the definition of a constructor is “the code that is always called when an object is created”. So maybe I should just stop thinking of initialize as a constructor and more as an initialiser.

Suppose you’ve been working in branch X and you’re about to merge those changes into branch Y …

• work in branch X and make your commits as needed
• switch to branch Y git checkout Y
• compare the differences between X and Y git log Y..X – the order here is important; it is merge-target..merge-source
• merge the changes from X (merge-source) into Y (merge-target) git merge X

Even better, before doing the merge, while you are still in branch X, you can squash multiple commits into a single one. This way, branch Y, when you examine the log, has a single entry “implemented feature X”, instead of thirty-five entries all related to feature X in some way.

• examine the log git log or even better git log --pretty=oneline
• choose the commit that immediately precedes the one that starts your piece of work (probably the last one before you branched to start work on feature X)
• start an interactive rebase session – this is git-fancy-talk for picking some commits and typing a message – git rebase -i THE-COMMIT-ID-OF-THE-COMMIT-YOU-SELECTED-ABOVE
• Your favourite editor (or nano) will open listing the commits for feature X – edit the word “pick” to “squash” for all except the top entry, save and exit
• Your favourite editor will open again – again showing your commit messages – add a line at the top stating something like “implementation of feature X” and leave the list of individual commits on separate lines below it (adding in a Fixes #123 or whatever your issue tracker demands as the last line), save and exit
• Now if you do a git log you will see a single commit with a nice “implementation of feature X” log message
• And you’re all set to merge to your master branch without cluttering its history with a long string of commit messages

All the tests passed. Poking it on the staging server worked well. On to production and all was good.

Time passed.

Then, the app needed to be moved to a new server. Actually to a cluster. Slightly more nerve-wracking, but some playing around with the ghost gem and DNS settings and the deployment completed.

More time passed.

And then disaster struck. Well, not disaster, but it wasn’t good. Not at all.

The app had a rake task that was cronned to run every night. And it failed silently. We found out the day after and I scanned the production log file looking for evidence of what went wrong. Nowt. It didn’t even look like the rake task had run – but it had, as I had the data in front of me.

And the day after, it failed again. Another scan of the production log revealed nothing. Things were getting serious.

I suspected the cluster deployment. Maybe cron didn’t have permission to write to the log file, even though Passenger did. But everything looked good on that front. Maybe there was something else weird about the cluster setup – all the cron jobs were set to run on one of the app-servers; maybe moving it would make a difference?

To test this, I opened the Rails console and started manually recreating what the rake task was doing, on a different app-server. Which is when I noticed that script/console wasn’t writing to the production log either. What?

After a couple of hours of head scratching, I finally found out that the cluster deployment was not at fault. In fact, it seems to be the move to Rails 2.2 that had happened many weeks before – in particular it appears that the logger object no longer auto-flushes itself after writing.

We added a quick Rails.logger.auto_flushing = 1 to an initialiser and both script/console and the rake tasks wrote to the log as expected. Phew!

From the customer’s point of view it’s pretty essential. You need to know how much you are spending before the work begins so you don’t get stung.

From the developer’s point of view it’s pretty difficult to do because you don’t know how long things will take until you know what you’re building. And you don’t know the details of what you are building until they become apparent, which is normally while you are doing the work.

For this particular client I wanted to get this piece of work out of the way as quickly as possible and the requirements were deceptively complicated. As trying to deal with intangibles is a recipe for disaster, and time was short, I was pretty sure that whatever estimate I came up with would be totally inaccurate.

So to deal with the intangibles I thought I may as well get stuck in with the actual work. And what’s the first thing that I do? Write a cucumber story.

I went through the requirements documents (which after weeks of to and fro were actually quite detailed and refined) and broke it down into four features, each with a number of scenarios. Each scenario was then broken down into individual steps – however, I did not write any step definition (ruby) files. This is cucumber as customer documentation, not as integration testing.

We agreed that the features met the requirements (the client was very impressed with the level of detail and the clarity that cucumber offered) and I simply went through each feature, scenario and step and came up with a price for the step individually. It’s still guesswork, but it’s guesswork on a small scale, so you’re less likely to be way off. Total the lot and there’s your estimate.

And even better, once the client has agreed the price and you’ve started development, you’ve got a development plan, a measure of progress and proof that it all works – all just a simple rake features away.

Image by lemon drop

However, the rest of my suite relied on transactional fixtures (which is Rails’ badly named way of saying that a transaction is started before each test and rolled back at the end, leaving your test database in a pristine state before the next case is run).

In particular, my spec_helper.rb had the following:

Spec::Runner.configure do |config|
  config.use_transactional_fixtures = true
  # stuff
end

The code being tested looked something like this:

begin
  Model.transaction do
    do_something # may fail with an exception
    do_something_else # may fail with an exception
  end
rescue Exception => ex
  do_some_recovery_stuff
end

I had a spec for the successful path (checking that the outcomes of do_something and do_something_else were what I expected.

However when I tried the same for the failure paths, the outcomes matched the successful path. The time-tested debugging method of sticking some puts statements in various methods showed that do_some_recovery_stuff was being called as expected. But the outcomes were still wrong.

And the reason? Transactions. This was a Rails 2.2 project, running on Mysql (innodb). As RSpec/Test::Unit starts a transaction before the specification clause runs (and then rolls it back on completion) when Model.transaction statement is reached, the spec is actually starting a second transaction, nested within RSpec/Test:Unit’s. Which means when the inner transaction is rolled back, the database doesn’t actually do anything – there’s still an outer transaction that may or may not be rolled back. (I think Rails 2.3 corrects this behaviour and if you roll back an inner transaction then the outer transaction reflects the correct state, but I’m not 100% on that).

So I had a choice – move the (production) app to Rails 2.3 to fix this one bug (which is very urgent) or figure out how to switch the outer transaction off for these particular steps. Google wasn’t very helpful – lots of stuff on how RSpec extends Test::Unit, lots of stuff on how Rails extends Test::Unit to add the fixtures (and transaction) support. But no concrete example on how to actually switch it off.

After much playing around (overriding methods like use_transaction?(method_name) and runs_in_transaction?) I eventually stumbled across the answer. And it’s pretty simple.

Set your default to be config.use_transactional_fixtures = true in spec_helper.rb. Then, for the specs that are not transactional, simply create a describe block and simply add the following:

describe MyClass, "when doing its stuff" do
  self.use_transactional_fixtures = false
  it "should do this"
  it "should do that"
  it "should do the other"
end

The only thing to be aware of is you may well need an after :each block to clean up after yourself.