3hv

Today, Caius made a discovery that shocked me.

He had a class, descending from ActiveRecord::Base, with a custom constructor (initialize method). To debug it, he had the constructor raise an exception. In the console, Thingy.new(params) raised the exception as expected. But wotsit.thingies.find_by_field(value) did not. Even though it was instantiating an instance of Thingy and returning it.

“It must not be calling the constructor” he said.
“Rubbish” said I, “it’s a constructor. Constructors are always called. That’s the point of them”.

But as he dug deeper it certainly looked like the constructor wasn’t being called.

And then he found an article explaining that you should never rely on things being set up on in an Active Record constructor. Mainly because Active Record uses allocate to instantiate associated objects. And what is this mysterious allocate? Why explains it all.

To be honest, I’ve got mixed feelings about this. I can see the use of “allocate” - why’s example of marshalling an object makes sense (I’m slightly less sure about the way that Active Record uses it to load associations). But, to my mind, the definition of a constructor is “the code that is always called when an object is created”. So maybe I should just stop thinking of initialize as a constructor and more as an initialiser.

I always forget how to do this so I’m writing it down (especially as it’s really easy and git, as usual, makes me feel stupid as Linus is so much smarter than me).

Suppose you’ve been working in branch X and you’re about to merge those changes into branch Y …

• work in branch X and make your commits as needed
• switch to branch Y git checkout Y
• compare the differences between X and Y git log Y..X - the order here is important; it is merge-target..merge-source
• merge the changes from X (merge-source) into Y (merge-target) git merge X

Even better, before doing the merge, while you are still in branch X, you can squash multiple commits into a single one. This way, branch Y, when you examine the log, has a single entry “implemented feature X”, instead of thirty-five entries all related to feature X in some way.

• examine the log git log or even better git log --pretty=oneline
• choose the commit that immediately precedes the one that starts your piece of work (probably the last one before you branched to start work on feature X)
• start an interactive rebase session - this is git-fancy-talk for picking some commits and typing a message - git rebase -i THE-COMMIT-ID-OF-THE-COMMIT-YOU-SELECTED-ABOVE
• Your favourite editor (or nano) will open listing the commits for feature X - edit the word “pick” to “squash” for all except the top entry, save and exit
• Your favourite editor will open again - again showing your commit messages - add a line at the top stating something like “implementation of feature X” and leave the list of individual commits on separate lines below it (adding in a Fixes #123 or whatever your issue tracker demands as the last line), save and exit
• Now if you do a git log you will see a single commit with a nice “implementation of feature X” log message
• And you’re all set to merge to your master branch without cluttering its history with a long string of commit messages

A project I’ve been working on was recently moved up to Rails 2.2 (Rails 2.3 migration coming soon, but we wanted to take things one step at a time).

All the tests passed. Poking it on the staging server worked well. On to production and all was good.

Time passed.

Then, the app needed to be moved to a new server. Actually to a cluster. Slightly more nerve-wracking, but some playing around with the ghost gem and DNS settings and the deployment completed.

More time passed.

And then disaster struck. Well, not disaster, but it wasn’t good. Not at all.

The app had a rake task that was cronned to run every night. And it failed silently. We found out the day after and I scanned the production log file looking for evidence of what went wrong. Nowt. It didn’t even look like the rake task had run - but it had, as I had the data in front of me.

And the day after, it failed again. Another scan of the production log revealed nothing. Things were getting serious.

I suspected the cluster deployment. Maybe cron didn’t have permission to write to the log file, even though Passenger did. But everything looked good on that front. Maybe there was something else weird about the cluster setup - all the cron jobs were set to run on one of the app-servers; maybe moving it would make a difference?

To test this, I opened the Rails console and started manually recreating what the rake task was doing, on a different app-server. Which is when I noticed that script/console wasn’t writing to the production log either. What?

After a couple of hours of head scratching, I finally found out that the cluster deployment was not at fault. In fact, it seems to be the move to Rails 2.2 that had happened many weeks before - in particular it appears that the logger object no longer auto-flushes itself after writing.

We added a quick Rails.logger.auto_flushing = 1 to an initialiser and both script/console and the rake tasks wrote to the log as expected. Phew!

For some reason ferret server on a site I have been managing has been giving me a load of grief.

I seem to have nailed it down to two problems.

Firstly, monit only seems to like monitoring ferret server under certain circumstances.

Secondly, the ferret indexes don’t seem to like being rebuilt.

The first one is weird … my monit config looks like this:

check process ferret with pidfile /home/rails/myapp/current/log/ferret.pid
  start program = "/home/rails/myapp/current/script/ferret_server -e production start"
  as uid rails and gid rails
  stop program  = "/home/rails/myapp/current/script/ferret_server -e production stop"
  as uid rails and gid rails
  group ferret
  if failed port 9010 type TCP then restart
  if 5 restarts within 5 cycles then timeout

But monit often says “not monitored”. And completely fails to restart the server if it does go down.

No solution to that as yet.

The second problem is even weirder.

If I stop the server, then rebuild the indexes (I have a custom rake task that rebuilds all the indexes for my app) and restart, every search crashes ferret.

If I rebuild the indexes while the server is running, it appears to run fine. Why? I have absolutely no idea.

Writing estimates up-front is a really tricky part of client work.

From the customer’s point of view it’s pretty essential. You need to know how much you are spending before the work begins so you don’t get stung.

From the developer’s point of view it’s pretty difficult to do because you don’t know how long things will take until you know what you’re building. And you don’t know the details of what you are building until they become apparent, which is normally while you are doing the work.

For this particular client I wanted to get this piece of work out of the way as quickly as possible and the requirements were deceptively complicated. As trying to deal with intangibles is a recipe for disaster, and time was short, I was pretty sure that whatever estimate I came up with would be totally inaccurate.

So to deal with the intangibles I thought I may as well get stuck in with the actual work. And what’s the first thing that I do? Write a cucumber story.

I went through the requirements documents (which after weeks of to and fro were actually quite detailed and refined) and broke it down into four features, each with a number of scenarios. Each scenario was then broken down into individual steps - however, I did not write any step definition (ruby) files. This is cucumber as customer documentation, not as integration testing.

We agreed that the features met the requirements (the client was very impressed with the level of detail and the clarity that cucumber offered) and I simply went through each feature, scenario and step and came up with a price for the step individually. It’s still guesswork, but it’s guesswork on a small scale, so you’re less likely to be way off. Total the lot and there’s your estimate.

And even better, once the client has agreed the price and you’ve started development, you’ve got a development plan, a measure of progress and proof that it all works - all just a simple rake features away.

Image by lemon drop

I’ve just been playing with emacs for the first time in about twelve years. All was good apart from I couldn’t get the meta-key to work. So Ctrl-B works as “move backwards” but Meta-B did not work as “move backwards one word”. Which is really annoying.

A quick hunt through Terminal.app’s preferences though and all is revealed.

Just tick the box and glory in all your meta-goodness.

I have just written a load of test code that needed to verify that a particular set of classes behaved correctly when a transaction was rolled back.

However, the rest of my suite relied on transactional fixtures (which is Rails’ badly named way of saying that a transaction is started before each test and rolled back at the end, leaving your test database in a pristine state before the next case is run).

In particular, my spec_helper.rb had the following:

Spec::Runner.configure do |config|
  config.use_transactional_fixtures = true
  # stuff
end

The code being tested looked something like this:

begin
  Model.transaction do
    do_something # may fail with an exception
    do_something_else # may fail with an exception
  end
rescue Exception => ex
  do_some_recovery_stuff
end

I had a spec for the successful path (checking that the outcomes of do_something and do_something_else were what I expected.

However when I tried the same for the failure paths, the outcomes matched the successful path. The time-tested debugging method of sticking some puts statements in various methods showed that do_some_recovery_stuff was being called as expected. But the outcomes were still wrong.

And the reason? Transactions. This was a Rails 2.2 project, running on Mysql (innodb). As RSpec/Test::Unit starts a transaction before the specification clause runs (and then rolls it back on completion) when Model.transaction statement is reached, the spec is actually starting a second transaction, nested within RSpec/Test:Unit’s. Which means when the inner transaction is rolled back, the database doesn’t actually do anything - there’s still an outer transaction that may or may not be rolled back. (I think Rails 2.3 corrects this behaviour and if you roll back an inner transaction then the outer transaction reflects the correct state, but I’m not 100% on that).

So I had a choice - move the (production) app to Rails 2.3 to fix this one bug (which is very urgent) or figure out how to switch the outer transaction off for these particular steps. Google wasn’t very helpful - lots of stuff on how RSpec extends Test::Unit, lots of stuff on how Rails extends Test::Unit to add the fixtures (and transaction) support. But no concrete example on how to actually switch it off.

After much playing around (overriding methods like use_transaction?(method_name) and runs_in_transaction?) I eventually stumbled across the answer. And it’s pretty simple.

Set your default to be config.use_transactional_fixtures = true in spec_helper.rb. Then, for the specs that are not transactional, simply create a describe block and simply add the following:

describe MyClass, "when doing its stuff" do
  self.use_transactional_fixtures = false
  it "should do this"
  it "should do that"
  it "should do the other"
end

The only thing to be aware of is you may well need an after :each block to clean up after yourself.

When you’ve got an application that has little or no test coverage it can be quite daunting making changes. What if you alter X and it breaks Y? Without running through the entire app by hand how will you know what you’ve broken?

Well you won’t.

Even worse, what if your client reports a bug in that application? That makes things even worse doesn’t it?

No. It’s actually an opportunity. Because even if your boss thinks “testing is a great idea, we’ll start on the next project when we’ve got more time”, a bug fix is one of those things where the time to fix varies. So take advantage of that.

Start by writing a test that reproduces some functionality in the same area that you know works.

This won’t be easy. You need to get the database into the correct state, set up the session correctly. To save you some time, try using an object factory - with the correct configuration you can concentrate on creating just the models you need without having to fill the entire database with test data.

Take a look at the code you are testing as you are writing the tests. But make sure your test checks the outcomes of that code, not the implementation - when using mocks it’s pretty easy to end up effectively rewriting the real code as a series of should_receive(:something) calls. Which looks great until you come to refactor, at which point it becomes a nightmare.

Get your test to pass. Remember this is a feature that works so it shouldn’t be that hard to get it to pass. And you are building some important foundations as you are setting up a configuration for your object factory a model at a time.

Once you’ve got the first test working, prove that it’s doing what it’s supposed to be doing. Comment out its of the implementation and watch it fail. If it doesn’t fail then you’ve got a problem - your test is testing something other than that particular implementation.

Now we’ve got a test that really checks your existing code. I like to add another test that checks the error handling in that existing code as well (there is error handling in your existing code isn’t there?) Follow the same process as before - test the outcomes (probably catching an exception if you’re at the model level, probably looking for a particular redirect and flash message at the controller level), and then prove that it works by commenting code out. Hopefully this should be pretty quick to write as most of the hard work was done when setting up the original test.

And finally we can get to the meat of it. Write a test that reproduces your bug (that is, your test exercises your app in the way expected and your app should fail). Again, most of your setup work should already be done, so it shouldn’t take too long. Now run the test and watch it fail. If it doesn’t fail then your test isn’t right.

After all of this, we are finally in a position to fix the bug.

Your test should prove that it’s been fixed. And prove that the bug won’t reappear in a future version. But you’ve also taken an important first step to wrapping your application in tests, making your life easier in future.

A while back Brightbox honcho, Jeremy Jarvis, tweeted about his best three decisions of the past two years. All three were big decisions (OK, two out of three).

Which got me thinking. It’s been almost two years since my last big decision (to quit my job and go freelance so I could work with Ruby full-time). Change is pretty much the only thing you can be sure of so you may as well make things happen rather than passively sitting by and waiting for them to happen to you.

Meaning it’s probably time for a big decision. But what is it to be?

A conversation between me and my (14 year old) daughter:

Me: Are any of your friends “into” computers? You know, writing stuff, making websites, coding?

Daughter: I think [girl's name] is. She does layouts and codes for web stuff.

Me: Really? Interesting. But just her then?

Daughter: Yeah.

Me: So why aren’t more of your friends into that sort of thing?

Daughter: I dunno. Just not that bothered I guess.

Me: What about the boys you know? Are any of them into computers?

Daughter: No, not really. Just [girl's name]. Why are you asking?

Me: Well this guy did a presentation at a conference with some saucy images on the slides. And now a load of people are up in arms saying “it’s this sort of thing that makes women not get involved with technology”.

Daughter: Oh no, it’s nothing like that. It’s just not very interesting is it?