Generating code coverage metrics for a Ruby on Rails project with simplecov

As part of my dive into unit testing Rails applications I was keen to set up a tool to give me code coverage stats.  Code coverage represents the % of your source code that your unit tests exercise.  100% code coverage is a good goal to have and the earlier you hit it, the more likely you’ll find testing a worthwhile endeavour.  Whilst 100% coverage doesn’t guarantee that you’ve tested every permutation in your app, not having 100% coverage guarantees there are bits where daemons may be lurking.

Setting up code coverage reports with Simplecov

I first looked at rcov, however, as soon as you run it, you get a lovely warning telling you that Ruby 1.9 support is experimental and that the results aren’t guaranteed to be correct! So I quickly moved on and found simplecov which has full support for 1.9.

To get started, add the gem to your gemfile:

gem 'simplecov'

Then run a ‘bundle install’ to grab the gem and set stuff up.  Next we need to hook simplecov into our tests.  Simplecov works with a range of test frameworks, so, depending on your framework of choice, the next instruction varies slightly.  If you’re using RSpec like me you need to open spec_helper.rb, if you’re running the default Rails test tools find test_helper.rb or if you’re using Cucumber find env.rb (other frameworks will have a similar initialization file).

At the very top of your helper file, enter

require 'simplecov'
SimpleCov.start do
end

Now, whenever you run your test suite, code coverage will be calculated and a fancy report will be available to view in coverage/index.html.  How easy was that!

We’re not done quite yet though, there are a few tweaks we can make to improve things further.  First up, we don’t want to include all of the stuff around the edge of a Rails app, for example, the config dir.  We can filter these bits out by adding some rules inside the initialisation block.

require 'simplecov'
SimpleCov.start do
  add_filter '/spec/'
  add_filter '/config/'
  add_filter '/lib/'
  add_filter '/vendor/'
end

I’ve included the /spec/ folder to be clear that I dont want coverage metrics run on my tests. If you re-run your test suite you should now see slightly cleaner results. simplecov has a powerful filtering engine that lets you define far more complex rules than these. For example, you can exclude short files that only have a couple of lines.

Our next tweak will be to make use of simplecov groups. Groups let you group together similar file types so that you can view them separately. In the example below you can see how I’ve set up groups for Models, Controllers, Views, Mailers and Helpers.

require 'simplecov'
SimpleCov.start do
  add_filter '/spec/'
  add_filter '/config/'
  add_filter '/lib/'
  add_filter '/vendor/'

  add_group 'Controllers', 'app/controllers'
  add_group 'Models', 'app/models'
  add_group 'Helpers', 'app/helpers'
  add_group 'Mailers', 'app/mailers'
  add_group 'Views', 'app/views'
end

Again, this only touches the surface of what you can do with simplecov. You can create far more complex rules to slice and dice your codebase as you wish.

There is one final tweak which you may find helpful, especially as your codebase grows. Running coverage metrics is SLOW! Much slower than just executing the test suite. So, you might not want to always run your coverage analysis – especially if you’re using a tool like autotest to re-run the tests every time the file system changes.

To make running coverage conditional, we’ll check for an environment variable before initialising simplecov.

SimpleCov.start do
  # rules here
end if ENV["COVERAGE"]

Now I can run coverage on demand by specifying the COVERAGE environment variable

$ COVERAGE=true bundle exec rake spec