RSpec, Cucumber, Webrat, RCov and Autotest are a powerful combination of tools for testing your Rails app. Unfortunately getting them to all work nicely together can be a bit of challenge. I recently configured a development environment from scratch on OS X 10.5 Leopard and kept track of all of the little details.
Prerequisites
I’m assuming you’ve got the following installed:
- ruby
- ruby gems 1.3.1
- Apple development tools
- git
- rails >= 2.3.2
- You’ve added github to your gem sources (gem sources -a http://gems.github.com)
RSpec & RSpec-Rails
First let’s grab the rspec1 and rspec-rails2 gems.
sudo gem install rspec
sudo gem install rspec-rails
Cucumber
Next we’ll install the cucumber3 gem
sudo gem install cucumber
Webrat
Webrat4 is used by cucumber to simulate a browser for your integration tests. Webrat will also install nokogiri5.
sudo gem install webrat
RCov
I thought RCov6 would get installed with RSpec, but it wasn’t for me. You might not need to do this, but just to make sure…
sudo gem install rcov
Autotest
Autotest7 comes from ZenTest8 and allows you to have a kick ass workflow where you are constantly running relevant tests and less-constantly automatically running your entire test suite.
sudo gem install ZenTest
Optionally, Thoughtbot’s Factory Girl
Factory girl9 is a really helpful fixture replacement (and more) gem to use in conjunction with cucumber, checkout their much better explanation
sudo gem install thoughtbot-factory_girl --source http://gems.github.com
Optionally, Carlos Brando’s Autotest Notification
While autotest normally runs in a terminal window, it can be setup to hook into applications like growl or snarl. The Autotest Notification9 gem helps make this setup a lot easier.
You will need growl installed and configured for this step the installation instructions on this gems github page are very easy to follow.
sudo gem install carlosbrando-autotest-notification --source=http://gems.github.com
Next you need to turn autotest notifications “on”
an-install
A Sample Rails App
Let’s create a sample rails app for the rest of this guide.
rails sample-app
Configuring Environment Variables
Autotest relies on some environment variables to run all of your features and specs correctly. If autotest “hangs” after you try to run it, or it just never seems to be watching your specs or features, this will most likely solve your problem.
Open the test.rb environment definition file in sample-app/config/environments/test.rb and add the following.
1 2 | ENV['AUTOFEATURE'] = "true" ENV['RSPEC'] = "true" |
These lines will test autotest to run, and look for changes to, your specs (rather than test unit tests) and your cucumber features.
Update
If you don’t want to add these environment variables to every rails project you’ve got on your machine, you can also choose to set them as environment variables in your .bash_profile or .bashrc (or whatever shell you’re using) files.
export AUTOFEATURE=true export RSPEC=true
Unpacking Gems
Next let’s freeze (unpack) some gems that we’ll be using in our app. I’ve run into problems trying to use the system gems with cucumber, rspec and webrat, especially when I have multiple versions of any of them installed. Unpacking them into my rails app solves this problem for me.
mkdir sample-app/vendor/gems cd sample-app/vendor/gems gem unpack rails gem unpack rspec gem unpack rspec-rails gem unpack cucumber
Because webrat (and nokogiri) are native gems, that is, they are built locally on your machine based on its architecture, we won’t unpack those.
config.gem support
The current accepted practice, when using rails 2.3, and as suggested by the rspec guy(s) is to use rails’ config.gem functionality.
Open sample-app/config/environments/test.rb and add the following lines:
config.gem "rspec", :lib => false, :version => ">= 1.2.0" config.gem "rspec-rails", :lib => false, :version => ">= 1.2.0" config.gem "cucumber", :lib => false, :version => ">= 0.2.3" config.gem "thoughtbot-factory_girl", :lib => "factory_girl", :source => "http://gems.github.com" config.gem "webrat", :lib => false, :version => ">= 0.4.3" config.gem "nokogiri", :lib => false, :version => ">= 1.2.3"
Your version numbers may be different, but these are all current at the time of writing.
Boot Strapping RSpec and Cucumber
Before you can get very far with rspec or cucumber you need to run the bootstrapping scripts to give yourself the default files and directories.
# From inside your rails app sample-app/ script/generate rspec script/generate cucumber
Factories
Depending on where you’re going to use your factories the most, you might want to save your file in either spec/ or features/. I chose the latter. Only complete this step if you plan to use the FactoryGirl gem.
touch sample-app/features/factories.rb
Getting Accurate RCov Data
By default RCov is setup to only use your specs when calculating code coverage. If you’re using Cucumber and RSpec, you’ll obviously want to include both types of tests to calculate your project’s true code coverage.
I picked up this rcov rake task from my co-worker Jay McGavren it does all of the heavy lifting for you, we’ll just need to make a couple of changes.
Drop this file into sample-app/lib/tasks/rcov.rake and use it by calling rake rcov:all from your terminal.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 | require 'cucumber/rake/task' #I have to add this require 'spec/rake/spectask' namespace :rcov do Cucumber::Rake::Task.new(:cucumber) do |t| t.rcov = true t.rcov_opts = %w{--rails --exclude osx/objc,gems/,spec/,features/ --aggregate coverage.data} t.rcov_opts << %[-o "coverage"] end Spec::Rake::SpecTask.new(:rspec) do |t| t.spec_opts = ['--options', ""#{RAILS_ROOT}/spec/spec.opts""] t.spec_files = FileList['spec/**/*_spec.rb'] t.rcov = true t.rcov_opts = lambda do IO.readlines("#{RAILS_ROOT}/spec/rcov.opts").map {|l| l.chomp.split " "}.flatten end end desc "Run both specs and features to generate aggregated coverage" task :all do |t| rm "coverage.data" if File.exist?("coverage.data") Rake::Task["rcov:cucumber"].invoke Rake::Task["rcov:rspec"].invoke end end |
The important part here is on line 7, we want rcov to exclude our features directory. We obviously don’t need or want rcov telling us that our feature files are not “covered”. To solve this problem we’ve simply excluded the features directory from rcov’s processing.
We also need to slightly modify sample-app/spec/rcov.opts to get the full rspec + cucumber coverage data.
Your rcov.opts should look like this:
--exclude "spec/*,gems/*,features/*" --rails --aggregate "coverage.data"
We again want to ignore our cucumber features and we also want to tell rcov to aggregate data in a file called coverage.data. This is used in the above rake task.
Write Some Specs and Features!
Act like you know what you’re doing and write some models, controllers whatever. Add some specs and features too.
Autotest Workflow
Open a terminal and make your way to your sample rails app and fire up autotest. You might see something like the following, depending on how many specs and features you’ve got.
$> autotest loading autotest/cucumber_rails_rspec opts ... Finished in 0.06276 seconds 3 examples, 0 failures ================================================================================ /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/bin/ruby /Library/Ruby/Gems/1.8/gems/cucumber-0.2.3/bin/cucumber --format progress --format rerun --out /var/folders/Aq/Aqp06i3dFnqse+tQgQA+1++++TI/-Tmp-/autotest-cucumber.75956.0 features ................. 4 scenarios 17 passed steps /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/bin/ruby /Library/Ruby/Gems/1.8/gems/rspec-1.2.2/bin/spec --autospec spec/models/intern_spec.rb -O spec/spec.opts ... Finished in 0.062995 seconds 3 examples, 0 failures ================================================================================ /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/bin/ruby /Library/Ruby/Gems/1.8/gems/cucumber-0.2.3/bin/cucumber --format progress --format rerun --out /var/folders/Aq/Aqp06i3dFnqse+tQgQA+1++++TI/-Tmp-/autotest-cucumber.75956.1 features ................. 4 scenarios 17 passed steps
The REALLY important stuff
- make sure you’ve got “ENV['AUTOFEATURE'] = true” in your test.rb otherwise autotest won’t run your features automatically
- make sure you’ve got “ENV['RSPEC'] = true” in your bash profile or else autotest won’t run your specs automatically
- make sure you’ve got “–aggregate = ‘coverage.data’” in your spec/rcov.opts file if you’re going to use the above rake task and hope to get combined rcov coverage data between rspec and cucumber
- make sure you’re excluding the features directory from rcov where required or else you’ll end up with misleading rcov data.
Gem Versions
Here’s a list of the current gems and their versions that I used in preparing this guide.
*** LOCAL GEMS *** actionmailer (2.3.2, 1.3.6, 1.3.3) actionpack (2.3.2, 1.13.6, 1.13.3) actionwebservice (1.2.6, 1.2.3) activerecord (2.3.2, 1.15.6, 1.15.3) activeresource (2.3.2) activesupport (2.3.2, 1.4.4, 1.4.2) acts_as_ferret (0.4.1) addressable (2.0.2) builder (2.1.2) capistrano (2.0.0) carlosbrando-autotest-notification (1.9.1) cgi_multipart_eof_fix (2.5.0, 2.2) cucumber (0.2.3) daemons (1.0.9, 1.0.7) data_objects (0.9.11) diff-lcs (1.1.2) dnssd (0.6.0) extlib (0.9.11) fastthread (1.0.1, 1.0) fcgi (0.8.7) ferret (0.11.4) gem_plugin (0.2.3, 0.2.2) highline (1.2.9) hpricot (0.6) libxml-ruby (0.3.8.4) mongrel (1.1.4, 1.0.1) mysql (2.7) needle (1.3.0) net-sftp (1.1.0) net-ssh (1.1.2) nokogiri (1.2.3) polyglot (0.2.5) rack (0.9.1) rails (2.3.2, 1.2.6, 1.2.3) rake (0.8.4, 0.7.3) rcov (0.8.1.2.0) RedCloth (3.0.4) rspec (1.2.2) rspec-rails (1.2.2) ruby-openid (1.1.4) ruby-yadis (0.3.4) rubynode (0.1.3) sources (0.0.1) sqlite3-ruby (1.2.1) term-ansicolor (1.0.3) termios (0.9.4) textmate (0.9.2) thor (0.9.9) thoughtbot-factory_girl (1.2.0) treetop (1.2.5) webrat (0.4.3) ZenTest (4.0.0)
El Fin
Hopefully this guide was useful or had that one little step that you needed to get everything working. I’m sure this will all be out of date in the coming weeks, but I’ll try to keep it as up-to-date as possible. If you see any errors, or can better explain some of the missing pieces, please post a comment. Thanks!
1 http://github.com/dchelimsky/rspec/tree/master
2 http://github.com/dchelimsky/rspec-rails/tree/master
3 http://github.com/aslakhellesoy/cucumber/tree/master
4 http://wiki.github.com/brynary/webrat
5 http://github.com/tenderlove/nokogiri/tree/master
6 http://rubyforge.org/projects/rcov/
7 http://www.zenspider.com/ZSS/Products/ZenTest/#rsn
8 http://www.zenspider.com/ZSS/Products/ZenTest/
9 http://github.com/thoughtbot/factory_girl/tree/master
10 http://github.com/carlosbrando/autotest-notification/tree/master
Updates
2009-12-08 – Removed “sudo” when describing how to unpack gems (h/t xdotcommer)
GAH autotest needs to be replaced its so annoying
The only other thing I’ve ever used was RSpactor, but I’m not sure of its current development.
Excellent writeup, nice and thorough. Thanks!
Instead of setting ENV['RSPEC'] you can just run the autospec script that’s included in the rspec gem.
had a problem with
sudo gem unpack rails
got this error
config.gem: Unpacked gem rails-2.3.2 in vendor/gems has no specification file. Run ‘rake gems:refresh_specs’ to fix this.
instead I ran
rake rails:freeze:gems
which put it into sample-app/vendor/rails and all worked fine
cheers
Michael,
I’ve experienced the same problem on Ubuntu, but for whatever reason i didn’t get this error when I did my setup on Leopard. I think I’ve written it off as a problem with the gem dependency stuff in rails so I’ve just ignored the errors. I’ll try your fix right now on my Linux box, thanks!
BTW, if you set up gems with config.gem, you can use these Rake tasks:
rake gems:unpack # Unpacks all required gems into vendo
r/gems.
rake gems:unpack:dependencies # Unpacks all required gems and their
dependencies into vendor/gems.
I think I just saw on the Cucumber Wiki where they were recomending spicy-rcov instead of the plain rcov. Have you heard anything about the differences?
Wayne
I was working with a rails 2.1.2 project that wouldn’t run rcov, it kept failing with a variety of errors. Installing the spicy-rcov gem was the only thing that worked. I think it has something to do with me being on the default 1.8.6 version that ships with OS X 10.5
I walked through this guide and got everything setup but I’m having a bit of a problem with rcov. When I run rake rcov:all it makes a coverage.data file. How do I get that to output html? Or how am I suppose to use that coverage.data file?
The coverage.data file is generated when the cucumber features are processed. The idea being that you aggregate all of the rcov results for cucumber, then run rcov for the regular specs and finally combine the two to get your total cucumber + rspec rcov score. If you’re just trying to run rcov for the cucumber features, the rcov rake task that is described here might not work, I’ve always run both features and specs to generate my rcov as I’ve yet to work on a project that only had cucumber for testing.
How do I do that? I mean, how do you use your coverage.data file? I’m simply missing something conceptual or a step that goes from coverage.data to something useful.
So the coverage.data file that is generated is eventually used to create the files in the coverage/ directory. If you open the index.html file that’s in
RAILS_ROOT/coverageyou’ll see your rails apps’ code coverage information.This is very comprehensive tutorial for setting up BDD.
It would be great if you can include the libraries install comannd before webrat. It failed for me at the webrat installation and I discovered following.
sudo aptitude install libxslt-dev libxml2 libxml2-dev
Very nice tutorial. It Helps.
Regards,
Rajan
Wow, that was an intimidating amount of work you had to do. Good documentation!
As far as RCov goes, the rcov gem from rubyforge is abandonware. I am the current maintainer of RCov now and the latest bits are found in the relevance-rcov gem.
gem install relevance-rcov -s http://gems.github.com
Im trying to get this running at the minute and i currently get the following errors
AUTOFEATURE=true RSPEC=true autotest > /dev/null
Warning: $KCODE is NONE.
/opt/local/lib/ruby/gems/1.8/gems/treetop-1.3.0/lib/treetop/ruby_extensions/string.rb:31: warning: method redefined; discarding old indent
/opt/local/lib/ruby/gems/1.8/gems/treetop-1.3.0/lib/treetop/compiler/node_classes/character_class.rb:13: warning: useless use of a literal in void context
/opt/local/lib/ruby/gems/1.8/gems/treetop-1.3.0/lib/treetop/compiler.rb:5: warning: global variable `$exclude_metagrammar’ not initialized
/opt/local/lib/ruby/gems/1.8/gems/treetop-1.3.0/lib/treetop/compiler/metagrammar.rb:232: warning: method redefined; discarding old space
/opt/local/lib/ruby/gems/1.8/gems/treetop-1.3.0/lib/treetop/compiler/metagrammar.rb:388: warning: method redefined; discarding old space
/opt/local/lib/ruby/gems/1.8/gems/treetop-1.3.0/lib/treetop/compiler/metagrammar.rb:771: warning: method redefined; discarding old space
/opt/local/lib/ruby/gems/1.8/gems/treetop-1.3.0/lib/treetop/compiler/metagrammar.rb:779: warning: method redefined; discarding old space
/opt/local/lib/ruby/gems/1.8/gems/cucumber-0.3.96/lib/cucumber/parser/feature.rb:28: warning: method redefined; discarding old white
/opt/local/lib/ruby/gems/1.8/gems/cucumber-0.3.96/lib/cucumber/parser/feature.rb:36: warning: method redefined; discarding old white
/opt/local/lib/ruby/gems/1.8/gems/cucumber-0.3.96/lib/cucumber/parser/feature.rb:679: warning: method redefined; discarding old white
/opt/local/lib/ruby/gems/1.8/gems/cucumber-0.3.96/lib/cucumber/parser/feature.rb:687: warning: method redefined; discarding old white
/opt/local/lib/ruby/gems/1.8/gems/cucumber-0.3.96/lib/cucumber/parser/feature.rb:813: warning: method redefined; discarding old white
/opt/local/lib/ruby/gems/1.8/gems/cucumber-0.3.96/lib/cucumber/parser/feature.rb:825: warning: method redefined; discarding old white
/opt/local/lib/ruby/gems/1.8/gems/cucumber-0.3.96/lib/cucumber/formatter/pretty.rb:195: warning: method redefined; discarding old cell_prefix
/opt/local/lib/ruby/gems/1.8/gems/activesupport-2.3.3/lib/active_support/core_ext/blank.rb:49: warning: method redefined; discarding old blank?
rake features works fine
rake spec works fine
but autotest fails have you come across this and maybe have a solution
I forked the RCov task on github, and added tasks that allow you generate separate rspec or cucumber coverage reports along with the aggregate.
Download at http://gist.github.com/231022
rake rcov:allfor the aggregate coveragerake rcov:rspecfor rspec coverage onlyrake rcov:cucumberfor cucumber coverage onlyClayton,
I wrote a Rails template with an approach to testing inspired from this blog post.
Check it out: http://github.com/mchung/yart
rcov only runs the tests for me, it doesn’t actually give me any coverage data when I run rake rcov:rspec. What gives?
Very informative, thanks
I also needed to do a “sudo gem install autotest-rails”
Sorry for posting this here, but I’m stumped with this curiosity and I’m not sure where else to post it. I configured autotest to run RCov as soon as all of my Cucumber tests pass. I did this using one of autotest’s hooks (http://gist.github.com/555162). That hook basically calls up the following rake task (http://gist.github.com/555163).
My problem with this though is that the tests get run twice. Once during the run_all_tests method of autotest, and then another time when the above rake task is called.
Is there a way to force this to just run the rake task once prior to displaying RCov results? Very minor problem if at all, I know, but I can’t stop obsessing over it!
I’m still very new to all this but I like Machinist a LOT more than Factory Girl. It just seems a more natural way to do things but of course it’s a personal thing.
What’s the Rails 3 equivalent? No more
Spec::Rake::SpecTask
I am trying to run my rspec test suite for models & always shows 8 failure in Ubuntu box
>> rspec spec/models
But same thing works well and gives 0 failures on windows box. Tried more than 10 times, but same result. Any clues on this? Using
Ubuntu 10.04
Rails 3
Rspec 2.6.4
Possible alternatives given the passage of time since the original post:
RSpec -> still RSpec!
Cucumber -> for rails, maybe Steak (“a minimal extension of RSpec-Rails that adds several conveniences to do acceptance testing of Rails applications using Capybara.”)
Webrat -> maybe Capybara (built to work nicely with Cucumber)
RCov -> use SimpleCov in ruby 1.9
Autotest -> maybe Guard?? haven’t tried it