A Case Against Cucumber

Cucumber has been a popular tool in Ruby’s TDD community for a while now. It offers a way to write tests in a way that anybody can understand, regardless of technical ability.

With the Gherkin syntax, you can write tests that prove to your product owners that your features work. Better yet, they can write their stories in Gherkin so you can just copy and paste them into some Cucumber files and implement the tests.

Reality

This all sounds well and good, but are any of the benefits of Cucumber really all that beneficial? Let’s start with the most obvious.

  Given I am a product owner
  When there are awesome cukes
  Then I can read the code!

In theory, this is a valuable thing, a bridge between the divergent worlds of developers and managers. In practice, however, I’ve never seen Cucumber used this way. Non-technical people don’t read code, no matter how easy it is to read. They care about the actual use cases and that means using the application. And if they use the application, who cares if there is some text claiming the application works!

So what else might Cucumber be good for?

  Given I am a product owner
  When I know how to write awesome cukes
  Then I can save time for developers!

This is another potential use case that I have heard of for Cucumber. The problem with this conclusion is that it forgets about the overhead for writing cukes. When I write a Cucumber feature, I have to write the Gherkin that describes the acceptance criteria, and the Ruby code that implements the step definitions.

Since the code to implement the step definitions is just normal RSpec (or whichever testing library you use), if someone else is writing the Gherkin, the amount of setup to create a working test should be about the same. So you’re only breaking even!

However, I don’t believe that it would really be breaking even. Cucumber adds another layer of indirection on top of your tests. When I’m trying to see why a specific scenario is failing, first I need to find the step that is failing. Since these steps are defined with regular expressions, I have to grep for the step definition, which in the above example could look like this:

Given(/^I am a product manager$/) do
  # make me a product manager
end

or like this:

Given(/^I am a (\w+)$/) do |user_type|
  # make me this user type
end

Have fun grepping for the latter. Chances are someone added a couple other step definitions of the form “I am a”.

Okay, so at its best, with a little help, Cucumber is about as good as some good ol’ RSpec. So what else can we do with Cucumber?

Given I am a user
When I go to some really cool page
And I click on some really cool buttons
Then I should see some really cool content

I’ll defer the discussion about testing your views within your normal test suite and instead focus on the capability to do so. This functionality is most commonly used with the cucumber-rails gem, which includes Capybara to do the view testing.

The problem with this approach is that, if you really need to, you can include Capybara in your project and test through the view in your RSpec tests. Either way, it’s all just a little Ruby and RSpec to set it up and use it. Why add the extra layer of complexity that Cucumber brings when you can do the same with less?

Given a set of requirements in the Gherkin syntax
When I write acceptance tests for my application
Then why should I also write my tests in Gherkin

I don’t prefer to use Cucumber for any of my testing, but I do enjoy the Gherkin syntax. Not for testing, but for gathering feature requirements. It provides a very clear and concise way of explaining a feature, without confusion. But that is where the line is drawn.

Cucumber is just a way to wrap RSpec tests with a non-technical syntax. Any supposed benefits of it go to waste because code is only interesting to those who are working in it. Quit writing cukes unless you can honestly say that there is someone reading them who would not understand pure Ruby.

Kevin Liddle, Software Craftsman

Kevin Liddle is a craftsman and enjoys physics and trivia.