Monday, July 14, 2008

test/unit, test/spec, RSpec, shoulda

Who has experience with two of these test frameworks, and can tell about this? Which one is better, and why?

2 comments:

Benjamin Behr said...

I already used RSpec, test/unit and shoulda.

For me RSpec is a nice framework if you want to have very fine grained tests. This fine grained tests are achieved by mocking and stubbing everything but the aspect you want to test. On the other side I am wondering if this level of isolation is a good tool for testing rails-apps. If I'm mocking and stubbing everything and my tests are green, I just need to make a change inside my model and my tests will stay green for many parts of the application although they are effected by that little change.

test/unit has a much more technical approach to write tests. There are no nice and descriptive error messages like in RSpec or shoulda. The approach to test with test/unit is not to isolate each test but to test the method in the context of the other methods. If you change something in your model for example, any other test which calls the changed method may fail. With that knowledge I trust my tests earlier than I do if I only have got one test that could (!!!) fail if something doesn't work as defined.

And last but not least: shoulda. I don't want to call shoulda a test-framework since it is a plugin for test/unit which adds some nice features to the test/unit-framework. shoulda allows us to set up contexts which equals the describe-blocks in RSpec. It also allows us to use the "it"-blocks from RSpec to define nice error-messages if a test fails. shoulda combines the advantages of both frameworks and makes it easier to test rails-apps because of adding some nice macro-methods which test a lot of common things you use in your daily work with rails.

Anonymous said...

Just wrote about this here: http://kylebanker.com/blog/2008/08/from-rspec-to-shoulda/

Having used RSpec for a while, I'm not entirely comfortable with the way it uses mocking.

I like Shoulda for its syntax and shortcuts.