Comments on: Testing Anti-Patterns: Overspecification http://jasonrudolph.com/blog/2008/07/01/testing-anti-patterns-overspecification/ puts Blog.new("nonsense") Thu, 04 Dec 2008 19:56:09 +0000 http://wordpress.org/?v=2.6.3 By: Jason Rudolph http://jasonrudolph.com/blog/2008/07/01/testing-anti-patterns-overspecification/#comment-14384 Jason Rudolph Wed, 02 Jul 2008 03:21:47 +0000 http://jasonrudolph.com/blog/?p=179#comment-14384 <p>Hi S.R.,<br/> You bring up a good point, but there's a difference between a robust set of well-specified tests and a bloated set of overspecified tests. The example I present above is a bloated and overspecified test. It's <em>aimed</em> specifically at testing the controller, and yet it include all sorts of logic about how to create, fetch, and validate a model object. I certainly want to be testing the model logic I write, but I want to perform those tests in a model test class, not a controller test class. <br/><br/> I like the example you mention regarding the <code>#sum</code> method. Let's take a closer look at that scenario. You argue that the developer could be adversely swayed by looking at the code before writing the test. There's an easy solution to this problem: just write the tests first. :-) Given a test-driven approach to development, I would have <em>zero</em> knowledge of the implementation up front. Therefore, I'd write all the tests necessary to fully exercise a method whose task is to take to values and return their sum: I'd write a test for a simple case (e.g., <code>sum(2, 1)</code>), a test with a negative value (e.g., <code>sum(-2, 1)</code>), a test with two negative values (e.g., <code>sum(-2, -1)</code>), a test where one value is nil, a test where both values are nil, etc. After writing each test, I'd implement just enough code to satisfy the test and nothing more. (Presumably this task would be trivial for simple math like we're talking about here, but the same idea extends to all code.) <br/><br/> I certainly share your desire to make sure my tests are catching bugs, but I can catch those bugs without polluting my controller tests with model-specific assertions. Take a good unit test suite that keeps your individual building blocks (e.g., models, controllers, library classes) in check, and couple it with a few good integration tests to make sure they all work together, and you've got a good set of developer tests to support your continuing development efforts. <br/><br/> I hope this helps. <br/><br/> Cheers,<br/> Jason</p> Hi S.R.,
You bring up a good point, but there’s a difference between a robust set of well-specified tests and a bloated set of overspecified tests. The example I present above is a bloated and overspecified test. It’s aimed specifically at testing the controller, and yet it include all sorts of logic about how to create, fetch, and validate a model object. I certainly want to be testing the model logic I write, but I want to perform those tests in a model test class, not a controller test class.

I like the example you mention regarding the #sum method. Let’s take a closer look at that scenario. You argue that the developer could be adversely swayed by looking at the code before writing the test. There’s an easy solution to this problem: just write the tests first. :-) Given a test-driven approach to development, I would have zero knowledge of the implementation up front. Therefore, I’d write all the tests necessary to fully exercise a method whose task is to take to values and return their sum: I’d write a test for a simple case (e.g., sum(2, 1)), a test with a negative value (e.g., sum(-2, 1)), a test with two negative values (e.g., sum(-2, -1)), a test where one value is nil, a test where both values are nil, etc. After writing each test, I’d implement just enough code to satisfy the test and nothing more. (Presumably this task would be trivial for simple math like we’re talking about here, but the same idea extends to all code.)

I certainly share your desire to make sure my tests are catching bugs, but I can catch those bugs without polluting my controller tests with model-specific assertions. Take a good unit test suite that keeps your individual building blocks (e.g., models, controllers, library classes) in check, and couple it with a few good integration tests to make sure they all work together, and you’ve got a good set of developer tests to support your continuing development efforts.

I hope this helps.

Cheers,
Jason

]]> By: S.R. http://jasonrudolph.com/blog/2008/07/01/testing-anti-patterns-overspecification/#comment-14381 S.R. Tue, 01 Jul 2008 15:30:48 +0000 http://jasonrudolph.com/blog/?p=179#comment-14381 <p>There is one thing to point out about overspecificated approach - it doesn't rely on any knowledge about main code and therefore it will potentially find more bugs. </p> <p>Example: </p> <p>def sum a - b end </p> <p>Looking in the code test engineer would say : "ah , it does a MINUS b so my test case would be: "</p> <p>def test_sum assert sum(2 , 1) == 1 end </p> <p>This is of cause an exaggeration but I can totally see that happening in more complicated cases.</p> <p>I'd be interested to know your opinion on this, thanks.</p> There is one thing to point out about overspecificated approach - it doesn’t rely on any knowledge about main code and therefore it will potentially find more bugs.

Example:

def sum a - b end

Looking in the code test engineer would say : “ah , it does a MINUS b so my test case would be: “

def test_sum assert sum(2 , 1) == 1 end

This is of cause an exaggeration but I can totally see that happening in more complicated cases.

I’d be interested to know your opinion on this, thanks.

]]>