After DoubleDawg 3, sam posted a gnarly question:
Should you not have a test in place that ensures that the processAnnotations method does not return null? After all someone could refactor it to return null, and now the changes you made would break. With a test at least they would know that it was not supposed to return null…
That’s wicked good, and props to Sam for wondering it. Here’s the short answer:
Should we test that processAnnotations never returns null?
Gnarly: I currently judge No.
Before I go any deeper on this, there are two parts of this answer I want to draw your attention to:
I use the term gnarly in the sense explained by Rudy Rucker in his book The Lifebox, The Seashell, And The Soul. He uses the word to indicate a computation for which there is no faster evaluation technique than itself. It’s rude of me to try and sum up gnarliness in a single sentence, but that’s it. (I also like the term because it reminds me of Fast Times At Ridgemont High.) For our shorthand, then, to say a question is gnarly is to say that there is no way to answer it definitively other than waiting to see what happens.
I also used the phrase currently judge. When I give an answer to any gnarly question, I am thrown back onto my judgment because there’s no better evaluation than that. By saying currently judge, I’m really reminding folks that a) it’s just a guess salted through with my instincts and experience, and b) I could easily change my mind later.
Factors I Considered
Okay, so, with that out of the way for now, let me describe the factors I consider in the judgment call of what to microtest.
My first thought was to say yes, because it would be more complete to cover that case. But then I remembered, the purpose of microtesting isn’t completeness, it’s productivity. I am doing this for money, not purity. Purity is for soap and scotch.
Then I thought about the risk factor. I judged the risk of someone on my team refactoring the code so as not to return a null is very small. If I’m not pure, the only sane thing is to calculate the odds and play them.
Then I thought about the fact that the function in question is private. This raised two related ideas: 1) that if I finish refactoring a class so that it has a significant untested private, I am being derelict, and 2) wow, I wonder if I can really stick to that. It’ll be a fun test.
Finally, I thought about the fact that we’re just getting started with this bastard. Truly proving that it can’t return null would require recreating the genuinely awesome McCabe complexity of the code right in its tests. When combinatorics threaten my testing, the answer is usually in refactoring or redesigning.
So there you have it. In my current situation, the price of rigorously proving that processAnnotations never returns null is quite high, and the benefit seems quite low, but to tell you the truth: I am falling back on my intuition, as we all do when confronting gnarly questions, and I am making a judgment call.
A programmer has to make judgment calls like these every working day.
(Note to self: Start writing your principles articles so you can start aiming at them, butthead.)