Feed on

After DoubleDawg 3, sam posted a gnarly question:

Should you not have a test in place that ensures that the processAnnotations method does not return null? After all someone could refactor it to return null, and now the changes you made would break. With a test at least they would know that it was not supposed to return null…

That’s wicked good, and props to Sam for wondering it.  Here’s the short answer:

Should we test that processAnnotations never returns null?

Gnarly: I currently judge No.

Before I go any deeper on this, there are two parts of this answer I want to draw your attention to:

I use the term gnarly in the sense explained by Rudy Rucker in his book The Lifebox, The Seashell, And The Soul.  He uses the word to indicate a computation for which there is no faster evaluation technique than itself.  It’s rude of me to try and sum up gnarliness in a single sentence, but that’s it.  (I also like the term because it reminds me of Fast Times At Ridgemont High.)  For our shorthand, then, to say a question is gnarly is to say that there is no way to answer it definitively other than waiting to see what happens.

I also used the phrase currently judge. When I give an answer to any gnarly question, I am thrown back onto my judgment because there’s no better evaluation than that.  By saying currently judge, I’m really reminding folks that a) it’s just a guess salted through with my instincts and experience, and b) I could easily change my mind later.

Factors I Considered

Okay, so, with that out of the way for now, let me describe the factors I consider in the judgment call of what to microtest.

My first thought was to say yes, because it would be more complete to cover that case.  But then I remembered, the purpose of microtesting isn’t completeness, it’s productivity. I am doing this for money, not purity. Purity is for soap and scotch.

Then I thought about the risk factor.  I judged the risk of someone on my team refactoring the code so as not to return a null is very small.  If I’m not pure, the only sane thing is to calculate the odds and play them.

Then I thought about the fact that the function in question is private.  This raised two related ideas:  1) that if I finish refactoring a class so that it has a significant untested private, I am being derelict, and 2) wow, I wonder if I can really stick to that.  It’ll be a fun test.

Finally, I thought about the fact that we’re just getting started with this bastard. Truly proving that it can’t return null would require recreating the genuinely awesome McCabe complexity of the code right in its tests.  When combinatorics threaten my testing, the answer is usually in refactoring or redesigning.

So there you have it.  In my current situation, the price of rigorously proving that processAnnotations never returns null is quite high, and the benefit seems quite low, but to tell you the truth: I am falling back on my intuition, as we all do when confronting gnarly questions, and I am making a judgment call.

A programmer has to make judgment calls like these every working day.

(Note to self:  Start writing your principles articles so you can start aiming at them, butthead.)

7 Responses to “DoubleDawg 3A: A Gnarly Judgment Call”

  1. sam says:

    thanks for the full reply. These completeness v pragmatic issues are ones that, with limited time and deadlines to meet, often have to be played off against each other. I think I would have probably made a similar call, and noted in the javadoc for the method that it will never return null. Then if someone did make it return null, I would at least have a little something to beat them with.

  2. Arielr says:

    Good read. I agree with all the “blah”.

    On the technical question at hand – the concept of code contracts solves this very issue.

  3. […] will pass just because to document the expected behavior? Sometimes yes, but not always as this post points […]

  4. Robert Higgins says:

    I once fail in love with a girl on site, in Duragno her name is Mary. She said her first girl would be Gnarleen and her Boy would be Radly.

  5. Jeff Grigg says:

    Question: How do I know that the processAnnotations method does not and cannot return null?

    The method looks like this:
    private List processAnnotations(Object o) {
    List result = new ArrayList();
    [… lots of nasty complex code …]
    return result;

    So really the question is, “What if someone assigns ‘result=null;’ in the middle of the method?

    Well, I looked. There is no such assignment.

    And if some later programmer introduces such a bug, then I think that they need to take responsibility for their actions. ;->

    And if I’m afraid that someone might accidentally do that, I can discourage them from doing so by adding the word “final”, like this:
    private List processAnnotations(Object o) {
    final List result = new ArrayList();
    [… lots of nasty complex code …]
    return result;
    Now everyone who looks at the first and last lines of this method will *KNOW* that it never returns null.

  6. Jeff Grigg says:

    Regarding “Should you not have a test in place that ensures that the processAnnotations method does not return null?”

    It’s very difficult to write a test to ensure that a function NEVER returns null. To do that, I’d have to test every possible scenario and ensure that none of them return null.

    But all is not lost…
    If we remove the null checks from the calling code and then run a good set of tests on the calling code, then the fact that the calling code does not suffer a NullPointerException is good evidence that the null checks were not needed. In fact, it’s a fairly strong test because we are already testing a case where we might fear that lists or arrays might reasonably be null or empty: We’re testing a class with no class annotations and no methods. We might worry that nulls might occur in an edge case like this. But running the tests, we see that there is no problem.

Leave a Reply

AWSOM Powered