I should probably start by pointing out that this post is more to help me get something straight in my head than anything else. It’s also covering a subject that I’m not sure I fully understand, so I may have completely missed the point.
One of the things that I was most interested in at Sync Conf was Behaviour Driven Development (BDD). I’ve never been great at following Test Driven Development (TDD), mainly because I couldn’t make the shift required to fully get my head round it. What I practiced, if it has a name, was Javadoc Driven Development; each method would have descriptive Javadoc that defined what happened on various inputs. I found that by doing this I built up a narrative of how a class would work and that provided me with concrete test examples.
This method of testing only really works if you write decent Javadoc. The following is next to useless, and on many levels:
* Sets the name.
* @param name the name
public void setName(String name);
What happens if I pass in
null? What happens if I call the method twice? Does passing a blank string cause issues? I’ve heard so many arguments that Javadoc is pointless and you should just read the code, after all, it’s just going to get out of date. I find that attitude utterly reprehensible; the behaviour of the method could rely on something that it’s calling and that could go more than one level deep. I don’t want to have to dive through reams of code to understand if what I’m calling does what I expect, nor do I want to have to guess. For me the Javadoc provides a contract for the method in a narrative form which is then implemented by the code. Change the logic of the code and you have to change the Javadoc, and I’d actually change the Javadoc first.
The mindset I try and put myself in is not “how will this method work logically”, but more “how will this method be used and how does it fit in with what the object is trying to do”. In the above example we’re setting a name. Does it make sense for a name to be
null, or blank? If we can have an empty name how is that represented?
null, empty string, or both? Is a blank string treated the same as an empty string? Should we trim the string? Are there other things we’re going to limit? You then document the answers to those questions:
* Sets the name to the given value, overwriting any previous value that was
* set. Passing <code>null</code> into this method will have the effect of
* clearing the name. Passing a blank or empty string will have the effect
* of clearing the name by setting it to <code>null</code>.
* @param name the value to set the name to
public void setName(String name);
All of a sudden you’ve got a load of things you need to test. I’d be writing tests to ensure that:
- Given a
null value, providing a value A set the value to A.
- Given a
null value, providing a
null value retained the
- Given a
null value, providing a blank value retained the
- Given the value A, providing a value A retained the value A.
- Given a value A, providing value B set the name to that value.
- Given a value A, providing a null set the name to
- Given a value A, providing a blank string set the name to
Those of you familiar with BDD may already be seeing something familiar here. What I have been doing is something very similar to BDD, albeit in an informal and haphazard way and at a very low level. I was defining functionality as a narrative in the Javadoc with my user in mind (another developer), and then using that to discover the scenarios that I should test. I was actually so used to this method of working that when I used Tumbler to test one of the classes I was working on it already felt natural and the tests practically wrote themselves. Interestingly enough the conventions used by BDD and Tumbler freed me from one of the constraints I was facing with testing; that of my test classes were getting too big. I will still structuring my tests classes as the mirror of my code, so for any object
Foo there was a
FooTest. By thinking in stories for the tests too and grouping scenarios that belong to a story I could break out of this habit and have as many classes as I needed testing the behaviour of
Happy with my new test scenario, and the output that Tumbler had produced, I proceeded to run Sonar against it. Sonar did not like the result. Most of what it complained about no longer matters. I can turn off the requirement for Javadoc on all methods because my test methods contain the scenario in the code and don’t require it elsewhere. The need for a descriptive string in my assertions can also be turned off as the way the tests are displayed by Tumbler provide a much more natural way of reading the test results. One critical marker stood out though: 0% code coverage.
It took me all of 30 seconds to work out what was going on. Tumbler uses its own JUnit test runner, produces its own reports and isn’t spitting out the files that Sonar is looking for and so there’s nothing for it to report on. This may or may not be something I can fix, although Google is yielding nothing on the subject. This got me to thinking: Do I need to know my coverage? Surely if I’m defining the behaviour of the class, then writing the code for that then I’ve pretty much got 100% coverage since I shouldn’t be writing code that produces behaviour that hasn’t been defined. This is where I got stuck.
Liz Keogh, who gave the Sync Conf BDD talk mentioned that BDD didn’t work so well for simple or well understood scenarios. Should I be using TDD here? That way I’d get my test coverage back, but I lose my new way of working. Finally, after much Googling I can across this blog and came to the realisation that I’m coming at this all wrong: there is no spoon. What I think Liz meant was that BDD at the level of the business helping to write the scenarios isn’t useful for well understood scenarios, because they’re well understood and anyone can write them, not that we just give up on BDD and go do something else… or maybe she did, but if I’m understanding Hadi correctly then using TDD instead of BDD is just a semantic shift and I could just as easily use BDD in TDD’s place.
We all know that 100% test coverage means nothing, and that chasing it can end up an exercise in pointlessness. Then there’s the farcical scenario where you have 100% test coverage, all tests running green and software that doesn’t work. So why the panic over not knowing my test coverage? I think it boils down to the Sonar reports let me browse the code, see whats not tested and then think up tests to cover that code. In other words chasing 100% (or 90% or 80%) and writing tests for testings sake. If I’m doing BDD properly then I’ll be thinking up the narratives, determining the scenarios and then writing the code to meet those scenarios. If my code coverage isn’t above 80% (which is a level I consider to be generally acceptable) then I’m doing something wrong as there is code and code paths not covered by scenarios which is, in theory, pointless code.
So how do I solve my Sonar problem? Simple, unhook the alerts on test coverage, remove the test coverage widget and keep my eye out for a plugin for Tumbler reports. In the mean time I can just use the reports generated by Tumbler to keep an eye on my tests and make sure they’re running clean and read up on getting my Maven builds to fail when Tumbler has a failed test.