Introduction

Code coverage is a way to measure the level of testing you’ve performed on your software. Gathering coverage metrics is a straightforward process: Instrument your code and run your tests against the instrumented version. This produces data showing what code you did—or, more importantly, did not—execute.

Coverage is the perfect complement to unit testing: unit tests tell you whether your code performed as expected, and code coverage tells you what remains to be tested.

Most developers understand this process and agree on its value proposition, and often target 100 percent coverage. Although 100 percent coverage is an admirable goal, 100 percent of the wrong type of coverage can still leave problems undiscovered. A typical software development effort measures coverage in terms of the number of either statements or branches to be tested. Even with 100 percent statement or branch coverage, critical bugs still may be present in the logic of your code, leaving both developers and managers with a false sense of security.

How can 100 percent coverage be insufficient? Because statement and branch coverage does not tell you whether the logic in your code was executed. Statement and branch coverage is great for uncovering glaring problems found in unexecuted blocks of code, but they often miss bugs related to both decision structures and decision interactions. Path coverage, on the other hand, is a more robust and comprehensive technique that helps reveal defects early.

Before you learn about path coverage, look at some of the problems with statement and branch coverage.

Statement Coverage

Statement coverage identifies which statements in a method or class have been executed. It is a simple metric to calculate, and a number of open source products exist that measure this level of coverage. Ultimately, the benefit of statement coverage is its ability to identify which blocks of code have not been executed. The problem with statement coverage, however, is that it does not identify bugs that arise from the control flow constructs in your source code, such as compound conditions or consecutive switch labels. This means that you easily can get 100 percent coverage and still have glaring, uncaught bugs.

The following example demonstrates this. Here, the returnInput () method is made up of seven statements and has a simple requirement: its output should equal its input.

Figure: Code sample

Next, you can create one JUnit test case that satisfies the requirement and gets 100 percent statement coverage.

Figure: Statement coverage

There’s an obvious bug in return Input (). If the first or second decision evaluates true and the other evaluates false, the return value will not equal the method’s input. An astute software developer will notice this right away, but the statement coverage report shows 100 percent coverage. If a manager sees 100 percent coverage, he or she may get a false sense of security, decide that testing is complete, and release the buggy code into production.

Recognizing that statement coverage may not fit the bill, the developer decides to move on to a better testing technique: branch coverage.