8 How Do We Measure Our Test Coverage?
Test coverage is a measure of the degree to which a test exercises some feature(s) or code. Test coverage
relates the tests produced to the software or features under test such that we can estimate:
1. The degree to which a test exercises the software or features.
2. The software or features which are insufficiently exercised.
3. Given the number of bugs found and the test coverage, the confidence we have in the system??™s
attributes at any moment.
4. The minimum number of tests which need to be run to provide some level of confidence in the
quality of the system.
Caveat:
test coverage is another example of small boys with hammers to whom everything appears to be
a nail. Thus there are many papers published, full of good advice on code coverage in unit testing. There
is very little on feature coverage; [Memon] is the rare exception.
Is your test coverage getting better, or worse? There are several ways of measuring it:
1. Test coverage by feature. The specification says the system has the following
n
features plus startup
and shut-down. Do we have a test (set) for every feature plus start-up and shut-down? See
section 15.2.
2. Test coverage by GUI icon. The user interface has a number of screens, buttons, pull-downs, tabs,
menus, etc. Do we have them all listed (see section 8.2 for an example), and do we have tests
which execute every one? See section 15.4 for more on this.
Pages:
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131