If this is an high-risk system then this may be good news,
particularly if the developers have an history of making very solid releases. Otherwise, particularly
if there is an history of severity-1 bugs being reported from the field, check:
??“ If the highest-severity bugs are correctly-classified
??“ If all end-to end scenarios have been completed
??“ If many variations of each scenario have been tested
System and Acceptance Testing 279
??“ If the bugs which have been found are found as often as you expect, but are more serious than
they appear (see section 15.15.1 for an example)
??“ If critical bits of the requirements specification have been correctly mapped to one or more tests
??“ If the system as a whole is approaching the stability profile you expect
??“ If fault injection in Chapter 13 has been used to test the tests??™ ability to find high-severity bugs
If the preceding seven statements are true it is probable that no severity-1 bug has been found
because it wasn??™t there. See section 18.2 in Chapter 18 for further tests to run to establish how
close the system is to exhibiting a bug. See Table 2.3 for a way of determining this.
??? Having to abandon test runs because they are untestable: testers are finding so many top-severity
bugs that they are losing faith in the possibility of a release succeeding. This is not your problem
directly, but it could help if you identify the parts of the system which can be tested such that you can
propose to management which bits might be worth keeping while the rest be completely rewritten.
Pages:
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533