Software Defect Detection

Подробнее о видео

Defect Detection and Remediation Patterns E33 2024 08 22
When I guided the Build and Test team at IBM Poughkeepsie, we gathered plenty of data about defect detection and remediation rates. We used this data to track the effectiveness of our test efforts. Over time, we developed a pair of measures that did a good job predicting ultimate product quality and helped reinforce our understanding of the development and test schedule.
One measure was the defect detection rate. Typically, the total number of defects found would rise quickly at first, and over time it would taper off. Testers are efficient and knock off the easy bugs first. The second measure was the reported severity of the defects. That would tend to stay in the low to moderate range, and over time rise as the more difficult problems surfaced. Those would take more time and effort to root out. These two curves can be overlaid. One curve rises rapidly than asymptotically approaches a nearly flat slope, meaning there aren’t many more defects to find. The other curve stays low, and over time picks up.
At a certain point the cost of finding the next bug would exceed the cost of moving into the next test phase. Unit testing – evaluating code execution step by step – runs till those issues have been found and resolved. Input screening occurs during this phase – buffer overflow, privileged execution mode, and such should be validated very early. Then the testers move into function/component test, looking at subsystem characteristics, usually in a mocked-up environment, with stubs for incomplete routines. When those are available, at the right time, testing moves into system test – looking at whole system characteristics. With a running system we can begin to explore load and stress, recoverability, the user and operator interface, input and environmental variations, installation and packaging, and failure modes.
The point is that test costs are very low compared with fixing defects in production, so staging your test initiatives to optimize your test efforts can save you a lot of heartache and long evenings downstream.
For more insight into testing, see “The Art of Software Testing” by Glenford Myers. I’ve given away more than a few copies, but I always have one on hand. The book reminds us that good developers generally make lousy testers – and how a good team makes the best use of all types of skills.
Defect Detection and Remediation Patterns
A Bit of Security for August 22, 2024
How do you measure the effectiveness of your testing? What is worth counting, and why? Listen to this -
Let me know what you think in the comments below or at wjmalik@noc.social
#cybersecuritytips #softwaretest #testphases #unittest #lastbug #BitofSec