Security Testing

Подробнее о видео

Software Test Phases E054 2025 01 28
A Bit of Security for December 27, 2024
Some time ago I published an IBM Technical Report “Large Systems Test Methodology” discussing the ways the IBM company then verified that a release of the mainframe operating system was ready for the market. While there are hundreds of books on writing software, there are a precious few that discuss how to test software.
Let’s begin by clearing up some confusion. When you test software, you are verifying that it does what it is supposed to do. That’s different from getting a medical test, which is designed to diagnose what might be wrong. Software testing is more clearly called quality assurance. A medical test is more clearly called a diagnostic procedure. The difference is important because some folk think that a test is an exploration – like testing a hypothesis, to see if a proposed scientific theory is correct or not. These three different types of tests can confuse people who aren’t in the middle of the work.
A software test is not an exploration - we already know the result. Instead of a hypothesis, we have a specification. The test merely lets us know whether the software fulfills its design goal or not. There are no unexpected results when testing software – either the code works or it doesn’t.
On the other hand, if a medical test doesn’t deliver a result, it’s considered a failed test – we don’t know what’s happening so we try to narrow down the possibilities by checking out some known parameters – chemicals in the blood, bacteria or other infectious agents possibly causing disease, metabolic parameters to see if the body is performing within expected ranges – but with no predefined specification of what the actual cause of the discomfort might be.
When testing a large software product, the specification needs to be comprehensive. What we test for is conformance to requirements – to use Philip Crosby’s phrase. Waterfall was never a linear process – the actual path is more like a “V” with requirements at the top left, and successive refinements to High Level Design, Low Level Design, and ultimately code. Code gives rise to Unit Test, which makes sure the code pathways work, doesn’t have any simple problems like reading past the end of file or outside the bounds of an array or buffer. From Unit test we move up the right-hand branch of the V to Function/component test, which means the set of modules performs the desired local functions, interprets API inputs appropriately, and produces desired local effects like a return code or a change in the status of an indicator.
The structure of the V now becomes apparent. The higher up one side you are, the more direct the relationship is to the function on the corresponding other side. So F/CT reflects what‘s discussed in low level design. System Test verifies whole system capabilities, reflecting the discussion of high level design. The later phases deal with specific capabilities that reflect the requirements – performance testing, installation verification, and distribution testing. In those days software was distributed by a group called the Program Information Division, so PID verification testing meant making sure the product package was complete and could be shipped.
Ultimately the product launch documentation should highlight the initial requirements – in other words, the delivered software should meet the initial needs that led the company to build the software in the first place.
All of which lets us now ask, where does security, privacy, trustworthiness, and safety testing take place? At each phase, there are specific items the requirements should highlight that guide the tester to design tests for privacy (information leakage) and integrity (no side effects that could alter data unexpectedly). Security problems have two possible causes: errors which would allow an unauthorized service or use to observe data or alter the flow of control; and errors which could guide a user to making an unsafe choice. User Interface testing happens late in the cycle, but code bugs are easier to spot very early in the process – checking that a field is properly verified before being used, that it doesn’t allow access to information that is out of range, that it doesn’t allow pother processes change its flow of control, and that it doesn’t allow unauthorized processes to access its internal controls or code paths.
References:
The Art of Software Testing, T. Badgett, Corey Sandler, Glenford Meyers 2105
Quality is Free, Philip Crosby 1979
Software Test Phases - A Bit of Security for January 28, 2025
We can test for security. Here’s a hint on how to do that. Listen to this -
Let me know what you think in the comments below or at wjmalik@noc.social
#cybersecuritytips #testphases #softwaretesting #securitytesting #BitofSec