What manual testing is in security

The SANS vulnerability management maturity model has an entire section on manual testing. That may not be a phrase you hear very often because there are several types of manual tests. So what is manual testing in security?

Manual testing is a form of security testing, namely, looking for security vulnerabilities in a non-automated or semi-automated fashion at most. It is not the same as vulnerability scanning like one does with tools like Nessus or Qualys.

Manual tests can be conducted by a third party or by an internal red team or purple team.

The reason SANS specifies manual testing is because some firms will simply conduct a vulnerability scan and call it a penetration test for regulatory or contractual purposes. Sometimes it’s an honest mistake and sometimes one or both firms should know better. Specifying a manual test hopefully makes both parties less likely to try to pass an unauthenticated Nessus scan off as a pen test.

Both types of tests have their purposes, but they are complementary. One is not a substitute for the other.

Types of manual testing: black box vs white box

Manual testing in security
Burp suite is a common tool for manual testing in security. Image credit: Mvb71, CC BY-SA 4.0, via Wikimedia Commons

Broadly speaking there are two types of manual tests: white box and black box. In a white box test, you have documentation and specifications. In a black box test, you start with little or no information.

Both white box and black box tests are types of manual tests. A white box test is more likely to find more issues and provide more data for root cause analysis. A black box test gives the outsider perspective, and theoretically a problem uncovered in a black box test is easier to find.

Manual testing vs automatic testing

Theoretically, a manual test can start by conducting a traditional vulnerability scan, probably without authentication, across all of the systems in scope to save time. But then you need to follow up on what you find to confirm it. Tools for this purpose include Burp Suite for testing a web application, or Metasploit or Core Impact for testing off the shelf software.

But hopefully this clears up any confusion over what a manual test is and the qualifications of an individual conducting them. I don’t recommend looking for a CISSP for this type of work, unless they also have other specific certifications. I wouldn’t recommend a CEH either, for the same reason. Look for GPEN or OSCP certifications.

Manual testing is more expensive, because automatic scans are largely passive work that finish in a matter of hours. Manual tests can take several days to complete. The more robust your security is, the longer it takes.

If you found this post informative or helpful, please share it!