Vulnerability assessment vs penetration test: What’s the difference?

Last Updated on December 23, 2019 by Dave Farquhar

You can expect any midlevel or higher security certification test to require you to compare and contrast a vulnerability assessment vs penetration test. The difference is important. But since I still see people confusing the two, let me explain them.

I conduct vulnerability assessments for a living. I’ve done some light pentesting in the past, but I’m happier doing assessments. I don’t want to be a penetration tester. That seems strange to some people. Maybe that gives me a good perspective to compare and contrast the two, because both have their uses. But they aren’t interchangeable.

Vulnerability assessment vs penetration test in a nutshell

In a nutshell, the biggest difference between a vulnerability assessment and a penetration test is one thing. A vulnerability assessment scans a computer or network and looks for vulnerabilities in a system, but doesn’t try to hack in. A pentest also scans your computer or network, but performing a penetration test includes following up with exploits to try to hack in.

To put it another way: If you ask me to conduct a vulnerability assessment on your building, I’ll assess the locks and the windows, but I won’t actually break into the building. A penetration tester will break into the building and tell you how he or she did it. It’s like the Robert Redford movie Sneakers. Exactly like the movie Sneakers.

That ought to tell you enough to get a test question right. If you’re making a purchasing decision or assessing a third party vendor, you may need a bit more to justify it. So let’s look at the advantages and disadvantages of each type of security testing.

Advantages of a vulnerability assessment

Since I do vulnerability assessments for a living, of course I think there are advantages to them. The main advantage is thoroughness. A vulnerability assessment does the best job it can to find every possible vulnerability in a system. Some tools are a little better at this than others, and I’ve written about that before, but even the worst vulnerability scanning tool is probably going to find things that aren’t on your penetration test report. Both of them provide lists of vulnerabilities, but the vulnerability assessment tries to provide a complete list.

A vulnerability assessment is also safer. A vulnerability scanner just looks at a system’s open ports and the responses on those ports, then looks at the filesystem. Then it uses those results to assess a system’s security posture. If it finds files associated with a particular vulnerability, it adds a finding to the list. A good vulnerability scanner does this as non-invasively as possible, and will even throttle itself if the system seems to be going into duress.

Vulnerability assessment can be, and when possible, should be a white box test. Give it credentials so it can authenticate to the system and validate as many of its findings as possible. I’ve covered vulnerability management best practices at some length. There are reasons to do an unauthenticated one, but those are niche uses.

Disadvantages of a vulnerability assessment

In a word, false positives. OK, that’s two words. But the biggest complaint about vulnerability assessment is the number of false positives. When you’re assessing security services, asking what they do with false positives can be a good differentiator. Some security professionals will work with your system administrators to investigate the details of each finding. Under that scrutiny, many findings turn out not to be false positives.

I once had a client complain about false positives and tell me they’d have their internal pentester try it out. The problem with that is your pentester not being able to exploit something doesn’t mean it’s a false positive. It means the exploit didn’t work under those conditions. It may work under different conditions. A different exploit may work better.

The other problem is simple information overload. A routine scan may find hundreds of things wrong with a single system. Yet the system seems to be working just fine. It’s perfectly natural to dismiss someone like me as alarmist

Advantages of a penetration test

The biggest advantage of a pentest is that it’s difficult to dispute the findings. I’m used to hearing people ask if I really think someone is going to use one of my findings to get into their network.

There’s no arguing that with a penetration test report. The penetration test report tells you what the assessor found, then what the assessor did to get into a system. It’s not theoretical. Someone got in. Once you have proof that someone used a vulnerability to get into a system, people fix it in a hurry.

Disadvantages of a penetration test

The disadvantage of a pentest is thoroughness. The main things you hear about are the vulnerabilities the pentester used to get in. There could be hundreds of other things wrong with each system in question. Some of them could be just as bad as the one the pentester used.

When you get a penetration test, you’re telling the assessor to break in, then tell you how he or she did it. You aren’t telling the assessor to find every possible way to break in and then do it.

There’s one more disadvantage, and that’s what happens when you exploit weaknesses in your system. An exploit is a controlled crash. Your pentester is actually crashing processes on your system, and attempting to maintain control long enough to get the system to run injected code before the process goes down in flames. It invites unintended consequences. This is why exploits don’t always work as intended. They’re dangerous contraptions by their very nature.

Vulnerability assessment vs penetration test: Which do you need?

Both penetration testing and vulnerability assessment are useful for good risk management and network security. Usually it comes down to regulatory requirements. Some regulations require one or both. It’s not a bad idea to get a penetration test once a year. Some industries require it. It identifies some things to fix and gives you a pile of evidence that it’s really a problem. It’s a good way to inject some fuel into a remediation management program. Vulnerability assessments really should happen once a month, and should be followed up with some serious remediation. You won’t fix all your findings every month, at least not at first. Fix as many of the new things that come out on Patch Tuesday as you can, and try to fix at least 10 old things too. Use something like Kenna to prioritize if there’s too much work for you to fix.

If you do a good job of remediating the findings in your vulnerability assessments, you’ll do better on your penetration tests. So the two things really do feed into one another.