Qualys severity vs CVSS

I’ve spent a little more time than I would like studying severity models in vulnerability scanners. One question that comes up occasionally is Qualys severity vs CVSS, or any of its competitors, for that matter. Let’s take a look at how Qualys severity compares against the industry standard.

Qualys’ severity score predates CVSS, which is now the industry standard. The two have similarities but there are various reasons for choosing one over the other.

Why Qualys has its own severity score

CVSS, the industry standard, was only launched in 2004. Vulnerability scanners had existed for about a decade at that point. Nessus came along in 1995, and Qualys came into being around 1999. With no standard, each vendor took its own shot at rating the severity of the vulnerabilities they found. And, frankly, I think their models had as much to do with marketing as they did with security.

A few years ago I heard an interview with Steve Katz, the first ever CISO. He said to report in terms of high, medium, and low. It cuts down on hyperbole, and you don’t really have to explain it. Everyone understands the difference between high, medium, and low.

That’s good security but some would say not the best marketing. Fear sells. Qualys went with Urgent-Critical-Serious-Medium-Minimal. Rapid7 went with Critical, Severe, and Moderate. Tenable went with Critical-High-Medium-Low.

Qualys severity levels explained

Qualys uses a numeric scale of 1-5, ranging from Minimal to Urgent. Higher numbers are worse. I get asked by non-security types why that is and if we can change it. That’s not configurable. I can’t say every vulnerability scanner in history has used a scale where lowest is best, but the surviving big three do, and so does CVSS. I explain it by saying sports statistics aren’t consistent either. In baseball, hitters want high batting averages but pitchers want low earned run averages. Leading the league in touchdowns in football is good, but leading the league in turnovers is bad. In hockey, leading the league in penalty minutes might be both.

• 5, Urgent: An attacker can easily gain control of the system, including full read and write access to files and/or remote code execution.
• 4, Critical: Intruders can possibly gain control of the system, or leak sensitive information.
• 3, Serious: Intruders can gain access to specific information stored on the host and/or deny services or misuse them, such as mail relaying.
• 2, Medium: Attackers may be able to collect sensitive information from the host, such as the precise version of software installed. This type of vulnerability can be used to find additional, more severe vulnerabilities.
• 1, Minimal: Attackers can collect information about the host, such as open ports or, services, and may be able to use this information to find other vulnerabilities.

Problems with the Qualys severity levels

Most of us know that serious isn’t as bad as critical. Critical condition means you’re closer to dying than being in serious condition. But the difference between critical and urgent is much more fuzzy. You don’t go to urgent care when your life is in danger. That’s what hospitals are for. Frankly, if you think someone might be in serious condition, you go to the ER, not urgent care. Urgent care is for when you need stitches, or you’re sick and you can’t get in to see your regular doctor.

Qualys terminology frankly makes almost everything sound like a big deal. And when everything, or almost everything, is an emergency, nothing’s an emergency. I like the CVSS terminology better. It’s more intellectually honest.

I pushed patches for a living for nearly a decade under those everything-is-an-emergency circumstances. The policies conflicted with each other so much I had nothing but a deadline and a passing score of 100 percent to go on. I paid zero attention to severity. It was all based on what I could reboot and when. To anyone who was watching, it looked like I was fixing things in random order. I got it done, fixing 800,000 vulnerabilities along the way, but I don’t think anyone was ever happy with exactly how I did it.

CVSS severity levels explained

CVSS uses a numeric scale of 1-10, ranging from Low to Critical. Higher numbers are worse. The numbers are a little fuzzier but the terminology makes a bit more sense.

• 9-10, Critical: An attacker can easily gain control of the system, including full read and write access to files and/or remote code execution.
• 7-8.9, High: Intruders can possibly gain control of the system, or leak sensitive information.
• 4-6.9, Medium: Intruders can gain access to specific information stored on the host and/or deny services or misuse them, such as mail relaying.
• 3.9 and below, Low: Attackers may be able to collect sensitive information from the host, such as the precise version of software installed. This type of vulnerability can be used to find additional, more severe vulnerabilities.

CVSS aims for more numeric precision while settling for broader terminology. But frankly I think the terminology ends up being more precise too. I don’t have to explain the scale with CVSS, you get it.

Interestingly, I find even when people use the Qualys numeric scale, they tend to use the CVSS terminology of Critical, High, Medium, and Low. Almost everyone has to look up what a Qualys SEV 1 vulnerability means.

When to use Qualys vs CVSS severity scores

CVSS is the newer standard, and it’s an open standard. No matter what scanner you use, the same vulnerability will have the same CVSS score. Two companies that use different scanners can agree to have the same policy, as long as they’re using CVSS to measure the results. If your policy states you have to fix Urgent or Severe vulnerabilities in a given length of time, your policy only works with Qualys (in the case of Urgent) or Rapid7 (in the case of Severe). It’s not a good idea to tie your policies to a specific vendor when there’s an open standard available.

That said, some policies have existed longer than CVSS has. That’s why Qualys and its competitors have kept their own severities along with supporting CVSS. That way you can continue to comply with older policies, as long as you haven’t changed tools since then.

I like CVSS better because it’s an open standard, and in a pinch, when someone wants to know why a vulnerability is critical instead of high, I can find out. The major components that go into the CVSS scoring are there in the scan results, and if you want more detail, you can export the knowledge base via the API and get every number that factored into the equation.

What about risk-based approaches?

Both CVSS and Qualys severity are calculated when a vulnerability is released. And I’ve specialized in vulnerability management long enough to tell you not every vulnerability ends up being as bad as we think at first. Meanwhile, others end up not getting any attention at all, even though they deserve it. When they come at a rate of 200 per week, it’s impossible to get anyone to pay attention.

A risk-based approach, such as what Kenna pioneered, recalculates severity based on whether attackers actually use the vulnerability or not. CVE-2018-15473 is my favorite example. It leaks the usernames on a Linux or Unix system via SSH. That’s a Medium or a Low the way Qualys and CVSS measure. Kenna rates it a high, because attackers use it a lot. It’s not as big of a deal as remote code execution, at least, reliable remote code execution. It is a bigger deal than remote code execution that doesn’t work reliably.

Some risk intelligence services use the CVSS equation and just factor in new risk intelligence in the parts of the equation that pertain to complexity and exploit code maturity. Others use their own equations.

Qualys risk intelligence

Qualys doesn’t recalculate risk in its Threat Protection module, which is included in its new VMDR plan. That’s my biggest knock on Qualys Threat Protection and VMDR. It provides threat indicators, but doesn’t provide any easy way to export them so you can do any kind of analysis on them outside the tool. Qualys says they leave it up to you to decide what kinds of threats matter to you. In my experience people really like that, but getting people to tell you which ones matter to them and why is difficult. If they do have an opinion, it’s because of something that happened to them in the past, which may not be a good indicator of future problems.