Qualys vs Rapid7

Where they rank any given year may vary, but there’s no doubt Qualys and Rapid7 are two of the big three in vulnerability scanning tools. Both tools have their pros and cons. Let’s look at Qualys vs Rapid7 so you can figure out which one is right for you.

Disclaimer

Let’s get the disclaimer out of the way. I am a former Qualys employee. We parted ways because I liked 1/3 of my job duties much more than the other 2/3. In my post-Qualys career, I have recommended Qualys at times, and competing products at other times. I am confident I can be objective. Qualys management isn’t going to be happy with everything I say here. Neither will Rapid7, if anyone from there ever reads this. But I learned 20 years ago that blogging is a lousy way to make friends. I’m not writing this to make friends.

I am also confident I can evaluate both tools competently. Before I was a security pro, I was a sysadmin, and I fixed somewhere between 800,000 and 900,000 vulnerabilities. I was a team of one. When I worked at Computer Sciences Corporation, they had me revamp how they handled patching for the entire region based on the methodology I developed. I started most Patch Tuesday cycles with a clean scan. When I didn’t have a clean scan, I had a risk acceptance for every remaining finding, and I did fix each finding before that risk acceptance expired. My risk acceptances were generally for 2-12 months.

I spent years of my career going through vulnerability scans line by line and fixing each finding, and fixing any patch that failed to apply. If there was a mistake in a scan, I found it.

This is all my opinion, not the opinion of Qualys, or my current employer, or any former employer.

Vulnerability scanning is not a commodity

I wish I had a nickel for every time some purchasing department tried to tell me they wanted one vendor to come down in price because they’re all the same. Changing between Qualys and Rapid7 isn’t like switching between HP and Lenovo. It’s more like switching between Apple and any given PC vendor. The PC vendor will beat Apple’s price every time, but if you need to run Final Cut Pro, it doesn’t matter. And if you buy Apple, you’ll have to jump through hoops to run Visio. Namely, installing virtualization software and Windows on it. So it’s a lot of hassle and extra expense.

You’ll find the same thing when you switch between vulnerability scanners. The capability you’re used to might be there. Or it might come at additional cost and additional hassle. Or it might not be there at all. Don’t assume just because the tool you’re used to has a report you like that the other tool has the same report. And the UI is going to be different, so you’re going to have to relearn some stuff even if both tools have every feature you need.

If you’re considering a switch, run a POC and make sure all the functionality you care about is there. Don’t assume, and don’t take the salesperson’s word for it. That’s how you get burned. I’ve seen more than one company switch tools and take the salesperson’s word they were interchangeable, and then get angry that the new tool didn’t work exactly the same as the old one.

There are times when a switch is necessary. But it won’t be painless.

I’ll also say I’ve observed a huge dropoff in capability once I got outside of Qualys, Rapid7, and Tenable. There’s a good reason why those are the three that people talk about.

Qualys vs Rapid7: Risk reporting

Qualys has been pushing its VMDR solution hard, which includes its Threat Protection product. Rapid7 claims it invented risk reporting. I’m pretty sure Kenna beat them both to it, but you’re not here to talk about that.

Qualys gives you a ton of risk data, and there’s some actual breach data behind it, which is nice. But the problem is that it’s too much data. You have to decide what threat matters most to you, and most organizations, frankly, don’t do a great job of figuring that out. What I typically see is they prioritize based on something that happened to them in the past, thinking that makes other types of attacks less likely in the future. Those blinders leave them open to those future attacks. Qualys just flags vulnerabilities as having capability of high lateral movement or high data loss, and it’s not something I find especially helpful.

Rapid7 gives you a score. Qualys counters that nobody knows what that number means, but everybody knows a vulnerability with a score of 100 isn’t as big of a deal as one with a score of 900. I’m not sure I know more than two people who really understand CVSS either, but we all accept and use it.

I’m not going to tell you what to do with Rapid7’s risk score (how I handle it is a trade secret) but let me just say that number gives me options. Lots of options. And most of them are good. With Qualys, I have to take their data and do some math first and I like how Rapid7 lets me skip the math and go straight to analysis.

The problem with Rapid7’s data is it’s just based on Metasploit. There’s more to the threat landscape than Metasploit. That data is expensive, though, and that’s why Rapid7’s intelligence costs less than everyone else’s. Qualys is buying data about actual breaches. This is a case where Qualys should be better, but Rapid7 hit its full potential while Qualys delivered much less than it was capable of.

The other nice thing Rapid7 does is make it easy to adjust the scores. Qualys also has this capability, but it’s buried and complicated to use. With Rapid7, you just create an asset group, set its importance to high, medium or low, and it adjusts everything for you. You still have to know what your critical assets are, but if you have that, Rapid7 puts that information to great use.

The Wording

All that said, I take great issue with the wording Rapid7 uses. While every other vendor uses High, Medium, and Low, Rapid7 rates vulnerabilities as Critical, Severe, and Moderate. That’s fearmongering. When everything’s an emergency, nothing is an emergency, and according to Rapid7’s terminology, nearly everything is an emergency. I can tell you from experience that leads to gridlock.

Actual breach data tells us it’s fewer than 10 percent of the vulnerabilities that get you. You can find that 10 percent with either tool, but they’re both going to make you work for it. Rapid7 does a better job of making it easy to find, but you’re going to have to look past their wording to find it. Rapid7’s wording makes you sound like Chicken Little.

Rapid7 does a better job here, but they sure did their best to try to mess it up.

Advantage: Rapid7

Qualys vs Rapid7: Informational findings

Rapid7 and Qualys trade barbs over informational findings. Qualys finds a ton of data as it assesses a device, and rather than throw it away or hide it from you, it presents that data in its findings. Rapid7 calls this confusing and deceptive. And to be fair, some people just hide all the informationals. But sometimes you need those. If a scan runs longer than expected, those findings tell me why. I can also use those findings to locate vulnerable systems even before anyone has signatures available yet. Qualys knows it has that piece of software on it, so all I have to do is export all the informationals, know where to filter, and we can work on remediation plans 24 hours before anyone has a signature for it.

And sometimes Qualys finds me compensating controls. I still have to ask questions, but at least I can ask very intelligent questions, armed with what Qualys finds me.

I’ve even used Qualys informational findings to prove a vulnerability was a false positive. Yes, I’ve used Qualys to disprove Qualys. For some reason I found it a lot more hilarious than Qualys did. But that was a quick and easy support ticket to get that QID fixed.

Most importantly, Qualys tells you the system’s uptime and whether it has a pending reboot. In some cases, I’ve “resolved” half the vulnerabilities in a large company just by using those informational findings to tell someone what systems to reboot. It makes me an instant hero.

It’s on you to figure out what to do with the trove of non-vulnerability data Qualys gathers, but it’s incredibly valuable. Qualys treats you like a power user, where Rapid7 treats you like a n00b. We were all n00bs once, but I prefer not to remain one.

Advantage: Qualys

User interface

I loved Nexpose’s user interface because it took me 15 minutes to learn it, maybe less. It was well laid out, intuitive, and looked reasonably good.

InsightVM has a modern, tablet-style user interface. VERY tablet-like. I found it jarring, though someone younger than me would probably love it. You can get used to it. The main thing is learning where everything is. Not quite everything is where you expect it to be, but most of it is. It will either take you 20 seconds or 20 minutes to figure out how to launch a scan in InsightVM, and it’ll probably have everything to do with your age.

Qualys’ UI is getting dated, but it was far and away the best when it first came out, and it remains usable today. The key there, too, is not quite everything is where you expect it to be. The major stuff is all someplace you can find it. But there may be that obscure thing you can’t find. Asset Merging was the most notorious example. Qualys cleaned that up, but at one point it was three different seemingly unrelated settings in three different places in the UI. Rapid7’s equivalent is on by default.

I like Qualys better but that may be personal preference. Part of that is probably because I have two years of experience with Rapid7 tools and seven years of experience with Qualys. I can do most of the major things you have to do in Qualys in my sleep. If I were more familiar with the Rapid7 UI, I could probably say the same. And I’ll tell you one thing, they both have a far better UI than Tenable.

Advantage: Toss-up

Qualys vs Rapid7: Accuracy

Qualys claims 99.9997% accuracy, and it’s probably pretty close. I’ll be blunt: Qualys’ weakness is assessing network gear. With Linux and Windows devices, as long as it authenticates, its false positive rate is absurdly low. Its accuracy on network gear is less good. Not terrible, but not top of its class.

When I was a sysadmin I would have had a field day with Rapid7. Even with authentication, you’re going to find false positives when you dig into the results. And you’ll get a much larger number of false negatives. Its checks just aren’t as thorough as Qualys. Its scans are faster, but that’s because it’s less thorough. I can’t put a number to it, and shouldn’t tell you the number even if I could remember it, but I was used to processing some number of false positives every quarter with Rapid7. How many depended on how well everyone was paying attention. With Qualys, I can go years without seeing a false positive, at least on Linux/Unix and Windows systems. And if I do find one, there’s a reasonably good chance when I rescan, the finding will go away because they revised their check.

Advantage: Qualys

Qualys vs Rapid7: Agents

In this COVID-19 world, we have to talk agents. When you have remote workers, the best way to assess their machines is using the agent. Both Qualys and Rapid7 have very capable agents. They’re lightweight, use small amounts of memory and CPU power, and do most of the processing in the cloud.

Qualys has a slight edge because you can limit how much CPU power it can use, and you can set blackout periods where it can’t do anything at all. If you turn all the settings up to 11 of 10, sure, you can make bad things happen. But if you use the default settings, you’ll barely know it’s there. If you turn them down a bit, you won’t notice the Qualys agent.

Both agents are multi-function, but Rapid7 offers two functions (three, the way Qualys counts) and Qualys offers about six, and has some crazy additional number in the works. Rapid7 bills its agent as one agent to rule them all, but Qualys is closer to delivering on that claim.

This is a close race, but Qualys gets the edge here.

Advantage: Qualys

Qualys vs Rapid7: Training

Qualys has free training and certification. You can do on-demand training at any time, for free, and take the test at any time, for free. Qualys also offers in-person training and doesn’t charge for that either, you just have to get there. And if you’re having Qualys issues, chances are taking the training will give you the background to resolve 90% of them, if not more. You can even get the training and certification if you’re not a Qualys customer. So if you’re considering Qualys, or if you’re interviewing for a job at a shop that uses Qualys, you can get training and hit the ground rnnning.

R7 has some free training online, but charges for its best courses, and it’s expensive. You’ll pay $2,000 per person to get the equivalent of the training Qualys gets you for free. If you just want to take the free classes and sit for the exam and get certified, that’s $200 per person. Rapid7’s training is good, but Qualys’ instructors are at least as good.

When I talk to people about VM, I can tell who’s had the training from their vendor and who hasn’t. It’s pretty obvious from their complaints. Regardless of which tool you select, you need the training. You’ll learn more in two days than you’ll learn from a year of using the tool and trying to figure it out yourself. But this is the place where Rapid7 charges you and Qualys gives it to you for free.

Advantage: Qualys

Qualys vs Rapid7: Reporting and Dashboarding

Both tools have the ability to build dashboards to visualize your data, and both tools have a large number of built-in reports and can output a large number of formats. Rapid7’s UI for moving dashboards around is much less clunky. The widgets just move when you want them to. In Qualys, sometimes when you go to move a widget, it just snaps right back where it was, with no indicator of why. Or it might resize when you meant to move it.

Both tools have adequate canned reports. Rapid7 takes things a bit further, letting you build your own reports, using whatever combination of elements you want from the canned reports. If you have someone who’s super particular about what they want to see, Rapid7 is much more likely to be able to deliver something close.

Both tools require you to change the way you work and report to fit them, but Rapid7’s reporting is much better. When I hear of buyer’s remorse after switching from Rapid7, it’s always the reporting people miss.

Advantage: Rapid7

Pricing

Pricing is the elephant in the room. Qualys is expensive, and they nickel and dime you. The new Qualys VMDR model changes that, bundling in CIS benchmarking (the element of the very expensive Policy Compliance module most people want), Threat Protect (risk intelligence), the cloud agent (formerly a 40% upcharge), and unlimited virtual scanners (formerly $995 a pop). Qualys VMDR is still more expensive than Rapid7, but if Qualys has given you sticker shock in the past. VMDR looks much more reasonable.

If you’re price-sensitive, Rapid7 is the cheapest option worth considering. At least up front. Rapid7 charges for training and certification, and pushes professional services. Those two things may eat up more of the price differential than you anticipate.

Advantage: Rapid7

Qualys vs Rapid7: Patching

Rapid7 integrates with Microsoft SCCM and IBM Bigfix, and unlike Tenable, it does so in a way that’s actually useful. It creates patch groups, so your sysadmin can turn around and easily deploy what you need deployed. It takes longer to pick out the patches you want than it does to actually deploy the update, so this breaks down one of the huge obstacles when it comes to security teams and patching teams working together.

The problem with the SCCM integration is it only handles Microsoft patches. Those are usually the patches that give organizations the least amount of trouble.

The problem with the Bigfix integration is it only sends new vulnerabilities to it. Your backlog is usually a much bigger problem than new vulnerabilities. You can work around this by deleting the asset and rescanning it so those vulnerabilities become “new,” but then you sacrifice your history.

Repid7’s integrations are far from perfect, but kudos to them for actually trying.

Qualys integrates only with its own patching solution. That’s fine if your infrastructure teams aren’t married to whatever patching solution they’re already using. But in my experience, they’re much more attached to their patching tool than you are to your scanning tool.

Let me give you one more pro tip. Even though neither Qualys nor Rapid7 integrate directly with Ivanti’s patching solutions, Ivanti’s most recent builds can ingest a CSV full of CVEs and build lists of patches from it. If you happen to be an Ivanti shop, take advantage of that functionality. Kudos to Ivanti for trying too.

Advantage: Rapid7

Qualys vs Rapid7: Searches and queries

Qualys provides QQL, its own Splunk-like query language to search your Qualys data. Rapid7 makes you do SQL queries. SQL is far more powerful, but you can learn QQL and be doing basic searches in minutes. SQL is fantastic if you have someone on your team who knows it. This is purely anecdotal, but I know exactly 5 current or former VM professionals who know SQL. I know at least 100 who do not.

With either tool, you’re going to make yourself an Excel sheet with cool queries in it that you find on the discussion forums, or that your sales rep shares with you. You’ll paste the queries in column A and a description in column B. Within six months, you’ll have some Qualys queries you developed yourself. With Rapid7, unless you know SQL, it’s always going to be a list of stuff someone else wrote.

Rapid7 is a jet fighter, and Qualys is a car with an automatic transmission. The jet fighter is much more powerful, but the car is more practical. If you know SQL, Rapid7 has the advantage here. Otherwise, I gotta go with Qualys here.

Advantage: Qualys

Qualys vs Rapid7: Solution data

Both Qualys and Rapid7 have very good solution data, but Rapid7 doesn’t bury it as deeply as Qualys does. Rapid7 provides the most recent update that fixes a given vulnerability. It’s right there in the UI or the export. Qualys provides the original update in that view. In some Qualys reports, you can suppress superseded updates, but that doesn’t supress them everywhere in the UI.

Qualys has a separate report, called the Patch Report, that you can pull against any asset or list of assets. The patch report is a get-well plan, showing the minimum number of patches to bring a system completely up to date. Qualys doesn’t guarantee the patch report is 100% accurate, because they know the supersedence data Microsoft provides isn’t quite 100% accurate, but it will be very close.

The Qualys patch report would be pure gold if your patching tool could ingest it somehow. For one-offs, such as remediating new builds before they go into production, it’s great. It’s a bit less useful against production systems, at least at enterprise scale.

Advantage: Rapid7

Qualys vs Rapid7: In conclusion

Rapid7’s solutions are compelling. But in the end, vulnerability management comes down to the quality of your data. Your IT department will attack your data, because it’s easier to discredit you and the tool than it is to fix millions of vulnerabilities. The Qualys data is more likely to hold up to scrutiny. The checks are more thorough and you’ll have fewer false positives and fewer false negatives. You can’t fix what you don’t know about, and Qualys is more likely to find everything, so you can do a better job of prioritizing.

Rapid7’s risk prioritization is much better, and its reporting is better. But there’s also a chance with either tool you’re going to download all of the data, process it yourself, or import it into some data visualization tool, and glean insights that neither tool gives you out of the box. Business intelligence and data science are wonderful things. I can probably tell you what you five biggest problems are, I just can’t tell you the order. Qualys and Rapid7 won’t give you that insight out of the box either.

There is no perfect VM tool. Everyone seems to think the other two tools, besides the one they use, is a great product. None of them are great. They’re all underachievers. It’s a class full of smart kids who’d rather play video games than study. The question is which of them eeks out the B+.

I could run a successful VM program with either tool. But my job as a security professional, above all else, is to provide competent advice to whoever’s paying the bill. That requires data I can defend, and Qualys is strong enough in other areas that Qualys gets the higher grade.

The most brilliant line I ever heard in a sales call happened when a prospect complained about their patching team. The sales guy asked, ‘Well, are you telling them what to do, or are you helping them do it?”

Rapid7 gives you the data to tell them what to do. Qualys gives you the data to help them do it, as long as you have the skill to read it.