On a recent episode of Down the Rabbit Hole, Rafal Los and James Jardine asked CISO-turned-CIO Joe Riesberg how he measures the effectiveness of a security program. He came up with five things, which are pretty much how we measure our effectiveness where I work too. That’s a pretty good indicator.
Incidents. How many incidents do you respond to and how quickly? Because no matter how good you are, you have incidents. As you gain maturity you spot more of them, and as your defenses improve you start to see fewer of them because you start preventing them, and as you get better you detect them faster and close them faster.
Patches. How quickly do you patch your systems and how many of them actually take?
System uptime. Good security doesn’t decrease uptime, contrary to popular belief. Poorly thought out security may, though. So when your security is uptime-neutral or uptime-positive, then you know you’re getting better.
Recovery time. This is what I’ve been preaching hardest lately. High achievers aren’t high achievers because they never make mistakes. They can afford to make mistakes because they can recover so quickly. Every security program I’ve ever inherited or (shudder) started was paralyzed by the fear a patch would break something. But if you can roll back that bad change in a matter of hours, would you do it? Most would. What if you could roll it back in minutes? If you’re really good, even that’s possible. When you won’t try because you’re afraid, though, the previous three will never come.
Opportunity cost. A good, hard measure of what you’re missing because of [x] is the key to moving forward. You can’t sell the next initiative without being able to give people a good idea of why they need it. A leader who just chases fads isn’t going to be in that leadership role for very long; leadership that’s going to last for the long haul will want something quantifiable to make a decision by.