“Threat detection and response” strategies are becoming par for the course in security today, which is undeniably a good thing. However, implementing a -DR solution is not a “set it and forget it” type of undertaking. Left to their own devices, these tools may or may not do the job effectively, though all will try.
Here’s how to keep an eye on your threat detection and response solution and make sure it is doing what you’ve asked it to do; namely, that it is operating at a level at which it can effectively combat the number of attacks it is up against, and in enough time to count.
Technique #1: Evaluate your detection coverage
First, you have to figure out how much of your environment is actually being covered. As Grant Oviatt, Head of Security Operations at Prophet Security, puts it, “Detection coverage measures the percentage of detections your team has implemented and tested that align to a known framework, namely MITRE ATT&CK. The purpose of this measure is to illustrate the likelihood that your operations team would receive some form of signal in the event of an incident.”
You figure this one out logically; out of so many behaviors outlined in a specific framework (say MITRE ATT&CK), how many detections do you have in place? The bigger solution, and better than just a number, is to identify which of those techniques are most likely and use your limited resources to watch out for those.
Technique #2: Find out your false negative rate
Now that you see which detection nets are in place, you need to find out how often they mistake real fish for fake ones. This can be even worse than generating false positives because while with one (false positives) you run around and waste a lot of time, with the other (false negatives), you let a malicious attack go by undetected and unobstructed. In other words, you give a green light to attacks.
These mistakes can occur when teams are tired or trained incorrectly, but they still need to be identified, analyzed, and stopped. While this may sound ambitious, you want a false negative rate of below 1% (and ideally 0%). You figure out your rate of false negatives, or investigation error rate, by sad trial and error, and this step can be unfortunately hard to do. Luckily, there are some tools on the market to help you with this.
Technique #3: What’s your MTTD (Mean Time to Detect)?
Mean Time to Detect (MTTD) is how long it takes from the time an attacker infiltrates your network to the time your team realizes what’s going on. To figure this one out, you just subtract the “Activity Started At” time (for the earliest spotting of the incident) from the “Alerted At” time. This will let you know how quickly you are on the draw.
If you’re doing well, you’re going to catch signs of nefarious activity between 30 minutes and 4 hours after they’ve happened. If you’re not, that time could be days - or, you’ll know about it when the data’s breached. It goes without saying that this metric also needs to be as close to zero as possible, and you can bring it down by auditing your detections: are they working? Are the traps set up in the right areas and catching things as they occur, or do you need to invest in detection coverage elsewhere?
Technique #4: MTTR (Mean Time to Respond)
Now, you have to see how long it takes you to get out there and launch a counter-attack (after you’ve been notified of the suspicious behavior). Mean Time to Respond (MTTR) is marked when you’ve successfully cauterized the wound, not just made an effort, and “measures the average time it takes to control and remediate a threat.”
Be fair with yourself; many companies opt to separate their MTTR rates by how severe the incident. So, one hour is a good goal for a critical case, while 8 hours might be natural for something not so serious. In any event, your MTTR should also be as low as possible. Here’s how to tighten them up:
- Check out which inputs result in the best MTTR rates (are you quicker off the bat with certain alerts - maybe certain people?). Follow those leads and try to see what you’re doing right.
- Pick apart your process - can you sharpen up one of the elements in order to bring times down?
- MTTR is a composite of waiting alerts, investigation times, actual response actions, and more, so determine which elements you’re good at and improve on the others.
Technique #5: Do you know your SOC capacity?
First off, do you know what your SOC capacity is? SOC capacity is simply how much time your team has to look into alerts (“Oh, that thing we never have” - yes). Along with this, you also want to know your workload, or Expected Work, every month. Subtract one from the other, and you will find the gap. You knew you had it, but now you know exactly how much space it takes up.
Ideally, you want your team to have 15% more time on their hands than total alerts to deal with in a month. If this sounds like a laughable pipe dream, keep in mind:
- It’s only going to get worse as AI continues to flood the landscape.
- It’s only going to get better as AI floods the landscape.
If your SOC has a negative capacity, consider (in addition to figuring out where things are breaking down):
- Fine-tuning your detections (low initial cost, you can do this in-house).
- Reaching out to managed SOCs to help carry the load.
- Investing in automation.
- Leaning heavily into AI-based solutions that can cut down on things like false positives, route investigations, and fatigue.
Conclusion
It’s time to do a little auditing and figure out how well your threat detection and response tools are really responding to the tasks at hand. By utilizing these self-investigation techniques, teams can take an X-ray of their -DR solutions and find out what’s really going on. Maybe knowing the truth won’t immediately bridge the gap, but it can tell you how it needs to be bridged and give you a good idea of where to start.
About the author:
An ardent believer in personal data privacy and the technology behind it, Katrina Thompson is a freelance writer leaning into encryption, data privacy legislation, and the intersection of information technology and human rights. She has written for Bora, Venafi, Tripwire, and many other sites.
0 Comments