What is our primary use case?
We're a big company with a small IT shop, specifically when it comes to IT security. We needed a partner that could be an extension of our existing team.
How has it helped my organization?
Binary Defense MDR has been that "bolt-on" that we need, giving us extra visibility into our environment and things that may or may not be seen by our existing products. That was really the goal of bringing an MDR in. We have a lot more visibility into things that we didn't before. It has made security operations a little easier, and that was the goal. Automation is king for smaller shops.
It has allowed us to have a little more peace of mind because we know, if somebody knocks on the door or gets on our network or even from the perimeter, Binary Defense is listening. They have adequately detected things and been able to let us know. Even if some things may not be actual threats, they're out there monitoring and letting us know when we might have an issue to look into. The biggest issue for most organizations is that they have people who don't know what to look for or who don't get enough visibility, especially internally on their network. Having a partner that is able to say, "Hey, you've got A, B, and C going on here," and actually filter what's an actionable threat versus what is just white noise, has been a great benefit for us. That was a big problem with our previous MDR: we got way too much white noise.
With every MDR, you go through a period of white noise first for a month or two. That period, for Binary Defense, wasn't anywhere near as long as it was with our last provider. There were reports that we would get from our last MDR that just weren't valuable, on a regular cadence. We also have other tools that Binary Defense integrates with, like Microsoft's Defender for Identity product, and they are able to respond at a faster rate. And there isn't a pile of noise that we have to go through and waste people's time. It's not just done with AI, but they actually have someone looking at a ticket and saying, "I've got X, Y, and Z," or "I've got possible indicators of compromise. Let me escalate to somebody."
Using Binary Defense has saved our organization time. I wasted a lot of time going through white noise tickets from our prior MDR. On average, I would see 10 or 12 tickets from them per day and, 90 percent of the time, they had no valuable information. They were things that really didn't need to be brought to my attention or that somebody in their SOC didn't really look at closely or correlate the data. Their white-listing time period was really long, and I gave them that feedback a lot of times in our one-on-ones.
With a new MDR, you expect a time period where you're going to get hit with white noise. We really didn't have that when Binary Defense flipped the switch. Sure, we've seen a little bit of white noise here and there, but nothing on the scale that we saw from our previous provider. We had a few oddball tickets, but they actually had valuable information, things that I didn't know were going on. We were able to change processes around those. That has been pretty helpful.
For example, our pen tester likes to be sneaky. They used one of our help desk tech's credit cards to create an account to do their tax with, and Binary Defence caught it. Through the purple test, we were able to identify that we needed to see this information. We changed our process so that when those tickets come in, we're able to copy our help desk so that they can verify things.
Binary Defense has definitely helped reduce our IT team's workload. It allows us to focus on things that are high-priority security issues. Our last MDR would just toss stuff over the fence. But this service has helped to reduce the footprint of escalations from a security perspective, and if something is a "911", they'll call and say, "You need to look at this immediately." That is very valuable because we're not wasting people-cycles trying to chase little things that may or may not be important.
Also, I've had phone calls from their SOC team where I have been shocked at how fast they have responded. For example, suppose one of our admins escalates somebody's rights. They have called me directly and said, "I've got a user here that's doing these changes," and that was within a few minutes of them detecting it. That is A1 customer service, especially when time matters from a security perspective. I believe the pitch that Binary Defense gives people is that they respond somewhere within 13 to 15 minutes, but they've always met or beat that.
What is most valuable?
Compared to our past MDR, their time to alert has been fantastic. Before, we had a lot of scenarios where we just weren't getting effective alerts. Binary Defense also has honeypot tech deployed and that has been really great. I've had some really good feedback from other ops folks about that.
Their processes and the way they document their tickets have been really valuable to us too. With our prior MDR, they would say, "Hey, we see this," but there would be no technical details. We're a very technical group because we're so small. We have to have experience in a multitude of things. The tickets from our prior MDR just weren't valuable. With Binary Defense, we don't just get an alert, but also a detailed rundown of why they're alerting us on it. They tell us what was executed, or the username, script, or IP. That way, we're not wasting time investigating.
Customizability with Binary Defense is better than a lot of companies that I've seen in terms of modifying their playbooks and tooling them the way you need them. We've done a few things, especially after doing pen tests and purple tests, and we have been able to tighten or customize a rule. That has been really great. With our last provider, one of their failures was that they didn't really want to talk about their playbooks and there wasn't that level of customization that Binary Defense has. I definitely give them top points for that.
What needs improvement?
Their integration with other applications and tools is not something I would call a complaint, but it is something they need to work on. In my experience, a lot of our integrations are done through APIs. A lot of what I've seen so far from Binary Defense—not all of it—seems to be beta integrations.
For example, their Duo and Proofpoint integrations aren't really what I would call ready for production. They have probably been working with those vendors to work out the kinks, but they're really not 100 percent production ready. And while there isn't really anything valuable we would get from Duo from a reporting standpoint, sometimes Proofpoint's SIEM tool or SOC can see something that might be valuable. We already get alerts from Proofpoint, so it's not a "make or break", but I have given this feedback to Binary Defense: This is something that should go the API route.
Their Microsoft integrations are top-notch and they do some third-party stuff really well for log ingestion, but I would like to see Binary Defense's development team change over to an API connection, versus how they do it today.
Also, if I were shopping for an MDR solution today, I would not only look for a company that has the ability to alert, detect, and remediate, but also the ability to integrate vulnerability management. That's a big thing that they're lacking today. We offset that with another product, but that should be part of their product offering. I've given this feedback to our account manager too.
Another point is that maybe they should have their own SIEM offering. Today, they offer AT&T's AlienVault, which is a good product in its own right, but it's not something that they offer directly from themselves. It's the same thing with Azure Sentinel. They just started offering that as a product you can buy as part of their service, but it's not their own SIEM. I would be interested in seeing them build out their own SIEM and offering that as a product you could buy. That would be very valuable to their customers because they would not have to rely on their folks learning another system.
For how long have I used the solution?
We've had Binary Defense MDR for about seven months.
What do I think about the stability of the solution?
So far, the stability has been good.
What do I think about the scalability of the solution?
The feedback we got from another company that had passed on Binary Defense was that they thought it had good scalability, but they didn't think it would work for them. The reason that it wouldn't work for them was that they had too many kinds of devices. They not only had workstations and servers and endpoints, but they had a lot of specialized manufacturing devices that they needed to monitor.
That is no fault of Binary Defense. That kind of customer isn't really their target audience. Their target audience is shops that are purely Linux/Macintosh/Windows, that don't have specialized industrial products.
How are customer service and support?
Their support varies on who you get, but in comparison with other companies that I've dealt with, like Rapid7 and a few others, I would give Binary Defense's support an eight out of 10. There are definitely some improvements that could be had, but those are more around training. I think they're going through expansion right now, building out their product and adding more folks.
My only call-out would be that they have some techs that need to take a closer look at account notes for customers. They may need to work with some of their newer folks and get them to be a little more attentive or a little faster in their responses.
I've had a couple of tickets where the replies were delayed, but those things happen when you're hiring new people or training them. Sometimes that slips through the cracks and there's nothing you can do about it.
Also, I can't say that Binary Defense has changed our security posture because our relationship, at least so far, has not been like that. They should probably add an SME or an adviser to each account. We have regular cadence meetings with our account manager, but other MDRs that I have PoC'd would always have, as part of their product stack, a time period where somebody from the SOC would say, "Hey, these are the things we saw." That's something our account manager does, but those other services would give recommendations on goals, from a security perspective, that we should look toward.
In terms of our partnership with them, I would give it a grade of 90 percent. Obviously, there are days when things fall through the cracks, you always have to calculate for human error, but it has been a great partnership so far. A year from now—and this is what we do with every product—we will reassess and evaluate and do a pen test to make sure that we're getting the security operations that we expect out of our products and teams. I would suggest that anybody in the midsize company space that needs a good partner to keep an eye on their environment, one that can be flexible in that environment, look at Binary Defense.
How would you rate customer service and support?
Which solution did I use previously and why did I switch?
We had a prior company for a year called Arctic Wolf that we were not happy with at all. One of my colleagues who is a pen tester had suggested Binary Defense, and we ran a PoC with them. They did really well, scored really high, and they have been a great partnership so far.
When we did the purple test with Arctic Wolf, they failed really badly. We gave them an opportunity over the course of a year to improve but they just weren't improving. They were wasting our time by pointing out things that were more "busy work" instead of actually addressing their problems. The big buzzword that they used was in saying, "Well, this is in beta." I would say, "Well, you're a production-level company. Everything in your playbook should not be in beta." I would hit them with random pen tests and, in almost every instance, they would fail. They might catch one or two things here and there, but they were things that I would have expected anybody to catch. And the things that they failed on were things that any MDR product that's worth its marketing should be able to catch.
How was the initial setup?
We don't have anything in the cloud, we have some other sensors on-prem, but we do have their agents deployed everywhere. That means we have some Azure assets that have their agents installed. We have API integrations with all of our Microsoft products and some third-party stuff such as Proofpoint and Duo MFA. All of our servers and workstations have their agents deployed and we have the AT&T AlienVault SIEM set up. We have logs from our firewalls being sys-logged up to that and a few other products. We're somewhat blended but it's mostly on-prem.
In terms of the initial setup, my career path has been engineer-level, so for me, it was pretty straightforward because I'm familiar with how their configurations work. Someone who is, say, only five years in, might have a little more of an issue. But their documentation is done really well, once you get access to the portal. They have links that walk you through it step by step to do the configuration. Obviously, there is an expectation that you are at a certain level of IT experience to install and deploy it. But it was well within my expectations of what an MDR deployment process should be.
The follow-up, from an implementation standpoint, could have been a little more fine-tuned. For example, with Proofpoint, Binary Defense doesn't use an API integration. They use some script with a VM to ship that data off to their cloud ingestion. I think there's a better way to do that but I can't fault them for that because that's what they're getting from Proofpoint. Overall, the complexity of the setup, between one and 10, was a seven or eight. For me, it was really easy.
The only maintenance that I've seen is agent-based, and that's all done through the cloud.
What about the implementation team?
I worked with Binary Defense to implement the system. They have an implementation team, a few engineers and a product manager, who walk you through the process. Our experience with them was good. It was better than my last MDR deployment, in terms of how they ran the process.
Near the end, it was our team pushing forward certain things, but the things we were griping about were things that are out of their control, waiting for third-party vendors to address. But that is something I would have liked to have been apprised of earlier in the process, that they really didn't have a great integration with this or that.
But overall, it was a night-and-day difference in comparison to Arctic Wolf.
What was our ROI?
We have seen ROI with Binary Defense, and my leadership is in agreement with me on that. Our CIO and our director of infrastructure both said they have had a really good experience with them, especially when compared to our last MDR.
It has saved us a lot of time, headaches, and money overall. Today, it's not if you're going to be breached or attacked, it's when. That's the reality of the security space, so monitoring and having those insights are key. There will always be threat actors who try to exploit something. For example, as soon as the SVB failure happened, there were reports of phishing attacks increasing exponentially for customers of that bank. The threat actors understood they had a little bit of a window to attack these folks because they were vulnerable to a phishing attack. That's something you have to address in this space.
With our prior MDR, there would be three or four days during the work week, and sometimes weekends, where I was wasting time responding to tickets. Now, going into the evening or the weekend, the only time we get a call is for something that has actionable intelligence. That has been a relief for our team. We know that if something crazy is happening, they will call us and tell us we need to address it immediately.
What's my experience with pricing, setup cost, and licensing?
All IT companies inflate their pricing to some extent. But sometimes companies don't have a big budget. The provider comes in with a high number and then they whittle it down to what both parties can accept.
Binary Defense is hindered by the fact that the SIEM they are offering is a big part of their price point and they have to eat it, sometimes, when they try to get a midsize or small company. They either have to filter down their log ingestion or lose some of those logs at the end of the month to meet their cap. That was one of the only positives with Arctic Wolf. I don't know who they were using, but they claimed to have an unlimited monthly amount, and then had cold storage for 90 days.
That is one thing that is lacking for any MDR. It's not necessarily an issue specific to Binary Defense. It's just how their pricing structure has to be.
From the initial cost that Binary Defense came in with, we pared it down quite a bit over the course of 30 or 60 days. My leadership would say that their cost was high, but realistically, they were in line with the market.
They have a good product offering in terms of their XDR strategy, but they could retool it a little bit. I've talked to CSOs and other people in the security space— and this isn't just a problem for Binary Defense—but they have what I call the "package problem" where they try to "line-item" things. I understand they're a service and trying to make money, but the big players like Binary Defense and CrowdStrike need to talk with their marketing and product line people and say, "We need to offer an all-in-one solution."
Binary Defense also offers things like deep web scans and a new product that is a collaboration with ExtraHop Networks. They should look at providing their own product. And that deep web stuff should be in an all-in-one package. The reason that we didn't go with that is I didn't think the cost was worth it when there are third-party or even free tools that you can use to supplement that. It just didn't seem like the value was there. If it had been an all-in-one package, as part of the MDR, there would be more value in that. Maybe a larger company that has a bigger spend might be more inclined to mix and match and buy parts of it, but a midrange company like ours needs an all-in-one solution.
Which other solutions did I evaluate?
We PoC'd, Binary Defense, Rapid7, and a third product, but we eliminated that last one pretty quickly because it didn't meet our needs. Rapid7 had what looked like an okay product offering but we got some feedback from some CSOs and other people in the security space who were not happy with the product overall.
I wasn't happy with Rapid7 because they would not let us PoC their SOC. They would only let us PoC their MDR bolt-on. Near the end of the process they said we could PoC their SOC, and that really annoyed us because they had wasted our time saying that we couldn't do that.
At the end of the day, from a technology standpoint, Binary Defense met our needs for an MDR provider and checked all the boxes. They went above and beyond and that really played into our final decision. I also got some feedback from pen testers who said they had gotten really great feedback on that company and that influenced our decision as well.
You can't just take response time into consideration. The information that they respond with is just as important. But Binary Defense succeeded in both those aspects every time. During the PoC, their response time was really fast, within a few minutes. In my experience, they have been in the 89th to 90th percentile for response time, given what I would expect from MDR.
With Arctic Wolf and Rapid7 as examples, they would alert within a pretty decent amount of time, but the information wasn't as valuable. They would link to MITRE ATT&CK documentation, but that doesn't do you any good. I want to know "who, what, where, when". That's essentially what I get from Binary Defense, the meat and potatoes information that is most valuable to me as an IT professional.
There are two types of MDR and security solutions today. There is the kind that deals with IT people who want to see a certain level of information so that they can investigate things on their own, and they need the right information in front of them. Other solutions are for IT folks who are at a certain level but they only want just enough information to allow them to check off a box.
The playbooks for Arctic Wolf and Rapid7 were not that impressive. Rapid7's technology was a little better than Arctic Wolf's, but neither was anywhere near as complex or as well thought out, as a product stack, as Binary Defense.
What other advice do I have?
Anybody who is looking for an MDR needs to have a serious conversation with their leadership about their needs or what problem they are trying to solve. That's what we did. In our case, the problem was that we didn't have enough people to keep our eyes on every single bit of day-to-day operations from a security perspective. We needed a partnership, an extension to our staff. And it has been great.
The big "gotcha" is you need to figure out what you need and what your expectations are for the cost. You have to weigh what it would cost to pay a full-time security person throughout, because they're not cheap. The market is screaming for security people right now. There are hundreds if not thousands of companies looking for security people. IT leadership has to say, "We can either pay X dollars per year for a SOC service to help us manage things, or we can get a manager and a handful of SOC analysts to bolt on to our existing staff and pay them 2X dollars a year."
Another driving factor is cyber security insurance. That space has changed a lot, especially in the last five years. That was a big talking point within our organization because we needed to tick the boxes or we were not going to be able to get cyber insurance.
Another factor to be aware of is long-term stability. Mandiant was on our shortlist when sourcing our prior MDR solution, and we didn't go with them because they were way overpriced. But what ended up happening was that the company was split up. So that was a concern of ours with Binary Defense. You see that in the market. There are companies that are really hot, they do really well, and then they get to a certain level and they're bought by a larger company. That was a conversation we had with our account manager. We asked what their "five-year" looked like and what their growth looked like.
That was the big concern for our CIO: Are we going to be replacing you in a year or two? Are we going to reevaluate this conversation because the relationship has changed or the quality of the product has changed because you've decided to have a third party invest in you and now you're not giving us the same product stack or customer service that we had? That has not been an issue so far. Based on the outlook that they gave us, it seemed that it was not going to be an issue.
Disclosure: PeerSpot contacted the reviewer to collect the review and to validate authenticity. The reviewer was referred by the vendor, but the review is not subject to editing or approval by the vendor.