CISO at a hospitality company with 1,001-5,000 employees
Real User
They take care of all first-line alerts, with eyes on glass, fingers on keyboard; they're doing the work, allowing me to focus elsewhere
Pros and Cons
  • "I also use their mobile app. It's very easy to use and very convenient to be able to respond to alerts wherever you are. I love the app. You can respond and communicate, per ticket, with their SOC in near real-time. The response is very quick."
  • "The updated UI is actually pretty bad. Regarding the intuitiveness, it is fairly easy to use, but the responsiveness, on a scale of one to 10, is a one. It's really poor performance."

What is our primary use case?

We needed a SOC operation, and we weren't going to build it in-house, so we were looking for exactly what they offer. They're an MDR service, and we were looking for somebody that would manage the SIEM tool as well as the endpoint management tool and have the ability to take action, when necessary, on endpoints and function as a full, hands-on SOC. That is why we selected them.

The service doesn't require us to make use of any hardware. The software required is Splunk, as a SIEM tool, which provides options as to how it's managed. We opted to have CRITICALSTART fully manage it, so we're hands-off with the SIEM tool, and it's hosted in AWS. Then you have to have an endpoint endpoint detection tool that CRITICALSTART has approved. I don't know what their current selection is, but a year-and-a-half ago it was either Cylance or Carbon Black. We're using Cylance.

Our use of the service covers 100 percent of our endpoints. We're covering 1,100 endpoints.

How has it helped my organization?

We didn't have a security team before. If I were to say the service had improved our organization, it might lead you to think we were doing security a certain way before, but we weren't. I came into the company as the first security professional for them.

The service has increased efficiency for me to the point that I can focus on other areas of the business. Again, as a department of one, and not having to attempt a one-person SOC operation, I'm able to focus on the strategic security posture, the architecture, for the company, and focus on where our keys to the kingdom are. I can also pay attention to compliance, which is part of my role. I'm able to do my job because I have this outsourced SOC.

What is most valuable?

The most valuable part of the service is that they are 100 percent taking care of all first-line alerts. With eyes on glass, fingers on keyboard, they're doing the work. If they have a question, or they haven't seen something in our environment before, then they will escalate it to me. The service takes care of Tier-1 and Tier-2 triage. They actually provide a report that gives details on how much that saves us. I looked at it when we first started, and it was multiple FTEs, on an annual basis, that they're saving us.

I also use their mobile app. It's very easy to use and very convenient to be able to respond to alerts wherever you are. I love the app. You can respond and communicate, per ticket, with their SOC in near real-time. The response is very quick. I can close tickets, I can escalate them. I have very close to all of the capabilities that I have on my desktop. All the things that I need to do in a ticket, I can typically do them from the app. I am a one-man show. I'm the only security analyst for our organization. I couldn't really do my job without the app. I can't sit in front of a computer all the time, so it's critical for us.

I communicate with CRITICALSTART's security analysts. I haven't spoken with them over the phone, except for one time, in a year-and-a-half, but their accessibility is very high. I always receive quick responses to my escalated tickets. When I'm commenting, they're following up, and they're very fast.

I feel I have full transparency to their SOC. Anything I want to go look at, I can do so. I can see all of the comments and discussions that the SOC team has on behalf of us. I have full transparency.

In terms of CRITICALSTART contractually committing to paying a penalty if it misses a one-hour SLA to resolve an escalated alert, I honestly haven't looked at the contract in a year and a half, so I don't remember if it's monetary. I believe that it is. They're very proud of their SLA and not missing it, so I've not ever had an issue or concern or had to think about it. This high commitment to SLAs was our CIO's primary concern when we were looking at CRITICALSTART. After seeing their record, 18 months ago, of not missing a single SLA, it became a moot point. It was a concern at the time but they satisfied that concern.

What needs improvement?

The updated UI is actually pretty bad. Regarding the intuitiveness, it is fairly easy to use, but the responsiveness, on a scale of one to 10, is a one. It's really poor performance.

I have shared this next point with them already, but I would like to see a monthly report to talk about advancements or new alerts, anything to do with what we call IOCs — indicators of compromise. When there is anything that they have changed on behalf of their customers on the backend, they should say, "Hey, we have made these modifications. We're now looking at these types of alerts." It would give the customer a sense that they're actively looking for new IOCs. So I would like a monthly recap of what they have done, not specifically for me, but what they've done for all of their customers. That would be good.

Buyer's Guide
Security Orchestration Automation and Response (SOAR)
February 2024
Find out what your peers are saying about Critical Start, Palo Alto Networks, ServiceNow and others in Security Orchestration Automation and Response (SOAR). Updated: February 2024.
756,650 professionals have used our research since 2012.

For how long have I used the solution?

I have been using CRITICALSTART for a year and a half.

How are customer service and support?

I would rate the customer support, post-deployment, as highly as it can be rated. Their focus on doing the right thing for the customer is how you would hope that every company you deal with would respond to customers. They are 100 percent focused on doing the right thing for the customer, and they back it up. I've seen that multiple times.

In terms of project management, in the lifespan of managed detection and response companies, I'm an old customer now, at 18 months. Back then, the project management was poor and that was part of the reason our roll-out was delayed. CRITICALSTART took all of the necessary steps to revamp that department and correct their mistakes, and that's why we were compensated monetarily, as well. It was poor then, and I haven't had the experience of working with the revamped project management team, because I'm already established.

In terms of delivering services on time, on budget, and on spec, we're a little bit of a unique customer. I know that because we had some early growing pains. They did miss the scoping of our network, which did impact the budget. I brought it to their attention and they stepped up. From a monetary standpoint, they made it right, with no fight. They just recognized it. They have a great ability to put themselves in the customer's shoes and do the right thing on behalf of the customer without any friction.

Which solution did I use previously and why did I switch?

Prior to CRITICALSTART, we were a customer of Arctic Wolf.

It's really not even fair to compare the two companies, because Arctic Wolf was not a 24/7 SOC operation, even though they sold themselves as that. It was more like a managed SIEM service. They used a proprietary SIEM. I cannot say anything positive about that company. Not a single thing. Right from the time for migration and sending the SIEM tools back to them, it was a very bad experience. They don't do what CRITICALSTART does. Even though they try to market themselves as an MDR, they're really not an MDR. They don't manage the endpoint tool, so it was really apples and oranges.

How was the initial setup?

There wasn't really an initial setup required at our end to use this service. The implementation of the endpoint tool, in this case Cylance, was a requirement for us. That involved some GPOs and the Splunk forwarders that we implemented in our environment. But as far as man-hours on our side to do the setup, it was very low.

It was straightforward. Pushing out software is something we do. Creating GPOs to make sure that the correct data from servers was being pushed and directed to the Splunk forwarders was all typical, sysadmin-type work. Nothing was complicated.

There were no data sources that this service wasn't able to integrate with.

From the time we entered into an agreement to use them, it was about four to five months until we started using it, but a lot of that was dependent on our ability to get the product rolled out, and our activity for base-lining the system, or our environment. Some of that time span was us, and some of it was them, but they made monetary compensations for the delay that we had. While it didn't go as fast as we wanted, the end result was positive.

What was our ROI?

We are absolutely seeing return on our investment from CRITICALSTART's services. They're doing the job of a 24/7 SOC at a fraction of the price that it would cost me to run it myself.

What's my experience with pricing, setup cost, and licensing?

You get what you pay for.

Which other solutions did I evaluate?

Compared to the competitors that we looked at, CRITICALSTART had a longer history, even though they were a young company. I liked that they were not using proprietary tools in the environment. That allowed us the freedom to move, if we wanted to, to another provider. They were just ahead of everybody else in terms of maturity.

What other advice do I have?

In terms of advice, I don't feel that implementing this service is any different than implementing any other system into your environment. A lot relies on your project management skills.

I would attempt to test your MDR choices against a framework. The framework that comes to mind is the MITRE ATT&CK framework, which everybody is familiar with. Have realistic expectations about what vulnerabilities your MDR partner is really going to mitigate. That's the lesson I have learned.

In terms of CRITICALSTART's Trusted Behavior Registry and the way it resolves things that are known as trusted, so that the focus is on resolving unknown alerts, I'm obviously not looking at all of the alerts that they work on. But what they escalate to me, only the alerts that I'm seeing —which is a small percentage — if I were to rate them on a scale of one to 10, I'd rate this aspect at eight. There are a few things that slip through, things that they'll escalate that I know should not have been escalated, but it's a very small percentage of what they actually escalate. It's a very small percentage where I'll have to just say, "Hey, did you mean to do this one, because we've been through this before," or a virus total shows that it's 100 percent clean, so why did it get escalated? It's not common but it does happen.

The service missed a pen test, but I still have a high level of confidence with the data and the actions they take. We had hired a red team, so the situation was a red team test. Red teams are generally 100 percent successful, or very close to it. With them, you always expect to uncover the unknown. But I do have confidence in the tool and the data that they are looking at.

The number of escalated alerts we receive, compared to the number the service's Trusted Behavior Registry resolves, is probably less than 5 percent of the total.

Disclosure: PeerSpot contacted the reviewer to collect the review and to validate authenticity. The reviewer was referred by the vendor, but the review is not subject to editing or approval by the vendor.
PeerSpot user
Systems Administrator at a energy/utilities company with 1,001-5,000 employees
Real User
They tell you they're going to cut your alerts by 99 percent and they did that, freeing me up for other things
Pros and Cons
  • "The most valuable feature of their service is their tuning... If we were getting 1,000 alerts a day without them, they tune it until they know what to do for 999 of them, and one will make it through to us per day. That tuning is the most valuable part of their solution."
  • "They just did a user interface overhaul to the website portal that you use for troubleshooting tickets. The old one was fine. The new one is not intuitive..."

What is our primary use case?

What I was looking to achieve with this service was to have less work on my plate, and to leverage people. Usually, when you buy a big product like an antivirus or endpoint protection, if it's a big solution and you have a big company, you need another person to just manage it or things like it. We didn't have those resources. We got the antivirus product, but we didn't have another person to add to it, so I needed someone to help me manage it.

CRIICALSTART is helping me manage this solution because I don't have time to manage it.

Originally, they were managing CylancePROTECT for us. Now, they manage CylancePROTECT, Carbon Black Defense, and Palo Alto Cortex XDR for us.

How has it helped my organization?

They take work off my plate and that frees me up to work on other things. The fact that I have time to do more of my job isn't game-changing for my company, but for me it's a huge deal. Otherwise, I'd be spread so thin. What would have happened if we didn't CRITICALSTART is that I would either have been getting thousands of alerts a day and having to ignore everything else, or we would have used a different security product that is less noisy but also less secure. And then, maybe, we would have been compromised and not even know it.

Our expectations have been met in terms of services delivered on time, on budget, and on spec. When you sign up with them, they tell you they're going to cut your alerts down by 99 percent, and they did that. They did that with Carbon Black Defense and they did that with XDR. That's all I could really hope for.

What is most valuable?

The most valuable feature of their service is their tuning. All the service really does is get things to the point where we get fewer alerts sent to us. If we were getting 1,000 alerts a day without them, they tune it until they know what to do for 999 of them, and one will make it through to us per day. That tuning is the most valuable part of their solution.

When we had Carbon Black, we were getting at least one escalated alert a day, maybe more, because it wasn't able to be tuned the same way that other services can be, or maybe Carbon Black itself alerts that much more. With Cortex XDR, we're only getting about one escalated alert a week, or one a month. It's much less.

What needs improvement?

They just did a user interface overhaul to the website portal that you use for troubleshooting tickets. The old one was fine. The new one is not intuitive and I hate it.

It's an information overload issue. When you go there, there is a bunch of stuff to look at. I had to get a walkthrough last week because I didn't know how to get to the one screen that I'm looking for when I use it, the one that shows the tickets that I have and the tickets that I don't have. I couldn't figure out how to get to that. In the middle of the main screen there's a little button that'll take you there. And at the top there's a search bar and a filter that helps you find tickets that are assigned to your organization or their organization, tickets that are open, tickets that are closed. But it's not intuitive.

For how long have I used the solution?

I have been using CRITICALSTART for one-and-a-half years.

What do I think about the scalability of the solution?

If they expanded the scope of what they can ingest and did so at good pricing for managing other services and remediating other issues, I would definitely look into expanding our usage. At this point, I don't know what else they take in, other than endpoint protection.

How are customer service and technical support?

From a project management standpoint they have performed very well. They're very organized. They're very reliable and responsive. Their customer support is a 10 out of 10. I'm always happy to hear from them and see them.

I haven't had any problems since they've been managing XDR, but back with Carbon Black I had a lot of problems trying to understand why something was being alerted this way and why this or that was being blocked. They helped me troubleshoot all of that stuff as well. And they do it within their SLA. It's nice to have that insurance that they should be responding within an hour.

Which solution did I use previously and why did I switch?

This is the first time I've used a managed service provider for managing anything like endpoint protection.

How was the initial setup?

There was an initial setup required at our end to use their service and they helped me take care of that. It was very straightforward. There were a few settings for me to change and there were a lot of settings for them to change, and they just remoted into my machine and helped me do it. Either way it was not rocket science for me.

We've used this service with three different products. For the first one, CylancePROTECT, there wasn't a portal for me to log into. That was all behind the scenes. We didn't get to know what was happening. They just took care of everything. 

When we had Carbon Black Defense, we had the old portal, but that was a year-and-a half-ago and I don't remember how long it took to get set up. It hooked in pretty quickly. 

With Palo Alto Cortex XDR, we were either their first or one of their first customers to use that service, so it took a little bit longer to get everything set up correctly, even though we were already connected to them through the old service. We were in the system immediately, but we weren't in full-on production mode for about four-and-a-half months. That's not that bad because they were actively managing it until then.

Which other solutions did I evaluate?

I looked at Arctic Wolf. There were some others as well. But the pricing of other services was so insane that they weren't even an option. And they don't do exactly the same thing. CRITICALSTART has a narrow scope that fit our requirements. I had a problem and CRITICALSTART specifically works with that thing. I don't know if they do other stuff now, but when we started working together, pretty much all they covered was antivirus.

What other advice do I have?

If you have people who already do this at your company, and they're paid well and they know what they're doing, and you have multiple products like this that they can manage, then you don't really need CRITICALSTART. But if you are a small group of IT people trying to support an entire company and you have a crazy, complex product like CylancePROTECT or Carbon Black defense or Palo Alto Cortex XDR, or anything like that, then it's probably better to leverage an expert company like CRITICALSTART.

The only data source we are using them to manage is our antivirus and they integrate with that. I don't know if they would have been able to integrate with our other data sources. We didn't try that.

I have used CRITICALSTART's mobile app but I haven't used it lately because we get so few alerts that I don't really need it. A lot of people use the mobile app for when they're home on the weekends and they need to get stuff remediated quickly. We don't have people working on the weekends, usually, so it's not a huge issue for us. If my company is working, I'm at my office and at my computer already so I don't need the mobile app for that.

The mobile app has the basic features that you need to use their service. I don't remember if it lets you link to the service they're managing; for example, I don't think there's a link to the Cortex XDR app from CRITICALSTART's mobile app. So you can't really dig deep into anything on there, but that's not their fault. It's just because you can't do that, period. But for quick remediation or quick alerting, it's perfect.

I haven't spoken to CRITICALSTART's analysts lately. During implementation, we had weekly meetings. Usually I only talk to them when things aren't going well, so the fact that I haven't talked to them in a while means we're good. But they were always available when I needed them. If I needed them quickly, they could join a meeting within a day.

Out of all the service providers I've had to work with over the years—I've been here six years—CRITICALSTART is my favorite to work with. I see them at almost every convention that I go to, no matter what city I'm in. I'm always happy to see them and they always recognize me. I feel like that's worth something when you're looking for someone to work with. They have a personal touch.

Disclosure: PeerSpot contacted the reviewer to collect the review and to validate authenticity. The reviewer was referred by the vendor, but the review is not subject to editing or approval by the vendor.
PeerSpot user
Buyer's Guide
Download our free Security Orchestration Automation and Response (SOAR) Report and find out what your peers are saying about Critical Start, Palo Alto Networks, ServiceNow, and more!
Updated: February 2024
Buyer's Guide
Download our free Security Orchestration Automation and Response (SOAR) Report and find out what your peers are saying about Critical Start, Palo Alto Networks, ServiceNow, and more!