CRITICALSTART Reviews

We were looking for a third-party managed detection response provider for our integrations with Cylance and Carbon Black. We had to deploy the Cylance and Carbon Black agents after we received them from CRITICALSTART.

Types of challenges that we were looking to address:

• 24/7 monitoring
• Reducing alerts.
• Getting Level 0 and 1 taken care of, along with that first triage of alerts. Those are taken care of before our team has to look at it.

The way that the user interface presents data enables our team to be able to make decisions significantly quicker, rather than have to dig into the details or go back to the original tools.

The transparency of data in the platform is perfect. The way they built it out, you are seeing everything as they are seeing it. There is not a black box; it's not the magic sauce happening behind the curtain. You have the ability to see everything that they do right there in the console.

The service has significantly increased our analysts’ efficiency to the point that they can focus on other areas of the business. We went from triaging an email inbox and a few other tools to being able to manage the queue appropriately at regular intervals. We also have begun looking for other tasks or items to further advance some of the analysts' careers.

Services have been fully delivered on time, on budget, and on spec. Whether it be for implementations, go-lives, or enhancements for anything that we want to add to the platform, they have always been consistent, ready, and willing to help out, build out, and troubleshoot should there be any issues.

I have a very small team and anytime I can maximize efficiencies within the work I'm trying to do with Kirby, it's a good thing. That's what I was trying to do by using CRITICALSTART.

The most valuable part of the service is the time saved. CRITICALSTART helps with so many of these alerts that my team and I don't get alert fatigue. It saves us time to concentrate on the more important things. It probably saves us a day or two, 10 to 15 hours, a week.

I also talk to CRITICALSTART analysts and the value in that is immense. I just talked to my Board of Directors about that this morning. The value from it is what I'm spending on the service versus what I would have to spend to build a team like that internally. It's at least one-fifth of the cost. There's value in that for me. And their availability is generally pretty quick. I've never really had to wait very long for anything. The availability of the analysts where they will say, "Hey, I know we sent an alert on this, but you should really take a closer look at it," via a phone call or a message, is just phenomenal.

In a given quarter, I get 589,000 security events and 584,000 of those get reduced by the service before they even get to me. The alerts that actually come through to me end up being about 1,400 in that quarter, which is a 99.7 percent efficiency rate.

We were looking for a managed service provider who could handle our endpoint alerts as well as our SIEM alerts. We were looking to address alert reduction, better correlation, and reduction in head count that would ultimately lead to a more secure environment.

We brought our own endpoint solution into the equation. We added a full functionality SIEM solution. There wasn't a whole lot of infrastructure. 

The transparency is extremely effective. The ability to maneuver through the GUI (the front-end) allows the team to be more effective and perform their job efficiently and effectively. They can bounce around, get in there, and know what they're looking at, which gives them the ability to really dive into the alerts. It's a user-friendly front-end.

From where we were prior to going into them, the service has increased our analysts’ efficiency to the point that they can focus on other areas of the business. It gives me the ability to allow analysts to do Level 3 and 4 work and stay out of the weeds of the alerts, where you tend to get alert fatigue. The service takes care of much of the Tier 1 and Tier 2 triage. It is more effective than what we had been used to, because it allows the filtering of Level 1 and Level 2 type alerts to be taken care of. This leaves less for us to handle, which is a good thing.

We remained within budget. They continue to work with us to add additional logging sources to effectively meet our budget requirements and are in line with our cost cutting efforts.

We are using it to try and improve our cybersecurity overall. We are also using it to reflect on our business growth whether we need to invest in more cybersecurity.

We started as a small, family-owned  business which was purchased by a U.S. company under the same umbrella. That company wanted to have all their portfolios have a higher level of security. This was an initiative taken by the parent company. This came at the right time because we started to get more phishing attacks as we started to manage more users. There has also been more requirements on the IT department to keep us secure along with more focus in today's world on IT security. Previously, we didn't really pay as much attention because we always thought we were a small company, and thought, "Who would want to hack us?" I guess that is no longer the case.

The service for endpoint protection needs to have an agent installed on the endpoint, and that is pretty much it. There is no specialized hardware required to use their service.

It removed a huge task from my shoulders onto someone who it's their profession to do this because I'm not from a security background. It definitely makes my life a lot easier. In terms of company, we have invested in something sophisticated and management knows that we have access to a 24/7 service. It makes them feel happier as well, especially these days when you hear about attacks, etc. For them, knowing we have a service like this in place is a good thing.

I receive probably less than five alerts per week. Most of them are caused by OpenDNS, which means there is not much they can do. These happen when our workstation is trying to reach a destination with IP addresses, then it will raise an alert because it suspects someone is trying to bypass the DNS security to go directly to a certain destination. With that kind of alert, the only thing we can do if we don't think it's safe is block it in the firewall. With the service from CRITICALSTART, they don't have the capability to actually block individual IP addresses. That's why those alerts keep coming in whenever there's a new IP. Our regular processes, like our ERP software, are mostly filtered and no longer come up as alerts. This has being cut down by probably more than 80 percent compared to day one.

On whatever CRITICALSTART does, it will show up and be logged. If there is an alert, and someone made a comment or did something, it will all show up in one place. That has sort of a paper trail of what people did. Because we have agents installed on endpoints, I don't know exactly all the details of information that are sent to CRITICALSTART. I assume since this is Zero Trust, they probably be sending everything because we keep thousands of processes with a playbook and a whitelist of filters. So, I never go in and actually check exactly what's being sent over. As far as I can see, if they done anything, like putting something to a whitelist or triggering/disabling a filter, it all shows up.

Now, all I need to do is just go in. Luckily, we're relatively small. With most of the alerts, I'm able to address them right away because I know exactly what they are and they have done most of the leg work, then I ask the team if they will take care of the rest. That definitely saves a lot of time on my side. I can't really make a comparison between now and before last November, because we didn't do much because we weren't equipped to do much then. There might have been something going on, but we didn't know because we didn't have the resources for this kind of service.

We now have the tools and the support to actually have a clear view of what's going on. Before, it was just the traditional antivirus installed on the computer. Whatever it did, it was done without us because we couldn't really do much except block something or whitelist it. There were no humans involved. I'm not spending too much time on this because most of the jobs are done by the team from CRITICALSTART. All I do is just help them confirm whether the alert is legitimate or another regular process that we haven't playbooked yet.

We're a small shop on the security side and our goal with CRITICALSTART was to alleviate some of the constant looking at our phones 24/7 and allowing somebody who is actually sitting in front of a computer 24/7 to handle the front end alerts that come through our automated services or systems. As those come in, we wanted them to be able to escalate to us as seen fit. We were looking to weed out the lower priority, false-positive portion of the alerts.

Due to our size limitations, we needed assistance with the lower level alerts so that we could focus with the real, priority alerts. Because of the use cases that they've built up in some of the logging systems that we already had they were able to amplify the type of alerts that we were getting in a way that gave us better and more visibility than we were receiving beforehand.

All of the hardware and software that we were already utilizing was already in place. We were able to offload the management of our Splunk environment. CRITICALSTART began to manage this for us. That alleviated a good portion of one of my analyst's time, to where they didn't have to manage that them self by allowing CRITICALSTART to manage it. We have it 24/7 so if something was to go wrong, they can look into it.

Sometimes the hardest part of showing a ROI with Security in general, is the fact that when you do not have an incident there.  Then comparing that to when a peer has an incident, and how much you are saving because of the tools in place to prevent specific types of attacks. 

My impression of the transparency of the data is that it has good detail. It allows you to see how many events have come in, how many of those events have made it down to their analysts to review, and then how many have been escalated to us. It's a good metric that we can share with my management. They see the value of what the SOC is bringing on top of what my team is already doing.

CRITICALSTART does not take care of the tier one and tier two triage on the Splunk integration that we utilize. If we move to the endpoint integration, then it could by providing the ability to lock down a system and providing additional contexts that Splunk logs are not capable of. But as of right now, no. That's just because of what we decided to go with on the first round to give them a shot by doing the Splunk integration.

They haven't missed a one hour SLA to resolve an escalated alert as of yet. We haven't had to enact on their commitment to pay a penalty. It's supposed to be that they will deduct it from our renewal rate.

This type of SLA commitment was reassuring. But I was already well established with CRITICALSTART long before deciding to go with them on the MDR. My relationship with them was what really drove this inner engagement. I used them for other services such as with Pen Testing, architect,  and  purchasing equipment.

We needed a company with expert solutions in the security field. We needed to secure our internal network, external users. CRITICALSTART has resources and know-how in those specific areas. The second part was that we needed assistance with security, hardware support, and implementation of Palo Alto firewalls, and they are the experts in that too.

There are additional features on the Palo Alto firewalls, security on the level of the apps. The users cannot go to certain places. There's a service that gets set up so we don't have to manage it; there is an automatic shield on those firewalls. Software-wise, we use CRITICALSTART to manage the ZTAP (Zero-Trust Analytics Platform). They manage an antivirus solution for us by Cylance and another protection level is Cisco Umbrella. They manage and monitor our systems with their MDR solution.

For example, alerts come in from the Cylance antivirus to their systems and the CRITICALSTART team informs us and helps us combine the white lists, the black lists, what's allowed, which machines are behaving abnormally, and they monitor various aspects.

It is deployed to over 100 people within our company. That is the user base.

In terms of the MDR, if we didn't use CRITICALSTART, we would have to hire a full-time person to sit and do that job. It frees up resources. It's far less expensive for the company to hire CRITICALSTART instead. And CRITICALSTART has a large knowledge base in the field, whereas we would have to learn within our company how things work. With CRITICALSTART, we tap into the knowledge of all the companies that they manage. It's definitely a win for us.

There was the initial adjustment period, as every environment is different. Initially, they came in and looked at our stuff, our alerts. We tweaked things a little bit, but then we could tell that out of thousands, or even hundreds of thousands of alerts, we were only getting, say, 10 tickets per week from CRITICALSTART, if that. The rest of the things they handle automatically, or their system handles them automatically. It really frees up our time quite a bit.

It allows us to free up our resources. We don't have to get into the super-deep details of the alerts if something is happening. They bring a vast knowledge of the threats to the table. We don't have to research them ourselves so it frees up our time.

And they've previously seen the resources we use for the Palo Alto designs, and they know our environment because we have a person that deals with us directly. It's so much easier to work this way, versus if we were to hire somebody from a large consultant like CDW or Softchoice. With a third-party like that there's always a learning curve — you have to invest so many hours first — before you get to the problem. With CRITICALSTART, we can engage them right away with problem solving. There's no onboarding every time. They already know what's going on.

We have a SCADA system which is something that our field team operates 24/7, all year round. It's a pipeline. We have the Cylance umbrella solution on those critical machines and if something gets blocked by an error we get an alert right away on the mobile phone. We respond and CRITICALSTART comes in and makes live changes. That prevents us from having any downtime due to a blocked file on some system. If it's a bad file, it will get blocked, obviously. That's great. But if it's a false positive, we are able to get CRITICALSTART, using the mobile app, to respond right away and prevent downtime of the SCADA system.

We needed a SOC operation, and we weren't going to build it in-house, so we were looking for exactly what they offer. They're an MDR service, and we were looking for somebody that would manage the SIEM tool as well as the endpoint management tool and have the ability to take action, when necessary, on endpoints and function as a full, hands-on SOC. That is why we selected them.

The service doesn't require us to make use of any hardware. The software required is Splunk, as a SIEM tool, which provides options as to how it's managed. We opted to have CRITICALSTART fully manage it, so we're hands-off with the SIEM tool, and it's hosted in AWS. Then you have to have an endpoint endpoint detection tool that CRITICALSTART has approved. I don't know what their current selection is, but a year-and-a-half ago it was either Cylance or Carbon Black. We're using Cylance.

Our use of the service covers 100 percent of our endpoints. We're covering 1,100 endpoints.

We didn't have a security team before. If I were to say the service had improved our organization, it might lead you to think we were doing security a certain way before, but we weren't. I came into the company as the first security professional for them.

The service has increased efficiency for me to the point that I can focus on other areas of the business. Again, as a department of one, and not having to attempt a one-person SOC operation, I'm able to focus on the strategic security posture, the architecture, for the company, and focus on where our keys to the kingdom are. I can also pay attention to compliance, which is part of my role. I'm able to do my job because I have this outsourced SOC.

The challenges we were looking to address were mainly around making sure that my team wasn't overloaded with alerts and that we could tune out things we don't care about or that aren't important to us at that particular time. That was really what I was trying to accomplish, since I knew I wasn't going to be able to build out a team large enough to be 24 by seven.

Before we really had anything in place, or when we didn't use them in a managed way, we felt overloaded with issues like: "How do we deal with all these alerts?" "Is this a problem? Is this not a problem?" "Are there other customers that are also experiencing this?" It was pretty easy for us to justify paying a little bit to get some help on those things and get the benefit of their experience with the other customers they have on their platform.

In terms of the transparency of data on the platform, what comes to mind is that I've asked them a few times, "Hey, we've got this weird alert that you've escalated to us and we don't really know what to think about it. Have you had any other customers that have experienced it?" Obviously they're not quick to say, "Oh, well, Company XYZ had the same experience," and for good reason. But when asked, they're usually pretty good about saying, "Yeah, we've had some other customers that found this, or we've worked with them to determine it was this or that." Some of that you get upfront, but there are times when you do have to prod to get more information about something. Once we learn more about it, it affects our security operations because we're pretty small. So if I know that a large organization has spent time on this and had other analysts looking at it, analysts who have determined it's this and that, I'm going to lean toward what they found. I often just don't have the resources to do that myself, or it may be because I have respect for the security organization of that company. It's definitely valuable.

Using CRITICALSTART has increased our analysts' efficiency to the point where they can focus on other areas of businesses. That's definitely been a benefit of the whole thing. Instead of worrying about every little alert coming in, we really only pay attention to the ones that we need to pay attention to, the ones that are escalated to us. Otherwise, we would just be thumbing through thousands of things that likely don't really matter that much.

We have different groups throughout our company that use the equipment that we give them in different ways. So we've reached out to CRITICALSTART to build out groups and we can update those groups ourselves with different peoples' usernames. That way we can say, "All right, Nmap for the engineering group is always allowed. Don't ever alert us about that," or perhaps we make it a low alert as opposed to high. But if it's any other group, or if a user falls outside of those groups, we want to know. And that's been really useful for us in bringing down the number of escalations to us, things that would pop up as "high" at 8:00 at night, because some guy's running Nmap or something similar.

CRITICALSTART also takes care of Tier-1 and Tier-2 triage. In terms of time saved, I've always assumed that if we did this ourselves, I'd have to have at least a minimum 24/7 staff, or at least a few shifts throughout the day to cover the amount of things that would have to be researched and looked at.