Apica Reviews

We're a banking software company, so we use it for Synthetic logins to test how one of our end users would log into our product for a customer, how long it would take, what loads, and then log them out. Then we test how long it takes to do that entire process.

On the Synthetic side, we only use it SaaS-based. We actually put it through an SSO. We use Okta for an SSO. That's how we're securing our connections there. Security-wise, Apica's got a couple of things in the works that are going to help them out, but they're not there now. In particular, they're coming out with a key chain that allows us to save. You can hash passwords and users, they don't have that right now. Passwords and logins are set in plain text.

Before we had this particular product we were using SolarWinds to do something like this. The problem with that, however, was that was an internal check, which means it was coming from our network. With Apica, we could do the same type of check, and then we could also make a scenario script that would go and click the things we wanted to click, but it would come from an external source that we did not control. That would give us a better baseline for what the customer sees, as opposed to what we expect to see from our network.

We like the scripting features and the scenarios. It allows us to set up exactly how a customer would log in, what they would type in, where they would click on the screen, and then takes screenshots of it so that we can actually see it happen and see what they see at that time.

We also use it for up-down checks for a lot of our websites that we make ourselves for our customers to make sure the sites are up or down. It's not part of the Synthetic side of it, but we also use the ZebraTester. We're actually implementing various homemade tools on our site as well by API.

We use ZebraTester for some of the sites and other things before they even become into the Synthetic side.

It is highly flexible when it comes to websites. There are a few things that it does fall down on, but for the most part for logging into a website to check to make sure elements are loaded on the screen, it's highly flexible. If I don't want a certain element to load, I can block it or I can ask it to ignore it. If I need to check for a certain element, I can do that as well.

As always, within the IT industry, everybody's always looking to upgrade and update everything else like that. Apica has been one of those things but it's really hard to replace because it offers us the unique capability to see what the customer is seeing. A lot of other ones can do Selenium script and things like that, but there's a lot in Apica that we use right now. We utilize a lot of the scenario options in Apica right now, and there's a lot of other ones that do parts of it, but it doesn't do everything that Apica does.

Apica is indispensable in a few things that we do. It currently is the only one that we have that catches CDN outages. We have many tools that monitor our customer sites, but a lot of those are API logins. If we had a CDN outage and the site didn't load all of its elements, we wouldn't be able to tell that. Apica can tell that because it's looking for particular elements on the screen. Indispensable may be a strong word, but we do highly rely on it for some things.

We use Selenium scripts and we were able to do more specific checks, so it makes it feel like we're actually a customer logging into one of our sites, checking their accounts, and logging out.

The scripting feature has kind of saved money and resources. When we first got it set up, it was a pain because we didn't have the script set up before, but now we have it setup and it's running on multiple checks. Multiple checks, meaning, our Synthetic login checks range around five to 550 checks. Now when we have scripts set up to make the Selenium check, I can pump out new Selenium scripts for one of our online banking customers in five minutes.

Alerts are always accurate, but they might not always be useful. Apica alerts on two different things: one, when an element that is in the script cannot load, and two, when a part of what's loaded comes up with a certain internet error code, a 500 or 402 or something like that. It's always accurate because those things are always not doing that, or they're getting the errors, but it may not actually be as useful. To deal with that, we generally either have to block the URL that's throwing the error code or whatnot, or we have to verify the elements.

It's very accurate but sometimes not useful. It's also noisy. When Apica alerts, it does not have a pull-in time or anything else like that, unless for elements or error codes. It does for SLA times and variances, but not for the other ones. It could be that it's a one-time blip and something didn't load on the screen, it alerts immediately right then. If it loads the next time, it's not going to alert. If it's still set up, it alerts. It can be noisy.

This level of alerting accuracy has saved us time and money in operational costs. With CDN issues, it lets us know, for instance, that we have a homemade monitoring system for our products as well that monitors to make sure that things that should be there are there, but it doesn't actually take into account if the webpage itself is loading. A number of times we've had major CDN outages where our homemade monitoring tool is fine because everything is loaded by an API, but the webpages are not. When that happens, Apica tends to go alert hard and that lets us know that "Hey, we need to go check over here as opposed to over here." That saves us time and money on troubleshooting.

We have two different approximations in terms of how much it's saving us. The way that we do our major incidents, is that we do it per customer. If we have five customers down for five minutes, that's 25 minutes of downtime. I don't have an exact number. I know that things like that when it affects our entire environment are pretty substantial.

It has also saved costs involved in managing monitoring. It has at least saved us in the cost of that it gives us one pane of glass to go to for Synthetic monitoring. I can actually send one of our analysts to go look and if they want to know if a page loaded, they don't actually have to go log in, they just have to log into Apica and check to make sure Apica's running well. That saves time, which saves them money.

Alerting needs improvement. It's a little noisy. It needs some better options. Currently, they have an issue, when you set up a Synthetic monitor, you can set up where it's monitoring from a data center that Apica owns. However, for each data center that you attach to a monitor, that's considered an extra license. That's a bit iffy. They're usually behind on the version of Chrome that they're using for the Synthetic monitors. Currently, they're using Chrome 85, they're 11 iterations out of date. They're trying to get that fixed up with something called Evergreen, which will basically be a Chrome browser that'll stay constantly up to date, but it hasn't been implemented yet. 

The problem with that is that we generally test our product with the newest versions of Chrome and everything like that, so sometimes we've run into issues. Also, when they updated to Chrome 74, we lost some monitoring capabilities that we had before that did not transfer over with this new version of Chrome.

I'd like easier access to the API. Their API, it's not bad, it's just bulky. It's a little unwieldy in the way it has to be used. One of our app developers is currently working with them and he wanted to do a number of calls to the API, and he was not able to do that. They had to make special changes to our API to make the number of calls he wanted to make. It didn't seem to be scaling as well as we thought it would. But they worked with us to actually get it to do that. That's a plus point.

I'd like to see more abilities to do mass changes to checks in the GUI, in the interface. Things like setting a mass amount of blocks for checking a bunch of checks and saying, "Make sure that this URL is blocked on all these checks." Currently, we can only do that through the API, and last time we had to do that, we actually had to use Apica support to do it. 

Finally, they have an audit log system called Journal. However, it can only check, if I remember right, two weeks at a time. That becomes really difficult when you need to check on something that you need to go back multiple times and you don't know the exact dates of the thing that changed. For example, I had a user who got changed in one of my checks and I needed to find out when it got changed. It ended up being three months ago, but I had to go back in two-week increments until I could find it. Their Journal, their auditing system, needs a little bit of work.

We've had Apica for five years. 

We are using the SaaS portal.

The stability is not bad. We've never actually had an issue with Apica, the product. The alerting really comes back down to how this thing alerts, how the alerts are sometimes not useful. That's the worst I could say about the stability. We can turn them off or we could filter them through a third-party.

From what it does, it scales pretty well. I can easily go in, pull on a check, assign a scenario to it and boom, I'm done. I've done that many times. A couple of weeks ago I spent the day creating very specific Selenium checks for very specific parts of a customer's website to make sure that they load properly. It handles it quite well.

We have roughly 100 users using this solution.

We take Apica data and we send it to our data warehouse so that we can do SLAs for our customers. There's me who sets up all the monitoring in there. And then we have our NOC, who will go in there and they use it to actually make sure sites are up and everything. It's used throughout the company in all ways: business side, maintenance, and monitoring.

I do the maintenance of Apica. 

I have 683 checks. For the Synthetic login, the checks, it's 400 checks. Those are the ones that we mimic the login like a customer would log in. For the VT checks, which are basically just up and down checks, we have 112 of those. That's not just our customer sites, we also use this product for our site as well for corporate sites.

We do have plans to increase usage. When a new customer gets added in, they get a check as well. Every customer gets a check.

Support has never been a problem. Their support is top-notch. We either email or get our client experience manager on the line. They have been top-notch, willing to help, willing to go the distance. I have very minimal complaints about support. The one complaint I did have, they actually addressed it very quickly.

We started with SolarWinds, and after that, we moved to Apica. We then got rid of our SolarWinds integration and went to LogicMonitor. LogicMonitor has its own website monitoring tools. However, the problem with LogicMonitor's website monitoring tools is that it's very hard to set up a script the way that Apica does. They also don't provide screenshots of what happens. We've looked at a number of other vendors as well. The problem always comes down to it doesn't do the things that Apica does.

The initial setup was very complex. I wasn't even part of the initial setup, but I know it was very complex because we needed an external source for our checks, but we needed to be able to mimic logging in like the customer did. This was back in December of 2014. I have a feeling nowadays though, they probably have this down to a fine science of how to get people implemented and their stuff up and running.

We have seen ROI. We use it for all of our customers and it does help us. A lot of times it can catch things that happened to the site, but don't happen to the API. We've seen a good return on that.

Pricing is based upon not so much users, but the number of checks you're going to create. Make sure when you set up an account with this, to request more licenses for checks, for any type of check, than you actually need. This will save in the long run. They're really good about setting this up and getting you more licenses but there's always a cost with that.

If you think you're going to need 100 checks, make sure you get 110 licenses. Then remember if you want to do multiple-site checks, not just one-site checks, you're going to get a license for each site.

With all companies, you get the base product, but the base product's not all that you want. You want it with a whole bunch of other stuff with it. We can safely assume that there are probably other costs to add things. Things like additional integrations with other products are not included in the standard license. 

The biggest lesson I have learned from this solution is the sheer number of sites that can load when you load one website. We do online banking, but when you load online banking, it also loads 50 other URLs as it loads through there. That might include Google, Facebook plugins, or things like that. It has really opened my eyes to how many things load when you just open up a single webpage, even if there's that much on the webpage itself. It's very comprehensive when it comes to website monitoring.

I would rate Apica Synthetic a seven out of ten. We've had our problems with it and we're still waiting on some add-ons and features, but for the most part, it's never wrong. It's just sometimes noisy and feels old. The UI is very basic. It's not bad, it's not ugly, but it's basic. It uses old browsers. 

We have various tools, applications, and websites and the use case for Apica is emulating user actions on those sites and in the tools. We use it for proactive action. Before the user starts getting errors, Apica will alert us because we have it monitoring the same actions we expect the users will be taking. Once Apica detects an error it will notify us so we can take necessary action, before it becomes widespread and users start to report it. Apica is doing an important job in monitoring because our company offers services through those sites and the application.

We're using it on-premises and we're using their agents on their cloud.

Because we're using this product to monitor, for example, the sign-up flow, meaning Apica is doing the same actions as a user who would like to sign up, we have been able to figure out when there's a failure in that flow, before it starts to impact users and prevent them from signing up through the services. This is something that has a very high revenue-impact on our company. Apica has helped us find issues with the sign-up before users have started to call customer care regarding the services.

The fact that the solution offers multiple deployment options — on-premise, hybrid, managed cloud solution, bring-your-own-cloud — helps our organization meet our security requirements. Some of the tools in our company can only be accessed internally. To have an internal, on-premise agent makes it easier to test these corporate tools. We have these tools also monitored with the on-premise installation. It helps us to monitor both customer tools, services, and applications, and the corporate tools and applications.

We use the solution’s ability to make use of our own scripts in Selenium and Postman. We're using Selenium to write scripts that can be run for the browser checks, and we're using Postman to run the ZebraTester scripts. Using the scripting feature saves us on resources. It is one of the things that makes the product easy to use. We don't need a specific type of engineer or operator to be able to write these scripts. There are many people who can do the scripting.

In terms of the alerting, since we started using Selenium, which is for the browser checks feature, about 95 percent of the alerts have been real issues. This level of alerting accuracy has saved us time. It helps us to identify root cause quicker. We used to spend something like an hour just to find the root cause, but the ability to have sensitive monitoring reduces it by half. We can identify root cause within 15 to 30 minutes.

Apica has saved us management costs as well. I'm not involved on the financial side, so I can't put a number to it, but I know that we resolve priority-one incidents faster.

We mainly use the ZebraTester and the browser checks. These are the most important scripts that we're using on Apica. 

With the ZebraTester, the ability to have and store dynamic variables, when setting up the monitors, means you can extract that value and use it in a subsequent service call. This is something that has made our lives easier. The most complex monitoring processes are for security purposes: You need to have a fresh token for the user when, for example, he tries to log in. That token keeps changing. To be able to get the results of other service calls that are depending on the login, you need to use that token in the subsequent service calls. Being able to extract that token, store it in a variable, and use it in the other service calls is one of the most complex things. This is one of the features that I like the most because it helps us in configuring these services, in a certain flow, without the need to re-record the whole thing. Being able to extract that value from the service calls is something that has made monitoring a lot easier.

For the browser checks, the screenshots that are available help the engineer or the operator who is on the shift figure out what's wrong or what step is failing.

Also, the flexibility of the solution in terms of the range of protocols it can monitor has been great. The product has been working as expected and it has helped us to cover something like 95 percent of the outages or issues that we have had.

There is room for improvement with the GUI. It's not a big deal, but it would be great to fix the way the GUI is loading. Sometimes when we want the manager alerts and manager checks, it takes time to load all the way. With the whole GUI, if the information appeared quicker, loaded faster than it does now, it would be great.

Also, when it comes to the way the internal agent is installed, because you can install an application on a server, I would love to see the application Docker-ized. If you could install internal agents using Docker or using containers, it would be easier for us to manage them and spin up internal agents. Most of the applications we have now tend to be Docker-ized applications. I'd love to see Apica going that way with its internal agents.

I have been using Apica Synthetic for about five years.

It's stable. Sometimes there's an outage, but it's not frequent.

They offer scalability.

We have about 25 users of Apica Synthetic, people who log in to the tool. About 14 out of the 25 users are engineers with the NOC team and the rest are senior management and engineering leads. We're using the dashboards for management to see the SLAs and the availability of the different websites.

At this point it's being used very extensively. We may increase the number of users in the future, as we have some new projects coming out.

Technical support is always there to answer my questions. They are very responsive. Typically, they all have the ability to support the product, whether it's updates, or issues that we have regarding scripting, or setting it up.

They're just 100 percent available. They always help us on any issues that we have.

We were using Keynote before. I was not involved in the reasons for the switch.

The initial setup was pretty straightforward. There aren't too many requirements in setting up Synthetic. The guides they provided are easy to read and easy to follow. 

Apica support was also always available so we could just shoot a question to them and they would answer right away. During my shift, when I reached out to them via email or a form, they would be able to jump up on a Zoom or a Slack call, to help us right away. They were very helpful. We switched to Apica from a different product and they definitely helped us with recording the scripts that we already had, and with introducing newer ones. They helped reduce the time that we would have had to spend going over the admin guides. Initially, we just wanted to record our existing scripts and they offered to record them. Their support took care of converting the scripts from the language that we used in our old tools to Apica. That saved us time. 

They anticipated our needs during the deployment. They had pretty much everything that we needed when we initially set it up. And when we had a feature request or some kind of additional setup, they tried to provide that feature or help us with a workaround to meet our requirements, even if the product at that point did not have those things directly.

Our deployment, overall, took two to three months.

We had a deployment plan. There was a project manager, and I was involved in writing the scripts and trying to figure out how to convert from the old solution to Apica. Afterwards, I just took care of just recording the new scripts, but there was a whole project for changing to Apica. For deployment and maintenance there were two people involved from our team.

I have not tried all the synthetic monitoring tools out there, but I have tried two of them. They had the same ability to assign the dynamic variables, which is the most complicated stuff that we're implementing, the dynamic variables from one service call to another. But I found Apica offers the easiest way to set this up. This is something that made us stick with Apica, because it's easier to set up the scripts, even with the most complex feature. The two other products I tried have that ability, but it was so complex to set it up. That's what makes Apica better than the others.

Every time I face an issue and reach out to the support, they point me to a part of the documentation. So read the administration guide or the documentation, because they have everything that you need in their Knowledge Base. This is something I learned from opening multiple tickets. It's there in the documents. It now saves me time when I read the documentation.

Apica Synthetic is one of the most important monitoring tools that we're using. 

I would rate it at 10 out of 10 because it's accurate. I've dealt with so many tools and applications, but their support is the most responsive support I've seen. The tool itself offers so many integrations with other applications. It's easy to set up, easy to configure. The documentation is great. The most important part is that the tool covered most of the issues we have and was able to help reduce the time that we needed to resolve the issues and the outages that we had.

Our primary use case is for monitoring. We've got a number of auto finance applications and hosted applications that my teams are supporting. Apica offers outside-in visibility of what a user would experience if they were actually logging into the platform. We noticed that we were missing that outside component. We had a lot of internal monitoring in place for making sure that the user experience was good, but when it came to being able to support our users and report back on issues our users might be experiencing, and work to remediate or identify and resolve issues that our users may be experiencing from the open internet connection that they've got into our hosted environment, it was just not sufficient. So Apica is what we're using that for today to actually give us an outside-in view of what the end-user would actually experience from the beginning to the end and from their overall use case experience for our hosted applications. 

We also use it to monitor the internal service platforms that we use to support our infrastructure, support our environments, and support our internal clients. We use it for monitoring port status and service statuses associated with network-based applications like FTP file transfer platforms, MQ platforms, shared services, SOA platforms, and a number of other internal platforms that we utilize the shared services across our application stacks to serve the service of our clients.

It gives us a clear line of sight into when we're actually having an external network event that's impacting our end users. Previously to implementing Apica, we would have to rely on our end users to tell us, "Hey, we can't get into the website." With Apica and regional monitoring that we have set up in our higher profile application stacks, we're able to tell if we've got a regional network issue, a national network issue, or some other network event that's occurring that may be an internal network issue that's being exposed as a manifestation of user login failures across our application stack. We didn't have that line of sight prior to implementing Apica and so it's really helpful there.

The other thing that it helps with, that is an indirect benefit of doing URL based monitoring with these types of frameworks is that we've actually caught a few expired certificates and we've also caught encryption changes that have impacted our users' ability to access the environment that maybe some service provider downstream to us has changed.

Prior to having Apica, we never really had a clear line of sight into either of those things, other than some automation that we had internally that were basically report based, and they weren't driven off of real-time data like Apica provides. When your cert expires, Apica comes back with an alert saying, "Hey, my check has failed. And the reason my check has failed is that I can't establish an SSL connection because the cert is invalid." That's a great benefit to reap from having that framework in place that wasn't anything that we ever thought of during the time that we were implementing it. Those things are really nice to have.

It's too early for me to actually give a definitive answer on whether or not it had addressed Edge because we haven't been able to build out sufficiently complex user scenarios in our synthetic monitoring areas with Apica. But from what we have set up, it will definitely give us more insight when we're dealing with a complex infrastructure-based failure event scenario. It gives us more insight as to where specific failures are occurring, because it's giving us a lot more data back that gives us detail into where the user experience is actually not functioning. From the diagnostic data that the synthetics that we do have set up from that diagnostic data that we get back because we have an incredibly complex application infrastructure and architecture for some of our apps, we are able to quickly narrow down where within that infrastructure we're actually having a problem with that diagnostic data from the synthetic logs that we get back from the alerts. I would say it does, but we don't really have really deep synthetics setup to the point where I can go to regression test my entire application stack for one of my apps. I just can't do that yet, but it's definitely something that we have in our roadmap of to-dos.

The fact that Apica offers multiple deployment options, like on-premise, hybrid, and managed cloud solutions definitely helps my company meet our security requirements. Some of the internal texts that we need to do require us to have on-premise infrastructure components. Having a hybrid option is definitely helpful. I think the other piece to that is the flexibility to be able to go entirely cloud with Apica is incredibly beneficial because then you get access to regions all around the world that you have a line of sight from that can help you with getting visibility into what's happening from a client use case perspective. For example, if I have a lot of clients in Canada because we do have application products in Canada, I could, in theory, have an Apica presence in a Canadian region that will give me a localized view of what the user experience is like in conjunction with other regional views to help me narrow down when I am seeing a problem; if it's a regional issue or if it's something that's more global in nature. That actually is usually beneficial for us.

We use its ability to use our own scripts. What we use Apica for right now has primarily been based on the importing of Selenium Flames groups that we've developed for mimicking our user transactions. We've also been working on utilizing their automation platform by a ZebraTester and we've been learning how to work with that so that we're still in the early stages with it. But we've been seeing a lot of additional potential from that ZebraTester framework as well. LoadRunner is something that we've been talking about but we haven't really explored that at this point in time.

What we've developed in Selenium is that we've been able to easily convert over into the native Apica workflow stuff for the synthetics that we have configured. Once those Selenium scripts are created, you use it once, and then it's in Apica. The results have been fantastic from that standpoint and the simplicity of being able to use something that's common and standard across the industry, in terms of using system Selenium to create your synthetic transaction scenario, makes it really easy and helpful to actually get into the platform and a little bit more of an in-depth way versus having to learn an entirely different scripting language or having to learn something new in order to do those types of things.

It's hard to say whether or not this scripting feature has saved us money or resources. Because of that flexibility, more people have been able to access that component than normally would be able to. From that standpoint, it has increased our adoption rate. It hasn't necessarily improved outside of that, but with an increased adoption rate, because it's easier to implement and easier to use by more people, we're getting more value out of the framework without having to have dedicated script or dedicated people writing automation for it. 

From our standpoint, there are a number of valuable features. The WebHooks are obviously really great. The alert framework is really good and then the reporting and visualizations that you get from the dashboards are good. Those three areas are primarily what my team's focused on in terms of usage from day to day.

The dashboard view tells you the health of the services that we have monitored, and how the health of the entire infrastructure is doing at a glance. My teams have given me a lot of good feedback that I just keep the dashboard up on my workstation during the day, and if we get an alert, I can immediately go and investigate if I'm in the dashboard. I can also sometimes catch an event as it's occurring so save myself a little bit of time and be able to get in and see what's going on more quickly as a result of that. From the alerting standpoint, that in conjunction with the dashboards that you get, really compliment each other because then you can drill down and actually get into what's happening from a transactional standpoint or transactional perspective, and see where within the transactions that we're monitoring, where what steps are failing, get more details on why those steps are failing and work to mitigate and resolve those issues based on that.

It's that visualization component that really ties everything together and the drill-down capabilities that you get starting from the dashboard that really makes Apica very useful from a day to day support standpoint.

There are a lot of capabilities that we're not really taking advantage of that we could. There are a lot of opportunities to grow in terms of how we're using the framework, especially when it comes to doing things that are more complex, like facilitating deeper checks via multi-protocol based scenarios that tie in with ZebraTester automations that get created or more advanced regression-based scenarios that we might want to set up in the synthetic checks. We're using around a third of all the capabilities that we have available to us so we definitely have a lot of room in terms of what Apica offers for growth and for expansion of our use cases.

The alerting is impressive. 98% of every alert I see come out of Apica is a valid alert. The other 2% of the time, we will get an alert or something will not be right which is because we overloaded our Apica infrastructure with something that we were doing. It was a self-inflicted thing. If you actually remove that from the equation, what you're really talking about is that it's nearly a 100% success ratio of events to real events. 

It's been fantastically accurate at identifying events. The sometimes frustrating part of that is convincing other people that what we're actually seeing, coming out of Apica is a real issue and it needs to be addressed because a lot of times people will just not be convinced by the data that they're seeing until well after the fact. As we've been using the platform more and more, there are more teams out there that are understanding that when a team member brings something up from Apica, it's not to be taken lightly.

From my perspective, I would say it has saved us costs involved in monitoring. It's enabled my resources to work more focused. It's enabled them to work more accurately. It's enabled them to work more authoritatively and it enabled them to work more adeptly. From an operational standpoint, I would say that it's at least improved our monitoring efficiency by 5% to 10%.

Having to install an application on your desktop to utilize something like ZebraTester is a little cumbersome. It would be nice to see that become a web-based application. 

Having the documentation a little more accessible, and easier to digest by people who are just learning how to use the framework, especially when it comes to more complex or more edge-based cases would be really helpful to have. That is really it, but I think the other thing that would be really nice to have, and it's not necessarily a big downside is when the browser agents need to be upgraded, it would be nice if that just happened automatically and it was transparent and seamless to us and to our infrastructure.

We started using Apica around two years ago. 

It's very stable. We don't ever see any issues with it.

It scales well.

We have about 200 technical users and they're mainly technical support application development and infrastructure support teams.

We have a couple of dedicated resources for deployment and maintenance. Obviously, they share responsibilities across different application stacks, but we do have resources available. They're a monitoring infrastructure support team.

We have 200 users. We have 100 Synthetic and another 100 regular licenses, and we've got a lot of room to grow so that we plan on increasing usage quite a bit over the next couple of years.

The initial setup was moderate. We have a moderately complex setup here in LA.

It was moderately complex because we've got the hybridized infrastructure for Apica. If we were entirely cloud-based, we would probably be a little more straightforward and simplified. But because we're using both on-premise and the cloud infrastructure, it just makes it a little more complex.

We've got a number of groups that we've got a center of excellence basically. The center of excellence is helping teams by enabling their use of the platform.

We used their support during our migration. The vendor engagement with Apica is second to none. I've worked with some really big companies out there and I have been seriously impressed with our technical account team, with our support teams, and with our account executives for Apica. They are great partners to us and they listen and they help and they try to help, and they're just a fantastic group to work with.

The level of support reduced the time and costs involved with switching. It also is an ongoing, continual improvement type of strategy. It helps us to implement new solutions more quickly because they're accessible and again, always there to help so it's great.

They anticipated our needs during deployment. The adoption rate has been a little bit slower than what they were anticipating, but that's no fault of theirs.

I'd say it took us a good six months just to get everything set up the way that we wanted it.

We have absolutely seen ROI. The percentage of time saved that we've gotten from this platform has been helpful. That translates itself into client success stories as well.

It would be nice to see them have an enterprise license where an enterprise can just buy unlimited.

Professional services are at an additional cost, but it's very fair.

We also evaluated AppDynamics. 

I would highly encourage organizations that have external applications, web-based applications to definitely consider this platform if they're looking for something to give them an end to end view in the overall user experience.

Having that outside-in view, you don't really think about it at the time when you don't have it. But once you actually have that outside-in view, that really gives you that same context that the end-user has. It's kind of surprising how much more you actually learn about things that aren't necessarily within the infrastructure that might impact your clients and potentially impact us so it's been very revealing.

I would rate Apica a seven out of ten. 

There are quite a few things that we use Synthetic for:

• We use it for error checking and geo-protection checking because we are very regulated since we are in gaming. Basically, if it doesn't see X text on a page, the geo-protection is not firing properly for blocked areas. This checks all our maximized databases. 
• We use it for timings in making sure the web page is optimal so we can tell if someone accidentally seeks a large image up through the CMS site. 
• For the load test side, it is pretty much obvious what it does. It is load testing outside through journeys what we have through Synthetic. It checks the API back to the login services and so forth. 

That's a great overview of what we use it for. We use probably around 75 checks on Synthetic across our three verticals that we do per casino. 

Accuracy is probably around 98 percent. Sometimes, there is a false alert on one of their pops, where it is just loading slow from their architecture and that affects the timing involved. However, that is nothing major. The alerting is pretty accurate. It does show you the correct results. While it may have some false alerts, that is few and far between. If we do see a false alert, we just report it back to them and they fix it.

It helps with releases because we monitor them in staging. We can tell if something is critically wrong before it gets into production, e.g., if it was load related or function related and also what was different in the dev stage. It then alerts us straightaway inside of our production monitors once it has been released. Therefore, it has improved how we run our systems since we monitor multiple environments.

All their pops: They have locations all around the world. Having all the locations around the world is very useful, especially when you're in a license market.

Their alerting system: The way their systems alert you is top-notch. 

Its flexibility is pretty high. You have all your points of pops where you can go to Ireland, Sweden, or X location. I don't think they really need to improve on their flexibility. There are so many settings, different optimizations, and scripting options that you can do. 

The way you script Apica is probably the easiest way of working that you've ever seen. For a QA person, it is very easy because they have the understanding of the tools and what they have to offer. From the complexity side, it is very possible to do pretty much everything on Apica: down to logging in and up deposit, doing other processes inside your website, and loading slot machines to make sure external providers are loading correctly. Because in the world of gaming, you don't buy all your own slot machines. You have the likes of Pragmatic, Betsoft, NetEnt, where you have to make sure all their services are up as well.

The initial screen on their dashboard could have a bit more data, but this is a small thing. It could have more data, so we do not need to drill down to a screen behind that initial information. I would like them to get a little better on the user interfaces that we need to go into.

With PokerStars, I was using them for two years before I left. Since working with GraySnowPoker, I have used them for almost three years (coming up in October). In total, I have used them for around five years.

We have never had a major problem, and there has never been a major outage at all. Your monitors are constantly monitoring. Stability-wise, the system is constantly up. It has few errors.  It is a very stable company with a very stable technical staff as well. You get the best of both worlds

Scalability is huge. You can ask them technical queries, then they will adapt their scripts. There is scalability for having as many monitors as you like. Load test capabilities are there as well. As much as I've ever needed to load test our site, they have been able to assist with it. It pretty much has everything.

Three or four developers use it from the technical teams. On the reporting side of it, the management uses it. For errors, the QA team uses it. Everybody uses alerts because we all get mobile phones. There are probably about 10 to 15 people in our small organization who have touch points on Apica. It even sends alerts to my customer support people in their channels, saying, "We have a problem here. Be aware that if a customer rings up that we can say 'We're already working on it.'" There are multiple touch points around the company because of how the system reacts and how the system generates recalls and alerts. It covers a wide amount of groups who can use it. It doesn't have to just be techies.

At the minute, the solution is being 100 percent utilized. Unfortunately, with COVID-19, we were meant to go up to 180 checks this year. When my new budget comes out, we will have to stay at the same level because we own land based casinos that are in the USA. Our budgets are basically being taken down a little. Because we have no revenue coming through on land based casinos, even though its coming back, it's probably only at 50 percent of the revenue now. Expansion-wise, we would love to expand and sell more checks. From a money point of view during COVID-19, it is very hard to justify increasing checks, even though that actually gives you massive discounts. From a strategic point of view, we can't justify doing that at this point in time until the situation gets a bit more stable with America. It's unfortunate, but it's not very stable there at all, as you can read on the news.

They are always adapting and improving the product. If you ask something from them, they will do a custom script for you. If you get stuck on any scraping for a specific check, they will jump in. Their support team will jump in, guide you through it, help you write the script, and so forth. The flexibility of their system and staff are huge. It's really good.

Don't be worried about asking your account manager for assistance because they will always be there to assist you. From the support side, I can ring my account manager up. My account manager will basically go to a technical contact who gets on the line straightaway (within reason). With another company or ESOP servicer providers, if you ring them up, then you have to set up a ticket. That ticket has to go through second line support, which probably goes through third line support and so forth.

Apica's gurus are very good at answering questions quickly. If you need to escalate to your account manager, they will go directly through to your technical leads. Your technical leads will then jump on the Slack group or phone. From a support and ease of use perspective, it's amazing.

The vendor has supported every need that we had, which is good. There are very few vendors who are able to do because they don't all support your needs, where Apica does. They go the extra mile as well. They are not just trying to take your money, as they're there to support you. They make that very clear in their ways of doing stuff and their support. They are pretty clear that these are good guys and they want to support you with whatever it takes. Especially during this COVID-19 time, they have been extra supportive and have really looked after us as a company. Many providers aren't doing that and just want your money every month.

In their tiered system of how they produce their areas, we know which ones are Tier 1 providers, etc. It is very clear from a technical level as to the ability of the company. I don't think they're going anywhere because they have a very good team. I have had the same case manager coming up on five years along with the same technical support. Thomas has been there from day one.

Before I was at GreySnow, I was using NCC Group (now Eggplant Monitoring). We moved away from them because we saw the ability of Apica's product. I did quite a bit of research at the time. The main differences for NCC Group at the time was they didn't have different regional pops nor did they have the coverage that Apica has. 

Looking back at PokerStars, when we migrated from NCC to Apica, their support wrote all the scripts with Apica. They duplicated all the NCC scripts to Apica, which was a massive cost savings.

With my current company, we started off with Apica. 

The initial setup is very straightforward. Ease of setup is amazing because they assign someone to you. From our point of view, it was less needed because I have been using them for quite a long time, but they even offered to help the likes of me if I get stuck. At the end of the day, due to the amount of times I've used them, I pretty much know how their systems work. If I was going to give it a scale, where one was very hard and 10 was very easy, I would say it was around an eight or nine. It's that simple to set up. You have multiple different contact points to go to. If you struggle, you'll be assigned an account manager and a technical contact. Those technical contacts can be invited into your site groups. They will use Slack and other mechanisms as well. It's pretty much spot on for setup. Everything is documented to a very high level, which is very good.

They assign you with a support agent who can help you with scripts, etc. From the point of view of time consumption, it is time consuming to set up multiple scripts, but that is not a negative. It is to make sure you get the content for right journey and user flow.

You could do deployment and maintenance with a minimum of one person if you're talking about just writing the script. If you're talking about deploying one of their internal things, I would say two people. The whole point about monitoring in Apica is cutting down the resources you need. In general, we can get alerts back sometimes quicker than the provider can get alerts back because of the way the system is functioning.

The alerting saves a vast amount of money for us in gaming. Because when your site is down, it alerts you. If it is something that you can fix, or alert another team to fix, then the gaming sites are backup the quicker that it's fixed. There have been occasions where something has broken through a release that we weren't aware of. It shows you that issue. Then, you get a developer to fix that issue and you're back online. If you didn't have alerts then you would have to wait for the customer reports. If you wait for customer reports, then you could be waiting X amount of time because customers are very good about complaining when they are losing money, but they're not so good about complaining if the site is not up. They just go somewhere else and play, then come back later and play on our site. We will lose money in that instance. It saves thousands of pounds if you look at the rewards it has against its setup costs. It saves us more than the license.

You have our QA team implementing all the scripts to track everything, but this far outweighs the rewards because it's so cost effective and saves a lot of money if there are errors on the site. The alerting is definitely there for a good reason: To prewarn us if there are any problems that we can then get fixed quickly.

We use scripts to compliment Apica or write Selenium scripts to prove points to see exactly the job Apica is doing. This has definitely saved us resources. We have one person looking after Apica with inside our team. If we were self-scripting it or doing our own monitoring system, we would need multiple different servers or a dedicated team of Synthetic people to be able to generate exactly what Apica has produced. Staffing-wise, it will save you a couple of head counts. Therefore, you are looking at probably saving about 120,000 pounds a year in head counts for what it's doing.

The pricing and licensing are very reasonable. At the end of the day, you are using their technology/software and getting X amount of checks for a very decent value. As for discounts, they try to meet your budgets as much as they can. For example, if you need 100 checks and you have X amount of budget for it, then they will try and get down to that price. Costing-wise, it is a reasonably cost product. They will always try and come down to your price if you need them to come down to it by knocking off certain areas. 

We haven't actually had to pay any additional costs for anything. We fit into their model quite nicely.

I have used multiple monitoring solutions throughout my career. Apica is one of the best. The Apica system is leaps and bounds above pretty much everything else out on the market, especially from Eggplant. The scripting tools at Apica are a lot easier than NCC Group (Eggplant) and their reporting is incredibly sophisticated in comparison. Also, their alerting system is more complete. It basically gives you alerts and shows you what asset has failed. It shows you timing for failing assets.

In general, you can do login routines with Apica. The deposit routine of a poker table (or many of our systems), we wouldn't be able to do with other monitoring systems. 

Apica is a complete package. You can monitor every single touch point that can fail inside a user's journey. With NCC, it was very hard to script the journey, point out different APIs, and different failures of suppliers. It would just alert on one individual thing, and it wouldn't really show you where the main error was. It would just show you there was an error with that journey. With Apica, it shows you exactly where the error is, even if it's a misloading asset like a Google Analytics tracking script. It will then show you that missing asset so your developers then don't have to worry because they can see that through Google. Obviously, we can't influence if Google or Google Fonts is up and down. It puts your mind at rest, showing exactly where the failure points are for many other systems.

I would recommend researching what is out there to see just how good Apica really is. Apica is top-notch and probably one of the best monitoring companies.

There are multiple different things you can take off of the solution:

• Your code is not correct. 
• Your image optimizations are not correct. 
• Your geo-blocking has a fault, which means you're in breach of your license.
• How your system is working, e.g., the speed, performance, errors, and missing assets.

There is a lot of in-depth content.

They do meet our security requirements because we are not sharing any private data with them from a software-as-a-service point of view. With on-premise, we had one or two licenses that folks tell us that we could install on our platforms, monitor, prime routines, and so forth. However, from where I am now, the security is fine because you are not injecting anything. If we were injecting any usernames into it, they're test users and marked as test users within it because it's a back-end system.

Even if someone got our Apica password, it would be pointless anyway because we're not exposing player data. We have specific users set up for specific tasks that we monitor, and they're marked as test. They don't go on any revenue streams. 

From a simple point of view, their security is top-notch. They offer different security platforms for different use cases. If I was a bank, then I would have it on-premise and it would meet their security profiles as well. So, I am aware of their security and appreciate the efforts they're going to, but we are just fine with software-as-a-service because we're not declaring any personal information.

I would rate them five stars out of five.