Apica Synthetic Reviews

Apica is used to perform availability checks of our IT services. We put what we call synthetic checks in place, and these are mainly used to check if a specific application is running correctly, or if it is not available.

Because it is a platform that allows us to check the availability of our services, we have a process in our company that allows us to open an incident when we receive an alert that is raised by Apica. It is really critical in our company to have a tool like Apica, because every time we have an alert we know that there's a real problem in our system and we can forward the problem to our internal team so that they can take charge of the problem and solve it as soon as possible.

We are very satisfied with the flexibility that is offered by Apica. In our opinion it is much greater than in other products, even more expensive products. We found a good balance between the cost, after our spending review, and the features that it offers. The alerting is very reactive as well as very accurate. We are really confident in the alerts that we receive from Apica. The alerting accuracy has absolutely saved us time because we can minimize false positives and that means that we don't have to spend the time dealing with them. On a monthly basis it is saving us about 20 hours of work. That is the amount of work we did on false positives that we received before adopting Apica. That might seem like a low number, but trust me, when we have a critical issue, 24 hours are a lot.

We decided to move from another provider to Apica because it offers, from our perspective, more features and more advanced use case coverage. For example, it has a feature that allows us to set up business logic on the availability of our services. We can apply multiple criteria to trigger an alert. We have availability checks that allow us to check two different services at the same time and, to trigger an alert, both of them have to be down. One of the very difficult things with this kind of product is the possibility of false positives. Thanks to the flexibility that Apica provides, we are able to minimize the false positives, and that means that when we take charge of a problem opened by Apica, we are very confident that it is a real problem.

We are also using some JMeter scripts. At the moment, the platform itself is not using JMeter scripts, but they provide a converter that allows you to convert a Jmeter script into another language called ZebraTester. Thanks to that, we are using our JMeter scripts without any problems. And that means we can implement automation in the scripting and, obviously, that adds up to spending less time and effort on these automated activities. It's quite critical to have a tool that provides you this kind of automation. Apica also provides public APIs that allow us to run these kinds of scripts on demand. That is a good thing when you have to develop some automation to achieve very specific needs and tasks that are very frequently executed.

Overall, Apica has definitely saved us costs involved in managing monitoring, although I can't put a number on it.

There are several features that are really good. The first one is the flexibility and the advanced configuration that Apica offers when it comes to configuring synthetic checks. It provides the ability to customize how the check should be performed and it is very flexible in the number of synthetic locations that it can use. It allows us to run scripts from different locations all over the world, and they have a really good number of these locations.

There is also the ease of use. The user interface it provides is really advanced, but at the same time, it is really easy to use. That's a really good feature when it comes to daily use and our daily processes on the platform.

It is also very good in terms of the range of protocols it can monitor. Even if, at the moment, we are only using the HTTP protocol and browser synthetic checks—it's mainly the emulation of the end user browser—they also provide other protocols, such as DNS verification.

The first thing that I would suggest they improve is the user interface. Not from the point of view of how to access the features, but how they are presented. The user interface is very clear, but there are some components of the user interface that are not up to date. Just to give you an idea, today we have web applications that are called single-page applications that are much faster than the old style of web application. If we can move faster into the flow of the graphic user interface, and in a more effective way, it will save us a lot of time.

Another feature they can improve is related to how easy it is to set up what they call on-premises locations. Apica offers locations all over the world, but they also offer a manual to install a location on-premises to check the availability of services that are not public. This process, at the moment, is not so easy to achieve. The last time we did it, we were forced to contact their support to set it up. The automation of this kind of setup is not good. It should be something that does not require human involvement to follow the deployment. The possibility of being totally independent in installing and using an on-premises location would be much better.

I've been using Apica Synthetic for about one year.

The stability is absolutely excellent. We haven't had any issues so far. And when there was some kind of unavailability of the service, because no software is perfect, they advised us before or, if not before, as soon as possible, to let us know about the problem. This is definitely a good approach since if you tell us the platform is under maintenance for a problem, we can change our internal processes to take the unavailability into account.

It's very flexible in terms of scalability. At the moment we have about 2,000 scripts running on the platform without any problem. It's absolutely critical, since we are quite a big company, and moving from a software that is quite famous—because CA Technologies is a famous vendor—to another vendor, Apica, that is smaller, could be a risk. But after the PoC, we really trusted the Apica product. We are very happy that the platform is reliable and very scalable.

We used CA Nimsoft Monitor and Dynatrace Synthetic.

We were migrating from another product to Apica, so the main task that we performed was to migrate all the scripts that we were using on the old platform to the new one. We were able to do that thanks to the professional services support that Apica provided us. They offered us exceptional help in performing this task. It was mostly implemented using the APIs that Apica offers, since migrating data from a platform to another one requires some automation. You couldn't think about doing it manually. The Apica platform was made ready for our use cases because they provided the API that we needed to perform the migration.

The second task that I performed internally was to let the company know about the new processes to be implemented using Apica. The technology is the "engine," but then you need to build the "car" around the engine. That meant we needed to develop processes to let the people who were interested in using the platform know how to do so.

In summary, the first technical task was to migrate all the scripts from the old solution to the new one and the second step was to develop new processes, based on how Apica works.

Overall, our deployment took one year. But the level of support we received from Apica during our deployment helped reduce the time and costs involved in switching to their product. Without them, it would have taken double the time. Thanks to them, the time needed was reduced by a factor of half. They anticipated our needs, meaning that every time we asked them something specific, they replied right away, "We can do that. Don't worry."

It's not possible to provide ROI numbers for a simple reason. Last year we only performed the migration of the platform from CA to the new platform, so we have only been using Apica officially starting this year. We need more time to collect this kind of number. But the perception that we have after the assessment that we performed at the very beginning, is that it will halve our cost and double the performance of the processes that are related to the adoption of Apica.

We checked the new Dynatrace Synthetic platform. We decided to dismiss that and move to Apica after we performed the PoC, since we found that Apica was much more flexible than Dynatrace Synthetic.

Another main difference between Apica and the other products was the cost. We really thought that the balance in Apica between the features and costs was the best among all the products on which we did a PoC. 

There is also the support and the innovation that they bring. One of the reasons we decided to leave CA and Dynatrace is that they are bigger companies, but they are slower when it comes to solving a problem or when it comes to implementing a feature that we request. Apica is smaller, but being smaller means that you are even more flexible and more available when it comes to solving a problem. For example, Apica provided us with a totally new feature that we requested, before we moved to them. That feature was not in the Apica platform and it was critical for us. In about one month, they were able to develop that specific feature for us.

If you are looking for a product that offers a huge technology modernization, and quick support, you should take Apica into consideration, for sure. It is a small company compared to others, but they are really quick in answering your needs and providing you modern technology. If your company is growing and is looking to add new monitoring that is up to date, I would warmly suggest Apica.

We decided to use the SaaS version because we are trying to change the model of services that we are using in our company. We are trying to minimize the on-premises products because we don't want to be in charge of the management of the infrastructure of things that are on-premises. We are absolutely confident that Apica respects our security needs and that we can use Apica safely.

We have an eCommerce company. so we manage a lot of online stores. Our main usage is to make sure that our store web pages are different types of web pages, so there can be a homepage, search page, or product list page for each online store that we manage worldwide. We leverage a lot of Apica Synthetic probes to make sure that every store is available and responding as we expected worldwide. This is to ensure that there are no outages in specific parts of the world. Also, we collect some performance metrics, like response time, time to first buy, etc.

We are using the web-based service. While we do have some on-premise probes installed, we use the service on cloud the all the way. We installed the probes two or three months ago.

We see the benefit almost every day. It allows us to be alerted whenever there is a store that is not responding properly around the world. We do have a network operation center (NOC) who receives these alerts, immediately checking if everything is okay. There are some false positives, depending on the website configuration that we are checking. Other times, the alerts are real. We receive a very quick alert through our NOC so we can immediately check and try to find the root cause. This is our biggest benefit. 

Another benefit that we leverage is the creation of reports every month, regarding the availability of all our stores. This is something that is needed by the executives of our company. They want to see this report with the SLA availability of the stores. We are able to do this straight away with Apica Synthetic.

We don't have a lot of very complex monitoring cases because most of our checks are basic HTTP checks. However, in some cases where we use the solution with ZebraTester for scripts and monitoring, we have been able to properly import the scripts used on other platforms, reworking them so that they would work with the platform and have no issues.

The features that we use probably 99 percent of the time are the HTTP and HTTPS checks. We set up a lot of them. This is 99 percent of our current usage of Apica Synthetic as well as some full browser checks, but this is a lower amount. We also use some scripts that utilize the platform, but our usage of them is very small. 

An very important point for using Apica is the ability to have Chinese probes, which is not common among these types of tools. Other important things were the Grafana native integration and PagerDuty integration, which are all tools that we use extensively.

When we started using Apica Synthetic, we lacked a very important feature that was readily available in the following months. The alerting is usually very good - it allows if any websites or web pages are not responding properly. What we missed was the ability to aggregate alerts. This means that if we were monitoring one website with multiple probes worldwide, like Tokyo, France, and London, then Apica Synthetic would initially alert us if any of these three probes were responding with errors. We wanted a less sensitive alerting. For example, we could be alerted if the website was unavailable from all three probes, not just one. However, an aggregated check was not initially available. 

We do have two main open topics with them, regarding the features that we would like improved or added. 

1. The reporting part that we use for our executives needs a bit more customization capabilities. Right now, you can use only the three main templates for reporting. We would like to be able to customize them. 
2. The management of their single-sign on authentication does not 100 percent fit our requirements.

I have been using this solution for around two years.

The stability is very high. We had probably one outage in two years. I am very happy with its stability.

It is very scalable. Of course, scalability comes with a cost. If you need to scale out, that means you would need to use more of your licenses. However, that is reasonable.

There are mainly three to five administrators of Apica who check the platform, configure things, etc. These administrators are all part of the monitoring team. Plus, there are another 40 to 50 users who access the tool to check the collected data. Sometimes, they will also add some checks, but most users only check the outputs.

Right now, we are around 80 percent adoption. We would like more people to use it on our side. On the Apica Synthetic side, we would like to cover some additional checks that we haven't had time to add.

Their support team is very helpful. If we miss something, they make sure this is easily taken into consideration in the future. This is very important for us, because we tend to evolve pretty quickly. Also, in terms of requirements, they are very responsive, which we really appreciate. On a scale from zero to 10, I would say that they are very close to 10. I would probably give them a nine. I am not giving them 10 because 10 is perfection. I am leaving some room for improvement, but it is a very high rating in my opinion.

For now, the coverage is 100 percent of what we need. We were using another tool before migrating to Apica Synthetic. So, we made sure that all our needs were fulfilled and Apica Synthetic was covering all those needs. 

The reason that we switched was mainly because the previous solution that we were using did not have all the features that we needed. It was not really well-maintained. Often, it had some outages, which were unplanned. Overall, its usability was very low. So, it looked like a very old, not updated tool. So, we needed to find a better tool for our purposes.

We were using Broadcom ASM. I know they changed the name a few times, but initially it was owned by CA, and now it is owned by Broadcom. 

The initial setup was very straightforward. 

The solution went through our InfoSec review, which is something very important for us. The fact that we can deploy the service in a hybrid way is very useful because we do have some web pages available that are only using our internal network. We used to have two separate kinds of solution in order to monitor Internet-facing websites and internal ones, but thanks to this hybrid solution with the on-premises probe, we are able to use only one tool for both use case scenarios. This is very good for us.

We did use the professional services provided by Apica to help us with the migration, since we were migrating from an existing tool. Therefore, it was not from scratch. So, we leveraged them, and it was a very straightforward process. I would rate the level of support that I received during the migration as a nine (out of 10). This level of support helped reduce the time and costs involved in switching. Without the support, we would have had to do that entire job on our own. This means a lot of man-hours, and we couldn't afford that much because we don't have a lot of free capacity.

A few times, they solved some issues or problems with what they were importing or moving. They notified us, providing us with a better option. So, they anticipated our needs.

The migration took around four to six months. 

One person from our company was involved in the migration. This person's role is as an application performance specialist. His main job is to manage our application performance and availability tools. During the scope of the migration, he supervised that the checks were migrated properly as well as responding to Apica in case they had some issues or suggestions.

Apica Synthetic is used to avoid losing money, rather than make money. 

Our JMeter script is loaded into the Apica Synthetic platform. If Apica Synthetic didn't have this feature, we would have been forced to either choose another solution or add an external service to adjust for this purpose. In this case, they saved us money by allowing us to use only one tool.

The aggregated check has since been implemented by Apica, and now we are using it. Thanks to this, we are seeing when there is really a problem instead of just seeing some glitches and issues with the alerting. This has saved us time in operation costs because instead of having to check every time for just one probe that is failing, now our NOC center is able to focus on the important ones. We are saving a third of the alerting, e.g., if we were alerted and had to check three times a day, we are now doing it only once. 

Apica Synthetic reduces the time required to fix issues. If our website is down for hours, then we lose money. So, the less time it takes for us to be alerted of a problem, then the less money we lose.

The pricing is very reasonable, but it is not cheap.

We did some PoCs with other tools before selecting Apica Synthetic.

I would recommend it to get in touch immediately with Apica Synthetic support to have a chat with them and discuss best practices. They are usually very helpful since they are knowledgeable about the tool, so they are able to suggest the best way to implement checks.

I would rate this solution as a nine (out of 10).

Apica Synthetic is definitely one of our core pillars on the synthetic side. We also use synthetics as a measure of external customer application availability. So, we do a daily report internally, which goes out to the tech leadership team, showing how their applications are performing and how available they are. So, it is an integral part of our monitoring tools, and the synthetics are huge.

These are complex multi-step synthetic checks. The intent is to mirror as closely as possible the points and clicks or API/system-to-system calls that our customers are using. So, if anything is not operating properly, then teams are alerted who can triage and ultimately resolve the issue.

The primary use cases are SaaS, but we do have an on-prem environment for Apica Synthetic as well. That option is very helpful because we do have a number of applications that don't have external endpoints. For those use cases that are only accessible internally, we do leverage the Apica on-prem model. This allows flexibility when monitoring applications that we couldn't with a strictly SaaS deployment.

Apica Synthetic provides insight on our application availability at the enterprise level.

What is huge for us:

• The availability of reporting.
• Finding issues before our customers do.

Our application SREs do script checks in such a way that closely mimic our customers' actions using the platform. Because there are so many different ways and options to be able to configure checks to closely mirror your applications' capabilities, it provides a lot of optionality for teams to create the right type of check that can notify when there are any issues. At the end of the day, we want our monitoring tools to be able to catch any outage before our customers do. This is where Apica Synthetic does a great job.

There is definitely a lot of flexibility. I haven't run into any issues or heard of any issues from our SRE teams that said they weren't able to get Apica Synthetic to monitor or script in such a way where it monitored their applications effectively from a synthetic perspective. 

We have had some use cases come up, like when we have teams logging on through a VDI or multi-factor authentication where we have to think about things a bit differently. We are still working through how we might leverage Apica for those types of use cases. However, generally speaking, it has enough flexibility to be able to monitor the complex apps that we typically use it for.

We have been focused on reducing polling times for synthetic checks. We have gone from 10 minutes down to five minutes for a pretty broad swath, but there is some appetite to reduce that further, which could be an improvement. 

We have been using Apica Synthetic for close to three years now.

Stability overall has been pretty good. We have had some isolated issues with a node going down here or there, but generally speaking, it has been good.

Check frequency has been a scalability challenge for them. Other than that, scalability has been good. As far as geographic distribution of nodes, there are no issues.

They have fantastic, outstanding technical support as well as outstanding account support in general. I can't say enough good things about the responsiveness from these teams. Whenever we have an issue, there is fantastic support.

The initial deployment was relatively straightforward, though it is very different between SaaS and on-prem.

There is certainly reputational impact when applications go down and customers find that before you do. There is obviously revenue impact when an application is down and customers are not able to use it. Pick your favorite MTTX number, e.g., Mean Time To Detect or Mean Time To Repair, and having strong monitoring capabilities from a synthetic perspective is a big part of that.

Our ROI on Apica Synthetic is risk reduction. It has increased revenue due to improvements in the mean time to detect and mean time to repair that the solution brings to the table, minimizing downtime. That certainly all goes into our return on investment. At the end of the day, we wouldn't be using the tool if we didn't feel that it is providing a significant benefit to the organization.

License management is another area that Apica could do better. We have already had these conversations with our account teams. This is something that they are looking at largely improving in upcoming releases. I believe that this is already on their roadmap.

Frankly, there are other tools in the space, but we have subject-matter experts on Apica Synthetic now who know it well. We have a good relationship with our account team and have had it for multiple years. So, it is a fairly sticky platform. Where if we were going to a different tool, we would have to learn it. We have established relationships from business and contractual perspectives, and Apica Synthetic has met all our requirements from a synthetic perspective.

We do some load testing internally on JMeter. I know that capability exists, and we have advertised that internally. I am just not sure how much traction that it has gotten just yet.

At the end of the day, it is a tool. You need to have teams using the tool correctly. That is just part of the onboarding and training, which is another thing that my team does. Generally speaking, if the script is instrumented correctly, then the results are correct as well.

We look at three broad strokes from a monitoring perspective: end user monitoring synthetics, application performance monitoring, and infrastructure monitoring. We look at those as three very separate pillars.

I would rate this solution as an eight (out of 10).

We monitor various applications provided as dependent services. We also monitor internal applications that are required for different departments. Then, we have a wide variety of applications supporting different departments as well as clients. Whether it be a background transaction, front-end UI, or vendor application, we try to monitor the DNS with Apica

We have been using it both on-prem and off-prem recently. The Apica platform has its own external instance.

You can tell from the operational space of people who are using and consuming this data that they are more integrated. It is not dependent on one team anymore. It saves a lot of time by capturing and pinpointing the exact problem that is happening quickly. We have moved from getting escalations manually to getting escalations synthetically.

It has centralized the monitoring into a single pane. With the wide variety of testing tools out there in the market, we have been able to streamline them more into a single, proactive dashboard that manages the scheduling, configuration, and alerting with a very robust API. Those big benefits really brought together our synthetic monitoring objectives.

They have given us a lot of flexibility in order to get past our security. Overall, they provide a wide range and customization, which helps us get through any issues, but there isn't one solution for every organization or a good wide-breadth of options.

Their platform does allow us to import JMeter on our files. That provides us some flexibilities, allowing us to use different tools. Their API is also very robust so we can do integrations quite easily by using their core product, synthetic monitoring, as a base. We use that as a trigger for other platforms that will compliment resolution. If the Apica platform continues to monitor successfully, then we can take that result and implement the next action with it.

It is very accurate. It certainly provides a lot of detail. The quality of the alert is based on the quality of your script and what you are monitoring. This does depend on the ability of the developer along with their strategy and design of the script to capture what they need to be able to monitor performance or availability. Overall, I think it provides a lot of details, if you script it to capture what you need it to.

We have been able to work with the different protocols. Some of them have been challenging to use inside of our network. Sometimes, we face some difficulties getting to a particular site or authentication. 

The scripting features are proprietary. I would like to see more enhancements where the core solution can take on more generally available open languages, like Python. While Apica does accept Java and some Python, I would like to see more of the ability to just execute external scripts for generally available languages out there in industry.

Learning the tool has always been a little difficult from a scripting perspective because the framework is proprietary and unique. Once we became used to what it does and how to perform it, then it became easier for my team and me. I would like to see some of the testing steps be part of a more well-known language, like Java or Python. That would be a big improvement.

We have been using it for two and a half to three years.

It is a very stable platform. It is consistent, e.g., we have not had any outages nor many patches. There are mostly upgrades due to add-on services. Since the last upgrade from version 11 to 12, it has been very stable. They also continue to upgrade into the next version patch for 12. The upgrade process seems to have really improved.

A lot of different groups across departments use it.

We are starting to see the benefits of its scalability at this point. Because we're trying to integrate it with other monitoring or alerting platforms, I do see in their own cloud instance that they have integrations with other tools, like AppDynamics. Therefore, they understand fundamentally that their tool is not a single solution that other industry tools covering critical areas, such as populations, make available. They have made an effort to design around the most common industry tools that have been adopted to create an integration. Thus, it's been very scalable for our operations teams.

Their support post-deployment is very good. I have gotten to know some of the staff very well at different levels. There are agents in the US and in Europe. Overall, we have never had an issue as far as time zones and where the teams work. We have a global workforce, and they have a global workforce. 

The initial setup was right in the middle of being straightforward and complex. It wasn't that difficult compared to other things that can be very complex. There wasn't a single server. At the same time, it does require a couple of different servers overall to run the platform, from the web layer to the aspects layer. Other than that, I don't think it was too difficult. They did provide a lot of professional services to guide us. With that partnership, it was fairly easy to do.

Getting up to 150 checks took about a month and a half to two months. We were able to show the dashboard with synthetic checks running early on, which didn't make the biggest impact, but as we grew, our operations matured, and the platform was adopted, the scripts stabilized more. Then, we were able to show value more further along in having the platform adopted across different applications.

The implementation strategy is to engage the L1 command center team as well as the L2 teams who are the application owners. The L2 teams are various support group teams who adopt the pickup platform. We need to show them how to convert their current RFPs onto Apica, show them its value, what type of tool it is, and that we're migrating to this platform.

We centralized our development team to become experts within ZebraTester, showing them the output of the waterfall diagrams and continually ramp up into developing a lot more checks. We grew and matured the number of scripts, then we grew and matured operations and L2 to be able to interpret their check results, showing them the value of having it all in one place. That also shows the accuracy of the checks as they are happening.

We received a very high level of support from their team. They want to always make sure that you are successful, providing you the guidance and partnership that you need. They're always offering additional help or suggestions to get you more acclimated and ramp you up quickly. I am very happy with the amount of effort that they put into ensuring you are successful.

Their level of support absolutely helped reduce the time and costs involved in switching. Their assistance to help establish our platform made us spend a lot less time managing the infrastructure and allowed us to focus on the actual synthetics, which help the business. That was really what made it successful. There was work involved to get the platform up and running, but we were able to really focus on the synthetics overall to bring the value in versus dealing with constant infrastructure issues.

They have worked with enough different companies to know the most common problems that other organizations have had with their product. Therefore, they were able to predict where we would have trouble with them. 

Also, their professional services teams are very experienced. They were able to bring us to production level very quickly.

If the monitoring is successful, then that will have a ROI value.

The level of alerting accuracy has saved us time and money in operational costs. Overall, it has automated a lot of manual efforts which have been more complex with some of our other scripting tools or monitors. So, it brings things together by doing things faster and saves us money.

We evaluated different products and chose the Apica platform over others. We looked at One UI, some on-prem solutions, and Dynatrace. Right now, we think Apica has been one of the products that has fulfilled most of our needs and requirements. Other platforms can perform some functions better than others. However, Apica's pricing and product abilities are just a better fit for us.

There are other tools which can perform similar tasks. At the web UI layer, other tools can do a good job. What Apica brings is they do provide a lot of good scripting flexibility within their own proprietary tool. Overall, Apica has been able to accomplish something very unique in our environment. We have used with a lot of different testing tools, but they haven't really brought together a consistent way of performing synthetics which Apica has for our operations team. Therefore, Apica did overcome some of our challenges, bringing together our command center and operations.

They offer a lot more on-prem than any other platform that we have adopted. For the external instance, they are very competitive with other tools from an external cloud-based solution standpoint. For the on-premise, we can take on a different strategy if we want to. For example, if we chose to go hybrid, onto the cloud, or just keep it on-prem, that's a choice for us, which is a very good option to have.

We have other tools besides Apica and they all live side-by-side. We will never have just one tool nor invest in just one platform.

Organize your operations and App Dev teams to get onboard quickly. That is probably one of the most successful factors that you'll have. If your teams really buy into the framework and understand your objective, then you'll have better success criteria. This is because you really need to have everyone onboard from the lower lanes to the production level in order to continuously be able to get your synthetics updated with each release so you can reduce false alerts. Then, you can continually have monitoring in your applications.

We certainly have more room to grow. We continuously develop new applications and cycle through changes. Right now, Apica has probably been one of the most popular synthetic monitoring platforms that we use. We do have other testing platforms. Apica is not the only platform that we use. However, for production operations, it is one of our more primary platforms to monitor the health of production applications. While we still have several monitoring tools that we use, Apica has been one of our go-to tools for synthetics.

I would rate this solution as an eight out of 10.

Synthetic: To replace and improve on the current solution and integrate it with the multiple platforms that are already available in the organization.

LoadTest: To replace the current solution with a newer and better platform which would not only expand on the current capabilities but also reduces the infrastructure maintenance.

We have a mix of hybrid models. We have the orchestration platform as SaaS in Apica's cloud. However, the agents that we run checks (or load test from) are located on-prem. 

One of the biggest advantages of moving to Apica is the ability get to a hybrid model with the orchestration engine in the cloud and our agents on-prem. We also have access to Apica's cloud agent across the globe. That has changed the way that we have our load testing and synthetic monitoring setup at this point. Previously, it was always internal. Now, with this change , we can test and run checks from anywhere across the globe. If a certain location is not available , it just takes an email to their customer support, then it is spun up within 24 hours. That flexibility has changed the way that we perceive our Synthetic monitoring and load testing, not just in the US, but globally.

Availability of extensive API options is an enabler integrating both synthetic and load testing platforms with a wide variety of tools and processes. 

The tool is flexible to handle multiple complex scenarios, which is one of the good things that led us to decide on Apica.

Doing the URL monitoring without having to write a script is pretty neat and straightforward. I can spin up a check within a few minutes using the URL monitoring functionality. That is one of the easiest features that gets my team onboarded to use the solution within a couple of days. It is the easiest feature to use, very helpful for doing a quick setup, and delivering to other partner teams in a very short time.

The alerting feature from Apica for email alerts and integrations with other alerting platforms has been pretty helpful.

The flexibility of Apica is big in terms of the range of protocols it can monitor and scale for multiple scripts . That has been one of the bigger influences when making the switch. Also, it is not just about the flexibility of the tool, but the flexibility of the team who we work with. The product teams with whom we had interactions have been very flexible. We were able to request Apica to implement new features into their tool set, and they were more than willing to accept and implement them in their production tools. That is flexibility in a vendor relationship I have never seen when I have worked with other vendors. The product team was flexible enough to listen, accept, and implement the features that we wanted.Another key deciding factor for us when going with Apica.

The accuracy of alerts can be improved a little bit. Right now, it's good in terms of alerting immediately about failures or changes in response times. However, what we have seen happen is the number of alerts that we are getting is very frequent, and we would like to tone down the number of alerts. An option to aggregate alerts for a check from multiple locations is not available and creates duplicates. As a platform, it does send us good alerts, but it could be improved a bit.

For the Synthetic platform, their dashboard could be a little better. I don't have many options to configure my dashboard to make it suitable for a power user versus a leadership person. The dashboard for our CTO is the same as the one that our technical staff is using. We have made this request to Apica, and they're going to make some changes to it. However, at this point, the dashboards and the way they present the checks could be better. For the level of metrics that it provides for each check, they could provide it at a step level or page level on the landing page rather than having us click a few more times to get that data. That would save a lot of time. These two would be really good changes to help increase efficiency from the current tool and current features that it provides for synthetic monitoring. 

As far as I am aware, the Apica platform doesn't allow us to execute scripts from JMeter or LoadRunner out of the box, but they do offer conversion utilities.

For the LoadTest platform, they could increase the efficiency in terms of the results produced. The reporting structure of the results could be improved a bit. Apart from that, it is a pretty good platform.

We have been using Apica Synthetic and LoadTest for about a year now.

It has been very stable and availability has been at 99 percent, if not more, so far. In the last year, there was only one issue with availability, which was resolved within a matter of minutes. We were informed right away from support team that there is an issue with availability. Therefore, it has been pretty good and reliable.

The scalability of the orchestration platform is pretty good. Because it's hosted on the cloud, we haven't seen any slowness in the scalability of the platform. It could be a little better for the LoadTest platform though. 

• For the Synthetic platform, the scalability is pretty good as we increase the number of checks. The application has been able to scale without issues. 
• The LoadTest platform could be a little better. We have seen improvement over the last couple of months and the vendor is ready to provide more scaled solutions for the LoadTest platform going forward.

There are a total of 300 to 350 users on the Synthetic platform. For the LoadTest platform, there are about 100 users in total across the globe.User roles vary in ranges from technicians (the people who write scripts and run load tests), all the way up to managers and directors. This is across the board for both platforms, Synthetic and LoadTest.

We have about 4,000 checks on monitors running at any one point in time. 

The thing that makes the product so easy to use is the support. They made our experience a lot better than what we had have to deal with working with other vendors. Their support is unbelievable. 

We wanted to replace our current existing solution for synthetic monitoring with a newer, advanced solution that integrates with multiple platforms.

We are replacing Micro Focus Business Service Management (BSM) for synthetic monitoring and Micro Focus Performance Center for load testing.

Our initially setup from Apica's side was pretty straightforward. Since this was a hybrid model, we just had to procure some servers internally to setup as on-prem agents, and that was it. Compared to other tools that we work with, this was at least 60 percent easier to set up.

Apica anticipated our needs during our implementation process. They had a pretty good survey of our requirements and use cases before the implementation. They came up with quite a few recommendations that helped us to design the solution better for our current setup. 

It took about three to four months for the basic setup to be completed because of the hybrid model. 

Our idea was to set up on-prem agents as quickly as we could to facilitate the communication from Apica SaaS to the on-prem agent. We also wanted to have both the incumbent team and the new team comfortable with the new setup and decommission the old tool. That was our implementation and migration strategy.

I would give them a five out of five on the level of support. From this particular team that I worked with, they were amazing. The willingness to change their tool, or even remove features, based on our feedback is unheard of. I have never seen a vendor do that. I have the highest regards for the Apica support team.

Apica was able to come in, bringing in technical resources, and help change/convert the old scripts. This helped us to speed up the process of the migration. Without their help, I think the migration would have taken a lot more time.

Based on our prediction model, we should be at least saving 10 percent of our investment with Apica versus using our incumbent tool.

Apica is pretty cost-effective if you buy both solutions together: Synthetic and LoadTest. If you are going for one solution, the cost is on par with any tool in the market. 

In terms of licensing, the major advantage with Apica is you don't have to pay for their cloud hybrid, which is included in the license. For example, our company is heavily dependent on using cloud agents for monitoring after load testing.

We also looked at Catchpoint. We looked at them in a very elaborate way, doing a pilot and PoC, but couldn't align them properly with our needs.

With Catchpoint, the con was the flexibility in terms of working through enhancements and changes. It was not flexible enough to accommodate the changes. The scripting engine for Apica is very flexible, which was another major con from the other tools that we evaluated. In Apica, I can write my own custom code, which I can't do in the other tools. This was a big pro for Apica when we were making our decision.

The major factor that influenced our decision to go with Apica is that it is the same platform for both load testing and synthetic monitoring in terms of scripts. I could write a script for my load testing and plug the same ones in for my synthetic monitoring. So, it helps us in terms of our DevOps model, where I don't have to rely on two different tools to write different scripts, which is the current model right now. Instead, it increases the synergy between multiple teams to use the same platform and reduces the overhead of script creation from testing to synthetic monitoring, which is a big plus. I'm hoping in terms of efficiencies over time to reduce the number of resources that we rely on for writing script.

For Synthetic, we still need to see the alerting in real-time to see how it improves.

There is around a 15 percent adoption rate in the company for tool use. However, for our target audience, the total is around 80 percent. Our idea is to get to 100 percent adoption for our intended group of users. That is the goal for the middle of next year.

Apica is a pretty flexible tool. The amount of features that they have to offer is very high compared to other tools on the market because of the flexibility that they have to offer. The product team behind it is very committed to making those changes and the customer happy. I would advise if you are looking at a solution which is easy to maintain and can handle a lot of checks at the same time, then Apica would be the best solution to go with. If you have to do a complete on-prem solution, that is available with Apica as well. It is a pretty good solution in the current market. 

At this point, I would rate it a nine out of 10 because there are still some enhancements that need to happen for the platforms: Synthetic and LoadTest.

We're a banking software company, so we use it for Synthetic logins to test how one of our end users would log into our product for a customer, how long it would take, what loads, and then log them out. Then we test how long it takes to do that entire process.

On the Synthetic side, we only use it SaaS-based. We actually put it through an SSO. We use Okta for an SSO. That's how we're securing our connections there. Security-wise, Apica's got a couple of things in the works that are going to help them out, but they're not there now. In particular, they're coming out with a key chain that allows us to save. You can hash passwords and users, they don't have that right now. Passwords and logins are set in plain text.

Before we had this particular product we were using SolarWinds to do something like this. The problem with that, however, was that was an internal check, which means it was coming from our network. With Apica, we could do the same type of check, and then we could also make a scenario script that would go and click the things we wanted to click, but it would come from an external source that we did not control. That would give us a better baseline for what the customer sees, as opposed to what we expect to see from our network.

We like the scripting features and the scenarios. It allows us to set up exactly how a customer would log in, what they would type in, where they would click on the screen, and then takes screenshots of it so that we can actually see it happen and see what they see at that time.

We also use it for up-down checks for a lot of our websites that we make ourselves for our customers to make sure the sites are up or down. It's not part of the Synthetic side of it, but we also use the ZebraTester. We're actually implementing various homemade tools on our site as well by API.

We use ZebraTester for some of the sites and other things before they even become into the Synthetic side.

It is highly flexible when it comes to websites. There are a few things that it does fall down on, but for the most part for logging into a website to check to make sure elements are loaded on the screen, it's highly flexible. If I don't want a certain element to load, I can block it or I can ask it to ignore it. If I need to check for a certain element, I can do that as well.

As always, within the IT industry, everybody's always looking to upgrade and update everything else like that. Apica has been one of those things but it's really hard to replace because it offers us the unique capability to see what the customer is seeing. A lot of other ones can do Selenium script and things like that, but there's a lot in Apica that we use right now. We utilize a lot of the scenario options in Apica right now, and there's a lot of other ones that do parts of it, but it doesn't do everything that Apica does.

Apica is indispensable in a few things that we do. It currently is the only one that we have that catches CDN outages. We have many tools that monitor our customer sites, but a lot of those are API logins. If we had a CDN outage and the site didn't load all of its elements, we wouldn't be able to tell that. Apica can tell that because it's looking for particular elements on the screen. Indispensable may be a strong word, but we do highly rely on it for some things.

We use Selenium scripts and we were able to do more specific checks, so it makes it feel like we're actually a customer logging into one of our sites, checking their accounts, and logging out.

The scripting feature has kind of saved money and resources. When we first got it set up, it was a pain because we didn't have the script set up before, but now we have it setup and it's running on multiple checks. Multiple checks, meaning, our Synthetic login checks range around five to 550 checks. Now when we have scripts set up to make the Selenium check, I can pump out new Selenium scripts for one of our online banking customers in five minutes.

Alerts are always accurate, but they might not always be useful. Apica alerts on two different things: one, when an element that is in the script cannot load, and two, when a part of what's loaded comes up with a certain internet error code, a 500 or 402 or something like that. It's always accurate because those things are always not doing that, or they're getting the errors, but it may not actually be as useful. To deal with that, we generally either have to block the URL that's throwing the error code or whatnot, or we have to verify the elements.

It's very accurate but sometimes not useful. It's also noisy. When Apica alerts, it does not have a pull-in time or anything else like that, unless for elements or error codes. It does for SLA times and variances, but not for the other ones. It could be that it's a one-time blip and something didn't load on the screen, it alerts immediately right then. If it loads the next time, it's not going to alert. If it's still set up, it alerts. It can be noisy.

This level of alerting accuracy has saved us time and money in operational costs. With CDN issues, it lets us know, for instance, that we have a homemade monitoring system for our products as well that monitors to make sure that things that should be there are there, but it doesn't actually take into account if the webpage itself is loading. A number of times we've had major CDN outages where our homemade monitoring tool is fine because everything is loaded by an API, but the webpages are not. When that happens, Apica tends to go alert hard and that lets us know that "Hey, we need to go check over here as opposed to over here." That saves us time and money on troubleshooting.

We have two different approximations in terms of how much it's saving us. The way that we do our major incidents, is that we do it per customer. If we have five customers down for five minutes, that's 25 minutes of downtime. I don't have an exact number. I know that things like that when it affects our entire environment are pretty substantial.

It has also saved costs involved in managing monitoring. It has at least saved us in the cost of that it gives us one pane of glass to go to for Synthetic monitoring. I can actually send one of our analysts to go look and if they want to know if a page loaded, they don't actually have to go log in, they just have to log into Apica and check to make sure Apica's running well. That saves time, which saves them money.

Alerting needs improvement. It's a little noisy. It needs some better options. Currently, they have an issue, when you set up a Synthetic monitor, you can set up where it's monitoring from a data center that Apica owns. However, for each data center that you attach to a monitor, that's considered an extra license. That's a bit iffy. They're usually behind on the version of Chrome that they're using for the Synthetic monitors. Currently, they're using Chrome 85, they're 11 iterations out of date. They're trying to get that fixed up with something called Evergreen, which will basically be a Chrome browser that'll stay constantly up to date, but it hasn't been implemented yet. 

The problem with that is that we generally test our product with the newest versions of Chrome and everything like that, so sometimes we've run into issues. Also, when they updated to Chrome 74, we lost some monitoring capabilities that we had before that did not transfer over with this new version of Chrome.

I'd like easier access to the API. Their API, it's not bad, it's just bulky. It's a little unwieldy in the way it has to be used. One of our app developers is currently working with them and he wanted to do a number of calls to the API, and he was not able to do that. They had to make special changes to our API to make the number of calls he wanted to make. It didn't seem to be scaling as well as we thought it would. But they worked with us to actually get it to do that. That's a plus point.

I'd like to see more abilities to do mass changes to checks in the GUI, in the interface. Things like setting a mass amount of blocks for checking a bunch of checks and saying, "Make sure that this URL is blocked on all these checks." Currently, we can only do that through the API, and last time we had to do that, we actually had to use Apica support to do it. 

Finally, they have an audit log system called Journal. However, it can only check, if I remember right, two weeks at a time. That becomes really difficult when you need to check on something that you need to go back multiple times and you don't know the exact dates of the thing that changed. For example, I had a user who got changed in one of my checks and I needed to find out when it got changed. It ended up being three months ago, but I had to go back in two-week increments until I could find it. Their Journal, their auditing system, needs a little bit of work.

We've had Apica for five years. 

We are using the SaaS portal.

The stability is not bad. We've never actually had an issue with Apica, the product. The alerting really comes back down to how this thing alerts, how the alerts are sometimes not useful. That's the worst I could say about the stability. We can turn them off or we could filter them through a third-party.

From what it does, it scales pretty well. I can easily go in, pull on a check, assign a scenario to it and boom, I'm done. I've done that many times. A couple of weeks ago I spent the day creating very specific Selenium checks for very specific parts of a customer's website to make sure that they load properly. It handles it quite well.

We have roughly 100 users using this solution.

We take Apica data and we send it to our data warehouse so that we can do SLAs for our customers. There's me who sets up all the monitoring in there. And then we have our NOC, who will go in there and they use it to actually make sure sites are up and everything. It's used throughout the company in all ways: business side, maintenance, and monitoring.

I do the maintenance of Apica. 

I have 683 checks. For the Synthetic login, the checks, it's 400 checks. Those are the ones that we mimic the login like a customer would log in. For the VT checks, which are basically just up and down checks, we have 112 of those. That's not just our customer sites, we also use this product for our site as well for corporate sites.

We do have plans to increase usage. When a new customer gets added in, they get a check as well. Every customer gets a check.

Support has never been a problem. Their support is top-notch. We either email or get our client experience manager on the line. They have been top-notch, willing to help, willing to go the distance. I have very minimal complaints about support. The one complaint I did have, they actually addressed it very quickly.

We started with SolarWinds, and after that, we moved to Apica. We then got rid of our SolarWinds integration and went to LogicMonitor. LogicMonitor has its own website monitoring tools. However, the problem with LogicMonitor's website monitoring tools is that it's very hard to set up a script the way that Apica does. They also don't provide screenshots of what happens. We've looked at a number of other vendors as well. The problem always comes down to it doesn't do the things that Apica does.

The initial setup was very complex. I wasn't even part of the initial setup, but I know it was very complex because we needed an external source for our checks, but we needed to be able to mimic logging in like the customer did. This was back in December of 2014. I have a feeling nowadays though, they probably have this down to a fine science of how to get people implemented and their stuff up and running.

We have seen ROI. We use it for all of our customers and it does help us. A lot of times it can catch things that happened to the site, but don't happen to the API. We've seen a good return on that.

Pricing is based upon not so much users, but the number of checks you're going to create. Make sure when you set up an account with this, to request more licenses for checks, for any type of check, than you actually need. This will save in the long run. They're really good about setting this up and getting you more licenses but there's always a cost with that.

If you think you're going to need 100 checks, make sure you get 110 licenses. Then remember if you want to do multiple-site checks, not just one-site checks, you're going to get a license for each site.

With all companies, you get the base product, but the base product's not all that you want. You want it with a whole bunch of other stuff with it. We can safely assume that there are probably other costs to add things. Things like additional integrations with other products are not included in the standard license. 

The biggest lesson I have learned from this solution is the sheer number of sites that can load when you load one website. We do online banking, but when you load online banking, it also loads 50 other URLs as it loads through there. That might include Google, Facebook plugins, or things like that. It has really opened my eyes to how many things load when you just open up a single webpage, even if there's that much on the webpage itself. It's very comprehensive when it comes to website monitoring.

I would rate Apica Synthetic a seven out of ten. We've had our problems with it and we're still waiting on some add-ons and features, but for the most part, it's never wrong. It's just sometimes noisy and feels old. The UI is very basic. It's not bad, it's not ugly, but it's basic. It uses old browsers. 

We have various tools, applications, and websites and the use case for Apica is emulating user actions on those sites and in the tools. We use it for proactive action. Before the user starts getting errors, Apica will alert us because we have it monitoring the same actions we expect the users will be taking. Once Apica detects an error it will notify us so we can take necessary action, before it becomes widespread and users start to report it. Apica is doing an important job in monitoring because our company offers services through those sites and the application.

We're using it on-premises and we're using their agents on their cloud.

Because we're using this product to monitor, for example, the sign-up flow, meaning Apica is doing the same actions as a user who would like to sign up, we have been able to figure out when there's a failure in that flow, before it starts to impact users and prevent them from signing up through the services. This is something that has a very high revenue-impact on our company. Apica has helped us find issues with the sign-up before users have started to call customer care regarding the services.

The fact that the solution offers multiple deployment options — on-premise, hybrid, managed cloud solution, bring-your-own-cloud — helps our organization meet our security requirements. Some of the tools in our company can only be accessed internally. To have an internal, on-premise agent makes it easier to test these corporate tools. We have these tools also monitored with the on-premise installation. It helps us to monitor both customer tools, services, and applications, and the corporate tools and applications.

We use the solution’s ability to make use of our own scripts in Selenium and Postman. We're using Selenium to write scripts that can be run for the browser checks, and we're using Postman to run the ZebraTester scripts. Using the scripting feature saves us on resources. It is one of the things that makes the product easy to use. We don't need a specific type of engineer or operator to be able to write these scripts. There are many people who can do the scripting.

In terms of the alerting, since we started using Selenium, which is for the browser checks feature, about 95 percent of the alerts have been real issues. This level of alerting accuracy has saved us time. It helps us to identify root cause quicker. We used to spend something like an hour just to find the root cause, but the ability to have sensitive monitoring reduces it by half. We can identify root cause within 15 to 30 minutes.

Apica has saved us management costs as well. I'm not involved on the financial side, so I can't put a number to it, but I know that we resolve priority-one incidents faster.

We mainly use the ZebraTester and the browser checks. These are the most important scripts that we're using on Apica. 

With the ZebraTester, the ability to have and store dynamic variables, when setting up the monitors, means you can extract that value and use it in a subsequent service call. This is something that has made our lives easier. The most complex monitoring processes are for security purposes: You need to have a fresh token for the user when, for example, he tries to log in. That token keeps changing. To be able to get the results of other service calls that are depending on the login, you need to use that token in the subsequent service calls. Being able to extract that token, store it in a variable, and use it in the other service calls is one of the most complex things. This is one of the features that I like the most because it helps us in configuring these services, in a certain flow, without the need to re-record the whole thing. Being able to extract that value from the service calls is something that has made monitoring a lot easier.

For the browser checks, the screenshots that are available help the engineer or the operator who is on the shift figure out what's wrong or what step is failing.

Also, the flexibility of the solution in terms of the range of protocols it can monitor has been great. The product has been working as expected and it has helped us to cover something like 95 percent of the outages or issues that we have had.

There is room for improvement with the GUI. It's not a big deal, but it would be great to fix the way the GUI is loading. Sometimes when we want the manager alerts and manager checks, it takes time to load all the way. With the whole GUI, if the information appeared quicker, loaded faster than it does now, it would be great.

Also, when it comes to the way the internal agent is installed, because you can install an application on a server, I would love to see the application Docker-ized. If you could install internal agents using Docker or using containers, it would be easier for us to manage them and spin up internal agents. Most of the applications we have now tend to be Docker-ized applications. I'd love to see Apica going that way with its internal agents.

I have been using Apica Synthetic for about five years.

It's stable. Sometimes there's an outage, but it's not frequent.

They offer scalability.

We have about 25 users of Apica Synthetic, people who log in to the tool. About 14 out of the 25 users are engineers with the NOC team and the rest are senior management and engineering leads. We're using the dashboards for management to see the SLAs and the availability of the different websites.

At this point it's being used very extensively. We may increase the number of users in the future, as we have some new projects coming out.

Technical support is always there to answer my questions. They are very responsive. Typically, they all have the ability to support the product, whether it's updates, or issues that we have regarding scripting, or setting it up.

They're just 100 percent available. They always help us on any issues that we have.

We were using Keynote before. I was not involved in the reasons for the switch.

The initial setup was pretty straightforward. There aren't too many requirements in setting up Synthetic. The guides they provided are easy to read and easy to follow. 

Apica support was also always available so we could just shoot a question to them and they would answer right away. During my shift, when I reached out to them via email or a form, they would be able to jump up on a Zoom or a Slack call, to help us right away. They were very helpful. We switched to Apica from a different product and they definitely helped us with recording the scripts that we already had, and with introducing newer ones. They helped reduce the time that we would have had to spend going over the admin guides. Initially, we just wanted to record our existing scripts and they offered to record them. Their support took care of converting the scripts from the language that we used in our old tools to Apica. That saved us time. 

They anticipated our needs during the deployment. They had pretty much everything that we needed when we initially set it up. And when we had a feature request or some kind of additional setup, they tried to provide that feature or help us with a workaround to meet our requirements, even if the product at that point did not have those things directly.

Our deployment, overall, took two to three months.

We had a deployment plan. There was a project manager, and I was involved in writing the scripts and trying to figure out how to convert from the old solution to Apica. Afterwards, I just took care of just recording the new scripts, but there was a whole project for changing to Apica. For deployment and maintenance there were two people involved from our team.

I have not tried all the synthetic monitoring tools out there, but I have tried two of them. They had the same ability to assign the dynamic variables, which is the most complicated stuff that we're implementing, the dynamic variables from one service call to another. But I found Apica offers the easiest way to set this up. This is something that made us stick with Apica, because it's easier to set up the scripts, even with the most complex feature. The two other products I tried have that ability, but it was so complex to set it up. That's what makes Apica better than the others.

Every time I face an issue and reach out to the support, they point me to a part of the documentation. So read the administration guide or the documentation, because they have everything that you need in their Knowledge Base. This is something I learned from opening multiple tickets. It's there in the documents. It now saves me time when I read the documentation.

Apica Synthetic is one of the most important monitoring tools that we're using. 

I would rate it at 10 out of 10 because it's accurate. I've dealt with so many tools and applications, but their support is the most responsive support I've seen. The tool itself offers so many integrations with other applications. It's easy to set up, easy to configure. The documentation is great. The most important part is that the tool covered most of the issues we have and was able to help reduce the time that we needed to resolve the issues and the outages that we had.

Our primary use case is for monitoring. We've got a number of auto finance applications and hosted applications that my teams are supporting. Apica offers outside-in visibility of what a user would experience if they were actually logging into the platform. We noticed that we were missing that outside component. We had a lot of internal monitoring in place for making sure that the user experience was good, but when it came to being able to support our users and report back on issues our users might be experiencing, and work to remediate or identify and resolve issues that our users may be experiencing from the open internet connection that they've got into our hosted environment, it was just not sufficient. So Apica is what we're using that for today to actually give us an outside-in view of what the end-user would actually experience from the beginning to the end and from their overall use case experience for our hosted applications. 

We also use it to monitor the internal service platforms that we use to support our infrastructure, support our environments, and support our internal clients. We use it for monitoring port status and service statuses associated with network-based applications like FTP file transfer platforms, MQ platforms, shared services, SOA platforms, and a number of other internal platforms that we utilize the shared services across our application stacks to serve the service of our clients.

It gives us a clear line of sight into when we're actually having an external network event that's impacting our end users. Previously to implementing Apica, we would have to rely on our end users to tell us, "Hey, we can't get into the website." With Apica and regional monitoring that we have set up in our higher profile application stacks, we're able to tell if we've got a regional network issue, a national network issue, or some other network event that's occurring that may be an internal network issue that's being exposed as a manifestation of user login failures across our application stack. We didn't have that line of sight prior to implementing Apica and so it's really helpful there.

The other thing that it helps with, that is an indirect benefit of doing URL based monitoring with these types of frameworks is that we've actually caught a few expired certificates and we've also caught encryption changes that have impacted our users' ability to access the environment that maybe some service provider downstream to us has changed.

Prior to having Apica, we never really had a clear line of sight into either of those things, other than some automation that we had internally that were basically report based, and they weren't driven off of real-time data like Apica provides. When your cert expires, Apica comes back with an alert saying, "Hey, my check has failed. And the reason my check has failed is that I can't establish an SSL connection because the cert is invalid." That's a great benefit to reap from having that framework in place that wasn't anything that we ever thought of during the time that we were implementing it. Those things are really nice to have.

It's too early for me to actually give a definitive answer on whether or not it had addressed Edge because we haven't been able to build out sufficiently complex user scenarios in our synthetic monitoring areas with Apica. But from what we have set up, it will definitely give us more insight when we're dealing with a complex infrastructure-based failure event scenario. It gives us more insight as to where specific failures are occurring, because it's giving us a lot more data back that gives us detail into where the user experience is actually not functioning. From the diagnostic data that the synthetics that we do have set up from that diagnostic data that we get back because we have an incredibly complex application infrastructure and architecture for some of our apps, we are able to quickly narrow down where within that infrastructure we're actually having a problem with that diagnostic data from the synthetic logs that we get back from the alerts. I would say it does, but we don't really have really deep synthetics setup to the point where I can go to regression test my entire application stack for one of my apps. I just can't do that yet, but it's definitely something that we have in our roadmap of to-dos.

The fact that Apica offers multiple deployment options, like on-premise, hybrid, and managed cloud solutions definitely helps my company meet our security requirements. Some of the internal texts that we need to do require us to have on-premise infrastructure components. Having a hybrid option is definitely helpful. I think the other piece to that is the flexibility to be able to go entirely cloud with Apica is incredibly beneficial because then you get access to regions all around the world that you have a line of sight from that can help you with getting visibility into what's happening from a client use case perspective. For example, if I have a lot of clients in Canada because we do have application products in Canada, I could, in theory, have an Apica presence in a Canadian region that will give me a localized view of what the user experience is like in conjunction with other regional views to help me narrow down when I am seeing a problem; if it's a regional issue or if it's something that's more global in nature. That actually is usually beneficial for us.

We use its ability to use our own scripts. What we use Apica for right now has primarily been based on the importing of Selenium Flames groups that we've developed for mimicking our user transactions. We've also been working on utilizing their automation platform by a ZebraTester and we've been learning how to work with that so that we're still in the early stages with it. But we've been seeing a lot of additional potential from that ZebraTester framework as well. LoadRunner is something that we've been talking about but we haven't really explored that at this point in time.

What we've developed in Selenium is that we've been able to easily convert over into the native Apica workflow stuff for the synthetics that we have configured. Once those Selenium scripts are created, you use it once, and then it's in Apica. The results have been fantastic from that standpoint and the simplicity of being able to use something that's common and standard across the industry, in terms of using system Selenium to create your synthetic transaction scenario, makes it really easy and helpful to actually get into the platform and a little bit more of an in-depth way versus having to learn an entirely different scripting language or having to learn something new in order to do those types of things.

It's hard to say whether or not this scripting feature has saved us money or resources. Because of that flexibility, more people have been able to access that component than normally would be able to. From that standpoint, it has increased our adoption rate. It hasn't necessarily improved outside of that, but with an increased adoption rate, because it's easier to implement and easier to use by more people, we're getting more value out of the framework without having to have dedicated script or dedicated people writing automation for it. 

From our standpoint, there are a number of valuable features. The WebHooks are obviously really great. The alert framework is really good and then the reporting and visualizations that you get from the dashboards are good. Those three areas are primarily what my team's focused on in terms of usage from day to day.

The dashboard view tells you the health of the services that we have monitored, and how the health of the entire infrastructure is doing at a glance. My teams have given me a lot of good feedback that I just keep the dashboard up on my workstation during the day, and if we get an alert, I can immediately go and investigate if I'm in the dashboard. I can also sometimes catch an event as it's occurring so save myself a little bit of time and be able to get in and see what's going on more quickly as a result of that. From the alerting standpoint, that in conjunction with the dashboards that you get, really compliment each other because then you can drill down and actually get into what's happening from a transactional standpoint or transactional perspective, and see where within the transactions that we're monitoring, where what steps are failing, get more details on why those steps are failing and work to mitigate and resolve those issues based on that.

It's that visualization component that really ties everything together and the drill-down capabilities that you get starting from the dashboard that really makes Apica very useful from a day to day support standpoint.

There are a lot of capabilities that we're not really taking advantage of that we could. There are a lot of opportunities to grow in terms of how we're using the framework, especially when it comes to doing things that are more complex, like facilitating deeper checks via multi-protocol based scenarios that tie in with ZebraTester automations that get created or more advanced regression-based scenarios that we might want to set up in the synthetic checks. We're using around a third of all the capabilities that we have available to us so we definitely have a lot of room in terms of what Apica offers for growth and for expansion of our use cases.

The alerting is impressive. 98% of every alert I see come out of Apica is a valid alert. The other 2% of the time, we will get an alert or something will not be right which is because we overloaded our Apica infrastructure with something that we were doing. It was a self-inflicted thing. If you actually remove that from the equation, what you're really talking about is that it's nearly a 100% success ratio of events to real events. 

It's been fantastically accurate at identifying events. The sometimes frustrating part of that is convincing other people that what we're actually seeing, coming out of Apica is a real issue and it needs to be addressed because a lot of times people will just not be convinced by the data that they're seeing until well after the fact. As we've been using the platform more and more, there are more teams out there that are understanding that when a team member brings something up from Apica, it's not to be taken lightly.

From my perspective, I would say it has saved us costs involved in monitoring. It's enabled my resources to work more focused. It's enabled them to work more accurately. It's enabled them to work more authoritatively and it enabled them to work more adeptly. From an operational standpoint, I would say that it's at least improved our monitoring efficiency by 5% to 10%.

Having to install an application on your desktop to utilize something like ZebraTester is a little cumbersome. It would be nice to see that become a web-based application. 

Having the documentation a little more accessible, and easier to digest by people who are just learning how to use the framework, especially when it comes to more complex or more edge-based cases would be really helpful to have. That is really it, but I think the other thing that would be really nice to have, and it's not necessarily a big downside is when the browser agents need to be upgraded, it would be nice if that just happened automatically and it was transparent and seamless to us and to our infrastructure.

We started using Apica around two years ago. 

It's very stable. We don't ever see any issues with it.

It scales well.

We have about 200 technical users and they're mainly technical support application development and infrastructure support teams.

We have a couple of dedicated resources for deployment and maintenance. Obviously, they share responsibilities across different application stacks, but we do have resources available. They're a monitoring infrastructure support team.

We have 200 users. We have 100 Synthetic and another 100 regular licenses, and we've got a lot of room to grow so that we plan on increasing usage quite a bit over the next couple of years.

The initial setup was moderate. We have a moderately complex setup here in LA.

It was moderately complex because we've got the hybridized infrastructure for Apica. If we were entirely cloud-based, we would probably be a little more straightforward and simplified. But because we're using both on-premise and the cloud infrastructure, it just makes it a little more complex.

We've got a number of groups that we've got a center of excellence basically. The center of excellence is helping teams by enabling their use of the platform.

We used their support during our migration. The vendor engagement with Apica is second to none. I've worked with some really big companies out there and I have been seriously impressed with our technical account team, with our support teams, and with our account executives for Apica. They are great partners to us and they listen and they help and they try to help, and they're just a fantastic group to work with.

The level of support reduced the time and costs involved with switching. It also is an ongoing, continual improvement type of strategy. It helps us to implement new solutions more quickly because they're accessible and again, always there to help so it's great.

They anticipated our needs during deployment. The adoption rate has been a little bit slower than what they were anticipating, but that's no fault of theirs.

I'd say it took us a good six months just to get everything set up the way that we wanted it.

We have absolutely seen ROI. The percentage of time saved that we've gotten from this platform has been helpful. That translates itself into client success stories as well.

It would be nice to see them have an enterprise license where an enterprise can just buy unlimited.

Professional services are at an additional cost, but it's very fair.

We also evaluated AppDynamics. 

I would highly encourage organizations that have external applications, web-based applications to definitely consider this platform if they're looking for something to give them an end to end view in the overall user experience.

Having that outside-in view, you don't really think about it at the time when you don't have it. But once you actually have that outside-in view, that really gives you that same context that the end-user has. It's kind of surprising how much more you actually learn about things that aren't necessarily within the infrastructure that might impact your clients and potentially impact us so it's been very revealing.

I would rate Apica a seven out of ten. 

There are quite a few things that we use Synthetic for:

• We use it for error checking and geo-protection checking because we are very regulated since we are in gaming. Basically, if it doesn't see X text on a page, the geo-protection is not firing properly for blocked areas. This checks all our maximized databases. 
• We use it for timings in making sure the web page is optimal so we can tell if someone accidentally seeks a large image up through the CMS site. 
• For the load test side, it is pretty much obvious what it does. It is load testing outside through journeys what we have through Synthetic. It checks the API back to the login services and so forth. 

That's a great overview of what we use it for. We use probably around 75 checks on Synthetic across our three verticals that we do per casino. 

Accuracy is probably around 98 percent. Sometimes, there is a false alert on one of their pops, where it is just loading slow from their architecture and that affects the timing involved. However, that is nothing major. The alerting is pretty accurate. It does show you the correct results. While it may have some false alerts, that is few and far between. If we do see a false alert, we just report it back to them and they fix it.

It helps with releases because we monitor them in staging. We can tell if something is critically wrong before it gets into production, e.g., if it was load related or function related and also what was different in the dev stage. It then alerts us straightaway inside of our production monitors once it has been released. Therefore, it has improved how we run our systems since we monitor multiple environments.

All their pops: They have locations all around the world. Having all the locations around the world is very useful, especially when you're in a license market.

Their alerting system: The way their systems alert you is top-notch. 

Its flexibility is pretty high. You have all your points of pops where you can go to Ireland, Sweden, or X location. I don't think they really need to improve on their flexibility. There are so many settings, different optimizations, and scripting options that you can do. 

The way you script Apica is probably the easiest way of working that you've ever seen. For a QA person, it is very easy because they have the understanding of the tools and what they have to offer. From the complexity side, it is very possible to do pretty much everything on Apica: down to logging in and up deposit, doing other processes inside your website, and loading slot machines to make sure external providers are loading correctly. Because in the world of gaming, you don't buy all your own slot machines. You have the likes of Pragmatic, Betsoft, NetEnt, where you have to make sure all their services are up as well.

The initial screen on their dashboard could have a bit more data, but this is a small thing. It could have more data, so we do not need to drill down to a screen behind that initial information. I would like them to get a little better on the user interfaces that we need to go into.

With PokerStars, I was using them for two years before I left. Since working with GraySnowPoker, I have used them for almost three years (coming up in October). In total, I have used them for around five years.

We have never had a major problem, and there has never been a major outage at all. Your monitors are constantly monitoring. Stability-wise, the system is constantly up. It has few errors.  It is a very stable company with a very stable technical staff as well. You get the best of both worlds

Scalability is huge. You can ask them technical queries, then they will adapt their scripts. There is scalability for having as many monitors as you like. Load test capabilities are there as well. As much as I've ever needed to load test our site, they have been able to assist with it. It pretty much has everything.

Three or four developers use it from the technical teams. On the reporting side of it, the management uses it. For errors, the QA team uses it. Everybody uses alerts because we all get mobile phones. There are probably about 10 to 15 people in our small organization who have touch points on Apica. It even sends alerts to my customer support people in their channels, saying, "We have a problem here. Be aware that if a customer rings up that we can say 'We're already working on it.'" There are multiple touch points around the company because of how the system reacts and how the system generates recalls and alerts. It covers a wide amount of groups who can use it. It doesn't have to just be techies.

At the minute, the solution is being 100 percent utilized. Unfortunately, with COVID-19, we were meant to go up to 180 checks this year. When my new budget comes out, we will have to stay at the same level because we own land based casinos that are in the USA. Our budgets are basically being taken down a little. Because we have no revenue coming through on land based casinos, even though its coming back, it's probably only at 50 percent of the revenue now. Expansion-wise, we would love to expand and sell more checks. From a money point of view during COVID-19, it is very hard to justify increasing checks, even though that actually gives you massive discounts. From a strategic point of view, we can't justify doing that at this point in time until the situation gets a bit more stable with America. It's unfortunate, but it's not very stable there at all, as you can read on the news.

They are always adapting and improving the product. If you ask something from them, they will do a custom script for you. If you get stuck on any scraping for a specific check, they will jump in. Their support team will jump in, guide you through it, help you write the script, and so forth. The flexibility of their system and staff are huge. It's really good.

Don't be worried about asking your account manager for assistance because they will always be there to assist you. From the support side, I can ring my account manager up. My account manager will basically go to a technical contact who gets on the line straightaway (within reason). With another company or ESOP servicer providers, if you ring them up, then you have to set up a ticket. That ticket has to go through second line support, which probably goes through third line support and so forth.

Apica's gurus are very good at answering questions quickly. If you need to escalate to your account manager, they will go directly through to your technical leads. Your technical leads will then jump on the Slack group or phone. From a support and ease of use perspective, it's amazing.

The vendor has supported every need that we had, which is good. There are very few vendors who are able to do because they don't all support your needs, where Apica does. They go the extra mile as well. They are not just trying to take your money, as they're there to support you. They make that very clear in their ways of doing stuff and their support. They are pretty clear that these are good guys and they want to support you with whatever it takes. Especially during this COVID-19 time, they have been extra supportive and have really looked after us as a company. Many providers aren't doing that and just want your money every month.

In their tiered system of how they produce their areas, we know which ones are Tier 1 providers, etc. It is very clear from a technical level as to the ability of the company. I don't think they're going anywhere because they have a very good team. I have had the same case manager coming up on five years along with the same technical support. Thomas has been there from day one.

Before I was at GreySnow, I was using NCC Group (now Eggplant Monitoring). We moved away from them because we saw the ability of Apica's product. I did quite a bit of research at the time. The main differences for NCC Group at the time was they didn't have different regional pops nor did they have the coverage that Apica has. 

Looking back at PokerStars, when we migrated from NCC to Apica, their support wrote all the scripts with Apica. They duplicated all the NCC scripts to Apica, which was a massive cost savings.

With my current company, we started off with Apica. 

The initial setup is very straightforward. Ease of setup is amazing because they assign someone to you. From our point of view, it was less needed because I have been using them for quite a long time, but they even offered to help the likes of me if I get stuck. At the end of the day, due to the amount of times I've used them, I pretty much know how their systems work. If I was going to give it a scale, where one was very hard and 10 was very easy, I would say it was around an eight or nine. It's that simple to set up. You have multiple different contact points to go to. If you struggle, you'll be assigned an account manager and a technical contact. Those technical contacts can be invited into your site groups. They will use Slack and other mechanisms as well. It's pretty much spot on for setup. Everything is documented to a very high level, which is very good.

They assign you with a support agent who can help you with scripts, etc. From the point of view of time consumption, it is time consuming to set up multiple scripts, but that is not a negative. It is to make sure you get the content for right journey and user flow.

You could do deployment and maintenance with a minimum of one person if you're talking about just writing the script. If you're talking about deploying one of their internal things, I would say two people. The whole point about monitoring in Apica is cutting down the resources you need. In general, we can get alerts back sometimes quicker than the provider can get alerts back because of the way the system is functioning.

The alerting saves a vast amount of money for us in gaming. Because when your site is down, it alerts you. If it is something that you can fix, or alert another team to fix, then the gaming sites are backup the quicker that it's fixed. There have been occasions where something has broken through a release that we weren't aware of. It shows you that issue. Then, you get a developer to fix that issue and you're back online. If you didn't have alerts then you would have to wait for the customer reports. If you wait for customer reports, then you could be waiting X amount of time because customers are very good about complaining when they are losing money, but they're not so good about complaining if the site is not up. They just go somewhere else and play, then come back later and play on our site. We will lose money in that instance. It saves thousands of pounds if you look at the rewards it has against its setup costs. It saves us more than the license.

You have our QA team implementing all the scripts to track everything, but this far outweighs the rewards because it's so cost effective and saves a lot of money if there are errors on the site. The alerting is definitely there for a good reason: To prewarn us if there are any problems that we can then get fixed quickly.

We use scripts to compliment Apica or write Selenium scripts to prove points to see exactly the job Apica is doing. This has definitely saved us resources. We have one person looking after Apica with inside our team. If we were self-scripting it or doing our own monitoring system, we would need multiple different servers or a dedicated team of Synthetic people to be able to generate exactly what Apica has produced. Staffing-wise, it will save you a couple of head counts. Therefore, you are looking at probably saving about 120,000 pounds a year in head counts for what it's doing.

The pricing and licensing are very reasonable. At the end of the day, you are using their technology/software and getting X amount of checks for a very decent value. As for discounts, they try to meet your budgets as much as they can. For example, if you need 100 checks and you have X amount of budget for it, then they will try and get down to that price. Costing-wise, it is a reasonably cost product. They will always try and come down to your price if you need them to come down to it by knocking off certain areas. 

We haven't actually had to pay any additional costs for anything. We fit into their model quite nicely.

I have used multiple monitoring solutions throughout my career. Apica is one of the best. The Apica system is leaps and bounds above pretty much everything else out on the market, especially from Eggplant. The scripting tools at Apica are a lot easier than NCC Group (Eggplant) and their reporting is incredibly sophisticated in comparison. Also, their alerting system is more complete. It basically gives you alerts and shows you what asset has failed. It shows you timing for failing assets.

In general, you can do login routines with Apica. The deposit routine of a poker table (or many of our systems), we wouldn't be able to do with other monitoring systems. 

Apica is a complete package. You can monitor every single touch point that can fail inside a user's journey. With NCC, it was very hard to script the journey, point out different APIs, and different failures of suppliers. It would just alert on one individual thing, and it wouldn't really show you where the main error was. It would just show you there was an error with that journey. With Apica, it shows you exactly where the error is, even if it's a misloading asset like a Google Analytics tracking script. It will then show you that missing asset so your developers then don't have to worry because they can see that through Google. Obviously, we can't influence if Google or Google Fonts is up and down. It puts your mind at rest, showing exactly where the failure points are for many other systems.

I would recommend researching what is out there to see just how good Apica really is. Apica is top-notch and probably one of the best monitoring companies.

There are multiple different things you can take off of the solution:

• Your code is not correct. 
• Your image optimizations are not correct. 
• Your geo-blocking has a fault, which means you're in breach of your license.
• How your system is working, e.g., the speed, performance, errors, and missing assets.

There is a lot of in-depth content.

They do meet our security requirements because we are not sharing any private data with them from a software-as-a-service point of view. With on-premise, we had one or two licenses that folks tell us that we could install on our platforms, monitor, prime routines, and so forth. However, from where I am now, the security is fine because you are not injecting anything. If we were injecting any usernames into it, they're test users and marked as test users within it because it's a back-end system.

Even if someone got our Apica password, it would be pointless anyway because we're not exposing player data. We have specific users set up for specific tasks that we monitor, and they're marked as test. They don't go on any revenue streams. 

From a simple point of view, their security is top-notch. They offer different security platforms for different use cases. If I was a bank, then I would have it on-premise and it would meet their security profiles as well. So, I am aware of their security and appreciate the efforts they're going to, but we are just fine with software-as-a-service because we're not declaring any personal information.

I would rate them five stars out of five.