What is our primary use case?
Most of the time, we use it for testing the performance of networks for voice and video. It's not designed to be exclusively that, but that's what we use it for most of the time because network performance for the voice and video is notoriously difficult to monitor and manage.
We've built a business around AppNeta because in the early days, it was, and it still is, a unique tool in the way it operates, and we do a lot of consulting work for much bigger companies. Most of our work comes from a company into video and voice solutions, but we have done a lot of work with other ISPs. Essentially, we've built the business around what AppNeta can do.
How has it helped my organization?
It works well for visibility into the internet and cloud environments. AppNeta works from a source probe, and it sends a low bandwidth stream of traffic to a target. If the target is not another probe, it has to use ICMP. One slight difficulty is that a lot of the cloud vendors don't allow ICMP into their cloud infrastructure. The way around that is to install a software probe in the cloud. So, AppNeta has installed software probes in nearly every Microsoft Azure location, and those probes are publicly accessible, which means we can reach them by using a different protocol. If we need to test somewhere that's not Azure, such as Amazon Web Services or Google, then we would need to install a software probe in the cloud. So, it's one more step than we've had to use in the past, but that's because of what cloud providers will allow and will not allow into their data centers.
In terms of active and passive monitoring for alerting us to the deterioration of digital experiences before users are impacted, the delivery side is always active because that's the way it works. The passive side is used to monitor the operational traffic that's on the network. It works fine, but we don't use it very much because there are always security implications. So, we only use the active side of the tool. The passive side looks at the operational traffic that's already on the network, which can become tricky security-wise because when you do that, you are able to see all of the traffic unless it's encrypted. We don't tend to use that. We have used it, and it's useful to work out notifications and alerting. From the active side, we can see if the utilization is high, but we can't tell what's causing the high utilization. If we can get security authorization, we can use the passive side to find out which applications are using all the bandwidth. Typically, we're mostly focused on the active side of the testing, and the alerting and the notifications are very useful.
We use the Automatic User Geo-Location feature. It happens automatically. It's not something that you can turn on or off. We do use it, but I'm always a little concerned about how accurate it might be. That has nothing to do with AppNeta. For example, if I try to find out where my home broadband is, quite often, it'll show me in London, whereas actually, I live 80 miles away from London. It's just where the IP address is logged. We do use it, but we use it cautiously. It's good for the remote workforce. We've done a lot of troubleshooting work for people working from home. Over the last couple of years, so many people have started working from home, and they've had to rely on commercial broadband rather than business broadband. Often, people have no experience in networking or troubleshooting. We're able to get them a software agent that can be installed on their machine, and then once it's on, we just remotely manage it from a central location. So, once a user has installed the probe, which is no more difficult than any other Windows or Mac application, they don't have to do anything else. We take it over from there. When AppNeta first brought the licensing out for working-from-home pros, it included 25 agents, and recently, they pulled that up to 50 agents. It's pretty useful. For the price of one standard license, you get 50 agents that you can either put in for people working from home, or you can put them in a branch office as if it was a hardware probe. We found it to be very flexible for that purpose.
In terms of ease of using the Automatic User Geo-Location feature to determine if a problem is user-specific or region-specific, it depends on how many agents are around. We do know what the ISP is because it automatically tells us the ISP, as well as the geolocation. If we see people in similar locations and they're having problems with the same ISP, we can pin that down to an ISP problem. It's a question of comparing and contrasting what all the users are doing, but the end-to-end thing is, AppNeta already has a system to work out where a problem might be, whether it's at the local end with the user, or at the remote destination end, or somewhere in between. It always has that ability. There's a combination of ways to find out where a problem might be getting caused and who's causing it and why.
It's very good for our Mean Time to Identify (MTTI) for performance issues across business-critical apps, locations, and users. Obviously, when you're dealing with an end-to-end path, the devices and the connections are not all owned by the same people. The local part of the network is owned by the customer or a third party. There may be one or more ISPs, and there might be an ISP at the local end and a different ISP at the remote end because we work globally. So, we may be testing between the US and the Far East, or with the UK and South Africa. There's always at least one telco, and normally, more than one telco. Although we can pinpoint the problem and find out who owns it, it doesn't mean we can fix it fast because it all depends on other third parties accepting what we say and getting on and fixing it for themselves. However, normally, if there's a problem that lasts more than, for example, 10 minutes, we pretty much know exactly where the problem is.
It hasn't had much effect on our mean time to resolve (MTTR) because we're not normally in a position to resolve it. We're all pretty much external consultants. We can explain to people where the problem is, but we don't have direct relationships with telcos, where we can instruct them to go and do this. The MTTR for a problem is a bit unpredictable, but that doesn't have anything to do with AppNeta. That has to do with convincing the person or the organization that owns the problem area to get on and fix it.
We have seen a reduction in open tickets using AppNeta. Before people used AppNeta, they didn't have a reliable method to find out where the problem was. Sometimes, when we work for a particular client, a problem ticket has been open for a month because the problem is periodic, and it is unpredictable when it happens. When it does happen, they don't have the right tool in place to monitor it, and then we come in and install AppNeta, and generally, we can get that ticket closed within a few days of us becoming involved. So, we often find that problem tickets have been open and unresolved for quite some time, and then we come in and put AppNeta in, and typically, we can get that ticket closed within a few days. It's just a matter of how quickly the people involve us in the problem. If we were involved right from the beginning, I'm pretty confident that it would definitely result in tickets getting closed earlier. Sometimes, tickets are getting closed within minutes because we've done our bit and we've said what the problem is and where the problem is. There's not really any point in holding the ticket open any longer because the resolution is up to some other third party like ISP or telco.
What is most valuable?
The main feature that we use is what they call Delivery, which is the testing of network paths end-to-end. They do provide synthetic web transactions, and they also provide the ability to actually look at traffic on the network, but we don't use web transactions very much. That's essentially because the customers we work with aren't interested in that side of the business.
What needs improvement?
They should try and make diagnostics run a bit quicker. When the problem occurs on a network, AppNeta runs automatic diagnostics on the end-to-end path. The path it was testing only to the destination, it now runs the same test to all of the devices and all the intermediate devices. Depending on the number of intermediate devices, it can take several minutes to run. If we're trying to find or diagnose a problem that only lasts two or three minutes, it may be that the diagnostics is still running by the time the problem is cleared. The only thing, which I have also mentioned to AppNeta in the past, is that there should be much faster and much more lightweight diagnostics, which can be completed within 30 seconds or one minute, rather than in 5 to 10 minutes.
Currently, when we have short-duration problems, we use a different tool, but we only use that different tool for short-duration problems. With AppNeta, as long as the problem exists for more than a few minutes, such as within 10 to 15 minutes, we can normally tell where the problem is. However, most of the problems that we deal with are intermittent. They're very rarely a permanent condition that needs to be addressed. That makes it more difficult to troubleshoot. We would look to see at least two or three events and hope they show the same results to raise our confidence that we've actually found the problem, rather than just a problem.
For how long have I used the solution?
Broadcom acquired AppNeta earlier this year, but I've been using AppNeta for about 12 years.
What do I think about the stability of the solution?
What do I think about the scalability of the solution?
It's scalable. In terms of its usage, we use AppNeta exclusively for end-to-end network and app visibility across managed and unmanaged networks. We have looked at other products in the past, but AppNeta is still the one we typically use as our first choice. We do use some other products specifically for troubleshooting rather than long-term continuous monitoring, but 99% of what we do uses AppNeta only. For only 1%, we use a couple of other products.
How are customer service and support?
It has always been excellent. It's always pretty much a 10 out of 10.
How would you rate customer service and support?
Which solution did I use previously and why did I switch?
We didn't use any other tool previously. We worked in much more general-purpose networking management, and when we were introduced to AppNeta a long time ago, we saw that we could quickly build a business specifically around it. We didn't suddenly change to AppNeta. We just created a brand-new business using AppNeta.
There are other products that do synthetic web transactions for the delivery side of what they call Delivery, which is the network performance, but AppNeta uses a pretty unique technology. AppNeta has been going for nearly 20 years. They used to be called Apparent Networks, and they changed their name around 10 or 12 years ago. They've been using pretty much the same technology, which I don't believe anybody else has. That's why we've always used AppNeta.
How was the initial setup?
It's a cloud-based portal, but the probes that we install at the customer sites can be software or hardware. The software has been very valuable over the last couple of years because, with COVID, so many customers closed some sites. So, we couldn't ship a hardware probe to a particular site because there was nobody there to receive it or install it. With a software probe, we just downloaded from the portal and installed it, and we were up and running within minutes. It's a software-as-a-service, but we can use dedicated hardware or software probes that are installed with the user.
Typically, we don't go on-site and install anything, but we do pre-configure any equipment so that the customer can just plug it in, and if the firewall requirements have been met, then it'll just work. It's simple to do that. AppNeta does offer a private on-prem version, but we never used it.
The process for configuring it is straightforward. We just ask the customer for various addressing information. It's a very simple device. It only needs about six pieces of information that the customer gives us. We configure it and ship it. If it's a software probe, we don't even need anything like that because the customers just download and install it themselves. There's no preconfiguration that we need to do. When the probe comes online on the portal, then we just configure the destination targets that we want to run the path test on, and we just press go. That's it.
We take about an hour to configure the probe, but actually, it only takes us a few minutes to do the configuration. We like to leave it running for a little while just to make sure everything is okay with it from the software side of things and the hardware. If the customer installs the hardware, or they install the software, we can be up and running and measuring the performance within about half an hour.
What about the implementation team?
For the configuration, there are only two of us. Either of us can do it. It just depends on who has got the hardware probe. A lot of the work we do is fairly long-term, so we might do a network assessment for the pre-deployment of video and voice, which might take two or three months if it's a very large network. Quite often, what happens is that once the customer sees what can be done with the probes for an assessment or a troubleshooting engagement, typically, they'll keep the probes on anyway. We only do the configuration once per probe, and that configuration may well last several years.
In terms of maintenance, the software upgrades are handled by AppNeta themselves. It does require maintenance, but AppNeta manages that centrally, and it's pretty much transparent.
What was our ROI?
We have seen an ROI. Most of our work is involved with voice and video. If there are voice and video problems, it makes a call practically useless. People have spent a fortune on installing voice and video room products, and even individual products now. The return on the investment for the application that we get working is much more valuable than the return on the investment for investing in the actual tool itself. We're not exclusively voice and video. If you have a web app that is running very slowly, we can use AppNeta to work out if the problem is at the user end, if it's the network, or if it's the actual application server itself. All of that is invaluable.
Every customer spends a massive amount of money on all the applications that they run. If the applications don't run correctly, they're not very productive, and all of that investment in those applications is wasted. If you have one application that the entire company uses, with thousands of people using it, the return on the investment to get that running properly again is almost incalculable.
What's my experience with pricing, setup cost, and licensing?
That's a little difficult because the licensing costs are different depending on the type of probe that you have. We typically don't get involved in the commercial side, but the list price is probably something like $3,000 for a small probe. However, that gives all of the features that the probe can do, whether or not you use them. In the old days, up until two or three years ago, each of the separate features was a separately licensable module so that you could add things that you wanted, and you didn't have to add things that you didn't want. They've changed all that now, and everything the probe can do is a part of the base license. It's a bit tricky to specify the pricing because if you do use all of the features, which we don't, then it's really good value for money, but as there's nothing else like it, it's still an essential purchase. We don't seem to have any problem with customers seeing the value in paying that.
The small probe is probably around $3,000 and the very large probe that they make for massive data centers might be $50,000 or $60,000. It's a subscription model, so the payment is per year.
One of the other products that we looked at was ThousandEyes, and we specifically didn't go for that. It has some similarities to AppNeta, but we specifically did not go for that because it had a very unpredictable licensing model. In that, you had to decide how many times you wanted to run a test. If you decide you are going to run a test every five minutes, and that becomes not good enough and you then want to change the testing to every one minute, effectively, you use up five times the number of licenses. It's very unpredictable what the cost of the ThousandEyes product is going to be. With the type of work we do, testing once every 5 minutes or every 10 minutes is nowhere near adequate.
What other advice do I have?
To those evaluating this solution, I'd recommend that they get some help from someone like ourselves because AppNeta does take a certain amount of interpretation of what the results are telling you. That might not be immediately obvious if somebody has never used it before, and they try to start working on it without any training. Since Broadcom acquired AppNeta, the training documentation has improved quite a lot. I would definitely recommend looking at the Broadcom training website for AppNeta because it can get you up to speed very much quicker than what used to be the case a few years ago.
I'd rate it a 10 out of 10. We've used it all the time. We were lucky in that we've always had really close access to the people in AppNeta, and we've made suggestions about product improvements and things like that. In general, they've always done it within a few months or a year. Now that they're Broadcom, I don't know if we'll have that close relationship, but as far as I can see, nothing has changed with the way we operate together. It has always been a really good, healthy, and cooperative relationship between ourselves and the AppNeta people.
Disclosure: My company has a business relationship with this vendor other than being a customer: MSP