OpenText LoadRunner Cloud Reviews

When we test retail applications, which are hosted across the United Kingdom, we try to save the load generators on a cloud and then do load testing. This is to do with holiday readiness and to certify a few deployments to production in order to make sure that performance is not degraded. We use Micro Focus LoadRunner Cloud both for integrating with the CI/CD pipeline and for standalone instances.

It's a fast product, so you don't have much trouble in terms of maintenance overhead. You don't want to just look into configuring load generators, look for upgrades, and end up having that take up a lot of your time. With this solution, you just log in and you start using it. This means that there is a huge benefit in terms of the overhead of maintaining the infrastructure and the maintenance effort.

Some retailers have a global presence now, and you can spin the load generator from whatever location you need. You can easily run distributed load testing, rather than procuring the load test across the globe and then getting tunneling, etc. That's one of the great features.

LoadRunner Cloud's reporting features are valuable as well. We can get a quick PDF without having to analyze the local data and then imaging it in the report.

Sometimes, on private or a standalone instances we see users getting dropped off. You may have started to run your test with 2000 users, and you may see that a few users got dropped off. Finding the reason is always hectic. I haven't seen these issues on LoadRunner Cloud.

If you get a raw file on a standalone instance, you are on your own to splice and dice the results. I want to see errors versus response time, and I want to see how throughput was performing when there was a spike in error or response time at a certain period of time. These type of options are not available on LoadRunner Cloud, and they would make the user's life easier and would help him drill down to the exact time.

In the next release, it would be nice to have more coverage in terms of load generators. Then, you would be able to drill down on the raw research and analyze more in terms of response time spikes or errors.

CI/CD integration could be a little bit better. When there's a test and if you see that there are high response times in the test itself, it would be great to be able to send an alert. It would give a heads-up to the architect community or ops community.

I've used this solution for about a year.

I haven't seen any issues with stability during six to eight hours of continuous exhibition. There were no packet drops due to connectivity issues or any problems with regard to line monitoring.

I didn't have any problems adding users in tests with a maximum of around 3000 users. Once in a while, we have issues with refreshing.

We used Performance Center for ten years. We compared other solutions and decided to switch from standalone to SaaS-based. So, we went with LoadRunner Cloud. We felt that it was a bit expensive, but feature-wise, it was pretty much on par with products from competitors. Also, we didn't see any issues with scalability when we ran a higher competency task.

Other tools are available in the industry, but they're not user-friendly, or they have limited features. Some have issues with scalability.

The initial setup was definitely difficult in the earlier version. My team was sometimes not comfortable with the earlier version because there were a lot of issues with regard to the spinning of loads and data. There were limited options in terms of reporting. Also, there were very few graphs where you can monitor online compared to those in the current one. The current version is more mature and better.

Because we don't own anything, we created our account and then got started with the license.

When we compare the price of LoadRunner Cloud with that of products by other SaaS providers, it is a little bit on the higher side.

I would rate LoadRunner Cloud at nine out of ten because it is user-friendly. You don't have to have much coding experience for configurations.

It is all about load testing. We have to do a lot of load testing in our IT environment. In the investment banking space, you do a lot of end-to-end endurance testing, to see how to benchmark performance based on resources and the CPU memory. So, we will have to benchmark the performance based on trade volumes. Trade volumes are the basic KPI.

Endurance testing needs to be done to see how many trades can be pumped in my environment. I do endurance testing. 

better decision making was done Business KPI & technical KPI can be set by automating loy of endurance testing where given resources thresholds capacity can be measured against  business kpis

The record and playback feature is the most valuable feature. It's all driven by the script, so it's a script-based tool where the background tracing starts. Java's background process does a lot of tracing. The process starts in the background. It sees what peaks of volumes the process can handle. It's easy to use because it's script based, record and playback. I play certain tests. I record those tests, see how they pass through the entire environment, and see how many breaks I get. 

Then we repeat that for multiple numbers. If the first time, I pumped in 1 million trades, and get 5% of the failures, then I try with 2 million trades and I see that there is a 7% failure. These scenarios are easy to do in the LoadRunner. LoadRunner has been the best tool for more than 20 years in the market.

You can do endurance testing, you can do load testing. These are the different types of load testing. It lets you see how much volume you can benchmark in your entire environment, given the resources like CPU and RAM. 

The record and playback is the best feature. You need to use a console. You log in to your console and see the dashboard with how many trades failed and how many trades passed, what the peak loads endurance load were. 

The performance has really improved in terms of running test cycles. The product used to crash on-premises and when it had a lot of trades being pumped in. Because it is memory intensive, it used to crash if it was running out of memory. That was the limitation of the on-premises thing. Running those states and cycles in the cloud is much faster. 

Everything is frozen in the cloud. The RAM, CPU, compute, and storage are provisioned in the cloud, which is becoming easier for running these test cycles. Test cycles are highly effective. Of course, you need to have a test strategy, like volume-based load testing. Configure some test cases and run those test cases in cycles. Cloud performance is much faster. Volume-based endurance testing is easier in the cloud.

It is an old solution, around 20 years old. It was called once Mercury LoadRunner and then it was acquired by HP, it became HP QTP. That was around 2007, it was known as Hewlett-Packard HP QTP. Since then it has been acquired by OpenText.

It is very stable.

We have around 100 users, they are functional experts who use this for load testing.

The people who have been responsible for setting the KPIs will decide if we will expand the solution. They are the end-users. It will depend on the dashboard and the results of the load testing.

LoadRunner is a very old product in this industry. It was a little difficult to get to the right support team. Now there's a professional service group that is better than they were before. 

If scaling becomes a problem, then we have to involve the support, the right kind of support to actually fine-tune and troubleshoot the entire configuration. But with the cloud version, that risk is minimized. It has adequate storage available.

It's a subscription-based, pay-per-user license. I wouldn't say it's expensive. 

Because it was an on-premise license, it was a costly affair. Only certain stakeholders could use it. Now, it is enterprise-wide, we can have any number of cloud users performing the load testing because it is pay-per-user.

I would recommend LoadRunner. It is an industry-leading tool. You can benchmark the volumes on the dashboard and it is easy to use.

I would definitely recommend it for investment banks and certain businesses like OTC derivatives.

I would rate LoadRunner an eight out of ten. It has good documentation. The recording and playback scripts can be easily done. I would like for there to be better integration with other tools so that when you do load testing you can also do a security check.

We used Load Runner Cloud particularly to do spike testing. 

The license is built such that it can be used across the entire year. The flexibility to use it across multiple applications for the year for various concurrencies and time periods works well for us.

The TCO has been optimized along with the total ROI. The product, being a part of the Gartner magic quadrant, helps score against some of the open-source products out there. This helps as there are certain regulators who look at technology adoption that is a part of the Gartner magic quadrant.

We used Load Runner Cloud particularly to do spike testing and it was great. 

The license can be used across the entire year. They make it flexible so that we can use it across multiple applications throughout the year for various concurrencies and time periods.

The TCO has been optimized along with the total ROI. 

The product being is part of the Gartner magic quadrant and scores well against open-source products. This helps us with certain regulators who look at technology adoption.

There are two features that I would want MicroFocus to work on.

1. It should have a feature to report with a 99.9 percentile success rate.

2. We should be creating a performance dashboard with InfluxDB OR ElasticSearch integration with Micro Focus Cloud LR. We need Micro Focus Cloud LR to send feeds to InfluxDB OR ElasticSearch for each run for:

1) Realtime PT result publishing
2) Trending between runs
3) Data mashing with Grafana/Chronograf
4) Live alerting

The feed from the Micro Focus Cloud LR instance to InfluxDB should be configured, or an integration touch point should be available for sending real-time feeds into the DBs.

I have been using and supporting Micro Focus LoadRunner Cloud for more than two years now. 

Features like spike testing and load per use are excellent features. Users who are looking at very high concurrency requirements yet need it only for a short interval can use the LR Cloud efficiently. The learning curve is very smooth, and there is enough documentation. 

We use OpenText LoadRunner Cloud for performance testing of our applications.

We migrated from on-premises LoadRunner Professional to LoadRunner Cloud because it offers geographically distributed compute resources that can be scaled up and down on demand, optimizing costs.

OpenText LoadRunner Cloud helps us save money through its flexible scheduling feature. We can shut down resources when not in use, minimizing costs.

OpenText LoadRunner Cloud eliminates the need for our own testing infrastructure when running tests.

We're interested in leveraging the scriptless automation capabilities available in several tools. Some of our less technically inclined manual QA testers find them insufficient. They crave drag-and-drop functionalities or more intuitive scriptless automation options. Scriptless automation is an area that can be improved.

I have been using OpenText LoadRunner Cloud for six months.

OpenText LoadRunner Cloud is scalable.

Our OpenText Account Manager coordinates with their technical support to ensure tickets are routed and resolved efficiently.

Positive

With nearly 400 virtual machines to migrate, transitioning from on-premises to the cloud wasn't quick, but it proved to be a smooth process.

OpenText helped us with the migration.

The pay-as-you-go licensing model included in our cloud migration is a valuable feature. It ensures we only pay for what we use, eliminating unnecessary fees during periods of low activity.

The pricing for OpenText LoadRunner Cloud is average.

I would rate OpenText LoadRunner Cloud a nine out of ten. While OpenText LoadRunner Cloud boasts many AML capabilities, we're still exploring its full potential.

We use LoadRunner Cloud for working with banks, and I find the tool incredibly versatile for our varied load testing needs across different business units. Whether it is testing the API backend for core banking systems or the web frontend for digital applications, LoadRunner Cloud handles it all with ease. 

One of LoadRunner's standout features is its extensive support for various TechStacks and protocols. Unlike other tools, LoadRunner excels in integration and protocol support, ensuring it can capture data behind proxies and work seamlessly with diverse technology environments. This versatility makes it invaluable for testing applications across different platforms and protocols.

One area for improvement in LoadRunner Cloud, especially for agile models, is its limited support for functional testing alongside its robust non-functional testing capabilities. Unlike some other tools in the market that offer both functional and non-functional testing within a single platform, LoadRunner requires separate test scripts for each, doubling the testing effort.

I have been using OpenText LoadRunner Cloud for almost ten years.

LoadRunner Cloud's scalability has been reliable for my needs, although I haven't tested it on a very large scale like e-commerce platforms might require. As a bank, I have found it to be cloud-based and easily scalable, with no issues in distributing load or managing large tests.

OpenText's technical support is generally responsive and helpful. They address issues effectively, although specific or niche protocol requirements might take longer to resolve due to prioritization. Overall, I would rate their support an eight out of ten.

Positive

We have used similar solutions like JMeter and NordLayer. LoadRunner's compatibility with mainframe systems, along with other monitoring tools like BMC Patrol, sets it apart from other options we have tried.

The solution is a bit expensive. The pay-as-you-go model offered by LoadRunner Cloud is important to us, especially when considering the cost-effectiveness of performance testing.

LoadRunner Cloud's infrastructure management-free approach is a big plus, especially for multi-geography organizations like banks. By leveraging cloud-based load generators, you can ensure native performance testing without the hassle of maintaining hardware. Plus, while there may not be direct cost savings, the convenience and flexibility of cloud deployment can lead to more efficient resource utilization and potentially lower operational overhead in the long run.

Overall, I would rate LoadRunner Cloud as a nine out of ten. It is technically superior and well-executed, but the cost is a drawback. Additionally, its focus on non-functional testing means separate scripts are needed for functional and non-functional testing, which can slow down testing processes. 

We use OpenText LoadRunner Cloud for different types of testing, including load testing, performance testing, etc.

The product’s most valuable feature is the Vuser license; it allows us to reduce the cost as per requirement. Additionally, its setup process is accessible as well.

We encounter hurdles while running the professional version for on-premise setup. They should work on this particular area of improvement.

We have been using OpenText LoadRunner Cloud for nine to ten years.

The product is more stable compared to other vendors like NeoLoad and JMeter.

It is a scalable platform. I rate its scalability a nine out of ten.

The product’s technical support team’s response time could be better.

Neutral

We use NeoLoad and JMeter.

OpenText LoadRunner Cloud’s setup is straightforward. I rate the process a nine out of ten. Although, I rate the process for the on-premises version a seven out of ten as we face challenges there. The on-cloud setup takes four to five hours to complete. It takes longer if there are some additional protocols included.

One executive from our team and one from LoadRunner’s team work on the on-cloud setup process.

OpenText LoadRunner Cloud is economical. However, it is expensive compared to other tools.

I recommend the product for performing testing of short windows. It would help if you opted for other tools, in case you are working with long windows. I rate the product a nine out of ten.

The best benefit would be budget. It's a really inexpensive if you are testing your application every spring or every month. Even for less frequent applications that you have to test regularly. You can hook up your infrastructure to StormRunner Cloud and you can get the best of both worlds.

In larger enterprises, we have different departments using Performance Center and StormRunner or Selenium. So, StormRunner can act like an umbrella and plug-in everything, get it executed or done across the world. Since they use both HPE Cloud, AWS, and Microsoft, it uses more access points geographically to test it making it best of the class.

StormRunner itself is a pretty good hybrid product of Performance Center. Keeping up with DevOps, thus the best feature of StormRunner is that we don't have to build and maintain infrastructure anymore. Whenever we have to test lab scale applications, and in a scenario where we don't have to test every day, we don't have to build the machines and pay for it. Instead, we can get the infrastructure from on-demand from StormRunner, and its ability to run it anywhere just by opening it in browser is the best part. The evolution of StormRunner starting with supporting LoadRunner along with the open source technologies like JMeter and Selenium. Their unit testing tools are actually very advanced in the region of the product. That's why I would recommend anyone to use StormRunner, even though something is not supported now, eventually it's going to be supported by StormRunner. That's the kind of credibility that's needed for any customer when it comes to relying on a product or going for a new product.

More insight into test results and allowances. It might be a tailor-made requirement for me, but I would like to download them offline and do my own customizations on the reports. Right now, we have some standard templates that generates reports. But if I had to do some customizations, include something else and create a report, it's not easy. So, if I can an order to download the raw data I can make custom report. That would be the case with every customer because even though they tested one product it might be part of a big project and they need to have other information included in it along with the report, so feasibly it is good to have.

Also, it's evolving, where there's too many features for me to handle and it's too much on my plate to any make ROI out of it.

There is a steep learning curve for the product, too.

Over the last couple of years, it's been evolving. We started using more in the last one year and we see it's pretty stable. Adding new features to StormRunner is going to slow down once it has every feature. That has to do with stability and I don't see any drawback in that.

We haven't really hit any scalability issues by using StormRunner, but I just now heard that they can support two million users, which is kind of astonishing. The max I have used is 40,000 users, but two million is good, too. It's big step up from 40,000.

At the same time, you don't have to get the infrastructure built and set it up. For example, let's say you had to test it for one million users. You don't have to procure all of the machines you need for the one test you're going to do. You can do it on demand, that's the best thing. Two million is still very much overboard, but it's good to know. It's good to know that there's no limitation nearby.

I'm not a big fan of the tech support to be frank. SaaS tech support (HPE tech support or Micro Focus), there's a gap between the people who access our tickets and the people who know the product. So, there is always a blind exchange of information within Micro Focus and most of the time get frustrated with the kind of ticket updates we get. What happens is the person handling the ticket might not be an expert in the product so we end up redoing everything. Communicating everything about our setup and infrastructure and the customer engagement for each ticket. Therefore, we started involving our technical correspondent with Micro Focus, but we're told that it's a pretty messy situation over there.

I always thought it would be good to have something like this from HPE because we rely a lot on HPE. Then, StormRunner was released and I knew where exactly it was going and what's it would be for us.

I was involved in the initial setup. It was straightforward and there was a lot of good information available, but I did not need it. I didn't go through any support to set it up. The documentation itself was good.

The other product we considered was BlazeMeter. We eventually chose StormRunner. I think only those two are pretty much in the market - nothing else.

I like that StormRunner incorporates the idea of accepting and adapting all open sources. It is my understanding that they are planning to continue accepting, supporting, and adapting all open sources.

For someone evaluating StormRunner and similar products, there are two parameters I would tell him to evaluate:

1. Is the application under test? Is it customer facing?
2. Is how often do you test it?

Most important criteria when selecting a vendor: We are a retail business. The first thing I look for is any customer use case, or case study for the vendor. I look to see if they have proven that they are good resource for any existing retail customers. I look for any kind of case study in those lines before I jump in.

We are using all the indexes and the features provided by the solution.

We have an infrastructure that is developed on the cloud. We use the product because it is more feasible to have our applications on the cloud. It is a trustworthy brand. The data I present to my customers are to the point. I do not have to check on the matrices again to verify whether it is false.

The reports are very relevant to the customers’ expectations. The tool has its own APIs for its customers. It is very easy to collect the performance data. In the last year, we have executed major tests multiple times. It's very easy to use the APIs to collect our results and create a dashboard as per our needs.

It is very important to us that we can plan and run tests using LoadRunner Cloud without having to manage testing infrastructure. It was a niche requirement for us not to bother about those things and focus on the main targets. It is the best thing that we can get. The solution saves us money by not having to maintain hardware and the power costs associated with the hardware. We have saved 25% of the cost.

The product must provide agents to monitor servers. If we want to monitor our servers, we should be able to do it by integrating the servers with LoadRunner Cloud’s dashboards. The added feature will enable us to see exactly what is happening in the servers at a particular time.

I have been using the solution for more than five years.

I haven’t seen any stability issues. The stability is good.

I have scaled the tool up to 1000 users. It comes with its cost.

I got a solution from the support team every time I had queries.

Positive

I have received an ROI from the product.

There are other tools that provide the cloud capabilities and key features that LoadRunner has, but LoadRunner‘s brand and trustworthiness are more important to me.

Overall, I rate the product an eight out of ten.