OpenText LoadRunner Cloud Reviews

The best benefit would be budget. It's a really inexpensive if you are testing your application every spring or every month. Even for less frequent applications that you have to test regularly. You can hook up your infrastructure to StormRunner Cloud and you can get the best of both worlds.

In larger enterprises, we have different departments using Performance Center and StormRunner or Selenium. So, StormRunner can act like an umbrella and plug-in everything, get it executed or done across the world. Since they use both HPE Cloud, AWS, and Microsoft, it uses more access points geographically to test it making it best of the class.

StormRunner itself is a pretty good hybrid product of Performance Center. Keeping up with DevOps, thus the best feature of StormRunner is that we don't have to build and maintain infrastructure anymore. Whenever we have to test lab scale applications, and in a scenario where we don't have to test every day, we don't have to build the machines and pay for it. Instead, we can get the infrastructure from on-demand from StormRunner, and its ability to run it anywhere just by opening it in browser is the best part. The evolution of StormRunner starting with supporting LoadRunner along with the open source technologies like JMeter and Selenium. Their unit testing tools are actually very advanced in the region of the product. That's why I would recommend anyone to use StormRunner, even though something is not supported now, eventually it's going to be supported by StormRunner. That's the kind of credibility that's needed for any customer when it comes to relying on a product or going for a new product.

More insight into test results and allowances. It might be a tailor-made requirement for me, but I would like to download them offline and do my own customizations on the reports. Right now, we have some standard templates that generates reports. But if I had to do some customizations, include something else and create a report, it's not easy. So, if I can an order to download the raw data I can make custom report. That would be the case with every customer because even though they tested one product it might be part of a big project and they need to have other information included in it along with the report, so feasibly it is good to have.

Also, it's evolving, where there's too many features for me to handle and it's too much on my plate to any make ROI out of it.

There is a steep learning curve for the product, too.

Over the last couple of years, it's been evolving. We started using more in the last one year and we see it's pretty stable. Adding new features to StormRunner is going to slow down once it has every feature. That has to do with stability and I don't see any drawback in that.

We haven't really hit any scalability issues by using StormRunner, but I just now heard that they can support two million users, which is kind of astonishing. The max I have used is 40,000 users, but two million is good, too. It's big step up from 40,000.

At the same time, you don't have to get the infrastructure built and set it up. For example, let's say you had to test it for one million users. You don't have to procure all of the machines you need for the one test you're going to do. You can do it on demand, that's the best thing. Two million is still very much overboard, but it's good to know. It's good to know that there's no limitation nearby.

I'm not a big fan of the tech support to be frank. SaaS tech support (HPE tech support or Micro Focus), there's a gap between the people who access our tickets and the people who know the product. So, there is always a blind exchange of information within Micro Focus and most of the time get frustrated with the kind of ticket updates we get. What happens is the person handling the ticket might not be an expert in the product so we end up redoing everything. Communicating everything about our setup and infrastructure and the customer engagement for each ticket. Therefore, we started involving our technical correspondent with Micro Focus, but we're told that it's a pretty messy situation over there.

I always thought it would be good to have something like this from HPE because we rely a lot on HPE. Then, StormRunner was released and I knew where exactly it was going and what's it would be for us.

I was involved in the initial setup. It was straightforward and there was a lot of good information available, but I did not need it. I didn't go through any support to set it up. The documentation itself was good.

The other product we considered was BlazeMeter. We eventually chose StormRunner. I think only those two are pretty much in the market - nothing else.

I like that StormRunner incorporates the idea of accepting and adapting all open sources. It is my understanding that they are planning to continue accepting, supporting, and adapting all open sources.

For someone evaluating StormRunner and similar products, there are two parameters I would tell him to evaluate:

1. Is the application under test? Is it customer facing?
2. Is how often do you test it?

Most important criteria when selecting a vendor: We are a retail business. The first thing I look for is any customer use case, or case study for the vendor. I look to see if they have proven that they are good resource for any existing retail customers. I look for any kind of case study in those lines before I jump in.

Its ability to work well with other HP products, particularly SiteScope, allows us to maintain consistency in our infrastructure. The different solutions work seamlessly together and free up resources that would otherwise be spent on conflicting vendors and products.

It's fast, easy to use, has a user-friendly UI, and you can split users. It also works well with SiteScope and other HP products.

It doesn't provide custom reports. You can only use the default reports which contain irrelevant data or is missing data that we need.

Also, a lot of our customers have issues logging into the cloud.

It's been stable.

It scales fine because our customers are usually larger.

Technical support is something we're not so happy with.

The initial setup is straightforward.

It's very valuable because with StormRunner we can validate the capacity of our systems. Specifically, we use it for one system that is from two years ago. We have an increment of the users, about 300% more, so we need to assure that the quality of the service keeps up with the users. StormRunner helps us to validate that.

Right now, we are just starting to use it. I don't have any comments on that.

Less than a year.

Yes, it's stable. We don't have any issue right now with the performance or anything like that.

It's very flexible. We can have load testing for 100 users, maybe 1,000 or even more, so yes, no problem with that.

It's very good. We also have support from our partner, Micro Focus, so it's basically good on that.

No, it was straightforward. It was very easy to implement and to have a load test.

This solution is SaaS based so we can utilize cloud technology, which is less time consuming and saves a lot of of money.

Sometimes, you are utilizing one of the low generators, then all of a sudden if you discontinue from one project, it actually deletes the entire low generator. You have to go through all the setup again. This kind of problem is on and off. It comes and goes.

We have just been using it for about two months now.

This is a new product, so there are a couple of flaws, sometimes things act up weirdly. We reach out to the support group, they solve the issues. It's an ongoing process. We understand that.

Since this is a new solution, they need to improve the support. They also have some lack of expertise. We understand that.

It is complex. You need to convince the leadership and the IT security team. You need to solve all these issues. There are a lot of things that you have to go through.

We chose this one because of the name. It's established, the last decade. We used to be with this company. Micro Focus is more than seven or eight years, we're good with them, they are good with us.

It's a new product and we are facing some issues. We believe, they are going to work out at their end. They are going make it work and fix everything, whatever the flaws are now.

Overall, we are using this to save money and utilize the cloud technology.

The most valuable feature is the ability to configure browser settings for different operating systems and on different versions without the need to install every single version on each machine and to manage them. This is the most useful feature that I'm interested in.

I'm also interested in mobile center services related to StormRunner functions because we are going to determine which will be the best solution for developing mobile testing. And we are interested in outsourcing mobile testing, so this kind of environment will be useful as we are not engaged in the configuration of infrastructure. Also, I think it will be useful because Storm Runner functionality does not need specific configuration because the configuration basis is in the UFT and the UFT can be in our environment. We do a lot of configuration because we develop a custom framework related to UFT. So, I appreciate that configuration can be managed by us, while the browser can be released on the remit version.

I'd like to see more ability to dive more deeply into the configuration.

I think the product is still immature because we have some problem reaching the mobile center connection related to a proper configuration on our machine. In fact, we are in touch with the R&D team to fix the configuration for our environment. I also think that it will be useful to have an object recognition tool because the specific configuration on the browsers are to be managed so as to fix our specific test. I need to recognize what was the configuration on the remit browser and I need a tool to analyze the impact of the browser configuration on the test.

I think the scalability is fine with this kind of solution because I have no problem with how many machines will be available to run the test; it's a matter of the cloud provisioning. It will be useful to run the tests in multiple environments in parallel. That will be a good tool for us when we have to make tests with a shortage of time.

I have not used technical support recently. HPE support is always prompt to solve the issues. That is the main thing I appreciate regarding support.

It's in the cloud. Everything goes in the cloud. We needed something that's easy to set up in the cloud so we can use it directly there.

It's very easy to set up a load test, to start it and end it. It is valuable that it is very easy to use.

We are not benefitting a lot at this time because it has its bad parts.

It needs more features; a lot more features. Everything that's in LoadRunner Analysis should be built up in StormRunner; so we can use everything, every functionality and every feature all together.

It's stable. It's not perfect, but it's stable.

It is not really scalable.

We didn't use it that much. It was a demo session set up especially for us.

It's very east to set up. I did it directly myself just once.

We compared it to LoadRunner and Performance Center because we use that currently for load and performance tests. We cannot use this solution in the cloud. StormRunner is very rudimentary at this time, but it's a great concept. We hope that it will be thrilling.

We chose StormRunner because of the cloud. That's a great part of it.

Wait with StormRunner.

JMeter script integration. This integration feature is a good approach provided by HPE. Sometimes, we need to run scripts developed in an open source tool like JMeter. Instead of spending time on creating new scripts, the scripts developed in JMeter can be uploaded and based on the protocol license because it is a DEV user protocol, it provides a valuable feature when integrated with StormRunner.

Monitoring and sample access to servers needs to be provided, so that we can explore this and keep up-to-date.

The licensed version / free edition needs to be configured with SiteScope or AppDynamics, which many customers might not have. The LoadRunner feature of Windows Resource Monitor should come in handy in such scenarios. It is not only for Windows-based monitoring but also for UNIX-based monitoring. Dependency on the other tools or HP tools (SiteScope) would be reduced.

It needs to be more mature.

The client breakdown is a separate report and is not part of the existing load test report. The client breakdown is basically the response time observed from different browsers. We need to take a snapshot and place it in the load test report and that is not available when the load test report is generated from StormRunner.

Also, for the VuGen Web protocol, it supports only 12.50 and above, which is tough and compatibility is lost.

More than a year and half

I did not encounter any issues during deployment, but there are not enough tutorials available for the TruClient protocol and also on the reporting side.

I have worked extensively on the various module supported by STRL2.1v and provided my suggestions/improvements/recommendation to the HP R&D Team

Good

The level of customer service / technical support is satisfactory, with a few times good.

For example, when trying to learn the monitoring feature of StormRunner, I requested access to the Web Tours application that is installed on the server so I could explore that feature in depth. However, access was denied (monitoring-only access was provided). My question was: When both the application and the tool belong to HPE, why was access denied?

The end user needs to have strong skills in any of the load testing tools, specifically if they already have experience with LoadRunner.

We've seen in the past where we have customers that use other performance testing products such as LoadRunner and Performance Center that they often have the need to do spike testing. Their testing for 1,000 virtual users running at the same time but before the application goes into production or before a big event, they need to scale up to 10,000, 20,000, 50,000, and so forth. We want to make sure that they can test to those specific limits and StormRunner allows us to do that.

I think the biggest issues that I've seen, and this is a personal view of mine, is that most of the HPE products have a common look and feel to them. I'd really like to see it be a little more customizable to a use and user standpoint. For example. I happen to be colors blind, so I'd like to see more vivid colors on the UI, and things like that. It would make it a little more flexible from the customer standpoint.

I was formally an HP employee, so I've used the HP products in the past. I also was a former Mercury Interactive employee which was acquired by HP.

It's been extremely scalable as far as the testing that we've done with our customers. They've all been really satisfied with the scalability of the HPE products.

We've gone through the standard customer support and it’s been good. It's not quite to the level that an R&D would be, because they really get really under the covers to fix things.

It really doesn't need to be as flexible as some other solutions because it's right out of the box, it's relatively easy to use. You open up, you create a test in ten minutes and then you're able to run it.