OpenText LoadRunner Cloud Reviews

It's the license and structure because it's so dynamic. Virtual user hours is much more dynamic than virtual user days, and even more dynamic than static licenses. I did a test a short time ago where we could keep the test to half an hour so we could really optimize this virtual user hour. That was wonderful.

It's used to improve our customers, but we don't use it in-house.

I'm a consultant and this was used for a small project. My customer bought around five consulting days from me. We ran into some undocumented limits around the number of concurrent users. It was really just set up to keep the guys out that wanted to do denial of service using the demo account. It was not clear that this was the limit it would hit. That's expensive for a company. Out of five days, maybe half a day. That's 20% of the budget went into trying to run a test. The test fails, contact support, ask what's going on. Make them move the limit and then re-run. That was bad. It should just be upfront - these limits exists, be sure before you start that you don't want to move them.

Quite good.

I trust it. I haven't tried it with thousands of virtual users. As true client, yes of course. We had arl 6000 true client users. That's 60 gigabyte memory. It's not a PC, so it's not large.

Quite responsive.

We were previously using LoadRunner for our customer, but we needed something that could be used in the cloud. I advised the customer to buy StormRunner Load as given the time frame it was clear from the beginning that we needed to use the true client scripts. We didn't want to set up the memory foot print for them so we had to go to cloud.

It was difficult because the Amazon stuff didn't work. I don't like to use Microsoft stuff. I was forced to put it onto Azure Cloud because that was the one that just worked out of the box. The Amazon launching died somewhere in the process. There was another error there as well.

I'm an HPE partner. I don't look anywhere else.

For all the in house stuff where we run into firewalls, it just gets bad. For public websites, it's perfect.

The flexibility makes it a viable solution.

The reason we chose it and the most valuable feature is the amount of virtual users we can run in a test. We have run up to 900,000 users.

We do online testing for the States as we have contracts with them. We have to be able to support a certain number of students, so this helps us confirm and validate that we can support the number of students that are taking online testing at one time.

I think the reporting could be better. I think you could have a little more variety of reports. Right now, you get an email saying, "Here's your report". You can't put in the analysis tool. You can't export it to Excel. Those are the things that would make it a little better. That's probably my biggest pain point, it is the reporting.

I think it's fairly stable. There's some features that I think could be made better but, for what they have, I think what they have works well.

I think it scales really well. I like that you can add users on the fly or remove users on the fly. That's a great feature.

I have used the support. We were one of the first users for Beta, so we have a relationship with the developers, so we tend to just go right to them when we have questions or problems.

They've been great. A lot of the times, it's not really a problem with StormRunner, it's just our understanding of what it's supposed to do, so they are very great about explaining the reasoning and what will be coming in the future. If we don't think that's really what we want, they'll explain how it's in the roadmap and how it's going to be made better.

We were using another HPE product, Performance Center. We haven't totally switched, but StormRunner Load is our option for the test where we need a lot of virtual users where we can run it in the cloud. Whereas, on Performance Center, you have to have load generators in-house, or what HPE provides. That's why we use it. It's for when we want to use a lot and perform a big test.

I wasn't involved in the deployment.

Not really because we're mostly an HPE shop and this was working with HPE they to meet our needs, so they provided this and it's worked out great.

I don't like their reporting. Other than that, I think it's great. It does what it's supposed to do and I know it's just the beginning, so there's going to be a lot more features added.

You should look into it. I know there are other solutions that have more bells whistles, but,when it actually comes down to function, I think StormRunner does the job.

• Ability to get started on the fly
• Reusability of existing HTTP/HTML scripts

I have been able to run several tests from outside the corporate LAN, which in the past used to be an infrastructure nightmare as it required opening up ports, disabling firewalls, etc.

It needs the ability to execute other protocols, especially the TruClient for Firefox without having to convert them.

We've used it for over a year.

There have been no issues with the deployment.

We have had no stability issues.

It's scaled to our needs.

Customer Service is immaculate.

Tech Support is immaculate.

There was no solution in place previously.

There was no set-up required.

It's done in-house, but there's no actual implementation needed.

There pricing and licensing options are very aggressive, but the holiday discount brought down the price further.

I think that StormRunner Load is the way forward and if HPE manages to support all protocols currently supported by LoadRunner/Performance Center eventually in StormRunner, it would be a massive game changer.

StormRunner Load reduces the overall time for performance testing engagement and is best suited for the products in agile process. Easy to automate and execute the tests, StormRunner Load provides a feature to compare the current test results with the previous results to see if the current build is better than the previous or not.

With the ability to integrate with monitoring tools like SiteScope and APM tools like New Relic, performance bottlenecks in the applications can be identified very quickly and easily.

Protocol Support

StormRunner Load supports Web, Web Services, SAP Web and JMeter scripts

Easy Automation

StormRunner Load uses Truclient and TruAPI (which uses node.js) which enables developers and testers to quickly automate the user actions at the browser level, which also helps measuring the end user experience by measuring the browser rendering times as well. Performance testers can upload their LoadRunner scripts that are developed using LR 12.XX to StormRunner Load directly.

High Scalability

StormRunner Load is highly scalable and supports upto 1 million concurrent users.

On Premise Load Generators

StormRunner Load introduces the concept of On Premise Load Generators to compare the application performance within the network to various other geographical locations.

Configure SLAs

SLAs can be configured in StormRunner Load during the test execution. The tool highlights all the transactions that did not meet the SLAs and further drill down can help the users understand the geographical locations and times where the SLAs are not met.

Real time anomaly detection

During the test execution, StormRunner Load detects any anomalies where the test results are deviated for a specific transaction / geographical location / script / counter to identify the issues real time and further drill down provides more details about the anomalies.

Integration with Monitoring Tools

StormRunner Load integrates with monitoring tools like SiteScope to provide the hardware and application level metrics during the performance tests. StormRunner Load can also integrate with New Relic and other similar APM tools that helps identifying the server side bottlenecks so that organization can reuse investments

HP StormRunner Load has provided us a) flexibility for on demand load generation b) reduced over all TCO on our load testing efforts c) Reduced the skill level to develop load test scripts.

Custom load patterns for the performance tests: Currently StormRunner Load supports only linear ramp up of the user load and doesn’t provide the user load to follow a step incremental load pattern or any other pattern that a performance tester wants to.

Load distribution among on premise LGs: StormRunner Load currently supports load generation from only one of the On premise load generators and the user load cannot be distributed to multiple on premise LGs.

Simulation of latencies: There is no provision to simulate network latencies to simulate realistic geographical locations and mobile networks.

Ability to create custom reports: StormRunner Load provides only the default reports and there is no provision available currently to generate the custom reports

Backward compatibility: StormRunner Load expects the LoadRunner scripts that are generated from LR 12.00 and above. Any scripts developed using the older versions are not supported.The work around to overcome this problem is to open the scripts in LR 12.00 or higher and save it again before uploading to StormRunner Load..

We have used the product for more than 14 weeks now and tested on some of our internal applications in the organization that would require performance testing.

HP StormRunner Load doesn’t need any installation or deployment unless you wanted to configure few on premise load generators. Configuring on premise load generators is pretty straight forward. All that you need is to have an account with HP SaaS to start using StormRunner Load.

HP SaaS web application is pretty stable and no issues were seen during the evaluation process.

There were no issues with the scalability, however, the time taken to initialize the test is directly dependent on the number of users to be simulated.

5/5.

The support provided to us very good which helped us to evaluate the solution for our context.

We are in the process of adopting a SaaS based option, this led us to evaluate a bunch of tools.

It is simple - just registering with HP SaaS platform called pronq.com.

We evaluated through our in-house resources.

Our estimated ROI is positive we need a couple of months to gain insight into actual ROI.

HP StormRunner load is SAAS offerings so the initial setup costs are very minimal when compared to the traditional approach (no or minimal capex ), and the day to day costs would be purely driven by the duration and number of virtual users required to simulate the traffic.

We have evaluated HP StormRunner Load against other market competitors that include Blazemeter, SOASTA’s Cloud Test, NeoLOAD, VSTS, Load Complete and StressStimulus.

HPStormRunner load is an apt product a) if your application stack has Web 2.0 traffic b) your requirement is for a service that allows for pay as you go model c) proven and reliable load testing solution with superior support.

This review was co-authored by Sravanthi Naga & Ramanjaneyulu Narra, Performance Architects at Gallop Solutions. Gallop is a US based Co-located Independent Testing Services operating since 2003 with offices in Philadelphia & California. World's largest and leading organizations have relied on Gallop's specialist independent software testing services for more than a decade and have achieved significant market acceleration, returns on investments in their software quality initiatives.

The platform is easy to use for uploading and backing up scripts.

I had to do overload testing on our homepage. I couldn’t use our regular tools because I wanted to simulate a real scenario and StormRunner Load does this. It enabled us to upload the scripts into the cloud and gave me unlimited resources.

The monitoring abilities were lacking. The reporting didn’t present the information in a clear manner for me. It took me four days to understand why I couldn’t upload the scripts from my computer to the cloud, and I discovered I was missing an important feature that StormRunner Load is missing, called Corver, and this converts the script for the tests.

I used it once in a testing session and have been using it for one month in a live environment.

No issues encountered.

There was a bug and we opened a ticket with HP, and it was fixed after a few days.

No issues encountered.

It was great, there was an issue and I worked with Kim from HP who was great.

It was great.

No previous solution used.

It was straightforward, we just needed to register on the site. In comparison to other HP products, it was very user-friendly.

We did it in-house.

The charts dashboard, especially the results screen, is the most valuable feature. I’ve also configured the tests, so I’ve also used the test definition page, which is very easy to use, self-explanatory, and simple. The real value comes from the results screen.

StormRunner Load helped us evaluate the performance of the product version we were about to launch. It showed us bottlenecks, where our product reaches its limits, and different characteristics that affect this. For example, we had metrics we thought were affecting our response time. Initially we thought there were two metrics, but we found another significant metric we should have taken into account. Without StormRunner Load we wouldn’t have known about this "missing" metric.

I think the report page could be improved, although we have used the reporting feature. It wasn’t accurate enough to use as a report that I could send my managers. I had to take additional screenshots to help fill in the reporting. I actually, ultimately, generated a report on my own.

I've been using it for two months.

No issues encountered.

No issues encountered.

No issues encountered.

There was no previous solution in place.

This solution immediately helped us to determine the priorities for what to develop next and where we would reach our limits. Because we had a clear prediction, we knew when it would be urgent to develop features, in various cases. Before StormRunner Load, we didn’t have a clear developer roadmap, and now its very clear where we should invest, our weakest links, and what we need to do to strengthen weak links.

If you’re using third-parties for networking, cloud based servers, load balancer, VPN providers etc., all of them should be informed about the expected load. We had a VPN provider that was expecting thirty users per hour and calibrated accordingly, and the minute we started our load test it shut down. This third-party was not ready. Once we informed them that we were increasing our load, they increased their ability to support the load test.

No, this was my first choice.

It is very cost effective to start with a low number of virtual users for a short amount of time in order to verify that your company system is running properly. Start with a small load test, run it until completion, verify that you see the logs you intended to see, and all people involved in the environment are ready to see what they expected.

• Load generator setup is not required so we can be running high concurrency tests in few minutes.
• Scenario setup for running tests was very easy.
• Distribution of the load from different locations was very easy.
• SiteScope and New Relic monitoring tools can be easily integrated with StormRunner Load.
• SLA configuration was available to monitor response times.
• Navigation breakdown report - to create a client navigation breakdown report.
• Real time graphs for monitoring results.
• Email reports.
• Benchmark- comparing previous test results with current tests.
• Anomalies detection.

Being able to setup and run 300,000 load tests using LoadRunner Performance Center takes around 1 to 2 hours when creating four scenarios of 75,000 each, including allocating/blocking load generators required for testing, initializing the fusers and running the test.

Running a 300,000 test was very easy using StormRunner. We just had to upload the script and create scenarios for 300,000 and run the test in 20 minutes.

Running back to back high concurrency test was very easy using StormRunner.

• Improved PDF reports and creating custom reports
• Real world- custom scenario setup
• Update existing scripts online without uploading them again
• Run time settings update in scenario

I've used it for five months.

None - there's no deployment required for running a load test.

During the initial days of high concurrency testing (300,000 and 600,000) we had issues with the StormRunner infrastructure e.g. graphs “disappeared” in the middle of a test, and HP had to give an online fix. Also, there were cases where tests appeared as “initializing” even though it actually started and HP again had to give an online fix. The StormRunner R&D team was very proactive in fixing the issues in very small turnaround time and supporting us running the high concurrency tests.

Scalability issues were seen during initial days of testing but it was resolved by StormRunner R&D team.

8/10.

8/10 - their team was very proactive in resolving the issues and provided hotfixes to support us running performance tests whenever required.

We were using Performance Center to run performance tests, so we had to switch to StormRunner to run high concurrency tests.

As StormRunner Load is a web-based application, it's simple setup, just registering users through an HP site.

In-house implementation and evaluation.

There were no other solutions evaluated.

StormRunner is one of the fastest-growing cloud based performance testing tools for running high concurrency tests very quickly.

I can use it just as much as I need. It is quick and intuitive.

We are able to ensure very quickly and frequently that our product is ready for prime time. Using StormRunner, Codefresh has managed to identify performance issues and when these issues are introduced.

When you start a test, it takes times to initiate.

We have been using the solution for 7 months.

No issues with deployment.

No issues with stability.

We never encountered issues with scalability.

Excellent customer service.

Excellent technical support.

Very intuitive.

In-house technical people.

StormRunner does the job in a very simple and cost effective way. HP has always lead this market.