OpenText LoadRunner Enterprise Reviews

Performance of load test applications, reliably, and with good reporting.

We have high confidence that the results are reliable. It's helped catch multiple bugs before they've gone to production. And we're able to more confidently release software into production, just more confidently release code.

I'd like to see better reporting, ultimately, and analysis. I'd like to see the analysis more accurately pin-point where these issues are; a little more feedback immediately in terms of the anomalies that we're seeing during the testing, so we can get alerted much quicker, instead of blowing a whole bunch of our day to test data during a test.

The reporting could be a lot better. It's cumbersome. The VuGen takes a long time to load up, and edit and execute things off those. So a lot of the basic infrastructure.

We've had a few crashes, but overall they seem to be more of the aging architecture, the hardware's not as reliable. A few software bugs in there, which we have to work around. But overall, it's been pretty reliable when you think about how much load goes through it and the number of people using it.

We've scaled it up to, well, to our limits at least. A good 1500 users and some 10 million transactions, or something like, that per hour. It's generally handled that pretty well. We're looking forward to moving into StormRunner next.

The tech support seems to be pretty good. There have been a few problems that they haven't resolved, but they've led me in the right direction to eventually figure it out on my own. It's very difficult for them to understand all the wide range of applications we use. So I'm really impressed with how ultimately knowledgeable they are, given that they don't actually use all these applications that are out there.

I wasn't involved in the initial setup but I've been involved in all the upgrades. They've done it multiple times. Each upgrade seems to be adding a lot more features. It seems like the pace of development has picked up in the last couple years especially, and we've been impressed with the direction that the product's going.

The upgrades actually were fairly complex for my team. The last one, they were actually giving up for a while. It turned out that there were some database versions that were incompatible. Eventually, once we were able to get connected with the support, we were able to get most of it worked out.

There was a big upgrade with the 12.5 version that was a little slower. But everything else has been very smooth since then. Still, it's difficult. Everybody's come to count on that software being at a certain version.

The most important criteria when selecting a vendor are, I suppose, support, and the reliability in the industry.

They have a good road map for moving forward. It's going to be integrating with a lot more other products that the DevOps world is starting to push upon us.

It's not quite perfect. It's still the best in the market. It holds up to a lot of the scrutiny that the developers constantly throw at us.

There have been a few issues, especially around the scripting and the IDEs, handling of some of the protocols, that still don't perfectly match with what we have in production. But overall, we're able to pretty well defend it. You can almost always reliably pull up the application, and the response times seem to match what we have coming out of LoadRunner.

I would definitely advise looking at Performance Center, it's still the best in the breed. Just make sure that you have a good team in place that can implement it.

One of the things in the airline business, we see that our number of users varies on a day to day basis, from season to season. So, from an airline business standpoint, we are looking at scalability as one the major things and how can we adapt the solution in an agile fashion. If we want to ramp up the "Our Views" account, from, let's say 10,000 to 50,000, how can we do that? That kind of scalability is the main key thing we are looking at.

One of the key things we use is simulating the actual user experience on the log. We have a huge set of applications from front end to the back end systems. How do we integrate all these systems and how do we simulate the real time user behavior? That's where we see a key value.

One of the things we were looking for is more of a DevOps support, like BlazeMeter has. It would be an ideal scenario to incorporate those kinds of features. I know there are some open software products which have that but it would be ideal to see those features in the product.

As of now, it's working great for us, it's excellent. We don't have any issues. That's one of the reasons we are pushing forward to version 12, to incorporate the other protocols, which version 12 comes up with.

We are at version 11.5 and we are in the process of upgrading it to version 12. We are pretty happy with the solution we have.

We do have a dedicated team. They work with our tech support and with their tech support in terms of the installations and stability of the product and usability. All those issues, they take it up with tech support.

Tech support is pretty excellent. We are getting pretty good responses back from tech support and, as of now, we are happy. We do have a contact too, from the United Airlines side too so I'm pretty happy with that.

Ours is more of a historical basis. We were on version 9, we moved to version 11, and we are right now at version 12. It's more for historical reasons rather that an impulse buy.

I didn't work on the installation of 11.5, but right now I'm working on version 12.

Internally, we have a lot of planning to do on our side, like a database upgrade, LGs, all that stuff but we are coordinating that with HPE and Micro Focus and making sure that our timelines and their timelines match. And we do have upgrade licenses, which should be pretty good to go.

I would say the relations between us and Micro Focus is straightforward because all we are looking at is basically license upgrades. On our side, it's more complex because we have to internally work with various teams to coordinate all this activity.

The most important criteria when selecting a vendor to work with are the product, how easy it is to use the product and, again, how scalable the product is and how it suits the needs of United Airlines. And, of course, the customer support, and how technical support deals.

Regarding advice to a colleague, it depends on the industry and what kind of problem they are trying to solve. If it is in the airline industry, I would definitely suggest to them, "Okay, this is a perfect product because of the number of protocols it supports," because we looked at other open source software and we couldn't find a product which matches Performance Center, which supports so many protocols. So, especially in the airline industry, we are using multiple protocols and we need that support. I would definitely recommend that.

High scalability. Web-based testing. The interface. If you're familiar with the days of using LoadRunner, when you had to have the 32-bit client, using a web-based client is fantastic. You can spin it up relatively quickly despite the fact that it's enterprise software. You can test a huge variety of applications, not just web-based systems, but SAP, Oracle, web services, pretty much anything out in the market place, but it's mobile-based testing.

In my current organization, I honestly don't know so much. But in my previous organization, when I was doing consulting, we helped huge amounts of customers prepare not to fail under scale. So whether you have a large amount of base driven things like Super Bowl, or a major sale, release of a new product like Samsung S8, iPhone 7, etc. Basically when you get a huge push.

Canned reports are always a challenge and a question with customers because customers want to see sexy reports. They want to be able to show something to the CIO. So I think the dashboards are one of the features I'd like to see most.

I think it's more of getting into the world where you've got tableau and dashboarding. I think that reporting needs to be a little bit more fancy, as people expect the sexier reporting. They don't expect just to have, "I ran a test. The test ran for this long." I think the consumer's expectations for what reporting looks like have changed a lot. You do an Excel report or a Word report versus, "No, it needs to be a very pretty dashboard."

The product itself, I think it's pretty good. I can't think of anything off the top of my head.

It's great. I don't have a problem with stability at all, as long as you have it scaled properly and you have sufficient hardware in place. If you're running it all on a VM, you're going to have a problem, but if you run it with the proper infrastructure, it's a very solid product.

The nature of Performance Center is scalable, so you have the application server and then, when you need to have more generators to generate more load, you spin those up pretty quickly. You can use cloud-based generators as well, so that's a huge plus.

It's been a long time since I needed to use tech support. Normally, as a consultant, I am the tech support, so I don't typically have to use tech support. But when I have, I normally am able to get quickly to either R&D-level or a level-two support because it's a real problem with the product, not necessarily just, "I can't figure this out."

I help customers with this process all the time. I'm usually advising them on what, why, when, what the feature benefits are.

Unfortunately, as is human nature, customers decide that they need Performance Center because they've had a disaster. Hopefully not a horrible disaster, but they've had some kind of case where they released a product and it didn't scale. They didn't plan for their own success. A classic example is HealthCare.gov. Politics aside, when you've got the entire American population ready to enroll for healthcare and it tanks, it's a very bad experience for everyone. And that's not an uncommon occurrence across the board.

So then they realize, "Oh, well, we better do performance testing," and then they realize they didn't plan for that in the project lifecycle, so now they need to come and talk to Micro Focus about standing that up, or to talk to a partner at Micro Focus about how to do that for them.

There was a reason, for the longest time, that it had one of the largest market shares of any type of solution in the world, and now that Micro Focus has Silk and the LoadRunner/Performance Center product, they've got that market cornered.

I have set up many, many instances of Performance Center. Recently, it's much more straightforward. A long time ago it was very complex. But it's pretty straightforward. You set up the application center, you set up your generators, you set up your controllers, database.

When selecting a vendor I would judge them on the criteria that I have myself: they've got to have experience, they've got to have done the testing on the solutions that they've worked on. I think seniority is good too, little gray hairs don't hurt anything.

Regarding advice to others, invest in training. Invest in mentoring. Invest in experienced people that have done the job before. Don't go into it thinking that you're going to open the box, get it out, and it's going to be perfect. It's a complicated tool for a reason. You don't want someone operating on you who says, "Well, I read a book on brain surgery." It's complicated for a reason.

We still have a legacy LoadRunner, but we are moving towards this tool. For our web applications, we are moving towards Performance Center, because it is more GUI and user-friendly, and with all the latest technologies. It's pretty easy to learn how to use.

I don't know if it is available or it depends on the licenses, but reports from Performance Center need to be improved. If we can improve the reporting structure and we can view the report on a smartphone, that would be huge.

We always have to get connected if there is a report. It's doesn't need to be the full tool, but just like how HPE Discover has an app, if we can have an app just for reports, that would help.

I've been using it for about six to eight months.

We are still learning and evaluating it.

All of our applications are not yet on Performance Center.

It was even before I joined. We were using LoadRunner. LoadRunner is a leading load testing tool in the market. Whenever a customer or anyone looks for a tool, the first thing which comes to mind is HP. We have seen StormRunner and we are here at HPE Discover 2016 to check out tools. We will see how they scale, and probably will make a decision soon.

Nothing wrong with it, but I have to learn more about it to see if it's going to match our needs. Performance Center has everything which LoadRunner offers, plus additional things.

Performance Center is actually run by a performance engineering group so in every release you have a performance engineering phase that runs at the same time as UAT. That's pretty much used to ensure that we're going to hit our production stability, scalability, etc., when a product finally goes in.

That is used in a phase gate, it's not continually run. That's one of the things we're eventually going to get to, that you continually run it.

Our performance engineering team has feedback on Performance Center, so they don't utilize as many of the features as they should. They have it, part of it could be training, but they're not using it as much as they should. They use it, but they're not taking full advantage of all the features in it, meaning they are not combining it with UFT, etc.

They just run it to run the virtual users and then load and stress test, and that's pretty much what they're doing. They're not really taking advantage of the whole stack like we do. That's another group as well.

Stable in the fact that it does what it does, it does it well. Why they use other tools, that's where I would say there's some poly-functionality that has to be improved in the product. I'd have to specifically interrogate them on what they're not getting out of it. It's stable, it's up, and it runs, but if you want to look at is it as highly leveraged as it could be, it's not so much.

We're already at enterprise scale, so it's used across the enterprise. I would say that we're at that point.

Originally we were using LoadRunner, and then we had to upgrade to Performance Center because with LoadRunner we didn't maintain currency with the license. That's another group that manages that. They just kind of were using it as it is. Then when we upgraded to version 11, we had compatibility issues and we had to go to Performance Center. I think they just didn't get used to it. I don't know exactly what they are or are not using in that stack, but part of it is we were forced to upgrade.

I have an entire team, so I'm a director and I have an entire tools team that does that. I did get involved in the planning and the strategy of how we're going to do it. My team said that first installation is relatively easy. When we go to upgrade and migrate, that's where there's pain.

We have to use other products like Selenium and a lot of custom scripting, et cetera, but that's the nature of the business. That ends up happening everywhere.

Have a well-defined process, have a strong reporting structure, meaning in your process you want a lot of measurability. If you define your output, the reports and the questions you need to answer from what you're doing, which your process should be managing for you. In our company, we are very specific about what our executives and stakeholders want.

We have a very well-defined set of measurements that we have to take. We then put a process designed to ensure those measurements are always taken. That then allows you to deal with your outputs and your reporting structure, which then allows you to properly architect your tooling. The technology is very flexible. You have to decide as a client area how you really want to use it and that's going to start with what your business needs are the values that you're trying to get out of it.

That's the biggest advice that I have, it's not even on the technology. The technology will do great things for you if you have a plan and a structure and you know what you want it to do for you. Half the time they don't know, they want the tool to do it for them and it's the other way around. So that's what I advise people to do.

Think about it, have a vision, have a plan, tie that to outcomes, and measure those outcomes. If you're answering the right questions and asking the right questions, your technology will really enable you. You've got to look at it from that standpoint.

It's the most flexible and best solution for our people who are both offshore and onshore because of the reporting capabilities. When we hire new employees, we look for people who have used this tool for testing experience.

It's allowed us to really focus on load testing and to get some results that we've reported to our business managers.

Another thing we really like is that it takes advantage of our monitoring tools when we're performance testing. They work well together.

The migration from LoadRunner to Performance Center was not as smooth as I would have liked. Some of it was not the tool but us, but it still could have been smoother and taken a little bit less time.

We started off with LoadRunner and we've been using that for probably a year, and then we just recently converted over to the Performance Center, which is their staff solution, in the last year.

We've had no issues with deployment, other than the initial migration.

It's stable now.

For situations in which we've had to scale up a little bit, they've been there to help us. We didn't have any trouble scaling.

I have not been as impressed as with their other products. They do respond, they do have the knowledge to do this but it hasn't been the same as what I've seen with other products. A little more challenging.

We were challenged in out setup, making it more complex, because we didn't us HP cloud and were using AWS instead. Some of the connections weren't set up properly.

Make sure you figure out what you want before you buy. Determine how many and what types of licenses you need.

Although there may be other products out there that do an amazing job, they don't necessarily have the skill set in the industry that makes it worth me going out there and looking at them.

During implementation, make sure you're talking about a cloud solution. You may be implementing in multiple areas, so planning is very important.

We can centralize and execute various performance tests for business units across the globe.

We are able to run performance tests for Asia, EU, and the US from a centralized controller where each team can come together in their respective time zones, build and run the tests, as a test manager can review, and suggest various improvements along with infrastructure teams.

It's expensive, so could be cheaper. Also, it should have third-party tool integration.

I've used it for two years.

No issues encountered.

No issues encountered.

No issues encountered.

We switched due to common licences.

It was straightforward.

It's expensive, but useful if the usage is across the globe; others are pay-as-you-go type license which can be used to minimize the costs.

The primary use case for clients is that they often have large application development teams and application development projects that they needed to scale. So, for instance, if they were developing a new banking website and they needed to check that the application that sits on that website was scalable from a few hundred concurrent users to many thousands of users, they could test the load response using LoadRunner.  

That is what LoadRunner does, it does the performance testing and measures load-bearing response.  

I think, for me, the most valuable part of the product is the way you can do the basic testing. You can create the test script and then simulate thousands of users very, very easily. Instead of having to have lots-and-lots of systems that would emulate users, you just needed a couple to emulate tens of thousands of users. So the scalability of LoadRunner while it was testing scalability is really valuable.  

To improve the product, I think the integrations could be a little bit more slick. It does handle a lot of great integrations, but then some of them can be a little bit clunky to implement. The integration with third-party tools needs to be stepped up a little bit.  

As far as other things that need to be added, it has changed quite a lot recently, and I have not had vast amounts of experience with the latest version. So I am afraid it would not be fair for me to go further in expounding on that question. Things that I talk about may already have been included.  

I was a presales consultant and so I was kind of a technical consultant as well. I was working with the solution end-to-end for about seven years. My main focus was not LoadRunner for the entire time, but I gained knowledge of LoadRunner and I gave presentations about it. It has been about eight to twelve months since I last did anything serious with it. However, I am still familiar with the product.  

The scalability is really very good. In some ways, it is the purpose of the product: testing by use of scaling loads.  

The initial setup is pretty straightforward. But I have got to say, having worked on-and-off with LoadRunner over a period of time, I knew kind of instinctively how to set it up after a while. In other words, in my case, I would say it was simple. On the other hand, I think it the first time I tried to set it up it was a nightmare. After that, it was easy because I learned a lot about it. If I had to score it out of ten for initial setup with ten being the best, it would probably be seven-out-of-ten. It is not really going to be super easy for first-timers to deploy.  

The deployment could take quite a while, even when I was used to doing it. Getting the software installed and running is pretty quick and that is not a problem. But creating the projects and creating the test scripts can take a little while. To get up and running and doing stuff within it, it is probably just around a week. Doing it professionally with the integrations and with all the correct testing scripts, it can take a month and more. It really all depends on the purpose and how you want to use it.  

My advice to people considering LoadRunner is that if you are going to use the product, use it as part of your everyday application development lifecycle. Do not just use it right at the end, because it gives you some great insights during the development phase as well as at the end. You will end up writing cleaner application code with it. So bake the use of LoadRunner into your full application life cycle.  

On a scale from one to ten where one is the worst and ten is the best, I would rate this solution as between and eight and nine-out-of-ten. I could be slightly biased, having worked for the company that sells it. But it is a very good, professional solution. With the latest updates, it is very comprehensive and one of the best products of the sort. Let's say nine-out-of-ten because there is always room for improvement.