OpenText Enterprise Performance Engineering (LoadRunner Enterprise) Reviews

Performance of load test applications, reliably, and with good reporting.

We have high confidence that the results are reliable. It's helped catch multiple bugs before they've gone to production. And we're able to more confidently release software into production, just more confidently release code.

I'd like to see better reporting, ultimately, and analysis. I'd like to see the analysis more accurately pin-point where these issues are; a little more feedback immediately in terms of the anomalies that we're seeing during the testing, so we can get alerted much quicker, instead of blowing a whole bunch of our day to test data during a test.

The reporting could be a lot better. It's cumbersome. The VuGen takes a long time to load up, and edit and execute things off those. So a lot of the basic infrastructure.

We've had a few crashes, but overall they seem to be more of the aging architecture, the hardware's not as reliable. A few software bugs in there, which we have to work around. But overall, it's been pretty reliable when you think about how much load goes through it and the number of people using it.

We've scaled it up to, well, to our limits at least. A good 1500 users and some 10 million transactions, or something like, that per hour. It's generally handled that pretty well. We're looking forward to moving into StormRunner next.

The tech support seems to be pretty good. There have been a few problems that they haven't resolved, but they've led me in the right direction to eventually figure it out on my own. It's very difficult for them to understand all the wide range of applications we use. So I'm really impressed with how ultimately knowledgeable they are, given that they don't actually use all these applications that are out there.

I wasn't involved in the initial setup but I've been involved in all the upgrades. They've done it multiple times. Each upgrade seems to be adding a lot more features. It seems like the pace of development has picked up in the last couple years especially, and we've been impressed with the direction that the product's going.

The upgrades actually were fairly complex for my team. The last one, they were actually giving up for a while. It turned out that there were some database versions that were incompatible. Eventually, once we were able to get connected with the support, we were able to get most of it worked out.

There was a big upgrade with the 12.5 version that was a little slower. But everything else has been very smooth since then. Still, it's difficult. Everybody's come to count on that software being at a certain version.

The most important criteria when selecting a vendor are, I suppose, support, and the reliability in the industry.

They have a good road map for moving forward. It's going to be integrating with a lot more other products that the DevOps world is starting to push upon us.

It's not quite perfect. It's still the best in the market. It holds up to a lot of the scrutiny that the developers constantly throw at us.

There have been a few issues, especially around the scripting and the IDEs, handling of some of the protocols, that still don't perfectly match with what we have in production. But overall, we're able to pretty well defend it. You can almost always reliably pull up the application, and the response times seem to match what we have coming out of LoadRunner.

I would definitely advise looking at Performance Center, it's still the best in the breed. Just make sure that you have a good team in place that can implement it.

One of the things in the airline business, we see that our number of users varies on a day to day basis, from season to season. So, from an airline business standpoint, we are looking at scalability as one the major things and how can we adapt the solution in an agile fashion. If we want to ramp up the "Our Views" account, from, let's say 10,000 to 50,000, how can we do that? That kind of scalability is the main key thing we are looking at.

One of the key things we use is simulating the actual user experience on the log. We have a huge set of applications from front end to the back end systems. How do we integrate all these systems and how do we simulate the real time user behavior? That's where we see a key value.

One of the things we were looking for is more of a DevOps support, like BlazeMeter has. It would be an ideal scenario to incorporate those kinds of features. I know there are some open software products which have that but it would be ideal to see those features in the product.

As of now, it's working great for us, it's excellent. We don't have any issues. That's one of the reasons we are pushing forward to version 12, to incorporate the other protocols, which version 12 comes up with.

We are at version 11.5 and we are in the process of upgrading it to version 12. We are pretty happy with the solution we have.

We do have a dedicated team. They work with our tech support and with their tech support in terms of the installations and stability of the product and usability. All those issues, they take it up with tech support.

Tech support is pretty excellent. We are getting pretty good responses back from tech support and, as of now, we are happy. We do have a contact too, from the United Airlines side too so I'm pretty happy with that.

Ours is more of a historical basis. We were on version 9, we moved to version 11, and we are right now at version 12. It's more for historical reasons rather that an impulse buy.

I didn't work on the installation of 11.5, but right now I'm working on version 12.

Internally, we have a lot of planning to do on our side, like a database upgrade, LGs, all that stuff but we are coordinating that with HPE and Micro Focus and making sure that our timelines and their timelines match. And we do have upgrade licenses, which should be pretty good to go.

I would say the relations between us and Micro Focus is straightforward because all we are looking at is basically license upgrades. On our side, it's more complex because we have to internally work with various teams to coordinate all this activity.

The most important criteria when selecting a vendor to work with are the product, how easy it is to use the product and, again, how scalable the product is and how it suits the needs of United Airlines. And, of course, the customer support, and how technical support deals.

Regarding advice to a colleague, it depends on the industry and what kind of problem they are trying to solve. If it is in the airline industry, I would definitely suggest to them, "Okay, this is a perfect product because of the number of protocols it supports," because we looked at other open source software and we couldn't find a product which matches Performance Center, which supports so many protocols. So, especially in the airline industry, we are using multiple protocols and we need that support. I would definitely recommend that.

For me, it's the fact that I can train my team on a single tool. It really is kind of our Swiss Army Knife in that it can handle so many different types of protocols and technologies, and I don't have to train my team on multiple tools to handle each of those.

It's really given us a lot of insight into, especially as we've moved into DevOps now, incorporating that into our CI lifecycle. We can start the performance testing earlier on, carry the same results through, and have it as part of our release cycle.

Mobility, I think, is the biggest for us right now. We're really getting into mobile app testing, native mobile app testing, so native Android and IOS devices.

As far as scalability goes, we're able to spin up load generators to handle an incredible size of load, so really we're just limited by the amount of hardware that we can spin up for it.

We use a third-party partner to do tech support and it's been fantastic with them.

We have, ourselves, gotten to Mircro Focus on a couple of issues that have been actually escalated all the way up to the team back in Israel, I think that's where they're located.

That was fantastic. We actually had them, they came out on site with us, some of the developers for Performance Center, and talked to us about some of the ways that we were using it. Some of the features that they thought customers no longer needed that we were actually telling them were still valuable to us, and they wound up bringing back a few features that were on the chopping block.

I wasn't directly involved in the decision to invest in Performance Center because we'd been using it since it was LoadRunner, and I've been using it for pretty close to 20 years now. So that was before my time.

I was involved in it. I'd say it was fairly straightforward for us because we used our support partner to help us out, guide us through some of the pitfalls of the initial setup. But he gave us a two-week timeframe that he said we'd be able to get it set up in, and we easily had it set up within a week.

When looking at vendors, support is the big one. Also, ease of training people on the tool, and just the variety of technology supported.

In terms of advice, I would say the biggest choice is that Performance Center really is for a very large enterprise. So I'd say evaluate if it's really what you need for the size of your organization. But if it is, it really can support pretty much any kind of technology you throw at it.

Performance Center is really our go-to tool for anything that we have to test. It's just our default tool for whatever technology we have.

High scalability. Web-based testing. The interface. If you're familiar with the days of using LoadRunner, when you had to have the 32-bit client, using a web-based client is fantastic. You can spin it up relatively quickly despite the fact that it's enterprise software. You can test a huge variety of applications, not just web-based systems, but SAP, Oracle, web services, pretty much anything out in the market place, but it's mobile-based testing.

In my current organization, I honestly don't know so much. But in my previous organization, when I was doing consulting, we helped huge amounts of customers prepare not to fail under scale. So whether you have a large amount of base driven things like Super Bowl, or a major sale, release of a new product like Samsung S8, iPhone 7, etc. Basically when you get a huge push.

Canned reports are always a challenge and a question with customers because customers want to see sexy reports. They want to be able to show something to the CIO. So I think the dashboards are one of the features I'd like to see most.

I think it's more of getting into the world where you've got tableau and dashboarding. I think that reporting needs to be a little bit more fancy, as people expect the sexier reporting. They don't expect just to have, "I ran a test. The test ran for this long." I think the consumer's expectations for what reporting looks like have changed a lot. You do an Excel report or a Word report versus, "No, it needs to be a very pretty dashboard."

The product itself, I think it's pretty good. I can't think of anything off the top of my head.

It's great. I don't have a problem with stability at all, as long as you have it scaled properly and you have sufficient hardware in place. If you're running it all on a VM, you're going to have a problem, but if you run it with the proper infrastructure, it's a very solid product.

The nature of Performance Center is scalable, so you have the application server and then, when you need to have more generators to generate more load, you spin those up pretty quickly. You can use cloud-based generators as well, so that's a huge plus.

It's been a long time since I needed to use tech support. Normally, as a consultant, I am the tech support, so I don't typically have to use tech support. But when I have, I normally am able to get quickly to either R&D-level or a level-two support because it's a real problem with the product, not necessarily just, "I can't figure this out."

I help customers with this process all the time. I'm usually advising them on what, why, when, what the feature benefits are.

Unfortunately, as is human nature, customers decide that they need Performance Center because they've had a disaster. Hopefully not a horrible disaster, but they've had some kind of case where they released a product and it didn't scale. They didn't plan for their own success. A classic example is HealthCare.gov. Politics aside, when you've got the entire American population ready to enroll for healthcare and it tanks, it's a very bad experience for everyone. And that's not an uncommon occurrence across the board.

So then they realize, "Oh, well, we better do performance testing," and then they realize they didn't plan for that in the project lifecycle, so now they need to come and talk to Micro Focus about standing that up, or to talk to a partner at Micro Focus about how to do that for them.

There was a reason, for the longest time, that it had one of the largest market shares of any type of solution in the world, and now that Micro Focus has Silk and the LoadRunner/Performance Center product, they've got that market cornered.

I have set up many, many instances of Performance Center. Recently, it's much more straightforward. A long time ago it was very complex. But it's pretty straightforward. You set up the application center, you set up your generators, you set up your controllers, database.

When selecting a vendor I would judge them on the criteria that I have myself: they've got to have experience, they've got to have done the testing on the solutions that they've worked on. I think seniority is good too, little gray hairs don't hurt anything.

Regarding advice to others, invest in training. Invest in mentoring. Invest in experienced people that have done the job before. Don't go into it thinking that you're going to open the box, get it out, and it's going to be perfect. It's a complicated tool for a reason. You don't want someone operating on you who says, "Well, I read a book on brain surgery." It's complicated for a reason.

• Enterprise-Level
• Centralized Platform
• Mobile
• Cloud
• CI
• Advanced Reporting
• Support

Hewlett Packard Enterprise is a recognized leader in the performance testing space, and with every new software release, it continues to solidify it's position as a front-runner and as an innovator.

More features to support testing in DevOps environment(s).

For the past six years.

No issues faced.

No.

No.

A nine out of 10.

An eight out of 10.

No, I have been using it for the past six years.

It gives us a run history of the tests we've planned and executed.

We are able to connect and run tests from different locations with this tool.

Storing the results of our tests requires the use of a lot of memory in our data center.

I've used it for three years.

No issues encountered.

No issues encountered.

No issues encountered.

I have mixed feelings about HP technical support. A few technicians know what they are doing, but the majority don’t.

No previous solution was used.

We implemented it with our in-house, but we used the help desk during the entire process of implementation. Their help desk needs to know more about their own tools.

No other options were looked at.

I like the tool.

Performance Center, in our company, is used for important applications where we have a lot of users, or special needs for performance that are important.

We have a central team that implements the scripts and executes the tests. It depends on the years of experience of the users. The investment goes down, then we have more issues. Then money is spent and then investment goes up. So it is a curve. Everything is going up, as it is in ALM. ALM is still a growing market.

With Performance Center, the version upgrade is easy. You just have to roll out the new patch or the new version. It is much easier. I'm not really the right person to say, because I run the environment. We have a specialized team that does development.

I’ve been using Performance Center since 2007.

Performance Center is more stable than ALM. We roll out a version, and I think it fits for our clients. If it is a very early version, then we have to implement a patch. Afterwards, it is quiet, hopefully, for at least one or two years.

For Performance Center, you have to add additional load generators, and then you can do more. I think it is a matter of the price, in terms of how many machines you can buy.

For such an experienced team as mine, who have been with the product for over ten years, sometimes working with technical support is not that easy. Support does not have our knowledge. It takes a while to train them in what our issues are and we have to connect to second or third level support.

The collaboration between us and HPE, especially over the past ten years, has been very good. This is the most important thing when looking at a vendor. For that reason, I try to bring in more HPE products, if needed.

ALM centralizes everything. It allows you to work out how well you are doing project-wise because you see the number of scripts done, the number of tests run, and whether you have mapped all your requirements to it. You can produce metrics there fairly easily for your line management and higher. So, overall, it is better than people using Excel spreadsheets.

Performance Center is good because it allows you to share resources, which wasn't happening with Load Runner. With Load Runner, everyone was very specific. I've just got these controllers and their mine and I might only be using them five percent of the time but I need them tomorrow. And I can’t allow anyone else to use them because it will disrupt my schedule.

With Performance Center, you start to get into position where people can say, "I need to run a test, how many assets are available? When can I plan to do it?"
It also provides discipline because you stop getting people saying, "We're ready to do performance testing," because they've got to schedule the test. They've got to use that period when they've scheduled it. If they don't we pull it back and somebody else can use it. You get a lot of people screaming they've lost their slot but what you've proven to them is that they're not ready for performance testing.

It's very good from that point of view. It focuses people's minds on actually using their time effectively.

I have been using ALM for eleven years. I used it when it was version 9.2 and continued with a lot of versions, all the way through.

We picked up Performance Center when we started introducing Load Runner. We kept that together until we realized we were had too many instances and it would be better strategically to go with Performance Center. I have been using it for ten years.

HPE Quality Center ALM is stable. It obviously has not got the attractiveness of Octane. As going forward, Octane probably does now take it to the next step.

The one thing I always said about ALM, and I'll say this to everybody. The worst thing about it is it did not have zero footprint on your PC. The amount of effort and the cost to upgrade to the next version, the amount of problems that it gave us in terms of trying to put a patch on, because it was particularly essential, was really bad for the business.

We had many different PC models out there on people's desks, so it wasn't just a case of patching or building a new MSI package for one PC. You had to do it for a whole range and then you had to deploy them at exactly the same time or somebody would find that they couldn't use Quality Center.

Octane, now being zero footprint, is probably going to be one of the biggest cost savings I see.

Performance Center seems to be stable. It's probably being utilized far more readily than, say, even Unified Functional Testing.
There are issues with it that mostly seem to be environmental. You'd be surprised how many people think they know about how to do performance testing and then they start using a server that's in one area of the UK to try and run a performance test on servers in another country.

I’m thinking, “why are you running such a transaction load across our network.” Whereas, they should really be in the local area. So, with Performance Center, most of the issues are more user-based. Technically, it seems to meet the task that you need it to do.

Without a doubt, both Performance Center and ALM are very scalable.

Sometimes support is good. Sometimes it's not so good. Sometimes you hit an issue and trying to get across the message of what the issue is, and then trying to get an answer back, can be a bit of a challenge sometimes. You hit an issue that everybody else has hit and it has a solution, then you get the response back. But in the majority of cases, the people that are on the case for you tend to do their best to try and answer what you've given them.

Adaptability is what I look for in a vendor. It tends to pull the others in. A good contact, ready to listen, to really know how to deliver what you want. Someone who can listen to what your problem is or what your challenge is that you need the tool to resolve. And if you're willing to adapt to that, then the tool might not be 100%, but it might make it's way there. If you're fixed in your ways, and say, "this is what our tool does, this is all that it's going to do," then to be honest, why continue?

The biggest issue is that ALM is a thick client and you can't patch it, because you've got hundreds and hundreds of PCs. Several different standards are on people's PCs. You can’t do it. You leave it until there's a big release and then you take a massive program to deliver it. Get rid of that thick client bit and you could patch on the server and it could be up and running the next day. Which is the neat bit about Octane.

The setup of Performance Center seems fairly reasonable. No real shakes about it. Obviously, you've got to have VuGen on the PC. It tends to have to be a meaty PC, but then you are running performance tests. My biggest challenge with Performance Center is having people who claim to do performance testing or know how do to performance testing and they're still wet behind the ears.
A good performance tester needs to have a good 18 months experience with them. They need to have done things with Performance Center. Delivered projects. They need to use SiteScope. They need to use analysis tools on that network. They need to know how to get the best value out of the tool. Somebody who's just come for the first time has probably done a week or two-week training course and says, "I know how to performance test."
They get results back and say, "We ran it for a 100 users and it failed." Well, okay, where did it fail? Where's the analysis that helps us fix the problem? And we didn't get that, which they would have done if they'd known to implement the additional bits like SiteScope against it.

So, with Performance Center, it's a skill issue for the people that are using it. Again, one of my guys says, “I’d like to see people be able to grade themselves in Performance Center or even in performance testing, "I'm at a Bronze level. I'm at a Silver level. I'm at a Gold level." Then you know how effective that person is going to be.