IT Central Station is now PeerSpot: Here's why
Buyer's Guide
Performance Testing Tools
June 2022
Get our free report covering Micro Focus, Apache, Perforce, and other competitors of Tricentis NeoLoad. Updated: June 2022.
610,229 professionals have used our research since 2012.

Read reviews of Tricentis NeoLoad alternatives and competitors

Allan Beumer - PeerSpot reviewer
Lead Performance Test Engineer at bol.com BV
Real User
Top 20
Enables us to performance test our warehouse management system for peak load
Pros and Cons
  • "The TruClient feature is the most valuable for us. An application with testing can only be scripted using TruClient, so it's part web-based, but it also has its own protocol combined with HTTP and HTML. So many other tools do not recognize this specific proprietary protocol. Using TruClient, we can still create scripts that cover everything that we need to cover."
  • "Reporting and analysis need improvement. Compared to the old school LoadRunner Windows application, the reporting and analysis are mediocre in LoadRunner Cloud."

What is our primary use case?

We use a warehouse management system, my company is like a Dutch Amazon. We use LoadRunner to performance test the warehouse management system for peak load. So if Valentine's Day, Christmas, or whatever comes up, we try to run all functional loads through LoadRunner Cloud and make sure the peak count can be handled by the WMS. We use it almost daily. We have peak periods where the testing will increase, but nowadays it's so busy that throughout the year, it's almost the same. It's the same high load every day and we're testing continuously.

The amount of load on the system wasn't that large a couple of years ago for us to test. So the business case for performance testing wasn't pointing to investing in it, but now we have such huge, increasing numbers and increasing load that it's worthwhile setting up a structured and automated performance test and to invest in something like LoadRunner Cloud.

We use the cloud. The controllers and the dashboard servers are in the Micro Focus cloud. But we use our own load generators, which are on-premise. So we use a hybrid setup.

How has it helped my organization?

Before LoadRunner, we weren't even testing performance and now we're performance testing. So we now do performance testing as part of the delivery of a new release, which is an improvement compared to a couple of years ago. 

What is most valuable?

The TruClient feature is the most valuable for us. An application with testing can only be scripted using TruClient, so it's part web-based, but it also has its own protocol combined with HTTP and HTML. So many other tools do not recognize this specific proprietary protocol. Using TruClient, we can still create scripts that cover everything that we need to cover.

It supports multiple protocols, which is important to us. We also use JMeter open source combined in LoadRunner Cloud. In our company, we also use open-source performance testing tools like JMeter, Gatling, and Locust. It's important to be able to embed those scripts as well into a LoadRunner-based performance test. So it would really help us to have a single place for all kinds of test scripts and the open-source ones.

It's also very important to us that LoadRunner allows full integration into the ACI pipeline. We were focusing on a shift left on not only functional testing but also performance testing. And we're trying to integrate the performance testing in our CI/CD pipelines currently. So the better it supports this, the better and easier it is for us.

What needs improvement?

Reporting and analysis need improvement. Compared to the old school LoadRunner Windows application, the reporting and analysis are mediocre in LoadRunner Cloud.

For how long have I used the solution?

I've been using LoadRunner since I started at this company one and a half years ago but the company has been using it for longer. 

What do I think about the stability of the solution?

We don't have any issues with stability. 

What do I think about the scalability of the solution?

When it comes to efficient and cost-effective scalability, it can be expensive. It is efficient, but also expensive. So the balance would then be just okay.

It's used intensively by the three performance engineers, daily. We have plans to have other teams also use LoadRunner Cloud. So we hope it will increase.

We do our own maintenance on the on-premise part and Micro Focus does the maintenance on the cloud part.

How are customer service and technical support?

We don't call customer support but we do online requests. We have put in service requests. They're helpful and the replies are quick and valuable. It helps us out.

What's my experience with pricing, setup cost, and licensing?

We make use of virtual user hours. We buy time in the LoadRunner Cloud. It costs around $80,000.

I don't think there are any additional costs to standard licensing.

Which other solutions did I evaluate?

We tried Neoload and looked at JMeter. We chose LoadRunner because that was the only performance testing tool that had something like TruClient.

What other advice do I have?

We're thinking about using LoadRunner Developer integration. We want to hand over the performance testing responsibility from our performance testing guilds to the actual feature teams, to the scrum teams, so they can do it themselves. The developers do their own performance testing in an early stage, and maybe that is a solution for that.

My advice would be to look at whether you need on-premise load generators or fully on the cloud. Invest in training. It can be complex. You need some training to get started.

I would rate LoadRunner an eight out of ten. 

Which deployment model are you using for this solution?

Hybrid Cloud
Disclosure: PeerSpot contacted the reviewer to collect the review and to validate authenticity. The reviewer was referred by the vendor, but the review is not subject to editing or approval by the vendor.
Quality Assurance Test Manager at a printing company with 5,001-10,000 employees
Real User
Top 20
Cost-effective and straightforward to deploy, but the reporting needs to be improved
Pros and Cons
  • "The recording and playback functionality is helpful."
  • "The reporting is not very good."

What is our primary use case?

When I was last using JMeter, we were simulating 200 concurrent users and evaluating performance based on transaction times. We were defining SLAs based on the results.

Essentially, we created load scenarios and testing different ones using different workload models.

What is most valuable?

The recording and playback functionality is helpful.

What needs improvement?

The reporting is not very good.

When we run with multiple users, it takes a lot of memory.

With respect to the recording and playback functionality, the auto-correlation parameterization is not easy and should be improved.

For how long have I used the solution?

I have been using Apache JMeter for about four years.

What do I think about the stability of the solution?

There are issues with stability when running with multiple users because it consumes a lot of memory.

What do I think about the scalability of the solution?

Scalability is fine, although it is important to remember that JMeter doesn't run on its own. It needs to work with load-generations such as BlazeMeter. LoadRunner is the same in that you need a cloud-based infrastructure to run it.

How are customer service and technical support?

There is no official support. There is a forum where you can ask questions and they respond to you, but the technical support that we have with LoadRunner or NeoLoad is not available.

Which solution did I use previously and why did I switch?

I have used many similar solutions in the past such as New Relic, AppDynamics, NeoLoad, and Micro Focus LoadRunner Enterprise.

JMeter is not as good as LoadRunner or NeoLoad, and it isn't as easy to use, but it's okay because there is no cost. LoadRunner is too expensive, in my opinion. NeoLoad is cheaper, although not significantly.

From what I have seen, many companies are adopting JMeter because it's free. Especially in Canada, using JMeter seems to be the new trend. Some companies are choosing NeoLoad over LoadRunner because it is easier, faster, and cheaper. Whatever they need to do can be completed quicker. The main problem with NeoLoad is that obtaining resources is harder.

Given all of the choices, my preference would be to implement NeoLoad.

How was the initial setup?

The initial setup is straightforward. I would not say that it is complex and if you already have the file downloaded then it will only take about half an hour to deploy.

What about the implementation team?

I took care of the deployment myself.

What's my experience with pricing, setup cost, and licensing?

I was using the free version of the software.

What other advice do I have?

My advice for anybody who is considering JMeter is to just install it and try it. Creating scripts is a different process when you compare it to LoadRunner or Neoload. There is different terminology compare to these two products, so if somebody has not used JMeter then it may seem difficult at first.

I would rate this solution a six out of ten.

Which deployment model are you using for this solution?

On-premises
Disclosure: I am a real user, and this review is based on my own experience and opinions.
Vipul-Kumar - PeerSpot reviewer
CTO at Intelli AI LLC
Real User
Top 20
Feature-rich, quick to create effective test cases, and timely support
Pros and Cons
  • "The most valuable feature is the ability to create performance test cases quickly and then execute them. It provides a lot of powerful features to do that very efficiently and effectively."
  • "The flexibility could be improved."

What is our primary use case?

I use this product primarily for performance testing in various software applications. There are web-related use cases, as well.

What is most valuable?

The most valuable feature is the ability to create performance test cases quickly and then execute them. It provides a lot of powerful features to do that very efficiently and effectively.

What needs improvement?

The flexibility could be improved. For example, there are some use cases where I prefer to use NeoLoad rather than LoadRunner because of the flexibility that it provides.

For how long have I used the solution?

I have been working with Micro Focus LoadRunner Professional for the past 10 to 15 years.

What do I think about the stability of the solution?

We have had no significant issues with stability or performance.

What do I think about the scalability of the solution?

It is easy to scale and configure. We have approximately 50 people that work with it.

How are customer service and support?

There are issues from time to time that we need help with. There have been specific problems that we reached out to the vendor about, and they have mostly been resolved on time.

I am satisfied with the support.

Which solution did I use previously and why did I switch?

I have used several other performance testing tools.

One of the other ones that I have used is NeoLoad, which is a more recent entrant in this space. It's a lot more flexible to use than some other products.

When it comes to setting up test cases, the process is slightly different in NeoLoad. It is a little more automated than LoadRunner.

NeoLoad is also more cost-effective than LoadRunner.

The suitability of one product over another depends on the use case. Both of them have their strengths. LoadRunner is a much more longstanding and mature product in the market.

How was the initial setup?

The initial setup is a complex process that is not too straightforward. However, with some expertise, it is not a big problem.

It took a couple of days for us to deploy.

What about the implementation team?

We did most of the implementation internally. I handled it myself.

What's my experience with pricing, setup cost, and licensing?

LoadRunner is more expensive than some competing products.

The pricing is a little too high. However, everything is mostly included in the initial license. It depends on how many nodes you want to set up. This is what determines some of the additional costs.

What other advice do I have?

My advice for anybody who is looking into implementing LoadRunner is to try it out. Start playing with the product and test it.

I would rate this solution an eight out of ten.

Which deployment model are you using for this solution?

Public Cloud
Disclosure: I am a real user, and this review is based on my own experience and opinions.
Flag as inappropriate
Global Delivery Head at a consultancy with 51-200 employees
Real User
Top 5Leaderboard
Lacking support and security, but has multifaceted capabilities
Pros and Cons
  • "BlazeMeter can be used for both API and performance testing, it is a multi-facility tool."
  • "If the solution had better support and the documentation was efficient it would do better in the market."

What is most valuable?

BlazeMeter can be used for both API and performance testing, it is a multi-facility tool.

What needs improvement?

If the solution had better support and the documentation was efficient it would do better in the market.

For how long have I used the solution?

I used BlazeMeter within the past 12 months.

What do I think about the stability of the solution?

BlazeMeter is a stable solution.

What do I think about the scalability of the solution?

We have not done a lot of scalability testing with BlazeMeter to determine the full capabilities of the solution.

Small enterprises would prefer to use BlazeMeter and those companies which have a favorable budget should choose Micro Focus LoadRunner or Tricentis NeoLoad.

How are customer service and support?

There is minimal support available for BlazeMeter. You have to use the internet and try to do some searches to find solutions to your problems.

Which solution did I use previously and why did I switch?

I have used many similar solutions similar to BlazeMeter, such as JMeter, which I believe BlazeMeter is a customized version of JMeter. It is difficult to compare BlazeMeter to either Micro Focus LoadRunner or Tricentis NeoLoad because it is more of an open-source solution. You do not receive the high level of support that you do with paid solutions. You have to refer to the internet to resolve issues.

How was the initial setup?

The difficulty level for the installation of BlazeMeter depends on how we want to implement it. It's easy to download and install, but some companies restrict the security side of it. The paid solutions are more secure compared to these open-source solutions.

What's my experience with pricing, setup cost, and licensing?

The solution is free and open source.

What other advice do I have?

BlazeMeter should release frequently updated demo versions of the solution to the market. They can let the internet community test it, such as crowdsourced testing, this would be good for them in the market.

I would recommend this solution to others if they have a small budget and do not have the option to buy another solution. Otherwise, Tricentis NeoLoad and Micro Focus LoadRunner are the best.

I rate BlazeMeter a four out of ten.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
Flag as inappropriate
Buyer's Guide
Performance Testing Tools
June 2022
Get our free report covering Micro Focus, Apache, Perforce, and other competitors of Tricentis NeoLoad. Updated: June 2022.
610,229 professionals have used our research since 2012.