We changed our name from IT Central Station: Here's why
Get our free report covering Postman, Broadcom, Micro Focus, and other competitors of Apache JMeter. Updated: January 2022.
565,689 professionals have used our research since 2012.

Read reviews of Apache JMeter alternatives and competitors

Michael Daniel
Sr. Technical Test Analyst at a educational organization with 1,001-5,000 employees
Real User
Top 20
Enabled us to eliminate load generators, and automatically triggers and produces reports
Pros and Cons
  • "The fact that the solution supports multiple protocols such as open source, VuGen, TruWeb, TruClient, and SAP is very important because these protocols help us to concentrate on what is really needed to produce performance tests. If something is not supported, you have to use other tools or find other ways of assimilating loads."
  • "We are trying to put it into a complete CI/CD pipeline, but there are still some challenges when you try to run it through different protocols. The challenges are around how you can containerize applications. There are some limitations to some protocols, such as desktop. And when it comes to database testing, there are some things that we can't do through CI/CD."

What is our primary use case?

I have been using it to analyze the performance of various enterprise systems like SAP, web applications, Oracle Labs, and mobile app applications as well. The objective of the performance testing is to assess the system and whether it can withstand an actual load.

How has it helped my organization?

One of the main benefits is that we don't need to worry about load generators. Before we had Loadrunner Cloud, we had 10 to 20 load generators, and we needed to maintain them. There were always upgrades. We would also have to configure the load generators when we were ready to test, and at times there was stuff to clean up. None of that is necessary anymore. We can concentrate most of our time on creating scripts and configuring Loadrunner based on our objectives. It's very easy for us. We don't need to worry about what happens in the backend or how the load is distributed.

A second advantage is the reporting system. Once a test is done, even if it's in continuous integration and continuous delivery, it automatically triggers and produces reports. We can send the reports to multiple dev teams whose developers are expecting the reports.

Another advantage of the solution is seen compared to others we have used, like JMeter, and IBM solutions. LoadRunner has reduced a lot of the time involved in the scripting cycle because of TruClient. We only need to record one time and we can configure it accordingly to create various scenarios. It has reduced scripting time by about 50 to 60 percent.

When we do mobile applications, every build can be tested with LoadRunner Cloud. Once the developer checks into a particular branch, we call it a mod branch, the performance test is already integrated into it. They can trigger it through their continuous pipeline. For mobile development platforms, it's very crucial because notifications and other things need to be tested before we send to prod. Notifications are very load-sensitive because they go to every user—it could be 10,000 or 20,000 users—and we previously had issues with this. Using LoadRunner Developer and continuous integration has helped us.

What is most valuable?

Recently I have found the TruClient tool to be very useful. My team is involved in a lot of performance testing of applications including mobile platforms and different types of web browsers. In those cases, LoadRunner TruClient is very crucial to testing the performance. TruClient is one of the best features, one I use every day to create load scenarios.

The fact that the solution supports multiple protocols such as open source, VuGen, TruWeb, TruClient, and SAP is very important because these protocols help us to concentrate on what is really needed to produce performance tests. If something is not supported, you have to use other tools or find other ways of assimilating loads. For example, when you are trying to create loads for web applications, if it's not TruClient, you need to find and analyze every call: What the HTTP request is, and what the other kinds of requests are that we need to call, and then correlate all the correlation identifiers. All of that is taken care of by TruClient and other protocols. That is actually very beneficial across all the platforms. For example, a SAP application might be tested for mobile. We can use a combination of the SAP protocol and TruClient protocol. The combinations are very helpful for performance testing.

LoadRunner Cloud gives you a lot of options, even for multi-browser or multi-device testing. It has been the main tool that can do everything; complete end-to-end performance testing.

The support for large-scale testing is also an important feature in our operations. We have thousands of users and it provides the best solution. You can have an unlimited number of users, although you would need to pay for them, but that's a different story. In our organization, the maximum number of users is currently about 20,000 to 30,000. It's a one-stop solution. I can configure my load on the cloud environment and have 30,000 virtual users on the cloud. I don't need to create the infrastructure locally and I don't have to maintain it. Everything is taken care of by the solution.

In addition, its ability to run unlimited concurrent tests without worrying about hardware availability helps eliminate hardware dependency. You don't need to have the load generators on your network. You don't need to maintain those systems and you don't need to have that kind of network capability. If you're testing on-prem, but you don't have the network capacity to scale up to millions of users, LoadRunner Cloud enables you to create virtual networks and use the cloud to generate those kinds of loads. You can then analyze what the impact will be to your system when you have millions of users. LoadRunner Cloud is the best way to do that.

What needs improvement?

When it comes to the CI pipeline, there were some limitations initially, but the latest version of LoadRunner is very helpful. They can integrate into the CI/CD pipeline. We are trying to put it into a complete CI/CD pipeline, but there are still some challenges when you try to run it through different protocols. The challenges are around how you can containerize applications. There are some limitations to some protocols, such as desktop. And when it comes to database testing, there are some things that we can't do through CI/CD.

For CI/CD, the previous versions may not be the right ones, but the latest version is definitely a step ahead. We are aiming for 100 percent, but we have achieved around 60 to 70 percent in CI/CD. Still, it's very good to have that capability.

Also, it would be helpful if Loadrunner Cloud had the same kind of enterprise environment where we had multiple models and options while creating the load profile. Not all the options are available in the LoadRunner Cloud. If they could be added, it would be good.

For how long have I used the solution?

I've been using a LoadRunner solution for more than 10 years as part of my performance testing.

What do I think about the scalability of the solution?

The efficiency of the scalability is a 10 out of 10. I have used multiple tools and LoadRunner is the best one in terms of efficiency. When it comes to cost-effectiveness of the scalability, I would give it an eight out of 10. Even though it's cloud, and you can have thousands of users, we are paying in the tens of thousands. It's not so cost-effective for a university like ours. We still have to justify why we need to spend so much money every financial year.

How are customer service and technical support?

I have used Micro Focus technical support a couple of times and they have been very supportive. They're good.

How was the initial setup?

The only problem with setup is that there might be some problems with the firewalls, configuring SSH and other things. We were initially using Micro Focus SiteScope but we had some issues getting all the metrics. With New Relic and Dynatrace, we stopped using SiteScope. Other than that, we didn't have any issues. It's pretty straightforward. You install LoadRunner, configure your virtual users, and create the environment.

Our initial deployment of the on-prem solution took a week to get everything done, including setting up the firewall, configuration files, and the protocols. The migration to Loadrunner Cloud is nothing. You can start triggering whatever code you had on-prem in the cloud. There really isn't any migration involved. It's pretty straightforward.

What was our ROI?

We have definitely seen return on our investment with Loadrunner Cloud. As I noted, we used to have many load generators and they are no longer used. That is saving us about $30,000. And Loadrunner costs 50 percent of what it used to cost us to run the same test in the cloud. We don't need a Windows Server license anymore or networking capabilities specifically for testing. Those are the kinds of savings we have seen from moving to the cloud.

Also, in the past, we used to write code. But with TruClient, while you need a performance tester, you don't need a programmer to write scripting. If you know the system, and if you know the objectives of performance testing, you can do the performance tests. No programming skills are needed. That also gives us leverage. We can use someone with performance testing capabilities, even though he might not be the best programmer. That has also reduced our costs by $10,000 a year.

Which other solutions did I evaluate?

I have extensively used JMeter as a performance testing tool. JMeter is free and also enables codeless scripting. Even without coding you can try running scripts. But where there is no comparison between JMeter and LoadRunner is when it comes to distributing load. LoadRunner stands out. With JMeter, it's very difficult to distribute the load. 

When it comes to creating reports, Loadrunner is best. You will spend most of your time analyzing what's happened with the test, analyzing bottlenecks and pain points with the performance parameters. But in JMeter, you have to manually collect everything: collate the results and produce the reports. Then you need to do a detailed analysis to find the bottlenecks and resource patterns. It is very difficult, but it's free. If you have the skill set and the time, you can use JMeter. But if you are time-constrained, and you want to actually concentrate on performance testing, use LoadRunner.

LoadRunner Cloud provides application performance and management tools to an extent, but not to the extent of New Relic or Splunk. We predominantly use New Relic to monitor application performance and in some cases we use Dynatrace as well. But LoadRunner Cloud doesn't have complete application performance monitoring metrics.

What other advice do I have?

LoadRunner's Developer integration enables developers to script and run tests without leaving the developer ecosystem. It gives a complete IDE where you can develop the code and add your script. For example, if you are using a Java platform to code, and it has all the libraries and the IDE, you can integrate your load testing into your development. For us, because we don't have a single development cycle or ecosystem—we constantly move to various methods—that's where the IDE has limitations. It supports certain languages but it doesn't support everything. If we are using Go, for example, we might need to add certain libraries, so that's where it isn't helpful. But if you are purely on Java or core platforms, it will definitely help.

The Developer integration enables developers to add performance testing to their day-to-day tasks, but shift-left depends on your company's development strategy. If your whole culture supports shift-left, so that your quality assurance is embedded from the start of the development cycle, then shift-left works out. But in our case, we use a lot of packaged applications like Workday or SAP where we don't have much opportunity to work with their code. We do a lot of configurations and integrations. In that case, shift-left doesn't work as well. But whether you follow shift-left or shift-right, Loadrunner Cloud works.

For non-functional testing, LoadRunner is the best tool. I can recommend it to other people, to create specific tests from the smallest load to the highest level of load. I don't think any other commercial tool has that capability to create load performance testing. There is no other tool that gives this kind of experience for a load-testing professional. From end-to-end, starting from creating the load scenarios, to running them, and then reporting, LoadRunner is the best tool. You save a lot of time and, with LoadRunner Cloud, you are saving a lot of money. Go for it.

Disclosure: PeerSpot contacted the reviewer to collect the review and to validate authenticity. The reviewer was referred by the vendor, but the review is not subject to editing or approval by the vendor.
Flag as inappropriate
Software Engineer at Birlasoft IndiaLtd.
Vendor
Top 10
Facilitates load testing, operational volume testing, and spike testing for the APIs
Pros and Cons
  • "It stores good reports, as in, improved reports if compared with the SoapUI. It also has in-built security. You just need to switch and check the security testing. My team has never used it, but I know ReadyAPI provides those facilities as well."
  • "This is an area for improvement with the tool. We unnecessarily use JMeter for some website testing, which we would like to avoid by introducing this tool for API and load testing because it provides load testing features."

What is our primary use case?

I'm a test lead person so we have multiple teams who work under me and they use WebView API, other members use LoadRunner, and some members just use other automation techniques. They use Selenium and Java automation.

What is most valuable?

It provides many features. It facilitates load testing, operational volume testing, spike testing for the APIs, and the API is very fast 

It stores good reports, as in, improved reports if compared with SoapUI. It also has in-built security. You just need to switch and check the security testing. My team has never used it, but I know ReadyAPI provides those facilities as well.

What needs improvement?

I have used JMeter but for load testing for local website testing. I'm not sure whether this particular tool supports all the websites connected to testing as well.

This is an area for improvement with the tool. We unnecessarily use JMeter for some website testing, which we would like to avoid by introducing this tool for API and load testing because it provides load testing features.

It supports the APIs but not for the websites. It would be good if that feature is integrated into the solution. 

For how long have I used the solution?

I have been using LoadUI for one and a half years. 

What do I think about the stability of the solution?

It's a stable product. 

How are customer service and technical support?

I am not the one who handles their technical support. My team has not raised any concerns about the support team. If they did, they would raise the concern. 

They have good documentation and blogs on the internet. 

How was the initial setup?

The initial setup is not very complex. For testers, they have to create their test cases and scenarios within ReadyAPI, and there they simply go for the load testing. It's automatically created. 

What's my experience with pricing, setup cost, and licensing?

There should be a free testing version that should be offered for free for students. Not free for enterprises but for the student level so that they can get transparency and can become familiar with it. 

SoapUI partnered with SmartBear who offers this for free under their platform. 

There should be a student version of this platform in which a student can get their hands dirty in order to get some expertise.

For an enterprise, the price is fine. Some features are at a very high level, especially with reporting, they should just disable those right away for the student level.

One free month is not enough to fully evaluate the solution. I would need more time to explore the features. They should offer a free student version in order to help expand their community. 

What other advice do I have?

I would rate it a nine out of ten. It is a very good product. 

It's a very good and handy product. There are so many features. It's getting more popular day by day. 

In the next release, they should include website testing, especially load testing. I don't think it supports it right now.

I'm comparing it with JMeter. JMeter also supports load testing themselves. If I'm paying the price for using the software, it would be better if I could use those features where I can discard the open-source ones.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
RajaRao
Associate at Tech Mahindra Limited
Real User
Top 5Leaderboard
Offers excellent support and the pricing is reasonable but it has a few glitches
Pros and Cons
  • "The stability is okay."
  • "LoadRunner offers a full protocol, whereas, with this product, only a few of the protocols are supported - not all."

What is most valuable?

The support is very good. They have excellent community support.

The pricing is pretty good.

The stability is okay. 

What needs improvement?

There are still some glitches that they need to improve. We have given support feedback as well when we have some issues. They're very responsive and they do work to fix and improve issues. 

LoadRunner offers a full protocol, whereas, with this product, only a few of the protocols are supported - not all.

For how long have I used the solution?

I've used the solution for five years.

What do I think about the stability of the solution?

We do occasionally deal with glitches. The previous versions are very stable. The newer versions have some glitches and sometimes it hangs, however, for the most part, it's okay.

There are issues around some protocol settings, like when we use encryption. When a person uses the banking application, the user and password are encrypted. We have issues with the encryption protocol. Lately, some settings may have changed and it seems to be working.

What do I think about the scalability of the solution?

While it's a good solution, I cannot speak to the scalability.

How are customer service and support?

The community support is good. Whenever we have an issue, they provide support to resolve it. If we need information about features, they help. Support is very good.

Which solution did I use previously and why did I switch?

We use the solution as projects demand. We are a consulting company. Whatever the client has already purchased, we will use that. We work with all kinds of tools.

What was our ROI?

I don't have any details related to ROI.

What's my experience with pricing, setup cost, and licensing?

I'm not exactly sure what the pricing is as it differs from client to client.

That said, LoadRunner is expensive and NeoLoad is less expensive whereas JMeter is free.

Which other solutions did I evaluate?

I'm familiar, for example, with LoadRunner. When you compare this with Load Runner, Load Runner is the best tool as it, number one, is very user-friendly when compared with the NeoLoad. It also supports the protocols. It is very mature, so people are very comfortable with it. The monitoring is good, and it has a lot of good features.

NeoLoad will not support all the protocols and it's not as user-friendly as LoadRunner. I use LoadRunner more than NeoLoad, and therefore am more comfortable with it.

I also am familiar with JMeter, which is an open-source option and therefore a free tool. In comparison, NeoLoad is a big tool and JMeter is quite small. With NeoLoad, you can get all kinds of reports, flexible reports and you can customize the reports as well. We can drill down on the reports if we want. That's not possible in JMeter.

What other advice do I have?

NeoLoad is a good tool for banking or telecom applications. When compared with the LoadRunner, NeoLoad will support almost all necessary features including reporting. It is a good tool.

Overall, I would rate the solution at a six out of ten.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
Flag as inappropriate
Balamurugan Arjunan
Manager at a financial services firm with 1,001-5,000 employees
Real User
Top 5Leaderboard
Easy to maintain and the Java support is good, but the performance needs to be improved
Pros and Cons
  • "The most valuable feature is the support for Java, where we can quickly code what we need."
  • "The performance of the tool needs to improve."

What is most valuable?

The most valuable feature is the support for Java, where we can quickly code what we need.

it is quite easy to maintain.

The integration with other tools is good.

What needs improvement?

The performance of the tool needs to improve. Part of the problem is that this is a cloud-based deployment in a firewall setting, and our application will also be online. This means that there is an inside firewall and an outside firewall, so it is a different IP between connecting and executing it. Sometimes we feel that it is this aspect of the environment that is delaying the responses, rather than the actual application.

The reporting needs to be improved. They should include some additional reporting components because compared to other tools, like Sovereign, it does less.

Integration with monitoring tools, such as SolarWinds, needs to be improved. We can get performance testing information into our monitoring tools, but we can't map information about monitoring to the performance testing.

What do I think about the stability of the solution?

We have had no trouble with crashes, although we have to keep monitoring the loads because every time we are planning to increase one, you have to notify them and then they will increase it. If we exceed the limit then it can get stuck.

What do I think about the scalability of the solution?

Across multiple users, this is a scalable solution. However, across multiple platforms, we have to go in a different direction for that.

How are customer service and technical support?

I think that technical support from the vendor is good. I would rate their service a four out of five.

Which solution did I use previously and why did I switch?

I also have experience with Sovereign and JMeter. The reporting capabilities in these tools are better.

How was the initial setup?

The initial setup is complex.

What other advice do I have?

All of the features in this product are good.

My advice for anybody who is considering Tricentis Flood is that it is good for end-customer facing applications. However, it is purely result-oriented, as opposed to reporting-oriented.  It is good from a user's standpoint, but not fro ma senior management perspective.

I would rate this solution a seven out of ten.

Which deployment model are you using for this solution?

Private Cloud
Disclosure: I am a real user, and this review is based on my own experience and opinions.
QA Engineer at SSCGI
Real User
Top 20
Has good recording features but it should have the ability to customize reports
Pros and Cons
  • "I really like the recording because when I use the JMeter the scripting a lot of recording it takes me a lot of time to get used to. The BlazeMeter the recording is quick."
  • "In terms of improvement, I would like to have the ability to customize reports."

What is our primary use case?

Our primary use case is to create processes for performance. I created a one time function and implemented it for 1,000 users. 

What is most valuable?

I really like the recording because, with JMeter, the recording takes me a lot of time to get used to. With, BlazeMeter the recording is quick.

What needs improvement?

In terms of improvement, I would like it to have the ability to customize reports. 

For how long have I used the solution?

We have been using BlazeMeter for around two months. 

What do I think about the stability of the solution?

It is stable. 

What do I think about the scalability of the solution?

I implemented it for 1,000 users. 

How was the initial setup?

The initial setup was straightforward. I deployed it myself. Because I'm new to BlazeMeter it took two days to deploy because I needed to study it first also.

I deployed and also maintain it myself. 

Which other solutions did I evaluate?

I also evaluated JMeter and LoadRunner but I prefer BlazeMeter because of the speed and because it's easier to use. 

What other advice do I have?

I would rate it a seven out of ten. I would recommend it. 

I would like for it to be Windo-based. It should have the ability to be implemented into Windows applications. 

Which deployment model are you using for this solution?

Public Cloud
Disclosure: I am a real user, and this review is based on my own experience and opinions.
Get our free report covering Postman, Broadcom, Micro Focus, and other competitors of Apache JMeter. Updated: January 2022.
565,689 professionals have used our research since 2012.