Alona Tupchei - PeerSpot reviewer
Engineering manager I - Quality at a financial services firm with 1,001-5,000 employees
Real User
Top 5
Great for comprehensive performance testing and helps measure how our applications are working
Pros and Cons
  • "We find the ease of use and the reports and graphs available valuable."
  • "The solution needs more metrics for reporting."

What is our primary use case?

Our primary use case for the solution is comprehensive performance testing. It measures how our applications are doing overall with a lot of users. For example, if we want to simulate 100 users doing transactions simultaneously, we want to see if our app handles it well and how fast it responds. In addition, we can utilize the solution to identify time spent on operations.

What is most valuable?

We find the ease of use and the reports and graphs available valuable.

What needs improvement?

The solution needs more metrics for reporting.

For how long have I used the solution?

We have been using the solution for two years.

Buyer's Guide
Apache JMeter
March 2024
Learn what your peers think about Apache JMeter. Get advice and tips from experienced pros sharing their opinions. Updated: March 2024.
765,386 professionals have used our research since 2012.

What do I think about the stability of the solution?

The solution is stable, and we have not had any issues.

What do I think about the scalability of the solution?

The solution is scalable, but I cannot comment on the extent of its scalability because we have yet to scale widely.

How are customer service and support?

We have not had experience with customer service and support.

How was the initial setup?

The initial setup is straightforward.

What's my experience with pricing, setup cost, and licensing?

We use the tool as an open-source tool, so we do not pay for it.

Which other solutions did I evaluate?

We chose this solution because of its affordable cost, ease of use, and popularity.

What other advice do I have?

I rate the solution a nine out of ten. I advise users considering this solution to take a course to get familiar with it quickly.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
ArtemCheremisin - PeerSpot reviewer
Performance Test Engineer at BETBY
Real User
Top 5Leaderboard
A good load testing tool that comes with a strong online support community, but lacks reporting and analysis functionality
Pros and Cons
  • "We really appreciate that the solution comes with a live community, which continuously provided plugins and support protocols."
  • "We would like some reporting and analysis tools to be added to this solution."

What is our primary use case?

We use this solution to load test functional behavior, and measure performance.

What is most valuable?

We really appreciate that the solution comes with a live community, which continuously provided plugins and support protocols.

What needs improvement?

We would like some reporting and analysis tools to be added to this solution.

We would also like the manual available for this solution to allow for better usability; it can be quite complex for new users, and the product is not always very intuitive. 

For how long have I used the solution?

We have been working with this solution for 12 years.

What do I think about the stability of the solution?

We have found this solution to be a stable and reliable tool.

What do I think about the scalability of the solution?

This solution is easily scalable, you can distribute the larger generation among several machines in the LAN, or you can use the cloud variation of the product.

How are customer service and support?

This solution doesn't come with technical support, but there is plug-in wide community support, where organizations can ask questions about issues they are experiencing.

How was the initial setup?

This solution isn't actually deployed, it is a zip file, which is extracted. It can also be deployed via public cloud, if an organization chooses the cloud-based model.

What's my experience with pricing, setup cost, and licensing?

This is an open-source solution, so there are no licensing costs involved.

What other advice do I have?

We would recommend that organizations considering this solution invest time in research of the product before implementing it. There are a lot of online resources that can assist with this, so the business can check if the solution will meet their requirements.

I would rate this solution an eight out of ten.

Which deployment model are you using for this solution?

On-premises
Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
Buyer's Guide
Apache JMeter
March 2024
Learn what your peers think about Apache JMeter. Get advice and tips from experienced pros sharing their opinions. Updated: March 2024.
765,386 professionals have used our research since 2012.
Engineer 2 at a computer software company with 10,001+ employees
Real User
Top 5Leaderboard
User friendly interface, open-source and easy to install
Pros and Cons
  • "It's very easy to install, and it's very easy to code and develop the script."
  • "The memory utilization in JMeter is very poor."

What is our primary use case?

I use the solution for performance testing (Load testing, Volume testing and Endurance testing).

How has it helped my organization?

The solution has a friendly user interface and is a very good open-source solution for load testing. It supports both web and mobile network recording via proxy and offers a wide range of protocols. It has strong community support across multiple platforms.

What is most valuable?

The solution is open-source. 

It is very easy to code and develop the script and it supports web and mobile scripting.

There is a lot of good documentation available. 

The solution is stable. 

What needs improvement?

The memory utilization in JMeter is poor. Limited in terms of scalability. The system gets hung up for complex scripts and heavy loads which requires improvement.

It needs improvement in analytics and statistics reporting. The reports offer limited information on a load test. I would highly appreciate it if JMeter can provide better reporting capabilities.

For how long have I used the solution?

I have been using this solution for two years.

What do I think about the stability of the solution?

It's a stable and reliable product. There are no major bugs or glitches and doesn't crash. 

What do I think about the scalability of the solution?

The solution is scalable. However, it has a lot of memory issues during scale-up. That is the one concern. It's a bit limited in terms of scalability.

How are customer service and support?

I have never contacted Apache customer service for any issues. However, the documentation is readily available online and very useful.

How was the initial setup?

The initial setup is pretty straightforward and easy to implement. It's not overly complex. 

I'd rate it a five out of five in terms of ease of implementation. 

What about the implementation team?

I can do the implementation by myself.

What's my experience with pricing, setup cost, and licensing?

It is an open-source tool.

What other advice do I have?

I'd rate the solution eight out of ten.

Which deployment model are you using for this solution?

Private Cloud
Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
Senior Automated Tester & Test Manager & AnalystSenior at a consultancy with 10,001+ employees
Real User
Top 5
Easy to setup, includes a wide range of plugins, but the UI has room for improvement
Pros and Cons
  • "The solution helps by detecting bottlenecks."
  • "The UI has room for improvement."

What is our primary use case?

We use the solution for performance distributing testing.

How has it helped my organization?

The solution helps by detecting bottlenecks.

What is most valuable?

The solution provides a wide range of plugins and extensions, such as multiple monitors and multiple samplers. Apache JMeter can be used for multiple purposes, including performance and API testing, as well as integration testing. Additionally, users can create their own cloud and build their own distributor, eliminating the need to pay for any SaaS solution.

What needs improvement?

The UI has room for improvement.

I would like to be able to measure web performance as well using the solution. Apache JMeter is only for infrastructure testing, and backend testing, but we cannot use it for performance testing because we need to do it through the browser. 

For how long have I used the solution?

I have been using the solution for one year.

What do I think about the stability of the solution?

The solution runs on Java software. 

What do I think about the scalability of the solution?

I give the scalability a five out of ten.

How was the initial setup?

The initial setup is straightforward. The documentation is very detailed.

I created a resource group in our private Azure cloud, and then I had multiple agents. This included one master and four or five slaves of Apache JMeter. I configured the Apache JMeter and began the distribution and testing.

What other advice do I have?

I give the solution a seven out of ten.

I suggest using the solution and following the basics.

Which deployment model are you using for this solution?

Private Cloud

If public cloud, private cloud, or hybrid cloud, which cloud provider do you use?

Microsoft Azure
Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
Associate Test Engineer at OSI Consulting, Inc.
Real User
Top 5Leaderboard
Good solution for performance testing that works with multiple plugins
Pros and Cons
  • "The solution has good transition controllers and distributed testing."
  • "The solution's setup could be easier and security could be improved to minimize vulnerabilities."

What is our primary use case?

Our company uses the solution for performance testing on web applications, API load testing, and automation of APIs and web applications. The solution provides three streams of use. 

We also integrate Selenium with the solution to perform automation testing. 

What is most valuable?

The solution has good transition controllers and distributed testing. 

What needs improvement?

The solution's setup could be easier and security could be improved to minimize vulnerabilities. 

For how long have I used the solution?

I have been using the solution for four years.

What do I think about the stability of the solution?

The solution is used to test the stability of other applications through a three-step process. 

Load tests look at the CPU, memory, and APM tool and assess the current load to determine if it can be increased from 60% to 70% for example. You continuously monitor the APM to see fluctuations in memory and CPU usage. If load tests are successful, then you move on to stress tests. 

Stress tests find bottlenecks and determine how many users the application can handle within the current infrastructure. Baseline tests can be performed to determine results such as response time when increasing users. 

Endurance tests determine if an application is stable and for how long stability can be expected. For an endurance test, you set up a particular user load and run it for at least eight hours but up to twenty-four hours. During this time, you see how the application is behaving and whether the CPU and memory are stable. This will allow you to assess overall stability without OOM leakages. 

The solution determines if there are lags in application performance. For one project with 500 continuous users per hour, we used the solution to find that there were only two lags from 50,000 samples generated.  

What do I think about the scalability of the solution?

The solution is used to perform baseline tests for applications and compare them to SLAs to determine user load and scalability. 

How are customer service and support?

I have not needed technical support. 

How was the initial setup?

The initial setup can be complex depending on the application you want to test. 

If the application is straightforward without much logic, then setup is easy and only requires a simple transaction controller. This scenario is common for users who log in, perform simple actions within 15 minutes, and log out. 

Applications with internal logic require scripting that includes sample JSON extractors, pre- and post-processes, transition and throughput controllers, and precision throughput timers. This scenario is common for users who perform multiple actions over a longer period of time, is not straightforward, and involves multiple transaction controllers. 

What about the implementation team?

Our company implements the solution for customers.

First, we ask our customers about critical work flows and categorize them based on user loads to create a workload model. Then, we create scripts and proceed with testing. 

Often, we request access to a remote desktop for installing scripts and resetting networks. We follow the workload model to initiate load, stress, and endurance tests. 

What's my experience with pricing, setup cost, and licensing?

The solution is open source with no user fees or licenses. 

Which other solutions did I evaluate?

The solution is comparable to LoadRunner because it works with multiple plugins but is free to use. 

What other advice do I have?

I recommend the solution and rate it a eight out of ten. 

The solution is open source so on occasion there are security vulnerabilities. 

Which deployment model are you using for this solution?

On-premises
Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
Technical Specialist at a financial services firm with 10,001+ employees
Real User
Contains all the features required for performance metrics and is very user friendly
Pros and Cons
  • "Very user-friendly and easy to use."
  • "It has some proxy-based dependencies which require specific proxies to be set up or disabled, which causes problems."

What is our primary use case?

We are in the financial industry in India and carry out performance testing. We deal with  SWIFT messages and financial messaging systems. A few years ago we had 2 million transactions in five hours; today it's 2 million in two hours. To measure these performances, I use JMeter for regression. We have Exadata, Exalogic infrastructure. We are users of JMeter and I'm a technical specialist. 

What is most valuable?

This tool is very user-friendly and easy to use. It's open-source so there are no costs involved for the non-production environments. It's easily available to anyone who wants to use it and it has all the features required for performance metrics.

What needs improvement?

I think it has some proxy-based dependencies which require specific proxies to be set up or disabled, which causes problems when we are working in certain specific environments that have a proxy setup. When we want it to do a record with some new scripts, there are some challenges there.

For how long have I used the solution?

I've been using this solution for two years. 

What do I think about the stability of the solution?

The solution is stable and easily scalable. 

How are customer service and technical support?

The support is very good, with good technical teams. Whenever we get stuck they support us and provide solutions. 

How was the initial setup?

The initial setup took only 10 minutes but there were some complications with the product initially having less features. That has been greatly improved. It's easy to do maintenance of the test tools which we develop, and it's easy to maintain as a product. We have around 20 users of this solution. 

What's my experience with pricing, setup cost, and licensing?

It's open-source, so there's no cost. If you need some support from the vendors there will be a charge for that. 

What other advice do I have?

I would recommend this solution. Most of the support, guides, and tutorials are available on YouTube. 

I rate this solution an eight out of 10. 

Which deployment model are you using for this solution?

On-premises
Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
Senior Performance Engineer II at a tech services company with 1,001-5,000 employees
Real User
Stable, has a user-friendly GUI, and is simple to set up, but it consumes a lot of resources when you increase the load
Pros and Cons
  • "What I like best about Apache JMeter is its user-friendly GUI because even if you don't have very good coding knowledge or understanding, or even if you don't come from a development background, you can still use the solution with just a few clicks. This is what's unique about Apache JMeter, in comparison with other tools in the market. As Apache JMeter is open source, when there's a missing feature, you can search in several community blogs for plugins that you can use to modify Apache JMeter to meet your requirements, and this is another advantage."
  • "What needs improvement in Apache JMeter is the very high load requirements when you want to scale it beyond certain thresholds. For example, small to mid-range testing is very easily done with Apache JMeter, but if you scale and increase the load, then it would be a problem because the tool consumes a lot of resources, probably because Apache JMeter provides an enriched UI experience, so it consumes a lot of memory and requires high CPU usage. This means you have to manage your infrastructure, or else you'll have high overhead expenses. As Apache JMeter is a heavyweight tool, that is an area for improvement, though I'm unsure if Apache can do something about it because it could be a result of the way it's architected. What I'd like to see from Apache JMeter in the future is for it to transition to the cloud, as a lot of cloud technologies emerge around the globe, and a lot of people prefer cloud-based solutions or cloud-native tools. Even if a company has a legacy system, it's still possible to transition to the cloud. I've worked with a company that was an on-premise company that moved to the cloud and became cloud-native. If Apache JMeter could transition to the cloud, similar to k6, then it could help lessen the intense resource consumption that's currently happening in Apache JMeter."

What is our primary use case?

We've been using Apache JMeter for load testing, spike testing, and endurance testing. We use the solution mostly for nonfunctional use cases, except for the security aspects, because those aspects require a different tool altogether.

What is most valuable?

What I like best about Apache JMeter is its user-friendly GUI because even if you don't have very good coding knowledge or understanding, or even if you don't come from a development background, you can still use the solution with just a few clicks. This is what's unique about Apache JMeter, in comparison with other tools in the market.

As Apache JMeter is open source, when there's a missing feature, you can search in several community blogs for plugins that you can use to modify Apache JMeter to meet your requirements, and this is another advantage.

What needs improvement?

What needs improvement in Apache JMeter is the very high load requirements when you want to scale it beyond certain thresholds. For example, small to mid-range testing is very easily done with Apache JMeter, but if you scale and increase the load, then it would be a problem because the tool consumes a lot of resources, probably because Apache JMeter provides an enriched UI experience, so it consumes a lot of memory and requires high CPU usage. This means you have to manage your infrastructure, or else you'll have high overhead expenses.

As Apache JMeter is a heavyweight tool, that is an area for improvement, though I'm unsure if Apache can do something about it because it could be a result of the way it's architected.

What I'd like to see from Apache JMeter in the future is for it to transition to the cloud, as a lot of cloud technologies emerge around the globe, and a lot of people prefer cloud-based solutions or cloud-native tools. Even if a company has a legacy system, it's still possible to transition to the cloud. I've worked with a company that was an on-premise company that moved to the cloud and became cloud-native. If Apache JMeter could transition to the cloud, similar to k6, then it could help lessen the intense resource consumption that's currently happening in Apache JMeter.

For how long have I used the solution?

I've been using Apache JMeter since 2020, so it's been two years since I started using it.

What do I think about the stability of the solution?

Apache JMeter is a stable product overall. A very large community in the software industry uses Apache JMeter, and though it's open source and there's a continuous phase of improvement going on with it, it has stable versions available for my company to continue testing it.

What do I think about the scalability of the solution?

Apache JMeter is scalable in the sense that there's a master/slave architecture available for it where I can scale as many slaves along the way, but when a huge number of users is required in a particular load test, and you have to scale a lot beyond a particular threshold, the resource consumption becomes too high and requires considerable overhead. Apache JMeter consumes a lot of memory, and that's a hurdle, but it's scalable up to a certain point. If a very, very large organization requires very heavy load testing to be done, it would be better for that organization to go with some other tool.

How are customer service and support?

Because Apache JMeter is an open-source tool, you get support from the community. The challenge with open-source tools is that if there isn't enough community, then the support and development you get would be limited, but as Apache JMeter is widely used, the community is vast and the support is sound. Unlike with Micro Focus LoadRunner, when you encounter an issue, you can report that to Micro Focus and the Micro Focus team will take care of your issue. It doesn't work that way with open-source solutions.

For the support I get from the community, on a scale of one to five, I'm rating it a three because there could be instances where you could not get a resolution for your issue. After all, Apache JMeter is a free product, so you can only rely on community support. Though the community for the tool is so big and Apache JMeter is widely used, and there'd be a lesser amount of circumstances where the solution for your issue isn't available yet, there could be between one percent to five percent chance of it happening, so that could be troublesome.

How was the initial setup?

The initial setup for Apache JMeter is pretty simple. There isn't much dependency, in general. For example, in macOS, you can just download the tar files entirely and just untar the files. There's no installation of specific extensions required. That's for macOS. For Windows, there would be .exe files. From that perspective, setting up Apache JMeter is quite easy. You can just run it locally, untarring it in any particular location. Just one setup that's required to be done is setting the Java home path to start the app. Otherwise, it's pretty straightforward and very quick to set up.

On a scale of one to five, I'm rating the initial setup a four because there's always some room for improvement.

What's my experience with pricing, setup cost, and licensing?

Apache JMeter is an open-source solution, so it's free to use.

Which other solutions did I evaluate?

I evaluated k6 and Locust.

What other advice do I have?

I'm using Apache JMeter for work. Locust is also used within the company, as well as k6, but I only touched base on those solutions.

The projects I'm working on currently use Apache JMeter, but my company works across regions and uses different tools, including Gatling and Locust.

Apache JMeter is used daily, for the range of projects I've been testing it on. A new government project came in, and I chose Apache JMeter for it because of its simplicity and user-friendly interface.

My rating for Apache JMeter is seven out of ten.

Which deployment model are you using for this solution?

On-premises
Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
RABBAHMahmoud - PeerSpot reviewer
Senior Technical Architect at RABBAH SOFT
Real User
Top 5
Function testing solution that is straightforward to setup and offers stable performance
Pros and Cons
  • "The most valuable feature of this solution is being able to launch many requests and scheduling simulating human interactions with the application."
  • "If JMeter could integrate with the EPM solution, it would be great. It could also be improved by offering more integrations for security. For example, most applications are secure with OpenID Connect protocols."

What is most valuable?

The most valuable feature of this solution is being able to launch many requests and scheduling simulating human interactions with the application. There are some other plugins for monitoring and collecting data for JMeter but we don't use them. 

What needs improvement?

If JMeter could integrate with the EPM solution, it would be great. It could also be improved by offering more integrations for security. For example, most applications are secure with OpenID Connect protocols.

For how long have I used the solution?

I have been using this solution for more than 10 years.

What do I think about the stability of the solution?

This is a stable solution. 

Which solution did I use previously and why did I switch?

We previously used Katalon. 

How was the initial setup?

The initial setup is straightforward. 

What other advice do I have?

I would rate this solution an eight out of ten. 

Which deployment model are you using for this solution?

On-premises
Disclosure: My company has a business relationship with this vendor other than being a customer: Partner
PeerSpot user
Buyer's Guide
Download our free Apache JMeter Report and get advice and tips from experienced pros sharing their opinions.
Updated: March 2024
Buyer's Guide
Download our free Apache JMeter Report and get advice and tips from experienced pros sharing their opinions.