Apache JMeter Reviews

We've been using Apache JMeter for load testing, spike testing, and endurance testing. We use the solution mostly for nonfunctional use cases, except for the security aspects, because those aspects require a different tool altogether.

What I like best about Apache JMeter is its user-friendly GUI because even if you don't have very good coding knowledge or understanding, or even if you don't come from a development background, you can still use the solution with just a few clicks. This is what's unique about Apache JMeter, in comparison with other tools in the market.

As Apache JMeter is open source, when there's a missing feature, you can search in several community blogs for plugins that you can use to modify Apache JMeter to meet your requirements, and this is another advantage.

What needs improvement in Apache JMeter is the very high load requirements when you want to scale it beyond certain thresholds. For example, small to mid-range testing is very easily done with Apache JMeter, but if you scale and increase the load, then it would be a problem because the tool consumes a lot of resources, probably because Apache JMeter provides an enriched UI experience, so it consumes a lot of memory and requires high CPU usage. This means you have to manage your infrastructure, or else you'll have high overhead expenses.

As Apache JMeter is a heavyweight tool, that is an area for improvement, though I'm unsure if Apache can do something about it because it could be a result of the way it's architected.

What I'd like to see from Apache JMeter in the future is for it to transition to the cloud, as a lot of cloud technologies emerge around the globe, and a lot of people prefer cloud-based solutions or cloud-native tools. Even if a company has a legacy system, it's still possible to transition to the cloud. I've worked with a company that was an on-premise company that moved to the cloud and became cloud-native. If Apache JMeter could transition to the cloud, similar to k6, then it could help lessen the intense resource consumption that's currently happening in Apache JMeter.

I've been using Apache JMeter since 2020, so it's been two years since I started using it.

Apache JMeter is a stable product overall. A very large community in the software industry uses Apache JMeter, and though it's open source and there's a continuous phase of improvement going on with it, it has stable versions available for my company to continue testing it.

Apache JMeter is scalable in the sense that there's a master/slave architecture available for it where I can scale as many slaves along the way, but when a huge number of users is required in a particular load test, and you have to scale a lot beyond a particular threshold, the resource consumption becomes too high and requires considerable overhead. Apache JMeter consumes a lot of memory, and that's a hurdle, but it's scalable up to a certain point. If a very, very large organization requires very heavy load testing to be done, it would be better for that organization to go with some other tool.

Because Apache JMeter is an open-source tool, you get support from the community. The challenge with open-source tools is that if there isn't enough community, then the support and development you get would be limited, but as Apache JMeter is widely used, the community is vast and the support is sound. Unlike with Micro Focus LoadRunner, when you encounter an issue, you can report that to Micro Focus and the Micro Focus team will take care of your issue. It doesn't work that way with open-source solutions.

For the support I get from the community, on a scale of one to five, I'm rating it a three because there could be instances where you could not get a resolution for your issue. After all, Apache JMeter is a free product, so you can only rely on community support. Though the community for the tool is so big and Apache JMeter is widely used, and there'd be a lesser amount of circumstances where the solution for your issue isn't available yet, there could be between one percent to five percent chance of it happening, so that could be troublesome.

The initial setup for Apache JMeter is pretty simple. There isn't much dependency, in general. For example, in macOS, you can just download the tar files entirely and just untar the files. There's no installation of specific extensions required. That's for macOS. For Windows, there would be .exe files. From that perspective, setting up Apache JMeter is quite easy. You can just run it locally, untarring it in any particular location. Just one setup that's required to be done is setting the Java home path to start the app. Otherwise, it's pretty straightforward and very quick to set up.

On a scale of one to five, I'm rating the initial setup a four because there's always some room for improvement.

Apache JMeter is an open-source solution, so it's free to use.

I evaluated k6 and Locust.

I'm using Apache JMeter for work. Locust is also used within the company, as well as k6, but I only touched base on those solutions.

The projects I'm working on currently use Apache JMeter, but my company works across regions and uses different tools, including Gatling and Locust.

Apache JMeter is used daily, for the range of projects I've been testing it on. A new government project came in, and I chose Apache JMeter for it because of its simplicity and user-friendly interface.

My rating for Apache JMeter is seven out of ten.

Our company uses the solution for performance testing on web applications, API load testing, and automation of APIs and web applications. The solution provides three streams of use. 

We also integrate Selenium with the solution to perform automation testing. 

The solution has good transition controllers and distributed testing. 

The solution's setup could be easier and security could be improved to minimize vulnerabilities. 

I have been using the solution for four years.

The solution is used to test the stability of other applications through a three-step process. 

Load tests look at the CPU, memory, and APM tool and assess the current load to determine if it can be increased from 60% to 70% for example. You continuously monitor the APM to see fluctuations in memory and CPU usage. If load tests are successful, then you move on to stress tests. 

Stress tests find bottlenecks and determine how many users the application can handle within the current infrastructure. Baseline tests can be performed to determine results such as response time when increasing users. 

Endurance tests determine if an application is stable and for how long stability can be expected. For an endurance test, you set up a particular user load and run it for at least eight hours but up to twenty-four hours. During this time, you see how the application is behaving and whether the CPU and memory are stable. This will allow you to assess overall stability without OOM leakages. 

The solution determines if there are lags in application performance. For one project with 500 continuous users per hour, we used the solution to find that there were only two lags from 50,000 samples generated.  

The solution is used to perform baseline tests for applications and compare them to SLAs to determine user load and scalability. 

I have not needed technical support. 

The initial setup can be complex depending on the application you want to test. 

If the application is straightforward without much logic, then setup is easy and only requires a simple transaction controller. This scenario is common for users who log in, perform simple actions within 15 minutes, and log out. 

Applications with internal logic require scripting that includes sample JSON extractors, pre- and post-processes, transition and throughput controllers, and precision throughput timers. This scenario is common for users who perform multiple actions over a longer period of time, is not straightforward, and involves multiple transaction controllers. 

Our company implements the solution for customers.

First, we ask our customers about critical work flows and categorize them based on user loads to create a workload model. Then, we create scripts and proceed with testing. 

Often, we request access to a remote desktop for installing scripts and resetting networks. We follow the workload model to initiate load, stress, and endurance tests. 

The solution is open source with no user fees or licenses. 

The solution is comparable to LoadRunner because it works with multiple plugins but is free to use. 

I recommend the solution and rate it a eight out of ten. 

The solution is open source so on occasion there are security vulnerabilities. 

I am using JMeter only to test some web services.

You can moderate some transactions which aren't available to too many users. You can use JMeter and compress as needed or to support as many users as possible. 

I am a basic user. I couldn't explain the different features deeply. I use, for example, the web services. I like the features where you are programming at an increased rate in web services. I use those features. For example, as part of using our web services, if there are 100 transactions with three threads, and I want to finish with 5,000 transactions with 20 threads on different pages, I can find the rate which my server used on each page. I like this feature in JMeter.

It's easy to set up.

Apache should have a graphic interface. That would help beginner users a lot. Sometimes it's hard to do what you need to do via the command line. 

I've been using the solution for around two or three years. 

The solution is stable. There are no bugs or glitches, and it doesn't crash or freeze.

The solution can scale. However, it requires configuration to do so. I don't have much knowledge in regards to how to scale exactly.

On the other hand, if you have a cloud server with 10 servers and you are using the same DNS for all servers, as all servers are working with the same application, and you have for example a load balancer that provides balancing to each one of these ten servers, I am not sure if Apache JMeter will report the different servers separately. I imagine not.

Different implementations of JMeter Runtime could be used to point to each one of these 10 servers. You can use JMeter in different OS, for example, Windows, Linux, or Unix. You have a lot of features to use different instances with JMeter. However, this load balancer concept would be harder to explain.

Right now, we have around two or three guys working directly with the product.

We are evaluating automatization apps currently. We are using Jenkins, and I am sure we will start with JMeter later. However, I have not defined a roadmap to expanding usage of this product just yet.

I have not used technical support.

I have used some other solution previously. However, I cannot recall the name. It was likely SoapUI. We picked up JMeter as it is free to start with and easy to pick up.

The product is simple to set up. 

For example, you can just support Apache JMeter with the Java installer feature, which downloads the JDK or Java Runtime from the Oracle pages, for example. Or you can add the Runtime as a part of your installer, and after the installation, have a common user in Windows, for example, as an option. There is some knowledge of Java needed, and you do need to use the command line. Some may feel a bit lost when they start. However, there are different ways to do it according to your comfort level. 

It should have something for the front-end user to help them prepare the environment correctly, however.

We have two to three people handling the maintenance of the product at this time. 

I have not really noticed a return on investment. 

I cannot recall the exact version number we are using. I always download the latest. 

We are currently using our own servers to provide the network.

I recommend JMeter if your goal is to design some stress tests. JMeter has some features. For example, you can start with tests on the command line. You can join instantly on our current deployment batches. Jenkins has some support for JMeter as well. On another hand, you can make a simple bridge to start the JMeter process with the command line in the same Jenkins deployment. You have a lot of features on hand.

I'd rate the solution eight out of ten.

We use this solution for functional API testing, and performance testing.

This solution allows us to test all the functional APIs through it, then record and run the APIs.

This solution is easier to use than any other tool in the market; there is not even a requirement to learn a lot of scripting in order to use it.

We also appreciate that the solution is open source, so it has a very big community to utilize if we need help with an issue.

At present, if the number of virtual users increases beyond 10,000 when testing, then it results in a Java heap which causes the solution to crash.

I have been working with this solution for around seven years.

The stability of this product is good, outside of the issues with adding virtual users during testing.

This solution is scalable, but only up to a certain point when adding virtual users, before performance issues begin to occur.

The initial setup of this solution was quite straightforward, and deployment only took one to two minutes.

We implemented this solution using our in-house team.

This is an open-source solution, so there are no licensing costs associated with it.

We would recommend this solution to any QA or developer as a 'must-try product'.

I would rate this solution an eight out of ten.

The use cases are mainly for performance testing and for test case management of the different APIs.

Initially, we didn't have any tool for performance testing, particularly open source. JMeter, since it is open source, we can test our APIs pretty quickly and we can work with different teams to provide performance testing very quickly. It doesn't take up much of our time.

The ease of scripting is great. 

The modular design is very useful.

It is easy to set up.

The solution is stable.

It can scale well. 

If JMeter could provide a web version of editing, that would be good. 

If JMeter can provide its own cloud version rather than depending on BlazeMeter, the commercial version of JMeter, that would be ideal. If we could have somebody right on the front end of JMeter, using it on any of the clouds, including AWS, GCP, or Azure, that would be very helpful. it would be better than me going for using commercial services.

I would like to have some kind of cloud version that can be implemented. Or we would like a Docker version. A Docker version is something that I would look for.

I've been using the solution for four or more years. 

It is a stable, reliable product. There are no bugs or glitched. It doesn't crash or freeze.

The solution has been scalable. 

We have almost 100 users on the solution. 

We likely won't increase usage in the future.

I do have some knowledge of technical support. 

This is an open-source tool. If I write any issue in the forums, somebody will answer it pretty quickly. The open source technical support they have on offer is good.

We did use a different solution previously. 

Most people know JMeter more than the other tools. It's easy to learn and quick to execute. It's also open source. You can get a lot of solutions across the internet. If you are having some issue or error, somebody or the other may have faced that same issue, and he may have found out the solution. It makes troubleshooting easy, having crowdsourced troubleshooting. That was a factor in our decision-making. 

The initial setup is not straightforward. It's simple and not complex at all. 

We only need two people to maintain the product. It doesn't take a lot of staff. 

We have our own in-house team that can handle the setup. 

We've definitely seen an ROI. 

We use an open-source version of the product. However, there are cloud services that we use. I'm not sure of the cost of the cloud services.

We looked into Micro Focus and Cavisson NetStorm. However, they were licensed tools, and we decided against them. 

We are using the latest version of the solution. 

We are using a cloud version of JMeter that is called BlazeMeter.

If you are a company that is evaluating other licensed tools, just go for JMeter. It's open source, easy to use, and the most widely used across platforms. It would be much simpler than using the other licensed tools. Other licensed tools may have pretty good technical support. However, most problems with JMeter can be solved using the solutions that are available on the internet.

I'd rate the solution eight out of ten.

We use this solution mostly for creating scripts, load testing, and performance testing. If my client doesn't have a tool or I don't have a tool, I try to leverage JMeter because it's an open-source tool.

I personally use JMeter for things which cannot be covered by our load testing tool, which is OATS, Oracle Application Testing Suite. This is a tool that was developed by Oracle but is no longer in use because Oracle has stopped developing it.

The solution is deployed on-prem and on private and public clouds.

JMeter should be more stable. Every time there is a new release coming up, a lot of its older functionalities or the new functionalities that are brought in are not very well-documented. It should be documented properly, and there should be proper use cases. A lot of the newer features don't work, and sometimes you have to spend a lot of time maintaining the scripts. That is something JMeter could probably look at.

For example, in JMeter 5 they brought in a lot of new controllers. But there isn't a lot of documentation available on the Apache site on how you're supposed to use those controllers. They've explained the controller functionality, but there aren't any proper use cases to show that. 

I have been using JMeter for a long time and do a lot of work with this solution.

It's not stable. With every new release, they come up with newer features, which aren't always very stable. So, stability is a concern.

It is scalable. You can scale up to 1,000 users in JMeter. If you can put up four slave servers, you can easily ramp up to 1,000 users. That's not a problem.

Because it's open-source, a lot of organizations try to leverage JMeter into everything. I don't think JMeter is currently very adaptable to all the technologies. That is something Apache should think of – incorporating other technologies and other tech stacks, which can be scripted from JMeter and tested using JMeter.

On a scale of 1 to 5, I would rate technical support 2.5

They're not very good because JMeter is an open-source tool. You can raise a bug to JMeter, and they might fix it or they might say that, "We will actually fix this in the next release." So, because it's an open-source tool, you do not get dedicated support. You have to raise tickets and wait. If you use BlazeMeter, the support is probably a little better because you can take your issues there.

I don't think support is great if you compare it with tools like NeoLoad or LoadRunner where they have a dedicated support team. You can raise tickets. You have a direct conversation with their engineers. They can help you understand where the issue is exactly. That kind of support is not in JMeter, at least not now. BlazeMeter has it, but I have not used BlazeMeter support, so I'm not sure how it is. But from what I have seen with JMeter, support is not great.

It is straightforward, but for JMeter 5, since they have introduced SSL handshake at the slave level, that becomes a little bit of a challenge for somebody who is doing it for the first time if you're doing a distributed execution, that becomes a challenge.

But since there was not enough documentation, I had to do a lot of troubleshooting on my own, since it was new. Now there's a lot of documentation available, but initially, there was not much documentation.

JMeter is pretty easy to use if you are using it for the purpose of only developing scripts. You can just uninstall the file and you can start using it. The only challenge happens if you're trying to access the internet over any kind of proxy. Then, you'll have to start it via the proxy mode.

Otherwise, deploying JMeter is pretty easy compared to other tools. You just have to download, unzip, and you are ready to go.

We installed JMeter ourselves.

I would rate this solution 7 out of 10.

Apache JMeter's key feature is its ability to manage load profiles, gradually increasing requests over time. This was crucial for us as we tested our application, handling unique protocols and increasing load steadily. JMeter helped identify bottlenecks by measuring response times as we increased request flow rates. This data guided us in optimizing our system's performance and scaling hardware when needed. Recently, new tools like Platinum Consultant have emerged, but I haven't explored them thoroughly. My colleagues prefer these newer tools over JMeter.

JMeter helps us track response times between request and response. As we increase our workload, response times also rise, indicating potential bottlenecks. We use JMeter to gauge when we need to upgrade hardware or optimize our application for better performance. It's effective in measuring various request types and their corresponding response times, making it a valuable tool for assessing system performance.

I appreciate JMeter's simplicity and power for performance testing. While I haven't used all its features, the ability to simulate heavy loads from multiple users is quite beneficial. However, in my current configuration, we haven't utilized this specific aspect of JMeter. Compared to other costly tools like Hewlett Packard, JMeter is free and easier to use, although there are newer tools like Gatling that I haven't tried. Overall, JMeter is simple and effective for performance testing.     

Improving JMeter's sync time could be beneficial. For example, compared to a Hewlett Packard tool that required four machines for load generation, JMeter reduced this to possibly just two machines for the same workload.

I've been a junior with a few years of experience using Apache JMeter for load testing. It's a straightforward tool with useful features, although not entirely unique. 

Scalability is near-linear, especially with custom configurations.

Setting up JMeter is straightforward, not complex at all. Deployment time depends on the code you write for JMeter, which is executed efficiently.I've used it on-premises, but it might also work in a cloud configuration

Since it's free, there's no need for extensive support or improvements in pricing.

Overall, I'd give JMeter a solid ten for its simplicity and effectiveness in typical tasks. While the UI could be slightly better, it's not a critical issue. JMeter provides valuable data and insights through its graphs, and its main benefit lies in being free, simple to use, and widely recognized.

We are using this for performance testing and some automation. We consolidate the test data in our own systems and then use an API to send this data to JMeter. The software testing team then uses it to perform all the tests they wish to do. Nowadays, we are mainly focused on regression testing and working with that.

It works really well. 

To be honest, the UI could be more user-friendly. As a UX guy, I believe it's quite cold. The solution that I have nowadays is a very traditional one. So, there is room for improvement in the UI of Apache JMeter. 

I have been using Apache JMeter for two years. We have it in-house. It's a solution that my company has been using for a long time.

There are around seven users in my company using this solution. 

In terms of setup, it's quite simple. It has its benefits, but we are already used to it. In terms of experience, it could be nicer, to be honest. Sometimes, I get a little bit lost.

Two people are required for the deployment and maintenance. They can be he managers, tech leads, engineers, or developers.

The users need to pay for licenses.

We are not switching because we have several testing teams, and we sell them to several customers. So the solution sometimes is not ours; it's the customer's, and they provide some licenses because usually, companies like Vodafone have corporate licensing. They can provide licensing without being responsible for it. 

We also have our own solutions, but we are not sometimes responsive before saying I don't want to use this or I don't want to use that. Because we are an outsourcing company, we put people accordingly to the field desired by our customers. So if I need a Selenium expert or an Optimum or TestRail, whatever, I go to the market, and I outsource this to the customer.

Overall, I would rate the solution a seven out of ten. I recommend it because I know a lot of people who use it, and they don't have any complaints.