Apache JMeter Reviews

My main use case of JMeter is for web application performance testing as well as for API performance testing. We are customers of Apache. 

The benefit of JMeter is that it does our performance testing and provides a report without the need to spend money on a licensed tool. It's a significant benefit for us and for the project we're currently working on.

The fact that the solution is open source makes a big difference as we're working on a low-budget project. It's quite user-friendly and easy to use.

There are issues with the plug-ins which you need for reporting purposes as they make the reports quite heavy so you have to run them in non-GUI mode. If you go above the 200 user mark, the application creates a bottleneck and that's one of its major drawbacks. It means you have to run with a master-slave configuration with one system being the master, and multiple slave systems. It's not ideal and I think it could be simplified with a UI that provides direct configuration. In addition, the solution doesn't support SIP applications and some other protocols.

JMeter is not designed for high loads, if you overdo it the tool becomes a bottleneck. Unless you're using JMeter in multiple systems and all the systems are connected through a particular LAN, there is a limit.

Because it's freeware there's no official tech support but you can raise a request on the JMeter site. We haven't had many problems. 

We were previously using LoadRunner for this project but moved to JMeter because it's a freeware testware.

The initial setup is quite easy. There is no deployment process, you download a file from the web application service from your JMeter site, and you can use it. Any user with a little bit of knowledge can do it. The download takes about 15 minutes so you can be using JMeter in about half an hour. From time to time there are version upgrades but they don't affect existing assets. We have four people using JMeter and they don't have any issues. 

BlazeMeter works on top of JMeter and there is a small cost factor to purchase that. It basically gives a slightly more advanced JMeter. 

If you're working on a low-budget project and don't have dependencies of a huge number of users then this is the perfect tool. If you have 2,000 or 3,000 users then it's probably best to look at other options.

I rate this solution eight out of 10. 

We use Apache JMeter for lower-end performance testing. Sometimes we use it for Rest API testing, as well.

This solution allows us to have a detailed look at performance using different metrics. We can look at the deviation and the median values and see how it is performing. We can also analyze our servers and see how they are performing in terms of CPU and how much memory is allocated.

The most valuable feature in JMeter is the Thread Group, which helps us to see whether the performance is good.

The plugin for git allows us to scan files directly from the GitHub site.

The reporting is good and there are a lot of different formats available.

If you want to see the data that is associated with a response then you can download it in JSON format.

There is a large user community that can help you to learn and use JMeter.

When we are testing with too many threads then the solution hangs.

JMeter does not support JavaScript.

Automation is difficult in JMeter.

We have been using JMeter for the past two years.

Stability becomes an issue if we use more than 10 or 20 threads.

We have not been in contact with Apache technical support.

There is a large user community and if you need help then you can get instant help from them.

We also use Postman and when it comes to making a very large request, it is the better option. It is also a user-friendly solution and you do not have to go through a lot of documents to start using it.

JMeter works better for iterative testing.

It is easy to set up and administer JMeter.

This is an open-source product.

This is a good solution but many features would have to be added to use this tool alone. Postman has more features, so JMeter is really only good for performance testing.

I would rate this solution a seven out of ten.

We use it to measure the performance of the system and for balancing.

We like that Apache JMeter has different features and different plugins and that they are free of charge.

We also like that lots of people use it. So, if we happen to get stuck at some point, then we can search for solutions on Google and find them.

Report generation needs to be improved. It is quite difficult to get to.

We've been using Apache JMeter for two years.

It is quite stable.

It is a scalable solution.

On a scale from one to ten, I would rate Apache JMeter at nine. I would recommend it for medium-sized businesses.

Our use cases are for applications such as E-commerce sites where there are multiple concurrent users. In some cases, there is also API load testing where a high number API calls are required for load testing. 

Apache J meter has proved to be the tool of choice for Performance testing with various features and also it comes without any cost.

A valuable feature is that when there's a high number of TPS I can achieve more transactions per second given the hyper-limitations. If the TPS is more than 5,000 TPS, the CPUs get very high (for Load runner)  and JMeter handles that well. I'd also say that the scripting flexibility like co-relation using regular expression is good, it's in the central control. Those would be the two major valuable features. 

We have some scenarios for diameter load testing where TPS requirements are very high, 30K or 40K TPS. In the telco area, this is for simulating mobile usage. However, diameter load testing can be difficult in J Meter. The only way to imitate Diameter requests and process the responses of these requests is to implement them in the code of JSR223 samplers. JMeter generally provides synchronous calls. It's something that could maybe be improved in the future, because for achieving that very high kind of TPS, more than 30K, 40K requires a asynchronous solution. It's not a common thing, it's really very specific to the telco domain and a very few projects. 

I've been using this solution for about 18 months. 

We haven't encountered any major issues with the stability.

Scalability is good. 

Support is more open-sourced. There is a lot of material available online so we haven't had to contact Apache technical support and we have our own team in the company who maintain the product.

We sometimes use LoadRunner for various purposes but currently, we use JMeter 80% of the time because LoadRunner's CPU consumption becomes very high for TPS more than 5000. Also, LoadRunner is not free so we have to pay for the license. We're using it for very few projects now. JMeter can easily achieve a high TPS but LoadRunner has a lot of high-resource requirements. 

LoadRunner's advantage is that the output report is very comprehensive. JMeter reports are only good for the people who understand the detailed logs. Out-of-box, LoadRunner has nice reports and ease of execution. In JMeter, it's also simple but requires some online training or knowledge on the loader and the setup. 

Initial setup is quite straightforward, it takes about two hours, no more, although each project is different. It depends on the scripting that goes into the environment. In terms of the product alone, the deployment doesn't take long. If you include the hardware, it can take about a week because the hardware calls for a lot of couplers and storage units and that takes time.

For the load test which requires high transactions per second to be achieved, in the range of 15K transactions per second, this is a very good tool to use and the biggest advantage is that it is free. It has a big user group, so we can find solutions easily if there are any challenges. It's really a very good open-source tool for any organization to look at in terms of performance tests.

I would rate this solution an eight out of 10. 

In Micro Focus LoadRunner we can go from the UI and we can configure it. There is no such feature in Apache JMeter. There should be UI-based recording history or logs.

I have been using Apache JMeter for one year.

Apache JMeter is stable.

I have found Apache JMeter to be scalable.

Since Apache JMeter is open source there is no vendor support, you have to use the online communities available.

I have used Micro Focus LoadRunner previously and it is better features than Apache JMeter. However, it is not free.

The initial setup was easy for Apache JMeter.

Apache JMeter is a free open source solution and it is why we are using it.

For those who do not want to spend money on tools then this solution would be a good choice. This is the best solution in the open-source category.

I rate Apache JMeter an eight out of ten.

Our primary use for it is testing websites: load testing and performance testing.

JMeter is very easy for a user who doesn't have too much programming knowledge, as scripting is not mandatory nor essential to create the testing scenarios. This means the user doesn't need to know how to write code to run a complete set of load tests. Although, If you're an advanced user, JMeter does gives the possibility of using BeanShell, Groovy or Java to script your tests.

JMeter has a very friendly and intuitive User Interface which allows users to configure the scenarios reusing components already defined, which you can also modify to fit the requirements of the tests.

One very important feature of JMeter is that it is open source and it can be easily extended through a huge set of available plugins.

Considering the kinds of tests we are performing here, where we launch several tests at the same time as a batch request, JMeter is not the best tool for the job. Those kinds of things could be done easily with other tools, like k6. It would be simpler that way. JMeter is a very old tool. It has been around for about 15 years. While it has been improved over the last few years, it is a little complicated to run several tests at the same time with different sites.

JMeter could be easier. It would be a great improvement if it was easier to integrate with the CI deployments, with tools like Jenkins or CircleCI.

Our teams have been using it for about five years.

It is a very stable tool.

In my opinion, it is not scalable. We haven't found an easy way to do that.

We would like to increase our usage but we are trying to figure out the best way to integrate it with our new circuit. Our plan is to use a tool like this with customers. That is the reason we are evaluating the pros and cons of different tools. We want to incorporate these tools into our web development process, mainly in our CI/CD process. We are trying to figure out how to use tools that are designed to empower developers.

We have used LoadUI and SmartBear with tools like Selenium. Some of the developers, at that time, were used to JMeter and they introduced it. It is a very nice tool.

The initial setup is simple. It is not a hard tool to use. It has a very nice UI and the end-users are very comfortable using the tool. People deploy it themselves.

I would definitely recommend JMeter. It is a very nice tool. It is easy to use and there are several components already developed in the tool that you can use to simplify the process of defining and redefining scripts. But I would like it to be more flexible and integrate with other tools.

We primarily use the solution for API testing.

The ease of use is the solution's most valuable feature. Also, the ability to easily create test cases is also very good. It's easy to just ramp up on the solution.

The solution is new to us. I'm not sure if we're using the full capabilities of the solution yet, but from what we have used, we're quite satisfied.

In future releases, it would be helpful if there was an integration with ALM Octane.

The solution is stable.

The solution is very scalable.

We're not using too many features of the solution itself, so we haven't needed the assistance of technical support yet.

We previously used LoadRunner. We decided to switch because on LoadRunner scaling up was a problem.  We needed licenses to do so and it was becoming a problem for us to simulate the number of users we were expecting. 

The general setup of the solution is straightforward.

Before we chose JMeter, we looked at NeoLoad, but it looked to have the same issues surrounding licensing that LoadRunner did.

We are using the on-premises deployment model.

The advice I would give to others would be to look at your requirements and decide if the solution is right for you. For us, it was API testing and JMeter is very good for this.

I'd rate this solution nine out of ten.

• Recording/Playback
• Templates for test plans (NEW)
• Integrate with Java and many other scripting languages
• Integrating with external plug-ins

• The recording feature makes it easy to create test plans
• They have templates that make starting very easy (this feature is added recently)
• Supports a lot of protocols (HTTP, HTTPS, FTP , SMTP etc)
• Simplicity of use
• As compared to commercial tools, it provides better reporting features through third-party plug-ins
• There's also a huge community around tool, and because of that when there's an issue, you can get solutions easily, and without cost For example, we dis a very complex testing of HL7 protocols, which JMeter doesn't support directly, but it's extendable to support the protocols
• Supports Java and other scripting languages and can extend capabilities.

Reporting could be better if it could be like commercial tools. For example, a nice chart and visualizations. You can’t rely on the first 100 test results because the ramp-up and ramp-down time gives false results with outliers. It would be great if an option to ignore first 100 and last 100 records from results and then generate reports, should be great. Needs to support more concurrent users by single JMeter instance (agent/slave)

I've been using it for over last five years, and it is a major, active open source tool.

No issues with recent releases

There are issues when tests are stopped with large number of concurrent users. Test is not stopped as expected.

There is a huge community around JMeter. Hence free support is available to the users.

Excellent (from the community)

Grinder. Grinder was not user friendly and had to do lot of coding and low level configurations. It was lighter than JMeter.

Initial setup is straight forward. You need to have have JDK or JRE for running JMeter.

We have a team of testers capable of managing the JMeter perfromance testing project effectively.

It's easy to use and free, a huge community, and it offers a risk free to start, so use it for application performance testing. It supports connectivies with databases, FTP, and other system protocols.