Apache JMeter Reviews

I use Apache JMeter primarily for performance testing projects. It is a day-in, day-out tool where we run performance test executions to create a production-like load on systems. We analyze the results and share recommendations with customers. We use it for load testing, stress testing, and endurance testing on systems.

Apache JMeter helps us meet our requirements for performance testing by allowing us to create scripts and run them. It is part of our daily routine, and it provides us the capability to test using different protocols like gRPC, which is essential for our operations.

The most valuable features of Apache JMeter include SCTP sampler, throughput controllers, stepping up thread group, parallel controller, timers, and the ability to use gRPC protocol. These features meet our requirements and allow us to effectively create and run our performance testing scripts.

Apache JMeter has room for improvement in handling larger infrastructures as it consumes a lot of CPU and memory. It requires integration with other tools for live metrics, which is time-consuming. 

Also, the failure response times are calculated in the overall response time analysis, which should be separate. Better script maintenance and integration with ALM or repository tools would be beneficial.

I have been using Apache JMeter for about 14 years.

Apache JMeter is generally stable, with a high stability rating of nine out of ten. Any instability could impact timelines, project deadlines, releases, and revenues.

While Apache JMeter can scale to the required number of user loads, it requires a lot of manual work and larger infrastructure, which are challenges to its scalability.

There is no dedicated support team for Apache JMeter as it is an open-source tool. We rely on community support, which is not enough and needs to be improved.

Positive

I previously used LoadRunner. We switched due to customer preference for a commercial tool.

The initial setup of Apache JMeter is rated between eight and nine out of ten. If you have knowledge, it's an easy setup.

No specific team is required to maintain Apache JMeter. Performance test engineers familiar with the tool can maintain it.

Apache JMeter is an open-source tool, so there are no costs associated with its setup, pricing, or licensing.

We evaluated NeoLoad and found LoadRunner to be more realistic than NeoLoad and Apache JMeter.

For those looking into open source, JMeter is an excellent tool and should be the first option. For commercial needs, BlazeMeter is a good choice, or LoadRunner is a good choice for a commercial setup.

I'd rate the solution ten out of ten.

Any front-end technologies like Angular, React, or others might access the services. Therefore, you must ensure that using SAP with JMeter works fine. Unless you ensure this, the front-end team may not access the services properly.

You can save the state of your tests and test plans after conducting tests, allowing you to showcase the results to others later. This feature makes it easier to demonstrate what testing was done and share those test cases.

From an improvement perspective, consider a scenario where your application needs to retrieve data from PayPal and store it in an Excel file in an external network. Connecting to such a source using JMeter might take some time. Previously, you might have linked your application to a server to obtain these results. However, achieving this kind of integration directly in JMeter can be challenging. If it were possible to configure JMeter to handle such server connections seamlessly, its capabilities would greatly be enhanced.

I have been using Apache JMeter since 2004-05.

It is a stable tool and has been stable for a long time. Organizations have been using it extensively since 2017. However, it's important to note that while JMeter is excellent for testing the performance of your service, you cannot directly forward the results to the end user.

You can increase the testing scale. When deploying a microservice in a production environment, performance testing is essential. You should test to handle at least one million users, a minimum benchmark. 

From a developer’s perspective, if your service can handle one or two million users with a response time of less than one second, it is considered acceptable, provided the data is accurate. In production environments with multiple servers, response times might vary, but it is crucial to perform thorough testing in a local environment to ensure the system meets performance criteria. JMeter helps verify that the service can handle the expected load and deliver accurate results.

I am using Azure and GCP.

JMeter is easy to set up, taking about half an hour to install. However, the real effort lies in configuring and developing the tests, which can be time-consuming. 

If there is an update, it doesn't typically take long to come. They may release an upgrade within one or two years. If you have a test suite, it can be easily imported, and you can start using it. This process hardly takes more than half an hour.

Apache JMeter is configuring it properly, especially when dealing with SSL configurations. It can be tricky and time-consuming if you're unfamiliar with handling SSL certificates, keys, and JMX configuration. However, the process becomes straightforward if you know where and how to pass the necessary keys and configurations. Without this knowledge, you'll likely need to spend time researching and Googling how to set everything up. In contrast, if you're familiar with these configurations, setting them up in JMeter is quick and straightforward.

Overall, I rate the solution a nine out of ten.

I consider Apache JMeter to be a fundamental tool in the industry for performance testing. Its primary advantage lies in being free, making it accessible to everyone. The platform boasts a wide range of plugins, regularly updated, and has a substantial community. You can find information on almost any topic in the community forums or Q&A sites related to code.

Apache JMeter is well-known and widely used among developers, particularly on popular developer forums. While it may not have the most user-friendly interface, it offers strong support through official manuals and various articles from companies providing load testing services. 

The tool is free, has a substantial community, and serves as a fundamental choice for testers, especially those new to performance testing. While other tools like K6 may be more developer-oriented, JMeter's affordability and accessibility make it suitable for those without extensive performance testing experience.

Given that Apache JMeter is a free and open-source tool, documentation improvement may not be a major concern, as it is mostly contributed on a voluntary basis. The essential information is already available. However, in terms of the interface, there are occasional bugs, and the tool may not address them as quickly as some users would like. Fixing defects and bugs might take a considerable amount of time, with users sometimes having to wait for several months or even a year for the next release to address specific issues.

I have worked with Apache JMeter for years. 

In terms of stability, there have been some instances of unknown bugs, but these are documented on the official pages and tracked in bug repositories where users can find descriptions and potential workarounds. Users with less experience may encounter unexpected behavior, but with online resources, such as Google, they can understand and address the issues. Some known problems, like internal loop issues when launching from the MacOS terminal, can affect stability, but they are not critical and usually have workarounds. While these issues may slightly impact the user experience, they do not significantly detract from it. On a scale of ten, the overall user experience with JMeter might be rated around seven or eight.

Scalability is possible for both small and large companies. In smaller companies, JMeter can be scaled across multiple machines, utilizing them as load generators. For more complex network setups, collaboration with the DevOps or system administration teams may be required to handle firewall configurations. In large enterprises, options like Docker Swarm or Kubernetes can be employed. Additionally, using JMeter images or containers within these frameworks allows for quick scalability.

While I haven't personally engaged in direct interaction with support, I have encountered challenges and utilized forums and official support channels to address and resolve those issues. So, yes, I've had experiences leveraging these resources for problem-solving.

A few years ago, I found Apache JMeter to be a reliable tool, and while it may have changed since then, I haven't used it recently for a direct comparison. Around five years ago, I had a positive experience with Neotys, which was an excellent tool, albeit expensive. However, as its pricing is now contact-based, I'm not sure about the current costs. Additionally, I've recently explored K6, which seems promising. K6 is actively developed and evolving rapidly. It caters more to a coding-centric experience, making it favorable for developers who appreciate JavaScript-based coding without extensive installations. This tool might see significant growth in functionality in the coming years.

If someone is completely new to Apache JMeter, it might take around one hour to half an hour to set it up initially. This includes spending time reading manuals, watching YouTube videos, and getting familiar with the tool. For those with some experience, the setup process could take as little as ten minutes. This involves downloading and installing JMeter, configuring Java if necessary, and adjusting path variables. The overall time may vary based on the machine's readiness and the user's familiarity with the setup process.

One piece of advice I often provide to those venturing into performance testing with Apache JMeter is to carefully consider the expected load. Due to its utilization of the Java Virtual Machine, JMeter can be memory-intensive. For extensive testing scenarios with a high volume of virtual users and transactions, it's crucial to meticulously configure JMeter, considering each element in the scenario. However, for simpler yet intensive testing needs, individuals might find less resource-intensive tools like Gatling or Locust more suitable.

If it were a commercial tool, I'd rate it an eight, but given that it's a free and non-commercial tool, I'd lean more towards a nine. While it's not perfect, I believe it's one of the best options available for now.

We have multiple use cases. We have APIs integrated for functionality. We're using JMeter for performance testing those APIs.

There are about 80 people working on this project, and about 12 people are using JMeter for performance testing.

Approximately 10% of people are using JMeter in my organization, and some people are using PureLoad, which is a paid tool.

We're using version 4. The solution is deployed on an Azure public cloud.

The reporting features are really good. There's a lot less latency than other solutions. We configure our API script and execute it through JMeter, get a response time over there, and compare that response time with the Azure cloud response time. The latency is low. JMeter is optimized very well. 

Another valuable feature is that we can distribute our load in different formats, like a responsive load. Then, we can distribute our load in a different manner by using plugins.

We decided to use JMeter because it's open source, and it's a very stable product. There are many plugins that are easily available to configure multiple things.

JMeter is lagging when it comes to GUI performance testing because we need to install some third-party plugins for recording the GUI script, and the performance isn't very reliable. I think it should be improved. The API is pretty optimized, but the GUI isn't.

I use JMeter for API and stress testing, and I use a different tool for GUI testing.

It's pretty stable.

My server is capable of processing 60,000 API requests in one minute. I'm putting 1.2 million API requests per minute on my server because I have configured some scaling rules in the Azure environment. After reaching the optimized level of 60,000 requests, it automatically scales out by one load in the Azure cloud. JMeter generates that load with the specific time we have provided in our script and configuration.

I haven't contacted the official JMeter support channel because if I'm facing any issues, I just Google it. I have found a lot of solutions there because it's open source and multiple people can give their feedback and answer queries. 

The reporting is better in PureLoad because it's a licensed project, but it's okay if the entire report is embedded in Excel format.

There are multiple formats, like tree format and CSV format. In CSV format, you can download your reports. By using some formulas in Excel, you can prepare a similar report. There's automation in PureLoad, but it's not really required in my projects.

Setup isn't complicated. I would rate the setup as eight out of ten.

The speed of setup will depend on your internet speed, but deployment usually takes one to two hours to complete. If you've already written the script, you can easily upload those scripts in JMeter and trigger them. It will take one to two hours to set up all the configurations for JMeter.

The solution is open source.

I would rate Apache JMeter as nine out of ten. There are a lot of features. In terms of stress testing and performance testing, it's really good. There are many plugins available that are compatible with JMeter. You can start your load according to your requirement. JMeter isn't very complex to understand. It's user-friendly and the test results are also available in different varieties, including start, view, trees, and tables.

My advice is that you need to understand the requirements for your project and what needs to be done in terms of performance testing. You also need to think about which plugins serve your purpose. If the plugins and reporting are there, then you should definitely work with JMeter. It's a really good tool with no cost.

You can easily find errors and add or drag in the problem. There are also browser plugins, which support multiple tests and protocol services. In my project, I'm testing MQTT services, which is message queuing telemetry transport. The notification service we're using in our application is related to that. I have tested that particular functionality with JMeter as well because of the security protocol.

JMeter is used by a team of four performance engineers. We monitor tasks on the API level, so all of the requests are hitting individually.

JMeter has helped us streamline our applications by identifying multiple bottlenecks in our systems and exercising tasks against APIs.

JMeter is a free tool with a large user population, which comes in handy because we have a vast knowledge base to tap into when needed. It's also easier to hire consultants who know JMeter.

JMeter's reporting is extremely rudimentary. The fundamental reporting mechanisms need to be drastically improved. It doesn't utilize an automatic session management mechanism or methods other tools use like parsing cookies and variables. Everything needs to be done manually. There's no automation.

We've been using JMeter for the past three years.

JMeter never crashed on us. It's highly stable.

JMeter is highly scalable.

There is no technical support for JMeter.

We previously used LoadRunner, but we switched because the organization had an accumulated knowledge of JMeter. Nobody really knew LoadRunner. However, the biggest advantage is that it's free, and we didn't have to invest in acquiring a performance testing solution.

JMeter is simple to install. We did it in-house, and it takes a day or two to deploy it to the cloud. There's no maintenance required. 

We looked at a number of different tools.

I rate JMeter five out of 10. You need knowledgeable employees to work with the tool because it's difficult to learn. There is a steep learning curve, so you shouldn't expect an entry-level engineer to pick up the tool quickly and become proficient. It takes a considerable amount of time.

We use Apache JMeter to test the performance of our platforms and applications.

The most valuable feature of this solution is that it is free. If not for that, we would be using a different tool.

The graphical user interface could be improved.

In this tool, automation in general is almost non-existent. Everything is done manually. I would advise those who put this together to try to simplify it for their end users, such as being able to automate at their desks. Such as manual relations and social management.

Purely on the feature set, it lacks automation, therefore it requires a lot of manual work.

As a performance engineer, I use it almost every day.

I have been using Apache JMeter for two years.

Apache JMeter seems to be stable enough.

In terms of scalability, we haven't pushed it to a large scale yet.

The majority of users are performance engineers, but there are also some developers.

There are four performance engineers and at least three developers who use JMeter on occasion.

We intend to increase usage, but it is already being used exclusively and extensively.

There is no technical support program for Apache JMeter. There is only an online community of people who help one another.

The community is helpful and fairly advanced.

In the past, I used to use various tools, primarily LoadRunner.

When I changed jobs, and the new company only used JMeter because we didn't have a license for LoadRunner, I started working with Apache JMeter.

The initial setup was very easy.

We used it on a desktop computer. It is also used in the cloud. The idea behind using it in the cloud was to spread loads across multiple machines.

We carry out the deployment ourselves.

We have one performance engineering consultant who maintains this solution.

That would not be applicable because it is a free solution.

Licensing costs are rare.

There are operational costs related to using Amazon Cloud, but the tool itself.

I looked into other options, such as Bluehost, SmartBear, Epic, and LoadRunner but Jmeter proved to be sufficient for our needs.

We chose JMeter based on its pricing structure.

My advice is that it requires a fairly consistent amount of initial learning. It is not simple to learn.

I would rate Apache JMeter a three out of ten.

We primarily use it for conducting different types of performance testing, such as load testing, spike testing, and endurance testing.

JMeter is basically the art of the entire performance testing process. We generate load on our application using JMeter and then monitor various metrics like CPU with different monitoring tools. It's the essential foundation for our performance testing.

JMeter is doing some good things with upcoming releases, but the main area for improvement is the extensions available. 

Another area of improvement is the reporting part, specifically regarding report generation. 

There are certain things like we can't merge custom metrics into the JMeter reports. We're limited to JMeter metrics, and other server metrics can't be integrated with JMeter dashboard. This forces us to rely on another tool. 

We should be able to add or custom-configure server details directly in JMeter reports.

We have been using it for three years.

I would rate the stability a nine out of ten.  It's generally quite stable, it hardly has crashes or issues.

Scalability is good for my use cases, but to generate a large load, you must go into distribution mode, which is more complex to configure and requires powerful machines. So, while it's fine for my needs, but the scalability wouldn't be a perfect ten. I would rate it a seven out of ten, as there are some limitations for large-scale testing.

I'm the one who uses it most extensively. And the other we have four to five people are using it just for their local testing. It's development testing kind of stuff.  

It's an open-source community. So we can post our queries there. We generally get good responses from the forums. So it's good enough.

I used another tool like HP LoadRunner. And now it's offered by some different companies, ownership-wise, but it was long back. 

I would rate my experience with the initial setup an eight out of ten, where one is difficult, and ten is easy. Even for new users, the installation is straightforward based on the documentation. 

However, customizing and building something extra can be complex. But overall, it's easy enough to download and start working with.

Our main system is on the cloud, so we primarily use JMeter on the cloud. However, some use cases require on-premises deployment, and we use it there as well.

One person is enough for deployment. The deployment won't take much time. It is very fast, typically within five minutes.

We use the open-source version. 

I'll definitely advise that you should at least give it a try. If it serves your initial needs and meets your expectations, you should go for it. 

It's quite an old and up-to-mark tool with a proven track record in the industry, and there's a strong community behind it. So it's definitely worth giving a shot.

Overall, I would rate the solution an eight out of ten. 

Load testing is something that we used to do on Apache JMeter. We have regular releases every two weeks with varying features and modules. For major releases, where most of the EPS are modified or new features are introduced with new pages, we perform a performance test once every three to four months. We have a fixed schedule for performance testing before the sale, such as Thanksgiving and Black Friday, to ensure stability and performance during the sales period.

It is an open-source tool that is easy to use. It can be easily integrated with multiple tools, including Selenium. The tool also provides a recording and playback feature, along with the ability for modification and parameterization. Since it is an open-source tool, it can be easily integrated with many other tools, such as BlazeMeter, making the integration process very helpful.

The UI of the solution needs to be better. The UI takes up a lot of our bandwidth. So, we always run on the command line. Hence, improving the UI is needed. If it can be more lightweight, the editing can also be made easier.

I have been using Apache JMeter for four to five years.

Apache JMeter is a stable tool. Around 10,000-12,000 users are using the tool. One can increase its usage, but we haven't done so since we will need multiple machines to be connected to a network to generate the load. So, we haven't exceeded its use beyond 10,000 users.

Apache JMeter is a scalable tool.

We haven't used the solution's technical support.

We used other solutions in the past, but they were costly since they require a license, whereas Apache JMeter is free. Hence, we prefer Apace JMeter.

The solution's initial setup process was straightforward since setup can be done by opening a JAR file. It can be deployed in half an hour. Experienced people can deploy it in 15-20 mins, and newer ones may take up to an hour.

Also, we don't deploy on any server. Normally, the tests are on local ones.

For deployment and maintenance, you need knowledge of scripting and APIs. But otherwise, maintenance should be easy. Once you record and have the script ready, until and unless there are modifications to the application, maintaining it should not be much hassle.

The implementation was done in-house.

There is no investment as such. Only the resourcing part is there in the tool.

Apache JMeter is a free tool.

I recommend Apache JMeter to others who want to use it. I rate the overall solution an eight out of ten.