PankajSingh4 - PeerSpot reviewer
Senior Specialist at Qualitest
Real User
Top 5Leaderboard
Simple to configure, easy cloud deployment, and plenty of resources
Pros and Cons
  • "The most valuable features of Apache JMeter are user-friendliness, large resource, and the quality of assistance they provide. Additionally, it is easy to integrate with cloud platforms, such as AWS."
  • "Apache JMeter could use improvement in reporting. Currently, it isn't easy to generate reports in PDF format. While receiving reports in PDF format is possible, it requires a lot of customization. Additionally, when comparing the load test to others solutions it could improve."

What is our primary use case?

Apache JMeter is utilized for load testing of applications such as APIs and web services. Various types of load testing can be conducted, including stress tests and load tests. It supports different types of protocols, such as HTTP, HTTPS, SFTP, and others.

What is most valuable?

The most valuable features of Apache JMeter are user-friendliness, large resource, and the quality of assistance they provide. Additionally, it is easy to integrate with cloud platforms, such as AWS.

What needs improvement?

Apache JMeter could use improvement in reporting. Currently, it isn't easy to generate reports in PDF format. While receiving reports in PDF format is possible, it requires a lot of customization. Additionally, when comparing the load test to others solutions it could improve.

For how long have I used the solution?

I have been using Apache JMeter for approximately five years.

Buyer's Guide
Apache JMeter
April 2024
Learn what your peers think about Apache JMeter. Get advice and tips from experienced pros sharing their opinions. Updated: April 2024.
768,415 professionals have used our research since 2012.

What do I think about the stability of the solution?

Apache JMeter is a stable solution.

What do I think about the scalability of the solution?

We have more than 100 people using Apache JMeter in my organization.

We have multiple clients using this solution and all of our testers are using the solution.

My advice to others is for them to go through online tutorials before using the solution. It is easy to use, and anyone can quickly learn and test their application.

I rate the scalability of Apache JMeter an eight out of ten.

How are customer service and support?

I have not contacted technical support.

How was the initial setup?

The initial setup of Apache JMeter is easy and simple to configure. The whole process of implementation takes approximately half an hour. The first step is to download Java and install it. The last step is to download Apache JMeter and install it. 

What's my experience with pricing, setup cost, and licensing?

The main reason we chose Apache JMeter is that it is cost-effective and easy to use. There is no need to pay for additional services. Additionally, it does not require additional payment to vendors. The solution is open-source and free.

What other advice do I have?

I rate Apache JMeter a nine out of ten.

Which deployment model are you using for this solution?

Public Cloud

If public cloud, private cloud, or hybrid cloud, which cloud provider do you use?

Amazon Web Services (AWS)
Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
Tushar Shankar - PeerSpot reviewer
Tech Lead - QA at a financial services firm with 10,001+ employees
Real User
Top 10
An excellent testing solution, that is easy to use, but suffers performance issues when over-scaled
Pros and Cons
  • "This solution is easier to use than any other tool in the market; there is not even a requirement to learn a lot of scripting in order to use it."
  • "At present, if the number of virtual users increases beyond 10,000 when testing, then it results in a Java heap which causes the solution to crash."

What is our primary use case?

We use this solution for functional API testing, and performance testing.

How has it helped my organization?

This solution allows us to test all the functional APIs through it, then record and run the APIs.

What is most valuable?

This solution is easier to use than any other tool in the market; there is not even a requirement to learn a lot of scripting in order to use it.

We also appreciate that the solution is open source, so it has a very big community to utilize if we need help with an issue.

What needs improvement?

At present, if the number of virtual users increases beyond 10,000 when testing, then it results in a Java heap which causes the solution to crash.

For how long have I used the solution?

I have been working with this solution for around seven years.

What do I think about the stability of the solution?

The stability of this product is good, outside of the issues with adding virtual users during testing.

What do I think about the scalability of the solution?

This solution is scalable, but only up to a certain point when adding virtual users, before performance issues begin to occur.

How was the initial setup?

The initial setup of this solution was quite straightforward, and deployment only took one to two minutes.

What about the implementation team?

We implemented this solution using our in-house team.

What's my experience with pricing, setup cost, and licensing?

This is an open-source solution, so there are no licensing costs associated with it.

What other advice do I have?

We would recommend this solution to any QA or developer as a 'must-try product'.

I would rate this solution an eight out of ten.

Which deployment model are you using for this solution?

On-premises
Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
Buyer's Guide
Apache JMeter
April 2024
Learn what your peers think about Apache JMeter. Get advice and tips from experienced pros sharing their opinions. Updated: April 2024.
768,415 professionals have used our research since 2012.
Product Manager at Worex Technology
Real User
Top 5Leaderboard
It's a free, scalable tool that's good for checking backend services
Pros and Cons
  • "I use all the tools, but one feature that stands out is JMeter's ability to test when services are sending a particular kind of request. We are using specific ports to send queries, and assess the performance based on the time it takes these queries to respond. You can use it with stuff other than the web performance."
  • "The UI needs some work. The first time I used JMeter, I couldn't record the full scenario to mimic the user experience. Since then, they have introduced some plugins and a third-party tool called BlazeMeter."

What is our primary use case?

I use JMeter for concurrency and some backend services. We are also using JMeter to apply our tests within our framework by sending some requests to JMeter to assess our application's performance. My company is developing software based on data analytics that produces insights and graphs like Tableau or Power BI. We use JMeter to test the graphs, how the tables are rendered, and how long it takes to render some applications. 

For instance, we have a problem with one of our clients that use some complex scenarios or queries that take too much time to retrieve from the application or render to the end user. We ran a couple of tests on the application using JMeter and spotted the service or request that was taking too much time.We alerted the developers about this, and they took the appropriate action to fix this problem. Afterward, we run another cycle of JMeter to ensure everything is working as expected.

There are two modules. The analytics module is the interface for the stakeholders and company decision-makers. The number of users isn't that big, unlike the retail applications websites. It's 500 users at most. The other module is deals with data volume. We are currently doing some POCs to check whether we will benefit from JMeter in this area because it's not a concurrency issue. It's a data volume issue.

We have a hybrid deployment because we are using JMeter to assess the performance of our products. If the product is deployed on the cloud, we use JMeter on the cloud. If it's on-prem, we are using it on-prem. At my previous company, we mainly used JMeter on-prem.

I expect that we will continue to rely more on JMeter, and we have multiple DevOps pipelines using JMeter to test another module in our application.

What is most valuable?

I use all the tools, but one feature that stands out is JMeter's ability to test when services are sending a particular kind of request. We are using specific ports to send queries, and assess the performance based on the time it takes these queries to respond. You can use it with stuff other than the web performance.

What needs improvement?

The UI needs some work. The first time I used JMeter, I couldn't record the full scenario to mimic the user experience. Since then, they have introduced some plugins and a third-party tool called BlazeMeter. It's working on this, actually.

It's an excellent plugin that you can use to record the scenario from Google Chrome, and it integrates easily into JMeter. They could also make it easier to generate the built-in report. Now, you run the tests and generate the charts in a separate column. The graphs and charts that display the test metrics could be better.

I worked with another tool called Web Performance Tester, and its interface is better than JMeter's. They have intuitive graphs while you are running the tests, so you can see how things are going. It shows you the number of concurrent users logged into the system, the number of failures, response times, etc.

For how long have I used the solution?

I worked in performance testing from 2011 until 2019. I was working with another tool, but in the last few years, I started using JMeter for a couple of projects.

What do I think about the stability of the solution?

It's hard to say definitively. When there's something wrong with JMeter, we're unsure whether it's something in the device because there are lots of requests coming from the machine. Maybe we didn't use it that many times from our local server. There are some bottlenecks, but in many cases, we only need to restart JMeter, and it works fine.

What do I think about the scalability of the solution?

JMeter is scalable. I did a test using it in a distributed environment with more than one client. We used JMeter to load from the server and also the plugin tool. It works fairly well in a complex environment. I still need to research the maximum concurrent JMeter can handle. Is it 1 billion or 10 million? 

In my work experience, I need to load maybe 1 million users. For example, most of my work is on the backend of e-commerce websites with a maximum of 1,000 users. We have many clients, but the load is not that big. 

How are customer service and support?

I don't think JMeter has technical support because it's an open-source application, but there is support for third-party apps that use JMeter. For instance, BlazeMeter has good technical support.

Which solution did I use previously and why did I switch?

I have more experience in a tool called Web Performance Tester, but it has some limitations. For example, it doesn't support some forms. A long time ago, their support was better, but lately, their support hasn't been so good. They always ask us to switch to the consultancy service and they don't give us the solution. If we face problems, they always tell us, "Okay. You can consult us." This is not a good practice, actually.

Also, Web Performance Tester isn't well-known, so some of our clients aren't confident in it because it doesn't have a reputation.  They were more familiar with JMeter, and it's something I've used.  However, I have no problem with Web Performance Tester. It's hard to compare the two tools because it's not apples to apples. 

I joined this company in October, and they had just started using JMeter. Previously, they were using a tool that they had developed in-house. They found that JMeter has more capabilities and specificity than the tools they were using.

How was the initial setup?

Setting up JMeter is straightforward. It doesn't need an installation like other tools. There is a batch file for Windows. Around four people in my organization are responsible for maintaining and managing JMeter.

What's my experience with pricing, setup cost, and licensing?

JMeter is a free open-source tool. There are some third-party tools built on top of JMeter that have a license or something like BlazeMeter. I think you can also purchase some additional services.

What other advice do I have?

I rate Apache JMeter eight out of 10.  Before using JMeter, you should find some tutorials because you need to be trained to use it. It's not a plug-and-play tool. This is what I did. Spend some time researching JMeter's capabilities. We met a lot of people who knew about JMeter, but when you tell them that it can do something, they're not aware of it. 

For instance, through taking some courses, we learned that JMeter has distributed performance capabilities. Some people still know that you can't make these recordable scenarios. This information hasn't reached everyone in the IT market.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
GeetikaVerma - PeerSpot reviewer
Application Development Analyst at a financial services firm with 10,001+ employees
Real User
Top 10
Recommended for usability but struggles to handle bigger loads
Pros and Cons
  • "The most valuable features are the ability to capture the entire traffic of particular pages and the proper readability of entire pages and entire APIs."
  • "One of the drawbacks of JMeter is that it can't handle a large amount of load, which forces us to switch to other tools when we need to load more than a 5,000 or 10,000 user load."

What is our primary use case?

I mainly use JMeter to capture the traffic of the most-visited page to see how much load a particular page is getting and how many users are using that page for a particular amount of time. I've also used it to capture APIs for particular pages.

What is most valuable?

The most valuable features are the ability to capture the entire traffic of particular pages and the proper readability of entire pages and entire APIs.

What needs improvement?

One of the drawbacks of JMeter is that it can't handle a large amount of load, which forces us to switch to other tools when we need to load more than a 5,000 or 10,000 user load. In the next release, I would like JMeter to be more compatible with other languages in the market.

For how long have I used the solution?

I've been using JMeter for six to eight months.

What do I think about the stability of the solution?

JMeter can't be used in the long run, so I'd rate its stability as five out of ten.

What do I think about the scalability of the solution?

I would rate JMeter's scalability as seven out of ten.

How are customer service and support?

Apache's technical support is pretty good, I've had no issues with them.

How was the initial setup?

The initial setup was straightforward and took about five to six months.

What about the implementation team?

I used an in-house team.

What's my experience with pricing, setup cost, and licensing?

JMeter is open source, so there are no licensing costs associated with it.

Which other solutions did I evaluate?

I evaluated SoapUI, Postman, and Visual Studio. JMeter was more reliable compared to these options.

What other advice do I have?

I would definitely recommend JMeter in terms of usability. If you're using AngularJS as a language for testing UIs, JMeter might not be a good idea. I'd rate this solution as six out of ten.

Which deployment model are you using for this solution?

On-premises
Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
Hind Zahraoui - PeerSpot reviewer
Digital Project and Quality Manager at a transportation company with 5,001-10,000 employees
Real User
Top 10
Provides efficient stability and good technical support services
Pros and Cons
  • "It gives accurate results and recommendations that we can implement to enhance the performance of websites."
  • "Apache JMeter could be a more user-friendly product from the end user's perspective."

What is our primary use case?

We use Apache JMeter for performance testing, including the response time of our websites for multiple users.

What is most valuable?

The product is specialized in performance testing. It gives accurate results and recommendations that we can implement to enhance the performance of websites.

What needs improvement?

Apache JMeter could be a more user-friendly product from the end user's perspective. It requires someone with technical knowledge to administer it. This particular area needs improvement.

For how long have I used the solution?

We have been using Apache JMeter for a year.

What do I think about the stability of the solution?

I rate Apache JMeter's stability a ten out of ten.

What do I think about the scalability of the solution?

We have around 12 Apache JMeter users in our organization. I rate the product's scalability an eight out of ten.

How are customer service and support?

The technical support team responds quickly to queries.

How was the initial setup?

The initial setup process is very complicated for the on-premises version. It would be better to use the cloud version.

What's my experience with pricing, setup cost, and licensing?

The product has reasonable pricing.

What other advice do I have?

It is a complete tool that integrates lots of functionality. I recommend it to others and rate it a nine out of ten.

Which deployment model are you using for this solution?

On-premises
Disclosure: I am a real user, and this review is based on my own experience and opinions.
Flag as inappropriate
PeerSpot user
RajeevSAwant - PeerSpot reviewer
Head Automation CoE at Truglobal
Real User
Top 5Leaderboard
Free, commonly used, and good for web API testing
Pros and Cons
  • "It's a free tool."
  • "The UI could be better."

What is our primary use case?

We primarily use the solution for performance and delivery testing.

What is most valuable?

It's a free tool. It's a very common tool. There's a lot of support in the community for this. It mainly supports web API testing.

What needs improvement?

The UI could be better. It can have some Reach UI also, which would be helpful, and maybe a relatively simpler way of using it. It needs simple modules. There are quite a lot of things which are kind of abandoned, so they can definitely improve on it.

Integration with some of the other features should be managed. However, it's open source, so there is not much to complain about there.

It's an open-source tool; we cannot ask for additional features really.

The product could use some kind of filtering and monitoring and different degree of dashboards and analysis. If that can be provided, that would be very, very helpful.

For how long have I used the solution?

I’ve used the solution since 2009.

What do I think about the stability of the solution?

The solution is relatively stable. There are no bugs or glitches, and it doesn’t crash o freeze. It’s reliable.

What do I think about the scalability of the solution?

It can scale. However, what others say about scalability is that you need to have some proper calculations to be done first.

How are customer service and support?

We have dealt with support. Technical support is found in more technical forums. It’s open-source, and communes are where you go for information. We have received good responses when we have inquiries. There are quite a lot of forums in general available.

Now, as we have grown as an organization or as a team, there are still questions such as, "What are the limitations of this tool?" And we put that to JMeter so we can learn what is best for the maintenance.

How was the initial setup?

The installation can be a bit complex. There are quite a lot of things and issues if you go deep into it and if you're setting up JMeter. For example, direct script captioning is slightly different. It's a bit more complex, the correlation parametrization. Setting up the workload model can be complex as it is based on a Java service mechanism.

It becomes a bit challenging to manage. If I want to put a 50,000 user load, I have to be very careful how the memory is utilized. I must be very aware of the underlying system's capability to execute this. I have to make it into multiple nodes and run them in parallel. There are some calculations, and there are some good power processes that will be required when you are using JMeter. You need to understand its limitations and load work them on.

The solution doesn’t really need maintenance. They're open-source tools, so we don't expect any maintenance. What we typically do is we pick up a particular version and understand the limitations of it. We then play within those limitations.

What about the implementation team?

We handled the initial setup in-house. I handled the setup myself.

What's my experience with pricing, setup cost, and licensing?

The solution is free to use.

What other advice do I have?

We may be using the latest version. I don’t know the exact version number.

It is on-prem. However, we can put it on the cloud as well. We install it on any machine, so it can be a local or cloud-based machine.

I’d recommend JMeter. One critical piece of advice is to plan properly. For JMeter, planning and having an understanding of the limitations are important. If you play within those limitations, it is a really great tool. There are trade rules that apply to a lot of things that we use. However, there's plenty of material available so far. Whoever is doing it, can plan it accordingly. They can create those tests, and execute those tests. That said, at the same time, be aware of the scalability of the JMeter. For example, a single JMeter line, which is running, can take 1,000 to 2,000 threats, not more than that. Again, it depends on the available hardware. If you want to scale to, say, one million or something, different parts of planning are needed.

I’d rate the solution eight out of ten.

Which deployment model are you using for this solution?

On-premises
Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
Hemant Chaudhari - PeerSpot reviewer
Senior Consultant Specialist at HSBC India
Real User
Top 20
It's easy to configure and adjust our virtual users, but we have performance problems with higher loads
Pros and Cons
  • "JMeter lets us generate virtual users and T-load, per our requirements. It's easy to configure and adjusting the virtual users according to the DPS we want to achieve."
  • "It should be easier to combine multiple scripts. If you have multiple scripts, you need to write a new script to combine those scripts. The virtual user generator is slow."

What is our primary use case?

We use JMeter for performance testing. Our testing team has 50 to 60 people.

What is most valuable?

JMeter lets us generate virtual users and T-load, per our requirements. It's easy to configure and adjusting the virtual users according to the DPS we want to achieve.

What needs improvement?

It should be easier to combine multiple scripts. If you have multiple scripts, you need to write a new script to combine those scripts. The virtual user generator is slow.

For how long have I used the solution?

We've been using JMeter for six months.

What do I think about the stability of the solution?

With an on-premises deployment, we run into some performance problems when we have a high load. It's not that stable if we increase the scripts or the load. But that could be something in our environment.

What do I think about the scalability of the solution?

JMeter is an on-premise solution, so it has limited scalability. We are looking for a cloud-based solution that is scalable and elastic. 

What other advice do I have?

I rate JMeter six out of 10.

Which deployment model are you using for this solution?

On-premises
Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
Azure Cloud Test Manager at a tech services company with 201-500 employees
Real User
Great community support, industry-standard, and works well
Pros and Cons
  • "The biggest thing I liked about it is that there is a huge user base out there, and being shareware and being Apache, if I have any question on how to get something done, I get 18 different answers. Out of those, there would be at least a few good approaches for what I was trying to do. So, the support system out there is most valuable."
  • "I sometimes found the documentation to be not as explanatory as I would've liked it. In the cases that I can think of, I was looking for a rather hand-holding approach with Step A, B, and C, but then I realized that with a product that is open source like this, you can't do handholding. That is because there are so many different uses and different unique environments and setups for it, but I remember thinking a few times that if they only just said this."

What is our primary use case?

I've used JMeter in conjunction with Selenium, Java, and Log4j for logging. I used it before
I ended that contract in August. Its version was up to date at that time.

It was used for an e-commerce site that is specialized in C-PAP or weaving machines in effect. Their max was a thousand people logged in at once. I, of course, pushed the boundaries on that, but it was to test the performance of the website, and of course, I'm had to try subsystems, database interactions, etc.

What is most valuable?

I'm a total geek, so I liked the fact that I got to program. The biggest thing I liked about it is that there is a huge user base out there, and being shareware and being Apache, if I have any question on how to get something done, I get 18 different answers. Out of those, there would be at least a few good approaches for what I was trying to do. So, the support system out there is most valuable.

What needs improvement?

I sometimes found the documentation to be not as explanatory as I would've liked it. In the cases that I can think of, I was looking for a rather hand-holding approach with Step A, B, and C, but then I realized that with a product that is open source like this, you can't do handholding. That is because there are so many different uses and different unique environments and setups for it, but I remember thinking a few times that if they only just said this.

If I were going to be Mr. Selfish and say anything I want, I'd say a full feature GUI that lets me drag and drop different modules in line. It could have a simple-to-use GUI.

For how long have I used the solution?

I have been using this solution for probably a year and a half.

What do I think about the stability of the solution?

I didn't have any issues with the stability of JMeter itself. There were definitely issues with the program I was testing, but that's why I was testing it.

What do I think about the scalability of the solution?

It was very easy to scale, but I was barely scratching the surface. I have spent 17 years at Microsoft, and for the performance testing that we did there, we had 8.3 million users at once, as opposed to a maximum of a thousand. If I'm scaling, I have to do it quite straightforward and simple, but it was very minimal.

Only I was using it. It was the QA department. I showed it to some of the devs, and they were very interested. A couple of them tried it, but none were actually using it day-to-day for testing out the environment.

How are customer service and technical support?

I would rate them an eight out of 10 because sometimes, they would take two or three days to get back to me. Of course, at that point, you're like, "I need the answer; I need to answer." So, it was a little bit unrealistic in terms of expectations.

Which solution did I use previously and why did I switch?

At that particular company, I was the one who was tasked with coming up with the solution, so that was the only one that I looked at simply because JMeter is industry standard, but at Microsoft, they wrote their own custom tools, so I used custom Microsoft tools.

How was the initial setup?

It was rather complex. It is a complex product, but that part of it was very well-documented. I didn't have any problems with it.

What other advice do I have?

Don't be shy in asking questions. Google/Bing is your friend. It is complicated. There's no reason to spend eight hours trying to figure out something, except unless you are trying to learn in-depth. There are a lot of people who've done exactly what you're trying to do, and it doesn't matter what it is.

I would rate it a 10 out of 10 because it is industry standard. It did everything I could've asked. I barely scratched the surface, but what I needed it for, it did well and in a very straightforward-to-implement way.

Which deployment model are you using for this solution?

On-premises
Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
Buyer's Guide
Download our free Apache JMeter Report and get advice and tips from experienced pros sharing their opinions.
Updated: April 2024
Buyer's Guide
Download our free Apache JMeter Report and get advice and tips from experienced pros sharing their opinions.