Alex Logan - PeerSpot reviewer
Senior Manager, Performance Engineering at a energy/utilities company with 10,001+ employees
Real User
Top 10
GUI needs improvement, and it is not easy to learn, it requires upfront learning
Pros and Cons
  • "The most valuable feature of this solution is that it is free."
  • "In this tool, automation in general is almost non-existent. Everything is done manually."

What is our primary use case?

We use Apache JMeter to test the performance of our platforms and applications.

What is most valuable?

The most valuable feature of this solution is that it is free. If not for that, we would be using a different tool.

What needs improvement?

The graphical user interface could be improved.

In this tool, automation in general is almost non-existent. Everything is done manually. I would advise those who put this together to try to simplify it for their end users, such as being able to automate at their desks. Such as manual relations and social management.

Purely on the feature set, it lacks automation, therefore it requires a lot of manual work.

For how long have I used the solution?

As a performance engineer, I use it almost every day.

I have been using Apache JMeter for two years.

Buyer's Guide
Apache JMeter
November 2022
Learn what your peers think about Apache JMeter. Get advice and tips from experienced pros sharing their opinions. Updated: November 2022.
654,658 professionals have used our research since 2012.

What do I think about the stability of the solution?

Apache JMeter seems to be stable enough.

What do I think about the scalability of the solution?

In terms of scalability, we haven't pushed it to a large scale yet.

The majority of users are performance engineers, but there are also some developers.

There are four performance engineers and at least three developers who use JMeter on occasion.

We intend to increase usage, but it is already being used exclusively and extensively.

How are customer service and support?

There is no technical support program for Apache JMeter. There is only an online community of people who help one another.

The community is helpful and fairly advanced.

Which solution did I use previously and why did I switch?

In the past, I used to use various tools, primarily LoadRunner.

When I changed jobs, and the new company only used JMeter because we didn't have a license for LoadRunner, I started working with Apache JMeter.

How was the initial setup?

The initial setup was very easy.

We used it on a desktop computer. It is also used in the cloud. The idea behind using it in the cloud was to spread loads across multiple machines.

What about the implementation team?

We carry out the deployment ourselves.

We have one performance engineering consultant who maintains this solution.

What was our ROI?

That would not be applicable because it is a free solution.

What's my experience with pricing, setup cost, and licensing?

Licensing costs are rare.

There are operational costs related to using Amazon Cloud, but the tool itself.

Which other solutions did I evaluate?

I looked into other options, such as Bluehost, SmartBear, Epic, and LoadRunner but Jmeter proved to be sufficient for our needs.

We chose JMeter based on its pricing structure.

What other advice do I have?

My advice is that it requires a fairly consistent amount of initial learning. It is not simple to learn.

I would rate Apache JMeter a three out of ten.

Which deployment model are you using for this solution?

Private Cloud

If public cloud, private cloud, or hybrid cloud, which cloud provider do you use?

Amazon Web Services (AWS)
Disclosure: I am a real user, and this review is based on my own experience and opinions.
Flag as inappropriate
PeerSpot user
Performance Engineer at a tech services company with 10,001+ employees
Real User
Top 5
Very user friendly and being open source makes it very beneficial for low budget projects
Pros and Cons
  • "User-friendly and open source."
  • "The plug-ins make the reports heavy and they have to be run in non-GUI mode."

What is our primary use case?

My main use case of JMeter is for web application performance testing as well as for API performance testing. We are customers of Apache. 

How has it helped my organization?

The benefit of JMeter is that it does our performance testing and provides a report without the need to spend money on a licensed tool. It's a significant benefit for us and for the project we're currently working on.

What is most valuable?

The fact that the solution is open source makes a big difference as we're working on a low-budget project. It's quite user-friendly and easy to use.

What needs improvement?

There are issues with the plug-ins which you need for reporting purposes as they make the reports quite heavy so you have to run them in non-GUI mode. If you go above the 200 user mark, the application creates a bottleneck and that's one of its major drawbacks. It means you have to run with a master-slave configuration with one system being the master, and multiple slave systems. It's not ideal and I think it could be simplified with a UI that provides direct configuration. In addition, the solution doesn't support SIP applications and some other protocols.

What do I think about the scalability of the solution?

JMeter is not designed for high loads, if you overdo it the tool becomes a bottleneck. Unless you're using JMeter in multiple systems and all the systems are connected through a particular LAN, there is a limit.

How are customer service and support?

Because it's freeware there's no official tech support but you can raise a request on the JMeter site. We haven't had many problems. 

Which solution did I use previously and why did I switch?

We were previously using LoadRunner for this project but moved to JMeter because it's a freeware testware.

How was the initial setup?

The initial setup is quite easy. There is no deployment process, you download a file from the web application service from your JMeter site, and you can use it. Any user with a little bit of knowledge can do it. The download takes about 15 minutes so you can be using JMeter in about half an hour. From time to time there are version upgrades but they don't affect existing assets. We have four people using JMeter and they don't have any issues. 

What's my experience with pricing, setup cost, and licensing?

BlazeMeter works on top of JMeter and there is a small cost factor to purchase that. It basically gives a slightly more advanced JMeter. 

What other advice do I have?

If you're working on a low-budget project and don't have dependencies of a huge number of users then this is the perfect tool. If you have 2,000 or 3,000 users then it's probably best to look at other options.

I rate this solution eight out of 10. 

Which deployment model are you using for this solution?

On-premises
Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
Buyer's Guide
Apache JMeter
November 2022
Learn what your peers think about Apache JMeter. Get advice and tips from experienced pros sharing their opinions. Updated: November 2022.
654,658 professionals have used our research since 2012.
Saroja Josephine Jillella - PeerSpot reviewer
Consultant at a computer software company with 10,001+ employees
Real User
Top 5Leaderboard
Good documentation, easy to install, and open-source
Pros and Cons
  • "It's very easy to install, and it's very easy to code and develop the script."
  • "The memory utilization in JMeter is very poor."

What is our primary use case?

The product is used for performance testing. 

What is most valuable?

The solution is open-source. 

It's very easy to install, and it's very easy to code and develop the script.

There is a lot of good documentation available. 

The solution is stable. 

What needs improvement?

The memory utilization in JMeter is very poor. The system gets hung up. I would like that to be improved.

I'd like to see better reporting. The reporting offers limited information or details for a load test. So it would be highly appreciated if JMeter can provide better reporting and support multiple protocols.

They could include a wide range of protocols, which LoadRunner already offers, while scripting. 

For how long have I used the solution?

I've worked with the solution for two years. 

What do I think about the stability of the solution?

It's a stable product. It's reliable. There are no bugs or glitches. It doesn't crash. 

What do I think about the scalability of the solution?

The solution is scalable. However, it has a lot of memory issues during scale-up. That is the one concern. It's a bit limited in terms of scalability.

I'm not sure how many users are on the solution in our company right now. However, we are a pretty big organization, and therefore, there may be quite a few. It's likely 100 or more. 

How often I use it depends on the project I am working on. When I worked on my last project, I used it daily for two years. Now, I am using Loadrunner. 

How are customer service and support?

I haven't contacted support, however, the documentation is readily available and very useful.

Which solution did I use previously and why did I switch?

I am also using LoadRunner. 

Installation-wise, LoadRunner is a bit difficult and a complex tool to install, whereas JMeter is very easy. 

JMeter is an open-source tool, so we can use it any time that we want and install it anywhere. LoadRunner is very costly. Compared to handling heavy loads and everything, LoaderRunner gives us a good advantage.

LoadRunner provides pretty good and reliable reports compared to JMeter. JMeter doesn't give as much information.

How was the initial setup?

The initial setup is straightforward and easy to implement. It's not overly complex. 

I'd rate it a five out of five in terms of ease of implementation. 

What about the implementation team?

I can do the implementation by myself.

What's my experience with pricing, setup cost, and licensing?

It is an open-source tool.

What other advice do I have?

We are customers and end-users.

I'd recommend the solution if a company is looking for something that is open-source. 

I'd rate the solution eight out of ten.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
Flag as inappropriate
PeerSpot user
Rahul Shah - PeerSpot reviewer
Principal Software Automation Engineer at a computer software company with 501-1,000 employees
Real User
Open-source, with a good online community, but the scalability, and data storage could improve
Pros and Cons
  • "The thread groups, samplers, and listeners, which are all determined by the script's requirements, are the most valuable features of this solution."
  • "There could be improvements in terms of memory utilization. We are going to migrate away from JMeter in the near future."

What is our primary use case?

We use Apache JMeter for the load generation scripts for performance testing.

What is most valuable?

The thread groups, samplers, and listeners, which are all determined by the script's requirements, are the most valuable features of this solution.

What needs improvement?

We use many plugins to customize our scripts, which is its main purpose. We wanted to be able to use a larger variety of customizable plugins to meet our needs. Along with our, JMeter, you would use a variety of plugins.

The number of customization plugins should be increased.

There could be improvements in terms of memory utilization. We are going to migrate away from JMeter in the near future.

The data storage should be improved.

Scalability could be improved.

It should support more protocols.

For how long have I used the solution?

I have been working with Apache JMeter for three or four years.

We use version 5.0

What do I think about the scalability of the solution?

We are using the cluster mode because one node is definitely not enough. Scalability can be improved.

When compared with other solutions, it's not as good, which is the reason we are moving to another solution.

Scalability must improve.

How are customer service and support?

Generally, we use open forums, to resolve any issues we may have.

Which solution did I use previously and why did I switch?

I also work with BlazeMeter.

I worked with Apache JMeter, from the beginning.

How was the initial setup?

The initial setup is straightforward.

We have a small team to maintain this solution.

What about the implementation team?

We completed the installation on our own. It was completed in-house.

What's my experience with pricing, setup cost, and licensing?

Apache JMeter is an open-source solution.

We don't use the paid version of this solution.

Everything is included, and there are no additional costs.

What other advice do I have?

I would suggest that instead of using a GUI-based implementation, try to make it code-based. Try to replicate the configuration. The plug should be job-ready, and ready to be integrated as well. Rather than having a UI, and limitations.

Three or four years ago, I would have given it a seven or eight, but now that there are more powerful competitors, I would give Apache JMeter a five out of ten.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
Flag as inappropriate
PeerSpot user
Performance Specialist at a renewables & environment company with 1,001-5,000 employees
Real User
Top 20
A free, stable, and feature-rich solution with good community support
Pros and Cons
  • "A lot of things are valuable. It is free. It has a lot of features, such as report generation and integration with CI/CD, which makes it very competitive with the other paid solutions available in the market. It is a good solution."
  • "They can improve it a little bit in terms of distribution load testing. We struggled with it during the distribution. In terms of reporting, runtime monitoring is not currently included, and it should be included. They can also improve it on the reporting side in terms of the comparison of the reports. They can also focus more on integration with CI/CD. Currently, people are using their own customized tools. It would be nice if Apache can provide some standard tools and procedures for integration with CI/CD tools like DPR. There are some tools, but it would be nice if official standard tools and procedures are available."

What is our primary use case?

I am using this solution for a banking use case. We have an older version of this solution. We are planning to upgrade to a newer version.

What is most valuable?

A lot of things are valuable. It is free. It has a lot of features, such as report generation and integration with CI/CD, which makes it very competitive with the other paid solutions available in the market. It is a good solution.

What needs improvement?

They can improve it a little bit in terms of distribution load testing. We struggled with it during the distribution. 

In terms of reporting, runtime monitoring is not currently included, and it should be included. They can also improve it on the reporting side in terms of the comparison of the reports. 

They can also focus more on integration with CI/CD. Currently, people are using their own customized tools. It would be nice if Apache can provide some standard tools and procedures for integration with CI/CD tools like DPR. There are some tools, but it would be nice if official standard tools and procedures are available.

For how long have I used the solution?

I have been using this solution for almost three months.

What do I think about the stability of the solution?

It is stable. There is good support, and fixes are also available immediately.

What do I think about the scalability of the solution?

We have not scaled it, but it is scalable. We plan to increase its usage. 

How are customer service and technical support?

I didn't have to call Apache because if anything comes up, we are able to find a solution in the community.

How was the initial setup?

Its initial setup was straightforward.

What's my experience with pricing, setup cost, and licensing?

We are using the free version, and if required, we can easily switch to the other version.

What other advice do I have?

I would definitely recommend this solution if it suits a use case. It is good, and it also looks promising. We plan to keep using it in the future.

I would rate Apache JMeter an eight out of ten.

Which deployment model are you using for this solution?

On-premises
Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
Abdulla Pathan - PeerSpot reviewer
Technology Competency and Solution Head at LearningMate
Real User
Top 5Leaderboard
It is easily deployable and extendable, but it should start supporting the presentation layer
Pros and Cons
  • "It is open source as well as relatively extendable. It allows us to extend and add additional functionality and features. Its deployment is also very easy."
  • "It should start supporting the presentation layer. It currently provides performance testing specifically at the application and API level. It can be extended to the presentation layer, which includes mainly Angular and React frameworks. It should also be easy to use and easy to train people."

What is our primary use case?

It is specifically used for performance systems. It is used for identifying the areas where we need to improve the application bottlenecks and for load testing. We are using its latest version.

How has it helped my organization?

It has helped us to build robust application cater to the learning domain and identify bottleneck prior go live. It helped us refine our deployment strategy and capacity planning.

What is most valuable?

It is open source as well as relatively extendable. It allows us to extend and add additional functionality and features. Its deployment is also very easy.

What needs improvement?

It should start supporting the presentation layer. It currently provides performance testing specifically at the application and API level. It can be extended to the presentation layer, which includes mainly Angular and React frameworks. 

It should also be easy to use and easy to train people.

For how long have I used the solution?

I have been using this solution for more than ten years.

What do I think about the stability of the solution?

It is stable.

What do I think about the scalability of the solution?

It is scalable. It allows us to extend and add additional functionality and features.

We have around 10 to 15 people who use this solution.

How are customer service and technical support?

It is open source, so I don't think any support is available.

Which solution did I use previously and why did I switch?

Load Runner, replaced with JMeter due to lower ROI

How was the initial setup?

Its deployment is easy. It didn't take much time. It took less than 15 minutes.

What about the implementation team?

We deployed it on our own.

What was our ROI?

1. Scalable Product and solutions

2. Plug and Play with CICD process

3. Reduction in licence cost 

What's my experience with pricing, setup cost, and licensing?

No Licensing cost for JMeter

Which other solutions did I evaluate?

Yes.

What other advice do I have?

I would recommend this solution. We plan to keep using this solution.

I would rate Apache JMeter a seven out of ten.

Which deployment model are you using for this solution?

Private Cloud
Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
Technical Specialist at a financial services firm with 10,001+ employees
Real User
Top 20
Contains all the features required for performance metrics and is very user friendly
Pros and Cons
  • "Very user-friendly and easy to use."
  • "It has some proxy-based dependencies which require specific proxies to be set up or disabled, which causes problems."

What is our primary use case?

We are in the financial industry in India and carry out performance testing. We deal with  SWIFT messages and financial messaging systems. A few years ago we had 2 million transactions in five hours; today it's 2 million in two hours. To measure these performances, I use JMeter for regression. We have Exadata, Exalogic infrastructure. We are users of JMeter and I'm a technical specialist. 

What is most valuable?

This tool is very user-friendly and easy to use. It's open-source so there are no costs involved for the non-production environments. It's easily available to anyone who wants to use it and it has all the features required for performance metrics.

What needs improvement?

I think it has some proxy-based dependencies which require specific proxies to be set up or disabled, which causes problems when we are working in certain specific environments that have a proxy setup. When we want it to do a record with some new scripts, there are some challenges there.

For how long have I used the solution?

I've been using this solution for two years. 

What do I think about the stability of the solution?

The solution is stable and easily scalable. 

How are customer service and technical support?

The support is very good, with good technical teams. Whenever we get stuck they support us and provide solutions. 

How was the initial setup?

The initial setup took only 10 minutes but there were some complications with the product initially having less features. That has been greatly improved. It's easy to do maintenance of the test tools which we develop, and it's easy to maintain as a product. We have around 20 users of this solution. 

What's my experience with pricing, setup cost, and licensing?

It's open-source, so there's no cost. If you need some support from the vendors there will be a charge for that. 

What other advice do I have?

I would recommend this solution. Most of the support, guides, and tutorials are available on YouTube. 

I rate this solution an eight out of 10. 

Which deployment model are you using for this solution?

On-premises
Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
AlexLogan - PeerSpot reviewer
Tech Lead, Performance Engineering at ENEL
Real User
Top 20
It's a free tool with a vast knowledge base, but the reporting is lackluster, and it has a steep learning curve
Pros and Cons
  • "JMeter is a free tool with a large user population, which comes in handy because we have a vast knowledge base to tap into when needed. It's also easier to hire consultants who know JMeter."
  • "JMeter's reporting is extremely rudimentary. The fundamental reporting mechanisms need to be drastically improved. It doesn't utilize an automatic session management mechanism or methods other tools use like parsing cookies and variables. Everything needs to be done manually. There's no automation."

What is our primary use case?

JMeter is used by a team of four performance engineers. We monitor tasks on the API level, so all of the requests are hitting individually.

How has it helped my organization?

JMeter has helped us streamline our applications by identifying multiple bottlenecks in our systems and exercising tasks against APIs.

What is most valuable?

JMeter is a free tool with a large user population, which comes in handy because we have a vast knowledge base to tap into when needed. It's also easier to hire consultants who know JMeter.

What needs improvement?

JMeter's reporting is extremely rudimentary. The fundamental reporting mechanisms need to be drastically improved. It doesn't utilize an automatic session management mechanism or methods other tools use like parsing cookies and variables. Everything needs to be done manually. There's no automation.

For how long have I used the solution?

We've been using JMeter for the past three years.

What do I think about the stability of the solution?

JMeter never crashed on us. It's highly stable.

What do I think about the scalability of the solution?

JMeter is highly scalable.

How are customer service and support?

There is no technical support for JMeter.

Which solution did I use previously and why did I switch?

We previously used LoadRunner, but we switched because the organization had an accumulated knowledge of JMeter. Nobody really knew LoadRunner. However, the biggest advantage is that it's free, and we didn't have to invest in acquiring a performance testing solution.

How was the initial setup?

JMeter is simple to install. We did it in-house, and it takes a day or two to deploy it to the cloud. There's no maintenance required. 

Which other solutions did I evaluate?

We looked at a number of different tools.

What other advice do I have?

I rate JMeter five out of 10. You need knowledgeable employees to work with the tool because it's difficult to learn. There is a steep learning curve, so you shouldn't expect an entry-level engineer to pick up the tool quickly and become proficient. It takes a considerable amount of time.

Which deployment model are you using for this solution?

On-premises
Disclosure: I am a real user, and this review is based on my own experience and opinions.
Flag as inappropriate
PeerSpot user
Buyer's Guide
Download our free Apache JMeter Report and get advice and tips from experienced pros sharing their opinions.
Updated: November 2022
Buyer's Guide
Download our free Apache JMeter Report and get advice and tips from experienced pros sharing their opinions.