Prashanth Hallur - PeerSpot reviewer
Vice President at Narwal
Real User
Top 20
Provides good metrics and allows you to write your custom code, but reporting could be improved
Pros and Cons
  • "The metrics part of it and the ability to write your custom code to do some specific tests in the performance testing space are the most valuable features."
  • "Its reporting could be improved. There should be a better visual representation. That would be helpful for easy consumption of the reports."

What is our primary use case?

We are in the service industry. We implement it for our customers. We recommend the right tool and set it up for them. So, I've not had any hands-on experience in my current role, but I have a good understanding or a fair idea of the tool's capabilities. I have a team that takes care of the technical aspects.

It is an open-source solution. So, typically when you don't want to make a heavy investment, and you want to do some level of performance testing, Apache JMeter is used. 

It is typically on-premises, and it has also been on the public cloud. It could be Azure, or it could be AWS. It is very rarely on GCP.

How has it helped my organization?

It helps them to look into several parameters. For example, when you have certified test cases that are predominantly repeated on an application, you can execute the same thing with increased load. You can see how the application responds and if there is an impact on the response time of the application. You can confine it to certain parameter conditions and then start making changes to see how it performs. You can see where the RAM or CPUs are stagnant and not increased.

What is most valuable?

The metrics part of it and the ability to write your custom code to do some specific tests in the performance testing space are the most valuable features.

It is easy to use. If you want to test your application out and not incur a lot of costs, it is probably the best tool.

What needs improvement?

Its reporting could be improved. There should be a better visual representation. That would be helpful for easy consumption of the reports.

Buyer's Guide
Apache JMeter
March 2024
Learn what your peers think about Apache JMeter. Get advice and tips from experienced pros sharing their opinions. Updated: March 2024.
765,234 professionals have used our research since 2012.

For how long have I used the solution?

I have been using it for about four to five years.

What do I think about the stability of the solution?

JMeter is probably good for lower loads. It is not comparable to LoadRunner when it comes to higher loads.

What do I think about the scalability of the solution?

It probably scales up to a few thousand users but not beyond that.

How are customer service and support?

Because it is an open-source community, their support is probably average. It won't be like the support for a commercial product. I would rate it a six out of ten.

Which solution did I use previously and why did I switch?

LoadRunner is one of the prominent tools. It was formerly HP, and now it is Micro Focus. It has good capabilities and features. It also has decent reporting capabilities. Because of the brand and the capability, it was probably chosen by most of the Fortune 500 clients that we work with. There are also some startup communities or organizations that ventured into other solutions, such as JMeter.

In terms of comparison, primarily, there are three to four parameters. The first one is the ease of use. The second one is about the protocols that need to be tested, whether it's web or API, HTTP, HTTPS, and all the native things. The third one is in terms of flexibility in setting it up and executing, and the fourth one is in terms of monitoring the execution and reporting pieces. Those are the key parameters for pros and cons. LoadRunner gives you a lot more capability and flexibility, but at the same time, it also consumes a lot of resources. JMeter is relatively simpler, cheaper, and easier to use.

How was the initial setup?

I don't have hands-on experience with it, but based on what I have heard from people, it's pretty straightforward in terms of the setup. 

The setup probably takes a week or two, and then the execution is probably a three to four weeks exercise.

What about the implementation team?

We are in the services business. Clients give us access, and we set it up there.

What was our ROI?

It is certainly good for testing out the applications for performance testing, especially when you have to test them out frequently and make sure that they are good for at least a few thousand users. It has a decent ROI.

What's my experience with pricing, setup cost, and licensing?

It is open source. There are no licensing costs associated. If you need enterprise support, you'll probably end up paying for a license.

You would also factor in the infrastructure cost, but that's not significant.

What other advice do I have?

I would rate it a seven out of ten. It is a decent choice from a small-scale perspective, but reporting could be better. If you want to get some performance testing done without spending money, JMeter is probably the best tool. It doesn't have the best reporting, but it is quite a handy tool.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
Harish Kawade - PeerSpot reviewer
Performance Specialist at DKATALIS
Real User
Top 5
A free, stable, and feature-rich solution with good community support
Pros and Cons
  • "A lot of things are valuable. It is free. It has a lot of features, such as report generation and integration with CI/CD, which makes it very competitive with the other paid solutions available in the market. It is a good solution."
  • "They can improve it a little bit in terms of distribution load testing. We struggled with it during the distribution. In terms of reporting, runtime monitoring is not currently included, and it should be included. They can also improve it on the reporting side in terms of the comparison of the reports. They can also focus more on integration with CI/CD. Currently, people are using their own customized tools. It would be nice if Apache can provide some standard tools and procedures for integration with CI/CD tools like DPR. There are some tools, but it would be nice if official standard tools and procedures are available."

What is our primary use case?

I am using this solution for a banking use case. We have an older version of this solution. We are planning to upgrade to a newer version.

What is most valuable?

A lot of things are valuable. It is free. It has a lot of features, such as report generation and integration with CI/CD, which makes it very competitive with the other paid solutions available in the market. It is a good solution.

What needs improvement?

They can improve it a little bit in terms of distribution load testing. We struggled with it during the distribution. 

In terms of reporting, runtime monitoring is not currently included, and it should be included. They can also improve it on the reporting side in terms of the comparison of the reports. 

They can also focus more on integration with CI/CD. Currently, people are using their own customized tools. It would be nice if Apache can provide some standard tools and procedures for integration with CI/CD tools like DPR. There are some tools, but it would be nice if official standard tools and procedures are available.

For how long have I used the solution?

I have been using this solution for almost three months.

What do I think about the stability of the solution?

It is stable. There is good support, and fixes are also available immediately.

What do I think about the scalability of the solution?

We have not scaled it, but it is scalable. We plan to increase its usage. 

How are customer service and technical support?

I didn't have to call Apache because if anything comes up, we are able to find a solution in the community.

How was the initial setup?

Its initial setup was straightforward.

What's my experience with pricing, setup cost, and licensing?

We are using the free version, and if required, we can easily switch to the other version.

What other advice do I have?

I would definitely recommend this solution if it suits a use case. It is good, and it also looks promising. We plan to keep using it in the future.

I would rate Apache JMeter an eight out of ten.

Which deployment model are you using for this solution?

On-premises
Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
Buyer's Guide
Apache JMeter
March 2024
Learn what your peers think about Apache JMeter. Get advice and tips from experienced pros sharing their opinions. Updated: March 2024.
765,234 professionals have used our research since 2012.
Quality Analyst at Hiup Solution
Real User
Top 10
A stable and scalable solution useful for load testing
Pros and Cons
  • "Due to process automation, I don't have to prepare reports, making it the perfect solution."
  • "At times when we overload the application, it gets stuck...After the solution gets stuck due to overloading, we have to restart our computers. In short, the solution keeps crashing."

What is our primary use case?

I use the solution in our organization for load testing.

What is most valuable?

The solution's automation capabilities are one of the best parts of the solution. Due to process automation, I don't have to prepare reports, making it the perfect solution. Whenever I require a report, the report gets generated by the solution, and then afterwards, I can look at the reports before copying and pasting them.

What needs improvement?

At times when we overload the application, it gets stuck. Especially with load testing, it gets stuck when we deal with huge numbers of 100 to 1,000-10,000 or 2,000 to 20,000. I believe that we should be able to hit around a million people without any issues. After the solution gets stuck due to overloading, we have to restart our computers. In short, the solution keeps crashing.

For how long have I used the solution?

Apache JMeter is a simple solution, and I have been using it for seven years. Also, I am using the solution's latest version.

What do I think about the stability of the solution?

After an update in the application, the stability gets hampered if we continue to use its non-updated version, causing the solution to crash. So, once I faced that problem since I was not updating the application, it crashed. Back then, I wasn't aware that an update was available for the solution, which I had to apply to the solution. So, for two or three days, I tried to solve the issue with the solution. Later, I found that I just needed to update the application. So, after I updated the application, it started working fine. I rate the solution's stability a seven out of ten.

What do I think about the scalability of the solution?

Since Apache JMeter is a scalable product, I rate the solution a ten out of ten. I am the only person using the solution in my organization.

How are customer service and support?

Since I have experience with the solution's customer support team, I rate the technical support of the solution an eight out of ten.

How would you rate customer service and support?

Positive

How was the initial setup?

On a scale of one to ten, where one is difficult, and ten is easy, I rate the solution's initial setup a ten out of ten.

The solution's deployment process took only a couple of hours. I believe it took me around an hour or so at the beginning during the initial setup phase. Right now, I can deploy the solution in fifteen to twenty minutes.

Maintenance for the application is only required when the application crashes due to its overuse.

What's my experience with pricing, setup cost, and licensing?

Since we are using an outsourced solution, it is not paid for by our company.

What other advice do I have?

My words to those planning to use the solution is that if the crashing issues in the solution get resolved, it is one of the perfect solutions in the market. Also, I have not used the paid version, so I have no idea how it works. It's a very resourceful solution.

I need to use the paid version to be able to comment on it. Since we are using the free version in a company, even if the solution makes any mistakes, we ignore those and go ahead.

Overall, I rate the solution a ten out of ten.

Which deployment model are you using for this solution?

Hybrid Cloud
Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
Quality Assurance Test Manager at a printing company with 5,001-10,000 employees
Real User
Cost-effective and straightforward to deploy, but the reporting needs to be improved
Pros and Cons
  • "The recording and playback functionality is helpful."
  • "The reporting is not very good."

What is our primary use case?

When I was last using JMeter, we were simulating 200 concurrent users and evaluating performance based on transaction times. We were defining SLAs based on the results.

Essentially, we created load scenarios and testing different ones using different workload models.

What is most valuable?

The recording and playback functionality is helpful.

What needs improvement?

The reporting is not very good.

When we run with multiple users, it takes a lot of memory.

With respect to the recording and playback functionality, the auto-correlation parameterization is not easy and should be improved.

For how long have I used the solution?

I have been using Apache JMeter for about four years.

What do I think about the stability of the solution?

There are issues with stability when running with multiple users because it consumes a lot of memory.

What do I think about the scalability of the solution?

Scalability is fine, although it is important to remember that JMeter doesn't run on its own. It needs to work with load-generations such as BlazeMeter. LoadRunner is the same in that you need a cloud-based infrastructure to run it.

How are customer service and technical support?

There is no official support. There is a forum where you can ask questions and they respond to you, but the technical support that we have with LoadRunner or NeoLoad is not available.

Which solution did I use previously and why did I switch?

I have used many similar solutions in the past such as New Relic, AppDynamics, NeoLoad, and Micro Focus LoadRunner Enterprise.

JMeter is not as good as LoadRunner or NeoLoad, and it isn't as easy to use, but it's okay because there is no cost. LoadRunner is too expensive, in my opinion. NeoLoad is cheaper, although not significantly.

From what I have seen, many companies are adopting JMeter because it's free. Especially in Canada, using JMeter seems to be the new trend. Some companies are choosing NeoLoad over LoadRunner because it is easier, faster, and cheaper. Whatever they need to do can be completed quicker. The main problem with NeoLoad is that obtaining resources is harder.

Given all of the choices, my preference would be to implement NeoLoad.

How was the initial setup?

The initial setup is straightforward. I would not say that it is complex and if you already have the file downloaded then it will only take about half an hour to deploy.

What about the implementation team?

I took care of the deployment myself.

What's my experience with pricing, setup cost, and licensing?

I was using the free version of the software.

What other advice do I have?

My advice for anybody who is considering JMeter is to just install it and try it. Creating scripts is a different process when you compare it to LoadRunner or Neoload. There is different terminology compare to these two products, so if somebody has not used JMeter then it may seem difficult at first.

I would rate this solution a six out of ten.

Which deployment model are you using for this solution?

On-premises
Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
PeerSpot user
Sr Manager - Quality Engineering at a computer software company with 501-1,000 employees
Real User
Top 5
A useful solution for load testing and testing web applications, but it does not provide page rendering capabilities for end-users
Pros and Cons
  • "When someone in our organization wants to test web applications, they use Apache JMeter since they face no hurdles while using the solution."
  • "Self-healing and page rendering for the end-users are not available in Apache JMeter."

What is our primary use case?

The solution is primarily used for load testing.

How has it helped my organization?

When someone in our organization wants to test web applications, they use Apache JMeter since they face no hurdles while using the solution. Also, we can even increase the number of users using it.

What is most valuable?

All the features mentioned under JMeter metrics are available to use. Feature-wise, the solution has everything in it that I want.

What needs improvement?

Self-healing and page rendering for the end-users are not available in Apache JMeter. Hence, these are the two areas that need to be made available in future releases of the solution.

For how long have I used the solution?

I have been using Apache JMeter for eight years. Also, I am using the solution's latest version, and any of the solution's versions support all the applications in my organization. Our company is an end-user of the solution.

What do I think about the stability of the solution?

Stability-wise, I rate the solution a seven out of ten.

What do I think about the scalability of the solution?

There are around 50,000 users in my company using the solution. Also, most of the users are involved in performance testing. Scalability-wise, I rate the solution a seven out of ten.

With Apache JMeter, banking processes may face some scalability issues.

How are customer service and support?

There is no support at all for the solution. Basically, it provides zero support.

How was the initial setup?

The solution's initial setup process was straightforward since one only needs to download Apache JMeter and use it after that. Basically, there is no need to do any installation.

There is no need for any maintenance in the solution since it is an open-source product.

What's my experience with pricing, setup cost, and licensing?

Since it is an open-source product, buying or paying for any license is unnecessary. One just needs to download it to be able to use it.

What other advice do I have?

I would recommend the product to those planning to use it as an open-source product, and there is no need to buy a license. So, it is free of cost application. However, I am not sure about the product from a security perspective. But nowadays, banking people are also using it, but it won't be scalable in a banking environment where a large number of people want to use it. Using the solution, one can only test web applications and not use the tool for page rendering. I rate the overall solution a seven out of ten.

Which deployment model are you using for this solution?

On-premises
Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
AmilaDantanarayana - PeerSpot reviewer
Presales manager at Mitra Innovation
Real User
Top 5Leaderboard
Easy to set up with good documentation but needs extended backend support
Pros and Cons
  • "It's stable and reliable."
  • "In terms of platform support, they need to extend the support for backend platforms and more of the legacy types of platforms."

What is our primary use case?

There's a third-party platform that is getting migrated to, so with the newer version migration, we were looking to see if there are any performance degradation with the migration. The product platform assures the user loads that we are planning to have on the system. Nevertheless, we wanted to have validation from a business perspective. We wanted to see that it'll still support the loads that we are looking at.

What is most valuable?

The ability to create a large number of user loads has been very helpful for us. 

It's typically straightforward to set up. 

The solution can scale well.

It's stable and reliable. 

There is lots of useful community support. 

What needs improvement?

In terms of platform support, they need to extend the support for backend platforms and more of the legacy types of platforms.

For how long have I used the solution?

We've used the solution for a long while. We've used it for a few years. 

What do I think about the stability of the solution?

It's a well-trusted product. A lot of people use it. There's a lot of support in the community also. Therefore, the stability is pretty good and it is quite reliable.

What do I think about the scalability of the solution?

We can create demand as we need. It's quite scalable. 

We have five people suing the product. The performance testers use it.

How are customer service and support?

We can get a lot of support from the community. It's pretty easy to get assistance. 

We've had no issue so far when we've needed to get help. We are satisfied with the support. 

How would you rate customer service and support?

Positive

Which solution did I use previously and why did I switch?

We did not previously use a different solution. 

How was the initial setup?

The solution can be easy to set up depending on the target system that we are testing. However, there are situations where Apache JMeter is not supported.

The deployment took two to three weeks to set up and deploy.

We have a team of five people that can handle deployment and maintenance. 

What about the implementation team?

We did the setup in-house. We have the capabilities within.

What's my experience with pricing, setup cost, and licensing?

Most of the time, it's the end client who manages the license for us. We don't purchase the product by ourselves; it's the end customer who does it for us. I'm not sure what the actual cost of the product is. 

What other advice do I have?

We likely are using the latest version of the solution. However, I can't be sure. I'm not sure which version number we're using. 

It's a tool that is being used by many, many others as well. There's a lot of community support out there. I'd recommend it, as long as it supports the platform.

I'd rate the solution seven out of ten. 

Which deployment model are you using for this solution?

Public Cloud
Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
Quality Engineering Delivery Leader at a financial services firm with 10,001+ employees
Real User
Good performance and easy to configure but could use better UI
Pros and Cons
  • "The performance of the solution is excellent."
  • "You really need a technical team in order to really utilize the product."

What is our primary use case?

We're primarily using the solution for its performance. It's on our data center.

How has it helped my organization?

The tool has been a great help to our organization, which is focused more on quality testing as opposed to performance testing. It's provided a lot of insights. With the digital transformation that is currently happening, the need for that shift was rather critical.

What is most valuable?

The performance of the solution is excellent.

They have designed the product so that it is very easy to configure. 

You can basically do anything you like with the product. It's not very restrictive.

We like the fact that the technology is open-source.

What needs improvement?

The user interface could be improved. If they had better UI, it might make it easier to use.

You really need a technical team in order to really utilize the product.

The scalability could be better, or the process of scaling itself could be a bit more clear.

For how long have I used the solution?

We've used the solution for a couple of years at this point.

What do I think about the stability of the solution?

The stability is quite good. There haven't been any issues with bugs or glitches. It doesn't crash of freeze. It's very reliable.

What do I think about the scalability of the solution?

Nobody ever has used any tool to its full capacity, including this one. Half of the time it's how you set up and how your environment is. We can easily scale on cloud. So far, we're worried about its ability to scale. Our setup is basically a hybrid cloud where we can have private data centers and we can know exactly where each is located. However, the scaling is a big concern for me. I'm looking at BlazeMeter as well, and, although I've only really read about it, it looks like it scales quite well. With scaling, I'm sure I'll find more complexities, especially if I look into kiosk testing. I'll want to explore further test cases.

Currently, those that are on the product are mostly testing communities approaching it from a performance aspect. Every project is different and therefore the number of users goes up and down.

How are customer service and technical support?

There isn't really a need for tech support with JMeter. There is a vibrant community, which is a good way for a lot of users or developers to go on and post a question and get an answer. The team prefers it this way. It's a positive aspect of JMeter.

Which solution did I use previously and why did I switch?

I have some experience LoadRunner, however, it is expensive for what it offers. This product is open-source which makes it affordable.

How was the initial setup?

The initial setup is pretty straightforward. My team was very hands-on. They didn't struggle with the UI at all and it wasn't too complex for them. That said, if a person with lesser skill levels, maybe someone who has a lower skillset, may struggle a bit with the implementation. It's geared slightly towards more technical users.

What's my experience with pricing, setup cost, and licensing?

The product is an open-source solution.

Which other solutions did I evaluate?

I understand the Apache also has something called BlazeMeter, which is something I recently looked into.

What other advice do I have?

I'm not quite sure which version of the solution our team is using right now. It's likely the latest version. We did the upgrade on our tools in 2019. It's open-source, and therefore everybody picks up the latest one.

As an open-source tool, there's a good community surrounding it. If you have automation frameworks or a DevOps pipeline, you can connect that and easily configure everything to streamline processes. Due to the fact that it's open-source, you don't have to wait to get the approvals and the budgets in order. That part alone can take months to sometimes years. If your company has the time, they should review the tools. If it seems to fit your organization, I'd recommend trying it out. 

Overall, I would rate the solution seven out of ten. It's a good open-source tool that you can configure easily, and it's very competitive in the market in terms of usability

Which deployment model are you using for this solution?

Hybrid Cloud
Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
Software Solution Architect at Beltone Financial
Real User
Helps in the capacity assessment planning for services
Pros and Cons
  • "The product helps me get the expected performance from applications or servers and reduces costs. It also enhances the performance of the services and helped them reach their ultimate capacity."
  • "The tool needs to have a better Graphical User Interface. Many of the solution's features are difficult to understand due to the complex user interface and user experience. The product needs to add plugins. It should also work on the integration with external partners like IDE and API gateways."

What is our primary use case?

The tool helps us in the capacity assessment planning for our services. It helps us to make sure that the APIs will be able to afford the traffic that we expect to get from these services in the production environment. 

How has it helped my organization?

The product helps me get the expected performance from applications or servers and reduces costs. It also enhances the performance of the services and helped them reach their ultimate capacity. 

What needs improvement?

The tool needs to have a better Graphical User Interface. Many of the solution's features are difficult to understand due to the complex user interface and user experience. The product needs to add plugins. It should also work on the integration with external partners like IDE and API gateways. 

For how long have I used the solution?

I have been using the solution for more than three to four years. 

What do I think about the stability of the solution?

The tool is stable. 

What do I think about the scalability of the solution?

The product is scalable. My company has around four to five users for the solution. 

How was the initial setup?

The product's setup is not complex. The tool's installation was very smooth.  It doesn't have any deployment process. You just need to implement a set of packages to get started with the product. 

What other advice do I have?

I would rate the product an eight out of ten. 

Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
Buyer's Guide
Download our free Apache JMeter Report and get advice and tips from experienced pros sharing their opinions.
Updated: March 2024
Buyer's Guide
Download our free Apache JMeter Report and get advice and tips from experienced pros sharing their opinions.