IT Central Station is now PeerSpot: Here's why

Apache JMeter OverviewUNIXBusinessApplication

Apache JMeter is #1 ranked solution in top Performance Testing Tools, #1 ranked solution in top Load Testing Tools, and #4 ranked solution in top API Testing Tools. PeerSpot users give Apache JMeter an average rating of 7.4 out of 10. Apache JMeter is most commonly compared to Postman: Apache JMeter vs Postman. Apache JMeter is popular among the large enterprise segment, accounting for 68% of users researching this solution on PeerSpot. The top industry researching this solution are professionals from a computer software company, accounting for 27% of all views.
Apache JMeter Buyer's Guide

Download the Apache JMeter Buyer's Guide including reviews and more. Updated: August 2022

What is Apache JMeter?

Apache JMeter is an open-source Java application that tests load and functional behavior and performance in applications. Created initially to test web applications, it has expanded its functionality to test other functions. For instance, you can test a server to see how efficiently it works and how many user requests can be handled simultaneously.

You can use JMeter to test functional performance and regression tests on different technologies. This Java desktop application has an easy-to-use graphical interface which uses the Swing graphical API. You can run JMeter on any environment that accepts a Java virtual machine, such as Windows, Linux, and Mac.

What protocols does JMeter support?

  • Web - HTTP, HTTPS
  • Web services - SOAP/XML-RPC
  • Email service - POP3, IMAP, SMTP
  • FTP service
  • Database via JDBC drivers
  • LDAP
  • Native commands or shell scripts
  • TCP
  • Java objects

How does JMeter work?

JMeter sends requests to a target server by simulating a group of user requests. Then it collects and calculates statistics on the performance of the target. This target can be a server or an application.

You can test the performance of static resources, such as JavaScript or HTML, and dynamic resources, such as JSP, Servlets, and AJAX. It is also helpful to determine how many concurrent users your website can handle.

There are two main tests you can carry out with JMeter: load test and stress test. The load test models expected usage of a server by simulating multiple users accessing the web server simultaneously. The stress testing aims to find the maximum load capacity of the server or application.

Apache JMeter Key Features

  • JMeter enables fast test plan recording, building, and debugging via a featured test IDE (integrated development environment).
  • Command-line mode allows carrying out load tests from any OS compatible with Java.
  • JMeter can extract data from most popular response formats, such as HTML, JSON, XML.
  • JMeter is entirely portable.
  • A multithreading framework allows you to simultaneously test multiple samplings and separate thread groups.
  • Thanks to the caching feature, you can conduct offline analysis or replay test results.

Apache JMeter Benefits

  • As open-source software, it is freely available and continuously improved.
  • The setup is user-friendly, and no installation is needed.
  • The GUI (graphic user interface) is intuitive and easy to use.
  • You can write your own tests and use the visualization plugins to extend the testing.
  • It is a platform-agnostic tool. For example, JMeter can work with Linux by clicking on the JMeter shell script. On Windows, you can call up JMeter by starting the jmeter.bat file.
  • Since the test plans are stored in XML format, you can generate a test plan using any text editor.
  • You can simulate a heavy load on a server, a group of servers, a network, or an object. Use JMeter to test an application or server's strength and performance under different load types.

The JMeter extensible core has numerous benefits:

  • Unlimited testing capabilities via pluggable samplers.
  • You can choose multiple load statistics with pluggable timers.
  • Visualization plugins and data analysis enable customization.
  • Continuous integration via third-party open-source libraries (Maven, Gradle, and Jenkins).

Reviews from Real Users

Stephen B., I.T. Architect, Analyst, and Developer at an educational organization, says, "The scripting ability is most valuable. It is easy to use. There is a UI, and you can go in there and figure those things out. After you've got a good set of tests, you basically have a scripted document that you can grab and execute in a pipeline. It is pretty quick to set up, and you can scale it and version control it."

"I like the fact that JMeter integrates well with other tools," adds the Founder and Principal Consultant at a tech services company.

A Quality Engineering Delivery Leader at a financial services firm says, “The performance of the solution is excellent. They have designed the product so that it is very easy to configure. You can basically do anything you like with the product. It's not very restrictive. We like the fact that the technology is open-source.”

Apache JMeter was previously known as JMeter.

Apache JMeter Customers

AOL, Orbitz, Innopath Software, PrepMe, Sapient, Corporate Express Australia, CSIRO, Ephibian, Talis, DATACOM, ALALOOP, eFusion, Panter, Sourcepole, University of Western Cape

Apache JMeter Video

Apache JMeter Pricing Advice

What users are saying about Apache JMeter pricing:
  • "It is open source. There are no licensing costs associated. If you need enterprise support, you'll probably end up paying for a license. You would also factor in the infrastructure cost, but that's not significant."
  • "I haven't looked into it. Most of our projects are nonprofit or grant-based. Everything is public commons, so we don't really have to worry about that so much."
  • "Apache JMeter is far less expensive than HP Performance Center."
  • Apache JMeter Reviews

    Filter by:
    Filter Reviews
    Industry
    Loading...
    Filter Unavailable
    Company Size
    Loading...
    Filter Unavailable
    Job Level
    Loading...
    Filter Unavailable
    Rating
    Loading...
    Filter Unavailable
    Considered
    Loading...
    Filter Unavailable
    Order by:
    Loading...
    • Date
    • Highest Rating
    • Lowest Rating
    • Review Length
    Search:
    Showingreviews based on the current filters. Reset all filters
    RajaRao - PeerSpot reviewer
    Associate at Tech Mahindra Limited
    Real User
    Top 5Leaderboard
    Easy to learn, and free to use but could be more user-friendly
    Pros and Cons
    • "We appreciate that the solution is free to use, as an open-source tool."
    • "We're like the solution to be more user-friendly."

    What is our primary use case?

    We primarily use the solution for testing. We have three types of testing. One is the APA testing, and the web, and one is mobile. All have been recorded through this protocol.

    What is most valuable?

    We appreciate that the solution is free to use, as an open-source tool.

    We're using all of the lightweight technology with the AngularJS and Springboard applications, all the EPAs only. We use these CTP samples for all the web, as well as the EPAs. 

    The initial setup is pretty straightforward.

    The product is easier to learn than paid tools. You can find training online on YouTube, or you can Google it to find out more about the solution and how to use it.

    The JMeter community has developed a lot of IoT protocols. 

    What needs improvement?

    We're like the solution to be more user-friendly. 

    As freeware, not everything is readily available. You can't play around with everything. That's just due to the fact that it's not a paid tool. When you pay for tools, you get a bit more. 

    Not everything is supported by JMeter. It's limited.

    With JMeter, with banking encryption, we have struggled a lot. It's not as good as other paid tools that provide support and configuration capabilities that JMeter lacks. 

    The solution doesn't really have good documentation, and, if you run into issues you can't simply raise a ticket. There's no help available to you.

    There are certain protocols that you can get on other solutions, such as LoadRunner, that you can't get on JMeter.

    What do I think about the stability of the solution?

    The stability is pretty good. You can execute any kind of tech with JMeter as well. It's an open-source community. There are a number of samples are available. You can achieve it in different ways, however, the stability is quite good. That said, we have experienced glitches. 

    Buyer's Guide
    Apache JMeter
    August 2022
    Learn what your peers think about Apache JMeter. Get advice and tips from experienced pros sharing their opinions. Updated: August 2022.
    622,358 professionals have used our research since 2012.

    What do I think about the scalability of the solution?

    Scalability is possible. You need to have the load generator for the scaling of your application. It's very simple to procure the load generator. You will have to install the JMeter agent file, where it'll be connecting to the monitor. It will not be as simple as a paid tool. For example, with BlazeMeter, they provide scaling with the cloud. BlazeMeter supports the JMeter items. If you have thousands of users, you will need the cloud and you'll need something like BlazeMeter. 

    Currently, as this is a banking application, we don't have that many users. However, I've tried it with the 5000 users with the five to six load generators in the cloud. If we want to really scale, however, it's best to go with BlazeMeter.

    How are customer service and support?

    As an open-source tool, support is not available for JMeter.

    Which solution did I use previously and why did I switch?

    We also use NeoLoad. JMeter is open-source. It suggests you need to install that and you can make the cloud as a distributor system. NeoLoad is paid. We have some licenses and discounts. We cannot use NeoLoad due to the licenses. That's why we moved the solution to JMeter which is free and open-source.

    How was the initial setup?

    The initial implementation is very simple. Initially, when setting up JMeter, you have to follow the proper documentation. It's very simple, however, if you do make some mistakes, you'll not be able to set up the distributed system. You need to have a little bit of networking knowledge so that all the systems should be in the same network and subnet. Connectivity should be established. It can be very difficult to execute. If you try to do everything all at once. I had some experience and I did it in the parts. It's very simple for me.

    What about the implementation team?

    I handled the implementation myself. 

    What's my experience with pricing, setup cost, and licensing?

    The solution is open-source and free to use.

    So long as you don't have to scale too much, it's very cost-effective. If you do have to scale your users, it's best to move over to BlazeMeter, which is reasonably priced, user-friendly, and works well.

    What other advice do I have?

    I would not recommend this solution for machine-critical applications such as banking or telecom applications. For those, we would go for paid tools. That way, if there are any technical issues or technical breaches with the tool, with the application we are supporting, we can call on support for troubleshooting. With JMeter, we don't have this option. It is good, however, for non-critical applications. In telecom or banking applications, they need to have critical releases and patches, and issues have a high likelihood of leading to a loss of business. We don't want to take chances. However, for non-critical items, JMeter is fine.

    I'd recommend the solution to other users so long as they keep in mind JMeter's capabilities are limited. The upside is it is free to use, however, there's also a limit, to some extent, on how you can use it. IF you have a small-scale organization and a small number of users, JMeter will work well.

    I'd rate the solution at a seven out of ten.

    Which deployment model are you using for this solution?

    Private Cloud
    Disclosure: I am a real user, and this review is based on my own experience and opinions.
    PeerSpot user
    Prashanth Hallur - PeerSpot reviewer
    Vice President at Narwal
    Real User
    Top 20
    Provides good metrics and allows you to write your custom code, but reporting could be improved
    Pros and Cons
    • "The metrics part of it and the ability to write your custom code to do some specific tests in the performance testing space are the most valuable features."
    • "Its reporting could be improved. There should be a better visual representation. That would be helpful for easy consumption of the reports."

    What is our primary use case?

    We are in the service industry. We implement it for our customers. We recommend the right tool and set it up for them. So, I've not had any hands-on experience in my current role, but I have a good understanding or a fair idea of the tool's capabilities. I have a team that takes care of the technical aspects.

    It is an open-source solution. So, typically when you don't want to make a heavy investment, and you want to do some level of performance testing, Apache JMeter is used. 

    It is typically on-premises, and it has also been on the public cloud. It could be Azure, or it could be AWS. It is very rarely on GCP.

    How has it helped my organization?

    It helps them to look into several parameters. For example, when you have certified test cases that are predominantly repeated on an application, you can execute the same thing with increased load. You can see how the application responds and if there is an impact on the response time of the application. You can confine it to certain parameter conditions and then start making changes to see how it performs. You can see where the RAM or CPUs are stagnant and not increased.

    What is most valuable?

    The metrics part of it and the ability to write your custom code to do some specific tests in the performance testing space are the most valuable features.

    It is easy to use. If you want to test your application out and not incur a lot of costs, it is probably the best tool.

    What needs improvement?

    Its reporting could be improved. There should be a better visual representation. That would be helpful for easy consumption of the reports.

    For how long have I used the solution?

    I have been using it for about four to five years.

    What do I think about the stability of the solution?

    JMeter is probably good for lower loads. It is not comparable to LoadRunner when it comes to higher loads.

    What do I think about the scalability of the solution?

    It probably scales up to a few thousand users but not beyond that.

    How are customer service and support?

    Because it is an open-source community, their support is probably average. It won't be like the support for a commercial product. I would rate it a six out of ten.

    Which solution did I use previously and why did I switch?

    LoadRunner is one of the prominent tools. It was formerly HP, and now it is Micro Focus. It has good capabilities and features. It also has decent reporting capabilities. Because of the brand and the capability, it was probably chosen by most of the Fortune 500 clients that we work with. There are also some startup communities or organizations that ventured into other solutions, such as JMeter.

    In terms of comparison, primarily, there are three to four parameters. The first one is the ease of use. The second one is about the protocols that need to be tested, whether it's web or API, HTTP, HTTPS, and all the native things. The third one is in terms of flexibility in setting it up and executing, and the fourth one is in terms of monitoring the execution and reporting pieces. Those are the key parameters for pros and cons. LoadRunner gives you a lot more capability and flexibility, but at the same time, it also consumes a lot of resources. JMeter is relatively simpler, cheaper, and easier to use.

    How was the initial setup?

    I don't have hands-on experience with it, but based on what I have heard from people, it's pretty straightforward in terms of the setup. 

    The setup probably takes a week or two, and then the execution is probably a three to four weeks exercise.

    What about the implementation team?

    We are in the services business. Clients give us access, and we set it up there.

    What was our ROI?

    It is certainly good for testing out the applications for performance testing, especially when you have to test them out frequently and make sure that they are good for at least a few thousand users. It has a decent ROI.

    What's my experience with pricing, setup cost, and licensing?

    It is open source. There are no licensing costs associated. If you need enterprise support, you'll probably end up paying for a license.

    You would also factor in the infrastructure cost, but that's not significant.

    What other advice do I have?

    I would rate it a seven out of ten. It is a decent choice from a small-scale perspective, but reporting could be better. If you want to get some performance testing done without spending money, JMeter is probably the best tool. It doesn't have the best reporting, but it is quite a handy tool.

    Disclosure: I am a real user, and this review is based on my own experience and opinions.
    Flag as inappropriate
    PeerSpot user
    Buyer's Guide
    Apache JMeter
    August 2022
    Learn what your peers think about Apache JMeter. Get advice and tips from experienced pros sharing their opinions. Updated: August 2022.
    622,358 professionals have used our research since 2012.
    Senior Cloud Performance Engineer at Oracle
    Real User
    Top 20
    Allows us to create scripts and is useful for load testing and performance testing but could be more stable
    Pros and Cons
    • "It is scalable. You can scale up to 1,000 users in JMeter. If you can put up four slave servers, you can easily ramp up to 1,000 users."
    • "JMeter should be more stable. Every time there is a new release coming up, a lot of its older functionalities or the new functionalities that are brought in are not very well-documented. It should be documented properly, and there should be proper use cases."

    What is our primary use case?

    We use this solution mostly for creating scripts, load testing, and performance testing. If my client doesn't have a tool or I don't have a tool, I try to leverage JMeter because it's an open-source tool.

    I personally use JMeter for things which cannot be covered by our load testing tool, which is OATS, Oracle Application Testing Suite. This is a tool that was developed by Oracle but is no longer in use because Oracle has stopped developing it.

    The solution is deployed on-prem and on private and public clouds.

    What needs improvement?

    JMeter should be more stable. Every time there is a new release coming up, a lot of its older functionalities or the new functionalities that are brought in are not very well-documented. It should be documented properly, and there should be proper use cases. A lot of the newer features don't work, and sometimes you have to spend a lot of time maintaining the scripts. That is something JMeter could probably look at.

    For example, in JMeter 5 they brought in a lot of new controllers. But there isn't a lot of documentation available on the Apache site on how you're supposed to use those controllers. They've explained the controller functionality, but there aren't any proper use cases to show that. 

    For how long have I used the solution?

    I have been using JMeter for a long time and do a lot of work with this solution.

    What do I think about the stability of the solution?

    It's not stable. With every new release, they come up with newer features, which aren't always very stable. So, stability is a concern.

    What do I think about the scalability of the solution?

    It is scalable. You can scale up to 1,000 users in JMeter. If you can put up four slave servers, you can easily ramp up to 1,000 users. That's not a problem.

    Because it's open-source, a lot of organizations try to leverage JMeter into everything. I don't think JMeter is currently very adaptable to all the technologies. That is something Apache should think of – incorporating other technologies and other tech stacks, which can be scripted from JMeter and tested using JMeter.

    How are customer service and support?

    On a scale of 1 to 5, I would rate technical support 2.5

    They're not very good because JMeter is an open-source tool. You can raise a bug to JMeter, and they might fix it or they might say that, "We will actually fix this in the next release." So, because it's an open-source tool, you do not get dedicated support. You have to raise tickets and wait. If you use BlazeMeter, the support is probably a little better because you can take your issues there.

    I don't think support is great if you compare it with tools like NeoLoad or LoadRunner where they have a dedicated support team. You can raise tickets. You have a direct conversation with their engineers. They can help you understand where the issue is exactly. That kind of support is not in JMeter, at least not now. BlazeMeter has it, but I have not used BlazeMeter support, so I'm not sure how it is. But from what I have seen with JMeter, support is not great.

    How was the initial setup?

    It is straightforward, but for JMeter 5, since they have introduced SSL handshake at the slave level, that becomes a little bit of a challenge for somebody who is doing it for the first time if you're doing a distributed execution, that becomes a challenge.

    But since there was not enough documentation, I had to do a lot of troubleshooting on my own, since it was new. Now there's a lot of documentation available, but initially, there was not much documentation.

    JMeter is pretty easy to use if you are using it for the purpose of only developing scripts. You can just uninstall the file and you can start using it. The only challenge happens if you're trying to access the internet over any kind of proxy. Then, you'll have to start it via the proxy mode.

    Otherwise, deploying JMeter is pretty easy compared to other tools. You just have to download, unzip, and you are ready to go.

    What about the implementation team?

    We installed JMeter ourselves.

    What other advice do I have?

    I would rate this solution 7 out of 10.

    Which deployment model are you using for this solution?

    Hybrid Cloud
    Disclosure: I am a real user, and this review is based on my own experience and opinions.
    Flag as inappropriate
    PeerSpot user
    Quality Engineering Delivery Leader at a financial services firm with 10,001+ employees
    Real User
    Top 20
    Good performance and easy to configure but could use better UI
    Pros and Cons
    • "The performance of the solution is excellent."
    • "You really need a technical team in order to really utilize the product."

    What is our primary use case?

    We're primarily using the solution for its performance. It's on our data center.

    How has it helped my organization?

    The tool has been a great help to our organization, which is focused more on quality testing as opposed to performance testing. It's provided a lot of insights. With the digital transformation that is currently happening, the need for that shift was rather critical.

    What is most valuable?

    The performance of the solution is excellent.

    They have designed the product so that it is very easy to configure. 

    You can basically do anything you like with the product. It's not very restrictive.

    We like the fact that the technology is open-source.

    What needs improvement?

    The user interface could be improved. If they had better UI, it might make it easier to use.

    You really need a technical team in order to really utilize the product.

    The scalability could be better, or the process of scaling itself could be a bit more clear.

    For how long have I used the solution?

    We've used the solution for a couple of years at this point.

    What do I think about the stability of the solution?

    The stability is quite good. There haven't been any issues with bugs or glitches. It doesn't crash of freeze. It's very reliable.

    What do I think about the scalability of the solution?

    Nobody ever has used any tool to its full capacity, including this one. Half of the time it's how you set up and how your environment is. We can easily scale on cloud. So far, we're worried about its ability to scale. Our setup is basically a hybrid cloud where we can have private data centers and we can know exactly where each is located. However, the scaling is a big concern for me. I'm looking at BlazeMeter as well, and, although I've only really read about it, it looks like it scales quite well. With scaling, I'm sure I'll find more complexities, especially if I look into kiosk testing. I'll want to explore further test cases.

    Currently, those that are on the product are mostly testing communities approaching it from a performance aspect. Every project is different and therefore the number of users goes up and down.

    How are customer service and technical support?

    There isn't really a need for tech support with JMeter. There is a vibrant community, which is a good way for a lot of users or developers to go on and post a question and get an answer. The team prefers it this way. It's a positive aspect of JMeter.

    Which solution did I use previously and why did I switch?

    I have some experience LoadRunner, however, it is expensive for what it offers. This product is open-source which makes it affordable.

    How was the initial setup?

    The initial setup is pretty straightforward. My team was very hands-on. They didn't struggle with the UI at all and it wasn't too complex for them. That said, if a person with lesser skill levels, maybe someone who has a lower skillset, may struggle a bit with the implementation. It's geared slightly towards more technical users.

    What's my experience with pricing, setup cost, and licensing?

    The product is an open-source solution.

    Which other solutions did I evaluate?

    I understand the Apache also has something called BlazeMeter, which is something I recently looked into.

    What other advice do I have?

    I'm not quite sure which version of the solution our team is using right now. It's likely the latest version. We did the upgrade on our tools in 2019. It's open-source, and therefore everybody picks up the latest one.

    As an open-source tool, there's a good community surrounding it. If you have automation frameworks or a DevOps pipeline, you can connect that and easily configure everything to streamline processes. Due to the fact that it's open-source, you don't have to wait to get the approvals and the budgets in order. That part alone can take months to sometimes years. If your company has the time, they should review the tools. If it seems to fit your organization, I'd recommend trying it out. 

    Overall, I would rate the solution seven out of ten. It's a good open-source tool that you can configure easily, and it's very competitive in the market in terms of usability

    Which deployment model are you using for this solution?

    Hybrid Cloud
    Disclosure: I am a real user, and this review is based on my own experience and opinions.
    PeerSpot user
    RajeevSAwant - PeerSpot reviewer
    Head Automation COE at Truglobal
    Real User
    Top 5Leaderboard
    Free, commonly used, and good for web API testing
    Pros and Cons
    • "It's a free tool."
    • "The UI could be better."

    What is our primary use case?

    We primarily use the solution for performance and delivery testing.

    What is most valuable?

    It's a free tool. It's a very common tool. There's a lot of support in the community for this. It mainly supports web API testing.

    What needs improvement?

    The UI could be better. It can have some Reach UI also, which would be helpful, and maybe a relatively simpler way of using it. It needs simple modules. There are quite a lot of things which are kind of abandoned, so they can definitely improve on it.

    Integration with some of the other features should be managed. However, it's open source, so there is not much to complain about there.

    It's an open-source tool; we cannot ask for additional features really.

    The product could use some kind of filtering and monitoring and different degree of dashboards and analysis. If that can be provided, that would be very, very helpful.

    For how long have I used the solution?

    I’ve used the solution since 2009.

    What do I think about the stability of the solution?

    The solution is relatively stable. There are no bugs or glitches, and it doesn’t crash o freeze. It’s reliable.

    What do I think about the scalability of the solution?

    It can scale. However, what others say about scalability is that you need to have some proper calculations to be done first.

    How are customer service and support?

    We have dealt with support. Technical support is found in more technical forums. It’s open-source, and communes are where you go for information. We have received good responses when we have inquiries. There are quite a lot of forums in general available.

    Now, as we have grown as an organization or as a team, there are still questions such as, "What are the limitations of this tool?" And we put that to JMeter so we can learn what is best for the maintenance.

    How was the initial setup?

    The installation can be a bit complex. There are quite a lot of things and issues if you go deep into it and if you're setting up JMeter. For example, direct script captioning is slightly different. It's a bit more complex, the correlation parametrization. Setting up the workload model can be complex as it is based on a Java service mechanism.

    It becomes a bit challenging to manage. If I want to put a 50,000 user load, I have to be very careful how the memory is utilized. I must be very aware of the underlying system's capability to execute this. I have to make it into multiple nodes and run them in parallel. There are some calculations, and there are some good power processes that will be required when you are using JMeter. You need to understand its limitations and load work them on.

    The solution doesn’t really need maintenance. They're open-source tools, so we don't expect any maintenance. What we typically do is we pick up a particular version and understand the limitations of it. We then play within those limitations.

    What about the implementation team?

    We handled the initial setup in-house. I handled the setup myself.

    What's my experience with pricing, setup cost, and licensing?

    The solution is free to use.

    What other advice do I have?

    We may be using the latest version. I don’t know the exact version number.

    It is on-prem. However, we can put it on the cloud as well. We install it on any machine, so it can be a local or cloud-based machine.

    I’d recommend JMeter. One critical piece of advice is to plan properly. For JMeter, planning and having an understanding of the limitations are important. If you play within those limitations, it is a really great tool. There are trade rules that apply to a lot of things that we use. However, there's plenty of material available so far. Whoever is doing it, can plan it accordingly. They can create those tests, and execute those tests. That said, at the same time, be aware of the scalability of the JMeter. For example, a single JMeter line, which is running, can take 1,000 to 2,000 threats, not more than that. Again, it depends on the available hardware. If you want to scale to, say, one million or something, different parts of planning are needed.

    I’d rate the solution eight out of ten.

    Which deployment model are you using for this solution?

    On-premises
    Disclosure: I am a real user, and this review is based on my own experience and opinions.
    Flag as inappropriate
    PeerSpot user
    AndrewRath - PeerSpot reviewer
    Azure Cloud Test Manager at a tech services company with 201-500 employees
    Real User
    Top 20
    Great community support, industry-standard, and works well
    Pros and Cons
    • "The biggest thing I liked about it is that there is a huge user base out there, and being shareware and being Apache, if I have any question on how to get something done, I get 18 different answers. Out of those, there would be at least a few good approaches for what I was trying to do. So, the support system out there is most valuable."
    • "I sometimes found the documentation to be not as explanatory as I would've liked it. In the cases that I can think of, I was looking for a rather hand-holding approach with Step A, B, and C, but then I realized that with a product that is open source like this, you can't do handholding. That is because there are so many different uses and different unique environments and setups for it, but I remember thinking a few times that if they only just said this."

    What is our primary use case?

    I've used JMeter in conjunction with Selenium, Java, and Log4j for logging. I used it before
    I ended that contract in August. Its version was up to date at that time.

    It was used for an e-commerce site that is specialized in C-PAP or weaving machines in effect. Their max was a thousand people logged in at once. I, of course, pushed the boundaries on that, but it was to test the performance of the website, and of course, I'm had to try subsystems, database interactions, etc.

    What is most valuable?

    I'm a total geek, so I liked the fact that I got to program. The biggest thing I liked about it is that there is a huge user base out there, and being shareware and being Apache, if I have any question on how to get something done, I get 18 different answers. Out of those, there would be at least a few good approaches for what I was trying to do. So, the support system out there is most valuable.

    What needs improvement?

    I sometimes found the documentation to be not as explanatory as I would've liked it. In the cases that I can think of, I was looking for a rather hand-holding approach with Step A, B, and C, but then I realized that with a product that is open source like this, you can't do handholding. That is because there are so many different uses and different unique environments and setups for it, but I remember thinking a few times that if they only just said this.

    If I were going to be Mr. Selfish and say anything I want, I'd say a full feature GUI that lets me drag and drop different modules in line. It could have a simple-to-use GUI.

    For how long have I used the solution?

    I have been using this solution for probably a year and a half.

    What do I think about the stability of the solution?

    I didn't have any issues with the stability of JMeter itself. There were definitely issues with the program I was testing, but that's why I was testing it.

    What do I think about the scalability of the solution?

    It was very easy to scale, but I was barely scratching the surface. I have spent 17 years at Microsoft, and for the performance testing that we did there, we had 8.3 million users at once, as opposed to a maximum of a thousand. If I'm scaling, I have to do it quite straightforward and simple, but it was very minimal.

    Only I was using it. It was the QA department. I showed it to some of the devs, and they were very interested. A couple of them tried it, but none were actually using it day-to-day for testing out the environment.

    How are customer service and technical support?

    I would rate them an eight out of 10 because sometimes, they would take two or three days to get back to me. Of course, at that point, you're like, "I need the answer; I need to answer." So, it was a little bit unrealistic in terms of expectations.

    Which solution did I use previously and why did I switch?

    At that particular company, I was the one who was tasked with coming up with the solution, so that was the only one that I looked at simply because JMeter is industry standard, but at Microsoft, they wrote their own custom tools, so I used custom Microsoft tools.

    How was the initial setup?

    It was rather complex. It is a complex product, but that part of it was very well-documented. I didn't have any problems with it.

    What other advice do I have?

    Don't be shy in asking questions. Google/Bing is your friend. It is complicated. There's no reason to spend eight hours trying to figure out something, except unless you are trying to learn in-depth. There are a lot of people who've done exactly what you're trying to do, and it doesn't matter what it is.

    I would rate it a 10 out of 10 because it is industry standard. It did everything I could've asked. I barely scratched the surface, but what I needed it for, it did well and in a very straightforward-to-implement way.

    Which deployment model are you using for this solution?

    On-premises
    Disclosure: I am a real user, and this review is based on my own experience and opinions.
    PeerSpot user
    Stephen W. Boyd - PeerSpot reviewer
    I.T. Architect, Analyst, Developer at a educational organization with 51-200 employees
    Real User
    Top 10
    Scalable and stable, easy to use, can be automated, and has good community support
    Pros and Cons
    • "The scripting ability is most valuable. It is easy to use. There is a UI, and you can go in there and figure those things out. After you've got a good set of tests, you basically have a scripted document that you can grab and execute in a pipeline. It is pretty quick to set up, and you can scale it and version control it."
    • "Because so much is being done these days with authentication processes, a better system for either getting bearer tokens or some kind of token-based authentication prior to executing APIs would benefit the product. It is there, and you can do things. It is just not real clean at this point. There should be a better authentication process for JMeter or some automation or better guidelines for gaining and utilizing tokens on the fly."

    What is our primary use case?

    We use Apache  JMeter for API testing. Our latest task combines test cases with our CI/CD pipeline for deployment to Kubernetes. Tests are developed locally and automated for the build using standard Unit and Integration testing techniques. Once deployment is successful to Kubernetes, we develop the JMeter tests for automation in promotion scenarios. Thereafter, once a deployment is successful, we can automate promotion to the next level based on test success.

    How has it helped my organization?

    Test automation moves our organization close to rapidly deploying products. Unit and Integration testing is easy to automate, and most organizations perform these as part of their day-to-day operations. However, end-to-end testing, smoke testing, load testing, and performance testing are much harder to automate. Apache JMeter has aided in that challenge.

    What is most valuable?

    While there is a User Interface, the scripting ability is highly beneficial and is easy to use. Tests can be added to a CI/CD Pipeline for integration with testing and deployment scenarios once finalized and operational.

    What needs improvement?

    This is a difficult question to answer. On one side, JMeter is very flexible and allows for a high amount of customization. On the other, some tasks are common enough that it merits simplifying the process.

    Authentication for API testing could use improvement. Currently, it is a multi-step process to call, extract, and utilize a bearer token securely for API calls. This process is becoming a common enough task that a "wizard" for creating and consuming popular authentication models is merited.

    For how long have I used the solution?

    I have been using this solution for about six years.

    What do I think about the stability of the solution?

    Apache JMeter is stable, and I personally have not encountered any issues. Depending on the size of test runs, one might need to adjust their JAVA settings to align with the test requirements.

    What do I think about the scalability of the solution?

    Its scalability works. It is a typically Java run. Therefore, it is limited only by what you can do in Java in terms of scalability.

    Developers write tests, verify tests, and maintain tests using version control. They identify and tag each to ensure they are appropriately labeled for test purposes (E.g., unit testing, integration testing, performance testing, and the like). Unit and Integration test coverage is normally high. However, we require testing from outside of the system, and JMeter allows us to create tests automating this process.

    How are customer service and technical support?

    Apache JMeter utilizes community support. It is well-documented and has an active community. As far as I know, there is not a "pay-for-support" option.

    Which solution did I use previously and why did I switch?

    I have used Postman in collaboration with other developers. However, I prefer Jmeter only out of personal familiarity and not for any technical deficiencies of Postman.

    How was the initial setup?

    Apache JMeter setup is easy. However, there is a medium-to-heavy learning curve for developing tests and getting started using it for practical uses. Depending on its intended uses, there could be a significant configuration task for a given set of tests.

    What's my experience with pricing, setup cost, and licensing?

    Apache JMeter is under Apache License, Version 2.0 licensing. Understanding licensing requirements is important for the implementation of any tool.

    What other advice do I have?

    Understand the use case. Choosing the correct tool for any task is always a challenge. Jmeter offers a significant amount of flexibility and will work for a lot of solutions. Jmeter requires a commitment to learning for optimal operation; without that investment, tests may not yield the appropriate outcomes.

    Which deployment model are you using for this solution?

    Public Cloud

    If public cloud, private cloud, or hybrid cloud, which cloud provider do you use?

    Disclosure: I am a real user, and this review is based on my own experience and opinions.
    PeerSpot user
    Technical Specialist at a financial services firm with 10,001+ employees
    Real User
    Top 20
    Reasonably priced, performs well, and has responsive technical support
    Pros and Cons
    • "It is cost-effective and simple to use."
    • "It will be much easier, and beneficial for the individual to run it on their own machines rather than having a high-end infrastructure, more CPUs, or more memory that has been consumed by Apache JMeter."

    What is our primary use case?

    Regardless of, whether it is used for personal performance, load, volume, UI test, or functional test Apache JMeter is the best tool on the market.

    What is most valuable?

    It is cost-effective and simple to use.

    Take, for example, a performance test with, the Micro Focus performance test tool, and compare it to Apache JMeter. Apache JMeter is far less expensive than HP Performance Center.

    What needs improvement?

    When you run tests with JMeter, it generates test version five, which is extremely large. Also, when you have a large number of tests to run, it requires a large size or memory size, which basically means it consumes a lot of memory. It would be helpful to come up with a way to be able to use Apache JMeter in a way where it did not use as much memory. It will be much easier, and beneficial for the individual to run it on their own machines rather than having a high-end infrastructure, more CPUs, or more memory that has been consumed by Apache JMeter.

    For how long have I used the solution?

    I have been working with Apache JMeter for two years now.

    I am working with the latest version.

    It's on digital platforms rather than the cloud.

    What do I think about the stability of the solution?

    I am absolutely satisfied with the stability of Apache JMeter.

    What do I think about the scalability of the solution?

    Apache JMeter is a scalable product. We are pleased with the scalability of this solution.

    How are customer service and support?

    When we have encountered difficulties and reached out to technical support, they respond very quickly and most of the things are readily available with them, for all of the challenges or support that are required. If I had to rate them from one to ten, I would rate the technical support an eight out of ten.

    How would you rate customer service and support?

    Positive

    How was the initial setup?

    It is very easy to set up. There have been no issues, it's hassle-free, and it doesn't require a lot of expertise or specialization to configure and set up JMeter. 

    It is very straightforward and very easy. That is something that anyone can do.

    What about the implementation team?

    Our team completed the deployment. It only requires one person and takes no more than two to four hours.

    In terms of maintenance, we only require one person.

    What was our ROI?

    When you compare it to HP because prior to using this Apache JMeter, we used this HP Performance Center, and in terms of ROI, while I don't have exact figures, when it comes to HP Performance Center, we have nearly five times more. You can say that it's five times less of an investment and five times written on the investment by using Apache JMeter. With Apache JMeter, you receive five times more. It's higher.

    What's my experience with pricing, setup cost, and licensing?

    It's a yearly license.

    What other advice do I have?

    I would rate Apache JMeter an eight out of ten.

    Which deployment model are you using for this solution?

    On-premises
    Disclosure: I am a real user, and this review is based on my own experience and opinions.
    Flag as inappropriate
    PeerSpot user
    Buyer's Guide
    Download our free Apache JMeter Report and get advice and tips from experienced pros sharing their opinions.
    Updated: August 2022
    Buyer's Guide
    Download our free Apache JMeter Report and get advice and tips from experienced pros sharing their opinions.