Apache JMeter OverviewUNIXBusinessApplication

Apache JMeter is the #1 ranked solution in top Performance Testing Tools, #1 ranked solution in top Load Testing Tools, and #3 ranked solution in top API Testing Tools. PeerSpot users give Apache JMeter an average rating of 7.4 out of 10. Apache JMeter is most commonly compared to Postman: Apache JMeter vs Postman. Apache JMeter is popular among the large enterprise segment, accounting for 69% of users researching this solution on PeerSpot. The top industry researching this solution are professionals from a computer software company, accounting for 23% of all views.
Apache JMeter Buyer's Guide

Download the Apache JMeter Buyer's Guide including reviews and more. Updated: December 2022

What is Apache JMeter?

Apache JMeter is an open-source Java application that tests load and functional behavior and performance in applications. Created initially to test web applications, it has expanded its functionality to test other functions. For instance, you can test a server to see how efficiently it works and how many user requests can be handled simultaneously.

You can use JMeter to test functional performance and regression tests on different technologies. This Java desktop application has an easy-to-use graphical interface which uses the Swing graphical API. You can run JMeter on any environment that accepts a Java virtual machine, such as Windows, Linux, and Mac.

What protocols does JMeter support?

  • Web - HTTP, HTTPS
  • Web services - SOAP/XML-RPC
  • Email service - POP3, IMAP, SMTP
  • FTP service
  • Database via JDBC drivers
  • LDAP
  • Native commands or shell scripts
  • TCP
  • Java objects

How does JMeter work?

JMeter sends requests to a target server by simulating a group of user requests. Then it collects and calculates statistics on the performance of the target. This target can be a server or an application.

You can test the performance of static resources, such as JavaScript or HTML, and dynamic resources, such as JSP, Servlets, and AJAX. It is also helpful to determine how many concurrent users your website can handle.

There are two main tests you can carry out with JMeter: load test and stress test. The load test models expected usage of a server by simulating multiple users accessing the web server simultaneously. The stress testing aims to find the maximum load capacity of the server or application.

Apache JMeter Key Features

  • JMeter enables fast test plan recording, building, and debugging via a featured test IDE (integrated development environment).
  • Command-line mode allows carrying out load tests from any OS compatible with Java.
  • JMeter can extract data from most popular response formats, such as HTML, JSON, XML.
  • JMeter is entirely portable.
  • A multithreading framework allows you to simultaneously test multiple samplings and separate thread groups.
  • Thanks to the caching feature, you can conduct offline analysis or replay test results.

Apache JMeter Benefits

  • As open-source software, it is freely available and continuously improved.
  • The setup is user-friendly, and no installation is needed.
  • The GUI (graphic user interface) is intuitive and easy to use.
  • You can write your own tests and use the visualization plugins to extend the testing.
  • It is a platform-agnostic tool. For example, JMeter can work with Linux by clicking on the JMeter shell script. On Windows, you can call up JMeter by starting the jmeter.bat file.
  • Since the test plans are stored in XML format, you can generate a test plan using any text editor.
  • You can simulate a heavy load on a server, a group of servers, a network, or an object. Use JMeter to test an application or server's strength and performance under different load types.

The JMeter extensible core has numerous benefits:

  • Unlimited testing capabilities via pluggable samplers.
  • You can choose multiple load statistics with pluggable timers.
  • Visualization plugins and data analysis enable customization.
  • Continuous integration via third-party open-source libraries (Maven, Gradle, and Jenkins).

Reviews from Real Users

Stephen B., I.T. Architect, Analyst, and Developer at an educational organization, says, "The scripting ability is most valuable. It is easy to use. There is a UI, and you can go in there and figure those things out. After you've got a good set of tests, you basically have a scripted document that you can grab and execute in a pipeline. It is pretty quick to set up, and you can scale it and version control it."

"I like the fact that JMeter integrates well with other tools," adds the Founder and Principal Consultant at a tech services company.

A Quality Engineering Delivery Leader at a financial services firm says, “The performance of the solution is excellent. They have designed the product so that it is very easy to configure. You can basically do anything you like with the product. It's not very restrictive. We like the fact that the technology is open-source.”

Apache JMeter was previously known as JMeter.

Apache JMeter Customers

AOL, Orbitz, Innopath Software, PrepMe, Sapient, Corporate Express Australia, CSIRO, Ephibian, Talis, DATACOM, ALALOOP, eFusion, Panter, Sourcepole, University of Western Cape

Apache JMeter Video

Apache JMeter Pricing Advice

What users are saying about Apache JMeter pricing:
  • "Apache JMeter is an open-source solution, so it's free to use."
  • "It is open source. There are no licensing costs associated. If you need enterprise support, you'll probably end up paying for a license. You would also factor in the infrastructure cost, but that's not significant."
  • "We didn't pay licensing fees for Apache JMeter because it's an open-source tool. We only paid for the machines where we installed Apache JMeter modules."
  • Apache JMeter Reviews

    Filter by:
    Filter Reviews
    Industry
    Loading...
    Filter Unavailable
    Company Size
    Loading...
    Filter Unavailable
    Job Level
    Loading...
    Filter Unavailable
    Rating
    Loading...
    Filter Unavailable
    Considered
    Loading...
    Filter Unavailable
    Order by:
    Loading...
    • Date
    • Highest Rating
    • Lowest Rating
    • Review Length
    Search:
    Showingreviews based on the current filters. Reset all filters
    MuhammadAwad - PeerSpot reviewer
    Quality Manager at Incorta
    Real User
    Top 20
    It's a free, scalable tool that's good for checking backend services
    Pros and Cons
    • "I use all the tools, but one feature that stands out is JMeter's ability to test when services are sending a particular kind of request. We are using specific ports to send queries, and assess the performance based on the time it takes these queries to respond. You can use it with stuff other than the web performance."
    • "The UI needs some work. The first time I used JMeter, I couldn't record the full scenario to mimic the user experience. Since then, they have introduced some plugins and a third-party tool called BlazeMeter."

    What is our primary use case?

    I use JMeter for concurrency and some backend services. We are also using JMeter to apply our tests within our framework by sending some requests to JMeter to assess our application's performance. My company is developing software based on data analytics that produces insights and graphs like Tableau or Power BI. We use JMeter to test the graphs, how the tables are rendered, and how long it takes to render some applications. 

    For instance, we have a problem with one of our clients that use some complex scenarios or queries that take too much time to retrieve from the application or render to the end user. We ran a couple of tests on the application using JMeter and spotted the service or request that was taking too much time.We alerted the developers about this, and they took the appropriate action to fix this problem. Afterward, we run another cycle of JMeter to ensure everything is working as expected.

    There are two modules. The analytics module is the interface for the stakeholders and company decision-makers. The number of users isn't that big, unlike the retail applications websites. It's 500 users at most. The other module is deals with data volume. We are currently doing some POCs to check whether we will benefit from JMeter in this area because it's not a concurrency issue. It's a data volume issue.

    We have a hybrid deployment because we are using JMeter to assess the performance of our products. If the product is deployed on the cloud, we use JMeter on the cloud. If it's on-prem, we are using it on-prem. At my previous company, we mainly used JMeter on-prem.

    I expect that we will continue to rely more on JMeter, and we have multiple DevOps pipelines using JMeter to test another module in our application.

    What is most valuable?

    I use all the tools, but one feature that stands out is JMeter's ability to test when services are sending a particular kind of request. We are using specific ports to send queries, and assess the performance based on the time it takes these queries to respond. You can use it with stuff other than the web performance.

    What needs improvement?

    The UI needs some work. The first time I used JMeter, I couldn't record the full scenario to mimic the user experience. Since then, they have introduced some plugins and a third-party tool called BlazeMeter. It's working on this, actually.

    It's an excellent plugin that you can use to record the scenario from Google Chrome, and it integrates easily into JMeter. They could also make it easier to generate the built-in report. Now, you run the tests and generate the charts in a separate column. The graphs and charts that display the test metrics could be better.

    I worked with another tool called Web Performance Tester, and its interface is better than JMeter's. They have intuitive graphs while you are running the tests, so you can see how things are going. It shows you the number of concurrent users logged into the system, the number of failures, response times, etc.

    For how long have I used the solution?

    I worked in performance testing from 2011 until 2019. I was working with another tool, but in the last few years, I started using JMeter for a couple of projects.

    Buyer's Guide
    Apache JMeter
    December 2022
    Learn what your peers think about Apache JMeter. Get advice and tips from experienced pros sharing their opinions. Updated: December 2022.
    658,157 professionals have used our research since 2012.

    What do I think about the stability of the solution?

    It's hard to say definitively. When there's something wrong with JMeter, we're unsure whether it's something in the device because there are lots of requests coming from the machine. Maybe we didn't use it that many times from our local server. There are some bottlenecks, but in many cases, we only need to restart JMeter, and it works fine.

    What do I think about the scalability of the solution?

    JMeter is scalable. I did a test using it in a distributed environment with more than one client. We used JMeter to load from the server and also the plugin tool. It works fairly well in a complex environment. I still need to research the maximum concurrent JMeter can handle. Is it 1 billion or 10 million? 

    In my work experience, I need to load maybe 1 million users. For example, most of my work is on the backend of e-commerce websites with a maximum of 1,000 users. We have many clients, but the load is not that big. 

    How are customer service and support?

    I don't think JMeter has technical support because it's an open-source application, but there is support for third-party apps that use JMeter. For instance, BlazeMeter has good technical support.

    Which solution did I use previously and why did I switch?

    I have more experience in a tool called Web Performance Tester, but it has some limitations. For example, it doesn't support some forms. A long time ago, their support was better, but lately, their support hasn't been so good. They always ask us to switch to the consultancy service and they don't give us the solution. If we face problems, they always tell us, "Okay. You can consult us." This is not a good practice, actually.

    Also, Web Performance Tester isn't well-known, so some of our clients aren't confident in it because it doesn't have a reputation.  They were more familiar with JMeter, and it's something I've used.  However, I have no problem with Web Performance Tester. It's hard to compare the two tools because it's not apples to apples. 

    I joined this company in October, and they had just started using JMeter. Previously, they were using a tool that they had developed in-house. They found that JMeter has more capabilities and specificity than the tools they were using.

    How was the initial setup?

    Setting up JMeter is straightforward. It doesn't need an installation like other tools. There is a batch file for Windows. Around four people in my organization are responsible for maintaining and managing JMeter.

    What's my experience with pricing, setup cost, and licensing?

    JMeter is a free open-source tool. There are some third-party tools built on top of JMeter that have a license or something like BlazeMeter. I think you can also purchase some additional services.

    What other advice do I have?

    I rate Apache JMeter eight out of 10.  Before using JMeter, you should find some tutorials because you need to be trained to use it. It's not a plug-and-play tool. This is what I did. Spend some time researching JMeter's capabilities. We met a lot of people who knew about JMeter, but when you tell them that it can do something, they're not aware of it. 

    For instance, through taking some courses, we learned that JMeter has distributed performance capabilities. Some people still know that you can't make these recordable scenarios. This information hasn't reached everyone in the IT market.

    Disclosure: I am a real user, and this review is based on my own experience and opinions.
    Flag as inappropriate
    PeerSpot user
    Senior Performance Engineer II at a tech services company with 1,001-5,000 employees
    Real User
    Stable, has a user-friendly GUI, and is simple to set up, but it consumes a lot of resources when you increase the load
    Pros and Cons
    • "What I like best about Apache JMeter is its user-friendly GUI because even if you don't have very good coding knowledge or understanding, or even if you don't come from a development background, you can still use the solution with just a few clicks. This is what's unique about Apache JMeter, in comparison with other tools in the market. As Apache JMeter is open source, when there's a missing feature, you can search in several community blogs for plugins that you can use to modify Apache JMeter to meet your requirements, and this is another advantage."
    • "What needs improvement in Apache JMeter is the very high load requirements when you want to scale it beyond certain thresholds. For example, small to mid-range testing is very easily done with Apache JMeter, but if you scale and increase the load, then it would be a problem because the tool consumes a lot of resources, probably because Apache JMeter provides an enriched UI experience, so it consumes a lot of memory and requires high CPU usage. This means you have to manage your infrastructure, or else you'll have high overhead expenses. As Apache JMeter is a heavyweight tool, that is an area for improvement, though I'm unsure if Apache can do something about it because it could be a result of the way it's architected. What I'd like to see from Apache JMeter in the future is for it to transition to the cloud, as a lot of cloud technologies emerge around the globe, and a lot of people prefer cloud-based solutions or cloud-native tools. Even if a company has a legacy system, it's still possible to transition to the cloud. I've worked with a company that was an on-premise company that moved to the cloud and became cloud-native. If Apache JMeter could transition to the cloud, similar to k6, then it could help lessen the intense resource consumption that's currently happening in Apache JMeter."

    What is our primary use case?

    We've been using Apache JMeter for load testing, spike testing, and endurance testing. We use the solution mostly for nonfunctional use cases, except for the security aspects, because those aspects require a different tool altogether.

    What is most valuable?

    What I like best about Apache JMeter is its user-friendly GUI because even if you don't have very good coding knowledge or understanding, or even if you don't come from a development background, you can still use the solution with just a few clicks. This is what's unique about Apache JMeter, in comparison with other tools in the market.

    As Apache JMeter is open source, when there's a missing feature, you can search in several community blogs for plugins that you can use to modify Apache JMeter to meet your requirements, and this is another advantage.

    What needs improvement?

    What needs improvement in Apache JMeter is the very high load requirements when you want to scale it beyond certain thresholds. For example, small to mid-range testing is very easily done with Apache JMeter, but if you scale and increase the load, then it would be a problem because the tool consumes a lot of resources, probably because Apache JMeter provides an enriched UI experience, so it consumes a lot of memory and requires high CPU usage. This means you have to manage your infrastructure, or else you'll have high overhead expenses.

    As Apache JMeter is a heavyweight tool, that is an area for improvement, though I'm unsure if Apache can do something about it because it could be a result of the way it's architected.

    What I'd like to see from Apache JMeter in the future is for it to transition to the cloud, as a lot of cloud technologies emerge around the globe, and a lot of people prefer cloud-based solutions or cloud-native tools. Even if a company has a legacy system, it's still possible to transition to the cloud. I've worked with a company that was an on-premise company that moved to the cloud and became cloud-native. If Apache JMeter could transition to the cloud, similar to k6, then it could help lessen the intense resource consumption that's currently happening in Apache JMeter.

    For how long have I used the solution?

    I've been using Apache JMeter since 2020, so it's been two years since I started using it.

    What do I think about the stability of the solution?

    Apache JMeter is a stable product overall. A very large community in the software industry uses Apache JMeter, and though it's open source and there's a continuous phase of improvement going on with it, it has stable versions available for my company to continue testing it.

    What do I think about the scalability of the solution?

    Apache JMeter is scalable in the sense that there's a master/slave architecture available for it where I can scale as many slaves along the way, but when a huge number of users is required in a particular load test, and you have to scale a lot beyond a particular threshold, the resource consumption becomes too high and requires considerable overhead. Apache JMeter consumes a lot of memory, and that's a hurdle, but it's scalable up to a certain point. If a very, very large organization requires very heavy load testing to be done, it would be better for that organization to go with some other tool.

    How are customer service and support?

    Because Apache JMeter is an open-source tool, you get support from the community. The challenge with open-source tools is that if there isn't enough community, then the support and development you get would be limited, but as Apache JMeter is widely used, the community is vast and the support is sound. Unlike with Micro Focus LoadRunner, when you encounter an issue, you can report that to Micro Focus and the Micro Focus team will take care of your issue. It doesn't work that way with open-source solutions.

    For the support I get from the community, on a scale of one to five, I'm rating it a three because there could be instances where you could not get a resolution for your issue. After all, Apache JMeter is a free product, so you can only rely on community support. Though the community for the tool is so big and Apache JMeter is widely used, and there'd be a lesser amount of circumstances where the solution for your issue isn't available yet, there could be between one percent to five percent chance of it happening, so that could be troublesome.

    How was the initial setup?

    The initial setup for Apache JMeter is pretty simple. There isn't much dependency, in general. For example, in macOS, you can just download the tar files entirely and just untar the files. There's no installation of specific extensions required. That's for macOS. For Windows, there would be .exe files. From that perspective, setting up Apache JMeter is quite easy. You can just run it locally, untarring it in any particular location. Just one setup that's required to be done is setting the Java home path to start the app. Otherwise, it's pretty straightforward and very quick to set up.

    On a scale of one to five, I'm rating the initial setup a four because there's always some room for improvement.

    What's my experience with pricing, setup cost, and licensing?

    Apache JMeter is an open-source solution, so it's free to use.

    Which other solutions did I evaluate?

    I evaluated k6 and Locust.

    What other advice do I have?

    I'm using Apache JMeter for work. Locust is also used within the company, as well as k6, but I only touched base on those solutions.

    The projects I'm working on currently use Apache JMeter, but my company works across regions and uses different tools, including Gatling and Locust.

    Apache JMeter is used daily, for the range of projects I've been testing it on. A new government project came in, and I chose Apache JMeter for it because of its simplicity and user-friendly interface.

    My rating for Apache JMeter is seven out of ten.

    Which deployment model are you using for this solution?

    On-premises
    Disclosure: I am a real user, and this review is based on my own experience and opinions.
    Flag as inappropriate
    PeerSpot user
    Buyer's Guide
    Apache JMeter
    December 2022
    Learn what your peers think about Apache JMeter. Get advice and tips from experienced pros sharing their opinions. Updated: December 2022.
    658,157 professionals have used our research since 2012.
    RajaRao - PeerSpot reviewer
    Associate at Tech Mahindra Limited
    Real User
    Top 5
    Easy to learn, and free to use but could be more user-friendly
    Pros and Cons
    • "We appreciate that the solution is free to use, as an open-source tool."
    • "We're like the solution to be more user-friendly."

    What is our primary use case?

    We primarily use the solution for testing. We have three types of testing. One is the APA testing, and the web, and one is mobile. All have been recorded through this protocol.

    What is most valuable?

    We appreciate that the solution is free to use, as an open-source tool.

    We're using all of the lightweight technology with the AngularJS and Springboard applications, all the EPAs only. We use these CTP samples for all the web, as well as the EPAs. 

    The initial setup is pretty straightforward.

    The product is easier to learn than paid tools. You can find training online on YouTube, or you can Google it to find out more about the solution and how to use it.

    The JMeter community has developed a lot of IoT protocols. 

    What needs improvement?

    We're like the solution to be more user-friendly. 

    As freeware, not everything is readily available. You can't play around with everything. That's just due to the fact that it's not a paid tool. When you pay for tools, you get a bit more. 

    Not everything is supported by JMeter. It's limited.

    With JMeter, with banking encryption, we have struggled a lot. It's not as good as other paid tools that provide support and configuration capabilities that JMeter lacks. 

    The solution doesn't really have good documentation, and, if you run into issues you can't simply raise a ticket. There's no help available to you.

    There are certain protocols that you can get on other solutions, such as LoadRunner, that you can't get on JMeter.

    What do I think about the stability of the solution?

    The stability is pretty good. You can execute any kind of tech with JMeter as well. It's an open-source community. There are a number of samples are available. You can achieve it in different ways, however, the stability is quite good. That said, we have experienced glitches. 

    What do I think about the scalability of the solution?

    Scalability is possible. You need to have the load generator for the scaling of your application. It's very simple to procure the load generator. You will have to install the JMeter agent file, where it'll be connecting to the monitor. It will not be as simple as a paid tool. For example, with BlazeMeter, they provide scaling with the cloud. BlazeMeter supports the JMeter items. If you have thousands of users, you will need the cloud and you'll need something like BlazeMeter. 

    Currently, as this is a banking application, we don't have that many users. However, I've tried it with the 5000 users with the five to six load generators in the cloud. If we want to really scale, however, it's best to go with BlazeMeter.

    How are customer service and support?

    As an open-source tool, support is not available for JMeter.

    Which solution did I use previously and why did I switch?

    We also use NeoLoad. JMeter is open-source. It suggests you need to install that and you can make the cloud as a distributor system. NeoLoad is paid. We have some licenses and discounts. We cannot use NeoLoad due to the licenses. That's why we moved the solution to JMeter which is free and open-source.

    How was the initial setup?

    The initial implementation is very simple. Initially, when setting up JMeter, you have to follow the proper documentation. It's very simple, however, if you do make some mistakes, you'll not be able to set up the distributed system. You need to have a little bit of networking knowledge so that all the systems should be in the same network and subnet. Connectivity should be established. It can be very difficult to execute. If you try to do everything all at once. I had some experience and I did it in the parts. It's very simple for me.

    What about the implementation team?

    I handled the implementation myself. 

    What's my experience with pricing, setup cost, and licensing?

    The solution is open-source and free to use.

    So long as you don't have to scale too much, it's very cost-effective. If you do have to scale your users, it's best to move over to BlazeMeter, which is reasonably priced, user-friendly, and works well.

    What other advice do I have?

    I would not recommend this solution for machine-critical applications such as banking or telecom applications. For those, we would go for paid tools. That way, if there are any technical issues or technical breaches with the tool, with the application we are supporting, we can call on support for troubleshooting. With JMeter, we don't have this option. It is good, however, for non-critical applications. In telecom or banking applications, they need to have critical releases and patches, and issues have a high likelihood of leading to a loss of business. We don't want to take chances. However, for non-critical items, JMeter is fine.

    I'd recommend the solution to other users so long as they keep in mind JMeter's capabilities are limited. The upside is it is free to use, however, there's also a limit, to some extent, on how you can use it. IF you have a small-scale organization and a small number of users, JMeter will work well.

    I'd rate the solution at a seven out of ten.

    Which deployment model are you using for this solution?

    Private Cloud
    Disclosure: I am a real user, and this review is based on my own experience and opinions.
    PeerSpot user
    Prashanth Hallur - PeerSpot reviewer
    Vice President at Narwal
    Real User
    Top 10
    Provides good metrics and allows you to write your custom code, but reporting could be improved
    Pros and Cons
    • "The metrics part of it and the ability to write your custom code to do some specific tests in the performance testing space are the most valuable features."
    • "Its reporting could be improved. There should be a better visual representation. That would be helpful for easy consumption of the reports."

    What is our primary use case?

    We are in the service industry. We implement it for our customers. We recommend the right tool and set it up for them. So, I've not had any hands-on experience in my current role, but I have a good understanding or a fair idea of the tool's capabilities. I have a team that takes care of the technical aspects.

    It is an open-source solution. So, typically when you don't want to make a heavy investment, and you want to do some level of performance testing, Apache JMeter is used. 

    It is typically on-premises, and it has also been on the public cloud. It could be Azure, or it could be AWS. It is very rarely on GCP.

    How has it helped my organization?

    It helps them to look into several parameters. For example, when you have certified test cases that are predominantly repeated on an application, you can execute the same thing with increased load. You can see how the application responds and if there is an impact on the response time of the application. You can confine it to certain parameter conditions and then start making changes to see how it performs. You can see where the RAM or CPUs are stagnant and not increased.

    What is most valuable?

    The metrics part of it and the ability to write your custom code to do some specific tests in the performance testing space are the most valuable features.

    It is easy to use. If you want to test your application out and not incur a lot of costs, it is probably the best tool.

    What needs improvement?

    Its reporting could be improved. There should be a better visual representation. That would be helpful for easy consumption of the reports.

    For how long have I used the solution?

    I have been using it for about four to five years.

    What do I think about the stability of the solution?

    JMeter is probably good for lower loads. It is not comparable to LoadRunner when it comes to higher loads.

    What do I think about the scalability of the solution?

    It probably scales up to a few thousand users but not beyond that.

    How are customer service and support?

    Because it is an open-source community, their support is probably average. It won't be like the support for a commercial product. I would rate it a six out of ten.

    Which solution did I use previously and why did I switch?

    LoadRunner is one of the prominent tools. It was formerly HP, and now it is Micro Focus. It has good capabilities and features. It also has decent reporting capabilities. Because of the brand and the capability, it was probably chosen by most of the Fortune 500 clients that we work with. There are also some startup communities or organizations that ventured into other solutions, such as JMeter.

    In terms of comparison, primarily, there are three to four parameters. The first one is the ease of use. The second one is about the protocols that need to be tested, whether it's web or API, HTTP, HTTPS, and all the native things. The third one is in terms of flexibility in setting it up and executing, and the fourth one is in terms of monitoring the execution and reporting pieces. Those are the key parameters for pros and cons. LoadRunner gives you a lot more capability and flexibility, but at the same time, it also consumes a lot of resources. JMeter is relatively simpler, cheaper, and easier to use.

    How was the initial setup?

    I don't have hands-on experience with it, but based on what I have heard from people, it's pretty straightforward in terms of the setup. 

    The setup probably takes a week or two, and then the execution is probably a three to four weeks exercise.

    What about the implementation team?

    We are in the services business. Clients give us access, and we set it up there.

    What was our ROI?

    It is certainly good for testing out the applications for performance testing, especially when you have to test them out frequently and make sure that they are good for at least a few thousand users. It has a decent ROI.

    What's my experience with pricing, setup cost, and licensing?

    It is open source. There are no licensing costs associated. If you need enterprise support, you'll probably end up paying for a license.

    You would also factor in the infrastructure cost, but that's not significant.

    What other advice do I have?

    I would rate it a seven out of ten. It is a decent choice from a small-scale perspective, but reporting could be better. If you want to get some performance testing done without spending money, JMeter is probably the best tool. It doesn't have the best reporting, but it is quite a handy tool.

    Disclosure: I am a real user, and this review is based on my own experience and opinions.
    Flag as inappropriate
    PeerSpot user
    Senior Consultant at Capgemini
    Real User
    Top 5
    Lightweight, simulates applications, and creates threads with good server utilization, but scalability and stability both need improvement
    Pros and Cons
    • "To me, what's most valuable in Apache JMeter is that it's a lightweight tool for application testing. It's the best load-testing tool for my company because Apache JMeter simulates your application during testing. Apache JMeter also creates threads with good server utilization. Apache JMeter allows you to focus on analyzing the situation, looking into measurements, response time, and client-server responses, which I find valuable."
    • "Both scalability and stability could be improved in Apache JMeter."

    What is our primary use case?

    We have a Neotys slave server configuration where we have one server that caters to three servers, and we test most of the load on Apache JMeter, particularly for a hundred users. We test the load for web applications, services, and the rest of the APIs, though our current setup for Apache JMeter isn't that big.

    How has it helped my organization?

    Initially, Apache JMeter had a complex configuration; its UI was tricky and required a lot of resources. Creating scripts and running tests on Apache JMeter was always confusing, but nowadays, with more documentation and UI enhancements, Apache JMeter has improved. Previously, recording and creating scripts was tricky, and you had to do it manually. Now there's a recording facility in Apache JMeter that lets you create and modify scripts and test faster, which helped improve my organization.

    What is most valuable?

    To me, what's most valuable in Apache JMeter is that it's a lightweight tool for application testing. It's the best load-testing tool for my company because Apache JMeter simulates your application during testing. Apache JMeter also creates threads with good server utilization. Apache JMeter allows you to focus on analyzing the situation, looking into measurements, response time, and client-server responses, which I find valuable.

    What needs improvement?

    Both scalability and stability could be improved in Apache JMeter.

    What I'd like to see in Apache JMeter in the future is ease of use in terms of scripting. A recording capability similar to what LoadRunner offers, where you can record scripts, make some modifications, then the script will be ready, is another advanced feature I'd like Apache JMeter to have. The two features would make it easier for new users to learn how to use Apache JMeter and help users utilize the tool more quickly.

    For how long have I used the solution?

    I've been using Apache JMeter for more than six or seven years.

    What do I think about the stability of the solution?

    Apache JMeter isn't as stable because it sometimes crashes when you're running a test. The performance of Apache JMeter could be improved because testing on it isn't always as smooth sailing.

    The tool is partially stable. You can't expect Apache JMeter to run well for enterprise-level, high-load applications. It's a good tool for more straightforward or lightweight web applications but not for CRM-type applications.

    What do I think about the scalability of the solution?

    Scalability-wise, Apache JMeter could be improved because if you try to implement it on multi-servers, the threads running on the tool don't hold up well.

    Which solution did I use previously and why did I switch?

    We used LoadRunner before using Apache JMeter. As Apache JMeter is open source, and we only needed to test lightweight applications, we were pretty sure we wanted to go with Apache JMeter.

    How was the initial setup?

    Apache JMeter is an open-source tool that you can install directly from the web with binary files, so setting it up on one to two machines is easy. The setup could be tricky if you hook Apache JMeter to three or more different machines, and it's also tricky when you execute it after.

    What about the implementation team?

    We implemented Apache JMeter in-house.

    What was our ROI?

    I've seen ROI from Apache JMeter, mainly because it doesn't cost much to maintain, and we can use it on a few lightweight applications.

    What's my experience with pricing, setup cost, and licensing?

    We didn't pay licensing fees for Apache JMeter because it's an open-source tool. We only paid for the machines where we installed Apache JMeter modules.

    What other advice do I have?

    I have experience with Apache JMeter, with version 5.5. as the most recent version I've used.

    Apache JMeter is deployed on-premises, but my company did a POC with Apache JMeter and BlazeMeter. BlazeMeter is a CA proprietor tool where you can hook up Apache JMeter scripts. BlazeMeter is a cloud-based tool where you can run tests with the help of Apache JMeter scripts.

    At the moment, only two people use Apache JMeter within my company. Two people can handle the deployment of Apache JMeter, while only one person is required to maintain it.

    My advice to people looking into implementing Apache JMeter is to make the decision based on the application portfolio. For example, if it's more diverse, then using Apache JMeter could be tricky, but if you're only testing lightweight applications, Apache JMeter will be a viable solution.

    Apache JMeter requires minimal investment, yet it has some returns, and it's a good tool, so I'm rating it as seven out of ten.

    Which deployment model are you using for this solution?

    On-premises
    Disclosure: I am a real user, and this review is based on my own experience and opinions.
    Flag as inappropriate
    PeerSpot user
    Senior Cloud Performance Engineer at Oracle
    Real User
    Top 10
    Allows us to create scripts and is useful for load testing and performance testing but could be more stable
    Pros and Cons
    • "It is scalable. You can scale up to 1,000 users in JMeter. If you can put up four slave servers, you can easily ramp up to 1,000 users."
    • "JMeter should be more stable. Every time there is a new release coming up, a lot of its older functionalities or the new functionalities that are brought in are not very well-documented. It should be documented properly, and there should be proper use cases."

    What is our primary use case?

    We use this solution mostly for creating scripts, load testing, and performance testing. If my client doesn't have a tool or I don't have a tool, I try to leverage JMeter because it's an open-source tool.

    I personally use JMeter for things which cannot be covered by our load testing tool, which is OATS, Oracle Application Testing Suite. This is a tool that was developed by Oracle but is no longer in use because Oracle has stopped developing it.

    The solution is deployed on-prem and on private and public clouds.

    What needs improvement?

    JMeter should be more stable. Every time there is a new release coming up, a lot of its older functionalities or the new functionalities that are brought in are not very well-documented. It should be documented properly, and there should be proper use cases. A lot of the newer features don't work, and sometimes you have to spend a lot of time maintaining the scripts. That is something JMeter could probably look at.

    For example, in JMeter 5 they brought in a lot of new controllers. But there isn't a lot of documentation available on the Apache site on how you're supposed to use those controllers. They've explained the controller functionality, but there aren't any proper use cases to show that. 

    For how long have I used the solution?

    I have been using JMeter for a long time and do a lot of work with this solution.

    What do I think about the stability of the solution?

    It's not stable. With every new release, they come up with newer features, which aren't always very stable. So, stability is a concern.

    What do I think about the scalability of the solution?

    It is scalable. You can scale up to 1,000 users in JMeter. If you can put up four slave servers, you can easily ramp up to 1,000 users. That's not a problem.

    Because it's open-source, a lot of organizations try to leverage JMeter into everything. I don't think JMeter is currently very adaptable to all the technologies. That is something Apache should think of – incorporating other technologies and other tech stacks, which can be scripted from JMeter and tested using JMeter.

    How are customer service and support?

    On a scale of 1 to 5, I would rate technical support 2.5

    They're not very good because JMeter is an open-source tool. You can raise a bug to JMeter, and they might fix it or they might say that, "We will actually fix this in the next release." So, because it's an open-source tool, you do not get dedicated support. You have to raise tickets and wait. If you use BlazeMeter, the support is probably a little better because you can take your issues there.

    I don't think support is great if you compare it with tools like NeoLoad or LoadRunner where they have a dedicated support team. You can raise tickets. You have a direct conversation with their engineers. They can help you understand where the issue is exactly. That kind of support is not in JMeter, at least not now. BlazeMeter has it, but I have not used BlazeMeter support, so I'm not sure how it is. But from what I have seen with JMeter, support is not great.

    How was the initial setup?

    It is straightforward, but for JMeter 5, since they have introduced SSL handshake at the slave level, that becomes a little bit of a challenge for somebody who is doing it for the first time if you're doing a distributed execution, that becomes a challenge.

    But since there was not enough documentation, I had to do a lot of troubleshooting on my own, since it was new. Now there's a lot of documentation available, but initially, there was not much documentation.

    JMeter is pretty easy to use if you are using it for the purpose of only developing scripts. You can just uninstall the file and you can start using it. The only challenge happens if you're trying to access the internet over any kind of proxy. Then, you'll have to start it via the proxy mode.

    Otherwise, deploying JMeter is pretty easy compared to other tools. You just have to download, unzip, and you are ready to go.

    What about the implementation team?

    We installed JMeter ourselves.

    What other advice do I have?

    I would rate this solution 7 out of 10.

    Which deployment model are you using for this solution?

    Hybrid Cloud
    Disclosure: I am a real user, and this review is based on my own experience and opinions.
    Flag as inappropriate
    PeerSpot user
    Quality Engineering Delivery Leader at a financial services firm with 10,001+ employees
    Real User
    Top 10
    Good performance and easy to configure but could use better UI
    Pros and Cons
    • "The performance of the solution is excellent."
    • "You really need a technical team in order to really utilize the product."

    What is our primary use case?

    We're primarily using the solution for its performance. It's on our data center.

    How has it helped my organization?

    The tool has been a great help to our organization, which is focused more on quality testing as opposed to performance testing. It's provided a lot of insights. With the digital transformation that is currently happening, the need for that shift was rather critical.

    What is most valuable?

    The performance of the solution is excellent.

    They have designed the product so that it is very easy to configure. 

    You can basically do anything you like with the product. It's not very restrictive.

    We like the fact that the technology is open-source.

    What needs improvement?

    The user interface could be improved. If they had better UI, it might make it easier to use.

    You really need a technical team in order to really utilize the product.

    The scalability could be better, or the process of scaling itself could be a bit more clear.

    For how long have I used the solution?

    We've used the solution for a couple of years at this point.

    What do I think about the stability of the solution?

    The stability is quite good. There haven't been any issues with bugs or glitches. It doesn't crash of freeze. It's very reliable.

    What do I think about the scalability of the solution?

    Nobody ever has used any tool to its full capacity, including this one. Half of the time it's how you set up and how your environment is. We can easily scale on cloud. So far, we're worried about its ability to scale. Our setup is basically a hybrid cloud where we can have private data centers and we can know exactly where each is located. However, the scaling is a big concern for me. I'm looking at BlazeMeter as well, and, although I've only really read about it, it looks like it scales quite well. With scaling, I'm sure I'll find more complexities, especially if I look into kiosk testing. I'll want to explore further test cases.

    Currently, those that are on the product are mostly testing communities approaching it from a performance aspect. Every project is different and therefore the number of users goes up and down.

    How are customer service and technical support?

    There isn't really a need for tech support with JMeter. There is a vibrant community, which is a good way for a lot of users or developers to go on and post a question and get an answer. The team prefers it this way. It's a positive aspect of JMeter.

    Which solution did I use previously and why did I switch?

    I have some experience LoadRunner, however, it is expensive for what it offers. This product is open-source which makes it affordable.

    How was the initial setup?

    The initial setup is pretty straightforward. My team was very hands-on. They didn't struggle with the UI at all and it wasn't too complex for them. That said, if a person with lesser skill levels, maybe someone who has a lower skillset, may struggle a bit with the implementation. It's geared slightly towards more technical users.

    What's my experience with pricing, setup cost, and licensing?

    The product is an open-source solution.

    Which other solutions did I evaluate?

    I understand the Apache also has something called BlazeMeter, which is something I recently looked into.

    What other advice do I have?

    I'm not quite sure which version of the solution our team is using right now. It's likely the latest version. We did the upgrade on our tools in 2019. It's open-source, and therefore everybody picks up the latest one.

    As an open-source tool, there's a good community surrounding it. If you have automation frameworks or a DevOps pipeline, you can connect that and easily configure everything to streamline processes. Due to the fact that it's open-source, you don't have to wait to get the approvals and the budgets in order. That part alone can take months to sometimes years. If your company has the time, they should review the tools. If it seems to fit your organization, I'd recommend trying it out. 

    Overall, I would rate the solution seven out of ten. It's a good open-source tool that you can configure easily, and it's very competitive in the market in terms of usability

    Which deployment model are you using for this solution?

    Hybrid Cloud
    Disclosure: I am a real user, and this review is based on my own experience and opinions.
    PeerSpot user
    RajeevSAwant - PeerSpot reviewer
    Head Automation COE at Truglobal
    Real User
    Top 5Leaderboard
    Free, commonly used, and good for web API testing
    Pros and Cons
    • "It's a free tool."
    • "The UI could be better."

    What is our primary use case?

    We primarily use the solution for performance and delivery testing.

    What is most valuable?

    It's a free tool. It's a very common tool. There's a lot of support in the community for this. It mainly supports web API testing.

    What needs improvement?

    The UI could be better. It can have some Reach UI also, which would be helpful, and maybe a relatively simpler way of using it. It needs simple modules. There are quite a lot of things which are kind of abandoned, so they can definitely improve on it.

    Integration with some of the other features should be managed. However, it's open source, so there is not much to complain about there.

    It's an open-source tool; we cannot ask for additional features really.

    The product could use some kind of filtering and monitoring and different degree of dashboards and analysis. If that can be provided, that would be very, very helpful.

    For how long have I used the solution?

    I’ve used the solution since 2009.

    What do I think about the stability of the solution?

    The solution is relatively stable. There are no bugs or glitches, and it doesn’t crash o freeze. It’s reliable.

    What do I think about the scalability of the solution?

    It can scale. However, what others say about scalability is that you need to have some proper calculations to be done first.

    How are customer service and support?

    We have dealt with support. Technical support is found in more technical forums. It’s open-source, and communes are where you go for information. We have received good responses when we have inquiries. There are quite a lot of forums in general available.

    Now, as we have grown as an organization or as a team, there are still questions such as, "What are the limitations of this tool?" And we put that to JMeter so we can learn what is best for the maintenance.

    How was the initial setup?

    The installation can be a bit complex. There are quite a lot of things and issues if you go deep into it and if you're setting up JMeter. For example, direct script captioning is slightly different. It's a bit more complex, the correlation parametrization. Setting up the workload model can be complex as it is based on a Java service mechanism.

    It becomes a bit challenging to manage. If I want to put a 50,000 user load, I have to be very careful how the memory is utilized. I must be very aware of the underlying system's capability to execute this. I have to make it into multiple nodes and run them in parallel. There are some calculations, and there are some good power processes that will be required when you are using JMeter. You need to understand its limitations and load work them on.

    The solution doesn’t really need maintenance. They're open-source tools, so we don't expect any maintenance. What we typically do is we pick up a particular version and understand the limitations of it. We then play within those limitations.

    What about the implementation team?

    We handled the initial setup in-house. I handled the setup myself.

    What's my experience with pricing, setup cost, and licensing?

    The solution is free to use.

    What other advice do I have?

    We may be using the latest version. I don’t know the exact version number.

    It is on-prem. However, we can put it on the cloud as well. We install it on any machine, so it can be a local or cloud-based machine.

    I’d recommend JMeter. One critical piece of advice is to plan properly. For JMeter, planning and having an understanding of the limitations are important. If you play within those limitations, it is a really great tool. There are trade rules that apply to a lot of things that we use. However, there's plenty of material available so far. Whoever is doing it, can plan it accordingly. They can create those tests, and execute those tests. That said, at the same time, be aware of the scalability of the JMeter. For example, a single JMeter line, which is running, can take 1,000 to 2,000 threats, not more than that. Again, it depends on the available hardware. If you want to scale to, say, one million or something, different parts of planning are needed.

    I’d rate the solution eight out of ten.

    Which deployment model are you using for this solution?

    On-premises
    Disclosure: I am a real user, and this review is based on my own experience and opinions.
    Flag as inappropriate
    PeerSpot user
    Buyer's Guide
    Download our free Apache JMeter Report and get advice and tips from experienced pros sharing their opinions.
    Updated: December 2022
    Buyer's Guide
    Download our free Apache JMeter Report and get advice and tips from experienced pros sharing their opinions.