Apache JMeter OverviewUNIXBusinessApplication

Apache JMeter is the #1 ranked solution in top Performance Testing Tools, #1 ranked solution in top Load Testing Tools, and #3 ranked solution in top API Testing Tools. PeerSpot users give Apache JMeter an average rating of 7.4 out of 10. Apache JMeter is most commonly compared to Postman: Apache JMeter vs Postman. Apache JMeter is popular among the large enterprise segment, accounting for 69% of users researching this solution on PeerSpot. The top industry researching this solution are professionals from a computer software company, accounting for 23% of all views.
Apache JMeter Buyer's Guide

Download the Apache JMeter Buyer's Guide including reviews and more. Updated: December 2022

What is Apache JMeter?

Apache JMeter is an open-source Java application that tests load and functional behavior and performance in applications. Created initially to test web applications, it has expanded its functionality to test other functions. For instance, you can test a server to see how efficiently it works and how many user requests can be handled simultaneously.

You can use JMeter to test functional performance and regression tests on different technologies. This Java desktop application has an easy-to-use graphical interface which uses the Swing graphical API. You can run JMeter on any environment that accepts a Java virtual machine, such as Windows, Linux, and Mac.

What protocols does JMeter support?

  • Web - HTTP, HTTPS
  • Web services - SOAP/XML-RPC
  • Email service - POP3, IMAP, SMTP
  • FTP service
  • Database via JDBC drivers
  • LDAP
  • Native commands or shell scripts
  • TCP
  • Java objects

How does JMeter work?

JMeter sends requests to a target server by simulating a group of user requests. Then it collects and calculates statistics on the performance of the target. This target can be a server or an application.

You can test the performance of static resources, such as JavaScript or HTML, and dynamic resources, such as JSP, Servlets, and AJAX. It is also helpful to determine how many concurrent users your website can handle.

There are two main tests you can carry out with JMeter: load test and stress test. The load test models expected usage of a server by simulating multiple users accessing the web server simultaneously. The stress testing aims to find the maximum load capacity of the server or application.

Apache JMeter Key Features

  • JMeter enables fast test plan recording, building, and debugging via a featured test IDE (integrated development environment).
  • Command-line mode allows carrying out load tests from any OS compatible with Java.
  • JMeter can extract data from most popular response formats, such as HTML, JSON, XML.
  • JMeter is entirely portable.
  • A multithreading framework allows you to simultaneously test multiple samplings and separate thread groups.
  • Thanks to the caching feature, you can conduct offline analysis or replay test results.

Apache JMeter Benefits

  • As open-source software, it is freely available and continuously improved.
  • The setup is user-friendly, and no installation is needed.
  • The GUI (graphic user interface) is intuitive and easy to use.
  • You can write your own tests and use the visualization plugins to extend the testing.
  • It is a platform-agnostic tool. For example, JMeter can work with Linux by clicking on the JMeter shell script. On Windows, you can call up JMeter by starting the jmeter.bat file.
  • Since the test plans are stored in XML format, you can generate a test plan using any text editor.
  • You can simulate a heavy load on a server, a group of servers, a network, or an object. Use JMeter to test an application or server's strength and performance under different load types.

The JMeter extensible core has numerous benefits:

  • Unlimited testing capabilities via pluggable samplers.
  • You can choose multiple load statistics with pluggable timers.
  • Visualization plugins and data analysis enable customization.
  • Continuous integration via third-party open-source libraries (Maven, Gradle, and Jenkins).

Reviews from Real Users

Stephen B., I.T. Architect, Analyst, and Developer at an educational organization, says, "The scripting ability is most valuable. It is easy to use. There is a UI, and you can go in there and figure those things out. After you've got a good set of tests, you basically have a scripted document that you can grab and execute in a pipeline. It is pretty quick to set up, and you can scale it and version control it."

"I like the fact that JMeter integrates well with other tools," adds the Founder and Principal Consultant at a tech services company.

A Quality Engineering Delivery Leader at a financial services firm says, “The performance of the solution is excellent. They have designed the product so that it is very easy to configure. You can basically do anything you like with the product. It's not very restrictive. We like the fact that the technology is open-source.”

Apache JMeter was previously known as JMeter.

Apache JMeter Customers

AOL, Orbitz, Innopath Software, PrepMe, Sapient, Corporate Express Australia, CSIRO, Ephibian, Talis, DATACOM, ALALOOP, eFusion, Panter, Sourcepole, University of Western Cape

Apache JMeter Video

Archived Apache JMeter Reviews (more than two years old)

Filter by:
Filter Reviews
Industry
Loading...
Filter Unavailable
Company Size
Loading...
Filter Unavailable
Job Level
Loading...
Filter Unavailable
Rating
Loading...
Filter Unavailable
Considered
Loading...
Filter Unavailable
Order by:
Loading...
  • Date
  • Highest Rating
  • Lowest Rating
  • Review Length
Search:
Showingreviews based on the current filters. Reset all filters
Programmatore software at a tech services company with 201-500 employees
Real User
A stable solution with a lot of connectors and easy setup
Pros and Cons
  • "The features that I appreciate are quite basic. It is easy to ramp up the threads and start calling the application. A lot of connectors can already be found within Apache JMeter, but we are not using the entire set because the integration between the customers and platform is based on HTTP. We are just going to produce lots of HTTP sequences."
  • "Currently, the integration pipeline is implemented by using Jenkins or a similar tool platform. These are continuous integration tools. As far as I know, integration is done by using custom scripts. It would be good if the integration with a continuous integration pipeline, like Jenkins or Hudson, can be done out of the box without using a script."

What is our primary use case?

We are using Apache JMeter for profiling and stress testing the application. It is a part of our delivery architecture to stress test the application. It is a part of the build chain. It is not just something that we use internally to verify what we are going to use or going to do.

In terms of a specific use case, we are using Apache JMeter to simulate user engagement. Our software is about customer engagement, and we use JMeter to simulate the customers and engage them on our platform as aggressively as we can.

What is most valuable?

The features that I appreciate are quite basic. It is easy to ramp up the threads and start calling the application. A lot of connectors can already be found within Apache JMeter, but we are not using the entire set because the integration between the customers and platform is based on HTTP. We are just going to produce lots of HTTP sequences.

What needs improvement?

Currently, the integration pipeline is implemented by using Jenkins or a similar tool platform. These are continuous integration tools. As far as I know, integration is done by using custom scripts. It would be good if the integration with a continuous integration pipeline, like Jenkins or Hudson, can be done out of the box without using a script.

For how long have I used the solution?

I have about two years of experience with JMeter. I started to use it two years ago with a new job.

Buyer's Guide
Apache JMeter
December 2022
Learn what your peers think about Apache JMeter. Get advice and tips from experienced pros sharing their opinions. Updated: December 2022.
655,774 professionals have used our research since 2012.

What do I think about the stability of the solution?

It, for sure, is stable. Apache JMeter is a consolidated tool that has survived for several years. 

What do I think about the scalability of the solution?

It is mainly used by automation tools. These automation tools are around whenever needed. We have about 70 people, and this is a spot adoption.

How are customer service and support?

I have never tried Apache support.

How was the initial setup?

The initial setup is quite easy because this is just a compressed archive. There is no real installation of the tool. You just need to extract it to integrate with Jenkins Pipelines and automatic tools by using the command-line calls. After you expand the archive, you have to adjust Apache JMeter to its proper configuration, which is not a big issue.

What's my experience with pricing, setup cost, and licensing?

In terms of open-source adoption, it is completely free.

What other advice do I have?

I would recommend this solution. 

I would rate Apache JMeter a ten out of ten. I don't have anything else to compare it with, and I don't know if a competitor can achieve better.

Which deployment model are you using for this solution?

On-premises
Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
Quality Assurance Test Manager at a printing company with 5,001-10,000 employees
Real User
Cost-effective and straightforward to deploy, but the reporting needs to be improved
Pros and Cons
  • "The recording and playback functionality is helpful."
  • "The reporting is not very good."

What is our primary use case?

When I was last using JMeter, we were simulating 200 concurrent users and evaluating performance based on transaction times. We were defining SLAs based on the results.

Essentially, we created load scenarios and testing different ones using different workload models.

What is most valuable?

The recording and playback functionality is helpful.

What needs improvement?

The reporting is not very good.

When we run with multiple users, it takes a lot of memory.

With respect to the recording and playback functionality, the auto-correlation parameterization is not easy and should be improved.

For how long have I used the solution?

I have been using Apache JMeter for about four years.

What do I think about the stability of the solution?

There are issues with stability when running with multiple users because it consumes a lot of memory.

What do I think about the scalability of the solution?

Scalability is fine, although it is important to remember that JMeter doesn't run on its own. It needs to work with load-generations such as BlazeMeter. LoadRunner is the same in that you need a cloud-based infrastructure to run it.

How are customer service and technical support?

There is no official support. There is a forum where you can ask questions and they respond to you, but the technical support that we have with LoadRunner or NeoLoad is not available.

Which solution did I use previously and why did I switch?

I have used many similar solutions in the past such as New Relic, AppDynamics, NeoLoad, and Micro Focus LoadRunner Enterprise.

JMeter is not as good as LoadRunner or NeoLoad, and it isn't as easy to use, but it's okay because there is no cost. LoadRunner is too expensive, in my opinion. NeoLoad is cheaper, although not significantly.

From what I have seen, many companies are adopting JMeter because it's free. Especially in Canada, using JMeter seems to be the new trend. Some companies are choosing NeoLoad over LoadRunner because it is easier, faster, and cheaper. Whatever they need to do can be completed quicker. The main problem with NeoLoad is that obtaining resources is harder.

Given all of the choices, my preference would be to implement NeoLoad.

How was the initial setup?

The initial setup is straightforward. I would not say that it is complex and if you already have the file downloaded then it will only take about half an hour to deploy.

What about the implementation team?

I took care of the deployment myself.

What's my experience with pricing, setup cost, and licensing?

I was using the free version of the software.

What other advice do I have?

My advice for anybody who is considering JMeter is to just install it and try it. Creating scripts is a different process when you compare it to LoadRunner or Neoload. There is different terminology compare to these two products, so if somebody has not used JMeter then it may seem difficult at first.

I would rate this solution a six out of ten.

Which deployment model are you using for this solution?

On-premises
Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
Buyer's Guide
Apache JMeter
December 2022
Learn what your peers think about Apache JMeter. Get advice and tips from experienced pros sharing their opinions. Updated: December 2022.
655,774 professionals have used our research since 2012.
Jayanta Kar - PeerSpot reviewer
Automation and Nft Manager at a computer software company with 10,001+ employees
Real User
Excellent for load testing in use cases with high TPS
Pros and Cons
  • "When there's a high number of TPS I can achieve more transactions per seconds given the hyper-limitations."
  • "JMeter output reports can be difficult to understand without training."

What is our primary use case?

Our use cases are for applications such as E-commerce sites where there are multiple concurrent users. In some cases, there is also API load testing where a high number API calls are required for load testing. 

How has it helped my organization?

Apache J meter has proved to be the tool of choice for Performance testing with various features and also it comes without any cost.

What is most valuable?

A valuable feature is that when there's a high number of TPS I can achieve more transactions per second given the hyper-limitations. If the TPS is more than 5,000 TPS, the CPUs get very high (for Load runner)  and JMeter handles that well. I'd also say that the scripting flexibility like co-relation using regular expression is good, it's in the central control. Those would be the two major valuable features. 

What needs improvement?

We have some scenarios for diameter load testing where TPS requirements are very high, 30K or 40K TPS. In the telco area, this is for simulating mobile usage. However, diameter load testing can be difficult in J Meter. The only way to imitate Diameter requests and process the responses of these requests is to implement them in the code of JSR223 samplers. JMeter generally provides synchronous calls. It's something that could maybe be improved in the future, because for achieving that very high kind of TPS, more than 30K, 40K requires a asynchronous solution. It's not a common thing, it's really very specific to the telco domain and a very few projects. 

For how long have I used the solution?

I've been using this solution for about 18 months. 

What do I think about the stability of the solution?

We haven't encountered any major issues with the stability.

What do I think about the scalability of the solution?

Scalability is good. 

How are customer service and technical support?

Support is more open-sourced. There is a lot of material available online so we haven't had to contact Apache technical support and we have our own team in the company who maintain the product.

Which solution did I use previously and why did I switch?

We sometimes use LoadRunner for various purposes but currently, we use JMeter 80% of the time because LoadRunner's CPU consumption becomes very high for TPS more than 5000. Also, LoadRunner is not free so we have to pay for the license. We're using it for very few projects now. JMeter can easily achieve a high TPS but LoadRunner has a lot of high-resource requirements. 

LoadRunner's advantage is that the output report is very comprehensive. JMeter reports are only good for the people who understand the detailed logs. Out-of-box, LoadRunner has nice reports and ease of execution. In JMeter, it's also simple but requires some online training or knowledge on the loader and the setup. 

How was the initial setup?

Initial setup is quite straightforward, it takes about two hours, no more, although each project is different. It depends on the scripting that goes into the environment. In terms of the product alone, the deployment doesn't take long. If you include the hardware, it can take about a week because the hardware calls for a lot of couplers and storage units and that takes time.

What other advice do I have?

For the load test which requires high transactions per second to be achieved, in the range of 15K transactions per second, this is a very good tool to use and the biggest advantage is that it is free. It has a big user group, so we can find solutions easily if there are any challenges. It's really a very good open-source tool for any organization to look at in terms of performance tests.

I would rate this solution an eight out of 10. 

Which deployment model are you using for this solution?

Public Cloud

If public cloud, private cloud, or hybrid cloud, which cloud provider do you use?

Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
UdayKumar - PeerSpot reviewer
Lead SDET at Unify Technologies
Real User
Good reporting, but it should be more user-friendly, support Ruby, and support dynamic throughput
Pros and Cons
  • "The most valuable features are the integration with Jenkins and the reporting."
  • "The interface could be made more user-friendly."

What is our primary use case?

We are writing scripts using various tools in multiple environments. We have tests set up where you have to pass the credentials in to get a session ID, and it sets and checks the cookies. Then, we check the response values and pass them on to the next function.

We have this solution integrated with Jenkins and we also use it for database performance testing.

What is most valuable?

The most valuable features are the integration with Jenkins and the reporting.

JMeter allows us to automate performance testing and raise diagnostic functions.

What needs improvement?

This solution should support the Ruby programming language for scripting.

JMeter should support dynamic throughput so that we can reduce or increase it during the execution of the scripts. 

For performance testing, we would like to be able to select different bandwidths such as 3G or 4G.

The interface could be made more user-friendly.

For how long have I used the solution?

I have been working with Apache JMeter for the past three months.

What do I think about the stability of the solution?

We have not had any problems with stability.

What do I think about the scalability of the solution?

So far, I haven't had any issues with scalability when performance testing.

How are customer service and technical support?

We have not yet been in contact with technical support.

Which solution did I use previously and why did I switch?

I have also started working with BlazeMeter. One of the differences is that the automation is not possible in BlazeMeter. However, BlazeMeters allows for the selection of different bandwidths, it takes less time to write tests, and it is more user-friendly.

How was the initial setup?

The initial setup is straightforward. We have not yet completed our full deployment with all of the tests.

What about the implementation team?

Our internal team was responsible for the implementation.

Which other solutions did I evaluate?

We are currently comparing the differences between Apache JMeter and BlazeMeter.

What other advice do I have?

JMeter is a good solution but it requires writing manual scripts and takes longer than BlazeMeter. It takes longer to cover the same number of use cases and is not as user-friendly.

I would rate this solution a four out of ten.

Which deployment model are you using for this solution?

On-premises
Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
Deepak Dhar - PeerSpot reviewer
DevSecOps Manager & IT-Business Partner - Digital Platforms (Brand - Comms - Events ) at a healthcare company with 10,001+ employees
Real User
Top 20
Good reports and analysis capabilities with a quick initial setup
Pros and Cons
  • "The reports and analysis tools are very good. They are the solution's most valuable features."
  • "It's not easy to get the data from one place or to do customizations."

What is our primary use case?

We primarily use the solution to manage our capabilities. Wherever the business goes, we need to work within the expected benchmark. In terms of the user bases we deal with, most of these are products from different locations with different types of functional flow-out. There are environmental integrations also running in the background, which a user is basically accessing while going through a particular workflow. 

We'll also handle the reporting section of our data management responsibilities utilizing this system.

What is most valuable?

The reports and analysis tools are very good. They are the solution's most valuable features.

What needs improvement?

The GUI could be improved. When we go into GUI mode, there are occasions where it will not sync with our expectations. There are crashes that happen that will stop the solution from performing. It seems we get minor glitches when we go into GUI mode.  

The data client architecture that we have isn't so great. If we are to consume the data, it won't clear because there is tech running on different agents. 

When I need to pull the reports from different agents, it's not user-friendly. The reporting can be difficult to handle. It's hard to increase it if you are working on a client's architecture.

It's not easy to get the data from one place or to do customizations. 

There are other solutions that allow users to model their load and structure with them. You can't do that on JMeter. 

On other solutions, like Silk Performer, you can do network packeting, which you can't do on JMeter. They should add this to the solution as a capability in the future.

The support management needs improvement. Support is coming from consultants; you will not be able to get on-premise support from all of their agents in one place. On Silk Performer, for example,  they have the capability where you can basically have a summarized report from different agents.

For how long have I used the solution?

Personally I've been using the solution for the last 10 years. However, in the program we are in, I've only been using this tool for six months.

What do I think about the stability of the solution?

If you want stability, you will have trouble, because the system is not uniform. You have to do a lot of modeling. 

If you're talking about mobile platforms, JMeter will not allow you to capture the performance of applications on your mobile platform. It's an area where the solution is lacking. It doesn't have the capability to allow developers to check the performance on an AP app.

What do I think about the scalability of the solution?

You can scale the solution, but you will run into issues if you want to run, say, 1,000 users.  You will have to configure multiple agents and then distribute your load. It's quite difficult to manage everything on a single agent.

How was the initial setup?

The initial setup is straightforward. We found it quite easy. For us, it wasn't complex at all. The solution is up and running quite quickly and doesn't take long at all to set up and deploy.

Which other solutions did I evaluate?

We've looked at Silk Impact and Load Balancer in the past. Each of these solutions offers a few different functionalities that aren't necessarily available on JMeter. For example, on both, there are more network features available that aren't included in JMeter and you can model your load and structure with them as well.

What other advice do I have?

If companies have mobile development, they should use JMeter. If they want to integrate it with the PA pipeline, JMeter will work well. However, there are other platforms and solutions as well.

Other options include Silk Performer and Load Impact. These two solutions have other capabilities that JMeter doesn't have. You can basically model your load and structure with them. You can't do that on this solution. On Silk Performer, you can also do network packeting, which you can't do on JMeter.

Then there are some network features available in Load Impact as well as Silk Performer that aren't in JMeter. 

I'd rate the solution eight out of ten. The solution has certain limitations, but it's still a good product. 

Which deployment model are you using for this solution?

Public Cloud

If public cloud, private cloud, or hybrid cloud, which cloud provider do you use?

Microsoft Azure
Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
AvinashYH - PeerSpot reviewer
Senior Software Test Engineer at Loyal Hospitality Pvt. Ltd
Real User
Good performance testing and reporting, helpful user community, but it hangs with too many concurrent threads
Pros and Cons
  • "The most valuable feature in JMeter is the Thread Group, which helps us to see whether the performance is good."
  • "Automation is difficult in JMeter."

What is our primary use case?

We use Apache JMeter for lower-end performance testing. Sometimes we use it for Rest API testing, as well.

How has it helped my organization?

This solution allows us to have a detailed look at performance using different metrics. We can look at the deviation and the median values and see how it is performing. We can also analyze our servers and see how they are performing in terms of CPU and how much memory is allocated.

What is most valuable?

The most valuable feature in JMeter is the Thread Group, which helps us to see whether the performance is good.

The plugin for git allows us to scan files directly from the GitHub site.

The reporting is good and there are a lot of different formats available.

If you want to see the data that is associated with a response then you can download it in JSON format.

There is a large user community that can help you to learn and use JMeter.

What needs improvement?

When we are testing with too many threads then the solution hangs.

JMeter does not support JavaScript.

Automation is difficult in JMeter.

For how long have I used the solution?

We have been using JMeter for the past two years.

What do I think about the stability of the solution?

Stability becomes an issue if we use more than 10 or 20 threads.

How are customer service and technical support?

We have not been in contact with Apache technical support.

There is a large user community and if you need help then you can get instant help from them.

Which solution did I use previously and why did I switch?

We also use Postman and when it comes to making a very large request, it is the better option. It is also a user-friendly solution and you do not have to go through a lot of documents to start using it.

JMeter works better for iterative testing.

How was the initial setup?

It is easy to set up and administer JMeter.

What's my experience with pricing, setup cost, and licensing?

This is an open-source product.

What other advice do I have?

This is a good solution but many features would have to be added to use this tool alone. Postman has more features, so JMeter is really only good for performance testing.

I would rate this solution a seven out of ten.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
Software Architect at a computer software company with 10,001+ employees
Real User
Script-generation is helpful for people with little programming knowledge
Pros and Cons
  • "JMeter is easy to use for a user who doesn't have too much knowledge of programming or certain languages."
  • "Considering the kinds of tests we are performing here, where we launch several tests at the same time as a batch request, JMeter is not the best tool for the job. Those kinds of things could be done easily with other tools, like T6."

What is our primary use case?

Our primary use for it is testing websites: load testing and performance testing.

What is most valuable?

JMeter is very easy for a user who doesn't have too much programming knowledge, as scripting is not mandatory nor essential to create the testing scenarios. This means the user doesn't need to know how to write code to run a complete set of load tests. Although, If you're an advanced user, JMeter does gives the possibility of using BeanShell, Groovy or Java to script your tests.

JMeter has a very friendly and intuitive User Interface which allows users to configure the scenarios reusing components already defined, which you can also modify to fit the requirements of the tests.

One very important feature of JMeter is that it is open source and it can be easily extended through a huge set of available plugins.

What needs improvement?

Considering the kinds of tests we are performing here, where we launch several tests at the same time as a batch request, JMeter is not the best tool for the job. Those kinds of things could be done easily with other tools, like k6. It would be simpler that way. JMeter is a very old tool. It has been around for about 15 years. While it has been improved over the last few years, it is a little complicated to run several tests at the same time with different sites.

JMeter could be easier. It would be a great improvement if it was easier to integrate with the CI deployments, with tools like Jenkins or CircleCI.

For how long have I used the solution?

Our teams have been using it for about five years.

What do I think about the stability of the solution?

It is a very stable tool.

What do I think about the scalability of the solution?

In my opinion, it is not scalable. We haven't found an easy way to do that.

We would like to increase our usage but we are trying to figure out the best way to integrate it with our new circuit. Our plan is to use a tool like this with customers. That is the reason we are evaluating the pros and cons of different tools. We want to incorporate these tools into our web development process, mainly in our CI/CD process. We are trying to figure out how to use tools that are designed to empower developers.

Which solution did I use previously and why did I switch?

We have used LoadUI and SmartBear with tools like Selenium. Some of the developers, at that time, were used to JMeter and they introduced it. It is a very nice tool.

How was the initial setup?

The initial setup is simple. It is not a hard tool to use. It has a very nice UI and the end-users are very comfortable using the tool. People deploy it themselves.

What other advice do I have?

I would definitely recommend JMeter. It is a very nice tool. It is easy to use and there are several components already developed in the tool that you can use to simplify the process of defining and redefining scripts. But I would like it to be more flexible and integrate with other tools.

Which deployment model are you using for this solution?

On-premises
Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
Executive Director/Consultant at a tech vendor with 1,001-5,000 employees
Consultant
Cost-effective and has good community support
Pros and Cons
  • "The most valuable feature for us is the available information on the forums and to be able to discuss and get answers from the people that are involved in using this tool."
  • "There is some work to be done with the integration."

What is our primary use case?

In general, the primary use case of this solution is for performance testing and load testing.

Typically it is for load tests. They have to focus on three metrics including throughput, response times, and the utilization of the resources including the CPUs.

We are delivering some bank houses with performance tests, through JMeter.

The deployment model varies depending on the client. Some have a private line.

What is most valuable?

From the customers' perspective, the most valuable feature is the price.

The most valuable feature for us is the available information on the forums and to be able to discuss and get answers from the people that are involved in using this tool.

What needs improvement?

They have to find a way to prepare the script or to prepare a detailed analysis. We have to collect all the information on each of the services we have to call. Based on this they have to collect in the phase of preparation for the performance test and then we can run our audit. It is easy to prepare a script to run a performance test.

You can't rely on the support. It's something that is not fully working.

The scalability of this solution needs some improvement.

There is some work to be done with the integration.

For how long have I used the solution?

I have been using this solution for over five years.

What do I think about the stability of the solution?

This solution is stable, but it does have one limitation with the number of throughputs. For example, if I want to generate four, five, or two thousand people working concurrently then they have to prepare our setup for ten PC servers.

What do I think about the scalability of the solution?

Normally they can run a load for five thousand people, customers, or users working concurrently. It is scalable but needs some improvement.

How are customer service and technical support?

The technical support on the site is not fully functioning.

How was the initial setup?

The initial setup is easy and straightforward.

You can have it running within thirty minutes.

What about the implementation team?

I had always implemented this solution myself, but It's been a couple of years since I did it last.

Now it is done through our company technicians.  I just coordinate activity-performance tests for our customers.

What's my experience with pricing, setup cost, and licensing?

This is an open-source solution, and there are no fees.

What other advice do I have?

The requirements for most of our customers is to provide them with cheap solutions, and most prefer not to pay for software. This is the right solution for them based on some of the open-source tools that are available.

There are some commercial solutions that provide better integration to the solutions that must be tested when running loads.

I would rate this solution an eight out ten.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
Intermediate Technical Test Analyst (Mobile Lab SME HP Mobile Center and Appium) at a financial services firm with 5,001-10,000 employees
Real User
A scalable, stable solution that's very easy to use
Pros and Cons
  • "The ease of use is the solution's most valuable feature. Also, the ability to easily create test cases is also very good. It's easy to just ramp up on the solution."
  • "In future releases, it would be helpful if there was an integration with ALM Octane."

What is our primary use case?

We primarily use the solution for API testing.

What is most valuable?

The ease of use is the solution's most valuable feature. Also, the ability to easily create test cases is also very good. It's easy to just ramp up on the solution.

What needs improvement?

The solution is new to us. I'm not sure if we're using the full capabilities of the solution yet, but from what we have used, we're quite satisfied.

In future releases, it would be helpful if there was an integration with ALM Octane.

For how long have I used the solution?

I've been using the solution for six months.

What do I think about the stability of the solution?

The solution is stable.

What do I think about the scalability of the solution?

The solution is very scalable.

How are customer service and technical support?

We're not using too many features of the solution itself, so we haven't needed the assistance of technical support yet.

Which solution did I use previously and why did I switch?

We previously used LoadRunner. We decided to switch because on LoadRunner scaling up was a problem.  We needed licenses to do so and it was becoming a problem for us to simulate the number of users we were expecting. 

How was the initial setup?

The general setup of the solution is straightforward.

Which other solutions did I evaluate?

Before we chose JMeter, we looked at NeoLoad, but it looked to have the same issues surrounding licensing that LoadRunner did.

What other advice do I have?

We are using the on-premises deployment model.

The advice I would give to others would be to look at your requirements and decide if the solution is right for you. For us, it was API testing and JMeter is very good for this.

I'd rate this solution nine out of ten.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
Senior Cloud Performance Engineer at Oracle
Real User
Top 10
A cost-effective solution that is easy to use and can be installed in any network

What is our primary use case?

This is a cost-effective tool, which provides people the ability to do POCs without worrying about costs.

How has it helped my organization?

Apache JMeter provided us with the opportunity to test applications for clients who did not have the budget to employ heavy-duty tools like LoadRunner.

Additionally, due to ease of installation, it can be installed in any client network, without the requirement for port opening.

What is most valuable?

API testing, Database Testing, and MQ testing can be done with ease.

What needs improvement?

The tool should be made a bit more robust, and better support should be made available.

For how long have I used the solution?

We have been using this solution for two years.
Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
ArtemCheremisin - PeerSpot reviewer
Performance Test Consultant at Avalanche Labs
Real User
Top 20
Continuously offers new features and plug-ins and has valuable coding features
Pros and Cons
  • "The solution offers a lot of plug-ins and a huge continuously developing community that is regularly offering new features and plug-ins."
  • "The solution could use some sort of educational features to offer tips and hints to help users navigate it better. They should improve the manuals and help files."

What is our primary use case?

I set a script reporting and execute test report and analysis on almost all stages of the performance cycle except in regards to requirements and the environment.

How has it helped my organization?

we saved a lot on licenses for commercial tool

What is most valuable?

The features that helpful are the coding and the opportunity to work with the SQL directly in order to report on the data. You can collect all the test data from DB with these and then do post test execution metrics collection as well(if you need DB)

What needs improvement?

RegEx Extractor needs improvement. It's a headache for many people. The solution could use some sort of educational features to offer tips and hints to help users navigate it better. They should improve the manuals and help files. I've searched the internet for answers over the past year or so, and I haven't come across anything that is helpful. The community help files are pretty good, but their own help files are not.

In the next release, it would be helpful to offer more flexible graph handling to be able to combine different metrics into one graph.

For how long have I used the solution?

I've been using the solution for eight years.

What do I think about the stability of the solution?

quitre stable but keep an eye on RAM consumption and smart execution tips (can see on their site)

What do I think about the scalability of the solution?

Scalability is good, though you need to be aware of distributed and non guis test execution modes. 

How are customer service and technical support?

It's a free tool, so they don't offer technical support. If you need help, you need to Google and search for answers on your own.

Which solution did I use previously and why did I switch?

I've used LoadRunner in the past, among other solutions. I switched to Apache because it is free. Other tools are much too expensive and can cost up to $50,000 a year if you are looking at commercial options.

Now Neoload probably the best one (better than Load runner as per my subjective perception) but it's a way too costly 

What's my experience with pricing, setup cost, and licensing?

please take a look at ROI and scal of performance tests. For big projects and organizations consider Load Runner or Neoload, but if you have small project, most likely you wil lstop on Apache Jmeter. Also it depends on the protocol support. But because of big community, Apache jmeter has constant income of new plugins developed

What other advice do I have?

For people thinking about implementing the solution, I would advise that they know what protocols they want to work with. I would recommend doing an evaluation of script maintenance. You need to see how stable your automation disk and performance is. 

I would rate this solution nine out of ten. The solution offers a lot of plug-ins and a huge, continuously developing community that is regularly offering new features and plug-ins.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
Demkó Sándor - PeerSpot reviewer
Test Team Lead at Passed Informatikai Kft.
Real User
A free solution that's stable and easy to use
Pros and Cons
  • "The solution is free. You don't need to worry about licensing costs."
  • "The user interface is a little bit tricky."

What is our primary use case?

We primarily use the solution for performance download testing.

What is most valuable?

It's a solution that's easy to use. It's a bit different from SoapUI and LoadUI.

What needs improvement?

The user interface is a little bit tricky.

For how long have I used the solution?

I've been using the solution for three to four years.

What do I think about the stability of the solution?

The solution is very stable.

What do I think about the scalability of the solution?

It's not 100% scalable like LoadUI, but LoadUI is very expensive. We're an outsourcing company, so we have approximately ten users. We may have plans to expand in the future.

How are customer service and technical support?

The solution doesn't have technical support. If we need answers, we look for them on the internet.

Which solution did I use previously and why did I switch?

Previously, we used the basic version of LoadUI. We switched because LoadUI is very expensive and Apache JMeter is free.

How was the initial setup?

The initial setup was complex. We did have it done in a day. It took approximately one hour. We have about five to six people that manage the solution.

What's my experience with pricing, setup cost, and licensing?

The solution is free. You don't need to worry about licensing costs.

Which other solutions did I evaluate?

We didn't evaluate other options before choosing this solution.

What other advice do I have?

We are using the private cloud version of the solution.

I would rate this solution eight out of ten. If it had a bit of a better user interface, I would rate it higher.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
PeerSpot user
Consultant at a tech services company with 10,001+ employees
Real User
It's very easy to download and setup the entire infrastructure.

Valuable Features:

Extremely light weight.

Very easy to download and setup the entire infrastructure (Controller +LGs)

Ease of scripting

Host of plugins which further boost its effectiveness

Availability Non-GUI mode which consumes even lesser resources than already lightweight GUI version.

Improvements to My Organization:

Robust scripts which could handle build changes

Easy LG(Master & Slaves) configuration setup

Very lightweight and low usage of resources 

Room for Improvement:

GUI tends to freeze and shutdown under more load

Ctrl+z( Undo doesn't work) so gets very inconvenient at times

Cannot do a mass replace( Ctrl+H) on the GUI ( can do it if script is opened via notepad++)

Re-iteration problems during errors - Usually we have login in once only controller, Action in Loop controller and logout in once only controller. Now if an error comes when the user simply logouts due to some unhandled error then the script expects that your session would continue but if that error has made you logout then your next iterations won't go through and vice versa. Suppose you put everything in loop controller and error doesn't log you out then the script start to login when you are already logged in and tends to fail.

Deployment Issues:

We've had no issues with deployment.

Stability Issues:

See above Room for Improvement.

Scalability Issues:

We've had no issues with scalability.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
it_user4032 - PeerSpot reviewer
it_user4032CTO with 11-50 employees
Vendor

Hello Kapil,
Thank you for your review.

Here are my comments on what you write:
- GUI tends to freeze and shutdown under more load => Load testing with GUI is a known ANTI-PATTERN. GUI is to create/record/correlate, NON GUI is for Load Testing. Since 3.1 a WARNING appears on startup.

- Ctrl+z( Undo doesn't work) so gets very inconvenient at times => True, we'll try to find a solution to this

- Cannot do a mass replace( Ctrl+H) on the GUI ( can do it if script is opened via notepad++) => With upcoming 3.2, you'll be happy to see this feature

- I don't understand the last item, so if you're willing to give more details maybe we can help on this, see http://jmeter.apache.org/issues.html

Thank you
Regards
Philippe M. from Apache JMeter Team

See all 2 comments
PeerSpot user
Technical Leader with 501-1,000 employees
Real User
Reporting feature can be improved but allows us to simulate a load of concurrent users on our application

What is most valuable?

Almost all features provided by JMeter are helpful.

How has it helped my organization?

We have a solution called Service Quality Manager portal and is a web based solution to do service management. We do performance testing of this solution using JMeter. It allows us to simulate a load of concurrent users on our application and find out performance bottlenecks in the application.

What needs improvement?

Reporting feature can be improved to provide better test results.

For how long have I used the solution?

We have been using JMeter for 5 years.

What was my experience with deployment of the solution?

I did not encounter any issue while deploying JMeter.

What do I think about the stability of the solution?

I did not encounter any issue with stability.

What do I think about the scalability of the solution?

Yes, While running with a large number of simulated users, sometimes we got thread exceptions and we restarted our testing simulation.

How are customer service and technical support?

As such there is no customer service provided by Apache for JMeter. This software is free to use under Apache licensing system.

Which solution did I use previously and why did I switch?

We tried BadBoy but it lacked features.

How was the initial setup?

Initial setup of JMeter is simple. A user can go ahead and start using it just by following it's documentation.

What about the implementation team?

We implemented JMeter using our own in-house team.

What was our ROI?

I am not sure yet.

What's my experience with pricing, setup cost, and licensing?

As it is java based software an can run on any platform supporting JVM.

What other advice do I have?

I would suggest to verify one's own business need and evaluate certain parameters first like Feasibility and Reliability of Automation, Ease in maintenance of automated test cases, Time saving in Manual or Automated Testing.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
it_user326337 - PeerSpot reviewer
it_user326337Customer Success Manager at PeerSpot
Consultant

How would you suggest improving the reporting features?

See all 2 comments
it_user324075 - PeerSpot reviewer
Founder & CEO at a tech services company with 51-200 employees
Real User
JMeter supports HTTP/HTTPS, DBs, FTP, and other system protocols and easy to crate basic scripts but has limitations on the reports

What is most valuable?

  • Recording/Playback
  • Templates for test plans (NEW)
  • Integrate with Java and many other scripting languages
  • Integrating with external plug-ins

How has it helped my organization?

  • The recording feature makes it easy to create test plans
  • They have templates that make starting very easy (this feature is added recently)
  • Supports a lot of protocols (HTTP, HTTPS, FTP , SMTP etc)
  • Simplicity of use
  • As compared to commercial tools, it provides better reporting features through third-party plug-ins
  • There's also a huge community around tool, and because of that when there's an issue, you can get solutions easily, and without cost For example, we dis a very complex testing of HL7 protocols, which JMeter doesn't support directly, but it's extendable to support the protocols
  • Supports Java and other scripting languages and can extend capabilities.

What needs improvement?

Reporting could be better if it could be like commercial tools. For example, a nice chart and visualizations. You can’t rely on the first 100 test results because the ramp-up and ramp-down time gives false results with outliers. It would be great if an option to ignore first 100 and last 100 records from results and then generate reports, should be great. Needs to support more concurrent users by single JMeter instance (agent/slave)

For how long have I used the solution?

I've been using it for over last five years, and it is a major, active open source tool.

What was my experience with deployment of the solution?

No issues with recent releases

What do I think about the stability of the solution?

There are issues when tests are stopped with large number of concurrent users. Test is not stopped as expected.

How are customer service and technical support?

Customer Service:

There is a huge community around JMeter. Hence free support is available to the users.

Technical Support:

Excellent (from the community)

Which solution did I use previously and why did I switch?

Grinder. Grinder was not user friendly and had to do lot of coding and low level configurations. It was lighter than JMeter.

How was the initial setup?

Initial setup is straight forward. You need to have have JDK or JRE for running JMeter.

What about the implementation team?

We have a team of testers capable of managing the JMeter perfromance testing project effectively.

What other advice do I have?

It's easy to use and free, a huge community, and it offers a risk free to start, so use it for application performance testing. It supports connectivies with databases, FTP, and other system protocols.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
it_user4032 - PeerSpot reviewer
it_user4032CTO with 11-50 employees
Vendor

Hello Janesh,
Thanks for this nice review.
Just one note, since JMeter 3.0 and even more in 3.1, JMeter has now :

- a new Web Reporting feature, see http://jmeter.apache.org/usermanual/generating-dashboard.html

- Scalable Real-time Graphs in NON GUI mode thanks to 3rd party open sources like Graphite or InfluxDB, see http://jmeter.apache.org/usermanual/realtime-results.html

Regards
Philippe M. from Apache JMeter Team

it_user281952 - PeerSpot reviewer
Sr. Performance Test Engineer at a university with 501-1,000 employees
Vendor
It can record, run, and create reports, but it hangs up on large loads.

Valuable Features:

I like the ease of using JMeter. It can be downloaded and can be used to record and run load tests. JMeter can record, run and create reports easily without the need to have licenses, complex installations and paid support. It can be used to performance test services, web applications, database queries, LDAP, etc.

Improvements to My Organization:

We use JMeter for performance testing. We are able to know which module is lagging behind in performance, whether the software or hardware upgrades have improved the performance or not.

Room for Improvement:

I think JMeter hangs up on large loads. I have seen JMeter becoming unresponsive in the middle of a test. It is because of garbage collection in JMeter. We use best practices to mute all listeners, use non GUI mode, etc., but even with many best practices JMeter pauses in between load tests.

Cost and Licensing Advice:

It is a open source tool.

Other Advice:

My advice would be to have load generator machines. Do not run JMeter (or any load testing tool) from a desktop. Do not have just one JMeter installation to generate load. Use several JMeter instances to distribute load. If possible, use JMeter in Client Server mode.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
it_user4032 - PeerSpot reviewer
it_user4032CTO with 11-50 employees
Vendor

Hello Rahul,
If in 2017 using JMeter 3.1 you still experience "hangs up on large loads." as you wrote here then it would be really nice to report this issue to JMeter team:
- http://jmeter.apache.org/issues.html

From my experience with it, I am able since JMeter 2.11 at least to load test with a single JVM on a 8 Core machine with 6g of memory to load test without any issue up to 2000/2500 Threads with a correctly developed test plan. And whenever I switch to distributed load testing run load test on big E-Commerce website with Millions of users per day.

So I am very interested to know what kind of issue you face so that we can either help you to fix your test plan or fix the issue in JMeter if it is there.

Thank you
Philippe M. from Apache JMeter Team and big user of Apache JMeter

PeerSpot user
Consultant at a consultancy with 501-1,000 employees
Consultant
3 Reasons for combining JMeter and WebDriver

The Apache Foundation states the following about JMeter:

The Apache JMeter™ desktop application is open source software, a 100% pure Java application designed to load test functional behavior and measure performance. It was originally designed for testing Web Applications but has since expanded to other test functions.

So in other words, it’s a tool you can use to generate functional load on an application or a platform. This also immediately describes what it is good at: generating load. Yes if implemented well you not only generate dumb-load but also hit the functional application layers with the tool. But the basic function of JMeter is aimed at generating load and measuring the (server) response times during this load.

What does JMeter NOT do?

Despite being capable of generating functional load, JMeter does not render pages nor is it very well equipped to execute embedded JavaScript (it is simply not equipped to do that actually), therefore JMeter will not tell you anything (out of the box) about the render times of pages. Especially not about render times when the server is heavily overloaded by your scripts!

What is WebDriver good at?

SeleniumHQ gives a wonderful description of what Selenium (or nowadays WebDriver) does:
Selenium automates browsers. That’s it! What you do with that power is entirely up to you. Primarily, it is for automating web applications for testing purposes, but is certainly not limited to just that.

In short, what WebDriver does is just about anything that happens within your browser. It does render pages, it does execute JavaScript, it retrieves the pages as if an actual human was clicking on a website. So for fully functional automated testing (or checking to stick with the more correct terminology) WebDriver is perfect.

What does WebDriver NOT do?

Well, it is not quite good at generating load. Since WebDriver basically requires a browser (yes, it is possible to run it headless of course) it is very difficult to generate multiple (virtual) users. That would require a bunch of browsers to start up, when talking about 10 users that may seem feasible, however when talking about generating real load (say several 1000’s concurrent users) a bunch of browsers becomes a lot more difficult to arrange.

Why combine them then?

The logical question then indeed is, why would you combine them? Below I have set out 3 clear reasons why combining JMeter and WebDriver scripts can be an excellent idea.

  • Impact of server-side load on render-times;
    When the load on a server increases, the response times of various parts of a web application may increase as well. These increased response times can have implications on the render time of the web application. For example: a web application heavy with Ajax requests is put under load, the server response times increase, this may result in all Ajax-requests becoming slower, therefore making the website extremely unfriendly to the end-user. When you just run a JMeter script, this will hardly be noticed, and if you do notice it, you cannot express the impact it will have on the user. You can merely speculate about it.
  • Impact of server-side load on functional behavior;
    Given that the server is experiencing increased load and therefore the business-logic of the application is working hard to handle all requests effectively, it can be safe to say the underlying database may also be stressed and therefore responding slower than expected. Slower response times of both application-logic and database requests can result in buggy behaviour of the application. For example incomplete data returned, or worse, a time-out on data or application-logic. How does the application deal with that? How are these errors reported to the end-user? Will the application still function normally within the browser when certain aspects of the application platform are malfunctioning? The best answer to this is by testing the functionality thoroughly while the application is under load. An easy way to test this repeatedly and consistently is by automating these functional tests, for example using (part of) the automated regression test while the servers are under increased load.
  • Advantage of screenshots of fully rendered pages and possible errors with the application under load;
    As a result of the two points mentioned above, it may be extremely useful for both developers/system engineers and your customer to see errors on the pages affected by the increased server load, such as stylesheets not loading or not loading properly, JavaScript not loading, images missing etc. Screenshots (or screen captures in movie format) will help make clear to the customer what the problem is and more importantly how big the impact on the end-user will be.

I have listed 3 reasons why combining JMeter and WebDriver can be a good idea. I’d love to hear your suggestions of more reasons to want to combine the two.

In a follow-up post I will go into more detail on ways to achieve an effective combination of JMeter and WebDriver running along side each other, well timed and generating consistent logging and results.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
it_user69342 - PeerSpot reviewer
it_user69342Consultant at a consultancy with 501-1,000 employees
Consultant

Hey, Peter-Thomas,

The easiest way I can suggest it to just start small: ensure you add timers to your existing webdriver scripts so you know what your application does under normal circumstances.
Once you have the functional side covered in a way that you know what you need to know, start building up a load generator in Jmeter and start running the two alongside one another manually. When you have that working properly and are getting some useful numbers drop the whole thing as a script into your CI environment for running after a full regressiontest.

See all 4 comments
PeerSpot user
Consultant at a consultancy with 501-1,000 employees
Consultant
JMeter is great as a stand-alone load generator, however when you need a distributed load you'll require high server use

Over the past year I have spent a lot of time in close collaboration with Apache JMeter. I used JMeter as a standalone platform, in collaboration with several tools out there and with my own servers. Over the next few posts I will elaborate a bit more on what I have learned from working with tools like BlazeMeter, Loadosophia as well as working with your own remote servers and Jmeter. On top of that I will share some things I consider good practices to use when working with JMeter and building elaborate test scenarios in it.

JMeter and remote servers

Meter is great as a stand-alone load generator, however the moment you need to generate a substantial load or a distributed load, you will require the use of more servers. There are several possibilities to add external servers to JMeter. You can use a load generation platform such as BlazeMeter and Flood.io or you can choose to use your own machines (Disclaimer: I have only worked with Flood and BlazeMeter as services, so I only mention these two. There may very well be other services out there I am not (yet) aware of).

Why use your own remote servers?

Although there are several commercial options to run JMeter tests from a cloud-platform, it might be useful to have your own servers running. Your own machines, for example Virtual Private Servers, will give you at least the following advantages, please bear in mind these are the main reasons for me to run private servers. It may very well be that you have different reasons for doing the same (I’d be happy to hear them btw!):

  • Availability within firewalls: Not all applications are available outside of firewalls. For quite a lot of companies it is not an option to allow external addresses, especially cloud services, within their firewalled environment. When this is the case in your organisation it can be very useful to have your own load generator hosted inside the firewall.
  • Geographical location: not each and every country has platforms available for load generation. Although platforms like Blazemeter have a lot of different locations available, it may very well be that your particular country is not available through a service provider. However getting your own, hosted, machines within a country is generally not too difficult to do. Keep in mind though whenever you want to use something like a Virtual Private Server to verify with the hosting company whether they allow load generation from within their network!
  • Control: a fairly straightforward reason is in order for you to have full, unlimited control over the load generating servers. If you have your own (virtual) servers running you can easily adjust settings, properties and other things in order to make it fully match your (customers) needs.

Why use a service?

Even though there are quite some reasons for using your own machines, the advantages of using a service like BlazeMeter or Flood.io are plenty, I will just highlight the few that for me have tipped the scales several times in favor of a service.

  • Maintenance: Setting up your own servers is time consuming and therefore not cheap. These servers need maintenance even though you quite likely do not constantly need them. A Service is just there. You buy a subscription, be it with a time constraint (monthly) or a load constraint (max amount of concurrent users) or something like that, but the service is simply at your fingertips. The moment you need it, you have it.
  • Multiple locations: the JMeter based services generally make use of the Amazon AWS cloud, thus giving you, the user, an immense amount of servers at your disposal. With this comes a huge worldwide distribution, which can be very advantageous for big world-wide used applications.
  • Support: both mentioned platforms have a very solid support base for their customers. If you have questions, if your tests seem to not run properly or you simply have issues getting something done, there generally is a good support platform available. You can use forums or the actual helpdesk, but either way, there is a good, solid, commercial support-base to help resolve your issues.

In the end, I choose per assignment what best fits my needs. Sometimes I use the VPS, sometimes I use a service. It is important you at least think about what best suits your needs for the test you are about to build and execute.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user

Hi Martin,

Thanks for your input, very useful to know. I am working as a consultant on a relatively new client site, and am preparing to head up the team to write an RFP proposal for a new performance testing tool to be used across the company.

Getting all the technical / protocol details for the various systems is proving to be more challening than it should - they seem to be all Web based (although only support old versions of IE like 8 and 9... I know). I am very tempted to strongly recommend JMeter as the tool (probably with BlazeMeter or Flood as the cloud based service), but I am concerned that a couple of months down the line some legacy system will come out of the woodwork that JMeter would not be able to support.

Hope that all makes sense. Would be interested in any other feedback that you had.

Cheers,

James

See all 5 comments
PeerSpot user
Senior Software Engineer at a tech vendor with 51-200 employees
Vendor
JMeter helps me in measuring performance of my product very effectively

What is most valuable?

I think CSV data config, regular expressions extractor, reporting and distribution testing are the strengths of JMeter.

How has it helped my organization?

It makes life easier by simulating production scenario to check the performance under heavy load. We are able to find out the bottleneck in our system with the results.

What needs improvement?

Better way to handle dynamic elements, to handle session id, browser cookie and cache manager.

For how long have I used the solution?

I have used JMeter for around 6-8 months.

What do I think about the scalability of the solution?

No, distributed testing was helpful for scaling the users in JMeter.

How are customer service and technical support?

Customer Service: 3 out of 5Technical Support: 3 out of 5

Which solution did I use previously and why did I switch?

I never used any performance tool before.

How was the initial setup?

Yes, the initial setup IS easy after following the blogs or Apache JMeter page.
Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
it_user254718 - PeerSpot reviewer
it_user254718Principal QA Engineer at a tech company with 10,001+ employees
Real User

I have started using JMeter recently, I am confused with the sampler number, I ran a simple Login test of 50 threads for 10 min, I see sampler numbers around 3000. Does that mean 3000 users are doing logins or 50 threads have done 3000 requests so far?

See all 2 comments
PeerSpot user
Head of Performance with 51-200 employees
Vendor
Good for vanilla web testing, less so for more complex requirements.
A mature opensource toolset that has been available for many years. Good for vanilla web testing, less so for more complex requirements. Like most opensource offerings it has relatively poor analytics and limited integration with analytics tools like APM although it is relatively easy to extend with plug-ins and there are quite of these available for free download. A good choice where budget is tight and the tech-stack requirement is straight-forward.
Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
it_user100902 - PeerSpot reviewer
it_user100902Head of Performance with 51-200 employees
Vendor

Hi Kobi,

JMeter is not well suited for this sort of requirement as you are likely to have to deal with some sort of API. If however the API is web service based then should be ok. Something more proprietary would be better dealt with scripting in C# in my experience so some other tool choice would be better suited.

KR
Ian

See all 10 comments
it_user5646 - PeerSpot reviewer
QA Expert at a tech services company with 51-200 employees
Consultant
User friendly Java tool, but doesn't support SSL recording or non-web apps

Valuable Features:

1. GUI is very user friendly 2. It's a pure Java tool, which enables me to run it on any platform (i.e. platform independent) 3. Free tool & easy to get.

Room for Improvement:

1. It does not support recording SSL 2. Supports only web based applications 3. Cannot change the recorded scripts
Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
it_user77718 - PeerSpot reviewer
it_user77718QA Expert at a tech company with 51-200 employees
Real User

JMeter DOES support SSL
JMeter DOES support LDAP, Database, Mail, FTP, etc. and can be easily extended with any protocol as plugin or as built-in extension like Beanshell - http://blazemeter.com/blog/queen-jmeters-built-componentshow-use-beanshell
JMeter scripts can be correlated with pre and post processors, variables, properties.

See all 4 comments
it_user1428 - PeerSpot reviewer
Architect at a tech vendor with 10,001+ employees
Vendor
Powerful open source with wide community support. Needs experience to match commercial alternatives.

Pros:
Open source, powerful, wide community of support - it works.

Cons:
Experience needed to build custom tools around Jmeter in order to match capabilities of most commercial alternatives.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
Buyer's Guide
Download our free Apache JMeter Report and get advice and tips from experienced pros sharing their opinions.
Updated: December 2022
Buyer's Guide
Download our free Apache JMeter Report and get advice and tips from experienced pros sharing their opinions.