it_user602865 - PeerSpot reviewer
Senior Quality Assurance Automation Engineer / Manager at a non-profit with 51-200 employees
Vendor
I like the ability to spin up as many load generators as I need, and get statistics on response times and the load generators themselves.

What is most valuable?

I really like the ability to spin up as many load generators as I need, and get statistics on response times and the load generators themselves. BlazeMeter support and flexibility is also hugely valuable.

To accurately test load on a webserver it takes traffic (load) from multiple “load generators”.
A load generator is a machine running load testing software (like Jmeter).
Clearly, one couldn’t accurately test 10 load balanced webservers with traffic from a standard desktop.
If one did and saw performance issues, the most likely source would be the desktop doing the testing, or the connection.
If no performance issues were seen it could be that the desktop was not able to emulate the specified load.

One blind spot of Jmeter is it does not tell the user how many threads their machine can handle. Users need to know how to monitor. Most users don’t, and set the threads higher than the machine can handle.
Blazemeter:

- Handles all that complication and makes using Jmeter simple

- Has expert staff that can point you in the right direction if your tests are not working

- Allows users to select how many load generators (machines running Jmeter) will be used during a given test.

o There are a few different contract options (1 server, 5 servers)

o Custom contracts for any number of servers you need. I really appreciate their flexibility on the contracts.

How has it helped my organization?

Building an in-house group of load generation JMeter servers to load test usually takes time and effort. BlazeMeter makes all that simple.

For how long have I used the solution?

I have used JMeter for 4 years and BlazeMeter on and off for 2 to 3 years.

What do I think about the stability of the solution?

There were some minor issues with stability.

Buyer's Guide
BlazeMeter
April 2024
Learn what your peers think about BlazeMeter. Get advice and tips from experienced pros sharing their opinions. Updated: April 2024.
768,924 professionals have used our research since 2012.

What do I think about the scalability of the solution?

We did not encounter any issues here. Scalability is one of the wonderful parts of working with BlazeMeter.

How are customer service and support?

Technical support has been great.

Which solution did I use previously and why did I switch?

I’ve used JMeter from several load generator servers I set up to work together.

How was the initial setup?

Initial setup was easy.

What's my experience with pricing, setup cost, and licensing?

If you don’t see the option you need, just ask them. You’ll be able to get a custom contract. It is not free, but it is a fantastic deal compared to anything else out there.

Most recently I have been using a custom contract with Blazemeter rather than a standard month to month contract. That allows me to pay by test hours times servers used. So I can spin up as many load generators as I need. The statistics on response times and the load generators themselves are standard Blazemeter features available to anyone with a contract. BlazeMeter support has several times helped me figure out what was going on with my tests, that saved me a chunk of time. It's always nice to have some insight from a true Jmeter expert.


Which other solutions did I evaluate?

We considered setting up our own set of load generation JMeter servers. We also looked at HPE LoadRunner and SoapUI.

What other advice do I have?

BlazeMeter is an easy way to get really accurate results from JMeters that you already have without having to build a set of networked JMeters (e.g., one with the UI running a bunch GUI-less slave machines).

Disclosure: PeerSpot contacted the reviewer to collect the review and to validate authenticity. The reviewer was referred by the vendor, but the review is not subject to editing or approval by the vendor.
PeerSpot user
Senior Engineer at CIBER
Real User
Provides great flexibility and we can run executions from multiple locations
Pros and Cons
  • "The solution offers flexibility with its configurations."
  • "Lacks an option to include additional users during a test run."

What is our primary use case?

I mainly use BlazeMeter for scripting. I have a performance testing background, and can directly record my workload with the help of a BlazeMeter plugin. Once I've recorded I can modify a couple of enhancements. I can design my signature in BlazeMeter and choose multiple dedicated IPs, depending on our load. We are system integrators and I'm a senior engineer. 

What is most valuable?

Providing the Cloud load generator is a good feature. We can run executions from multiple locations. Very few tools provide that. They have recently started with separate API testing. I haven't had a chance to use that yet but it might be very helpful. The solution offers flexibility with its configurations, taking around two to five minutes each. You can see the metrics in a graphical view while running the tests. Response times are also visible and can filter whatever we need and get the results of error loads. 

What needs improvement?

Documentation for the solution could be improved because there are some areas, such as licensing costs, where there is a lack of information regarding structure. I'd also like the ability to see a comparison feature after carrying out several tests. I'd like to know the difference in terms of response times and other details. That would be a great feature for them to provide. 

Sometimes we'd like to include additional users during a test run to check application sustainability. We can do it from a scripting end but it would be great if BlazeMeter would provide the option of adding a few more users while running a test.

For how long have I used the solution?

I've been using this solution for five years. 

What do I think about the stability of the solution?

The stability is good. 

What do I think about the scalability of the solution?

The scalability is good. 

How are customer service and technical support?

If there are issues I update it in a chat and the technical support creates a ticket ID for follow-up. Sometimes they respond very quickly and other times they are very slow at resolving the issue. Response times are inconsistent. 

How was the initial setup?

The initial setup is relatively straightforward and user-friendly. We have less than 50 users, we don't do any load tests.

What other advice do I have?

I recommend BlazeMeter because of the accuracy of the results. 

I rate the solution eight out of 10. 

Which deployment model are you using for this solution?

On-premises
Disclosure: My company has a business relationship with this vendor other than being a customer: Integrator
PeerSpot user
Buyer's Guide
BlazeMeter
April 2024
Learn what your peers think about BlazeMeter. Get advice and tips from experienced pros sharing their opinions. Updated: April 2024.
768,924 professionals have used our research since 2012.
it_user577332 - PeerSpot reviewer
Development Manager at a comms service provider with 51-200 employees
Vendor
Can quickly scale out a load test across the world, to simulate realistic world traffic conditions.

What is most valuable?

  • The ability to quickly scale out a load test across the world, just to simulate realistic world traffic conditions.

How has it helped my organization?

For us in particular, we're a product that gets heavily used during the election season. We've always had trouble handling this burst in the past, so this past year, I've started using BlazeMeter at this company. We took from January to July using BlazeMeter to really model out what the traffic looked like in previous years and to be able to handle the load when November came and it went through just fine without a glitch.

What needs improvement?

At the moment, it's heavily based on putting a load on systems through API endpoints/HTTP endpoints. There is a lot of existing tooling out there in organizations that deal with a Selenium-based test and they only have a Selenium web driver option right now, which is not the nicest thing to use. Organizations currently have Selenium tests written and, like us for example, a huge suite in Java. It would be nice to be able to just throw them in there.

For how long have I used the solution?

I’ve been using BlazeMeter for about four years.

What do I think about the stability of the solution?

This is a cloud-based solution. I've never really had any trouble with it.

What do I think about the scalability of the solution?

We have not had scalability issues for our needs. We're talking about scaling stuff out to hundreds of thousands of requests per second and those in turn generate a lot more traffic internally. So with whatever we use, it's perfectly fine.

How are customer service and technical support?

We did not use technical support.

Which solution did I use previously and why did I switch?

We didn't have a previous solution. I personally came to this company in the past year and a half. I've always used BlazeMeter at my other companies and there was a big, gaping hole here. They were doing everything manually with running JMeter tests and this was just a natural fit.

How was the initial setup?

For this audience, if you know JMeter, everything is pretty straightforward. You just sign up and upload your test. As far as support goes, if you do run into stuff, they have a little button on the side that lets you chat with them.

What about the implementation team?

This is all in-house.

What was our ROI?

That's tough to calculate, but I can tell you that BlazeMeter helped increase quality-of-service (QoS) which in turn leads to customers having a positive experience with the product. That, of course, leads to less churn which is a good thing.

What's my experience with pricing, setup cost, and licensing?

The pricing works. They're flexible. They work with you.

Which other solutions did I evaluate?

There was LoadStorm, loader.io, and blitz.io.

What other advice do I have?

Definitely ask your engineering staff if they're familiar with JMeter. If they're not, it would be good to get a quick two-hour tutorial on it and then BlazeMeter will make a lot more sense.

Disclosure: PeerSpot contacted the reviewer to collect the review and to validate authenticity. The reviewer was referred by the vendor, but the review is not subject to editing or approval by the vendor.
PeerSpot user
it_user602877 - PeerSpot reviewer
Senior Performance Engineer/ Staff Qa Member at a tech vendor with 51-200 employees
Vendor
It can generate a heavy load and provide a report that shows how the system behaved under the load.

What is most valuable?

Ability to generate heavy load (up to 5000 concurrent users)

Provide performance transactions summary & statistics

These features are valuable because, in my company, there is a requirement for the product to support up to 5000 concurrent users. As a performance engineer, it’s important to have a tool (like BlazeMeter) to generate heavy load and at the same time provide a report that shows how the system behaved under the load and provide performance statistics.

How has it helped my organization?

By using this tool, a performance engineer was able to identify performance bottlenecks, and after fixes was applied, overall product quality improved.

What needs improvement?

It would be very useful if there was a mechanism to schedule performance runs.

For example, a mechanism to automatically schedule performance tests execution, by using the calendar scheduler. This way, it would be possible to schedule multiple runs overnight, without the need to manually start each performance test.

For how long have I used the solution?

I have used BlazeMeter with JMeter 3.0 and Java 1.8 for six months.

What do I think about the stability of the solution?

Stability was great; no issues.

What do I think about the scalability of the solution?

Scalability was great; no issues.

How are customer service and technical support?

I did not contact support.

Which solution did I use previously and why did I switch?

We did not previously use a different solution.

What's my experience with pricing, setup cost, and licensing?

I think prices are reasonable.

Which other solutions did I evaluate?

Before choosing this product, we did not evaluate other options.

What other advice do I have?

It is an excellent and reliable tool. I liked it very much.

Disclosure: PeerSpot contacted the reviewer to collect the review and to validate authenticity. The reviewer was referred by the vendor, but the review is not subject to editing or approval by the vendor.
PeerSpot user
it_user613551 - PeerSpot reviewer
Lead Software QA Engineer at a tech services company with 51-200 employees
Consultant
It can push large numbers of simulated users at our applications and provides access to detailed reports.

What is most valuable?

The ability to push large numbers of simulated users at our applications and having access to detailed reports: This helps us to have peace of mind that our apps can stand up to production-level loads and beyond.

How has it helped my organization?

We no longer have to guess whether deploying new code will affect performance or not. This ensures we are ready to meet customer demand, even at peak traffic times.

What needs improvement?

I would like to have better insights when tests fail to run properly or when the load engines occasionally break. There have been times when a test fails to start, sometimes due to a bug in my script and other times due to a problem on BlazeMeter’s side, that have required contacting support. It would be better if the interface proved enough feedback via logs, etc., for me to determine the cause and possibly fix it without needing to wait for BlazeMeter support.

For how long have I used the solution?

I have used BlazeMeter for three years.

What do I think about the stability of the solution?

Occasionally, we have long wait times or trouble starting tests.

What do I think about the scalability of the solution?

One load engine was not sufficient to run the JMeter scripts that I’ve written. Scaling up to seven load engines largely eliminated these issues.

How is customer service and technical support?

Support is adequate. Getting initial responses for support issues can sometimes be slow.

How was the initial setup?

It was very straightforward to upload the JMeter scripts and supporting files, and start running tests.

What's my experience with pricing, setup cost, and licensing?

It is somewhat expensive, but the amount of testing and peace of mind it gives us is well worth the cost.

Which other solutions did I evaluate?

We considered creating our own internal JMeter farm, but in the end decided BlazeMeter would do a better job of end-to-end testing.

What other advice do I have?

If you are looking to improve your website performance and ensure maximum uptime by load testing your system from end to end, then there is no better solution that I have found. It is extremely simple to setup.

Disclosure: PeerSpot contacted the reviewer to collect the review and to validate authenticity. The reviewer was referred by the vendor, but the review is not subject to editing or approval by the vendor.
PeerSpot user
PeerSpot user
Chief Technology Officer at a non-profit with 51-200 employees
Real User
We like the ability to run a test repeatedly and that it is easy to make small changes to the test.

What is most valuable?

  • Easy to make small changes to the test without reconfiguration, so we could step increases in load and find the limits without risk in breaking the test scripts. Consistent testing allows for confidence in the results and ease of use by the team.
  • Integration with New Relic - provides insight as to why the test had the results and how we can improve them. New Relic was already in our tool belt, so it was nice to not need to change to an unfamiliar tool.

How has it helped my organization?

Provided an easy to use tool to measure the performance of our most critical business application, along with the expertise of BlazeMeter to properly generate the test case.

It has allowed us to test the impact of vendor updates to their software on the performance of the web component, which we are now doing as updates get released (6 month release cycle, so only one retest thus far).

System Administrators are performing the testing, as the software isn't ours and we don't presently develop our own software this was a new need and we don't have anything to compare to other than anecdotal evidence from users. We were able to compare results post-testing by comparing Google Analytics data and found that the improvements we put into place as result of the testing did have positive effect on both pageload and generation timings along with improving the overall conversion rate for the tool as well.

What needs improvement?

It's a rather expensive tool for running just a few tests — which is our primary use case, if there was a pricing structure that met our use case price wouldn’t be of concern. Worth noting is that we requested they donate our use of the tool — and they did donate the tool for our use — so it did not cost us anything (which we are very thankful for as a nonprofit).

What do I think about the stability of the solution?

We encountered no stability issues in using it.

What do I think about the scalability of the solution?

We encountered no scalability issues, though we were testing at relatively low volumes.

How are customer service and technical support?

Any questions we had were addressed promptly by knowledgeable individuals.

Which solution did I use previously and why did I switch?

We wanted a solution that was easy to interpret and wouldn't require custom tooling beyond building the test. JMeter was the right tool for the job, and BlazeMeter's expertise and easy to use interface allowed us to readily view and compare results without needing to build a custom workflow.

How was the initial setup?

Creating the JMeter test case required some time on our part, partially due to JMeter not supporting JavaScript and the application we were testing requires JavaScript support. We were able to create a test to work around that limitation, but that's more due to BlazeMeter utilizing JMeter than it being a limitation of BlazeMeter as a platform.

What's my experience with pricing, setup cost, and licensing?

Cost is critical as a non-profit, and they donated our access as noted prior. Functionality is also critical — and they had the function we needed, were easy to configure, and integrated with New Relic which saved us a lot of time.

Which other solutions did I evaluate?

The alternative was to use Open Source tools to analyze the results, this solution was selected as we felt it would be faster and easier than rolling a custom solution which just wasn't warranted for how simple our situation is.

What other advice do I have?

It's free and easy to sign up and try, so give it a whirl and see if it meets your requirements. We found it met ours and more, helping us gain confidence from our customers and solve a nagging problem in the performance of a third-party application.

Disclosure: PeerSpot contacted the reviewer to collect the review and to validate authenticity. The reviewer was referred by the vendor, but the review is not subject to editing or approval by the vendor.
PeerSpot user
Associate QA Manager at Xcel Serv Solution
Real User
Great performance testing and API Mocking with reliable stability
Pros and Cons
  • "It supports any number of features and has a lot of tutorials."
  • "The reporting capabilities could be improved."

What is most valuable?

I like the BlazeMeter extension. It is a very good expansion. With it, you can record any type of performance script scenario, and you can generate in a JMeter script, Selenium, et cetera. That is a very good feature. 

If you want to do that, it is used mostly on performance testing, API Mocking, et cetera. The API Mocking capabilities are great.

For performance, it is a very good tool and we don't have too much to do. 

It is UI-based and you can select the load generator and it is auto-scalable. You can select the origin where we want to generate load from it. We can select the bandwidth, for example. 

It supports any number of features and has a lot of tutorials.

The solution is stable.

What needs improvement?

I cannot recall coming across any missing features.

The reporting capabilities could be improved.

It would be ideal if it could incorporate tools such as APM or Dynatrace.

For how long have I used the solution?

I've been using the solution for six years. It's been a while. 

What do I think about the stability of the solution?

We have found the solution to be stable. There are no bugs or glitches. It doesn't crash or freeze. It's reliable. 

How are customer service and technical support?

For the paid version, we have our own technical support for BlazeMeter. They are helpful.

What's my experience with pricing, setup cost, and licensing?

The cost is a bit high for the Indian consumer. For a European or American user, the solution may not seem that expensive.

What other advice do I have?

We are just customers and end-users.

I'd rate the solution at an eight out of ten. We're pretty happy with its capabilities in general.

I'd recommend the solution to other users and organizations.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
Real User
Enables us to monitor our application instances as well as our database to see where the bottlenecks are
Pros and Cons
  • "It has helped us simulate heavy load situations so we can fix performance issues ahead of time."
  • "My only complaint is about the technical support, where sometimes I found that they would not read into and understand the details of my question before answering it."

What is our primary use case?

We use BlazeMeter to run simple POST requests against our application to simulate heavy load. We can then monitor our application instances as well as our database to see where the bottlenecks are.

How has it helped my organization?

It has helped us simulate heavy load situations so we can fix performance issues ahead of time. This ensures that we are ready for real customer load.

What is most valuable?

The most valuable feature is simply the ability to have a basic POST request while simulating many users at one time. Being able to determine either a number of iterations or having all users hit the application for a duration, are both useful options.

What needs improvement?

My only complaint is about the technical support, see further details below regarding customer support.

It would improve the product if their Chrome extension allowed you to modify the JMeter settings.

For how long have I used the solution?

Less than one year.

What do I think about the stability of the solution?

So far I haven't encountered any issues so it seems very stable to me.

What do I think about the scalability of the solution?

We have only used basic, but for one thousand users, it works great.

How are customer service and technical support?

Their support is pretty good and normally gets back to me within one day. One time, instead of the support rep taking the time to understand the details of my inquiry and source of my confusion, they pointed me to documentation to assist me in finding the answer on my own.  On the documentation page, I had to go into a few links in order to find the answer. The rep could have easily said "here is the documentation where we have the information, and you'll need to go here and do this to fix your problem."

I appreciate when support makes it feel like they understand your exact issue and then tell you exactly how to fix it. I also prefer when support gives you exactly what you need as they know the product best.

Which solution did I use previously and why did I switch?

BlazeMeter was the first tool that I used.

How was the initial setup?

I found the initial setup to be pretty simple. I'm using JMeter, so I had to do a bit of research myself on how to change some of the JMeter settings that their Chrome add-on does not allow you to modify.

Which other solutions did I evaluate?

This solution was recommended by a colleague, and I did not evaluate other options.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
Buyer's Guide
Download our free BlazeMeter Report and get advice and tips from experienced pros sharing their opinions.
Updated: April 2024
Buyer's Guide
Download our free BlazeMeter Report and get advice and tips from experienced pros sharing their opinions.