What is our primary use case?
Our primary use case for BlazeMeter is performance testing. We leverage BlazeMeter as our enterprise performance testing platform. Multiple teams have access to it, and we execute all of our load tests with BlazeMeter and do all the reporting through it. We also use it for mock services.
We have a hybrid deployment model. The solution is hosted and maintained by BlazeMeter. We also have on-premise locations within our network that allow us to load test applications behind our corporate firewalls. That's for test environments and non-production applications that are not externally available. It's a hybrid role that is mostly SaaS, but the on-premises component allows us to execute those load tests and report the results back to the BlazeMeter SaaS solution.
The cloud provider is GCP. BlazeMeter also grants access to Azure and AWS locations which you can execute load tests from. They engaged with all three of the major cloud providers.
How has it helped my organization?
BlazeMeter gives us a centralized place to execute load tests, do reporting, and have different levels of user access control. BlazeMeter has a full API, which is the feature that's given us a lot of value. It allows us to integrate with BlazeMeter in our CI/CD pipelines, or any other fashion, using their APIs. It helps increase our speed of testing, our reporting, and our reporting consistency, and gives us a central repository for all of our tests, execution artifacts, and results.
BlazeMeter added a mock services portion. We used to leverage a different product for mock services, and now that's all done within BlazeMeter. Mock services help us tremendously with testing efforts and being able to mock out vendor calls or other downstream API calls that might impact our load testing efforts. We can very easily mock them out within the same platform that hosts our load tests. That's been a huge time saver and a great value add.
BlazeMeter absolutely helps bridge Agile and CoE teams. It gives us both options. BlazeMeter is designed so that we can grant access to whoever needs it. We can grant access to developers and anyone else on an Agile team. It allows us to shift left even farther than a traditional center of excellence approach would allow us.
It absolutely helps us implement shift-left testing. One of the biggest features of shifting left is BlazeMeter's full, open API. Regardless of the tools we're leveraging to build and deploy our applications, we can integrate them with BlazeMeter, whether that's Jenkins or some other pipeline technology. Because BlazeMeter has a full API, it lets us start tests, end tests, and edit tests. If we can name it, it can be done via the API. It tremendously helps us shift left, run tests on demand, and encode builds.
Overall, using BlazeMeter decreased our test cycle times, particularly because of the mock service availability and the ease with which we can stand out mock services, or in the case of an Agile approach, our development teams can stand out mock services to aid them in their testing.
It's fast, and the ability to integrate with pipelines increases our velocity and allows us to test faster and get results back to the stakeholders even quicker than before.
The overall product is less costly than our past solutions, so we've absolutely saved money.
What is most valuable?
The orchestration feature is the most valuable. It's like the tourist backend component of BlazeMeter. It allows me to essentially give BlazeMeter multiple JMeter scripts and a YAML file, and it will orchestrate and execute that load test and all those scripts as I define them.
The reporting feature runs parallel with orchestration. BlazeMeter gives me aggregated reports, automates them, and allows me to execute scheduled tests easily on my on-premise infrastructure.
BlazeMeter's range of test tools is fantastic. BlazeMeter supports all sorts of different open-source tools, like JMeter and Gatling, and different web driver versions, like Python and YAML. If it's open-source, BlazeMeter supports it for the most part.
It's very important to me that BlazeMeter is a cloud-based and open-source testing platform because, from a consumer perspective, I don't have to host that infrastructure myself. Everything my end users interact with in the front-end UI is SaaS and cloud-based. We don't have to manage and deploy all of that, which takes a lot of burden off of my company.
The open-source testing platform is fantastic. They support all of the open-source tools, which gives us the latest and greatest that's out there. We don't have to deal with proprietary formats. A secondary bonus of being open-source and so widely used is that there is a tremendous amount of help and support for the tools that BlazeMeter supports.
What needs improvement?
BlazeMeter needs more granular access control. Currently, BlazeMeter controls everything at a workspace level, so a user can view or modify anything inside that workspace depending on their role. It would be nice if there was a more granular control where you could say, "This person can only do A, B, and C," or, "This user only has access to functional testing. This user only has access to mock services." That feature set doesn't currently exist.
For how long have I used the solution?
I have used this solution for almost five years.
What do I think about the stability of the solution?
The stability has absolutely gotten better over the years. They had some challenges when they initially migrated the platform to GCP, but most of those were resolved. Overall, they have very high availability for their platform. If there's an issue, they have a status page where they publish updates to keep customers in the loop.
If you email their support team or open a ticket through the application, they're always very quick to respond when there's a more global uptime issue or something like that. Overall, they have very high availability.
How are customer service and support?
Technical support is absolutely phenomenal. I've worked with them very closely on many occasions. Whether it's because we found a bug on their side, or an issue we're having with our on-premises infrastructure, they're always there, always willing to support, and are very knowledgeable.
I would rate technical support as nine out of ten.
How would you rate customer service and support?
Which solution did I use previously and why did I switch?
We previously used HP Performance Center. We used HP Virtual User Generator as a predecessor to JMeter for our scripting challenges.
We switched because it's a very outdated tool and toolset. BlazeMeter is a more modern solution. It supports many more tools, and it allows us to solve problems that were blocked by the old solution.
The BlazeMeter platform is designed to be CI/CD, so it has continuous integration, it's continuous delivery-friendly, Agile-friendly, and it has all of the modern software development methodologies.
Our old solution didn't really cooperate with that. It didn't have the API or any of the test data functionality that we've talked about with generating or pulling test data. It didn't have any of the mock services. BlazeMeter gave us the kind of one-stop-shop option that allows us to accelerate our development and velocity within our Agile space.
How was the initial setup?
From my company's side, I'm the "owner" of BlazeMeter. I worked with a support team to set up the on-premises infrastructure. I still work with them.
Deployment was straightforward and simple. We pulled some Docker images and deployed them. The whole on-premise deployment methodology is containerized, whether it's standalone unit servers running Docker or a Kubernetes deployment, which allows you to deploy on-premise BlazeMeter agents through a Kubernetes cluster and your own GCP environment or on-premises Kubernetes environment.
What about the implementation team?
We worked directly with BlazeMeter.
Which other solutions did I evaluate?
We evaluated Load.io and a couple of other solutions. When we brought on BlazeMeter five years ago, they were absolutely the leader in the pack, and I believe they still are. They have a much more mature solution and an enterprise feel. The whole platform is much more developed and user-friendly than some of the other options we evaluated.
I don't know if there are any features in other platforms that BlazeMeter didn't have; it was mostly the other way around. There were things BlazeMeter had that other platforms didn't have, and existing relationships with the company that used to own BlazeMeter, Broadcom.
What other advice do I have?
I would rate this solution an eight out of ten.
It's a fantastic solution and can do so many things. But unless you have a team that's already very experienced with JMeter and BlazeMeter, there will be some ramp-up time to get people used to the new platform. Once you're there, the features and functionality of BlazeMeter will let you do things that were absolutely not feasible on your previous platforms.
We don't really leverage the actual test data integration and creation functionality, but we leverage some of the synthetic data creation. BlazeMeter will let you synthetically generate data for load tests, API, or mock services. We have leveraged that, but we have not leveraged some of the more advanced functionality that ties in with test data management.
The ability to create both performance and functional testing data is not very important to us. A lot of the applications we test are very data-dependent and dependent on multiple downstream systems. We don't leverage a lot of the synthetic data creation, as much as some other organizations might.
We don't extensively use BlazeMeter's ability to build test data on-the-fly. We use it to synthetically generate some test data, but a majority of our applications rely on existing data. We mine that in the traditional sense. We don't generate a lot of synthetic test data or fresh test data for each execution.
BlazeMeter hasn't directly affected our ability to address test data challenges. We don't directly leverage a lot of the test data functionality built into BlazeMeter, but we're trying to move in that direction. We have a lot of other limitations on the consumer side that don't really let us leverage that as much as we could. It certainly seems like a great feature set that would be very valuable for a lot of customers, but so much of our testing is done with existing data.
We haven't had any significant challenges with getting our teams to adopt BlazeMeter. There were just typical obstacles when trying to get people to adopt anything that's new and foreign to them. Once most of our users actually spent time using the platform, they really enjoyed it and continued to use it.
There were no significant hurdles. Their UI is very well-designed and user-friendly. Perforce puts a lot of effort into designing its features and functionalities to be user-friendly. I've participated in a few sessions with them for upcoming features and wire frameworks of new functionalities.
Which deployment model are you using for this solution?
Hybrid Cloud
If public cloud, private cloud, or hybrid cloud, which cloud provider do you use?
Google
*Disclosure: PeerSpot contacted the reviewer to collect the review and to validate authenticity. The reviewer was referred by the vendor, but the review is not subject to editing or approval by the vendor.
