What is our primary use case?
We have been using Sauce Labs to test various browsers and OS combinations as well as test our applications. Our existing automation scripts are written in various technologies, which could be Java, JavaScript, Selenium, Cypress, etc. Jenkins is the tool that we use to typically run our jobs. Through Jenkins, they get scheduled and run in Sauce Labs. This is where we choose to run them, through various browsers and OS combinations.
We use Sauce Labs core and whatever services that we choose to go with, like browsers and mobiles. From there, we can choose the browser and OS versions, etc.
How has it helped my organization?
Without Sauce Labs, we wouldn't be able to test as quickly or simultaneously on various browsers. That is the capability Sauce Labs has brought to us.
The automated functional testing, visual regression testing are pretty important to us, as those are the use cases for which we use Sauce Labs. Specifically because it is hosted in the cloud, we don't have to use as much capacity from our own servers. We don't have to maintain and install various versions of browsers nor do we have to maintain them. It is a good solution and works well for us.
Most of our teams who have solutions to test on mobiles are using the solution's mobile emulators and simulators. Having that availability is very important for a complete testing experience.
We use the solution's emulators, simulators, and different browser version combinations for all our testing. For every release, we validate across various browsers. If it is a mobile application, then we need to validate across various Android and iOS devices as well as the previous few versions of each of the operating systems.
The customer base that we typically serve is vast, diverse, and varied. This makes it very common to have our applications used across various applications, systems, and screen sizes. To test across all that, we needed a system in place. That is why we are using Sauce Labs. Without it, we would either develop something in-house or rely on possibly testing in only one particular system. For example, with Android, our entire customer base who is not Android wouldn't be able to validate and reproduce the user experience.
When we are doing releases, the releases happen across all teams and various services. This means all of them need to test for a particular release deployment in whatever environment simultaneously. Based on the number of concurrent VMs that Sauce Labs provides, we can add or reduce VMs as needed. The amount of concurrency that we have purchased is specifically to support our various teams to do deployments simultaneously. Doing these simultaneous tests has reduced our test execution time.
What is most valuable?
There are a lot of analytics that you can do and look at when you run your jobs. It also gives you how much throttling has been seen across various teams over the past month. It tells you which particular line of code has been failing for however many runs that you have done.
The concurrent usage and VM availability are its strengths. We have found concurrent throttling very useful. So, we know exactly which job or team is using more than their capacity and might need additional capacity.
We have also found the code analysis that gets run on Sauce Labs very useful. It tells us the line of code that has been failing or not received a new command.
Sauce Labs is optimized for automation and integration with the major CI/CD platforms and developer tools. Most of our teams are creating CI/CD pipelines. Some of our teams do many deployments in a day. Without CI/CD, that pace is not possible. Having that capability, for whichever automation tool that we go with, is critically important for us.
What needs improvement?
Some of the trends that it shows. It only allows you to view the last month. Having it go beyond a month, e.g., yearly trends, would be good. While the yearly trends are available, they are available to Sauce Labs administrators, who are internal to Sauce Labs. I don't know if they are available for customers yet.
Buyer's Guide
Sauce Labs
March 2024
Learn what your peers think about Sauce Labs. Get advice and tips from experienced pros sharing their opinions. Updated: March 2024.
768,415 professionals have used our research since 2012.
For how long have I used the solution?
My teams use it. I have about eight or nine teams and most of them have been using it at least for a year. Since I joined the company a year ago, I have been watching it being used.
What do I think about the stability of the solution?
The stability is good. I haven't seen any issues with it. We haven't experienced any latency or downtime issues.
What do I think about the scalability of the solution?
It has been pretty scalable. We are looking at increasing some of its concurrency capacity.
We are currently using about 40 to 50 current VMS.
One part-time resource is primarily needed to design the user groups where jobs will be run. Initially, we had all of our jobs run under a single user name, but then we decided to segregate that and divide it up amongst the teams. So, one part-time resource is needed just to analyze and manage how your jobs get run, then analyze the trends after that.
How are customer service and support?
The technical support has been pretty good. We haven't seen a lot of technical issues in which to engage their technical resources. Occasionally, we used them when we couldn't see something, didn't have an admin account, or for account creation. All those things got resolved very quickly, maybe in a day or so. I would say the technical support is good and rate it as 10 out of 10.
How would you rate customer service and support?
Which solution did I use previously and why did I switch?
It was implemented before I joined the company.
How was the initial setup?
It was already set up when I joined the company.
What was our ROI?
From an infrastructure support perspective, the number of VMs, browsers installations and versions that we would be maintaining without Sauce Labs would be a lot. This includes not only the infrastructure costs, but also the maintenance costs and people's time. The labor cost associated with maintaining all of that would be considerably high. In terms of efficiency, having concurrent VMs with various browser combinations available has allowed us to run multiple executions by all our teams.
Typically, in a night, eight or nine teams have 40 to 50 jobs running simultaneously, which is amazing. Otherwise, if that was not concurrent, then it would have to be maintained in our own infrastructure. That would be a lot of money.
What's my experience with pricing, setup cost, and licensing?
Sauce Labs has room for improvement with its price point. Using a real mobile device, and having that dedicated to your team, costs more than actually purchasing a mobile device. We haven't tried the real devices yet. This is because of their price point.
The number of concurrent VMs that Sauce Labs provides depends on your purchase license level.
Latency has not been a concern due to Sauce Labs being a cloud-based solution. This comes back to the number of VMs and licenses that you have purchased. For example, if I have a capacity of 70 VMs, but I am running 100 jobs, then 30 of them will be throttled and we will see the latency. However, if I were to up my licensing, then I wouldn't see that latency. That is why it just depends on the license tier that you have ordered.
Which other solutions did I evaluate?
I have not compared it with other solutions.
What other advice do I have?
Plan for how you want to use it and how many teams will be using it as well as the types of accounts that it makes sense to have, different access levels, and who should have it on their team. If you plan ahead, then you don't have to fix it afterwards.
We haven't tried the front-end performance testing.
I would rate it as eight out of 10.
Which deployment model are you using for this solution?
Public Cloud
Disclosure: PeerSpot contacted the reviewer to collect the review and to validate authenticity. The reviewer was referred by the vendor, but the review is not subject to editing or approval by the vendor.