Sauce Labs Reviews

We have two kinds of applications using Sauce Labs in our company. One is the website, and we're using it to test across browsers, such as Chrome, Safari, Firefox, Edge, and mobile, to see that all the components we have in a web page work. 

The second product we use the solution for is our driver application. We make and deliver pizza. Every time an order is received, we dispatch a driver to bring it to the customer. We have an application for driver dispatch on iOS and Android, and we run test cases for those as well.

I cannot imagine our company without Sauce Labs. We are using it a lot. It gives us a provider in the cloud for running test cases, scenarios, and validation, and that is really important nowadays. You could always run testing locally or have your own environment, but it's difficult to scale and difficult to maintain. You need to dedicate a lot of resources to do that.

We ran, for example, 500 test cases in the last week for one specific team, and those test cases were running at 2:00 AM or 3:00 AM, for some specific scenarios. We could not do that without Sauce Labs. We are integrated for continuous integration using Jenkins and we're triggering all these tests nightly for different teams on different schedules. It would definitely not be possible to run that manually or in a different way. The cloud is the right environment to run it in, and for that, Sauce Labs is the best option in the market.

Also, the Failure Analysis helps us prioritize remediation. If we have a very high number of test cases failing for a given feature, we can analyze them, see the root cause, see the code in our repository, and create a Jira ticket to address those failures. That feature is very helpful for us. It's pretty straightforward and doesn't consume too much time. I generally spend about 15 minutes analyzing all the failures across the teams to see which three test cases are failing most.

Another benefit is that Sauce Labs saves us on manpower. We have manual testers, but it's not possible for them to cover all the different scenarios and all the different features before release. That means we need to rely on the automated test cases, and to do that, we need to run those test cases on a cloud platform. Sauce Labs saves us a lot of time doing those validations. It probably saves us from having to hire hundreds of people. Every team within our organization has one or two manual testers, but for every team we have hundreds of automated test cases that we run before release. It's good enough, for our operations, to have one or two manual testers. We are seeing good results with that, but to run hundreds of different scenarios I don't know how many people we would have to hire.

From my perspective, as an automation architect, the most helpful feature is the test history. I can easily go to the dashboard and see how many test cases we ran and how many failures and errors there were. I can segregate things by team and by specification. And I can tell a teammate, "You need to fix this test case, it is failing too much. The percentage of failure is too high." 

The most important historical data is for the last seven days. I don't go too much beyond that period of time because my feedback is about how we did in the previous week. In our environment, everything is changing all the time. We are testing different products and running different test cases. So for me, it's the recent data that is key.

The Failure Analysis feature is really important for us, one of the most important aspects. What is the root cause? Is it because we have a defect or is it that we have a test case that we need to fix or modify? The Failure Analysis is one of the main functionalities that I am exploring all the time in Sauce Labs. I can see a test case that is failing and on which particular platform or device it is failing. And the most important part is identifying the problem with the code. We can always go to the line of code and see the possible solution. The Failure Analysis helps us to discover which test cases we need to work on.

Before the test history became the most important feature, the most important part was the configuration that made it easy to connect test cases to the cloud. As a result, we can easily test different devices at once. 

And from a manual test perspective, testing in different browsers using the live test solution in Sauce Labs is very helpful.

Every time that we run scenarios where we need to discover the geolocation of our customers, by default it shows as Palo Alto, California. That's a problem for us and we need a workaround for those cases. It would be great if Sauce Labs didn't provide any geolocation by default, and we could provide the geolocation that we want. It would be helpful if we could enter a latitude and longitude into Sauce Labs configuration and say, "When you run a virtual Chrome device or an iPhone, make this your default location. Then, provide me that device so I can run my scenarios," because we have stores in different regions across the United States.

That is the only "ask" that I have for a feature, a geolocation that we can set by configuration. That would be awesome.

I have been using Sauce Labs since I started with this company, back in 2018, so that would be about three years and six months.

I am an automation architect and my work includes providing cloud testing to different teams. Sauce Labs is one important piece for us.

It's pretty stable. We don't have any problems with the solution. We probably need more virtual machines internally, but that's not related to the performance or availability of Sauce Labs. When it comes to stability, everything is going great.

The scalability is connected to our budget. The virtual machine concurrencies that we have are not great for our demand. I am talking with management about increasing the budget for 10 or 20 more virtual machines. But Sauce Labs itself provides the opportunity to scale very easily. For us, it's just a matter of budget.

Sauce Labs has a pretty reasonable price and is worth it for the solution and what we are doing with it.

There are a lot of other options on the market and I have tried some of them in the past. But Sauce Labs has a lot of advantages. For example, one feature that I really love is that when we are doing troubleshooting or have a problem with test cases, they answer every single question so quickly and are very helpful. The customer experience using Sauce Labs is great in terms of support. That is an aspect other companies don't really have.

I started with our company as an automation engineer and their support was able to help me with every single configuration, every single problem, and every single question. I cannot remember a single time that they were unable to help me. They are the best.

The usage of Sauce Labs was not as great, when I first started using it, as it is now. We have a great approach to testing using Sauce Labs. We test mobile and we do live testing and we are running automated testing. We're integrated with CI. Nowadays, it's better than three years ago and I look for it to improve even more in the next year.

Sauce Labs is a SaaS service hosted on the company's cloud. They have a number of cell phones and mobile devices housed on their cloud, and my team can perform tests on these devices from around the world. We have about 50 users right now, including business analysts, testers, automation, QA, developers, product owners, and third-party vendors. 

Sauce Labs helped us get ready to roll out our mobile apps for the first time. Before implementing Sauce Labs, we tested physical devices that team members had to share. It was more feasible when we were all located in one office, but we couldn't leverage our offshore capacity. With this solution, we can do everything remotely, which is essential now that most of us work from home. 

There's no need for other physical devices anymore because everyone's using Sauce Labs. It's efficient and it works. We still have some older devices, but nobody bothers with them anymore because Sauce Labs works just as well.
While it doesn't necessarily help us get the product out faster, it does make us more confident in the quality. Sauce Labs Visual also helps us catch bugs earlier in our development cycle.

You can access Sauce Labs remotely from different locations. This feature has been especially useful during the pandemic because the members of my home team aren't in one location anymore. The ability to remotely access devices has been essential.

Sauce Labs provides a single visual snapshot for visual and functional regressions, which we can use as proof when there are defects. It also helps us leverage our existing tools, especially the automation component. Sauce Labs has broad coverage, allowing us to test both functional and visual aspects of the UI. This is crucial because we need to see how the application displays on the various types of devices out there.

We've been using Sauce Labs for around a year and a half.

Sauce Labs' stability is good. We haven't experienced any downtime.

Sauce Labs' scalability is good. It's possible for us to scale higher, so I don't foresee any issues with that.

Sauce Labs' support has been great. I would rate them 10 out of 10 because they respond quickly when we have a problem, but we seldom have issues.

Positive

Setting up Sauce Labs was straightforward because most of the work was on the vendor's side. I think it only took a few days. We didn't deploy it into our system. Sauce Labs set it up for us. It requires some maintenance but not a dedicated person. It's one person spending around 10 percent of their time on this. We have an admin who adds users, so it's not really maintenance—more like administration. 

We purchased the solution through a reseller, and everything went fine. 

I don't have any hard data, but we have seen a return. Sauce Labs expands the number of tests we can run with our staff. For example, if we were limited to five physical devices, we could only test with five people. Now, our testing is only limited by the number of staff. From that perspective, it's a return on our investment.

The pricing model of Sauce Labs could be more flexible. Sauce Labs has just one price for the type of solution and a set number of devices. Other solutions have a fee for the base solution and an additional cost per device. 

If you're a smaller organization, you have to consider your needs. Some smaller companies still need to test various devices, so my advice is to start small and scale up as needed. We had initially planned to start big, but that would have been a big waste.

Eventually, I got approved to expand, but it didn't make sense, so I would advise starting smaller. Everything is about costs. Structure it in different phases. If this is something new for you, start with fewer devices to see if the solution is the right for you because it's costly.

You always have to weigh how much the solution costs versus buying physical devices, so I would say go for a phased approach. That way, you can determine whether it pays for itself. We did a phased approach, and we didn't have to go any higher for at least a year. We might go a little higher based on what we're doing this year. We're still evaluating that right now, so we'll see. 

I don't remember the names of the other solutions we evaluated, but we were going through resellers, and those resellers were using other companies. 

I rate Sauce Labs nine out of 10. If you're thinking about implementing the solution, you should talk to Sauce Labs about your needs and challenges. Their salespeople have a technical background, so they have enough knowledge to help you determine if this solution is right for you.

We use it for automation testing of our e-commerce product. We also have some apps that use React Native and they deploy to mobile devices. We also do responsive mobile testing. That means we test anything that hits a website with a browser, or on a phone through React Native, through Sauce Labs.

We also use their VMs and their video recordings.

We use the automation testing and the ability to run it against many device configurations. It's very convenient.

Infrastructure provisioning is a big thing. The whole point of having this expensive license for Sauce Labs is so that we don't have to maintain multiple versions of Chrome, Safari, Firefox, or Edge. We don't have to maintain our own device farms or our own servers, and that means no security patching, compliance, or auditing. A whole bunch of infrastructure headaches are offloaded to Sauce Labs.

Using it every day, and having all the manual QAs get some experience working with it, has saved us multiple person-hours. Just having an automated testing solution, Sauce notwithstanding, means an army doesn't have to sit there and click, click, click, multiple times, every time we do a release, to test the same old things and make sure the same old features still work. Having Sauce Labs on our side, we can actually do all of that at scale with the automation.

The number of testing environments is definitely mission-critical because it plays a part in a release. We run these automations so that we are able to catch issues and so that a customer does not experience issues. And not having to do manual QA frees people up to do exploratory testing. It frees them up to use their intuition and domain knowledge to find bugs that have come in from new features and that might affect old features. It's absolutely essential that we have Sauce Labs. There's no way we could accomplish releases at our current rapid cadence without it.

We also run tests in parallel. It would take way too long to do it one by one. It saves us tens, hundreds, even thousands of hours. And Sauce Labs has reports on that, telling you if you're maximizing your concurrency and whether your licensing is affected by concurrency units, which is great. Knowing that we can run tests in parallel means we can focus on the tests themselves and the quality of the tests. We don't want to create duplicate tests because that would increase test maintenance. Running them in parallel means that we're getting the most for our CPU buck.

We send over a configuration object in JSON and it's very convenient to be able to do it that way.

Also, Sauce Labs is optimized for automation and integration with the major CI/CD platforms and developer tools. We have an integration with App Center that we're working on. They have a storage API that lets us retrieve APK and IPA, iOS and Android builds, install them on the phone, so that we can continue testing. They integrate well with Jenkins.

It's super-important that the solution is optimized for integrating with these major CI/CD platforms and tools because at the manager level, they want integrations out-of-the-box. They want to reduce internal tooling or internal custom stuff.

We use the browser/OS combinations, mobile emulators, and real mobile devices. It's huge having multiple types of testing available in a single platform. It's definitely a competitive differentiator. For example, Microsoft has its own test automation through App Center and there's also BrowserStack and other competitors. It's very important to be able to tell the decision-maker, "Hey, Sauce Labs already has it, so don't worry about it."

They also have a huge number of browser OS combinations, mobile emulators, and real mobile devices. The solution covers a ton of combinations, probably almost any combination you would encounter when a custom reports a bug. That is great for QA to be able to reproduce that issue on that exact same device. 

Sauce Labs maintains physical devices in their data center. They go out and buy the device and provision it for you when you have a real-device contract and licensing, and that's also huge. You're on a physical device.

And for the mobile emulation, which is great as well, they not only have Apple devices, but different iOS versions, which is a huge feature, including different Safari versions on different macOS versions and different Windows versions. More often, you only have a subset of what Sauce Labs offers because people will be mostly using cutting-edge stuff or people might be using mostly legacy. But Sauce Labs runs the gamut and they have all kinds of devices. You'll run out of combinations that are relevant to you before you run every single combination that Sauce Labs has.

I'm pretty happy with the areas of the product that I've been using. The Appium part, even though Appium feels pretty new, is still supported. They support Selenium 4 as well as several other test frameworks, such as Cypress, XCUITest, Puppeteer, and Espresso. Sauce Labs also has artificial intelligence, the AutonomIQ test framework. With AutonomIQ you can have manual QA where you submit an Excel file and then it just automatically creates a test. That's a killer feature.

They offer so many things that we haven't even tried yet, like performance testing and courtesy Docker containers. They are continually updating the documentation. They have performance testing and visual testing. They even acquired Backtrace, which is some sort of error monitoring solution.

I've been using the Sauce Labs solution for about a year and a half. Our company has been using it since 2016.

We haven't had any issues. Sauce Labs has been more reliable than we have.

Scalability is connected to the pricing. The solution is scalable if you have the money to scale. It's based on what they call concurrency units, and they can get expensive.

We have about a dozen users of the solution. They are mostly involved in test automation, SDET.

Support is great, including the support ticketing. Every time I've had a support ticket, they have replied. If they need to, they escalate it. They'll answer technical questions about things like IP whitelisting, and they'll take a look at the screenshots we provide or links to tests that are failing. Their support is empowered to really probe and ask questions.

We haven't used their expertise to help integrate automated testing into our CI/CD pipeline. We have generally solved every issue that we've encountered so far, but they do offer software architecture assistance. It's good to have someone at the software level, and not just sales or product support. If I say I'm having a development issue, it's good to be able to talk at that level, using the jargon.

Positive

When I came in, the solution was already set up. Tweaking it has been easy.

One of the great things about it is that there's no maintenance. We just throw a JSON content object over and then they take over from there.

I can't speak about metrics, but we're able to run automation tests in parallel and that helps with releases. It's definitely a critical part of the whole process. And even moving forward to cloud, it's definitely a big part.

They could improve on the pricing because it seems pretty expensive. I'm sure it's justified, but it's expensive.

For some of the features we aren't using yet, I believe we do need to add new licenses, but for others, we just need to try them out. We just need to have the bandwidth and time.

We use other products for front-end testing but there's no significant reason we couldn't do it with Sauce Labs. It's not lacking in that solution. We use other tools mostly due to dev team mindset. They prefer something more local to their use and something they're familiar with. If we were to push the QA side to do performance testing through Sauce Labs, they'd be open to it. 

It's not only Sauce Labs, as a vendor, that offers automation, but there's BrowserStack and others that also offer it. But using Sauce Labs has been great.

Definitely try it out. They are very friendly about giving you trials and then following up with monthly syncs. They'll connect you with a sales rep, an engineering-type salesperson, and you can have monthly chats with them. They'll keep you updated about their product updates. It's free to try it. Once you try it, I think you'll see the benefits.

Latency due to Sauce Labs being a cloud-based solution hasn't been a concern at all. It runs automatically and sometimes it runs during off-hours, so any latency is not a big deal for us. For flaky tests we use Ruby, which has a rescue retry pattern that we use a lot and that's really helped. Test flakiness is just a reality of test automation and we have good workarounds for it. So cloud latency in Sauce Labs hasn't been an issue.

We've been pretty happy with Sauce Labs. I'd probably have to think pretty hard about what it is lacking. It's been working for us and whatever we throw at it, including Appium, mobile device simulation testing, and being able to support multiple apps. The automation testing has been great. The SC (Sauce Connect) Proxy is pretty friendly. There are the VMs and the video recording. Overall, we've been pretty happy with it. I'd be hard-pressed to find a glaring issue that hasn't been addressed.

We mainly use it to run our test cases for different platforms. We're able to run it for multiple browsers and multiple devices. We use it for about 90 percent of our CI/CD test cases.

We are using it for automated testing and not for visual testing or performance testing.

It's important that the solution gives us access to automated functional testing, as well as browser/OS combinations, mobile emulators and simulators, and real mobile devices, all in a single testing suite. That's particularly so for our client-facing applications so that our customers are able to access our products and offerings through any device and platform. For those types of applications, it is a huge help. The number of browser OS combinations, mobile emulators and simulators, and real mobile devices it offers is really good. There has not been a case where we say we want a given combination and we don't have it.

One of the most valuable features is that we do not have to have the cross-platform testing vehicles in-house. Sauce Labs gives us the ability to test across platforms and that really helps give us confidence in our products.

One of the challenging areas for us is the reporting and the matrix. It should be based on roles, but right now it is only available for the admin role. The admin role can really do a lot of stuff, but our infrastructure team, which holds the admin role, is not ready to hand it over to us on the testing team. If Sauce Labs had permissions associated with roles, where this role could do this activity and that role could do that activity, it would be easier for us.

We have raised this suggestion with our product manager and she has put it in the product backlog, but, of course, she cannot guarantee when that will be looked at.

I joined this company seven months ago, so I've been using Sauce Labs for that long. But the company has had it for about three years.

From a solution stability point of view, it's good.

We are able to run some of our tests in parallel on the solution, within the constraints of how many licenses we have. We have some 50 testing teams across our organization, and we use them. There has never been an issue with not being able to execute tests.

Latency, due to Sauce Labs being a cloud-based solution has not been a concern.

We did have downtime in late September last year, but it was because of our lack of knowledge in terms of the configuration. Sauce Labs support was really quick in responding and triaging and fixing it, which was really great.

With scalability, they are going in the right direction. I have attended some of their new product webinars on API testing, and I really found that to be cool. We may use it, or evaluate it at least, once we start going into that direction. That goes for visual testing as well. If they are able to deliver on some of their promises I really look forward to that and seeing how we can utilize them.

We could use Sauce Labs even more. While we are using it in our testing teams we are not yet there for things like API testing and visual regression.

Our next step, in terms of our exploration of the solution and how we want to use it as part of the CI/CD, is that we are moving into GitHub Actions. We were using Jenkins and it worked well with that. We are moving to GitHub Actions and trying to figure out how that will work.

Overall, technical support has been really good. They have responded to our questions and have triaged pretty quickly and followed up.

The reports that our account manager at Sauce Labs provides are helpful to us to see where our gaps are. She also helps us evaluate where we could be from an industry-standard perspective. She gives us an idea of what others are doing, what she's seeing out there, and in which areas we can improve.

In terms of using Sauce Labs’ technical expertise to help integrate automated testing into the CI/CD pipeline and DevOps toolchain, we have not done that yet. That is more due to the way our organization is structured. We, as the testing team, do not have access to how the tool is managed or how it was set up. There are probably gaps in terms of our understanding of how the tool can be used effectively. But we have discussed this with Mandy, our account manager, and she has offered to do a free session for our associates, which is scheduled for next Friday, and we are looking forward to that. It should help us understand what the best practices are and whether we are using it right.

From the perspective of our organization's roadmap, we are new to testing automation in general. Our focus is on automated functional testing. There are certain use cases for visual testing, and there have been talks about performance testing, and an evaluation will be going on for that. As part of the workshop next week, Sauce Labs will also be presenting some of their offerings, so we'll be looking at those as well.

Positive

Sauce Labs is a tool. It can fit the needs that you have, but you first have to figure out your needs. Once your team has been able to successfully use it, and things have stabilized, go to the next need you have. That's exactly what we are doing. Once we get our functional testing to be stable, we'll look into the next performance test case.

Don't try to do everything together. When you're going with a new solution, it takes time for people to adopt it.

The process could be different for a large company like ours, versus a smaller company where there are just a couple of products. They may be able to move faster. For a financial institution like us, there are so many roadblocks that we have to go through. I do understand that the licenses are expensive, and you do not want to get these licenses and have them sitting idle.

I would rate Sauce Labs a 10 out of 10 because I have not been able to find any use case that Sauce Labs does not provide.

We use the Sauce Labs test automation VMs to help our quality assurance engineers run automated tests concurrently, using a platform that we build in-house.

We use the browser VMs and we definitely use it for end-to-end testing. We may use it for performance testing, as well.

The way that we use it is pretty straightforward.

This product is a big time-saver, so by extension, it's a cost-savings.

It's helped us remove the need for extensive manual testing, which is how it's freed up a lot of time for quality assurance engineers to work on other things. They can work on the automation itself, for example, whereas otherwise, they'd be spending all of that time doing these tests manually and synchronously.

Sauce Labs is optimized for automation and intention with major CI/CD platforms and developer tools. In our case, we've plugged it into our pipelines quite easily. It's an integral tool in that regard, so it is very important to us that it is optimized for integration.

We run Sauce Labs with almost every code commit.

Although Sauce Labs is a cloud-based solution, latency has not been a concern for us. We haven't run into any situations where latency is causing any major issues.

The most valuable feature is the ability to run concurrent automated tests up to a specified value, depending on what we are currently paying for.

I am pretty satisfied with the options in terms of the number of browser and OS combinations they support, as well as the number of real devices, mobile emulators, and simulators. I haven't had any needs that they couldn't satisfy. This enables us to ensure the quality of our products, cross-platform.

A really nice feature would be a pay-as-you-use model, as opposed to a strict number of VMs that you pay for continuously.

I have been working with Sauce Labs for two years and nine months. It has been used by the company since before that.

This product is not super scalable, because you have a very specific number of VMs that you can use. If there was a pay-as-you-use-it model, I think that would really increase the scalability of the solution, but right now Sauce Labs is a bottleneck because you have a specified number of VMs.

We use, on average, 75 of their VMs per hour. Optimally, we're using as much of the VMs as we pay for. As such, the goal is to have our tests running often enough to maximize that usage.

I can't speak to how widespread this product is used in the company, although in my team, there are 10 of us using it. My team is serving between 30 and 40 quality assurance engineers. Other teams in the company are also using it.

I have not personally used Sauce Labs' technical expertise to assist us with integration or for implementing solutions, although, in the past few years, I've submitted a couple of trouble tickets based on some technical difficulties.

I found the technical support to be great and timely. I haven't had any complaints, and the issues have been resolved.

I don't have any experience with other similar technologies.

I was not with the company when it was first implemented.

We pay for a specific number of VMs. The licensing model should be changed to pay-as-you-use, which would make it more effective.

The biggest lesson that I have learned from using Sauce Labs is that concurrency and automation save a lot more time than you initially think.

I would rate this solution a nine out of ten.

Our company has different divisions, like research and development, manufacturing, and research laboratories. There's also a department called Global Infrastructure Services. In total, we have five or six areas. In some cases, Sauce Labs is only used in a segment of a department. Take, for example, research and development. They might not use it in the laboratories, but they have a digital health section that works on apps or digital solutions for medical diagnostics.

In this example, the primary use case is developing different software applications for medical diagnostics. They use Sauce Labs for testing applications in different operating systems and environments. We do regression testing across various ways someone might use an application. Instead of securing physical machines, they use Sauce Labs to do that. Many of the applications are internal to the company, so we use Sauce Connect to link up to the internal network to test those applications.

We do a combination of parallel and sequential testing in Sauce Labs. Whenever you log in, you see all the tests on the dashboard. You have concurrent and parallel tests running, but some people will also use a sequential one.

Before we started using Sauce Labs, they needed to use physical machines to test applications. We required a Mac or Windows laptop to try different combinations, so it used to be a cumbersome process. Sauce Labs made testing more straightforward, so we could run more elaborate tests. 

Sauce Labs provides us with more combinations to test, so we can keep adding platforms and devices to our network. That's been a very seamless experience. If there's an iOS or a private device we need, Sauce Labs has helped get all those set up when needed.

As platform administrators, we get requests from different departments. Project teams using Sauce Labs come to us and say, "We have these three or four devices, but we might need another one that is private to us with this combination of operating systems and browsers." We get those spun up in Sauce Labs and have customer support. 

The ability to run tests in parallel has cut down our test execution time for every application we use. The more we can reduce testing time, the easier it is to deploy an application, so running parallel tests is obviously an excellent way to do that.

It offers a decent range of browser-OS combination simulators and mobile device emulators. Our company tries out lots of different applications and solutions. It's not just for product development but for any application we use internally. For example, we use JIRA for project management, but we tried many solutions before deciding on that particular one. 

We've been using Sauce Labs for a long time, even though there were other candidates we could have used. It's a robust platform that has met our needs, especially with Sauce Connect. We have internal applications that aren't available externally, and Sauce Connect makes that possible.

Nothing comes to mind right now, but this might be because I'm an administrator and not an end-user who is familiar with the application's use case. If we have teams asking for something that isn't already there, we communicate that back to Sauce Labs. That hasn't happened so far in the time that I've been in this role. 

I may not know what should be improved on the platform, but I think it could offer a greater variety of testing solutions. I know there might be competing solutions that cover more aspects that I'm unaware of, and it could expand a bit more. We've done a lot of automated testing scenarios, and that's all the rage these days, so I think Sauce Labs could look at potential ways to improve and cover other scenarios.

I work for the IT team of a pharmaceutical company, but different departments use Sauce Labs. I belong to the team that manages all applications the company uses, so we provide Sauce Labs for other users but don't use the product ourselves. I support it from the administrative side, but I've only been in this role for a few months. 

I haven't seen any downtime in the few months that I've been in this role, and we haven't heard any complaints about latency, but we've always used a cloud-based application. We wouldn't know the difference. We could compare if we had an on-premise version, but we've always used the cloud version, and it's the fast version.

We have purchased a consistent amount of licenses over the years, and we are looking to grow with a focus on automated testing this year.

Sauce Labs' support has been awesome. We have monthly support calls to catch up on any requests that we might have. When there is a new update or release, somebody from Sauce Labs comes to talk to us. It's not just us, but all users. It's like an open house. We have been planning that for some time now, but COVID has put a damper on those plans. When we meet monthly, we share any findings on our site, and they come up with best practices to share with us as well. We are satisfied with Sauce Labs' customer service.

A supply management team deals with pricing and licensing. We look at the utilization of licenses every month and see if we need to purchase new ones then shoot them an email saying we need more. We don't get into the inner workings of licensing and procurement.

I rate Sauce Labs eight out of 10. There are more and more different testing scenarios for various machines and environments we use every day. Sauce Labs provides a robust platform to test combinations of what a user might encounter when accessing any application.

Sauce Labs reduces the testing time and covers all the possible scenarios that could come up, so we don't have to worry about any scenario that we might not have missed testing manually. It's a great platform to cover all your bases.

We create banking software and we use this product for testing. We have different business units and we set up an enterprise license, so everybody feeds into it from each of our business units.

We have about 750 websites and approximately 50 mobile apps, and we will test the different types of browsers against our automation. A good section of the work we do is running automation against different combinations, and that will expand into mobile devices once we kick off the new year.

For the most part, it's heavy automation, but there is also testing that is manual, where they can log in and pick their devices or browsers.

Our environment includes VMs on the cloud, as well as public and private devices. we have the CrossBrowser and we have the private and public cloud.

Sauce Labs allows us to expand our coverage without having to run them separately against different browsers on your local machine. You can just spin up an instance and run against whatever browser and operating systems combinations you need.

Especially on the mobile side, things are improving. We had an on-premises lab, where we were managing the physical devices on a device cart. That's going away, as we are now able to use the public cloud offering, which has 2,000 plus devices.

For locking down devices that we prefer to use without registering with other clients, we use the private Cloud. We split up the distribution 50/50 between private and public.

Sauce Labs is what enables our different testing combinations. We run our different suites that include smoke tests, regression tests, and functionality testing. All of these types of testing need to be verified against different browsers and mobiles, and this is our solution to that problem.

This product helps us to increase test coverage by testing both functional and visual aspects of the UI. This is very important to us because we have different types of customers. Some are end consumers, whereas others are banking customers. We won't release software without validating all of our use cases.

Sauce Labs helps us to deliver new features and products to the market. It's part of the use cases that we have to validate, so we need tools to complement our test cases. We're trying to eliminate manual testing or how we kick off scripts. We want to be able to automate testing of our suite of use cases against our devices and operating systems.

The most valuable feature for us is the browser. That is all we're using right now, although come to the start of the year, we will be using the mobile feature.

The most critical thing is that this software aligns with our Agile and DevOps way of doing things. It integrates with kickoff scripts through DevOps. Time to market is key, which means that we can't have a lot of manual interventions, and this fits into our automation program.

We had some specific features that we opened tickets on, although they were not earth-shattering. For example, the way the menus scroll could be improved because it does not have a bar, the way that people are used to, where you can move up and down. It is something that you have to flick in order to activate.

We have been using Sauce Labs for between three and four years.

This solution is very stable. It's generally up 24/7.

We have about 12 business units that use the service.

Scalability is good on the public Cloud. You've got all of the different devices and most of the operating systems, including beta. It's all that we can ask for in that respect.

The browsers are pretty straightforward. They usually deploy any new versions within 48 hours of the operating systems for the browser side.

We have a thousand registered users and their roles are a combination of traditional QA testers and developers. To support them, we have a small team that I call admin support. There are only a couple of us on my team and if there are any issues, we centralize that before deciding if we need to open a ticket up to the Sauce Labs support center.

Our small admin team works very well. Two people are sufficient to handle the training, education, and any configuration changes. In our company, one of them is the lead over all of the different tools and we license from different vendors. This person is a little bit more technical when it comes to the Sauce Labs configuration setup. The second is an admin for the end-users, teaching them how to use the tool appropriately. This person is also responsible for guiding end-users on how to test applications properly.

The sales group was easy to work with. We had a service account representative, which is a nice setup. They handle the day-to-day activities and support needs, which helps to make us successful by answering questions and giving us solutions or enhancements.

We meet with our service rep on a weekly basis and if we have any issues, we can discuss those solutions and opportunities. Basically, we centralize the communication flow between a small admin team under my team for the organization, and it seems to work very well.

Overall, I would rate the customer service an eight and a half out of ten.

We merged as a company two and a half years ago, and we had licensing agreements with BrowserStack, Kobiton, Sauce Labs, and SeeTest, which is part of Digital.ai. Some of the solutions were on-premises for the devices and we wanted to move to the cloud to simplify the infrastructure.

We moved forward with Sauce Labs because they offered both public and private cloud options. The other ones did not necessarily have that combination. It's a win, having private and public both, as it is important to companies like us.

The initial setup was very straightforward.

We get a service rep that guides us on how to set up the tunnels to access our intra-sites, and then the device procurement is pretty straightforward. You give a list of what devices you want and they are put into each of the data centers, San Jose or Germany.

They do offer training as part of the agreement, as well, and we're in the midst of doing that because we're rolling out the mobile component in January.

Everything that I've needed, they've been there.

In terms of how much effort and how long it took to deploy, because it's on the cloud, the majority of the effort is on the Sauce Labs side. We've had some basic internal procedures to follow, but that doesn't really align with Sauce Labs. This included things like security approvals and stuff like that but overall, it was very little effort on our side.

Our implementation strategy was pretty straightforward. They make it available to us and then we do the training for the end-users. We have a registration list of about a thousand people, to which we have provided demos and documentation. Sauce Labs has a very robust document site. It's a combination of self-service education for the testers and developers in training. So, that's essentially our rollout plan.

We had to rightsize how many licenses we needed, based on user consumption, which was part of the planning phase. We did a full pilot where Sauce Labs gave us an area to access the mobile environment, and we ran our use cases before selecting the software.

We deployed in-house and we are happy with the support provided by the vendor.

The licensing agreement that we have has reduced our cost through solution consolidation by 30% to 40%. I don't have a precise measure of how much it saves the end-user because I don't have a good metric for it. That said, it expands our coverage and saves us from having to kick off the scripts against each of the different combinations. I would estimate that we save maybe 20% in this regard.

In total, considering the bundled discount, we're going to save several hundred thousand dollars per year in licensing fees. The vendor has a tiered pricing structure and by going to one vendor and having enough licenses, we were able to get a more competitive price.

With respect to pricing, they did a bundled discount because we went with Sauce Labs for both mobile and browser. They were very competitive on pricing and provided a bundle discount for us as a larger customer.

I like the licensing model because it is a system of shared licenses. This is different from BrowserStack, for example, which I didn't like because they charge on a per-user basis. This matters to us because we have a situation where there are heavy users and light users. In this case, we prefer to have shared licensing.

Shared licensing is like having a seat at the table and when we have a thousand registered users, it is easy to understand that some are heavy users and some are very light users. With the shared licensing, we don't have to manage the registration of whether they're heavy users or not. We don't worry about that. Rather, we just worry about the consumption of the licenses, which are shared amongst all.

There are no costs in addition to the standard licensing fees.

We recently went through consolidation to get rid of BrowserStack, Kobiton, and SeeTest. We chose Sauce Labs over the other vendors and consolidated to one.

With Sauce Labs, we did a full pilot before selecting it. The selling point and the reason that we ultimately chose this product is because we had our own service account representative that helped us to be successful.

The features met our needs, which was the other evaluation point against the other software providers. They had both browser and mobile solutions, and it's easier to work with one vendor versus four. 

We have usage reports, so we're taking the logs and evaluating that over time. Because we're consolidating from four tools to one, we considered the usage reports of all four of the vendors and came up with what our go-forward is. We expect our usage of the product to increase a little bit over time; maybe a 10% increment as we move forward and get more teams engaged.

Although a 10% increase is in order, we won't purchase licenses without having the data to support that. We will have detailed usage reports for each business unit, which is what will help to determine how much we expand our usage of Sauce Labs. 

My advice for anybody who is considering this product is that they need to understand their solutions and match it up against what is provided. Sauce Labs is leading in the area of both mobile capabilities and CrossBrowser and I would highly recommend looking into Sauce Labs. My decision was to choose Sauce Labs over those other leading contenders.

The biggest lesson that I learned from implementing this solution had to do with what our options were. We had to do a comparative analysis that included breaking down all of the different features and comparing them against those other tools, to make sure they satisfied the features and functionalities we needed. Not having that knowledge across those vendors completely, we were able to do that through a pilot. Essentially, we came away with knowing the nitty-gritty by running the pilot and doing the feature analysis. As such, we were able to make sure that Sauce Labs met our needs.

Overall, we are happy with this solution. We validated most of the functionality when we ran our pilot. Nothing critical stands out in terms of improvements that it needs.

I would rate this solution a nine out of ten.

At our company, Applause, we offer software testing as a service and we always get a lot of interesting, uncommon or challenging use cases from our clients. We sometimes get ones that require specific devices or browsers to work. For example, we have clients who want to mix testing on desktop browsers and apps or test on multiple apps to achieve some kind of scenario; perhaps you are at a restaurant, and you are ordering something on your personal phone/tablet, which shows up on the restaurant's tablet or desktop browser. 

Our clients are not only looking for executing the test cases manually, but their target is to automate all of them and be able to integrate that into their CI/CD pipeline and get faster feedback about the stability of the changes that the development team produces on a daily basis.

Sauce Labs covers all of our automation needs and also allow us to do manual testing in case we are verifying bugs or testing something else.

My main focus is on automation testing and Sauce Labs is an integral part in our success as a company as they offer 1000+ different combinations of desktop browsers, real devices & OS versions. This allows us to pretty much cover all the automation test scenarios we need. We've also integrated them in our internal SDKs and are using their platform on a daily-basis to test the quality of our clients' apps and websites.

The number of the real devices they offer is also very important to our business, because most clients want to support multiple devices & OS versions. We have clients who want us to run automation tests on Android 7, 8, 9, 10, 11 and sometimes even on Beta versions. Sauce Labs offers all of that, enabling us to support all of these versions, which is really nice. The same goes for their desktop browsers, although most of our clients usually use either the latest version or the one before that. I haven't seen a client who wants to use and old browser version like Firefox 50, but it's a great feature that Sauce Labs offers, in case it is needed for any reason.

In terms of delivering software faster by getting feedback with every commit, it depends on the client and the system under test (SUT); some of our clients are testing manually and that would take them days, others have a CI/CD pipeline and run some smoke tests after each commit. Part of our job at Applause is to enhance the overall testing process of our clients & automate it as much as possible. Instead of taking days to do a full regression, it might take a couple of hours to get the same results. 

Because we're working with Sauce Labs, and we're doing automation for our clients, they get a pretty fast cycle of feedback that allows them to make the right decisions at the right time.

Without Sauce Labs, it would be really hard to achieve the software quality standards or the automation coverage that our clients need.

Towards the beginning of this year, we had to constantly switch between Sauce Labs and their legacy solution for real device testing, TestObject. This was causing a lot of issues or confusion for our engineers & clients. Now, with the new unified platform, we get everything we need in one single place and it is so intuitive and straightforward. You can easily do manual testing, view running automation scripts or switch between data centers from one single place.

Overall, we're very happy with the combinations of browsers and devices they offer, especially because they always have multiple instances of the same device. If you want an iPhone 12 for your tests, you can get five of them for example and run your test scripts in parallel, which reduces the total execution time.

Sauce Labs also allows specifying the browser version as "latest", which means that they automatically pick the latest version they support and run the tests against it. Previously, we had to manually change the browser version in our scripts whenever a new one becomes available and it was wasting a lot of time (as it needed to be done for a lot of projects).

Their platform also offers some cool features if you're working with apps. You can just pass them a URL to an app build that is stored somewhere on your servers or on AWS, and they will install that on the phone/tablet and run tests against it on the spot. This has allowed for an easier integration with our SDK, as opposed to competitors who don't offer such feature.

The market is changing and we're seeing a lot of shift towards testing on OTT devices, like Apple TV, Fire TV Sticks, Chromecast and Roku. This is really an emerging market and has the potential to grow exponentially. We've been getting a lot of clients asking us to test on OTT devices, either manually or via automation. We have been able to automate testing on OTT devices, but it is done with local devices hosted on our premises, so we can't scale that fast or cover all of the huge demand we are seeing. We're looking for the ability to do this in the cloud, so if Sauce Labs offered such a feature, that would be really great. Working on OTT devices usually includes testing on desktop browsers or phones/tablets and having all of those supported in one platform would be ideal.

While Sauce Labs pretty much covers everything we need, another exception is HarmonyOS. I know it's a new thing and it's not supported by a lot of providers, but it is something that clients have started asking for. That is something we'd be really interested in seeing, for both manual and automated testing.

Also, latency, due to Sauce Labs being a cloud-based solution, has been a concern. We work in different continents and countries, but last time I checked, Sauce Labs was only offering two data centers, one in the EU and another in the US. If you're not in either of those two places, you would have latency and issues running your test scripts. However, I'm working with clients in different countries and we've seen some latency issues, depending on the country, although it's not huge. In comparison to other providers who have data centers in different places, there is some noticeable latency.

I've been using Sauce Labs for over five years.

Sauce Labs is pretty stable in comparison to other providers. We still get some issues every now and then or random failures, especially when there is a new OS or browser version, but it's not a big deal and we can easily contact Sauce Labs' support to get things fixed.

One of Sauce Labs' competitors does not force you to select the data center you want to run against. They just figure out which data center is closest to where you are executing your test scripts from. They offer a single, unified endpoint or URLs when it comes to automation and handle everything else internally. When we're writing automation scripts, we don't need to specify, "I want to run on the European data center," or "I want to run on the American data center." We can just say, "I want to run a test case," and depending on the location you come from, the platform is smart enough to direct it to the nearest data center to reduce latency.

This may not be directly related to scalability, but this kind of capability would make it easier for us to build our SDKs faster and focus on other features, which in return would allow us to scale faster as a company. The learning curve for newcomers would also be easier, because they wouldn't need to worry about figuring out which data center to run against.

While I love the fact that we can specify the data center ourselves, because it gives us more freedom, I would love to see more data centers in different places around the world, to reduce latency, and the selection of the data center implicitly done by the platform, so we don't need to worry about it.

We open support tickets, and they're pretty responsive, they get back to us on time. They're really good about fixing things and making their platform more stable.

I've run into a couple of situations where tickets have been left for weeks without feedback, but that has not happened often. When it does happen, I can go to our customer success manager and say, "We need some attention to these tickets." She can take care of it and that makes the support pretty good.

Positive

Pay attention to selecting the correct data center, otherwise you would run into some latency-related issues.

Although we are not using visual regression testing, that is something that would be really interesting, because a lot of our media clients want to test visual validations on their apps. If Sauce Labs is offering that out-of-the-box, it would be really interesting for us.