Sauce Labs Reviews

Ability to run many different browsers and tests in parallel. This has allowed us to keep our total test times under 25 minutes (even as we've continually added hundreds more tests and up to 10 browsers) and run tests more often—decreasing the time from bug introduction to detection.

Being able to test mobile devices (iOS, Android, and different browsers on each device) has helped us catch tricky mobile-specific bugs in the past.

We've also found the web-based review features (video playback, step-by-step screenshots, "click to manually take over test", full logs) and API handy for diagnosing failing tests and looking back at the result and timing history.

We also found Sauce Labs to be more flexible than alternatives during the setup phase—if you go just one parallel instance over the limit, for example, I noticed they won't instantly fail your tests like another major provider does, which saved a lot of time and frustration when getting parallel tests set up to start.

We now find most browser-specific breakages before they hit production. This lets us push code out knowing it won't break in specific browsers, letting us be bolder in making larger changes across our codebase, frontend to backend.

We are now getting our Sauce Labs tests integrated with our CircleCI tests using Sauce Connect, which should dramatically decrease the time-to-issue discovery and save us time tracking down bugs once our tests catch them.

Finding ways to debug our flakier tests is an ongoing pain for us (we've fought with our own tests' flakiness across all different providers, often e.g. due to some test steps using implicit rather than explicit waits). We could always use more insights in to the causes of different failures, common advice when a failure type is detected, or more detailed guidance on best practices for test runner setup.

There are a few features that makes this service a top-tier solution. First there is the automatic video recording of the test runs that records the actions taken. For any debugging that you have to do this makes this service invaluable. Often times I would look at the recording first before I started debugging the test locally since I would likely find the issue there faster than starting up my own debugging session. 

Second best feature is the use of real mobile devices as an option for some of the test runs. There is no substitute for running tests on real devices. After setting up and maintaining a local testing grid of 9 mobile devices I can assure you using a service that does it for you is so much better. There were some weeks that were spent on just fixing issues when using my own devices.

The best feature though is the fact that they use the WebDriver protocol for executing actions on their service. A lot of companies just used their own variations on the WebDriver. This then meant that you had to write another set of wrapper functions just to make the current framework work with whatever vendor you were using. Sauce Labs just used the standard remote WebDriver. Which meant that all you needed to change was the hub url to point to their hub and you were done. If you wanted to there is a few lines of code that let you mark the test runs as successful or failed from your side instead of their generic completed message. All told that is 5 minutes of work and it would have better integration with whatever your process is.

The ability to share the tests that failed with the developers improved the overall efficiency of the team since there was no ambiguity on why a test failed. In addition we used some of the recorded test runs with the business to clarify the functionality of some of the more complicated features.

My company wanted test coverage for some devices that were not available on Sauce Labs so I still had to manage a small "test lab" with those devices. Some of the devices were Windows Phone 8 and Blackberry 7 and 10. On our site the only reported instances of those devices accessing our site was the test devices after 6 months so I am still unsure why we wanted to support those.

Actually since their service was so stable it helped identify some flaky tests. Before those tests were executed on bad machines so if they failed they would be executed again and then it would pass. But by running the tests on Sauce helped identified that the test itself was flaky so the tests were refactored into stability.

We did not encounter any issues with scalability. I personally wished that we had a budget to get more parallel executions but the cost was not justifiable.

Their customer service reps are fantastic! They have heard of all of the typical issues that can occur in business and offer solutions to it such as using one of the professional tiers to make sure that their service integrates into the workflow before going to the full blown enterprise plans.

We had used Perfecto Mobile but was unhappy since it was not easy to integrate with our existing framework. Business wanted the devices to be real devices everywhere in the world so went ahead and purchased Perfecto Mobile without trying the service.

The initial setup was so easy that I thought I must be missing something. Since their service provides so many features I was certain that there must be some other options that I have to declare in the code to get those features, but no it was really that straight forward. 

I implemented it in-house since I was tired of managing 25 virtual machines and 12 mobile devices and 5 physical machines on a seemingly daily basis.

Truly agile companies that are constantly testing should get the enterprise solution that they have with dedicated machines. For more ad hoc testing or just testing before the release the pricing on their site would be the best.

I evaluated Perfecto Mobile and BrowserStack before testing out Sauce Labs and they did not compete well enough with Sauce Labs. Perfecto did have more devices available around the world than Sauce Labs but the connections were slow.

The session duration is too short; it's only about 3 hours long.

We haven't had any issues with deployment.

We haven't had any issues with stability.

We haven't had any issues with scalability.

The most valuable feature would have to be the cross-browser testing functionality. Our product specifications say that we support the Big Four browsers – Chrome, Firefox, IE, Safari – latest versions and up to two versions back. When including mobile, that’s 10 different browsers (and browser drivers) to maintain. This would be a near impossible task for our small testing team.

It also gives us the ability to replay our tests back in video form. This makes attempting to debug failed tests not a breeze, but easier. It also helps convincing (stubborn) developers that there is a problem.

We have been able to successfully integrate Sauce Labs into our automated test environment. From that we are able to run our test suite across multiple browsers overnight. This gives us a level of confidence that we previously didn’t have.

The other huge benefit we get from this product is the manual testing powers. When a client tells us that they’re having issues on a particular browser. We’re able to fire up that exact version and reproduce the problem ourselves.

Speed is the key. We’re a business in Australia, so any tests that we perform are having to travel to America and back. This is time that we have to wait on for us to get responses back. In an active business every second counts.

We've been using it for about 12 months.

We've had no issues with deployment.

We've had no issues with instability.

We haven’t had any problems that we couldn’t work around, or there wasn’t any documentation on. The only problem is that hit that point where it’s unfeasible to expand our UI testing framework as the cost to run a shed load of tests is too much for our business.

This has been an area of focus from Sauce Labs. Their customer service is unmatched, and their help documents have all been recently updated.

We have used a local Selenium grid, and still do to write our tests. However, we valued the cross-browser accessibility over speed for our nightly automated test suite.

Initial setup was somewhat difficult originally. That was due to multiple reasons, namely lack of actual programming experience on our behalf, and the documentation at the time was a mountain to climb. Sauce Labs has since updated this documentation, and we have since gained experience.

Our in-house team performed the implementation. For advice, it’s a lot like setting up Selenium Grid. One thing we did was have an Environmental Variable “ISTESTLOCAL”, we use this as a gate to whether or not our tests should run on Sauce Labs or not. This is very helpful in increasing the speed which we can write and debug our own tests.

Also, if using the Bamboo plugin, make sure to check the settings page as there is a list of already in use environmental variables set when Sauce Labs runs.

The biggest things you have to consider is how much testing you’re planning on doing, who is your client base, and what browsers they’re using, and what can’t be tested using API testing.

If you have any issues with the product just shoot the support team an email. They always get back to me promptly with the best solution they have available. Also, spend the time to read the documentation.

If your company is security conscious, as it should be, look into Sauce Connect which creates a secure tunnel between Sauce Labs and yourself.

Multiple operating browsers, screenshots and high video quality.

A cross browser test platform is really important for a web designer, Sauce Lab providing a really fast, functional and with good quality service.

It's a little bit slow sometimes.

No issues with deploying it.

It's been stable.

Scaled for our needs.

The beyond convenience to test websites and applications on any platform necessary.

In the past, I would have to find all the devices I could to live test pages. But now, I can virtually test the devices I need, plus a lot more devices that I wouldn't be able to get my hands on.

Just continue to support new versions as they come out. The speed can be a little slow, but other than that it's great.

At least five years.

We've had no issues with deployment.

It's been stable.

No issues with scalability.

It can be used in a very wide spectrum of environments and has a high speed of access when running tests against these environments. 

The environments are really easy to configure and the UI is very friendly. 

Another great feature is the monitoring of test executions in real-time as well as the ability to perform actions on a testable web page. 

It also has video recording and full logging, which are great and helpful functionalities.

I think that I found maybe only a few gaps which I don't think are very important. These are more like improvements which need to be added. 

It would be very good to see a short example of the code that needs to pass into Driver capabilities, something like auto-generation of this block. 

Also, it's not obvious where to look for the API key. It would be very nice to have a button on the top navigation menu.

No issue with deployment.

No issue with stability.

No issue with scalability.

My general overall impression of Sauce Labs is very positive. I use Sauce Labs and recommend it to every engineer.

Sauce Labs is an amazing product for the work that I do, and there are many different solutions for testing out there. Sauce Labs in particular, I have used on many occasions within different roles and I found it to be a good alternative to options such as Browserstack. 

The most important and valuable feature is the ability to select multiple browsers and environments to test and configure how your site runs and works. This process eliminates the need for any virtual machines taking up disk space on servers, and now Sauce Labs streamlines the testing process by being able to have these virtual instances of OS and Browsers right on-demand. 

I particularly found the tabbed view of Sauce Labs useful, allowing me to have multiple browser sessions open (one for, say, each operating system as well as different browser brands) along with its clean and minimal interface. These are features which aren't found in other testing suites and are definite bonus! 

As a newbie to automation testing and having experience in using a wide range of on-demand tools, my main aspect of Sauce Labs involves using the 'Manual Testing' option, which is brilliant. You simply selected the URL and choose the browser and environment, and within minutes you'll simply have a virtual environment to play with, and it is really that simple.

Another useful aspect of Sauce Labs' product is that you can allow it to take screenshots, videos and logs, which are really, really handy. Having dealt with many issues that end users see whilst a website is being developed on, often is the case that issues are hard to replicate. Knowing Sauce Labs is there to record this (in case I can't replicate it), makes this a tool which you can share the experience with other users. 

The configuration and options within Sauce Labs are endless, which makes it an excellent product. A well-designed and minimal interface has also helped me to stay productive and keep focused on the main tasks which I use it for (manual on-demand testing and using different environments to see expected behavior for our platform).

Over time, I aim to expand into the automation area which looks great from the outset and be able to automate some tedious tests which could reduce the amount of time that I spend testing, and ensure I can continue to capture bugs and issues more effectively.

This hasn't necessarily changed the way in which the organization functions, but has added to my skill set. Currently, Sauce Labs fits very well within the other services (Ghost Inspector, Browserstack) which I currently use and it's a tool that is there for me to be able to have the flexibility to test on demand and use whichever configuration is right for me. For example, our user base will mainly be clients using Internet Explorer 10, 11 and Edge on Windows 10 or IE and, therefore, having these environments at my disposal is really handy. 

As a QA person, it is ultimately my responsibility to ensure that our product is tested on as wide a range of devices as possible and Sauce Labs helps to achieve this through the multiple environments that it offers and being able to multitask is an added benefit. 

For me, it has improved the way I function as I don't require the use of virtual machines and the headache of setting them up, and running tests on local computers meaning adding to the myriad of devices already on my desk! Streamlined, easy access to testing when I need to - that's already the benefit to me of Sauce Labs.

The main interface, although clean, is sometimes very slow to load on slower connections and selecting a machine and waiting for it to boot up the environment takes a lot longer than other products I have used. This sluggishness also increases when the user tries to navigate the environment and interact with it. Selecting Microsoft Edge on Windows 10 gave me a really slow response time and took a while to boot up. All virtual environments are expected to be slow, however the interaction between the user and keyboard presses / mouse clicks wasn't very synchronized, and often the lag that is caused distracts away from the full experience. 

Manual testing also opens up in a separate browser tab, which I'm not sure why, but should ideally load over the main dashboard.