Sauce Labs Reviews

The Primary use caes would be the following :

• Using the device farm and also using the ability to use the virtual and real devices for the purposes of software Quality Analysis testing.
• There are projects that sometimes require a variety of devices to perform software testing on and having the repository of devices to choose from that my company might not have greatly assisted with this use.
• Especially in handling massive loads of users. Since there is virtual emulation as well as device farms with actual devices this allows a great number of devices for a team to test with.

Sauce Labs has been a saving grace for our company. More so in recent years if I'm going by my own personal experience.

When you consider the positive plus points relative to having multiple devices, especially with the recent pandemic being under lockdown as well as just a general work from home situation before. Having Sauce Labs farm has been greatly beneficial to many people within the organization and has helped keep the project turn around time on track.

Supplying devices to a testing team of possibly close to one thousand testers and developers is a great undertaking but Sauce Labs has made this very easy and a welcomed solution.

The most valuable feature is as I mentioned would be the benefit of having mobile devices on hand even if it's virtually so in abundance for our testers In my company. 

Another positive is having an up to date device farm and virtual emulation of devices. The time it takes to allocate and purchase devices on the scale it would normally need is completely deducted from the issues we would have to face from a day perspective or even project-wise. On top of that, having to maintain these devices would be another undertaking and require resources that we as a company do not have to concern ourselves with anymore.

If I had to speak of an area that could be improved it would probably have to be the speed of interaction with the devices. There is at times a considerable amount of lag while using some of the virtual and at times even physical device farms. While this is absolutely better than not having it situation Sauce Labs certainly would benefit as well as we the user would certainly benefit from gaining a software interaction speed boost. Having it more consistent would be greatly appreciated if the long run. This is the only thing affecting the testing time at the moment and can even be seen as a disadvantage in comparison to having an actual physical device in hand.

i have been using saucelabs for about 6 years or so within the company I work for

Stability-wise it's just okay. The speed of this software needs to improve a lot of possible.

Very good as it has the ability to appeal to a lot of users despite some lagful moments.

Customer service is good. I have personally not had an issue myself but from those that have it has never been anything but positive .

I have not personally used another solution prior to Sauce Labs.

Pretty straightforward for the most part.

We deployed with an in-house team.

In terms of projects, it's per-project basis. So it varies but it certainly has saved time.

I'm unaware if my company has done so but I imagine they did and still chose Sauce Labs as the best option.

It's certainly worth a try if anything . I can confirm it has definitely been a help to myself and the company I work for and it probably will do the same for others. 

As we began architecting a CI/CD pipeline in our SDLC, a need quickly arose for left-shifted, continuous testing of our web applications in real browsers. Specifically, a service that could integrate seamlessly with our Jenkins CI servers and executing tests continuously against our Dockerized applications and firewalled environments. Furthermore, our company does not have the resources to implement and maintain a proprietary Selenium Grid for test distribution. Sauce Labs addresses all of these concerns, and furthermore offers a platform for manually debugging applications in any combination of platforms, browsers, and devices needed, either from an automated test script or from a manual test session. The largest selling point for our needs is its relatively seamless integration with our Jenkins CI servers, including detailed playback and test reports per build job, as well as easy configuration of the proxy tunnel used to access our firewalled environments and desired browsers-under-test, right from the Jenkins UI via their Sauce OnDemand Jenkins Plugin.

Prior to moving to a continuous delivery model, we were on a classic two week regression testing cycle, mostly conducted via manual testing and manually-executed automated tests scripts in a large integrated testing environment. Upon shifting our SDLC to be more continuous, and rearchitecting our delivery pipeline, we were able to very quickly shift that long testing cycle to be integrated into the delivery pipeline itself by left-shifting these deep functional/regression tests to be executed during build time on our CI servers, and the implementation of these automated tests scripts being integrated into the development effort itself. Sauce Labs allowed all of this to happen by supplying the service needed to execute automated browser tests in from a CI server without having to implement or maintain our own proprietary grid of virtual machines.

Of all the services explored, Sauce Labs offers the single best solution for integrating deep automated browser testing in a CI/CD pipeline, and integrates best with Jenkins. The Jenkins UI plugin offers a way to easily configure browser settings for tests being executed continuously. The Sauce Connect Proxy is a necessary technology for companies who have firewalled testing environments, as well as localhost testing. Test reports (that also integrate seamlessly with Jenkins jobs) are also detailed and incredibly useful. Browser support is quick to be updated as new versions of browsers are released. The custom capabilities that can be provided to Sauce Labs VMs during automated testing sessions are a valuable option for experimental or niche testing.

The Jenkins Sauce OnDemand plugin could have more options available to tap into more of the custom capabilities Sauce Labs actually supports. Currently, in order to tap into these capabilities, a programmatic solution is required from the test code rather than being able to configure them via the Jenkins Plugin UI, e.g., desktop resolution, browser automation binary versions, remote Selenium version, and browser console logs. These are all things it would be nice to be able to customize straight from the Jenkins Plugin UI. 

In our experience, we have also had some issues with the Sauce Connect Proxy on our Jenkins servers failing to start, which makes the optimal CI/CD pipeline come to a halt. 

User account management needs an overhauls, allowing for user groups rather than just a hierarchy structure. Test job queuing is a first-come-first-serve, meaning there is no inbuilt way to queue batches of jobs from different Jenkins servers/accounts.

Occasionally there are issues with the stability of the Sauce Connect Proxy during Jenkins builds, which can cause builds to fail seemingly inexplicably, since determining the root cause is oftentimes difficult.

The only issue is cost. Each new unit of concurrency you add to your enterprise license represents a relatively linear increase in cost.

When you reach the Enterprise licensing tier, base level being with 10 concurrent test sessions, pricing is essentially per-unit-of-concurrency thereafter with a relatively linear increase and not much benefit for "bulk".

BrowserStack was also explored, but did not integrate with out intended CI/CD model as extensively or seamlessly as Sauce Labs was able to.

• Running tests in parallel
• Sauce Connect Proxy tunnel
• JIRA/Bamboo plugins

Enables faster execution of regression tests without having to support/maintain the software needed to validate regressions against different devices, OS, and browsers.

Speeds up the time it takes to run end-to-end user interface (UI) tests inside their virtual machines (VMs).

Less than a year.

None yet.

Not yet.

Not yet.

Excellent, so far.

Excellent, so far.

Yes. We built and managed all of our testing infrastructure in-house.

Pretty straightforward, except when you begin working within an enterprise organization that operates nearly all of its web applications behind a firewall. To get our automation running inside Sauce Labs's infrastructure, we first have to install and configure the Sauce Connect Proxy server. The complexity of setting up the tunnel comes mainly from the protocols our organization has for getting the approval to implement Sauce Labs behind our firewall. Explaining the details of the tunnel, why we need Sauce Labs for regression testing, and facilitating the connection between our Server's team with a Sauce Labs technical resource were all fairly complex.

In-house.

It is difficult to estimate at this time.

Start small, even if you are a large-to-enterprise level organization. The setup cost, if you're an enterprise-sized organization, is negligible. Establish a few quick and easy "wins" for one or two product teams. Hold a post-Sauce review with all the stakeholders within each team with the goal of reaching a consensus on whether the product is a good long-term fit. If the answer is yes, then calculate ROI, and if it makes sense, scale up and spend the money so every product that should use Sauce has access and the correct bandwidth.

SmartBear, Perfecto, and Ranorex.

The on-demand provisioning, the multiple operating system and browser choices (including multiple versions), as well as the really nice screenshots and videos, as well, are key.

Having full logs, with videos and screenshots, has really helped us work more closely within our own team, as well as, more importantly, being able to show developers (automatically!) what the precise problem is.

Since around August of 2011

Integration with Sauce has been a breeze for us here at Mozilla.

At various times, there are high-latency launches, or outright timeouts, but the Sauce Ops team has been pretty good about both 1) acknowledging the issue/investigation 2) working steadily to resolve it.

At times, the brief outages or slow service responses have temporarily hampered our fairly large (in terms of concurrency) test-automation runs, but that's not very often.

Sauce support has been very helpful -- always friendly, pretty prompt, and thorough, too -- even when it's not always a direct problem with their service or our usage of it.

For our needs, the customer service and technical support come through the same channel, and we've been very happy with the level and quality of responsive support we receive.

Being an open-source organization, we wrote our own custom plugin to handle integration of both our in-house Selenium Grid as well as Sauce Labs (https://github.com/mozilla/pytest-mozwebqa), and although I didn't personally do any of that coding work, can attest that the APIs Sauce provides have made it pretty easy for us to do that integration and setup work.

We are lucky to enjoy Sauce Labs's "Open Sauce" program, as an open-source, not-for-profit organization: https://saucelabs.com/opensauce

We did look (back in 2001) at other vendors, but found the business side and open-source/project relationships fit well with us.

As a disclaimer, I have done a number of Selenium Meetups with Sauce Labs, as well as a few blog posts which have covered our usage of the service, as well as, finally, authored a Featured Customer case study, in which I expound on nearly all the points I've made here:

https://saucelabs.com/customers

• They have a video recording feature which I find tremendously helpful. It helps me debug and find defects quicker. When I see something fails, it shows me the video of the failed part, the video playback feature is amazing.
• Easy to setup.
• Very responsive. Get back within the same day – overall great company.

My company was doing manual testing, I came in and built a framework using Sauce Labs; now it takes less than an hour for regression testing. Easy integration, increased the amount of codes that go out. I used to have to write the tests and perform manual tests; Sauce Labs, has changed all that.

Documentation is pretty extensive, but at an advanced level could be a bit better. But they are always helpful. Besides that everything is great, they help out right away, so nothing is a big deal.

About three and a half years.

No issues with deployment.

No issues at all with the product's stability.

Very good, highly scalable product.

They are amazing. Always get back to me, follow up, everything is amazing.

They are amazing. Always get back to me, follow up, everything is amazing.

Previously I used my own virtual environment – had to create and manage it on my own, now I don’t have to worry, infrastructure, setup, anything, Sauce Labs does it all for me.

Straightforward for me.

I implemented in-house.

In terms of value, we increased our release cycle to twice a week as opposed to once every two weeks – so its doubling productivity.

I did. I looked at BrowserStack. Did not have video-playback, which is tremendous for finding bugs and code.

I think its excellent and everyone should get it – I recommend it to everyone. No brainer. Nothing else like it out there right now.

Ability to run many different browsers and tests in parallel. This has allowed us to keep our total test times under 25 minutes (even as we've continually added hundreds more tests and up to 10 browsers) and run tests more often—decreasing the time from bug introduction to detection.

Being able to test mobile devices (iOS, Android, and different browsers on each device) has helped us catch tricky mobile-specific bugs in the past.

We've also found the web-based review features (video playback, step-by-step screenshots, "click to manually take over test", full logs) and API handy for diagnosing failing tests and looking back at the result and timing history.

We also found Sauce Labs to be more flexible than alternatives during the setup phase—if you go just one parallel instance over the limit, for example, I noticed they won't instantly fail your tests like another major provider does, which saved a lot of time and frustration when getting parallel tests set up to start.

We now find most browser-specific breakages before they hit production. This lets us push code out knowing it won't break in specific browsers, letting us be bolder in making larger changes across our codebase, frontend to backend.

We are now getting our Sauce Labs tests integrated with our CircleCI tests using Sauce Connect, which should dramatically decrease the time-to-issue discovery and save us time tracking down bugs once our tests catch them.

Finding ways to debug our flakier tests is an ongoing pain for us (we've fought with our own tests' flakiness across all different providers, often e.g. due to some test steps using implicit rather than explicit waits). We could always use more insights in to the causes of different failures, common advice when a failure type is detected, or more detailed guidance on best practices for test runner setup.

Cross-platform mobile testing. Before using Sauce Labs, I would have to setup a Selenium Grid cluster and install all versions of mobile simulators/emulators in order to test apps on different platforms. With the help of Sauce Labs, all I need to do is to configure the settings in my codes and let the tests run on the cloud. It saves me a lot of time and efforts.

For mobile testing, Sauce Labs platform is great for functional tests. But the option to run tests on real physical devices is missing right now, so it’s impossible to run performance stress tests which is also very important to test mobile apps. This feature is actually supported by some other mobile testing vendors like Appurify. Hopefully, Sauce Labs will add this feature soon.

Good. According to my experiences, the technical support engineers typically reply to emails within a day.

Straightforward. Just integrate the client libraries and setup processes into your existing automation framework. Detailed documents are available on Sauce Labs’ website and its Github report.

The client libraries are quite compatible with other popular testing frameworks. Chances are you can directly integrate it with your existing automation infrastructures, so the original setup cost is very low.