IT Central Station is now PeerSpot: Here's why

Sauce Labs Room for Improvement

Joel Alonzo - PeerSpot reviewer
Product Manager - Data & APIs at a marketing services firm with 201-500 employees

Multi-domain SSO is a big concern for us right now, especially as we've been merged into a larger company. I suddenly have teams coming from 20 different domains, and because the main master Sauce Labs account is locked down to one SSO domain, there are teams that can't run a test right now. 

I've heard they're working on a solution and they've been very communicative with us about it. A solution to that would help us a lot.

The other issue is that when you're behind a corporate proxy and inside a corporate network, it's a nightmare in general. And the problem that we've come up against over and over again is that a lot of our network staff don't want to open up giant CIDR blocks to internal traffic. Opening it up to one endpoint which is Sauce Labs, through a little API is perfectly fine. That's one IP address and it's no big deal. 

But for their virtual machines they have one jump box and that jump box contains an entire CIDR block that changes each time the tunnel starts up. In other words, if every call went into one IP and that got bounced back to a load balancer or net gateway, that would help a lot too. When I go to the network team and say, "Hey, we're using a product, a piece of software, and I need an entire CIDR block opened," the first and almost eternal response to that will be "absolutely not."

Getting things set up and running is always going to be a bit of a struggle within any corporate environment. Most of that is not Sauce Labs' fault, but the things I just mentioned are things they could do to help deal with that struggle.

View full review »
Savio De Souza - PeerSpot reviewer
Sr. IT Architect at a healthcare company with 10,001+ employees

When we were in development, it was a bit of a pain because we have onshore and offshore development. One of our development shops is in India, and we were running tests over there. When some of the users tried to log in, it was slow for them or we didn't have enough licenses. That was during the core development before we even launched. 

We got around that by purchasing more seats, tinkering with some of the virtualization pieces, and scaling. Now we don't have that issue, because we scaled back the offshore team significantly, so when we run it overnight, there's really no effect. We come in the next morning and review the results. It doesn't affect the overall business or the offshore team. 

There were also some bottlenecks because of the amount of time testing takes, so we started using more tunnels and running it in parallel. That was the main issue that we faced initially, but now that it's all set up, we're good to go. We were struggling with the volume of tests, and Sauce Labs suggested we run everything in parallel.

Sauce Labs isn't lacking any features that we want, and it has several we're not using, like mobile and API testing. They've also introduced a ton more features since we launched, so I don't see anything missing on their end. 

View full review »
RL
Director of Quality Assurance - Shared Service at a financial services firm with 10,001+ employees

We had some specific features that we opened tickets on, although they were not earth-shattering. For example, the way the menus scroll could be improved because it does not have a bar, the way that people are used to, where you can move up and down. It is something that you have to flick in order to activate.

View full review »
Buyer's Guide
Sauce Labs
June 2022
Learn what your peers think about Sauce Labs. Get advice and tips from experienced pros sharing their opinions. Updated: June 2022.
610,518 professionals have used our research since 2012.
Hassan Radi - PeerSpot reviewer
Head of Automation R&D at Applause

The market is changing and we're seeing a lot of shift towards testing on OTT devices, like Apple TV, Fire TV Sticks, Chromecast and Roku. This is really an emerging market and has the potential to grow exponentially. We've been getting a lot of clients asking us to test on OTT devices, either manually or via automation. We have been able to automate testing on OTT devices, but it is done with local devices hosted on our premises, so we can't scale that fast or cover all of the huge demand we are seeing. We're looking for the ability to do this in the cloud, so if Sauce Labs offered such a feature, that would be really great. Working on OTT devices usually includes testing on desktop browsers or phones/tablets and having all of those supported in one platform would be ideal.

While Sauce Labs pretty much covers everything we need, another exception is HarmonyOS. I know it's a new thing and it's not supported by a lot of providers, but it is something that clients have started asking for. That is something we'd be really interested in seeing, for both manual and automated testing.

Also, latency, due to Sauce Labs being a cloud-based solution, has been a concern. We work in different continents and countries, but last time I checked, Sauce Labs was only offering two data centers, one in the EU and another in the US. If you're not in either of those two places, you would have latency and issues running your test scripts. However, I'm working with clients in different countries and we've seen some latency issues, depending on the country, although it's not huge. In comparison to other providers who have data centers in different places, there is some noticeable latency.

View full review »
SM
Sr Staff Software Engineer, QA Enablement at a tech vendor with 501-1,000 employees

The process can be a little slow to configure and get started. It's a cloud platform and you're trying to run automated tests. It first has to be configured, then the test downloaded, then you have to configure the environment that you're going to run on, then kick off the test based on what your needs are. All of this takes time.

We have faced challenges with the availability of mobile devices. There was once or twice where there were no mobile devices available. These instances may have been due to a service outage.

View full review »
AD
Senior Manager - Software QA at a hospitality company with 10,001+ employees

Some of the trends that it shows. It only allows you to view the last month. Having it go beyond a month, e.g., yearly trends, would be good. While the yearly trends are available, they are available to Sauce Labs administrators, who are internal to Sauce Labs. I don't know if they are available for customers yet.

View full review »
JoseMorales2 - PeerSpot reviewer
Automation Architect at a hospitality company with 10,001+ employees

Every time that we run scenarios where we need to discover the geolocation of our customers, by default it shows as Palo Alto, California. That's a problem for us and we need a workaround for those cases. It would be great if Sauce Labs didn't provide any geolocation by default, and we could provide the geolocation that we want. It would be helpful if we could enter a latitude and longitude into Sauce Labs configuration and say, "When you run a virtual Chrome device or an iPhone, make this your default location. Then, provide me that device so I can run my scenarios," because we have stores in different regions across the United States.

That is the only "ask" that I have for a feature, a geolocation that we can set by configuration. That would be awesome.

View full review »
Sarah Barefoot - PeerSpot reviewer
Senior Quality Assurance Manager at a financial services firm with 1,001-5,000 employees

I can't remove team members that have left the organization. I can only set them as inactive. It would be really nice to clean up my data and delete them from the team management.

View full review »
Nitu Singh - PeerSpot reviewer
Test Automation Snr. Consultant at a insurance company with 5,001-10,000 employees

One of the challenging areas for us is the reporting and the matrix. It should be based on roles, but right now it is only available for the admin role. The admin role can really do a lot of stuff, but our infrastructure team, which holds the admin role, is not ready to hand it over to us on the testing team. If Sauce Labs had permissions associated with roles, where this role could do this activity and that role could do that activity, it would be easier for us.

We have raised this suggestion with our product manager and she has put it in the product backlog, but, of course, she cannot guarantee when that will be looked at.

View full review »
AP
LMTS, Software Engineering at Salesforce

Sauce Labs' logging features could be improved. For example, when we run a mobile test on iOS, we want a way to capture the logs generated. It has some drawbacks there. Sauce Labs isn't able to give us everything we test for. 

Sauce Labs' dashboard could be improved by adding more filters and allowing more customization options. There was one instance where the dashboard on the Sauce Labs UI didn't meet our requirements, so we had to use the Sauce Labs API to create some apps and dashboards on our own. The API endpoints could be a little more robust and customizable. 

View full review »
PB
Tech Lead at a pharma/biotech company with 10,001+ employees

Nothing comes to mind right now, but this might be because I'm an administrator and not an end-user who is familiar with the application's use case. If we have teams asking for something that isn't already there, we communicate that back to Sauce Labs. That hasn't happened so far in the time that I've been in this role. 

I may not know what should be improved on the platform, but I think it could offer a greater variety of testing solutions. I know there might be competing solutions that cover more aspects that I'm unaware of, and it could expand a bit more. We've done a lot of automated testing scenarios, and that's all the rage these days, so I think Sauce Labs could look at potential ways to improve and cover other scenarios.

View full review »
BL
Application Engineer at Discover Financial Services

Something that needs improvement is the speed. Relatively speaking, it's slow, but that's because we are communicating with the Sauce Labs cloud service and, obviously, there are network delays. We're hoping that in the future it can just be a bit quicker.

Another aspect that could be improved is having the virtual devices boot up a little quicker. Sometimes we wait up to a minute before a device is available. It would be nice if that was 30 seconds or so.

View full review »
AP
IT Analyst at a manufacturing company with 10,001+ employees

The real concern is the load time of applications or real devices when we start our tests. It takes some time to load the application or web browser. Sometimes, it is frustrating too. Since they are real devices, we understand it takes some time to load. However, if it were to improve, then that would be a great asset to the solution. So, we would like better responsive times when opening applications and running tests.

View full review »
reviewer1398594 - PeerSpot reviewer
Senior Software Development Engineer in Test at Autodesk, Inc.

As a web product QA team, we sometimes need to spot check some new child site on multiple browsers and OS(es). It was a little time consuming for us since we need to click on each of the browser/OS combinations and start a new session to test. Every sprint, with new features and child pages being added, we mostly need to do the same steps over and over again. We would like to see if Sauce Labs can be able to open multiple different browsers on the same website at the same time, just by 1 click. It will reduce a lot of time for manually click and change the devices/OS combination.

View full review »
reviewer1396422 - PeerSpot reviewer
Senior QA Automation Engineer at Bleacher Report

Overall, I think Sauce Labs provides us with a valuable tool and resource. As far as what could be improved, I would say the overall test execution time. Some of the calls take a bit longer than I expect, for example in web browser tests; while the execution time isn't obnoxious, it could be improved so that overall tests/test suites finish faster.

Within the insights section of the Sauce Labs tool, I think it would be nice if you could go back farther than 30 days worth of data. I like all the custom filters available, but being able to go back say 90 days (or more) would show more statistics on our data/test results.

In the next release, I'd like to see continual new additions of OS's. This area is usually pretty up-to-date, please keep at it!

View full review »
reviewer1394535 - PeerSpot reviewer
Lead Quality Analyst at Derivco

If I had to speak of an area that could be improved it would probably have to be the speed of interaction with the devices. There is at times a considerable amount of lag while using some of the virtual and at times even physical device farms. While this is absolutely better than not having it situation Sauce Labs certainly would benefit as well as we the user would certainly benefit from gaining a software interaction speed boost. Having it more consistent would be greatly appreciated if the long run. This is the only thing affecting the testing time at the moment and can even be seen as a disadvantage in comparison to having an actual physical device in hand.

View full review »
Andy Antes - PeerSpot reviewer
Senior Software Engineer at a computer software company with 10,001+ employees

A really nice feature would be a pay-as-you-use model, as opposed to a strict number of VMs that you pay for continuously.

View full review »
Ryan Craven - PeerSpot reviewer
Quality Assurance Engineer at Optum

From a company use standpoint - I have no complaints. The one issue I have is the 14-day trial that a new user gets for free. I understand the concept of the trial period; however, I think this could be revamped to a free 30-minute run time every few months or after a significant update once the trial period has ended. Personally speaking - I like to maintain my automation framework and would like to occasionally test that it continues to run on Sauce Labs after releases without stealing from Sauce Labs.

View full review »
it_user495984 - PeerSpot reviewer
Senior Software Test Automation Engineer at a tech vendor with 10,001+ employees

1. They still have very slow support and ask many excessive questions before starting to fix the issue.

When we submit tickets in their reporting system, we expect them to start tracking down the problem as soon as possible. Instead, it looks like they ask standard questions just to delay work to a few days. E.g. 
1st day: "Could you confirm you use the latest version" - "Yes, we do"

2nd day: "Ok. Can you then try to run that with extra flag Y and grab the log for us" - "Yes, please, take the log"

<2 days of silence>

5th day: ... - "Guys, is there any progress on that?"

and so on...

Sometimes questions are really unrelated and take time like "Have you read the Getting Started guide?"

2. They still have platform-specific issues, especially on Mobile platforms. They wouldn't affect trivial tests though.

E.g. they have issues related to SauceConnect traffic routing specific to iOS 9 and Android 4.

This affect tests that want to sniff the traffic only though. Please, let me know if you want more technical details.

3. Sometimes they change configuration options (e.g. rename...

E.g. recently they changed desired capability for iPhone Simulator in a way like "iPhone 6 simulator" > "iPhone 6"

That affected multiple tests suddenly. They haven't made any notification about that though.

View full review »
reviewer1400664 - PeerSpot reviewer
QA Analyst II at Regal Entertainment Group

A service that could be improved is some of their Wiki documents are either outdated or point to broken URLs; however when contacting the customer service they are able to respond in a quick manner with either an updated document or with the steps to fix the issue. 

Another feature that could still be improved on is more error clarity. Sometimes when running automated scripts the test will fail on the device side instead of the script and errors only show a 500 try again message instead of a detailed script that could of a been a timeout error from the code. 

View full review »
Alejandro Garduno - PeerSpot reviewer
Quality Assurance Lead at ZX-Ventures

The testing process is difficult. I need to prove the complete competency of the tool, and I am finding that challenging. 

View full review »
reviewer1394979 - PeerSpot reviewer
Lead QA Engineer at a healthcare company with 201-500 employees

We have used Sauce Labs extensively for the last 5 years across automated and manual testing and have found some problems during this time. Firstly as we use Sauce Labs to connect the tunnel to connect to apps in our internal network we have found that during automated testing this can be very slow. This causes inconsistencies with the tests. It's very difficult to rely on a service when you can't be sure if a test will pass or fail the next time it runs. This means building in a lot of sync time into the tests which in turn slows them down.  If this speed could be improved then the service would be much better.

View full review »
reviewer1396401 - PeerSpot reviewer
Senior QA Engineer at Indeed.com

Improve easy way to automation testing and generate script automatically while running manual test cases on Sauce. 

1. When running manual test cases, provide us a way to record and playback the same steps

2. Provide JIRA integration plugin so that we can easily log issues.

3. Provide easy way to create scripts automatically for manual steps and use sauce labs UI to execute it again

4. Sauce labs should provide their own email client and easy to send email in that user session

5. One-click setup for sauce proxy

View full review »
OB
Software Engineering Manager at a healthcare company with 10,001+ employees

I would like for there to be more detail in regards to the quality of our code i.e. how many failures occurred, how many passed based on industry standard metrics, etc.

View full review »
reviewer1396425 - PeerSpot reviewer
Senior Automation Engineer / Senior SDET at Setel

The only drawback is the speed, it will be good if we have a server in Asia too. It will be great if we can improve speed while initialization and execution.

View full review »
Buyer's Guide
Sauce Labs
June 2022
Learn what your peers think about Sauce Labs. Get advice and tips from experienced pros sharing their opinions. Updated: June 2022.
610,518 professionals have used our research since 2012.