Please share with the community what you think needs improvement with Sauce Labs.
What are its weaknesses? What would you like to see changed in a future version?
The testing process is difficult. I need to prove the complete competency of the tool, and I am finding that challenging.
A service that could be improved is some of their Wiki documents are either outdated or point to broken URLs; however when contacting the customer service they are able to respond in a quick manner with either an updated document or with the steps to fix the issue. Another feature that could still be improved on is more error clarity. Sometimes when running automated scripts the test will fail on the device side instead of the script and errors only show a 500 try again message instead of a detailed script that could of a been a timeout error from the code.
As a web product QA team, we sometimes need to spot check some new child site on multiple browsers and OS(es). It was a little time consuming for us since we need to click on each of the browser/OS combinations and start a new session to test. Every sprint, with new features and child pages being added, we mostly need to do the same steps over and over again. We would like to see if Sauce Labs can be able to open multiple different browsers on the same website at the same time, just by 1 click. It will reduce a lot of time for manually click and change the devices/OS combination.
From a company use standpoint - I have no complaints. The one issue I have is the 14-day trial that a new user gets for free. I understand the concept of the trial period; however, I think this could be revamped to a free 30-minute run time every few months or after a significant update once the trial period has ended. Personally speaking - I like to maintain my automation framework and would like to occasionally test that it continues to run on Sauce Labs after releases without stealing from Sauce Labs.
Overall, I think Sauce Labs provides us with a valuable tool and resource. As far as what could be improved, I would say the overall test execution time. Some of the calls take a bit longer than I expect, for example in web browser tests; while the execution time isn't obnoxious, it could be improved so that overall tests/test suites finish faster. Within the insights section of the Sauce Labs tool, I think it would be nice if you could go back farther than 30 days worth of data. I like all the custom filters available, but being able to go back say 90 days (or more) would show more statistics on our data/test results. In the next release, I'd like to see continual new additions of OS's. This area is usually pretty up-to-date, please keep at it!
Improve easy way to automation testing and generate script automatically while running manual test cases on Sauce. 1. When running manual test cases, provide us a way to record and playback the same steps 2. Provide JIRA integration plugin so that we can easily log issues. 3. Provide easy way to create scripts automatically for manual steps and use sauce labs UI to execute it again 4. Sauce labs should provide their own email client and easy to send email in that user session 5. One-click setup for sauce proxy
The only drawback is the speed, it will be good if we have a server in Asia too. It will be great if we can improve speed while initialization and execution.
We have used Sauce Labs extensively for the last 5 years across automated and manual testing and have found some problems during this time. Firstly as we use Sauce Labs to connect the tunnel to connect to apps in our internal network we have found that during automated testing this can be very slow. This causes inconsistencies with the tests. It's very difficult to rely on a service when you can't be sure if a test will pass or fail the next time it runs. This means building in a lot of sync time into the tests which in turn slows them down. If this speed could be improved then the service would be much better.
If I had to speak of an area that could be improved it would probably have to be the speed of interaction with the devices. There is at times a considerable amount of lag while using some of the virtual and at times even physical device farms. While this is absolutely better than not having it situation Sauce Labs certainly would benefit as well as we the user would certainly benefit from gaining a software interaction speed boost. Having it more consistent would be greatly appreciated if the long run. This is the only thing affecting the testing time at the moment and can even be seen as a disadvantage in comparison to having an actual physical device in hand.
More physical devices.
Progress towards reducing application testing time can be made. There should be better and programmatic controls on request/response recordings and sharing with developers similar to new testing frameworks on the market.
* Ability to configure the memory and CPU for the test machines * Ability to install profiles on iOS real mobile devices
It would be nice for it to ask for the URL along with browser/operating system I want to try, but not for it to simply ask for a combination. I want to be able to start ten at the same time. I would also like to put a username and password before the test launch instead of entering ten tabs and having to put the same information over and over again.
Integration with Github, as well as several other similar tools, could be improved. However, Sauce Labs appears to be on that track. With all of the innovations that they have made over the past year and a half, I am looking forward to the changes they have planned for the future.
An image comparison would be a nice feature to include in the Sauce Labs product.
Sometimes pasting text while using "text object" does not seem to work, and it slows down testing times quite a bit.
It should provide more examples of script code.