We performed a comparison between AWS Device Farm and Perfecto based on real PeerSpot user reviews.Find out in this report how the two Mobile App Testing Tools solutions compare in terms of features, pricing, service and support, easy of deployment, and ROI.
"I rate the stability an eight out of ten."
"What I like best about AWS Device Farm is that it offers actual physical devices that let you do more accurate testing because physical devices depict the live testing scenarios much better as opposed to emulated devices. AWS Device Farm is a pretty nice solution. Because it's an AWS service, you can use the CLI to tie in several steps that can create the pipeline, and run it efficiently. AWS Device Farm also gives you monitoring ability, observability, logging, etc., so I'm pretty satisfied with the solution."
"It creates a faster production cycle and is quick to market. Things get deployed earlier because the testing happens on time. We can do a lot of panelization, so a lot of test phases can happen in a panel. People don't have to wait for a device to come to them. They can access multiple devices at the same time and do testing at the same time."
"It saves on the cost and effort of having to maintain our own virtual testing environment. Even our onshore team is not in the city that we work in, so that helps a lot. Even if we didn't invest a lot in getting multiple devices, having to share those devices would become a hassle."
"The quality of our software has improved since we implemented this solution."
"There are a whole bunch of things that I like about the solution, but I really love the interaction it has with mobile devices, the testing capabilities, as well as reporting capabilities that we get from the application. The reports are very detailed."
"I also like the reporting functions. We are constantly downloading these reports and sharing them with our final customers. They help us understand what kind of bugs are happening through the applications. The recording feature is handy because it lets us see a video of the process we run through the pipeline and discover the point at which the automation is breaking."
"In terms of cross-platform testing, they offer all of it, every device available in the market. It covers real scenarios that mimic production so that we don't miss out on any devices that our clients might be using to run the applications we develop. It's been great and very helpful."
"We are able to offer a quality product that has been tested fully, which improves our customer satisfaction. That is a good thing. It has also reduced our IT infrastructure cost. We don't have to spend a lot on setting up infrastructure, which becomes redundant or obsolete very soon. It helps in offsetting that cost."
"In terms of Perfecto's ability to perform cross-platform testing, I would rate it a ten out of ten."
"It is slow. It is super slow. Performance is an area that can be improved."
"An area for improvement in AWS Device Farm is that it lacks a lot of features that would tie it in with other AWS services. The solution doesn't have great connectivity with other services offered by AWS, for example, AWS Secrets Manager. This should be improved because a lot of times, that missing functionality hampers the quality and engineering standards in terms of deploying the full AWS suite of services. What I'd like to see in the next version of AWS Device Farm is for it to link better, or have some type of enrollment that would tie it in with other AWS services, such as EventBridge, Lambda, Secrets Manager, and any other new service from AWS."
"Previously, we used the cradle. Every time the mobile was blocking it, we would have to ask Perfecto to provide another one. That took a lot of time away from us."
"One improvement would be speed of execution. If it is an iOS native app, we have noticed that the speed is a bit slower. Perfecto might need to make some improvements in this area."
"We've had a couple of issues lately with videos not loading or browsers dying after some execution, although that happens very rarely."
"We don't use Perforce's BlazeMeter with Perfecto. From my perspective, it's not really relevant."
"I'm hoping they can support on-premises instances. We have been working on a JIRA integration with Perfecto—and I'm extremely impressed that they have that—but at this time they're not supporting onsite JIRA instances, which is what we have."
"We feel that Perfecto is a little slow. If they could improve on that slowness in accessing the app, when we want to click a button, that would be great because we feel the difference. An improvement in the connectivity speed is required."
"If we could run an accessibility test in Perfecto against builds, it would help us a lot. Currently, that's a very manual process for us. We haven't found a tool that can do accessibility scans for iOS and doesn't depend on engineering effort. Having a feature related to that would be really awesome for us."
"Going by the dashboard or analytics capabilities that Perfecto or Perforce is looking to offer in its roadmap, it will certainly help if they also cater to executing and enabling decision-making, rather than just focusing on standard testing metrics such as execution, efficiency, and defect rate. These are good metrics, but they don't necessarily enable decision-making for SLTs. Any improvements in the dashboards and reporting tools should focus on metrics or SLAs that can help with decision-making."
AWS Device Farm is an application testing service that lets you improve the quality of your web and mobile apps by testing them across an extensive range of desktop browsers and real mobile devices; without having to provision and manage any testing infrastructure. The service enables you to run your tests concurrently on multiple desktop browsers or real devices to speed up the execution of your test suite, and generates videos and logs to help you quickly identify issues with your app.
Perfecto is a cloud-based mobile app testing tool that is designed to help test applications on different operating systems and devices. The solution lets users automate processes related to advanced test scenarios and access browser versions and devices according to individual requirements. With Perfecto’s smart test execution capabilities, teams can manage testing across web and mobile platforms using a single script, conduct parallel testing for large test suites, and design advanced user-based test scenarios to accelerate app release cycles. The solution is also helpful for IT departments, providing insights into biometrics, device vitals, audio injection, network throttling, packet loss, and rotation data to meet desired end-user conditions that align with business priorities.
Perfecto has many valuable key features. Some of the most useful ones include:
There are many benefits to implementing Perfecto. Some of the biggest advantages the solution offers include:
Reviews from Real Users
Perfecto is a solution that stands out when compared to many of its competitors. Some of its major advantages are that it has fantastic testing, reporting, and automation capabilities.
Roland C., QA Lead at BMO Financial, says, “The most valuable would be their Live Stream analysis, where I can see the live analysis of all the executions on a single device or multiple devices as well as track them. The live analysis and reporting would be the single most valuable feature.”
Andy B., Systems Engineer at a financial services firm, writes, “I really love the interaction it has with mobile devices, the testing capabilities, as well as reporting capabilities that we get from the application. Perfecto is excellent when it comes to executing cross-platform testing.”
A Director, QE Automation & Analytics at a financial services firm, says, “The most valuable feature is cloud-based automation and testing.”
Another PeerSpot reviewer, Kristina B., Product Manager at a manufacturing company, states, “The solution allows us to do seamless cross-platform testing, saves time, and comes with excellent customer service.”
AWS Device Farm is ranked 5th in Mobile App Testing Tools with 2 reviews while Perfecto is ranked 3rd in Mobile App Testing Tools with 18 reviews. AWS Device Farm is rated 6.6, while Perfecto is rated 8.6. The top reviewer of AWS Device Farm writes "Gives more accurate results through physical devices that depict the live scenarios much better than emulated devices, but it doesn't have great connectivity with other services, and it's expensive". On the other hand, the top reviewer of Perfecto writes "Its reporting allows us to have a clear view regarding what tests have been executed". AWS Device Farm is most compared with Tricentis Tosca, Kobiton Mobile Device Testing, OpenText UFT Digital Lab, pCloudy and Digital.ai Continuous Testing, whereas Perfecto is most compared with BrowserStack, Sauce Labs, Appium, Katalon Studio and Tricentis Tosca. See our AWS Device Farm vs. Perfecto report.
See our list of best Mobile App Testing Tools vendors.
We monitor all Mobile App Testing Tools reviews to prevent fraudulent reviews and keep review quality high. We do not post reviews by company employees or direct competitors. We validate each review for authenticity via cross-reference with LinkedIn, and personal follow-up with the reviewer when necessary.