AWS Device Farm vs HeadSpin comparison

 

Comparison Buyer's Guide

Executive Summary
 

Categories and Ranking

AWS Device Farm
Ranking in Mobile App Testing Tools
5th
Average Rating
6.6
Number of Reviews
2
Ranking in other categories
No ranking in other categories
HeadSpin
Ranking in Mobile App Testing Tools
7th
Average Rating
8.0
Number of Reviews
6
Ranking in other categories
Functional Testing Tools (20th), Mobile APM (6th)
 

Mindshare comparison

As of June 2024, in the Mobile App Testing Tools category, the mindshare of AWS Device Farm is 0.5%, down from 6.8% compared to the previous year. The mindshare of HeadSpin is 1.4%, down from 3.0% compared to the previous year. It is calculated based on PeerSpot user engagement data.
Mobile App Testing Tools
Unique Categories:
No other categories found
Functional Testing Tools
0.6%
 

Featured Reviews

OC
Sep 26, 2022
Gives more accurate results through physical devices that depict the live scenarios much better than emulated devices, but it doesn't have great connectivity with other services, and it's expensive
An area for improvement in AWS Device Farm is that it lacks a lot of features that would tie it in with other AWS services. The solution doesn't have great connectivity with other services offered by AWS, for example, AWS Secrets Manager. This should be improved because a lot of times, that missing functionality hampers the quality and engineering standards in terms of deploying the full AWS suite of services. What I'd like to see in the next version of AWS Device Farm is for it to link better, or have some type of enrollment that would tie it in with other AWS services, such as EventBridge, Lambda, Secrets Manager, and any other new service from AWS.
Saorabh Singh - PeerSpot reviewer
Nov 13, 2022
It fulfills everything from automation to manual performance
We use HeadSpin for automated, manual, and performance testing of our apps. Presently, we have 70 users, including QA, developers, product managers, and the operations team HeadSpin allowed us to manage remote work during Covid, enable continuous automation execution, and facilitate collaboration…

Quotes from Members

We asked business professionals to review the solutions they use. Here are some excerpts of what they said:
 

Pros

"I rate the stability an eight out of ten."
"What I like best about AWS Device Farm is that it offers actual physical devices that let you do more accurate testing because physical devices depict the live testing scenarios much better as opposed to emulated devices. AWS Device Farm is a pretty nice solution. Because it's an AWS service, you can use the CLI to tie in several steps that can create the pipeline, and run it efficiently. AWS Device Farm also gives you monitoring ability, observability, logging, etc., so I'm pretty satisfied with the solution."
"The most valuable feature is that this is the first connected intelligence all-in-one platform."
"It has an interesting feature called AV box testing. A lot of companies that are in the OTT segment don't really understand what their streaming is like. They can't test for streaming quality. There are restrictions where you cannot simulate live streaming. For example, on Netflix, you can't simulate how a movie is being streamed on a remote device. That's why HeadSpin has got this AV box testing feature. It is a patented feature. They send an AV box to your location, and you can test live streaming, which is something that no other company does."
"The initial setup of HeadSpin was very easy and user-friendly. It was easy to configure and write a script."
"The technical support is really helpful because we can set up direct calls with them if we want to. We can use Zoom or Google Meet to interact with them directly, and if there is an issue in our system, they will help us by reproducing the issue in their machines and trying to figure out a solution. The support is really smooth, and we like that they're very supportive."
"The most valuable feature of HeadSpin it's the integration with other solutions. It is great. I can search for an element or do a quick debugging on the application right on HeadSpin. It's very useful."
"The most valuable features of the product are the performance parameters it gives us."
 

Cons

"An area for improvement in AWS Device Farm is that it lacks a lot of features that would tie it in with other AWS services. The solution doesn't have great connectivity with other services offered by AWS, for example, AWS Secrets Manager. This should be improved because a lot of times, that missing functionality hampers the quality and engineering standards in terms of deploying the full AWS suite of services. What I'd like to see in the next version of AWS Device Farm is for it to link better, or have some type of enrollment that would tie it in with other AWS services, such as EventBridge, Lambda, Secrets Manager, and any other new service from AWS."
"It is slow. It is super slow. Performance is an area that can be improved."
"If you want to do some testing or check the devices manually or check the application in a particular device manually, it is really laggy. That's a disappointment because sometimes we would like to do manual testing when our local devices are not available."
"Sometimes, devices go offline and some features are not functioning on some devices, specifically on iOS."
"They should automate their onboarding. A lot of things are still manual. They can create a video assistant or something like that to completely automate the entire process."
"HeadSpin needs to improve the hardware. With the mobile, the battery life reduces and must be continuously charged."
"Support and pricing could be improved."
"HeadSpin could improve on the user interface because it is very poor. The checks that are done on the iOS devices are very difficult, but for Android, it runs great. For all iOS devices, the user interface and how it interacts with the device are very poor."
 

Pricing and Cost Advice

"AWS Device Farm is an expensive service overall. You pay per device, and the cost for each device isn't cheap. My company paid for a device slot. It's a yearly subscription for a single device slot, so that's more cost-effective for my company. On a scale of one to five, where one is very expensive and five is very cheap, I'd give AWS Device Farm a two. It's on the more expensive side."
"It has a yearly license. There is no other option. It is expensive. There are a lot of other cheaper players in the market, but it is like a Mercedes. You pay an extra premium for it, but you get the benefits. I would love to see them come up with project-based costing. Companies that are low on funds or new-age can do with pricing that is easily digestible. They can give them a pricing model for three months. They can provide a startup package."
"I believe the licensing cost is cheap because it's a total solution, hardware, license and software."
"It's not cheap, but there are a few different packages and different prices for enterprises with different product versions."
"We have a yearly license for 16 devices."
report
Use our free recommendation engine to learn which Mobile App Testing Tools solutions are best for your needs.
787,779 professionals have used our research since 2012.
 

Top Industries

By visitors reading reviews
Financial Services Firm
19%
Computer Software Company
18%
Insurance Company
7%
Manufacturing Company
7%
Financial Services Firm
20%
Manufacturing Company
15%
Computer Software Company
12%
Real Estate/Law Firm
8%
 

Company Size

By reviewers
Large Enterprise
Midsize Enterprise
Small Business
No data available
No data available
 

Questions from the Community

What do you like most about AWS Device Farm?
I rate the stability an eight out of ten.
What is your experience regarding pricing and costs for AWS Device Farm?
AWS Device Farm is an expensive service overall. You pay per device, and the cost for each device isn't cheap. My company paid for a device slot. It's a yearly subscription for a single device slot...
What needs improvement with AWS Device Farm?
It is slow. It is super slow. Performance is an area that can be improved. Whenever we upload our Zip files to that particular platform, you know, it takes quite some time, especially if the file i...
What do you like most about HeadSpin?
The most valuable features of the product are the performance parameters it gives us.
What is your experience regarding pricing and costs for HeadSpin?
We have a yearly license for 16 devices costing 35 - 40 lakhs in rupees.
What needs improvement with HeadSpin?
HeadSpin would benefit by adding code coverage. On the front end, when we execute our automation or do a manual change, we don't know what percentage of the code has been tested. If HeadSpin could ...
 

Comparisons

 

Overview

 

Sample Customers

NFL, Etsy, Tableau, Gannett, Miniclip, Allstate, Rainforest, goibibo, mysmartprice, Zillow
Zynga, Tinder, Pinterest, Akamai, Microsoft, Airbnb, Jam City, TMobile, Mozilla, CNN, Cognizant, Yahoo!, ebay, Quora, Walmart, Kohls, Telstra
Find out what your peers are saying about AWS Device Farm vs. HeadSpin and other solutions. Updated: May 2024.
787,779 professionals have used our research since 2012.