What is our primary use case?
I use AWS Device Farm as an emulation environment, and it's also where you can utilize real, physical devices that can install applications, then you can use Appium to run and perform tests on your applications on the devices set up on AWS Device Farm.
How has it helped my organization?
AWS Device Farm is beneficial to our company, and the benefits we get from it are the same as the benefits we receive from Appium because we use Appium in conjunction with AWS Device Farm.
What is most valuable?
What I like best about AWS Device Farm is that it offers actual physical devices that let you do more accurate testing because physical devices depict the live testing scenarios much better as opposed to emulated devices.
AWS Device Farm is a pretty nice solution. Because it's an AWS service, you can use the CLI to tie in several steps that can create the pipeline, and run it efficiently. AWS Device Farm also gives you monitoring ability, observability, logging, etc., so I'm pretty satisfied with the solution.
What needs improvement?
An area for improvement in AWS Device Farm is that it lacks a lot of features that would tie it in with other AWS services. The solution doesn't have great connectivity with other services offered by AWS, for example, AWS Secrets Manager. This should be improved because a lot of times, that missing functionality hampers the quality and engineering standards in terms of deploying the full AWS suite of services.
What I'd like to see in the next version of AWS Device Farm is for it to link better, or have some type of enrollment that would tie it in with other AWS services, such as EventBridge, Lambda, Secrets Manager, and any other new service from AWS.
For how long have I used the solution?
I've been using AWS Device Farm for about two years now.
What do I think about the stability of the solution?
AWS Device Farm is a stable solution.
What do I think about the scalability of the solution?
I didn't see any scaling issues with AWS Device Farm. It's a scalable solution, but it's a bit expensive to scale.
How are customer service and support?
AWS support is quite responsive, and whenever my team had any issues with the AWS Device Farm setup, the AWS support team has always been very responsive and always gave guidance.
Which solution did I use previously and why did I switch?
We only use AWS Device Farm and have not used any other solution. AWS Device Farm is one of the most cost-effective solutions out there, and it also ties in with other AWS services. As we use AWS extensively in our organization, it just made more sense to use AWS Device Farm instead of others that would offer the same functionality.
How was the initial setup?
Setting up AWS Device Farm wasn't difficult. It didn't take up much time.
What was our ROI?
We haven't been making money off of AWS Device Farm. There have been cost savings or additional revenue, but those aren't revenue that's directly linked to the solution.
What's my experience with pricing, setup cost, and licensing?
AWS Device Farm is an expensive service overall. You pay per device, and the cost for each device isn't cheap.
My company paid for a device slot. It's a yearly subscription for a single device slot, so that's more cost-effective for my company.
On a scale of one to five, where one is very expensive and five is very cheap, I'd give AWS Device Farm a two. It's on the more expensive side.
What other advice do I have?
I'm using the latest version of AWS Device Farm. The solution is specifically provided by AWS, and it doesn't have a non-cloud version. My company runs it on AWS.
Between four to five people use AWS Device Farm in my company.
A single person can deploy AWS Device Farm, but it would still depend on how much you want to scale. If you have hundreds and hundreds of test cases running at a single time, then that would require more people, but to get AWS Device Farm up and running, you don't need more than one person to do that.
My company doesn't have plans of increasing AWS Device Farm usage because it's pretty expensive, but it's being used for a lot of work streams within the organization.
My advice to anyone who wants to use AWS Device Farm is that there's an expiration, and you have to check the expiry date on the test packages and test specifications. It would be nice to automate that part of AWS Device Farm.
The rating I'd give AWS Device Farm is six out of ten.
My company has a partnership with AWS.
Which deployment model are you using for this solution?
Public Cloud
If public cloud, private cloud, or hybrid cloud, which cloud provider do you use?
Amazon Web Services (AWS)
Disclosure: My company has a business relationship with this vendor other than being a customer: partner