What is our primary use case?
We need to test our mobile automation by using iOS as well as Android for a lot of our use cases.
We have a couple of use cases where the Perfecto team supports us, e.g., where single sign-on will move from the mobile app to the browser app, and then come back to the mobile app.
The finger signature is one of our main use cases required for testing, and it is supported by Perfecto.
It provides the simulator and emulator that we are using in our project for testing.
How has it helped my organization?
It improves our product quality by testing multiple times and integrating Perfecto into our CI/CD pipelines.
We are continuously doing testing on different environments, devices, and platforms. It executes seamlessly on multiple devices without having any connectivity issues. It has been really helpful for us to test on cloud devices.
What is most valuable?
It has a lot of features as well as 24/7 support. Everything is captured on their website.
The QR code automation for mobile is something that we have trained on using Robot Frameworks, which has not been supported well. We have used Perfecto to automate the QR code automation, which provides a very good value to us. It records the image file injection. This is a very good feature where you can inject the image and test it with the QR code, which is one of the main advantages of using Perfecto.
Currently, we are moving to the cloud. The performance and automation testing are very good. We compared the physical devices with the Android simulation, and its physical devices gave us better performance than the Android simulation.
The cloud and cloud mobile are very good features. They provide us with better performance.
Perfecto's reporting and analytics are very useful for us, and what we can see from Live Stream is really awesome. There are no delays at all. Whatever we are seeing in the mobile, the same thing will show in the Live Stream tool. It provides insights in the pie chart as well as the history and trends. So, you can go through it and find the number of tests executed, the pass/fail cases, etc. They have a report library that gives data about all the executions, including details of the history.
The time stamp helps us out a lot by providing the duration for the previous execution and the current execution. So, there are a lot more reporting configuration tools, which helps us. Also, we can export this particular report and give it to anyone for review.
The lab has been very helpful for us when pushing API or image files. Anything you can push into the repository, we can test it. It is seamless. We can test it through cloud devices, which is very useful for us.
It supports a lot of open-source technologies. We have been using Quantum Framework for more than eight years, which has been really helpful. Perfecto supports Quantum Framework, and this is where we can get Selenium as well as APM functionalities, which is really helpful for us.
They have in-built functionalities, which are very useful for us, e.g., loading the APK and applications. We can also switch contacts from mobile to web and web to mobile. Those are all really helpful for us when implementing the new functionalities into Perfecto by using Selenium as well as the APM.
What needs improvement?
Previously, we used the cradle. Every time the mobile was blocking it, we would have to ask Perfecto to provide another one. That took a lot of time away from us.
For how long have I used the solution?
We have been using Perfecto for eight to 10 years.
What do I think about the stability of the solution?
It has good device stability as well as the support from their team. Devices aren't going down.
What do I think about the scalability of the solution?
Scalability-wise, it is great. Devices are always available.
If we need devices, we can just send a request to Perfecto.
We have 300 to 400 devices in Android and 300 to 400 devices in iOS.
How are customer service and support?
Whenever we have a doubt about mobile device testing, we just go to the Perfecto site and what we need will be there. For example, we were facing some issues when scrolling down. So, we went to the Perfecto site and found that particular solution.
You just need to create a ticket for Perfecto, and most things will be immediately addressed. Complex solutions are also available on the Perfecto website.
I would rate the support team as eight out of 10.
How would you rate customer service and support?
Which solution did I use previously and why did I switch?
We were using the simulator and the emulator with Robot Framework, which was very slow compared to our current solution. It did not support many functionalities so we were struggling with the QR code automation and finger signature. When we switched to Perfecto, they helped us implement those testing functionalities and supported our entire end-to-end testing.
Earlier, we were using Perfecto cradle and GigaFox. Perfecto cradle and GigaFox, had higher maintenance costs and Wi-Fi issues. For example, during the pandemic when we were looking into devices, the issues were because of battery heat. It would buzz and need to be replaced. Sometimes, it would be down because of the Wi-Fi.
Once Perfecto started utilizing the cloud, we didn't need to worry about anything, saving us on costs. We didn't need to keep an eye on devices because Perfecto took care of it, giving a more efficient way to save. Also, the performance is awesome, which helps us a lot. We don't need to babysit the devices anymore.
What was our ROI?
It used to take 18 minutes in the simulator and emulator to test our applications. Perfecto takes around nine minutes. Our testing time was reduced starting from our first POC. Performance-wise, Perfecto has been awesome. Test-wise, it also provides seamless execution.
Which other solutions did I evaluate?
We have approached many solutions, like Sauce Lab and Kobiton. Comparatively, Perfecto is a good product. Feature-wise, finding the element is very easy compared to other devices. It has Object Spy, where we can go through a device to find and identify locators.
We are using Perfecto cloud with iOS and Android devices. It is pretty fast compared to GigaFox and Kobiton, which we have also used in the cloud. Comparatively, Perfecto gives us more performance while doing testing. So, we have done multi-level, multi-device testing for both Android as well as iOS, and it has very good performance compared to the cradle solution. The cloud is the best one.
What other advice do I have?
We are planning to do performance testing by using BlazeMeter and other tools, then we can do the load testing through mobile devices. We are hoping to start using this within a month.
I would rate it as nine to 10 (out of 10) because the solution is scalable and stable. It also has support, which is important.