Manager at a insurance company with 1,001-5,000 employees
Real User
Top 20
Reasonable price, reduces infrastructure cost, and provides access to all browser versions we need
Pros and Cons
  • "We are able to offer a quality product that has been tested fully, which improves our customer satisfaction. That is a good thing. It has also reduced our IT infrastructure cost. We don't have to spend a lot on setting up infrastructure, which becomes redundant or obsolete very soon. It helps in offsetting that cost."
  • "Going by the dashboard or analytics capabilities that Perfecto or Perforce is looking to offer in its roadmap, it will certainly help if they also cater to executing and enabling decision-making, rather than just focusing on standard testing metrics such as execution, efficiency, and defect rate. These are good metrics, but they don't necessarily enable decision-making for SLTs. Any improvements in the dashboards and reporting tools should focus on metrics or SLAs that can help with decision-making."

What is our primary use case?

We are using it to test the digital application that we are building. It is an insurance application. It is a web application that we test on multiple browsers and mobile devices. It is not a mobile native app.

We are an insurance company. As we expand the feature set or offerings that we have for our customers, we have to ensure that we are catering to the gamut of users. More and more people are becoming digital savvy, and they are using multiple devices to access applications online and on the road. So, we need to test our application on all possible devices that people may use. Even though it is not a mobile application, people do use mobiles for accessing web applications.

How has it helped my organization?

We have been using a competing product. We encountered some challenges in that competing solution in the cross-browser area. Perfecto has certainly brought positive change. We no longer have the defects or issues we were encountering earlier. It has been great from that aspect.

We use Perfecto’s cloud-based lab to test across devices, browsers, and OSs. It is very important for us. It saves the effort and the cost of investing in all different devices, which can become obsolete very soon. Continuous maintenance is not needed, and there is a single platform to access everything. All these benefits don't just make it cost-effective but also make things administratively light. It is very useful for us.

They provide same-day access to new devices, but we didn't require that. The majority of our tests focus on the in-market version. So, we haven't used it, but that's a good capability to have.

Perfecto’s device lab has helped us to scale our testing operations. One good thing is that you just need an email address, and you can share it with anyone. It is very prompt.

We are able to offer a quality product that has been tested fully, which improves our customer satisfaction. That is a good thing. It has also reduced our IT infrastructure cost. We don't have to spend a lot on setting up infrastructure, which becomes redundant or obsolete very soon. It helps in offsetting that cost.

It certainly has brought more capabilities to our coordinating teams. We are able to cater to different testing asks from the business and IT teams, and we are able to meet some of the key areas that our enterprise testing strategies are requiring us to do.

It is very important for scaling our testing. Investing in all different devices and then managing them, when it is not your primary job as a company, is not easy. Using virtual devices is certainly something that we can easily do at an enterprise level.

It affects our software quality. Our solutions are now tested for compatibility and accessibility. In the Ontario region where we are operating, accessibility is mandated by the government. It has also made us compliant with the accessibility requirements.

What is most valuable?

The inventory of browser options with the latest versions is valuable. There are various iOS and Android options. They are useful to us. We are using its cross-browser capability. It is certainly helpful for us. We have CrossBrowser for Mobile. We use Edge as our enterprise browser, but we know people use Chrome, Safari, and other browsers. We test our application through those browsers.

We also like some of the plugins and integrations that Perfecto has.

What needs improvement?

Going by the dashboard or analytics capabilities that Perfecto or Perforce is looking to offer in its roadmap, it will certainly help if they also cater to executing and enabling decision-making, rather than just focusing on standard testing metrics such as execution, efficiency, and defect rate. These are good metrics, but they don't necessarily enable decision-making for SLTs. Any improvements in the dashboards and reporting tools should focus on metrics or SLAs that can help with decision-making.

Buyer's Guide
Perfecto
November 2022
Learn what your peers think about Perfecto. Get advice and tips from experienced pros sharing their opinions. Updated: November 2022.
653,522 professionals have used our research since 2012.

For how long have I used the solution?

My team and I started using Perfecto a couple of months ago. It has been about three months, but different units in our organization have been using Perfecto and various other tools. 

What do I think about the stability of the solution?

It is on the cloud. So, it is available anytime. Its stability is very good.

What do I think about the scalability of the solution?

It is positive. We haven't noted any challenges so far.

How are customer service and support?

So far, so good. We haven't created an official service ticket yet, but the inquiry and the follow-up seem to be good. 

How would you rate customer service and support?

Positive

Which solution did I use previously and why did I switch?

We used SmartBear. There were certain challenges that were noted by the users of the tool. The testers on my team and on the vendor's team noted a few challenges. The tool was showing some issues that we couldn't see on the actual mobile device or browser. There were some false red flags that the tool was throwing. Sometimes, the UI controls were not clickable. We had to invest a lot in defect analysis and resolution. So, we started looking at alternatives, and we started to evaluate Perfecto. 

In terms of the overall usage in the testing space, we have very limited use of Perfecto. We still rely on major enterprise tooling sets. As we transform our IT services, at least in the area where we have identified its use, it has brought positive change.

Perfecto hasn't helped to speed up our testing efforts. We were using a similar tool, and the testing effort has been the same as before.

How was the initial setup?

I was not involved in its setup.

What was our ROI?

It has saved us the costs and efforts of having to maintain our own virtual test environment, but I don't have the metrics. We haven't done an ROI yet. At the end of the year, we might undertake that exercise.

When assessing the cost of our investment in the solution against the benefits we receive from it, we start at an equal playing field because we had another tool previously. However, the cost of investing in this tool to do testing to catch defects before going live would be six times less than the cost of finding defects and fixing them in production. We will use that as a benchmark. We will also consider the effort that it has taken for us to execute test cases specifically on devices and browsers. Those will be the basis for us to calculate the ROI.

What's my experience with pricing, setup cost, and licensing?

Pricing-wise, it is fine. It is not as expensive as what we used to have in the past from HP, IBM, and others. It is decently priced.

Which other solutions did I evaluate?

It came through recommendations from other teams that were using it. So, we started with a trial. We used BitBar, and we also used CrossBrowser for Mobile.

Even though we are not using a lot of integrations, we do like some of the plugins and integrations that Perfecto has. Essentially, the tool is not integrated into our entire testing life cycle. 

With BitBar, we did not see a direct Jira plugin. So, the scope of use may be limited, or we may have not explored that integration. Perfecto seems to have that integration, and based on the documentation I had from the team, it seems that it is also well integrated with the Jira life cycle, which is valuable to me as a QE Manager.

What other advice do I have?

I would advise specifically identifying the use cases that you want to try with the solution. The range of offerings that Perfecto is planning to bring to the market seems good. There is a lot of innovation happening in the space of test data management and usability and accessibility testing. If you are not looking at spending a lot of licensing money on enterprise tools, Perfecto is certainly good for starting small and then scaling up at the enterprise level. 

Perfecto supports open-source technologies, such as Selenium and Appium among others, but we are not using these capabilities. Based on my theoretical understanding and assessment of other tools, the range seems really impressive. Their roadmap is also impressive. It is good to see the integrations and collaboration it has with most of our testing tools. That's really impressive. 

They are certainly a good provider of services and solutions. We do not use Perforce BlazeMeter with Perfecto, but its collaboration with BlazeMeter excites me. I'm happy about that.

We don't leverage Perfecto's reporting and analytics capability, and we also don't use it for cross-platform testing.

Based on the size of the solution offering and the way they operate, I would rate it a nine out of ten. 

Which deployment model are you using for this solution?

Public Cloud
Disclosure: PeerSpot contacted the reviewer to collect the review and to validate authenticity. The reviewer was referred by the vendor, but the review is not subject to editing or approval by the vendor.
Flag as inappropriate
PeerSpot user
reviewer1963857 - PeerSpot reviewer
Head of Quality at a comms service provider with 1,001-5,000 employees
Real User
The network testing can be accessed from anywhere in the world, but the UI needs revamping
Pros and Cons
  • "It creates a faster production cycle and is quick to market. Things get deployed earlier because the testing happens on time. We can do a lot of panelization, so a lot of test phases can happen in a panel. People don't have to wait for a device to come to them. They can access multiple devices at the same time and do testing at the same time."
  • "There could be some improvements done on the interface. At times, there has been a bit of a struggle when finding things on the interface. A UI revamp would be a better option in future. That UI hasn't changed much in a long time, so I think they could just make it a bit better so that people could find stuff easily and intuitively."

What is our primary use case?

We are using Perfecto to do remote testing on physical devices. 

The device is quite significant for us because we are part of a telecom company and quite a lot of testing is sensitive. We need to have private cloud hosting as well as public cloud hosting. Perfecto gives us both options, and we are using both private as well as public cloud. When a new feature comes out, we cannot just put it on the public cloud for everyone to test. That is why we use a private cloud. We can access it over the internet anywhere in the world, which is very important for us.

We use it to work across multiple teams and departments.

How has it helped my organization?

Perfecto plays a very crucial part in our day-to-day test executions. We do a lot of mobile testing, so Perfecto is very useful for us for either manual testing or automation.

It is very important to us that the lab provides same-day access to new devices. We have a very fast-moving, agile team. Therefore, we need access to the lab all the time.

We have a small development team who uses their emulators for development purposes. For testing, we still rely a lot on the actual devices. However, at times, when we launch something, we do use the virtual devices as well. Virtual devices are important when testing, although they don't give the same level of experience as real devices. I can understand that it isn't possible to purchase every single device all the time, especially when you are testing later versions of Android and don't have a virtual device or any version of iOS. We have a couple of native application teams who deploy new applications on those virtual devices all the time, depending on the requirements.

Perfecto helps a lot with productivity overall. We have a distributed team across different geographics. Because people are spread out across different countries, not everyone works at the same time, but whenever they work, they want access to the devices. The only feasible solution is to look at the cloud. Perfecto has come in really handy since it is always up and active. People can access it in their own time and at their own comfort, i.e., anyplace and anywhere. So, it has definitely helped with our fast, agile deliveries over the course of time.

Any testing is good testing. The more you test, the better software quality gets. It has definitely helped.

What is most valuable?

Recently, the network testing component where we can use Perfecto on devices has been the most valuable feature. There is very little competition out there in this regard. We use it for 4G and 5G testing of launched 5G and 4G brands. So, the network testing can be accessed from anywhere in the world. That is very valuable for us.

Perfecto is very good when it comes to executing cross-platform testing. We have just gotten started on cross-browser testing. We already use Perfecto for the native-application testing on multiple devices. I think now we also use Perfecto for browser-related testing. So, all the cross-browser testing works well. We got a couple of licenses that we use.

We can mostly see usage over time using the device logs. This is what matters the most to us, e.g., what team is using a device for how long.

We use Cypress, and it supports it quite well. We use APM, and it also supports that quite well. We used to use the native quantum framework as well in the past, which was obviously well-supported, but now we have moved onto Cypress and APM. Also, we have Applitools, which is not open source, but a paid proprietary tool.

What needs improvement?

There could be some improvements done on the interface. At times, there has been a bit of a struggle when finding things on the interface. A UI revamp would be a better option in future. That UI hasn't changed much in a long time, so I think they could just make it a bit better so that people could find stuff easily and intuitively.

For how long have I used the solution?

We have been using Perfecto for over six years.

What do I think about the stability of the solution?

It is mostly stable. There could be times when the system or environment is down, then we get notified on those things. They have situations where it is not stable, and we have reached out to the team, saying, "Yeah, it just goes blank at times," or "People are not able to log in." We receive ample support for that. 

Maintenance is required, but we outsource it to a third-party vendor.

What do I think about the scalability of the solution?

The product is pretty scalable. 

Perfecto’s device lab has helped us to scale our operations. We use the public cloud as well as the private cloud. With the public cloud, we can scale up devices very often. Our public cloud is also growing at the moment. This is definitely important to us.

How are customer service and support?

The support that we are getting is really valuable. The technical support is top-notch. There is a dedicated person who takes care of us. We reach out to him, then he reaches out to his team. I would rate the support as seven or eight out of 10.

How would you rate customer service and support?

Positive

Which solution did I use previously and why did I switch?

Previously, we used to purchase a lot of physical devices for testing. The main constraint was that the team was distributed across the globe. Someone would purchase a solution, then someone else needs to distribute it over to those people, which was a mess. Because Perfecto is on the cloud, we can now have devices in one place and everyone can log into the interface. We can look at their testing or whatever they want to do. That has been a massive update.

I was using Xamarin Test Cloud in the past, which had the same mobile capabilities. We were also using a tool that wasn't as scalable as Perfecto, and that was one of the reasons why we switched.

How was the initial setup?

The initial setup was straightforward. While there is always complexity involved, the rollback mechanism works fine.

Deployment is usually fast. It takes about one night.

What about the implementation team?

Not many people are needed for the deployment. We have a small DevOps team and it takes two to three people tops.

What was our ROI?

It creates a faster production cycle and is quick to market. Things get deployed earlier because the testing happens on time. We can do a lot of panelization, so a lot of test phases can happen in a panel. People don't have to wait for a device to come to them. They can access multiple devices at the same time and do testing at the same time.

Perfecto has definitely saved us on the costs and efforts of having to maintain our own virtual test environment. We lost about 20 devices in the past to maintenance and audit. That was a massive loss for us, as a company, because we were giving devices to someone, but don't know whether we would get it back or not. Having those virtual labs, we don't need to worry about these kinds of things. We are easily saving $5,000 to $10,000 a month on device costing.

It easily saves us a couple of hours a day, every day, in terms of waiting on the devices, preparing devices, etc.

We are definitely getting benefits out of using Perfecto. That is why we have an ongoing relationship with them.

What's my experience with pricing, setup cost, and licensing?

We use it on the enterprise level. The pricing is reasonable.

Which other solutions did I evaluate?

We haven't evaluated any other options in five or six years.

What other advice do I have?

Perfecto is doing a very good job. Definitely give it a go. Do some PoCs on your environment with the cloud solution to see if it works for your team. If it does, then definitely give it a go. It is worth the money.

We haven't explored the BlazeMeter component yet. We have had a lot of demos on it and are looking really closely at BlazeMeter.

I would rate Perfecto as seven out of 10. Overall, we are happy, but there is room for improvement.

Disclosure: PeerSpot contacted the reviewer to collect the review and to validate authenticity. The reviewer was referred by the vendor, but the review is not subject to editing or approval by the vendor.
Flag as inappropriate
PeerSpot user
Buyer's Guide
Perfecto
November 2022
Learn what your peers think about Perfecto. Get advice and tips from experienced pros sharing their opinions. Updated: November 2022.
653,522 professionals have used our research since 2012.
Govindaraj Ponnusamy - PeerSpot reviewer
Automation Architect at a tech consulting company with 10,001+ employees
Real User
Improves our product quality by testing multiple times and being integrated into our CI/CD pipelines
Pros and Cons
  • "We are continuously doing testing on different environments, devices, and platforms. It executes seamlessly on multiple devices without having any connectivity issues. It has been really helpful for us to test on cloud devices."
  • "Previously, we used the cradle. Every time the mobile was blocking it, we would have to ask Perfecto to provide another one. That took a lot of time away from us."

What is our primary use case?

We need to test our mobile automation by using iOS as well as Android for a lot of our use cases. 

We have a couple of use cases where the Perfecto team supports us, e.g., where single sign-on will move from the mobile app to the browser app, and then come back to the mobile app. 

The finger signature is one of our main use cases required for testing, and it is supported by Perfecto.

It provides the simulator and emulator that we are using in our project for testing.

How has it helped my organization?

It improves our product quality by testing multiple times and integrating Perfecto into our CI/CD pipelines.

We are continuously doing testing on different environments, devices, and platforms. It executes  seamlessly on multiple devices without having any connectivity issues. It has been really helpful for us to test on cloud devices.

What is most valuable?

It has a lot of features as well as 24/7 support. Everything is captured on their website.

The QR code automation for mobile is something that we have trained on using Robot Frameworks, which has not been supported well. We have used Perfecto to automate the QR code automation, which provides a very good value to us. It records the image file injection. This is a very good feature where you can inject the image and test it with the QR code, which is one of the main advantages of using Perfecto.

Currently, we are moving to the cloud. The performance and automation testing are very good. We compared the physical devices with the Android simulation, and its physical devices gave us better performance than the Android simulation.

The cloud and cloud mobile are very good features. They provide us with better performance.

Perfecto's reporting and analytics are very useful for us, and what we can see from Live Stream is really awesome. There are no delays at all. Whatever we are seeing in the mobile, the same thing will show in the Live Stream tool. It provides insights in the pie chart as well as the history and trends. So, you can go through it and find the number of tests executed, the pass/fail cases, etc. They have a report library that gives data about all the executions, including details of the history. 

The time stamp helps us out a lot by providing the duration for the previous execution and the current execution. So, there are a lot more reporting configuration tools, which helps us. Also, we can export this particular report and give it to anyone for review.

The lab has been very helpful for us when pushing API or image files. Anything you can push into the repository, we can test it. It is seamless. We can test it through cloud devices, which is very useful for us. 

It supports a lot of open-source technologies. We have been using Quantum Framework for more than eight years, which has been really helpful. Perfecto supports Quantum Framework, and this is where we can get Selenium as well as APM functionalities, which is really helpful for us. 

They have in-built functionalities, which are very useful for us, e.g., loading the APK and applications. We can also switch contacts from mobile to web and web to mobile. Those are all really helpful for us when implementing the new functionalities into Perfecto by using Selenium as well as the APM.

What needs improvement?

Previously, we used the cradle. Every time the mobile was blocking it, we would have to ask Perfecto to provide another one. That took a lot of time away from us. 

For how long have I used the solution?

We have been using Perfecto for eight to 10 years.

What do I think about the stability of the solution?

It has good device stability as well as the support from their team. Devices aren't going down.

What do I think about the scalability of the solution?

Scalability-wise, it is great. Devices are always available. 

If we need devices, we can just send a request to Perfecto.

We have 300 to 400 devices in Android and 300 to 400 devices in iOS.

How are customer service and support?

Whenever we have a doubt about mobile device testing, we just go to the Perfecto site and what we need will be there. For example, we were facing some issues when scrolling down. So, we went to the Perfecto site and found that particular solution.

You just need to create a ticket for Perfecto, and most things will be immediately addressed. Complex solutions are also available on the Perfecto website.

I would rate the support team as eight out of 10.

How would you rate customer service and support?

Positive

Which solution did I use previously and why did I switch?

We were using the simulator and the emulator with Robot Framework, which was very slow compared to our current solution. It did not support many functionalities so we were struggling with the QR code automation and finger signature. When we switched to Perfecto, they helped us implement those testing functionalities and supported our entire end-to-end testing.

Earlier, we were using Perfecto cradle and GigaFox. Perfecto cradle and GigaFox, had higher maintenance costs and Wi-Fi issues. For example, during the pandemic when we were looking into devices, the issues were because of battery heat. It would buzz and need to be replaced. Sometimes, it would be down because of the Wi-Fi. 

Once Perfecto started utilizing the cloud, we didn't need to worry about anything, saving us on costs. We didn't need to keep an eye on devices because Perfecto took care of it, giving a more efficient way to save. Also, the performance is awesome, which helps us a lot. We don't need to babysit the devices anymore.

What was our ROI?

It used to take 18 minutes in the simulator and emulator to test our applications. Perfecto takes around nine minutes. Our testing time was reduced starting from our first POC. Performance-wise, Perfecto has been awesome. Test-wise, it also provides seamless execution. 

Which other solutions did I evaluate?

We have approached many solutions, like Sauce Lab and Kobiton. Comparatively, Perfecto is a good product. Feature-wise, finding the element is very easy compared to other devices. It has Object Spy, where we can go through a device to find and identify locators.

We are using Perfecto cloud with iOS and Android devices. It is pretty fast compared to GigaFox and Kobiton, which we have also used in the cloud. Comparatively, Perfecto gives us more performance while doing testing. So, we have done multi-level, multi-device testing for both Android as well as iOS, and it has very good performance compared to the cradle solution. The cloud is the best one.

What other advice do I have?

We are planning to do performance testing by using BlazeMeter and other tools, then we can do the load testing through mobile devices. We are hoping to start using this within a month.

I would rate it as nine to 10 (out of 10) because the solution is scalable and stable. It also has support, which is important.

Disclosure: PeerSpot contacted the reviewer to collect the review and to validate authenticity. The reviewer was referred by the vendor, but the review is not subject to editing or approval by the vendor.
Flag as inappropriate
PeerSpot user
Automation Consultant at a consultancy with 10,001+ employees
Consultant
We can execute tests in multiple browsers through Quantum framework, but test history should be extended
Pros and Cons
  • "The CI dashboard tool is very good, as is the Live Stream monitoring. Whenever I want to monitor execution, I can open multiple tabs in Perfecto and it is easy for me to refer to the CI dashboard and the Live Stream."
  • "We feel that Perfecto is a little slow. If they could improve on that slowness in accessing the app, when we want to click a button, that would be great because we feel the difference. An improvement in the connectivity speed is required."

What is our primary use case?

We use it to automate regression test cases for the mobile app of our application. We use Android and iOS for execution, in both English and French. We also use the solution to execute tests in different browsers.

How has it helped my organization?

It has really helped us speed up our testing process. We are able to execute tests overnight and see the results quickly.

Our devices are in the Perfecto cloud and that's extremely important because we have monthly releases and a lot of OS upgrades in our mobile application. We always have to validate our app for both forward and backward compatibility, as well as in the browser, to make sure that all the features of the application are working. Perfecto facilitates delivering the product on time, through the cloud devices, so that there is no impact to our end-users. That's extremely important for us.

The Perfecto lab also provides same-day access to new devices, which is another very important aspect. If we want to do some tests to make sure the features work in the latest Android 11 version, we don't have to contact the Perfecto team and ask them to provide us with access to that particular device. We can get that access and execute tests against the devices.

What is most valuable?

The CI dashboard tool is very good, as is the Live Stream monitoring. Whenever I want to monitor execution, I can open multiple tabs in Perfecto and it is easy for me to refer to the CI dashboard and the Live Stream. 

It's also very user-friendly. I can see and locate any issues that come in and it's easy to see which test case has failed. Under the Perfecto job name, if you click on it, it will show how many passed, how many failed and due to what issues, such as "element not found" or "object not found" or a Java error.

It also has video recording with the screenshots which is really good.

Another very good feature is the cross-platform testing. We are able to execute in multiple browsers and multiple Chrome, Safari, and Microsoft Edge versions. We have the capability in Perfecto to develop scripts. We just need to capture the capabilities of the different browsers and update them in our Quantum framework and we are able to execute. It's very easy. 

There is an option in Perfecto to capture the capabilities using the tool itself. If we want to run it in, let's say, a particular Android device, we can capture it from there and put it into the code and we can update the information and execute.

The range of open-source technologies that Perfecto supports is really good. I don't see any issue in using Perfecto for that.

In terms of its reporting and analytics, whenever I'm doing any execution for a new release, I just put in my release number and it's easy for me to look for what happened in the test case execution I ran for the previous month's release. I can easily locate it and I can get the information.

In addition, I can schedule the devices I need in Perfecto and I can block those devices so that other users on the team do not try accessing them.

What needs improvement?

The reporting can be improved due to the scrolling you need to do. We have an option for filtering and generating reports, but we are limited to three months of history. After that, we are no longer able to see the information. If they would maintain the history for a minimum of two years, that would be helpful.

Another issue is that sometimes, when we have to locate a button through physically accessing a mobile app, versus through Perfecto, we feel that Perfecto is a little slow. If they could improve on that slowness in accessing the app, when we want to click a button, that would be great because we feel the difference. An improvement in the connectivity speed is required.

For how long have I used the solution?

I have been using Perfecto since 2019.

What do I think about the scalability of the solution?

I would rate the scalability as an eight out of 10.

How are customer service and support?

I work closely with my Perfecto vendor and he is very supportive. Every time there is a new update he will share it with us and get our feedback to help enhance the tool.  They help update the core into the Quantum framework. We have had some key, latest, knowledge-sharing sessions with their team regarding these new updates.

We also have the Perfecto support portal, where we'll raise incident tickets. If, for example, a device is not working, it's not connecting, or a tool is not supporting what we're doing, we will raise a defect ticket. We can indicate the severity and wait for updates from the Perfecto customer solution team.

In terms of objects, we had some challenges initially and we told our point of contact about them. They updated the solution as a result. Also, sometimes we were not able to access iOS devices. Every time we wanted to go to iOS devices, we had to trust the app, otherwise the lab would not be launched. And we had some challenges accessing the iOS app from the Quantum framework. We worked with the Perfecto team and we got solutions to those issues.

That process is ongoing. We ask them about whatever ideas we have for them to improve. One of the ideas I asked about is because there are a lot of things we need to export into ALM Test Lab. It would be great if Perforce could create a plugin that could be added to the Perfecto portal so that we could just provide the ALM URL domain name and password to connect it to Perfecto. Right now, we need to write code to integrate it with ALM and export the results once the execution is done. An integration with Jira would also help.

How would you rate customer service and support?

Positive

Which solution did I use previously and why did I switch?

I have not used any other tool for a long time.

How was the initial setup?

I wasn't involved in the initial setup but there is some ongoing script maintenance involved.

What other advice do I have?

It's user-friendly, adaptable, extendable, and scalable. We are able to understand everything through the UI. They just need to enhance it further to adopt the kinds of changes we would like to see.

Disclosure: PeerSpot contacted the reviewer to collect the review and to validate authenticity. The reviewer was referred by the vendor, but the review is not subject to editing or approval by the vendor.
Flag as inappropriate
PeerSpot user
Bharat Malik - PeerSpot reviewer
Senior Automation Engineer at RBC
Real User
Top 20
Keeps video records of tests, supports and provides virtual devices, and the technical support is good
Pros and Cons
  • "The quality of our software has improved since we implemented this solution."
  • "The monitoring features, in particular network traffic monitoring, could be improved."

What is our primary use case?

We use Perfecto for our mobile testing; both real devices and virtual devices. We also use it in web online test cases, for testing the UI.

How has it helped my organization?

Overall, Perfecto has significantly helped us because we are not dependent on maintaining the infrastructure for various devices. Everything is maintained by Perfecto, which means that we can focus on testing.

We leverage Perfecto's reporting and analytics using the CI dashboard, reporting dashboard, and the live stream view. This functionality is quite important because it gives us the history of our past test cases and failures. From there, we can determine at which build a particular feature starts to fail.

Perfecto gives us the ability to test using virtual devices, and the testing can be done in parallel. This means that we are able to test against approximately 300 different devices.

In terms of scaling our testing operations, the option to use virtual devices is very important. If we didn't have the virtual device option then we wouldn't be able to scale as much, and our test execution time would increase.

When it comes to test creation, Perfecto supports open-source technologies such as selenium and APM. This is very good because initially, we were using most of the open-source technologies that were available. This meant that integrating with Perfecto was very smooth, as they supported what we were using.

The quality of our software has improved since we implemented this solution. Firstly, it helps that Perfecto maintains the hardware, allowing us to focus primarily on our software. Secondly, we have been able to find more defects in our applications.

What is most valuable?

The most valuable feature is the ability to record the video of every test case that we run during the automation. This means that we can go back at any time and actually see the video where the test passes or fails.

We use Perfecto's cloud-based lab for testing across multiple OS versions. The online banking team uses it to test different Chrome versions on different OS versions, which is something that really helps. It holds all of the emulators and simulators, so it's very important because we depend on those tools for our testing.

Perfecto provides same-day access when new devices are released, which is important because we sometimes make requests for these. For example, when a new iPhone is launched, it is quite important to test. This is true for all of the latest devices.

Using the device lab has definitely helped us to scale our testing operations. At this point, we run tests across 300 devices and this is possible because our test execution time has decreased significantly.

What needs improvement?

The monitoring features, in particular network traffic monitoring, could be improved.

For how long have I used the solution?

We started using Perfecto approximately 15 months ago, in March of 2021.

What do I think about the stability of the solution?

With respect to stability, I would rate it a seven out of ten.

What do I think about the scalability of the solution?

The scalability is good. We have been able to scale up with the introduction of new devices, and there are a lot of them in total.

I don't expect that our usage is going to increase.

How are customer service and support?

The Perfecto technical support is good. We have had to raise multiple tickets for some of the issues that we've faced. They have a decent turnaround time when they provide solutions. Sometimes, they were not able to provide a solution, but the response time was decent.

I would rate the technical support an eight out of ten.

How would you rate customer service and support?

Positive

Which solution did I use previously and why did I switch?

Prior to Perfecto, we were using in-house solutions such as local simulators and emulators. We did not have another provider.

We switched because we were looking for something that was more flexible than our in-house solution. We wanted a product that didn't require maintaining our infrastructure.

How was the initial setup?

I was involved in onboarding and setting up the framework. In terms of complexity, I would rate it medium. It was not very complex but not entirely straightforward. We just had to make changes in our framework and make sure that we did test executions on the cloud instead of on our local architecture.

What about the implementation team?

Our in-house team was responsible for the deployment.

The only maintenance that is required is for our real devices. In these cases, we have to make sure that the operating systems are up to date, and provide the latest ones that are in production.

The majority of the maintenance is taken care of by Perfecto. We quickly test to make sure that our scripts are not broken. This is a small team that takes care of this.

What was our ROI?

I am not directly involved in the aspects concerning costs, although if you consider the time it saves us in terms of test execution, this product has helped us.

It is correct to say that Perfecto has sped up our testing efforts. We are now about to test 300 devices in just a couple of hours with our whole regression suite.

What other advice do I have?

My advice for anybody who is evaluating this solution is to make sure that for whatever framework the teams are currently using, they are prepared to support that.

When any team is doing a POC with Perfecto, they should be able to run all of their edge test cases. Do not focus on the usual test cases but rather, what is most different. Edge-level test cases, which are difficult to execute, should be able to run on Perfecto while doing the POC.

I would rate this solution an eight out of ten.

Which deployment model are you using for this solution?

Public Cloud
Disclosure: PeerSpot contacted the reviewer to collect the review and to validate authenticity. The reviewer was referred by the vendor, but the review is not subject to editing or approval by the vendor.
Flag as inappropriate
PeerSpot user
IT Quality Control at a insurance company with 10,001+ employees
Real User
Top 10
Excellent support, good reporting, and a solid product for mobile testing and web testing
Pros and Cons
  • "The reporting feature is really tough to find in some of the other products that are competitors. Having your CITB type dashboard, where we can see the test results and see recordings of each test that passed or failed, is probably one of the distinguishing aspects of Perfecto."
  • "Its performance should be improved. Anything to speed up the user interface would be a great help. We've had a lot of pain with their migration from a product UI that was based on Adobe Flash to the new product that is based on HTML5. Migrations like that seemed to be very painful and not a real smooth process. We're still sort of recovering from that migration from old technology to new, and we haven't got all the functionality ported over that we used to have on the old UI."

What is our primary use case?

We have a cross-section of mobile devices and web devices, and we use those for browser compatibility and device compatibility testing, as well as functional testing of several apps and websites. So, we have testers that pull up the appropriate device, whether it's an Android or iPhone, and validate that all the functionality of the app or website is working as expected.

It is hosted by Perfecto. They take care of all the hosting.

How has it helped my organization?

Benefit-wise, by being able to share all the devices across 1,000 users, we get a much better cross-section of testing across a wide platform of devices versus having to buy one or two phones for each tester. To be able to share a pool of devices is a huge saving. It is also a huge benefit to be able to test on a wide variety of devices. So, it saves us from having bugs go out that would only be found on, for instance, two versions back of OS or a device that's not real common but is used by our customers. If a Google Nexus 3 is having a certain problem, we're going to find it because we have Perfecto with all different devices.

What is most valuable?

The reporting feature is really tough to find in some of the other products that are competitors. Having your CITB type dashboard, where we can see the test results and see recordings of each test that passed or failed, is probably one of the distinguishing aspects of Perfecto.

What needs improvement?

Its performance should be improved. Anything to speed up the user interface would be a great help. We've had a lot of pain with their migration from a product UI that was based on Adobe Flash to the new product that is based on HTML5. Migrations like that seemed to be very painful and not a real smooth process. We're still sort of recovering from that migration from old technology to new, and we haven't got all the functionality ported over that we used to have on the old UI.

It is sort of unfair to ask, but we don't have SSO right now. So, we're working towards that. If that had been available from day one, it would have been super awesome. Once we get to that point, it would be a whole lot better. There could be some additional administrative tools and reporting tools to get more direct hands-on reporting for usage, user database, and things like that.

For how long have I used the solution?

I have been using this solution for five years.

What do I think about the stability of the solution?

I would give it pretty high marks. It doesn't have a hundred percent uptime, but it is pretty good. It would probably rate it a four or a four and a half out of five.

What do I think about the scalability of the solution?

Its scalability is very good. We're a fully agile organization, so most of the users are testers, but we also have developers, product owners, and maybe a handful of business analysts who use it.

It is currently being used extensively. I'm guessing that if and when we renew the contract, we will expand the usage by adding more devices. We're using it a lot through automation, and our automation is increasing. We'll probably end up dedicating some of the mobile devices and web devices to automation.

How are customer service and technical support?

I commend their tech support. Their support is very efficient, very helpful, and very personalized. They just go above and beyond. I never had any complaints about their tech support. I am definitely satisfied with their support.

How was the initial setup?

I was not involved. It was set up at our company probably six years ago. I've only been here for four and a half years or so. So, I wasn't personally involved with the setup. I had been an administrator for the last four and a half years. So, from an administration standpoint, it seems fairly smooth.

There are four of us who do the admin for Perfecto. Mainly, it is a liaison with Perfecto on a routine basis. We don't do any updates to the software since it is all in their hands. We're paying for software as a service. So, our main duties are adding users, troubleshooting problems that they have on their desktop or mobile devices, and then managing which devices are going into the cradles. I and the team of three others are responsible for those admin duties. Everyone is a tester.

What other advice do I have?

I would recommend trying it. Go through the trial period. I would imagine that anyone who does so would compare it favorably to the competition. It may not be the lowest cost product out there, but you get a lot of value from the tech support and from the feature set that is out there.

I would rate it an eight out of 10. It is a great solid product for mobile testing and web testing. It has got great features, and we've utilized pretty much all of them, but it still has some growing pains, and it is a little bit slow to roll out the features that we've requested. 

Which deployment model are you using for this solution?

Private Cloud

If public cloud, private cloud, or hybrid cloud, which cloud provider do you use?

Other
Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
Automation Tester at IGM Financial
Real User
We've been able to get our testing done a lot faster, and multiple devices means manual testing is not delayed
Pros and Cons
  • "We're working in Agile and we need results ASAP. The fact that the lab provides same-day access to new devices is extremely important to us."
  • "When using devices on the cloud, it lags quite a bit at times. I know that these are real devices that are being projected on our laptop screens and monitors, but if the speed could be improved, that would be good."

What is our primary use case?

We have a website as well as a mobile app, Android and iOS. We do a lot of functional, UI-related testing. And we have automated a lot of test cases through Perfecto, and we have even started to automate mobile now.

How has it helped my organization?

Perfecto's device lab has helped us to scale our testing and it has made testing easier for us. We've been able to run our regression testing. We have hundreds of test cases, the majority of which are mobile-related, and we use Perfecto for them.

We've also been able to get our testing done a lot faster, especially during COVID when we weren't going to the office and didn't have our physical devices. We ended up using and being dependent on Perfecto a lot.

We have access to 15 or 20 devices. Even if some of them are being used by other testers, I can utilize another one. That was not stopping my testing when I was doing manual testing. That has helped us a lot

And for automation, we run our jobs at night when these devices are not being used by manual testers. Just the fact that we can automate our tests on Perfecto and run our jobs and have test results in the morning has given us a lot of flexibility and has helped a lot.

What is most valuable?

We've been able to automate test cases that are mobile-related. That's helping a lot with our automation. Obviously, you can't automate on a physical device, it's something that we do on a virtual machine. The fact that we can automate on mobile, Android and iOS, is working well for us.

We use the CI Dashboard. Perfecto's analytics are very important because that is where we have our test results, which we later input into Jira as well.

We use the device lab in our testing operations and mobile is one of the things we use. We have multiple devices, about 20, including Android and iOS. That's what we are running our automated test cases on. The device lab is extremely important because we are doing our automatic test cases and even our manual testing there. We don't have physical devices. We go into the Perfecto website and use all the mobile devices available on the cloud.

Also, we're working in Agile and we need results ASAP. The fact that the lab provides same-day access to new devices is extremely important to us.

We are mostly using Selenium and we are slowly putting our hands on Appium as well. That's working out well for us too.

What needs improvement?

When using devices on the cloud, it lags quite a bit at times. I know that these are real devices that are being projected on our laptop screens and monitors, but if the speed could be improved, that would be good. For example, if I open up a Google Pixel on my laptop, using Perfecto, just navigating on Google Pixel is quite slow. Improving on that would help us a lot.

For how long have I used the solution?

I've been doing manual testing with Perfecto for a while, but I started automation a few months ago. That's when I really got to work with Perfecto properly.

What do I think about the stability of the solution?

My main area of concern when it comes to improvement of the solution is that it lags often. If that could be improved, then it would make our testing a lot faster and more efficient. Apart from that, I'm pretty happy with Perfecto. It's going in the right direction.

What do I think about the scalability of the solution?

It's definitely scalable. For mobile testing, if Perfecto gets faster then we can utilize it in our bigger test cases. We do have some data-related test cases that really slow it down. If Perfecto can find a way to handle that, it can definitely grow with us.

How are customer service and support?

The technical support gets back to us within a few days. I have not used it that much and I don't have complaints, nor have I heard complaints. My team lead is the one who uses the technical support more often, and he's quite happy with it.

How would you rate customer service and support?

Positive

What other advice do I have?

It's a great tool for running functional, UI-related test cases. But I would give a heads-up about the speed that the mobile devices run at on our laptops or monitors, because it can slow us down a bit.

But in the bigger picture, it helps us because, obviously, even when doing manual testing, you're not going to be testing on your own personal devices. With a lot of people working from home, you cannot give cell phones to everyone to test on. Overall, it definitely helps.

Disclosure: PeerSpot contacted the reviewer to collect the review and to validate authenticity. The reviewer was referred by the vendor, but the review is not subject to editing or approval by the vendor.
Flag as inappropriate
PeerSpot user
Sameer Bakshi - PeerSpot reviewer
Test Specialist at IBM
Real User
Top 20
Helps us to scale our testing operations and test the last versions of operating systems
Pros and Cons
  • "It saves on the cost and effort of having to maintain our own virtual testing environment. Even our onshore team is not in the city that we work in, so that helps a lot. Even if we didn't invest a lot in getting multiple devices, having to share those devices would become a hassle."
  • "There was a discussion about having the capability to export the test results to a certain tool that we use in our project. If that were added it would be great not having to manually take screenshots, put them in a document, and share them on the different test management tools."

What is our primary use case?

We need to test a certain bank's application on different devices, mobile and web. We're using Perfecto to set up devices on the cloud, and that way, the whole distributed team can use it and test different devices.

How has it helped my organization?

The Perfecto device cloud has helped us to scale our testing operations. We can do much more testing than we did previously. Business has picked up, and the latest version of OSs needs to be tested. That would not have been possible if we had physical devices and had to give them out to the whole team. And that would involve a lot of cost. Having shared devices makes it much easier for the business.

What is most valuable?

People from all over the world can use it at the same time. It's not for a particular location. That's most valuable. Their cloud-based lab to test across devices, browsers, and OSs is the main part of it. People from offshore and onshore can access the devices. We can open everything on our system so we don't have to go looking for physical devices.

It saves on the cost and effort of having to maintain our own virtual testing environment. Even our onshore team is not in the city that we work in, so that helps a lot. Even if we didn't invest a lot in getting multiple devices, having to share those devices would become a hassle.

And same-day access to new devices is quite important. When OSs are updated or something new comes onto the market, upon our request, we can have new devices installed on the cloud and have access to them. We don't have any wait time and that's great.

Overall, it has helped massively to speed up our testing efforts and not just from a device point of view. Over the last two years, the performance of the Perfecto tool has picked up as well. It's much more real-time now, compared to what it was when I started.

Also, when I started using it, we did not have the web capabilities and we used to test only on mobile. We had Android and iOS, and that was great, but around 2020, they added web functionality as well, with Windows and Mac devices on the Perfecto cloud. It's great having everything together on one platform.

We use Perfecto's reporting and analytics for automation testing. Obviously, we need to see what the results are for whatever tests we perform from an automation perspective. Reporting results to our business team plays a huge role.

One other feature is the screenshots that Perfecto has. If I'm testing something and I want to take a screenshot, it's just a click away. That makes it much easier to record all the results.

What needs improvement?

There was a discussion about having the capability to export the test results to a certain tool that we use in our project. If that were added it would be great not having to manually take screenshots, put them in a document, and share them on the different test management tools.

For how long have I used the solution?

I've been using Perfecto since 2018.

What do I think about the stability of the solution?

Stability-wise, it's fine. There have been no unforeseen outages.

What do I think about the scalability of the solution?

There is a huge possibility to scale it up. It depends on our project environment if we actually need to, but scalability-wise it is quite good.

We have a huge team of around 100 people who are using it, but it's distributed across teams. It's not like everyone uses it at once. We have a certain number of devices available and we are only able to use those devices.

How are customer service and support?

Their technical support is very good. They immediately respond to our queries and try to resolve them in the best way possible. Obviously, there are times when they need more information from us, but when they have the full information, it's quite fast.

How would you rate customer service and support?

Positive

How was the initial setup?

I wasn't involved in the deployment, but there is maintenance required from time to time. That's mostly on a ticket basis. If we need something done, we raise a ticket with Perfecto support and they help us do it. It's not like we get regular maintenance notifications. It's on-demand maintenance.

What other advice do I have?

In terms of Perfecto's effect on our software quality, it's neutral. It does improve our software, but there's no direct impact from Perfecto on our quality.

My advice about Perfecto is "the earlier the better." I haven't looked into competitors of Perfecto, but from my experience, this is one of the best tools.

Disclosure: PeerSpot contacted the reviewer to collect the review and to validate authenticity. The reviewer was referred by the vendor, but the review is not subject to editing or approval by the vendor.
Flag as inappropriate
PeerSpot user
Buyer's Guide
Download our free Perfecto Report and get advice and tips from experienced pros sharing their opinions.
Updated: November 2022
Buyer's Guide
Download our free Perfecto Report and get advice and tips from experienced pros sharing their opinions.