Worksoft Certify Reviews

My current use case for Worksoft Certify is mainly for test automation with SAP.

Worksoft Certify helps in reducing regression cycle time because manual testing takes substantial time and can lead to errors. With Worksoft Certify, it will definitely help reduce resources as time and ROI will also increase.

I have used Continuous Testing Manager (CTM) to run some tests, and it is based on lights-out testing, which helps significantly with overnight execution. CTM is effective for managing the testing process because it is helpful; you don't have to sit in front of the machine and see the execution, as you can schedule the executions and see the results.

Worksoft Certify is a no-code tool that works effectively with SAP, and I appreciate all the features compared to other tools.

Teams who are planning for automation and are still in manual testing can use Worksoft Certify, and significant time and resources will be reduced.

Reporting for Worksoft Certify is also excellent; we have the BPP reports customized, and recently WRS has been implemented, so all these features are beneficial.

If any AI-powered features are included, that would be great. We've already seen the beginning of self-healing capabilities, but additional integrations with AI would be beneficial. Focusing more on AI and its integration with Worksoft would really help enhance our processes.

I have been working with Worksoft Certify for more than 12 years.

For stability, I would rate Worksoft Certify a nine out of ten.

For scalability, I would rate it as the best.

I would rate their technical support a nine out ten. I am satisfied with it.

The setup process for Worksoft Certify is quite easy; it requires just installation to use it. I have worked in many different environments, including both on-premises and cloud options such as AWS.

We purchased Worksoft Certify through a partner because we are a vendor.

I would rate the pricing for the product as reasonable. As it is a quality tool, the price is slightly higher, but that is acceptable compared to other tools, especially considering the ROI.

I can compare Worksoft Certify with Selenium, as we need to write code in Selenium but not in Worksoft Certify, and other tools such as Katalon Studio can also be compared with Worksoft Certify.

Overall, I would rate Worksoft Certify a nine out of ten.

Our main use case for Worksoft Certify is front-end automation, and we're also looking at offline testing where we can leave Worksoft Certify running while we go offline, so there's continuous testing.

I am not the day-to-day user myself, but many members of my team are.

We're getting good value from continuous testing management, and the fact that it's also codeless is valuable. It requires minimal to no coding at all, helping to ramp up the knowledge for our testers. I've not heard of any issues related to integration from our team.

Based on my colleagues' feedback, there are still some minor issues concerning continuous testing, particularly related to the timeout feature. Overall, we don't have major issues with it. We are satisfied with the functionality part.

I have been working with Worksoft Certify for a couple of years.

It's stable. I would rate it a nine out of ten for stability. 

I would rate the scalability of Worksoft Certify a nine out of ten. We are an enterprise company. It covers companies of all sizes.

Their technical support is responsive, which is good, and their solutions are timely. We just had a couple of instances that took a bit longer, but overall, we are satisfied.

Positive

The initial setup is not so complex, but it still needs some work. Overall, it's pretty easy and straightforward. Our deployment model is hybrid.

I am not certain which competitors Worksoft Certify has on the market, but Selenium is always present. Additionally, a few AI-driven automation tools are emerging.

We are not using any AI integration with Worksoft Certify yet, but I think they are coming up with AI-supported features.

I can definitely recommend it to other users. I would rate Worksoft Certify a nine out of ten.

We primarily used Worksoft Certify to automate SAP applications, and it is also well suited for web application automation as well.

Worksoft Certify has improved in many places in my organization, mainly in automating SAP applications. It helped reduce manual testing efforts and it's minimizing the chances of human error. Its well suited to automate both SAP and web applications made it highly adaptable to our needs. With its integration into our Agile and continuous testing workflows, we were able to deliver faster and with greater confidence. Overall, it has enhanced the efficiency and quality of our testing process.

SAP objects don’t usually change much, but I found the dynamic object identification really helpful for web apps as well as SAP like after upgrade when something is changes. It helps keep the scripts stable and saves time on maintenance.

From my experience, Worksoft Certify is a good tool for automating SAP, and it also works fine with web apps. But while creating or running scripts, we do face some automation abort issues, which break the flow and need rework. Also, sometimes it feels a bit slow, especially when running more scripts together. If the speed and stability can be improved, that would really help. We are using CTM for scheduling and managing runs it’s useful, but there’s still some scope to improve things like live monitoring and checking results more easily.

I have been using Worksoft Certify for three and a half years.

From what I’ve seen, it scales well. We started with fewer scripts and later added more without issues, especially with CTM handling the runs. But yeah, we need to plan properly to manage the load smoothly.

Customer service and support were good overall. I’d rate it around 80% based on my experience.

Positive

I have only experience with Worksoft Certify itself, so I'm not completely aware of other products for comparison.

Yeah, the setup and license cost is a bit high as per I known but if you’re working mostly on SAP, it’s worth it definitely. You just need to plan based on your project and how much you’ll use it.

I can perfectly recommend this product to others because Worksoft Certify is very user-friendly and we can easily automate the test cases. I would be interested in being a reference for Worksoft. The vendor can contact me if they have any questions about my review. I rate Worksoft Certify eight to nine out of ten overall.

It serves as our primary automation tool, specifically utilized for testing purposes. The tool predominantly finds application in functional testing and integration testing for our customers.

The tool itself is highly effective, especially when it comes to comprehensibility for newcomers. Even during the initial learning phase, I found it remarkably user-friendly. It facilitates quick onboarding and training of new resources. It offers features for building automation scripts, such as search and certify capture. A notable advantage is that it doesn't rely on other tools like Micro Focus UFT or Micro Focus ALM for script execution. Worksoft can run independently without the need for support from Microsoft Office, although the option to use it for maintaining data exists. The tool also includes debugging features and comprehensive reporting capabilities, generating PDF reports for easy analysis.

The primary area for improvement is the support service. The support process can be time-consuming, with tickets sometimes remaining unresolved even after considerable follow-up. Also, it doesn't offer a training or trial version for users to explore the tool before committing. Enhancing direct connectivity between radio and Worksoft Certify would streamline and improve efficiency. Although there are YouTube videos for specific features, the Worksoft community lacks corresponding content. This has been a common concern among users who find information on YouTube but struggle to locate it on their community portal.

I have been working with it for over eight years now.

I would rate its stability capabilities eight out of ten. Stability concerns are significant in automation, especially when dealing with new applications or features. Challenges arise in assessing the feasibility of automation without adequate support.

I would rate the scalability ten out of ten. At the moment, we are involved in four distinct projects out of the overall portfolio, which consists of around thirty projects. The focus is primarily on high-scale business initiatives.

While having a partner with dedicated support can help in critical situations, as a customer, the overall support experience might not always meet expectations. It often involves multiple calls and explanations to various teams across different locations, making it a bit cumbersome. I would rate it six out of ten.

Neutral

The initial setup was straightforward. I would rate it nine out of ten.

The deployment typically takes one to two weeks. The first step involves setting up the database, determining the server specifications, and installing the servers based on the specific requirements and data activities of the project.

While I acknowledge that it's a good tool, the challenges with support are notable. Despite the tool's excellence and the introduction of new features, the ground-level implementation should align with the promised benefits. In comparison to other tools in the market, some offer superior support and features. However, it's worth noting that it remains one of the best tools for learning, especially for users who might not have many alternatives. Overall, I would rate it ten out of ten.

Worksoft Certify is an automation tool used for functional testing.

Worksoft Certify Mostly is an easy-to-learn tool and does not require any extensive training. It is easy to record and play this tool. Anybody can learn and start working on the solution. As per industry standards, Worksoft Certify is the first choice for SAP applications. It has a lot of built-in objects and repositories for SAP applications.

The solution's support could be improved. Worksoft Certify's support team should respond more promptly when we are stuck with certain issues and looking for a solution. They have been helpful so far, but quick conversations can happen if they function within the timelines.

Worksoft Certify has already come up with a new tool, a continuous testing manager for lights-out testing. It's good that they are working in those areas to develop new tools to support the lights-out testing. Worksoft Certify should also try to develop some solutions that can be more beneficial for non-SAP applications, like web-based automation or Excel-based automation. Worksoft Certify can be made more robust in those areas.

I have been using Worksoft Certify for four years.

I rate Worksoft Certify a seven out of ten for stability.

Worksoft Certify is a scalable solution. I rate Worksoft Certify a seven out of ten for scalability.

Worksoft Certify is priced higher than most tools. I rate Worksoft Certify an eight out of ten for pricing.

Different solutions are available per the client's needs, but I still recommend Worksoft Certify as the number one or number two choice for SAP-based applications. For EPC and SAP, Worksoft Certify would be the first go-to choice.

Overall, I rate Worksoft Certify an eight out of ten.

In my company, the solution is used for SAP automation.

The technical support of the product is an area of concern where certain improvements are required.

I have been using Worksoft Certify for more than three years.

I am satisfied with the tool's stability. There have been no issues with the product's stability.

Stability-wise, I rate the solution a seven to eight out of ten.

The tool's scalability features did meet my company's goals. Scalability-wise, I rate the solution a seven to eight out of ten.

The solution's technical support was pretty good. There is a ticketing system and a call system to contact the product's support team.

I rate the technical support a six to seven out of ten.

Neutral

For automation purposes, my company works with multiple open-source tools, like Selenium and Java, along with a few license-based tools, like Worksoft Certify, QuickTest Professional, HP ALM, LoadRunner, IBM Rational Performance Tester, and OpenText Silk Test.

The solution is deployed on an on-premises model.

There is a separate team in my company who has taken care of the product's deployment phase.

I chose the tool for my company since it suits the purpose for which I use it.

A specific feature that I found to be the most valuable in the solution for our company's work processes stems from the fact that it is useful as a low-code automation tool.

The tool is good for dealing with end-to-end testing scenarios in our company, and in my organization, we use it for SAP testing.

The tool is useful for functional testing, and it did add some value to my company.

The tool is useful to automate business workflows. My company used the product to automate the test cases for whichever systems were under testing phases.

I used the low code automation functionalities provided by the product. I did not use the scriptless test automation feature of the tool.

I rate the overall tool an eight out of ten.

We use this solution mainly for SAP automation and for some web applications and we also use the reporting feature. I'm a quality assurance project manager.

I would recommend Worksoft mainly for SAP. It has all the in-built functionality and Worksoft is a wonderful tool. They've introduced the hook feature in this latest version and it's great. In general, the latest version is significantly better than the earlier ones. 

I would like the Worksoft web application to be more robust. For example, SRM has different modules in SAP. When we automate some of those objects in the web app, it's initially fine, but after a couple of executions, we need to again record that particular object or get Worksoft to identify that particular object again. It works well with the SAP R/3 or GUI but there are limitations with the web application and Worksoft needs to improve that area. In terms of additional features, we'd like to see the default report which we generally get after the execution is completed. There should be an option to customize reports according to our requirements. Tosca provides that feature but it's unavailable in Worksoft.

I've been using this solution for five years. 

The solution is stable. When we're undergoing upgrades or version changes, some maintenance is required but it's quite easy. 

The solution is scalable. In my project, we have a team of around 100 people, 20 of whom are using Worksoft. We have a dedicated automation team. To date, we have automated more than 3,000 test cases using Worksoft.

Worksoft provides good technical support. They've helped us resolve issues numerous times. It's wonderful. 

Positive

We have introduced other tools like Tosca which we use as a codeless tool because it wasn't possible for us to automate through Worksoft. Where there are web application limitations in Worksoft, we switch to Tosca.

Because we have a dedicated team that takes care of deployment, it's a smooth process for us. 

I believe licensing costs are based on the number of users and licensing is on an annual basis. 

Worksoft is a good tool to start with as it is a codeless tool. There's no requirement for coding language knowledge. 

We rate this solution eight out of 10. 

Initially we were automating the regression suite for SAP ECC.

From there we moved into a web application called HVAC Partners, which is something that we've developed that is a type of customer portal. That application also connects to SAP, but it does some other things that don’t necessarily connect to SAP. It is a kind of front end for quotes and sales orders that go into SAP, but it's also reporting status of orders and status of warranty claims and the like for the customers.

From there we moved into the Middle East SAP ECC instance and automated their regression suite and, from there we rolled out S/4HANA for our service business. With S/4, SAP releases updates every quarter. Because the S/4HANA instance is in the public cloud we have two weeks, essentially, to regression test and test any new functionality. We started with the last release and we did about 70 percent of the testing with Worksoft. We also used the S/4 automation tool, which is more for unit testing so it's not as valuable as Worksoft. We're wrapping up that automation in about the next month and we'll be moving on to a European rollout of S/4. We'll just start working our way across Europe and those implementations.

We have it on a virtual server and we're using remote desktop access for the offshore automation engineers to access it.

We're using it with Fiori and it's working fine. We have integration in and out of S/4 to Salesforce.com so we also automated those. The test cases were end-to-end. We start in Salesforce, which is a web application, with, for example, a quote, and then it goes into S/4 and gets reviewed and approved. It then goes back to Salesforce with the approval and a sales order is entered that ends up going back to S/4. And then there's fulfillment, back and forth, and eventually billing and collections. We were able to do that whole automation with Worksoft, plugging into Salesforce as well as integrating to S/4 and doing the S/4 automation, back and forth. It's been incredibly useful. We saved something like 80 percent of the time it would have taken to manually test, using this tool.

In terms of using the Capture feature without knowledge of testing tools, we brought on some new support people. One of them is our web support person and she had no background in Worksoft. She's been using it to do all the initial captures for our HVAC Partners. She's been able to use it very easily. Our more experienced automation engineers will follow up, after she's done the Capture piece, and troubleshoot some of the stuff that she might not understand yet. They're working with her so that she does learn it. But she's been able to use it very easily.

Worksoft’s ability to build tests and reuse them is very good. We ended up obsoleting the tests and not using them with the other tool we used, whereas now, we rerun these, at a minimum, every month. We do that for a few reasons. One reason is to keep the health of the tests up. Suppose a material is obsoleted. The test that has that material in it is going to fail because it's going to say, "Material not found." Or suppose a customer is no longer a customer and he has been blocked or archived. We run the tests to make sure that the scripts don't need any changes. We also use them in case a process has changed. We're releasing changes to SAP about every two weeks: support tickets, enhancements, maintenance, etc. If a business process changes, then the automated test needs to change to reflect that change. Running them every month, at a minimum, helps make sure that everything is healthy.

The other reason is to identify anything in our quality system that could unintentionally impact other things that the programmers didn't realize. We've caught a couple of those in queue and they said, "Okay. I didn't mean to do that. I only meant to change this one thing,” but it changed all kinds of things and we were able to catch that before it went into production. So the reusability is fabulous if you create the tests properly: no hard-coding, and you’re using data tables to hold any of your field selections, and you're using good automation standards, so you create and consume your data. If you create it and consume it, when you rerun it, it does the whole thing again. You don't have to worry about finding a sales order that works, for example. You really have to create a logical test design to make it reusable but as long as you do that, it's very reusable.

It dramatically reduces the time we spend on testing. Before we started using this tool, everyone was pretty much doing testing manually and test events were taking from two to six weeks. What they did in two to six weeks, depending on the scope of what they were doing and how many people they had involved, we can usually do in one to two days.

The most dramatic was when we finished the Middle East automation. They were bringing up another company code and they wanted us to run regression testing on all of their current company codes, about seven of them. We completed it in about four days. The IT director came to us and said that it reduced their labor by 93%. “Quite frankly,” he said, "we would never have been able to do all of that testing. We would have had to engage a minimum of 28 people, and it would've taken them a minimum of eight weeks, and we still would not have been able to do all of the tests. We wouldn't have gotten them done." We were able to do it in a fraction of the time and with a broader scope than they would've been able to do. They would've done as much as they could and then they would have gone live and hoped for the best.

And we've also been able to use it for other things like certain recurring tasks that had been done manually. We had people who were manually monitoring Tidal jobs, which are batch jobs that have been scheduled to run. If a Tidal job fails, somebody has to go in and figure out why it failed and either restart it or fix it, and then rerun it. These are jobs like billing jobs and we automated them. They probably spend 15 minutes a week on billing jobs now, whereas we had somebody doing this about 12 hours a week. And then that person would have to send out an email to whomever the relevant person was saying, "Hey, check your batch job. This isn't running." They now spend about 15 minutes running it. It saves the emails to the users, documents the results in a spreadsheet, and puts it out to a SharePoint where the auditors can pull them any time they want. It was the same thing with monitoring the claims jobs. We've done a few things like that which have added to the value.

Automation using Certify has also saved testing time, big-time. As I said, the Middle East: 93 percent. For the S/4HANA project, what we did in three or four days, they had been taking two weeks and not getting through at all. With the release, you don't get to say to SAP, "Hey, testing is running behind, we need another week," because it's in the public cloud. Like it or not, they're going live. The drill is supposed to be: You test during week one and you remediate in week two and you go live that weekend. We got our stuff done, 70 percent of the work, in about three days, and it was our first time, out-of-the-gate, so it'll go easier with the next release. The rest of the team took the entire two weeks to do their 30 percent. And within the 30 percent they were doing, a lot of them were smaller tests. We were doing end-to-end tests that go through Salesforce and S/4, etc.

In terms of defects, the value is finding the defects prior to moving something into production. There are two I'm thinking of that we found in Mexico. One of them would've brought shipping to a halt and the other one would have brought receiving to a halt. If you shut down factories, even for a short period of time, there is this domino effect. The value of those finds is huge. And this wasn't even something that the guys making this change were testing. They were testing the piece that they changed, which was working. What they didn't realize is that they changed all items instead of just that subset. It was a minor goof in the programming. It was just too broad of a statement. 

I started in IT about nine years ago and we did total manual testing. We would have defects in the high hundreds to 1,000 during the implementation testing. Now, we're probably under 100, so it's much lower. It could be that we're just getting better at implementations.

It's pretty seamless with SAP and Salesforce because they've built in the field definitions and all the things that you need. You literally turn it on and execute your script and it records it. It's very simple. Then you can go back and put in some of the other functions. For example, instead of hard-coding field selections, you put in a data table so you can run it multiple times or with multiple data. It was actually written to work very well with SAP.

Salesforce came a little later. Obviously, big companies like ours don't just use SAP. We have integrations with Salesforce and CRM and CPQ and all these other programs that integrate with SAP. Worksoft started looking at its customer-base and saying, "Okay, what are the popular ones you guys use?" Salesforce was one that came up. So one of their releases, about a year or so ago, included the ability to record in Salesforce like it has for SAP, so it's super-easy.

We’ve used the Capture not only to train people on how to do things, but also to provide the output to our users so they can validate that what we tested was proper. Capture is very good. It is lengthy, though, because it documents every keystroke that you do. At the beginning it will list all the field selections that we use and then it will give you each step: what it is, pass or fail. If we put in a screenshot, that shows up. It's up to you as to whether you put in screenshots or not. A lot of the times the documentation we provide as a PDF is lengthy but it's also very thorough, which is good.

Certify provides codeless, end-to-end process automation across packaged applications. It works well with SAP and Salesforce, for example. Another one that they have done all the definitions for is Oracle. We haven't started on our Oracle ERPs yet, but it's good to know that we can. We just don't have automaters who know Oracle, so we're sticking to what we know right now. The process automation makes for relatively fast automation compared to the other tool that we've tried to use. It makes it so much easier because you don't need any technical programming knowledge. A lot of the other ones are Java-based or based on other tech languages. That's a skillset that the average tester or support person does not have. It makes it very easy for those guys and the learning is quicker too, because the troubleshooting is easier. You look at the code, you can read what it's doing. You understand the business process and you say, "Okay, that's failing because we failed to set this flag, or fill out this field." It’s pretty simple.

I would like the ability to more easily modify the report from the Capture feature. One of the things I don't like is that it keeps repeating all the field selections throughout. To me, if we put them up front, we shouldn't have to repeat them at the different steps. It should just be Pass/Fail and show the screenshot. I've talked to them about this in the past.

There's another part of the Worksoft suite that probably does a better job at documentation for training purposes and providing an understanding of business process. It's the Certify BPP which we're not using right now because we're really focusing on automating all these different ERP systems. Whereas the testing is very detailed, which is great for the auditors and it's great for the users because they see everything we're doing, it makes for some big PDFs. It's a double-edged sword.

Also, with the codeless process automation across packaged applications, once in a while, if we get a weird application that's not widely used, it gets a little stickier. First, the software has to learn the fields, so you have to identify all the fields. Once you do that, as long as there isn’t any non-standard code in the application, then it works fine. But there's that one step that you have to do, a step you don't have to do with SAP and Salesforce, for example.

In addition, Worksoft definitely needs to continue the march toward bringing in more and more of the software that people commonly use. They're doing that, but they can only march so fast.

I know Worksoft is doing some stuff with RPA. There are other tools that strictly do RPA, but aren't automated testing so I'm not sure if they will be able to compete with those. I know that we did do some automation, what we call "bots," with Worksoft, and it was clunkier than some of the RPA tools that are currently on the market. I suspect that they'll come up with a very competitive offering. 

I would also like to see some better reporting of testing status, reporting that we can easily generate to say "Okay, we're 50 percent done and we've got 10 fails and 800 passes." That's what test management software is for and Certify integrates with that. Bang-for-buck, it's probably not a great place for Worksoft to invest. They're probably better off with RPA and bringing on the ability to more easily test software, like Salesforce and CPQ. I'd love to be able to do that as easily as I can with SAP. I would like that same ability to use Capture in CPQ, instead of using Silverlight.

We started using Worksoft Certify in 2016, so it's been about four years.

Stability is very good. They do a lot of releases. They are probably using an Agile methodology, so every time I turn around, they have three more releases out there. It would be helpful if they could release once or twice a year, but I understand why they are doing it. They are adding new features because they want to get them out as quickly as they can. I just don't have time to stop, do an upgrade, and move on.

We haven't had a problem yet with the solution. It's been very good, and you don't have to upgrade every time they do a release. We do it probably once a year.

Scalability is fabulous.

We've certainly taken on more projects. When I first started about nine years ago, there was one major implementation at a time. At the moment we have about six major projects going on and, with the unwinding due to the spinoff, there are probably about 50, but those are not being tested with automated software. We're focusing on just the two SAP ERPs, S/4, and the ancillary web apps. It does allow them to implement faster. Since we did the Middle East, they've brought up two new companies in six months, which is amazing for them. It probably would have been one at a time over a year and a half or two years, otherwise.

We don't use Certify to create RPA at this point. We have so many ERPs to automate that we're sticking to that right now. We're trying to get to where we can pick up more licenses and build up the team so we can start doing some of these other things. Right now, with the spin-off from our parent, everybody is hyper-focused on unwinding. When you're part of a big organization like we were — we're still pretty big but we were huge, Fortune 50 — and you start unwinding things, there are so many shared services and servers that are on their domain, etc. It's going to take us two to three years to unwind all that. So we're marching ahead on our ERPs and I'm keeping my head down. I have my seven licenses, although I want to get about 10 more, but I'm not going to raise my hand until we get unwound.

Most of our issues have been our own internal infrastructure issues. We have a very tightly controlled infrastructure, so I'm always banging up against that. Worksoft has been able to help us solve these problems, and they're not even their problems.

It's far easier than other solutions. We previously had HP Quality Center and we could not maintain it. Prior to my taking over testing, they had implemented that tool. They brought in some outside contractors who did the initial automation and they handed it off to the support team to maintain. But it was so complex to update it when there was an error, or just for general maintenance that needed to be done, that they found it easier to just manually test. They quit using the tool. It was a complete waste.

With Worksoft, in stark contrast, there was a little bit of a learning curve up front because for about 70 percent of your effort you can use its record function that just records your keystrokes. But then you have to go in and harden the script, and put in data tables and screenshots and validations, that type of stuff. But compared to the other tool, there are no real programming skills needed. You learn how to use the functions and when you look at the script or the test, it's not like looking at code. You can actually read it and say, "Oh okay, that's inputting the month and the year," or "That's validating that the sales order posted." It's in English and it's very clear to follow. There's a drag-and-drop, and delete and all the things that you're used to using with other applications, like Word and Excel, that makes it very simple to use. Initially we had a little bit of training involved, but since then it has been incredibly easy compared to the old tool. The old tool didn't make it past a couple of years. It's been four years with Worksoft and we've got interest, globally, from other parts of the company that are asking, "When are you going to automate our regression suite?" So it has been very well received.

Setting it up was pretty straightforward. My biggest frustration was with our infrastructure. We set it up as a remote desktop but our company has all these firewalls and restrictions around access, and my team is mostly offshore contractors.

The offshore contractors have different access than I do. I spent a lot of time whitelisting different web sites to give them the access to the software we are testing.

Deployment took about a month and a half, mostly due to the infrastructure problems. However, now, when we need to upgrade the system, we can pull it down and run the installation. Then, we always get on a call with Worksoft, because if we miss one step and it doesn't work, we can't afford to have the team down. So we get on a call and spend about an hour running through the update.

Worksoft was fabulous help with the setup. They would get on a call anytime. They would help us walk through issues and help us figure them out; even how to navigate our systems. Their assistance during the setup was phenomenal.

We have seen ROI but it's very hard to capture because a lot of the benefits are hard to monetize. We have seen a huge reduction in the time to test and a huge reduction in the number of people needed to test. Rather than lay off a bunch of people, we've chosen to do more projects, so our rate of implementations has gone up. 

The 93 percent reduction in labor that the Middle East calculated was pretty impressive. I would say that, on average, it would be more like a 70 percent reduction in test time, because you still have to have people review the tests to make sure they're comfortable. Even though we say everything passed, they're going to want to review them. And then there's the retesting of any remediation that needs to be done.

The initial investment is probably a little high. It was a little hard for me to sell, but it was a one-shot deal and that's why it's so high. All we are doing now is paying annual maintenance, which we don't have to do if we don't want upgrades, but we do.

It is based on the number of licenses. If we had bought a larger number of licenses, our costs would have come down significantly, which is fair. I did struggle a little bit trying to sell it because our company had already had one failure with a testing solution, and here I was asking for money to try again. However, since we got it in, we have had great success.

We have seven licenses today. The people using it are three automation engineers/quality assurance testers who do SAP ECC. We also have three who do web application testing. They are the ones creating the automation for our portals, e.g., customer portal. I have one test lead who oversees this team and bounces between both SAP and web testing. We haven't bought a whole lot of licenses and haven't rolled it out to a massive number of users. We're doing all the work ourselves.

Since these are concurrent licenses, we could double the number of users with our current licenses because six out of the seven are offshore. While we are sleeping, they're using them.

The solutions we evaluated were all Java-based and they all took skills that we didn't have and we would have had to hire people to use them, or we would have had to train people. The people those solutions might be good for would be developers, but I'm not going to get a budget for a bunch of developers on a test team. And developers don't want to test; developers want to develop. I wouldn't even be able to hang onto these people. That's what failed with our initial attempt. We brought in programmers, they came up with a test, and nobody could maintain them afterwards. It was an investment that we threw out.

There are a number of lessons I have learned from using Certify.

• When you get started with it, you need to make sure that you have an executive sponsor so that you get the cooperation you need.
• Pick up some mentoring services from Worksoft to help you get started.
• You need to document your test cases well. Don't just start without good documentation, because then you make mistakes and then you have to rework that particular test script.
• Be very organized in the naming conventions and the standards you're using to do the automation. For example, don't shortcut. Fill out the fields that explain what the test objective is. That way, when somebody else comes in a year later and they ask, "What does this test do?" it's right there. Be organized.
• Try not to do too much with a single test. We wrote some that were crazy long: 500 to 600 steps because our process was a very complicated process. Step back and think in terms of logical chunks, because a script which is that big is difficult to maintain. You fix one thing and you get 20 percent of the way through and something fails. So you fix that and then you get another 20 percent and something else fails. It will take somebody half a day to fix one script. You can't have that delay when you have 500 that you're maintaining.

I would put Worksoft Certify right up there at a 10 out of 10. It's been the easiest package that we've done. The S/4HANA tool that comes pre-written, where we just go in and change our data to make it applicable to us, is pretty simple but it's not flexible enough. You can only test S/4HANA within those four walls and almost nobody uses just S/4HANA. There are always integrations. So Certify, as a tool that works across integrations from one package to another, documents the results, is easy to maintain, and easy to use, is a 10. I have not seen a package that is this easy and we did look at other ones. This one was just head-and-shoulders above them. It's really a fabulous product, I'm so impressed with it.

Our use cases are mainly from different domains for mobility solutions. It is also for HR and eCommerce.

We implemented more than 200 test cases for critically defined processes. 

It helps us understand the end-to-end scenario and how it can be divided into multiple subprocesses. Those sub processes can be used whenever required. Based on that, we design automation scripts, then reuse them wherever required.

The solution has enabled us to find defects since it is a test automation tool. However, irrespective of the tool used, we are testing to find defects. Thus, it has found defects in our projects.

Certify is a codeless automation tool with scripts that are easy to maintain. Since it doesn't use any coding language, a person (who is unaware of any language) can directly deploy just by training him/her on how the tool works. So, if a person is well aware of at least basic programming knowledge, it can then be deployed by providing basic training.

For SAP, we have many use cases. Certify is helping us to automate things, which is really awesome. The time spent to automate is less when compared with other tools.

Initially, there were challenges because there is a concept called XF definitions, where each application type, e.g., the vendor and workflow, has to provide these XF definitions. Last year, these XF definitions were not provided. However, whenever we face any issues, we have to raise a support case, then they update the XF definitions in our enrollment. This year, they improved that and have the last XF definitions for SAP Fiori updated as of February 2021.

For integration, projects are usually agile. The customers are looking for integration with CI/CD tools, like Jenkins, Jira, Xray, Zephyr, etc. There is no clear documentation on how to integrate Certify with these tools. Also, we didn't receive the required support when needed. Worksoft used to have webinars on this, but those webinars used to be on after the integration was established and how the integration works, not on how to create an integration.

They should come up with a solution on how to do the integration. Jenkins, Jira, and Certify should be the same in every company. The only thing required is the pipeline code required to integrate Jenkins, Jira, and Certify if they make it available for everyone. Also, if an expert team could help customers to integrate, then that would really help our customers a lot. 

As part of our weekly regression, we wanted to use Execution Manager. However, from 2017 until March 2021, Execution Manager was not working as expected in our enrollment. It could have been better. If Execution Manager had worked well, then we could have doubled our productivity. Unfortunately, it had problems. 

We have been using the solution since May 2017.

The stability of the solution depends on how good the Configuration of your environment is. We haven't faced any issues w.r.t the stability and performance. So we can say the solution works good. 

We have increased the number of tests that we run based on customer requirements.

Sometimes, the support engineers will be unaware of the solution for which a support case has been raised. Their competency or knowledge on the tool could be improved. When calling them, they will see the problem but need a senior engineer who will not be available. We will then be forced to have a second meeting. Waiting for meetings and solutions can be really hard for us because we can't deliver to our customers.

But since last some moths, Customer Support has improved as they conduct bi-weekly follow-up meetings to address our pain areas and expedite the solution for the Support cases which are open.

Neutral

Compared to our previous solution, Worksoft Certify reduces our test maintenance work by 50%.

The initial setup was straightforward.

It took us a lot of effort and nerves to get it really running. I would also expect for the major CI/CD pipeline tools that these are somehow handled and covered.

The last upgrade of Certify happened in the month of March. That upgrade's time was around 30 minutes to one hour. There were also preparations needed because of internal policies for taking care of backups, which were a bit time-consuming.

There are separate server and database teams who have their own SLAs for deployment. It all depends on different project specifications. 

We have seen ROI from using Worksoft Certify. For example, in the mobility sector, we are frequently finding issues or defects that might have come into production if not properly tested. Some of them might cost some $10,000 if we had waited to solve them after they went to production.

Automation using Worksoft Certify has saved our testing times by 40% to 50%.

We got subscription model licenses three or four weeks ago. Now, we are setting up enrollment for other products of Worksoft. It is recommended that we should do our migration and upgrades in a test enrollment environment first, then move to a production environment. 

In general, they changed their license model. Before you had to buy licenses for each component, and now they changed it so you can buy a license and use it for nearly all their applications.

We also evaluated Tricentis Tosca. Their pricing includes all their training. This is free of cost to everyone, even users without a Tosca license. If Worksoft came up with something similar, I think that they would get more customers.

We haven't bought any projects or use cases to automate Salesforce.

Worksoft Process Capture is a tool that captures the user actions. Usually, the functional consultant knows the functionality that has to be automated. We then have to install the feature on his machine for him to use Process Capture. The challenge is, since they will be unaware of the tool, we first need to educate him how to use the tool. Because of this challenge, we aren't using Process Capture right now.

I would rate this solution as eight out of 10. We are very satisfied with the solution.

Our primary use case is for SAP regression testing. We wanted to have our all SAP processes for regression to be optimized using Worksoft Certify. It took around one year for us to take that process and learn how it works, gradually improving it until now. 

We do regression testing on a weekly basis using Worksoft Certify. This does not include just SAP. We are also using the web-based tools when we are integrating end-to-end processes. When we are moving within one end-to-end process, we are shuffling between all the scripts and then going back to SAP. It works quite well.

At some point, we will have an RPA initiative to use this solution. For example, we need to obtain data from our development environment to check reports and verify them, then create transactions using Worksoft Certify.

We have more than 160 end-to-end processes. In our repository, we can have one transaction process or more than 40. There are big end-to-end processes and there are smaller ones. There are processes that can take hours to run and there are processes that can take a couple of minutes to run, but most of the end-to-end processes have multiple transactions. In weekly/monthly releases, we don't have any type of manual testing. We are just relying on Worksoft Certify for its test automation.

We also use Worksoft Certify for test automation in projects.

For scheduling, we are using an Execution Manager from Worksoft Certify.

Our business and IT users have been able to use the Capture feature, even without knowledge of testing tools. We have different IT people from each business area, like production planning or material managements. These people, without even having any knowledge of programming or Certify, just need to know how to use Capture. They are using it because it's really easy to use. You just have to initiate a Capture and do the same work that you are doing every day. Then, Capture will record everything.

You don't need to do any coding when creating end-to-end processes; it's a codeless because it is just recording. We don't need a specific person (or multiple persons) to work with the processes to do coding for the automation, which saves time. It increases the confidence of the process owner or IT person working on it because we provide them the capability to record their own process without doing any coding and just using Capture. They are confident that whatever they have recorded is their own process. It's not someone else recording their processes. The test automator can polish that process, but we are not recording it since they initiated the Capture. Overall, this saves time.

It has reduced our test maintenance time by more than 50 percent because we don't have to do manual test processes. We have saved over 150 man-hours monthly. It has increased our delivery times. We went from 200 man-hours down (three weeks work time frame) to approximately 40 man-hours (three days work time frame).

Using Worksoft Certify, we have found and repaired more bugs, though not that many. Users cannot manipulate information during the process. We schedule our tests at the night time and leave them working. Next morning, when we come into the office, we just check our system. These tests are pass and fail. For the failed test, we have screen captures where we can go and check the screenshot to know what the issue is. No one has to sit in front of the screen when you are running regression tests or scheduling them. I can schedule them and continue with my other daily routine work.

It is really good if you are using Capture in the new version of 11, where you can have a visual of the capture. The new version is much better. If I am using the Capture, I just initiate the capture and start working normally with my transactions. Capture will start recording all the steps that I'm working with. Before sending that one, I can even edit the capture. I can add more steps or remove them. This is one of the best features that Worksoft Certify has.

Worksoft Certify is really easy to use. You don't need any programming skills. If you just know a bit about programming, then you can easily use it because most things record by themselves. 

It's fully capable to automate testing for packaged applications, like SAP and Salesforce. You can do whatever you want and even create one specific process. For this process, you can create multiple processes. For the one transactional process that you have automated, you can utilize that process in multiple other processes too. 

They have different tools for different types of documentation, like Certify and Analyzer (for business processes). In Certify, from the repository, you can get really good, detailed documentation of what has been recorded, how it works, what steps have to be taken, and what data to use for every step. From a documentation point of view, it is valuable and can be used for future training.

Worksoft Certify's ability to build tests is really easy. You can build as many tests as you want with Capture. It is very reusable to apply one process to multiple processes. 

The scripting methodology is easy to learn. It is easy to maintain because it is presented in a simple, narrative way. You don't need to know programming.

When you are using an older version of Worksoft Certify, like version 10, Capture will be running under your status bar. Then, you don't know what transactions you are currently recording. In the old version, you don't have the visual of the capture.

It works well with other tools, but there is some integration required with Solution Manager. If you are using test repository in your solution manager, then there is no direct integration between Solution Manager and Worksoft Certify. 

There is an improvement needed in the reporting within Worksoft Certify. We have either a detailed report or a summary report. We don't have report that can be used for training purposes. A different tool from Worksoft has to be utilized for this.

Worksoft Certify needs a bit of improvement for its web-based processes. It can be difficult because you need to recall the maps, then you still have to add-on for your browser. When you are using the browser-based testing, you cannot even move your mouse or do anything on your system when you are using the web-based testing. Therefore, it needs a bit of improvement on that side. While it does work, it needs improvement. From the SAP side, there is nothing better than Worksoft Certify. However, from the web-based, we are moving towards Fiori. SAP will soon be totally web-based. For Fiori, they need to be great with SAP testing. Thus, Worksoft has to improve the web-based testing part for Certify.

They have a lot of versions coming out every year, like four or five versions. They need to reduce this number. There should be one or two versions every year with add-ons, if necessary. Because if you want to upgrade your Worksoft Certify from an older version to a newer version, you almost need to reset the solution and we don't have that much time. We cannot dedicate four times a year to having the newest version of Worksoft Certify. Though, if I don't need the need changes or improvements, I can skip the most recent upgrade until the next version comes out.

Almost six years. We started launching it in 2014.

It is very stable; nothing happens.

Two people are responsible for maintenance, deployments, and updates: the test automation administrator and the test automator. One is mainly responsible for all the administration and maintenance of the scripts and everything that goes into the integration with SAP Solution Manager. The other guy mostly works with the execution of the script. 

The solution enabled us to scale up testing because we don't need to use manually testing. We are also using it for project testing. 

There are overall 12 to 13 users. We have three users identified for test automation. We have another 10 users who use different packages from models of SAP. They use it to capture their processes.

If you have any issues, cases, or discussion, you can go on the Community portal and ask from there. They have people and technical experts there who can provide you with solutions. It's one of the best things that Worksoft Certify is doing. You can discuss with others at different companies, or possibly the technical experts from Worksoft Certify if we have an issue. Then, we can open a case directly there. 

I would rate the technical support as a nine out of 10. They are very good and available at all the times. The technical support can assist through the Community portal. You can also email them directly. Their response time to contact you back is 24 hours.

Before Worksoft Certify, we needed almost two weeks or more of regression testing to move our changes from regression to production. We were spending more than 10,000 euros for that on top of our man-hours. Even then, it was not proper testing because the user could manipulate things because they were using manual testing. For example, if something was not working, they could try to use some other way to work it. After the implementation of Worksoft Certify, we have increased our scope for regression and have more than 162 processes from all the models. We have also increased the test coverage and have less issues.

We went from two weeks and 200 man-hours to three days of testing. Within three days, we can test for the monthly release. For the weekly release, we need maybe one day or less using Worksoft Certify to test our releases and move them to production. It has really improved the collaboration between teams and our dependencies between various processes. On top of that, it provides really quick reports and results for management. We are capable of providing quicker reports and results for regression, like, "What has happened? What has gone wrong? Is everything okay to move to production?" Management is also very happy with it because it is saving time and money.

I was not involved in the process.

We have seen ROI with its efficient reporting and ability to provide results to management. It has also increased collaboration between different teams, such as, purchasing and sales.

We have used Worksoft Certify as an RPA tool when we have needed to create the master data or upload some documents. This saves a lot of time. It can reduce 10,000 transactions from weeks down to a day and a half.

Purchasing and licensing are okay. Go for the perpetual licenses. In that way, you own a license, then you can purchase maintenance and support on top of that, so you don't have to pay every year for it. Even if you don't want it a contract with Worksoft Certify in the future, you will have your own license of it. Then, if your usage is not that much, you can have one or two perpetual licenses. However, if you want to run your processes, you will need more licenses, e.g., using the run-only licenses. They are really cheap compared to the full licensing.

There are multiple licenses available from Worksoft depending on your requirements. You can take any one of them, and they are really good. For example, if we are using five to 10 perpetual licenses where we need to update or create scripts and run parallelly, simultaneously multiple scripts with just one license, it will take months for us to run 162 processes. Therefore, we have run-only licenses that we use when it comes to executing our test scripts on releases.

It is integrated with SAP. It's one of the best tool to be used if you want to automize your testing for SAP. No other tool is as good as Worksoft Certify.

I have evaluated other tools, but I find Certify really tops them. Most of them are using the command line-based tools. With some of them, you need a expert who knows how to do programming. For Worksoft Certify, you don't need programming skills. For the SAP GUI, I haven't found another tool that can work as efficiently with SAP as Worksoft Certify does.

You just have to be clear about your processes and define them before going to capture one. If you capture some processes, you can just easily manipulate and merge them to the other processes. It is really easy to use Certify, but you do need to be focused. There is also integration with Execution Manager, which can be useful when you are using Worksoft Certify for scheduling and other things.

It is not that easy to work with SAP Fiori as compared to SAP's GUI.

It can be used with non-SAP platforms. 

Overall, the solution is a nine out of 10. For just the SAP GUI, it is a 10 out of 10. For the other tools, like the web part and Fiori, it is about an eight out of 10.