Worksoft Certify Reviews

The primary use is developing automation on SAP at a medical device company moving from Micro Focus UFT scripts.

We were able to increase the percentage of automation from between 40% and 50% to 80% within a year.

The most valuable feature is the ability to automate quickly and to maintain and update scripts.

We would like this to be able to be used outside of SAP applications, as it would be good for other types of products.

I have been using Worksoft Certify for three years.

We expanded licenses after a year.

Prior to this product, we used Micro Focus UFT.

We evaluated an SAP escalator from SAP in UFT.

My advice for anybody who is implementing this product is to get consulting and on-site training.

Its primary function is test automation. We record the test procedures and execute them in an automatic way. It gives us reports, in a centralized way, of all the information that we need to manage test automation for the ERP system and a few other systems as well.

We moved from manual regression testing, once a month, to continuous testing every second day, with the same scope or even an increased scope now. We don't need to focus on the manual regression tests, so we've freed up hands for support activities and new developments. I don't think that it has changed how our organization works directly, but we have become more efficient and more flexible.

In our estimation now, we save about 160 hours of manpower every month because we use Certify. It's 80 hours of robots executing the tests, and they're doing it roughly three times faster than humans. To do the same manually, and keep up with the general pace of development, we would spend 240 hours on testing every month.

In terms of codeless, end-to-end process automation across packaged applications, technically speaking there is some programming or development, like you would learn in middle school. But you don't need to be a professional in any programming language to run it. That's how we've done it. We don't deploy any additional software packages with it. We haven't developed anything else, other than using Certify steps to run end-to-end test automation.

Nevertheless you still have to have a mindset of a developer or an engineer. You need to have understand cycles and other primitive functional blocks. Still marketing-wise, I would call it codeless due to its simplicity. 

SAP is our core and the people involved with it are not always developers. Some of them are analytics or system experts. They don't know how to write C++ or Python or Java. But they can do testing automation in Certify themselves without help from a centre of competence (CoC). The only time CoC will participate is when truly complicated logic required such as runtime variables staking.

In usual situations, the skill set to start doing test automation needed is much lower.

There are a few aspects that are valuable. First of all, there is the screen-grabbing where you can pick up objects and it automatically recognizes them on the screen. Worksoft call it Certify Capture or LiveTouch. It's automatic capturing properties of the UI objects on the screen or on a mobile device for use in test automation scripts.

Other important feature is the modularization of the tests. That's an important feature not only of Certify but of IT tools overall where you create and reuse the components. Our test scripts are done with a "Russian doll" approach. That's standard for modern IT, however sometimes you see IT products without such functionality. Worksoft Certify's ability to build tests and reuse them is done pretty well and balances simplicity with flexibility.

Due to it, the learning curve improved even further allowing non professional test developers to plug-and-play test scripts. Such approach allows you to have that nice, modular end-to-end test. Can it get better? Probably not without sacrificing some other nice features and simplicity.

In terms of the core functionalities important to us there are the actual coding, great test data management, execution and reporting. Those are the basis, which are shared across many test automation tools. 

We also value that the test execution and reporting are stored in a database. We can extract the results and get the data out more easily comparing to text log files. That's different compared to other tools on the market. In Certify, we have a structured test execution and overall structured data.

Last not least is multi interface support is important for our applications landscape. With certify we can test SAP, Net, Java, Web and Android applications with one tool.

Certify's web UI testing abilities for testing of modern applications like SAP Fiori was good when we started and they developed it to be even better. We all know that web applications can change objects in DOM quite fast and it breaks tests. To counter it Certify has made object recognition more flexible and generic, so we don't have any troubles.

For standard and UI stable applications like SAP ECC 6.0 we have no trouble at all due to predefined objects and rich library of methods. That means we don't need to spend time writing a library, which could happen in some of the open source frameworks.

One caveat is that if you start running models in different parts of the end-to-end process — when you really try to hit the sky and make everything automatic, to cover multiple supply chain tiers processes in one e2e test, or similar processes that are really complicated — then tool simplicity turns into disadvantage. On other hand it stops us from unnecessary over engineering the test automation framework.

Architecturally, because Worksoft specifically built a database-oriented application, you essentially store code in the database. Git and text files orientation is more traditional approach with boundless set of tools to control versions, manipulation and analysis. But at the same time, Worksoft supplies us with their own version control inside Certify that has sufficient functionality for now. 

When it comes to mobile testing, we have a small bottleneck there. You have to buy third-party separate licenses if you want to test on a mobile. Business wise we see room for improvement there, although it's that really critical for us. 

None of these are showstoppers for our operations. Worksoft proved to delivery significant improvements in last 3 years and more we wait from 11.5 version. Overall, we are quite well covered with test automation related tools and nothing special is needed.

We've been using Worksoft Certify since 2017.

It's stable. Our approach is that we don't deploy the latest fresh version to production straight away. We wait for a few patches and follow with upgrade instructions. It doesn't matter if it's Certify, DB server, Linux or anything else.

Only case I could recall related to stability problem is a one mess-up in the database during over-few-versions upgrade about three years ago. It required some database cleanup with scripts provided by vendor support.

But looking at the overall stability, it doesn't go down or crash. SLA is 100 percent for us.

Because it is three times faster than manual testing, we can test more with fewer people. That gives us operational scalability already. Platform also has the Execution Manager tool if we would like to have technical scalability via a cluster of virtual machines. In addition Certify itself supports multi-agents, APIs, and integration with Jenkins and other applications including PowerShell. That means we can also develop scalable setup ourselves

As for overall scalability of Certify, licensing is the main bottleneck. You cannot scale more than you have licenses for. You cannot uplift your digital workforce just for one month and run 300 nodes strong cluster for a big upgrade tests. It is not different for majority of IT products, we don't see much of a problem for now.

Certify is our main tool for test automation for ERP, for PLM, and HR. It's covering two major applications that are at the core of any big machinery or manufacturing corporation. We continuously increase the scope, because systems are developed and extended.

We use their enterprise-level support and it pays out. We mostly use it for tricky situations. We often talk with 3^rd level, with architects and other experts behind the certify. It gives us good insights. Separate gratitude goes to active key account management on vendor side.

As for our improvement requests, we speak with the community and we speak with architects and provide necessary information. We don’t know do they take it into their backlog or is it just coincidence that Worksoft have provided most of things we asked for. In any case we get what we require for our testing, we in constant contant and we feel that we are listened to.

We did not have a previous technical solution.

The initial setup was pretty straight forward, in general. There is a database, there are clients to install, and you need a licenses server.

The key thing that you need to know is what you are doing, how you do test automation, in general. That's above the specifics of any tool. You must know how to slice system-under-test into functional components, make the right model for reuse. It is important to break down end-to-end testing into smaller reusable items. That's not trivial, and you faced such challenge with any tool. You have to know what you're doing.

The deployment included one of week training and then a proof of concept for a couple of months. We ran typical end-to-end process in the proof of concept and found our ways to deal with above mentioned slicing, development guidelines and how to establish roles. There are useful guidelines provided by vendor as well. Overall it took about one quarter from the GO decision to start running regression tests.

Our approach to testing is as a "safety net," so we don't need to have end-users who run it and hunt for new defects. It's just continuously testing given scope and raises a red flag if something goes wrong. Such approach secures an immediate feedback for the development team or for the quality assurance team.

End users are not involved into testing usually although we could run hybrid testing with a mix of automatic and exploratory tests.

We didn't work with any third-party. There was initial training from the Worksoft itself for one week. That was it.

The return on investment, when we count the 160 man-hours we save monthly, will take about seven years for our scale

License model is a traditional one. As with any other manufacturing company, the software testing is not our core business, it's not in our interest to invest capital into the licenses. It would be preferable to have a pay-per-use model. At the same time, it's a fair game, for now. 

Aside from their standard licensing fee, if you want to have enterprise support, there's an additional cost.

We have done a few comparisons and we're continuously looking at the market to see if there is anything better, cheaper and more suitable. Market is changing a lot, RPA solutions expanding to test automation market, Robot Framework ecosystem is thriving and some other established vendors are operating. We're keeping an eye on it for evaluation. So far, cost-wise, feature-wise, and in terms of the learning curve or skills requirements, we are good with Certify. From our comparisons now, we would pick Certify again.

As always, you have to know what you're trying to achieve. You also have to think about how do you model your system-under-test; how it is written, the quality requirements and standards. The key benefit comes from reusable test scripts. 

When it comes to Certify itself as a tool, keep your eye on objects. If you map something, keep a good inventory so you understand it and you don't put all the objects in one big basket.

For me, the biggest lesson from using Certify is that you can do quite interesting and complicated things with the codeless approach. You don't really need a complicated programming language to do testing. 

Overall, I would rate Worksoft Certify an 8 out of 10. Of course we would like it to be cheaper and more powerful and developed faster. But are there products that I would rate a 9 or 10? I don’t thinks so. It's at the top of the market from what we know about the market.

Our primary use case is for SAP regression testing. We wanted to have our all SAP processes for regression to be optimized using Worksoft Certify. It took around one year for us to take that process and learn how it works, gradually improving it until now. 

We do regression testing on a weekly basis using Worksoft Certify. This does not include just SAP. We are also using the web-based tools when we are integrating end-to-end processes. When we are moving within one end-to-end process, we are shuffling between all the scripts and then going back to SAP. It works quite well.

At some point, we will have an RPA initiative to use this solution. For example, we need to obtain data from our development environment to check reports and verify them, then create transactions using Worksoft Certify.

We have more than 160 end-to-end processes. In our repository, we can have one transaction process or more than 40. There are big end-to-end processes and there are smaller ones. There are processes that can take hours to run and there are processes that can take a couple of minutes to run, but most of the end-to-end processes have multiple transactions. In weekly/monthly releases, we don't have any type of manual testing. We are just relying on Worksoft Certify for its test automation.

We also use Worksoft Certify for test automation in projects.

For scheduling, we are using an Execution Manager from Worksoft Certify.

Our business and IT users have been able to use the Capture feature, even without knowledge of testing tools. We have different IT people from each business area, like production planning or material managements. These people, without even having any knowledge of programming or Certify, just need to know how to use Capture. They are using it because it's really easy to use. You just have to initiate a Capture and do the same work that you are doing every day. Then, Capture will record everything.

You don't need to do any coding when creating end-to-end processes; it's a codeless because it is just recording. We don't need a specific person (or multiple persons) to work with the processes to do coding for the automation, which saves time. It increases the confidence of the process owner or IT person working on it because we provide them the capability to record their own process without doing any coding and just using Capture. They are confident that whatever they have recorded is their own process. It's not someone else recording their processes. The test automator can polish that process, but we are not recording it since they initiated the Capture. Overall, this saves time.

It has reduced our test maintenance time by more than 50 percent because we don't have to do manual test processes. We have saved over 150 man-hours monthly. It has increased our delivery times. We went from 200 man-hours down (three weeks work time frame) to approximately 40 man-hours (three days work time frame).

Using Worksoft Certify, we have found and repaired more bugs, though not that many. Users cannot manipulate information during the process. We schedule our tests at the night time and leave them working. Next morning, when we come into the office, we just check our system. These tests are pass and fail. For the failed test, we have screen captures where we can go and check the screenshot to know what the issue is. No one has to sit in front of the screen when you are running regression tests or scheduling them. I can schedule them and continue with my other daily routine work.

It is really good if you are using Capture in the new version of 11, where you can have a visual of the capture. The new version is much better. If I am using the Capture, I just initiate the capture and start working normally with my transactions. Capture will start recording all the steps that I'm working with. Before sending that one, I can even edit the capture. I can add more steps or remove them. This is one of the best features that Worksoft Certify has.

Worksoft Certify is really easy to use. You don't need any programming skills. If you just know a bit about programming, then you can easily use it because most things record by themselves. 

It's fully capable to automate testing for packaged applications, like SAP and Salesforce. You can do whatever you want and even create one specific process. For this process, you can create multiple processes. For the one transactional process that you have automated, you can utilize that process in multiple other processes too. 

They have different tools for different types of documentation, like Certify and Analyzer (for business processes). In Certify, from the repository, you can get really good, detailed documentation of what has been recorded, how it works, what steps have to be taken, and what data to use for every step. From a documentation point of view, it is valuable and can be used for future training.

Worksoft Certify's ability to build tests is really easy. You can build as many tests as you want with Capture. It is very reusable to apply one process to multiple processes. 

The scripting methodology is easy to learn. It is easy to maintain because it is presented in a simple, narrative way. You don't need to know programming.

When you are using an older version of Worksoft Certify, like version 10, Capture will be running under your status bar. Then, you don't know what transactions you are currently recording. In the old version, you don't have the visual of the capture.

It works well with other tools, but there is some integration required with Solution Manager. If you are using test repository in your solution manager, then there is no direct integration between Solution Manager and Worksoft Certify. 

There is an improvement needed in the reporting within Worksoft Certify. We have either a detailed report or a summary report. We don't have report that can be used for training purposes. A different tool from Worksoft has to be utilized for this.

Worksoft Certify needs a bit of improvement for its web-based processes. It can be difficult because you need to recall the maps, then you still have to add-on for your browser. When you are using the browser-based testing, you cannot even move your mouse or do anything on your system when you are using the web-based testing. Therefore, it needs a bit of improvement on that side. While it does work, it needs improvement. From the SAP side, there is nothing better than Worksoft Certify. However, from the web-based, we are moving towards Fiori. SAP will soon be totally web-based. For Fiori, they need to be great with SAP testing. Thus, Worksoft has to improve the web-based testing part for Certify.

They have a lot of versions coming out every year, like four or five versions. They need to reduce this number. There should be one or two versions every year with add-ons, if necessary. Because if you want to upgrade your Worksoft Certify from an older version to a newer version, you almost need to reset the solution and we don't have that much time. We cannot dedicate four times a year to having the newest version of Worksoft Certify. Though, if I don't need the need changes or improvements, I can skip the most recent upgrade until the next version comes out.

Almost six years. We started launching it in 2014.

It is very stable; nothing happens.

Two people are responsible for maintenance, deployments, and updates: the test automation administrator and the test automator. One is mainly responsible for all the administration and maintenance of the scripts and everything that goes into the integration with SAP Solution Manager. The other guy mostly works with the execution of the script. 

The solution enabled us to scale up testing because we don't need to use manually testing. We are also using it for project testing. 

There are overall 12 to 13 users. We have three users identified for test automation. We have another 10 users who use different packages from models of SAP. They use it to capture their processes.

If you have any issues, cases, or discussion, you can go on the Community portal and ask from there. They have people and technical experts there who can provide you with solutions. It's one of the best things that Worksoft Certify is doing. You can discuss with others at different companies, or possibly the technical experts from Worksoft Certify if we have an issue. Then, we can open a case directly there. 

I would rate the technical support as a nine out of 10. They are very good and available at all the times. The technical support can assist through the Community portal. You can also email them directly. Their response time to contact you back is 24 hours.

Before Worksoft Certify, we needed almost two weeks or more of regression testing to move our changes from regression to production. We were spending more than 10,000 euros for that on top of our man-hours. Even then, it was not proper testing because the user could manipulate things because they were using manual testing. For example, if something was not working, they could try to use some other way to work it. After the implementation of Worksoft Certify, we have increased our scope for regression and have more than 162 processes from all the models. We have also increased the test coverage and have less issues.

We went from two weeks and 200 man-hours to three days of testing. Within three days, we can test for the monthly release. For the weekly release, we need maybe one day or less using Worksoft Certify to test our releases and move them to production. It has really improved the collaboration between teams and our dependencies between various processes. On top of that, it provides really quick reports and results for management. We are capable of providing quicker reports and results for regression, like, "What has happened? What has gone wrong? Is everything okay to move to production?" Management is also very happy with it because it is saving time and money.

I was not involved in the process.

We have seen ROI with its efficient reporting and ability to provide results to management. It has also increased collaboration between different teams, such as, purchasing and sales.

We have used Worksoft Certify as an RPA tool when we have needed to create the master data or upload some documents. This saves a lot of time. It can reduce 10,000 transactions from weeks down to a day and a half.

Purchasing and licensing are okay. Go for the perpetual licenses. In that way, you own a license, then you can purchase maintenance and support on top of that, so you don't have to pay every year for it. Even if you don't want it a contract with Worksoft Certify in the future, you will have your own license of it. Then, if your usage is not that much, you can have one or two perpetual licenses. However, if you want to run your processes, you will need more licenses, e.g., using the run-only licenses. They are really cheap compared to the full licensing.

There are multiple licenses available from Worksoft depending on your requirements. You can take any one of them, and they are really good. For example, if we are using five to 10 perpetual licenses where we need to update or create scripts and run parallelly, simultaneously multiple scripts with just one license, it will take months for us to run 162 processes. Therefore, we have run-only licenses that we use when it comes to executing our test scripts on releases.

It is integrated with SAP. It's one of the best tool to be used if you want to automize your testing for SAP. No other tool is as good as Worksoft Certify.

I have evaluated other tools, but I find Certify really tops them. Most of them are using the command line-based tools. With some of them, you need a expert who knows how to do programming. For Worksoft Certify, you don't need programming skills. For the SAP GUI, I haven't found another tool that can work as efficiently with SAP as Worksoft Certify does.

You just have to be clear about your processes and define them before going to capture one. If you capture some processes, you can just easily manipulate and merge them to the other processes. It is really easy to use Certify, but you do need to be focused. There is also integration with Execution Manager, which can be useful when you are using Worksoft Certify for scheduling and other things.

It is not that easy to work with SAP Fiori as compared to SAP's GUI.

It can be used with non-SAP platforms. 

Overall, the solution is a nine out of 10. For just the SAP GUI, it is a 10 out of 10. For the other tools, like the web part and Fiori, it is about an eight out of 10.

Initially we were automating the regression suite for SAP ECC.

From there we moved into a web application called HVAC Partners, which is something that we've developed that is a type of customer portal. That application also connects to SAP, but it does some other things that don’t necessarily connect to SAP. It is a kind of front end for quotes and sales orders that go into SAP, but it's also reporting status of orders and status of warranty claims and the like for the customers.

From there we moved into the Middle East SAP ECC instance and automated their regression suite and, from there we rolled out S/4HANA for our service business. With S/4, SAP releases updates every quarter. Because the S/4HANA instance is in the public cloud we have two weeks, essentially, to regression test and test any new functionality. We started with the last release and we did about 70 percent of the testing with Worksoft. We also used the S/4 automation tool, which is more for unit testing so it's not as valuable as Worksoft. We're wrapping up that automation in about the next month and we'll be moving on to a European rollout of S/4. We'll just start working our way across Europe and those implementations.

We have it on a virtual server and we're using remote desktop access for the offshore automation engineers to access it.

We're using it with Fiori and it's working fine. We have integration in and out of S/4 to Salesforce.com so we also automated those. The test cases were end-to-end. We start in Salesforce, which is a web application, with, for example, a quote, and then it goes into S/4 and gets reviewed and approved. It then goes back to Salesforce with the approval and a sales order is entered that ends up going back to S/4. And then there's fulfillment, back and forth, and eventually billing and collections. We were able to do that whole automation with Worksoft, plugging into Salesforce as well as integrating to S/4 and doing the S/4 automation, back and forth. It's been incredibly useful. We saved something like 80 percent of the time it would have taken to manually test, using this tool.

In terms of using the Capture feature without knowledge of testing tools, we brought on some new support people. One of them is our web support person and she had no background in Worksoft. She's been using it to do all the initial captures for our HVAC Partners. She's been able to use it very easily. Our more experienced automation engineers will follow up, after she's done the Capture piece, and troubleshoot some of the stuff that she might not understand yet. They're working with her so that she does learn it. But she's been able to use it very easily.

Worksoft’s ability to build tests and reuse them is very good. We ended up obsoleting the tests and not using them with the other tool we used, whereas now, we rerun these, at a minimum, every month. We do that for a few reasons. One reason is to keep the health of the tests up. Suppose a material is obsoleted. The test that has that material in it is going to fail because it's going to say, "Material not found." Or suppose a customer is no longer a customer and he has been blocked or archived. We run the tests to make sure that the scripts don't need any changes. We also use them in case a process has changed. We're releasing changes to SAP about every two weeks: support tickets, enhancements, maintenance, etc. If a business process changes, then the automated test needs to change to reflect that change. Running them every month, at a minimum, helps make sure that everything is healthy.

The other reason is to identify anything in our quality system that could unintentionally impact other things that the programmers didn't realize. We've caught a couple of those in queue and they said, "Okay. I didn't mean to do that. I only meant to change this one thing,” but it changed all kinds of things and we were able to catch that before it went into production. So the reusability is fabulous if you create the tests properly: no hard-coding, and you’re using data tables to hold any of your field selections, and you're using good automation standards, so you create and consume your data. If you create it and consume it, when you rerun it, it does the whole thing again. You don't have to worry about finding a sales order that works, for example. You really have to create a logical test design to make it reusable but as long as you do that, it's very reusable.

It dramatically reduces the time we spend on testing. Before we started using this tool, everyone was pretty much doing testing manually and test events were taking from two to six weeks. What they did in two to six weeks, depending on the scope of what they were doing and how many people they had involved, we can usually do in one to two days.

The most dramatic was when we finished the Middle East automation. They were bringing up another company code and they wanted us to run regression testing on all of their current company codes, about seven of them. We completed it in about four days. The IT director came to us and said that it reduced their labor by 93%. “Quite frankly,” he said, "we would never have been able to do all of that testing. We would have had to engage a minimum of 28 people, and it would've taken them a minimum of eight weeks, and we still would not have been able to do all of the tests. We wouldn't have gotten them done." We were able to do it in a fraction of the time and with a broader scope than they would've been able to do. They would've done as much as they could and then they would have gone live and hoped for the best.

And we've also been able to use it for other things like certain recurring tasks that had been done manually. We had people who were manually monitoring Tidal jobs, which are batch jobs that have been scheduled to run. If a Tidal job fails, somebody has to go in and figure out why it failed and either restart it or fix it, and then rerun it. These are jobs like billing jobs and we automated them. They probably spend 15 minutes a week on billing jobs now, whereas we had somebody doing this about 12 hours a week. And then that person would have to send out an email to whomever the relevant person was saying, "Hey, check your batch job. This isn't running." They now spend about 15 minutes running it. It saves the emails to the users, documents the results in a spreadsheet, and puts it out to a SharePoint where the auditors can pull them any time they want. It was the same thing with monitoring the claims jobs. We've done a few things like that which have added to the value.

Automation using Certify has also saved testing time, big-time. As I said, the Middle East: 93 percent. For the S/4HANA project, what we did in three or four days, they had been taking two weeks and not getting through at all. With the release, you don't get to say to SAP, "Hey, testing is running behind, we need another week," because it's in the public cloud. Like it or not, they're going live. The drill is supposed to be: You test during week one and you remediate in week two and you go live that weekend. We got our stuff done, 70 percent of the work, in about three days, and it was our first time, out-of-the-gate, so it'll go easier with the next release. The rest of the team took the entire two weeks to do their 30 percent. And within the 30 percent they were doing, a lot of them were smaller tests. We were doing end-to-end tests that go through Salesforce and S/4, etc.

In terms of defects, the value is finding the defects prior to moving something into production. There are two I'm thinking of that we found in Mexico. One of them would've brought shipping to a halt and the other one would have brought receiving to a halt. If you shut down factories, even for a short period of time, there is this domino effect. The value of those finds is huge. And this wasn't even something that the guys making this change were testing. They were testing the piece that they changed, which was working. What they didn't realize is that they changed all items instead of just that subset. It was a minor goof in the programming. It was just too broad of a statement. 

I started in IT about nine years ago and we did total manual testing. We would have defects in the high hundreds to 1,000 during the implementation testing. Now, we're probably under 100, so it's much lower. It could be that we're just getting better at implementations.

It's pretty seamless with SAP and Salesforce because they've built in the field definitions and all the things that you need. You literally turn it on and execute your script and it records it. It's very simple. Then you can go back and put in some of the other functions. For example, instead of hard-coding field selections, you put in a data table so you can run it multiple times or with multiple data. It was actually written to work very well with SAP.

Salesforce came a little later. Obviously, big companies like ours don't just use SAP. We have integrations with Salesforce and CRM and CPQ and all these other programs that integrate with SAP. Worksoft started looking at its customer-base and saying, "Okay, what are the popular ones you guys use?" Salesforce was one that came up. So one of their releases, about a year or so ago, included the ability to record in Salesforce like it has for SAP, so it's super-easy.

We’ve used the Capture not only to train people on how to do things, but also to provide the output to our users so they can validate that what we tested was proper. Capture is very good. It is lengthy, though, because it documents every keystroke that you do. At the beginning it will list all the field selections that we use and then it will give you each step: what it is, pass or fail. If we put in a screenshot, that shows up. It's up to you as to whether you put in screenshots or not. A lot of the times the documentation we provide as a PDF is lengthy but it's also very thorough, which is good.

Certify provides codeless, end-to-end process automation across packaged applications. It works well with SAP and Salesforce, for example. Another one that they have done all the definitions for is Oracle. We haven't started on our Oracle ERPs yet, but it's good to know that we can. We just don't have automaters who know Oracle, so we're sticking to what we know right now. The process automation makes for relatively fast automation compared to the other tool that we've tried to use. It makes it so much easier because you don't need any technical programming knowledge. A lot of the other ones are Java-based or based on other tech languages. That's a skillset that the average tester or support person does not have. It makes it very easy for those guys and the learning is quicker too, because the troubleshooting is easier. You look at the code, you can read what it's doing. You understand the business process and you say, "Okay, that's failing because we failed to set this flag, or fill out this field." It’s pretty simple.

I would like the ability to more easily modify the report from the Capture feature. One of the things I don't like is that it keeps repeating all the field selections throughout. To me, if we put them up front, we shouldn't have to repeat them at the different steps. It should just be Pass/Fail and show the screenshot. I've talked to them about this in the past.

There's another part of the Worksoft suite that probably does a better job at documentation for training purposes and providing an understanding of business process. It's the Certify BPP which we're not using right now because we're really focusing on automating all these different ERP systems. Whereas the testing is very detailed, which is great for the auditors and it's great for the users because they see everything we're doing, it makes for some big PDFs. It's a double-edged sword.

Also, with the codeless process automation across packaged applications, once in a while, if we get a weird application that's not widely used, it gets a little stickier. First, the software has to learn the fields, so you have to identify all the fields. Once you do that, as long as there isn’t any non-standard code in the application, then it works fine. But there's that one step that you have to do, a step you don't have to do with SAP and Salesforce, for example.

In addition, Worksoft definitely needs to continue the march toward bringing in more and more of the software that people commonly use. They're doing that, but they can only march so fast.

I know Worksoft is doing some stuff with RPA. There are other tools that strictly do RPA, but aren't automated testing so I'm not sure if they will be able to compete with those. I know that we did do some automation, what we call "bots," with Worksoft, and it was clunkier than some of the RPA tools that are currently on the market. I suspect that they'll come up with a very competitive offering. 

I would also like to see some better reporting of testing status, reporting that we can easily generate to say "Okay, we're 50 percent done and we've got 10 fails and 800 passes." That's what test management software is for and Certify integrates with that. Bang-for-buck, it's probably not a great place for Worksoft to invest. They're probably better off with RPA and bringing on the ability to more easily test software, like Salesforce and CPQ. I'd love to be able to do that as easily as I can with SAP. I would like that same ability to use Capture in CPQ, instead of using Silverlight.

We started using Worksoft Certify in 2016, so it's been about four years.

Stability is very good. They do a lot of releases. They are probably using an Agile methodology, so every time I turn around, they have three more releases out there. It would be helpful if they could release once or twice a year, but I understand why they are doing it. They are adding new features because they want to get them out as quickly as they can. I just don't have time to stop, do an upgrade, and move on.

We haven't had a problem yet with the solution. It's been very good, and you don't have to upgrade every time they do a release. We do it probably once a year.

Scalability is fabulous.

We've certainly taken on more projects. When I first started about nine years ago, there was one major implementation at a time. At the moment we have about six major projects going on and, with the unwinding due to the spinoff, there are probably about 50, but those are not being tested with automated software. We're focusing on just the two SAP ERPs, S/4, and the ancillary web apps. It does allow them to implement faster. Since we did the Middle East, they've brought up two new companies in six months, which is amazing for them. It probably would have been one at a time over a year and a half or two years, otherwise.

We don't use Certify to create RPA at this point. We have so many ERPs to automate that we're sticking to that right now. We're trying to get to where we can pick up more licenses and build up the team so we can start doing some of these other things. Right now, with the spin-off from our parent, everybody is hyper-focused on unwinding. When you're part of a big organization like we were — we're still pretty big but we were huge, Fortune 50 — and you start unwinding things, there are so many shared services and servers that are on their domain, etc. It's going to take us two to three years to unwind all that. So we're marching ahead on our ERPs and I'm keeping my head down. I have my seven licenses, although I want to get about 10 more, but I'm not going to raise my hand until we get unwound.

Most of our issues have been our own internal infrastructure issues. We have a very tightly controlled infrastructure, so I'm always banging up against that. Worksoft has been able to help us solve these problems, and they're not even their problems.

It's far easier than other solutions. We previously had HP Quality Center and we could not maintain it. Prior to my taking over testing, they had implemented that tool. They brought in some outside contractors who did the initial automation and they handed it off to the support team to maintain. But it was so complex to update it when there was an error, or just for general maintenance that needed to be done, that they found it easier to just manually test. They quit using the tool. It was a complete waste.

With Worksoft, in stark contrast, there was a little bit of a learning curve up front because for about 70 percent of your effort you can use its record function that just records your keystrokes. But then you have to go in and harden the script, and put in data tables and screenshots and validations, that type of stuff. But compared to the other tool, there are no real programming skills needed. You learn how to use the functions and when you look at the script or the test, it's not like looking at code. You can actually read it and say, "Oh okay, that's inputting the month and the year," or "That's validating that the sales order posted." It's in English and it's very clear to follow. There's a drag-and-drop, and delete and all the things that you're used to using with other applications, like Word and Excel, that makes it very simple to use. Initially we had a little bit of training involved, but since then it has been incredibly easy compared to the old tool. The old tool didn't make it past a couple of years. It's been four years with Worksoft and we've got interest, globally, from other parts of the company that are asking, "When are you going to automate our regression suite?" So it has been very well received.

Setting it up was pretty straightforward. My biggest frustration was with our infrastructure. We set it up as a remote desktop but our company has all these firewalls and restrictions around access, and my team is mostly offshore contractors.

The offshore contractors have different access than I do. I spent a lot of time whitelisting different web sites to give them the access to the software we are testing.

Deployment took about a month and a half, mostly due to the infrastructure problems. However, now, when we need to upgrade the system, we can pull it down and run the installation. Then, we always get on a call with Worksoft, because if we miss one step and it doesn't work, we can't afford to have the team down. So we get on a call and spend about an hour running through the update.

Worksoft was fabulous help with the setup. They would get on a call anytime. They would help us walk through issues and help us figure them out; even how to navigate our systems. Their assistance during the setup was phenomenal.

We have seen ROI but it's very hard to capture because a lot of the benefits are hard to monetize. We have seen a huge reduction in the time to test and a huge reduction in the number of people needed to test. Rather than lay off a bunch of people, we've chosen to do more projects, so our rate of implementations has gone up. 

The 93 percent reduction in labor that the Middle East calculated was pretty impressive. I would say that, on average, it would be more like a 70 percent reduction in test time, because you still have to have people review the tests to make sure they're comfortable. Even though we say everything passed, they're going to want to review them. And then there's the retesting of any remediation that needs to be done.

The initial investment is probably a little high. It was a little hard for me to sell, but it was a one-shot deal and that's why it's so high. All we are doing now is paying annual maintenance, which we don't have to do if we don't want upgrades, but we do.

It is based on the number of licenses. If we had bought a larger number of licenses, our costs would have come down significantly, which is fair. I did struggle a little bit trying to sell it because our company had already had one failure with a testing solution, and here I was asking for money to try again. However, since we got it in, we have had great success.

We have seven licenses today. The people using it are three automation engineers/quality assurance testers who do SAP ECC. We also have three who do web application testing. They are the ones creating the automation for our portals, e.g., customer portal. I have one test lead who oversees this team and bounces between both SAP and web testing. We haven't bought a whole lot of licenses and haven't rolled it out to a massive number of users. We're doing all the work ourselves.

Since these are concurrent licenses, we could double the number of users with our current licenses because six out of the seven are offshore. While we are sleeping, they're using them.

The solutions we evaluated were all Java-based and they all took skills that we didn't have and we would have had to hire people to use them, or we would have had to train people. The people those solutions might be good for would be developers, but I'm not going to get a budget for a bunch of developers on a test team. And developers don't want to test; developers want to develop. I wouldn't even be able to hang onto these people. That's what failed with our initial attempt. We brought in programmers, they came up with a test, and nobody could maintain them afterwards. It was an investment that we threw out.

There are a number of lessons I have learned from using Certify.

• When you get started with it, you need to make sure that you have an executive sponsor so that you get the cooperation you need.
• Pick up some mentoring services from Worksoft to help you get started.
• You need to document your test cases well. Don't just start without good documentation, because then you make mistakes and then you have to rework that particular test script.
• Be very organized in the naming conventions and the standards you're using to do the automation. For example, don't shortcut. Fill out the fields that explain what the test objective is. That way, when somebody else comes in a year later and they ask, "What does this test do?" it's right there. Be organized.
• Try not to do too much with a single test. We wrote some that were crazy long: 500 to 600 steps because our process was a very complicated process. Step back and think in terms of logical chunks, because a script which is that big is difficult to maintain. You fix one thing and you get 20 percent of the way through and something fails. So you fix that and then you get another 20 percent and something else fails. It will take somebody half a day to fix one script. You can't have that delay when you have 500 that you're maintaining.

I would put Worksoft Certify right up there at a 10 out of 10. It's been the easiest package that we've done. The S/4HANA tool that comes pre-written, where we just go in and change our data to make it applicable to us, is pretty simple but it's not flexible enough. You can only test S/4HANA within those four walls and almost nobody uses just S/4HANA. There are always integrations. So Certify, as a tool that works across integrations from one package to another, documents the results, is easy to maintain, and easy to use, is a 10. I have not seen a package that is this easy and we did look at other ones. This one was just head-and-shoulders above them. It's really a fabulous product, I'm so impressed with it.

The reporting is good.

It is very user-friendly.

Certify integrates with other tools and it works very well with other machine testing applications.

If we can import process from other automation tools like UiPath, Selenium, or Automation Anywhere, then it would be more helpful.

I have been using Worksoft Certify for more than seven years.

This is a stable product and we used it on a daily basis.

This is a scalable solution.

We have created tickets in the past and the support is absolutely fine. Their responses are very fast.

We did not regularly use a similar solution prior to Worksoft Certify, although I have worked with others here and there.

The installation is simple, taking less than an hour to complete.

Our internal IT team performed the installation and there is no maintenance required.

I have been comparing other tools, such as Micro Focus UFT One, to see whether they are feasible for our purpose.

Overall, this is a very good tool and I recommend it.

I would rate this solution a ten out of ten.

The reporting feature is something we look forward to. Also, the Worksoft training that we have done was mainly on the web application along with SAP, SNOW, and Silverlight integrations with the tool. It works efficiently with any tool. The projects we mainly used were for the testing of SAP applications over the web. 

Automating the User Acceptance Testing (UAT) process is the most valuable feature of this tool which helps us save loads of time for our clients on the projects and also generating testing and other reports.

We are doing Worksoft Online Training and also take up various Worksoft automation testing projects. I have totally trained about 50 participants on this tool. Most of the trainees are happy to learn and use this product in their jobs now. We have also done two projects, one for retail sales chain and another for a banking project. Both of the projects were delivered on time and the clients are happy to generate some automated OOB reports and also create their own Receipts. The clients were happy with our timelines and also with the training we offered at the end of the implementation.

Most of the features are good. However, being codeless is the best feature that makes it easy to use for those who have done any coding but understand simple English like commands. Anyone with basic knowledge of using a basic tool can learn and implement the solutions with Worksoft. This is why we have been able to train so many consultants on this tool and they use the same in their jobs currently and keep referring us for more candidates. We recommend using this tool for your automation testing projects.

Pricing is a bit high and we would like to have the availability of a trail environment for beginners and training would be great to have and easier to expand and use by more and more consultants.

The bigger reports needs to be more customized by the tool for better use, also tool also can be further simplified by a better Graphical user interface (GUI) will help us a great deal while taking up projects. 

We look forward for these upgrades so we can enjoy using the tool more and help others learn this tool.

I have been using this for the past three years. It has evolved as a codeless automation tool for testing. We used sed 10.0.1 and are currently using 11. 

We have also used few other tools for our trainings. 

ROI is pretty good so far.

We would like to have a cheaper version for a single user/trail version of the same for our trainings. The cloud based environment should be available free for the trainings for 15 days or so.

We do trainings on a needed basis so we understand the demand of various tools in the market. We strongly believe while Worksoft is a good tool it is losing its potential a bit to other tools in the market. The demand for training has gone down in the last two years.

We use it to do end-to-end testing for the business. After development has occurred and once we're into verifying that no regression has been broken, it's at that stage of testing that we deploy it.

We are a subsidiary of a larger company and we are focused on rolling it out, at the moment, to our larger company. With the tool's simplicity of use, where we are able to have a code review occurring, in that sense it will be useful in being able to roll it out to the greater company. We will be able to give it to the people who are experts in their areas, rather than trying to pass off test cases to one centralized location. It will be centralized automation and we'll just have one central COE.

Automation using the solution has saved testing time. I couldn't give you a number of hours or days because we're still in the beginning stages of trying to roll it out globally. We haven't been able to use the product and reuse automation. The whole point of automation is that the upfront cost to automate something is heavier and then, as you reuse it, it reduces the testing cycle. We're still investing in the earlier stages where perhaps we have spent equal parts right now, but we intend to see a reduction as we capture more and more.

Certify has also enabled us to find more defects. While I'm focused solely on automating and testing, so I don't have access to the defect count number, I know we have found defects, which tells me that we are finding defects that wouldn't have been found otherwise, or defects that wouldn't have been found as quickly.

• The dataset.
• The reusability.

It does allow for good reusability. When it's designed properly and utilized properly, we can put things in a way that allows for reusability, meaning a lot of reuse of VA01, if they're very similar flows, to keep it simple. And if we do have problems with a more complex flow, we'll make another version of VA01 that targets edge cases.

In terms of web UI testing, we've done very limited Fiori testing, but we have done Salesforce and a few others. Our experience is that when we get that stuff applied properly and working properly, it works very well. They're usually built well and if we do have problems with them we can get Worksoft to fix them. A lot of the times, if we're running on something that doesn't have an XF definition for it, by understanding how it's building objects, we are able to easily map objects fairly well and quickly.

The solution's ability to automate testing for packaged applications like SAP and Salesforce is related to when they do have that XF definition, but I do think it works very well. That's especially true for the SAP integration. That interface is very solid and objects are just about always discovered properly.

Since they updated the Capture feature to a more "Snagit" look and feel, it has become our primary tool. We've moved off of the old LiveTouch functionality. We will use it occasionally, but with Capture being built-in, it's easier for users to be trained on one tool. That tool has enough capability to be able to do both verifying the properties and recording the playback. It works well for us.

Looking at it as a product fully packaged, I would like to see more documentation or ease of use of the documentation. Sometimes documentation does exist but we have to search three different sites to find the proper way to do things or track down the technical document that explains certain fields. 

That, in turn, relates to the ease of use and how objects interact with each other. The application could lend itself to be simpler.

Another area that I would like to see improved is how the permissions are applied. If you're applying permissions groups to a user, one of the options is to delete the group entirely and lose the entire permission group, rather than just deleting the permission from the user, which seems a little silly. In my opinion, that whole module of permissions is very confusing and lends itself to common errors. We have to rebuild permissions occasionally.

The functionality is all there. I just think the way it's packaged can be confusing. We are successful and we can get things working the way they're intended to in Worksoft. It's just that sometimes finding how to do that, or where it is described, can be difficult.

I've been using Worksoft Certify for about the last year-and-a-half.

It's a pretty stable application. When it works, it works well, and it seems to work consistently. And when it doesn't work, it does not work — if that makes sense. When we see it functioning, we've got everything just right, it frequently seems to function solidly. And then, when we seem to have problems, it seems to not function at all, meaning tests will not run, or we cannot get a script to work in this or that particular way at all. But we've been able to work through all of our non-functioning issues through their support.

The solution will enable us to scale up our testing. With our focus being more on regression testing, it increases the testing of existing functionality first, and then we'll bring in that new functionality.

We are planning on rolling this out to more people, multiples of the number we have using it today. We think that it should be scalable but we haven't done it yet on that scale so we don't know for sure. But we do feel it will be scalable and that it will scale well.

Our extent of usage is pretty narrow at the moment. Approximately 10 people are using it right now and they are mainly automation engineers. There are a few directors using it to understand what the product is. People who we would consider to be "automation champions," who will help champion the product at our global headquarters, are being trained on it right now. They're not actually going to use the application, they're just going to understand it so they can help champion it and bring it on, full-scale, with user acceptance. 

Our main users in the future will be those information business analysts who know their respective products very well, the ones who are making the changes in targeted areas and who can easily reach out. They will be able to quickly test and record whatever they need to record for testing. We're looking at anywhere between 20 and 50 additional users within the next year, depending on how well user-acceptance goes, and expansion will continue from there.

I'll start with our positive experience. We always end up with some kind of resolution whenever we do submit something through support. 

There have been times though where their support has been very slow or difficult, where we end up with a level-one support for what feels like much longer than we should have a level-one for the issues we have. These are high-end issues that mean we can't function. That's been a frustration point for us. We've had to meet with Worksoft to talk about the support that we're getting.

As we start to build better in-house knowledge of some of the caveats of Worksoft though, that support has been needed less. That has made things a little better for us and that's why we focus heavily on training and having supporting documents on what we're doing.

I have used Winshuttle as well as DataLoad, which is an open-source and much more simplistic. Winshuttle is used more for something like an RPA function.

Certify has a much deeper bench in terms of what it can actually do. Winshuttle is only functional, to my knowledge, with SAP applications because it's built on the scripting portion of SAP. Its focus isn't for testing, so it's not a good tool for testing. But it is more simplistic in the sense that it looks like a spreadsheet and the result is provided in the last column of what the status bar gave you. It is really designed for one Pcode at a time, in my opinion. Whereas with Certify, you can run a larger-scale test or function or even a larger-scale RPA function, compared to what Winshuttle can support. The complexity involved in that is much harder. It's something of Catch-22, but Certify does enable you to do much more.

I can't speak to about the installation process, as we have a different person who manages installation. As far as setting up users goes, it's fairly simple within the application, once it's installed and functioning on the servers.

We started out with one model of being centralized and we're rotating to a decentralized model of sharing this out with more users and increasing usage. It's almost like we're in a second deployment of the product, and using more of the tools.

We're rolling it out to the specialists in each business area, on the information systems side. These are the people who are producing changes and who understand the changes and updates quite well. We'll have them write the scripts themselves, with our support as the center-of-excellence team. The idea is that they will be submitting the scripts that they've written back to us for code approval and then promotion to gold, to be able to be run regularly, as a script that's been validated. It should work well and be successful for them. We'll give them help with training, etc., in the Worksoft product itself. We're trying to focus on somebody becoming an application expert, for each application we're testing, and to be an application expert for the automation product, allowing them to function well enough within the Worksoft application.

The person who is responsible for installation is also responsible for maintenance of the solution. Like me, he is an automation engineer, but we have different focuses.

My advice would be to develop a very good training program to go with it. Also, understand how to build a good structure to allow for success and to limit exposure where people are editing things that they shouldn't be editing. You should also partner or work with other businesses that have used the solution successful. Build up industry contacts who can help you understand where they're going and where they're having problems, as well, with the model they're implementing.

The biggest lesson I have learned from using Certify is that you can design it to be way more complex than you need to, and you need to be very careful, when you're designing the solution, to design it in a very simplistic manner. It's almost like code in that it enables you to do things that are very complex, but you need to be very cognizant that you shouldn't always do the most complex flow, and that you shouldn't overly design logic out of any one script. They should be relatively simple.

Regarding ease of use, once you understand how to use it you can use it very effectively. But at times it's difficult to understand what the application is doing, what you are actually editing, within the application. So at times, when it comes to certain objects, you might not realize you're editing another object, in a way, unless you've used the application and understand how it actually builds together. It is simple once you know what you're doing, once you understand how all the objects work together, but leading up to that it can be more complex. We overcome that with training, reference documents, and a lot of training documents. We did an intro training with our team just yesterday. We're rolling out more globally, so we're training and trying to have a center of excellence team that can help out with these concepts. For example, they can help design better training to understand, "Hey, when you're editing here, you're doing this." We're trying to do more targeted training to the things we do with our standards inside of Worksoft.

As far as the Capture documentation goes, for us, it's almost too detailed. We've actually implemented a custom solution for documenting, because we need something that's simple, almost like what users would experience for test cases for manual testing. We also designed our own solution for that, in part, because we utilize a lot of Selenium-style code and we need to be able to record results that are occurring in that application. We'll call Selenium and Worksoft and we need to have a consolidated results report. We don't utilize, and, just to be clear, we've never purchased, BPP (Business Process Procedure) so I don't know any of that functionality. But with our unique set up, it did not make sense to utilize those reports. The reporting that is built into Worksoft is good for development cycles, developing scripts, but we don't use it for result-reporting, in the sense of whether the test passed or failed. We've narrowed it down into a custom application.

While it does allow for good reusability, even if best practices are followed, at times it's hard to identify if you have the same components or processes being built. That can be hard to recognize. For example, there will be duplicate login scripts. The application doesn't seem to lend itself to being easy to manage for duplication of processes. We are trying to put workflows in place on our team to help identify duplication and to reduce it. We do intend to use Analyze as a way to help catch duplicate workflows.

We are working towards use of the solution for RPA testing, but our primary charter is to industrialize our testing cycle, and then we can move into something like that.

We get user stories from the DevOps teams, in conjunction with a recording they make with another Worksoft tool. Then we will investigate if it has already been automated and, if not, to automate the process which has been delivered to us.

This is all set up in a Citrix environment. We have SAP being used at the moment and we still have the old SAP ECC up and running. I'm not sure which part of this is cloud-based, but the Certify solution is installed on Citrix.

Because of the fact that we started just six months ago and we have a small team. But slowly but surely we are gaining more audience; more people are starting to get interested. That should lead us to be able to start implementing it the way it should be done. We have done some regression testing and, when doing so, we found real issues. So it has proven itself to be useful during regression testing at least.

We have definitely seen savings in testing time. Scripts are executed five or even 10 times faster than any one of us could do by hand. While we don't do so at the moment, we are going to start executing them in a lights-out environment. We will run tests during the night and get more numbers, execute more tests. That should also help us save time. We have to get the experience and the numbers for this, but I think it will save us a lot of time.

The biggest feature is the fact that it's codeless. It takes away the problem of finding people with the correct programming language, since there are multiple such languages. It saves time in introducing people to the solution because they don't need programming knowledge, they just need to be able to think logically. This makes it vastly usable by more people who are not even acquainted with IT at all.

Also, the solution's web UI testing abilities for testing of modern applications is pretty awesome. Like with every product there are some parts which can be improved, but overall it's great.

It's very easy to use and to install. You have to know, as a user, what your exact application is on the test; you need to know which object recognition files you need to use.

You use the tool to do your automated testing. As far as I know at this moment, it can do a lot of stuff. It's usable in DevOps, so with regards to packaged and non-packaged software, it's good.

I use Capture from within Certify. I also have a stand-alone capture that I have up and running. If you look at the whole cycle, it takes the user a lot of time to create the records. During the capture, the responsiveness of the system is really slow. But after that, when you send it to Analyze, the documentation is really easy. Just click the button and choose the format. Automate is the same. You just create automation and choose a file name. Then, when you need it, you just download it into Certify and start using it. We've been doing it for some time now.

The Capture feature helps find the actual processes to test for and to create end-to-end testing. We ask the users, when we are making the recordings, not only to enter the proper data but also to provide us with comments or LiveTouch images of messages that need to be recorded. They know, "Okay, when I see this message then it's up and running." Because they take the end-to-end as a whole in the recording, we can use that as process knowledge as well. So the process is, in fact, being captured in the Analyze software.

In terms of the solution's ability to build tests and reuse them, I would rate it at eight out of 10. We record it on one environment. We make it completely environment-agnostic, data driven. Once recorded, we can reuse it on every single environment in the development cycle, which is awesome.

Performance on the web UI part, especially with some of the more comprehensive Fiori features, like the complex tables that are being used, could be improved. In those cases we have noticed a lot of execution-time increase with regards to the Certify solution.

I have been using Worksoft Certify for six months now. I started using it in September of 2019.

We've had no Certified-related stability problems. We have issues with things like Execution Manager and Analyze. I'm not sure if those problems are infrastructure-related or due to the Worksoft setup, but with regards to Certify it's stable. Sometimes there is a crash, but I think it's more related to the fact we're doing a lot of complex stuff in a Citrix environment with low resources.

The solution hasn't enabled us to scale up testing yet. We're on the way forward, but because of some issues in our own architecture we are not able to execute those tests. But I know how the setup is working, and I think you can scale up really easily; just add more machines, add more users, and have a go.

When I started within the program itself, no one else was using it. There were two users on the Railnova team. At this moment, about 10 or 12 people are using but within a couple of months we will be around 50 users in total.

I would rate Worksoft's technical support somewhere between a six or seven out of 10. In two-thirds of the cases we get a proper technical support member who has the knowledge to help us with our problem. But at other times we get someone who doesn't really know what he or she is doing or doesn't really understand the issue.

Another big part of the grade I gave is the fact that when you are in contact with the call center, a lot of times there really is a lot of background noise. With the accent, it's already really tough for me in fact to understand them, and with the background noise the problem gets bigger. But I've had also a lot of support from the Germans and from all over the world. Most of them are really capable.

It's an easy setup. There are some small configuration settings and then you can have a go. It's all up to the user to do the updates on the definition files. That's also easily available to us.

I started using the software without hearing from Worksoft. I only had to do some courses on the Worksoft University web page they provide. I didn't have any real, proper training, and I was up and running within two or three weeks. And within two months, I was able to provide enough support to get multiple teams within our company up and running with Worksoft. It's really straightforward.

I wasn't part of it, but I believe the initial setup and further configurations took two or three months in total.

Because of the fact that it's also able to do orchestration and because of the fact that our company is moving from the old SAP towards SAP Fiori — they wanted to have the main focus on Fiori for the UI part, in conjunction with the orchestration which Worksoft is able to do — at first it was only UI-driven. But we will expand into more and more Worksoft uses.

The company used a Worksoft consultant for the deployment but I don't know her name.

We, as a company, have good contact with one of the Dutch Worksoft managers and he introduced the integrator to us, as that manager is from the United States. The consultant came over for a week to give us some training on a number of things because we are not only using Certify.

They were really happy with her. A lot of questions were answered, a lot of issues were resolved. She was able to contact Worksoft support really fast. They had a blast while she was here.

We haven't seen ROI yet because we are in start-up mode with Certify. At this time we are only investing in the solution. Hopefully, we will be able to have some insights into ROI within a half-year from now.

The licensing is yearly.

My advice would be to think outside of the box. If you've chosen to work with Worksoft, you have to embrace the product as a whole. You will find, as with any other product, that some things that don't operate the way you want them to or would have expected them to. But if you teach yourself to view your problems from different angles using the software, then you will be able to come up with the most brilliant solutions. You can do much more with a codeless tool than you might think upfront.

The biggest lesson I have learned using comes back to the codeless part. I view myself as a smart guy, but I don't have the proper coding language knowledge. I was working for myself over the course eight years, before working here, and oftentimes the jobs were really cool, but most of the time I had to do Python and this and that. That was always a struggle because sometimes, when you've learned a language but you're not using it for a year or two years and you want to go back, you have to start remembering it. So I was turned down for those jobs. In this case, and we can show the world that it can be done codeless, if you have the proper tools.

When I was first introduced to Worksoft and they told me it was codeless, I was really skeptical. I said, "I don't see that happening," because I had been doing this for quite a while and was used to doing some coding. But the tool convinced me otherwise, which is really nice.

Overall, it's capable of being used in modern technology environments. I have been using it for six months now and I still have a lot of learning to do. And as a company, we need to start using more of the Certify features, not only scripting and rerunning those scripts.

Most of the people who are using it right now in our company already have some testing experience, but it's our goal to have business and IT people use the Capture feature as part of the process for DevOps.

We don't do test maintenance at the moment. We started out with test automation. We had to set up a base for the DevOps teams and then support them from that point onwards. So we are slowly moving into the maintenance part. Because we have split the data from the script itself — everything is data-driven — so it should be fairly easy for us to make the necessary changes. I think execution is faster when compared to human hand movements. But for changing or maintenance, I don't know.

The solution hasn't enabled us to find more defects at the moment, because we have been focusing on "happy path" testing. We need to get to the end-point of the end-to-end testing. But I believe, and I'm rather positive about this, that if defects are entered into the system, given that our regression test set covers a big percentage of the complete solution, it should be able to find defects really fast. Faster than we can.

The Certify users within our company are all in scripting. We're developers. And because we are in a scrum team, we don't have different roles in our team for test automation. A lot of things are being delivered by DevOps the teams, which you can view as functional consultants. As for the deployment and maintenance, a lot of it is outsourced to one of our partners. We do have functional and technical maintenance or support. I'm the technical guy and then we have two functional guys as well.

We are using the on-premise version and testing an eCommerce platform: SAP Hybris. This would take in all of our vaccine orders over the Internet. That is primarily what we are testing it on.

The autotesting piece has allowed us to complete testing more quickly, which helps us with our agile sprint. We are getting our releases out at the end of every four to six weeks, then pushing them out. Thus, we're delivering value in terms of our website being faster. The whole autotesting piece has helped us get there and deliver that. This has saved us time.

We are able to take about 1500 manual test cases, working with the Worksoft resources and system, and clobbered them down to 600 to 700 test cases. Our testing has gone from what would have been a course of six weeks down to a little over a week's worth of testing on the autotesting. This is possibly because our systems aren't all in sync at the moment, and we're still in the process of fine tuning this. When we finish fine tuning them, the testing may even be quicker.

For the tool, its valuable features are:

• The ability to build objects. E.g., if you have 50 test cases, but in those fifty test cases, they all had embedded a place order flow. You could build an object of place order, then all 50 of those test cases could use those steps in that object. This has been a real sharp feature, because you don't have to do those order steps 50 times like we used to do on our manual tests. You now have an object sitting out there where you can just reference the object, if you will. That has been a sharp feature.
• The general Capture tool, where you can walk through and mimic what you think is a test case, and the whole time it's capturing your steps. You can then use that to fine tune it and pull a test case out of it. We have it hooked up with Micro Focus ALM. This is where we have all of our test cases living, and it's sitting in that repository for us.

It is a pretty easy tool to use as far as automated testing tools go.

We went into this with the thought that we wanted to be able to hand this off to a business user, so the business user could develop their own test cases automatically through automation. We are not seeing that. We still have it assigned to an IT professional, someone who is certified in Certify. We constantly have to have that type of person around who can build these test cases for us. At the moment, there is not an automated testing tool out there that will allow a business user to develop their own test cases, and certainly not at the level that we want it to be it. So, this may not have been a realistic goal on our side to expect that one of our business people, who has their real job, could spend a couple hours here and there developing test cases on an automated testing tool, like Worksoft or any other.

It's a software package, and you have to know the software to be good at it. You have to have a certification in the tool to be able to be really good at it.

It would be great if our business testers could develop their own automated test cases. However, we either have to bring them up to a level of certification on the software or go hire somebody to do it. Worksoft, in essence, is the Mercedes-Benz of testing tools. If you want a Mercedes-Benz, you have to pay a bit more money. 

With every release you do, you have to go back and touch your old test cases and bring them up to speed, or develop new test cases. In the beginning, that is a challenge because you have to have someone who is certified in the tool to help you develop these test cases.

It is a little complex for someone who is not in the autotesting space to learn it. Like any software, you don't show up to use Oracle Database on day one and think you know it. You have to learn it, get certified in it, and understand it. This tool is similar in that sense. You have to have someone who knows the tool and knows how to use it. It's not something that your business users are gonna pick up, especially if they have a day job. It will take a long time for them to pick it up without full dedication and going to get certified.

We have had no issues with stability. It's been stable.

We haven't jumped yet to other systems. We are definitely looking into that. There should be no scalability issues.

We came to Worksoft with the hope of doing end-to-end testing. When Worksoft came in, we challenged them for a week to show us that it worked where you jump systems. They literally put people on our site for a week. It was a quick little RFP. We did see it work, so we know it works. We saw it jump back to our CARS system. Literally, we pulled Revitas CARS system up, and it logged into our CARS system, that's a web-based system as well. It started going through the steps that we needed to go through in CARS, then passed the status back to our eCommerce system. So, we know it works for us.

We know that the systems that we have involved will work with the end-to-end testing, but we haven't gone there yet. It's mostly on our side, not the software nor Worksoft. We are just doing other projects right now.

We currently have ten people (tops) using it in our organization. 

The technical support was pretty sharp, friendly, and responsive. There wasn't anything glaringly wrong with them.

We have a guy from Worksoft sitting onsite. If I have a problem, I talk to him first.

Long ago, we used HPE QuickTest Professional (UFT). The reason that we had to get off it, because it was liked by our business people, was when we went to Hybris in 2015, they didn't work together. So, we jumped ship at that point and went back to manual testing.

While we already had manual test cases, we wanted to move to autotesting because we are doing agile sprints. Our sprints were down somewhere between four to six weeks, depending on what is in that particular sprint and various conditions of trying to get that sprint out the door. We are trying to get down consistently to four weeks. Consequently, we had these test cases, which was up at around 1500 before, and also manual. We needed to get them to run in quicker, shorter periods. That's where autotesting came in.

The initial setup was pretty straightforward. We had manual test cases already in place. Before you even got into the entire tool, you would have to do basically a cleansing of your manual test cases. We worked with Worksoft on an analysis period for about a month upfront, where they run through your test cases and make sure they understand what you're trying to test. Then, they try to map out a way forward as the best way to automate. Therefore, the gear up is the homework you do with them in that month before you even touch the tool.  That is the setup piece. 

The setup for us was wanting to know our test environments and putting together solid test cases. In our case, the account setup with customer emails, names, and addresses with all that testing data that you need. We spent that month getting the tests in good shape and all the prerequisites needed to run our tests lined up. Doing homework ahead of time then makes the autotesting run smoothly.

We are still deploying. We started with what we called a 'pilot'. We gave them around 300 manual test cases, which we got down to 141 test cases as part of our pilot. We ended up reskinning out website, and consequently, we had to go back and touch all 141 test cases. Some of them have now become irrelevant with the reskinning of our site. Therefore, we had to go back and reanalyze all of them, and find the ones which were still relevant, which were 90 to 95 percent of them. Then, we had to touch them up. We are in the process of doing that now, touching up our old test cases and building new ones on top of them. This took us a good six months, but we are starting to run regression tests now, though they are not formally in place.

We hired resources from Worksoft, but we do have our own testing groups. We have also hired some Worksoft certified people. We have been using Wipro for Certify testing and development, but our experience with them is not so good. 

Wipro was our testing vendor for manual test cases before we brought on Certify. My guess is that they did not want to lose our business because they told us that they knew how to work with autotest cases, and they didn't. They told us they knew Worksoft Certify, and they didn't. They were given a second chance and hired some Worksoft certified people, but it was a really big headache.

For deployment, we have two people from Worksoft and five or six Wipro people. The Worksoft people are far more productive, since they know the tool better than anyone, but they are more expensive as well. The Worksoft people are sharp. They notice things in our testing and point things out. Their understanding of test cases is off the charts. They picked up our systems very quickly. Wipro has been a bit more of a drag. It is because they're learning Worksoft Certify and don't really know the tool. We also have one or two business people involved who are not developing test cases, but they're project managing.

We are still building test cases, but we are running the testbed that we have, which is a few hundred test cases. However, we only need one person from Worksoft to maintain this.

What autotesting has helped us do is consolidate our test cases because our company departments were testing in their own individual silos, running their own test cases manually. Now, with autotesting, we have been able to eliminate duplication of test cases across those departmental areas. This has helped us knock down our number of test cases. Our test cases are also running more optimally. Therefore, it has very much helped in that sense, so we were able to eliminate a lot of test cases and get out of manual silos by running on autotesting, which is more efficient.

Autotesting can runs overnight. It can run faster than someone doing it manually. The assumption is the system is not making any errors, but someone who is manually doing all the testing could miss something, get tired, etc. Be human, basically. Consequently, the autotesting eliminates some of those types of errors. Put all that together, and we are able to run autotesting and get our whole testing cycle done along with regression testing for an upcoming release, which is being done on these agile four to six-week sprints.

Overall testing has gone from six weeks down to pushing a button on a Friday. We may come in a couple days later, and the testing is done. At the moment, it takes less than a week for the testing, as opposed to six weeks in the past.

We haven't really seen the cost saving come in yet.

The price is in line with everyone else's in the market. They are not cheaper nor more expensive than anyone else who was in our RFP. 

There is a cost involved to doing it, but once you get over the initial cost, then you'll start reaping the benefits and seeing that testing is getting done more quickly and efficiently. We are still early on with it, but the expectation and what we're seeing is that we will start seeing some savings coming out on the back-end once we have this done. 

We looked into TCS and Cognizant.

There is an initial mountain to climb, where you have to get all your test cases in order and have the data ready. This will make it a much smoother setup when it comes to having Certify people coming in. I recommend hiring Certify people who really know the software. Once you get it humming, this is where you will see everything you are dreaming of, where you start a testbed one day and within a week your whole testbed is running, then you have figured out all the issues and can rerun it again. This is where you start seeing the benefits of autotesting.

We have the Capture tool, but I don't know the version that we have.

We are not doing web UI testing for modern applications, as we have SAP ERP, SAP Hybris, and Revitas CARS.

SAP HANA S/4 HCM, FI, BI, Payroll, SuccessFactors, Fiori, other web HTML interface applications.

Our company automates a lot of functional tests for SAP. The automated tests are included in larger integration tests of SAP transaction, workflows and end-to-end processes. Successful implementation and execution of Worksoft Certify tests has been faster than execution of the tests manually. With other tools, it was hard to get successful technical execution right quickly. Automation is not easy. Worksoft Certify makes it easier yet flexible enough to handle some of the most complex automation tasks. Every time steps are automated, test execution times are reduced and application problems are identified faster. Every time that the automated script runs (many times if developed early enough and applied with each change of the application being tested - in the current environment and leveraged in another environment), the investment of work made to develop the script is realized. The more times that the test is executed, the risk of problems going undiscovered is reduced. Testing time is shortened. Avoiding the tedious tasks of manual testing easily repeatable steps saves money.

Worksoft Certify provides the ability to develop test scripts with a low bar of entry among automation tools. It’s very simple to develop. However, great care must be taken to use the tool as intended, otherwise efficiencies and effort savings will not be realized. Like all automation projects, well-designed automation, with thoughtful planning and application, is necessary or you will unable to get back the return after spending the cost in investing time and effort with the tool.

Worksoft Certify provided outstanding feature support for SAP test automation which is unrivaled by other tools in the market. If implementing and need to automate SAP, first pilot the tool with a representative scenario, and then evaluate that retesting the scenario multiple times will ultimately provide compelling overall value for SAP test automation effort.

Worksoft Certify can better identify web test objects by providing libraries through its Extensibility framework. It has improved with it's latest version through Extensibility Tools support and additional test object attributes. Object identification is a necessary barrier for the technical execution of a test script, aside from the script's needs to verify business rules. Automation engineers must affirmatively answer the question: Is the application automatable?

If Worksoft Certify or any other test automation tool can get object identification down consistently and flexibly so that maintenance of web object identification (and hence the web test script) is easy and done with no execution problems, it will allow more scripts to be developed and applied faster with the human test automation staff resources on hand.

A good part of the technical test automation work is to make the script run and affect script changes when the application under test changes. When done right, the business part of automating the test application functions remains the same - leveraging your test investment.

Certify may not the best solution for your needs, but among the commercial and open source offerings in the automation tools space, it should rank very high. It is also a leader in rapid SAP test automation, and can be used as general all-purpose test automation tool for varied clients like PC, SAP, and especially Web.

Only one or two tools I have experienced could be suitable in place of Worksoft Certify. However those tools also carry their own set of disadvantages.

In terms of stability, Worksoft Certify is a polished and mature commercial offering. We rarely encounter problems with the tool. When we do, Worksoft is very responsive to the issue.

We also find the new features in version 11 of the tool to be innovative. The more friendly and fast UI, greater support for test object Extensibility support, Capture 2.0 functionality and global search are all market-driven features that benefit the bottom-line in creating and maintaining Worksoft Certify automation script assets. The improvements are welcomed and we look forward to even more such improvements to make Certify a top-tier competitor in the automation market.

In terms of scalability, we only have a few licenses. We manage and maintain the underlying database actively to avoid scalability issues. We are confident that Worksoft Certify can scale to fit our automation testing needs as we:

• Scale-up with many more automated tests of varying length
• Scale-out with unattended lights-out execution.

I would give technical support a rating of 8/10, 10 being the highest. Worksoft is very quick to respond and address customer problems identified with Certify.

My organization has been using Certify since 2007. We are committed and very happy with Worksoft. I picked up experience with Worksoft Certify when I joined my organization and leverage my existing automation skills to quickly become effective with the tool. I still use Selenium/Java/Python and was expert-level proficient several market leading enterprise QA automation tools.

The setup was done prior to my joining the organization. The existing work helps make my drop-in experience very smooth. It would be harder from the ground up, since there is a database and application server that needs to be installed and administered, and procedures and policies to define so that development is standardized (establishing naming conventions, specific ways to do things when there are multiple options available, etc.)

We implemented and developed in-house expertise. A vendor team would have accelerated and made adoption smoother.

We get at least a 2x return on investment (time, cost, labor) over manual efforts. When tests repeat execution the ROI can go higher. Tests often repeat when executed annually with planned annual system upgrades and additional time, labor and cost to maintain factored in. Including and reusing the tests in other testing activities such as sanity checks and regression cycles increases the overall ROI and investment value.

Conduct a proof of concept (POC) so that you understand what needs to be invested with Worksoft Certify. You will need to understand what sort of time you need to initially set aside to learn and apply the tool, and how that time can be leveraged to shorten continuing development of Worksoft Certify test scripts (processes) as facility in the tool increases.

• Someone needs to know to administer it.
• Someone needs to know how to develop the tests.
• Someone needs to develop a representative automated test script for a candidate application in your organization.
• That person also needs to execute the scripts and then report results of the scripts.

That script needs to execute several times. Now a comparison needs to be done with the time it takes to execute the multiple test runs against the same time it would take to do the same thing manually. In this way, you can now quantify your savings.

Now apply the need over the instances in your organization and compare that to the pricing and licensing to see how many instances of this effort would give you a return on your investment.

You will find that you will have many instances that would justify the price and licensing costs.

We evaluated QTP and Selenium. Worksoft Certify was chosen because of:

• The comprehensive coverage
• Extensive support with SAP
• Additional web test automation support
• Ability to develop tests fast and execute them reliably
• Technical support of the tool

The combination of all narrowed down our choice to Worksoft Certify. It was a great choice in 2007, and if evaluated today instead of 2007, it still would have to be Worksoft Certify since the importance of SAP support is priority and is significant over any of the other criteria and support provided by the other competitors.

Perform a pilot. Apply it to your app under a test with representative workflow. Understand the process of developing Worksoft Certify tests by doing it. Execute the develop tests multiple times. Can you live with developing the tests for Worksoft Certify? Is there value in the execution over multiple times quickly? Can you justify this effort over the cost of your current practices? If you can, then this incremental step can be leveraged and then take you further into more automation successes and favorable outcomes with Worksoft Certify.

Our organization usually tests on beta and production releases of Worksoft Certify so we are privileged to be able to work with the tool ahead of the general market. In that, we experience more issues that would not get out to production releases. Also even after release we work closely with Worksoft Certify support to identify and resolve Worksoft Certify functionality and may receive such priority support than a normal Worksoft Certify client would experience since we have such intimate technical knowledge, experience and relationship with Worksoft and the Worksoft Certify automation tool.

Because we are an active participant in a robust enterprise commercial process to help improve the stability of Worksoft Certify (staged releases to select customers to improve quality prior to some production release), I would say we’re contributing towards a good job of keeping up the stability of the automation tool by testing it first hand in our complex situation and environments and responding back with realistic and practical feedback encountered. We accomplish our organization's mission to develop and run diverse, varied and large amounts of Worksoft Certify processes automation and also assist Worksoft mission to develop and improve the Worksoft Certify automation software.

When speaking on stability, I refer to the stability of the Worksoft Certify software tool. Execution stability of the developed test scripts (Worksoft Certify processes) is a whole separate concern. Even so, Worksoft Certify also shines here in comparison with other tools. To really address this properly and clearly, you need to automate a small test scenario in an application that is to be tested and compare the tool script execution with another tool to see that Worksoft Certify achieves a high-level of repeatability and resiliency in script execution.

Even so, we do work with an occasional problem that comes out in production releases and experienced that the interaction and speed to which the problems are addressed and resolved by Worksoft Technical Support to be extremely satisfactory. I am not sure if this is representative of all of Worksoft’s clients, but we are very happy with their speed and service with which they handle our issues. This contrasts with the unsatisfactory paid subscription support experienced with commercial HP QTP/UFT and IBM Rational tools, and the lack of any support (other than searching the internet and relying on the user and development community) for open-source tools like Selenium and JMeter.

Without the interactive and engaging support, some of these problems would be extremely difficult to solve on our own. However Worksoft solutions were technical in nature beyond what a normal client user could identify and resolve on their own. Using the available and comprehensive technical support and educational services increases the likelihood of success and a positive outcome and benefit with Worksoft Certify. Successes with Worksoft Certify will come from meaningful collaboration to address automation issues rather than handling problems through independent or isolated effort (e.g. the problem could have been addressed with another customer and the resolution could have been worked out already. Worksoft support would be in the best position to know the possibility that a solution exists). 

This is typical of automation tools and the experience of other tools in the automation tool trade. The knowledge and experience is specific to the tools. As such, going to the source who has the most experience is the recommended strategy.

We automated a big end-to-end process: Hybrid to SAP to Manhattan, then back to SAP.

We have web UI automation testing. Hybrid is a type of web UI testing. We have SaaS automation testing, along with CRM testing. We also have Manhattan, which is a third-party application.

We are now trying to automate all our applications: SAP, web, third-party, and legacy. Instead of multiple tools, we have been able to have Worksoft handle a lot of our applications. This has saved us a lot of time and effort. 

The test automation for SAP is very good. It works with the automated dial up menu, which helps us compared to the other marketing tools in SAP.

People who don't have coding knowledge are capable of doing automation with Certify. It reduces coding and scripting dependencies.

We would like it to support mobile automation going forward.

The stability is pretty good.

The technical support is pretty good. However, their first response is usually late. Once they start responding, it will be on time.

Our upgrade last year was straightforward.

We no longer have any manual efforts for our regression testing, which we run on a weekly basis. So, we are saving two resources at 80 hours a week.

I would recommend Worksoft Certify. It is a library for everything. It supports cloud applications, and the market is moving into the cloud.

The Capture 2.0 feature worked very well with the maintenance testing versus the regular development.

We bought the product in 2014, but did not have the right structure in place to use it properly. It wasn't until 2017 that we started to receive good value out of the product. Automation is now part of our strategy.

We use this solution for end-to-end testing of our packaged and SaaS applications. It is very much a part of our key strategy. We have it running end-to-end updates all the time.

We do web UI testing of SAP and ServiceNow. This was difficult in the beginning, as we had a homegrown product, and we had to do the proper object naming. With SAP, Worksoft does the object naming for us. This improved our website from 2014 until now. This is part of their process improvement.

Our business users are doing regression testing as their day job. This is an add-on to their daily work. With everything so pressured in the industry, automation takes the pressure of these users. This is essential for us.

It is not IT dependent. You can go to a business user, and say, "Just record what you're doing." They don't have to be very technical and can just do their job, then you can do the technical side or evaluation. This product helps us so much in capturing real business processes. This makes the overall process much easier.

A technical person thinks differently to a business person, and having to actually see what they do is the part that's so good for us. We can see the processes exactly as they are, not how we think they are. This makes a difference.

I would love to see more data mining modules. 

I want to see more stability in the Execution Manager.

There have been major improvements in stability since we started until now. So, it is becoming more stable. 

Also, we are being taught how to make it more stable, since stability is not just about the product. It is how you use the product. There are base principles that you have to keep on your data in the system. It is how you write the script and if you write the script to be sustainable. This will make it stable.

I love its scalability. I like records set that you can just add onto it. We are going into a new country every year and the ability to increase from test scripts makes the product so much simpler.

Their technical support works with us, which is good. Working with them has been a collaborative process.

Worksoft Certify was originally purchased for a product. We thought that it was not be used to its full extent, so we ran it through a PoC. Then, we decided that the product could work for us.

Ownership wasn't taken for the product during the initial setup, so it was a difficult process at first. Once ownership was taken, it wasn't that difficult.

We have seen ROI.

We have not looked at other vendors.

If you're looking at automation, it has to be part of your strategy. It's not something you can push bottom up. It's not something you can just do for a project. Automation is how we work. It is the thing of the future.

I haven't personally used the Capture 2.0 feature, but we've reviewed it. Worksoft has definitely made it simpler again by putting in comments and adding in some extra things. This has really helped, and our business users have been using it.

The primary use case is to automate their SAP and Web applications. We use Worksoft Certify for end-to-end regression and integration testing across our systems.

It helps us to implement automation testing as part of most projects, so the need for manual testing can be reduced. This really accelerates the testing process as a whole. Before, where it could take ten days to test a project, now it takes only one or two days to do the complete testing. So, it helps us to reduce our testing timelines.

We no longer need ten people sitting and manually testing something. We can just have one person running the entire regression automation testing suite, and this has saved us dollars.

I have found Capture 2.0 helpful compared to Capture 1.0. It allows you to have access to all the screenshots when you use it so you see what steps you are capturing, and if you are capturing it right or not. It is very user-friendly.

It is easy to use and learn the application. For example, I have an intern who joined me three months ago. Today, she delivers the same number of scripts as my experienced developers with great quality.

One feature that could be added to Capture 2.0 is generating a PDF file from your capture, so you can see your screenshots and steps. This will really help teams leverage the documents generated as part of requirement/training.

Right now, when we do regression testing, we manually have to generate all the reports and populate all the results in HP ALM. We really are looking for a solution to have send all the results to HP ALM once Execution Manager completes the execution, then automatically logs them.

Our offshore teams experience a lag/delay when using the Worksoft interface. As of now we use VPN and Remote Desktop to help us with this issues, it was be great to see how much Certify 11 has improved in terms to offshore accessibility.

Every time there is a new release of Worksoft, they present it in a conference. However, there is no training document nor one point of solution where I one see what new changes/feature have been implemented, like a portal. If I don't know how to use a feature, there is no training nor documentation available. When you reach out for support, it takes time for them to research it and get back to us.

I would like more use cases or at least a weekly email update to all the customers saying, "These are new features which have been included in the last week." That would really help.

Worksoft with SAP is pretty stable. 

When it comes to the web, you need a lot of extra effort in making sure that the tests are maintained, but that is the nature of the web application. You have stuff that is changing all the time, so you have to ensure that you maintain your tests regularly.

We have learned the scalability part over time. We were very successful with SAP, but not very successful with the web. Over the period of about two years, we were able to prove to our company that we can use Worksoft for different type of applications. While there is a learning curve, it is all about trying things out and failing few times before you get a success.

I would rate the technical support as a seven out of ten. There is a delay in time zones when we reach out to them. The response is not as quick as we expect it to be with the other solutions that we have. There is definitely a delay in timing, as the speed is lacking.

The initial setup was easy. We had all the information and the requirements that we needed to do the prep work before doing the implementation.

We have Worksoft help us each time that we do an upgrade or implementation. Our  upgrades have gone smoothly.

An example of saving time and dollars: We had project going on that used to require manual testing. The first time that they did manual regression testing, we had a group of about 15 testers who sat in our office for a period of two weeks to do the testing. Now, when we have to regression testing with automation scripts ready in our Worksoft Certify, we do it in a day or two. 

We looked at quite a few vendors and automation tools in the market. We chose Worksoft because of the ease of use and mentorship support they provide in making sure we were successful.

For a new customer who is to implement Worksoft Certify, I would suggest 'Start the right way'. Have a Worksoft mentor come in and help you with your automation journey specific to your organization so you can have expert support until you become successful with it. Once you are successful, you'll know what to do. E.g., we had a team of interns who got trained and they tried to work with it, but it did not work. Then, we had Worksoft help us (after two years), and it worked. 

My team and I do the regression testing. We are a team of three to four people. We are not working on just one project, we are working on five or six different projects.

What's next, well I hope we are able to present that next success story in next conference ;)

Our organization is building a platform which supports a client across SAP end-to-end. As part of it, we have reconfigured go-to market solutions across industries and functions. Now,  specifically for SAP, we use Worksoft to test these reconfigured solutions, as these are very critical. As there are newer versions of SAP and we are building our solutions, we need to ensure our solutions can do regression testing.

The second important aspect is these are real visible assest which we take to the client. So, not only having preconfigured systems, but being able to test it where clients have Worksoft, it gives us an edge. 

Worksoft has helped us position our company better because the product lets us show our value in terms of the benefits that we bring.

Reusability: Once we develop an asset, we can easily move it to a client. There is something called a "reusable script". We adapt it for the client because it can be used in different ways. This saves the client 40 to 50 percent in time. The cost reduction has been anywhere from five to 30 percent for reuse.

We find the import/export utility to be critical.

For business users, the product needs to be more intuitive and user-friendly.

The updates for SAP Fiori have been great, where previously we saw a lot of issues. A year ago, it used to fail miserably. Worksoft teams worked with our product teams to resolve this issue.

I would like to see more integration features. It needs to provide the automation across the right set of plugins and integrations, along with cloud solutions, with the ability to quickly adapt.

Going forward, I am looking for end-to-end testing in DevOps.

The Worksoft team has worked with us to help us improve the stability.

The license cost is quite high. This might not be a consideration for a large company, but it will be for a small company. E.g., Tricentis (their competitor) offers certain exclusive use cases where a company can use it in a certain way, so this is another option that companies consider.

We do an evaluation of the tools which are best for automation with SAP, and Worksoft covers more SAP use cases.

Worksoft is our preferrred partner, but we do work with other testing tools.

Competitors include Tricentis and Micro Focus Quality Center.

There is training required with the tool, but it is easy for business user to understand.

The primary use case would be for automating SAP. We did attempt to automate some non-SAP processes, but had mixed results. We do have some areas which are already using it for RPA-like use cases.

We use this solution mostly for end-to-end testing of SAP.

These are some cases where automation helped:

1. Mostly in the area of project testing, the most immediate benefit is when you historically have manual testers do a certain job, and a full regression testing was previously done 100 percent manually. We have had cases where the release testing for an entire region would take around 12 weeks. With Worksoft, we are now down to two to three weeks.
2. The second benefit would be production execution runs. This would be where we run test cases to do certain transactions, like mergers and acquisitions. Or with asset migrations, there are hundreds and hundreds of lines of assets, that we just need to set up, which can be done by Worksoft.

Codeless automation helps speed things up, versus previous automation tools. It is easier for people to automate.

First, product stability needs major improvements. Our projects get delayed and our executions fail because the product is not stable enough.  We hear consistent reports of performance issues, Execution Manager crashes, and so on.  What is not helping is Worksoft's ability to respond to issues. They do not have clear SLA's on when issues get closed.  We have had High criticality issues that were open for months.

Second, there are mixed results for non-SAP Automation.  We tried to do web UI testing on SAP Fiori and some internal applications, but the results were mixed. In some cases, we are able to automate, but it takes a very long time to do it. There are other cases where we totally couldn't do it or our customers back out somehow, because of the length of time or limitations of the technology. Also, Worksoft testability assessments take quite awhile.

The stability is very bad. We have a lot of projects, which are run by our service integrators offshore in India. There have been performance issues, system crashes, etc.

While we recovered from these, it does not give me confidence, because it is happening every week or two.

Scalability is still to be seen. We have seen a lot of performance issues, so I cannot say for now. From a technical standpoint, it seems relatively easy to scale and we have attempted to do so. However our developers report frequent performance issues and system crashes.  We have not made Automator's mode work after one year in Worksoft. So in theory, it works, but if these issues persist, it does not give me the confidence to scale out another 100 projects.

Our experience has not been good. There are no SLA's for issue resolution which is an issue for a large enterprise customer like us. For us, there needs to be a certain level of reliability and predictability when things go wrong. We are not getting this today because of lack of SLA's.

We used to do UFT, which was a pain to maintain. 

The initial setup was pretty straightforward. However, we noticed that there were a lot of issues with the configurations and we needed to work back with Worksoft support.

We mostly deployed it ourselves. We had some consulting services from Worksoft. They helped in the initial set up, especially with setting up the Best Practices.

Automating our manual processes has saved us 70 to 80 percent of manual effort.

We evaluated Worksoft vs Tricentis. Our criteria for evaluation:

• Cost
• Internal experience and experience of other customers
• The company as a strategic fit with us for the long-term. 

We applied percentages across these criteria, with cost having the highest percentage. Because of our company's profile which was majority SAP, we chose Worksoft. This is with full knowledge of Worksoft's technical limitations versus tools like Tosca.

Today, our team recently launched Selenium. This is because of technical limitations with Worksoft. 

Test automation is a must for any company. It is not just about the tool. It is about the processes, how you maintain it, run it, and how you respond if you have any issues. The toolset that you choose must support the entire ecosystem for the automation process. You need to have an engagement model, robust lifecycle, and sustainable executions.

We used Worksoft Certify with SAP ECC and Fiori applications in the beginning. We extended that to the ServiceNow application. We have up to 15 modules in ServiceNow with 750 test cases. Our regression testing suite is around a 1000 processes.

Previously, the business was spending a lot of time on processes. So, we started automating and executing these processes on a weekly basis. Last year, we had planned and resourced only one major upgrade, but due to this automation work, we were able to add one extra upgrade.

It provides a lot of time savings. We are always ready to execute a task whenever the business asks us. We saved approximately 7000 hours in 2018.

The solution is easy to use.

The stability has been good, so far.

The scalability has been good. At first, we targeted complete automation only for ECC systems. Then, we extended it to CRM and ServiceNow, not only for the North American region, but for the European and Asian regions, as well. Our company is global, and our next regional rollout will be Latin America.

The level of technical support depends on the engineer who comes to the call. I'm not really happy with it. I want to have quick response for my issues. I'm not the technical person. I'm paying for them to support me. 

They come on, and say, "Oh, so-and-so link is here, go through that link, and make the modifications." I'm not comfortable in making those changes. I want to schedule a call, share my screen, and have them fix it for me. That would be a beneficial time savings for both of us.

We were using UFT before Worksoft and were not happy. UFT was too technical. We wanted to switch over to an easier tool, which was how we switched to Worksoft.

The initial setup went well. It was straightforward. We wrote all down instructions and procedures that we need to have before the installation. Once we had everything in hand, the configuration was easy.

Worksoft did the deployment. They were awesome. 

We worked with Charlene Smith. She did the installation for us.

We saved $1.3 million using Worksoft Certify in 2017. 

Our 2018 numbers aren't available yet.

We are not using Capture 2.0 at this point, because we don't have a Worksoft Analyze license.

This solution came to us as part of IBM ELA Worldwide. The company had ELA at that time and told us that this was part of our ELA, "Why don't you go try it out?" We did a PoC, and we liked it. Without a second thought, we signed up for it.

I definitely recommend Worksoft Certify. We went with a vendor and have seen a lot of problems with it, so go with the best partner for automation. I also want to advise to go with the best practices, as there is a difference technically when you follow the best practices. 

We have integrated Worksoft Certify with Micro Focus ALM. From ALM, we execute some tasks from Micro Focus Quality Center.

Our test maintenance time is really low at this point. In the beginning, we were having around a 30 percent of failure test cases and spending a lot of time on those test maintenance cases. Now, there is less than ten percent failed. We use Execution Manager, and once the execution is done, we spend time on the less than ten percent of failed test cases failed. For the last six months, we have not crossed more than ten percent.

The primary use case is for SAP applications. We do some end-to-end testing of applications. We created a checking and boarding process for a flying scenario, which was innovative. For the back-end systems, we use it to do financial valuations.

We also have a SAP HCM module and SAP FI module where we are actively using Worksoft for our current automation.

We have Worksoft Certify on everyone's desktop. So, when we want to trigger automation, it is a advantage.

We use Worksoft Certify for smoke testing and regulation testing purposes.

There is no porting and it is easily adaptable. If you give it to a functional tester or business analyst, anyone can go, run the test, and analyze the results. It is easy to introduce automation tools to the team members. We have installed Worksoft for all of our team members. We have functional testers, business analysts, and manual test leads. We have an offshore automation team, as well as a functional team.

It is easily adoptable. I don't need to hire a highly skilled/trained person to learn it. For other tools, personnel need to be very professional and need to know how to use that tool beforehand, but very little education or help is required here. They can easily adopt and trigger the automation. 

We are not using Certify in the development area, only in the functional or end-to-end areas, and there is a lot of activity going on in the development area recently. Right now, the development teams are using open source tools, like Jenkins. This would be a game changer if Worksoft could start in the development area.

Going forward, Worksoft will be integrating with Jenkins, which will be great for us.

A part of our CI/CD pipeline, we have to deploy through Cloud AWS. So, it is good to hear that they are moving to AWS too.

The stability is pretty good. I haven't seen any stability issues.

One year ago, our SQL Server database was running slowly with it. Then, we upgraded our SQL Server database to the 2016 version. Before that, we had only upgraded Worksoft without upgrading our database, so we were noticing some latency issues. Once we upgraded our database, I have not received any complaints from our team members.

It is highly scalable and reusable. It is easy for team members to maintain and use with confidence. There is great versatility.

We have 16 applications that we do end-to-end testing using Worksoft.

Worksoft technical support is pretty good. A few times where we upgraded, our team members communicated everything to Worksoft well, and issues were resolved pretty quickly. They are pretty supportive. 

With one of our applications where we do check-in, Worksoft is not able to identify the Java-based application. We raised the ticket, but we were unable to resolve this using Worksoft.

Worksoft already existed when I joined. I am part of the team that maintains it.

If someone in the company wants Worksoft, they call the technology desk. Then, they get the proper license and credentials. The initial setup for these cases is straightforward.

Worksoft did the implementation. Our experience with them was great.

Later on, Cognizant took over from Worksoft.

The product has save us time by not needing highly skilled people, since it is hard to retain experienced development resources. Once we train our functional testers who are running the automation, they can the tool well and stable.

Cost-wise, compared with other tools, it is a great product.

Worksoft Certify is a great tool, which is easy to use and maintain, as no porting needed, development, nor high technical skills are needed. 

Worksoft Certify is a great automation tool for SAP applications.

We do test UI. The UI is very dynamic, as it keeps changing, now and then. It's a very critical, where we are using Worksoft to create innovation. However, our goal is to validate financial valuations. 

The company mainly uses Selenium for CI/CD processes for check-in. Therefore, each time a developers does a check-in, it automatically gives a bill with some pass/fail information, which includes Jenkins.

While we are not using the Capture 2.0 feature, it seems like it will be useful going forward with our end-to-end automation.

It is not for the entire automation process. It is mainly for anything which needs to be automated. Instead of just replacing human work, which it is doing now. It also includes robotic process automation for when we are replacing centralize manual work.

Our clients use Worksoft Certify for end-to-end testing of packaged applications and web UI testing of modern applications.

I have been using BPP and Execution Manager for more than seven years. I sell the product to different clients.

It is a great improvement, especially from my client side, where I have implemented it. One of my clients wanted to replace weekend production and support. They were using mini applications that did not need to work on the weekends. So, it was purely replaced with Certify's automation, so they did not need to come into the office on the weekends. This saves a lot of time and frees employees up on weekends.

The tool itself is a great tool. There are some other tool out there which support only web applications as opposed to only Windows-based applications, etc. It is a huge improvement, as it recognizes complex general applications, where most tools won't recognize the objects within them.

It is very easy to use, but there are some places where they need to improve their security. E.g., the BPP tool is just a URL, which does not ask you for a username and password. Anyone can login and can see it. This is a challenge.

Regarding the Execution Manager, which is good, since it comes with a username, password, etc. However, once you are logged inside, anyone can execute anything. It shouldn't allow this. Worksoft should create restrictions at the folder level structures, or somewhere they know they can align only few people to do the job, instead of allowing anyone do anything that they like.

The stability is good.

The scalability is working well for our needs.

The technical support is good. I would like them to get straight to the problem and provide solutions instead of asking a thousand questions.

Worksoft Certify has a very simple language. Our clients want to save on cost and ensure everyone who is manual testing knows it takes more effort.

The solution has cut our clients' test maintenance time for changes, like patches or system upgrades. They used to take a lot of time until the production validation completed. Now, it is just seconds until it is ready, so they can do executions within couple of minutes. 

This solution has enabled our clients to automate and save time (approximately 40 to 50 percent).

The product has saved our clients' money

The product works well with SAP and non-SAP products.

Use this tool. It is really good.

We're using it to move manual tests over to automated tests to use on major programs and projects going on within the company. We are also moving to Dell EMC Embedded Boxes for an agile approach from a waterfall approach, and part of our new addition for 2019 after merging with a new company. Our role now is to automate as much as we can for usability.

We're doing mostly end-to-end testing scenarios. We use it on SAP and ServiceNow web services, along with SRM, CRM, and various applications.

We have an internal web portal, as well as an external. We did automation for it to do regression testing and build a test space.

The most valuable feature is its time saving. Once development is complete, the short time that it takes to execute a test is invaluable. It saves a number of dollars and man-hours. 

The Capture 2.0 feature is very intuitive, useful, and user-friendly. You can do so much with it now, versus the older version.

We had a lot of issues with the optics changing because they're dynamic. We just recently learned they are already worked on fixing it.

Another feature that they are also working on is being able to export processes from one project and upload it to another project. Therefore, we can change our both structures within Worksoft. We prefer to run and set it up based on business use for separate projects. It's exciting to know that this is coming. We want to be able to move assets from project to another project without having to be in the same project or the same folder structure.

I would like the GUI to be more user-friendly and intuitive. E.g., previously, the buttons were gray. Now, they have color to them and are fun for the user. It also makes them easier to identify.

It is very stable. For example, we used it with ServiceNow. We did automation all last year for an upgrade. Once they did the upgrade, we ran a test after the upgrade and everything was successful.

It is very scalable.

The technical support has been good, but sometimes there are little delays. A lot of times when we need support, it's an emergency situation. However, we are able to reach out to our contacts, and they're able to expedite it, which has been very helpful.

Our upper management wanted to be able to do testing much quicker than what we're actually doing with the manual process. We had to research and find a tool which could provide value for the company.

We had QTP, which wasn't very user-friendly from a coding perspective. There was only a small group of people who could actually use the tool. With Worksoft, we were able to push it out to the business.

I set it up. I just followed the instructions. It was easily done.

We've saved over 80 percent in time savings.

Worksoft Certify was the only tool that we found in which you did not have to know coding or know how to script to create a test. It's more user-friendly, more intuitive, and we also have business users who are actually developing as we speak on major projects and programs.

Worksoft was able to come in on a PoC and actually do automation development on the fly. The previous vendors that I worked with for months, they never could do one test within our environment. Our environment is very complex. It has a lot of policies turned on and off, which is a challenge for a lot of vendors.

Give them a chance, because you won't regret it.

We primarily use it for ERP testing. SAP is the primary application for the client, which we test using Worksoft.

We use Worksoft Certify for end-to-end testing of packaged applications. We have applications, like SAP, connecting with third-party applications, such as Oracle Transportation Management and other web applications. We have these type of use cases implemented and they are working pretty well.

We also use this solutions for web UI testing of modern applications.

With one of our top clients based in the Midwest, we have around 2500 automation scripts. These are not like unit level scripts, they are like big scripts. Previously, our development time for a medium complexity script was approximately three to five days. Now, it has come down to one and a half to two days. 

The turn around time for getting the automation tester familiarized with the tool is very quick, as it doesn't have any coding. It is fairly simple to understand.

The Capture 2.0 feature is very impressive. When creating documentation and automation, it is pretty quick, depending upon the size of the process you're trying to automate or capture. It has brought down the documentation and scripting time by at least 50 percent. It works in the background. It captures only the SAP window, it is not capturing their other work left in the emails or some confidential work that they have on the system. It's pretty seamless. Because it has security, I don't see it getting hacked. It's also pretty stable and works well.

We're really hopeful for the mobile testing in Worksoft Certify going forward. 

I would rate the stability as a seven out of ten. If there is a test script I'm running five times, I have apprehension that it might not be successful five times. The tool might be slow in +xcel or the tool will not be able to identify the same object/property. 

I have never seen the tool crash before.

It is scalable.

We use technical support quite a lot. I would rate them as an eight out of ten. They're pretty responsive, but it takes a long time to get issues resolved.

We usually replace tools, the Micro Focus UFT, with Worksoft Certify.

For a user to install it, the product can be complex.

Make sure you involve Worksoft while you do the installation. Follow their recommendations, which are very important, from framing of fuller structure designs or choosing the right capacity for the server. Their box of recommendations are definitely required.

We have cut maintenance testing time by 30 to 35 percent.

The solutions has saved us money. If we were doing everything manually, it would be two and a half times the cost.

We have a long partnership with Worksoft.

We integrated this solution with Jenkins and Micro Focus ALM for continuous testing. While Jenkins integration went pretty smoothly, Micro Focus ALM initially had hiccups.

The primary use case is SAP applications and customized web-based applications. Now, the company is moving into Salesforce, which is something big for us.

I've worked on multiple projects and our customers are very happy using it. The company is also very happy with the tool. 

We have something called automatic processing in SAP. Previously, the company was doing it manually, which used to take hours to days. Now, we have a script that processes 200 to 300 orders per day.

Everything is integrated with SAP and the web, and it works smoothly. However, we are not yet using it for web UI testing.

• The ease of use.
• It is understandable.
• Everything is integrated.

The Capture 2.0 feature is good. Our clients like using it. It does not take long to create documentation. Once the client knows how to use it, they capture the steps, like a recording. They give it to us for putting on the best practices, and we give them back the results. Efficiency-wise, it is pretty good.

I am looking forward to a feature which will be available in the next release, version 11. The search feature, where you can search for any process that you have created. 

I would like BPP to have more filtering options during the report creation. This would make our customers happy.

The product is stable.

The technical support is very good. They are helpful and provide solutions on time and very quickly.

Our customers come up with their requirements, then we come up with a PoC for them. If it works and their happy, they go ahead with it. 

I work with customers for new implementations.

The initial setup is pretty straightforward. There are a few packages that you need to install, then you have the tutorials which guide you on how to do it. It is that easy.

Our customers are saving money.

It saves us time: approximately 50 percent.

We have cut test maintenance time by 70 to 80 percent.

Experience working with the tool and explore the features of it, because I think the Worksoft Certify is the best and can do anything.

Primary use case is for SAP support testing.

People focus on what they actually want to test and define it in a more detailed way. It shines a light on what they are testing, along with the speed and adaptability.

• Reusability
• Scheduling ability
• The modularization of testing: It brings a more functional level, as opposed to the technical level.

A feature that I am looking forward to in version 11 is a search capability, where you can search within the script themselves for keywords. That will be really helpful.

More than five years.

It is stable from an application standpoint. Sometimes, there is complexity within the virtual machines that they run on, or the hardware, depending on the configuration that their client has setup. Sometimes, this is a bit of an issue to overcome, but I don't see this as a Worksoft Certify issue.

If done properly, it's very scalable. It creates automation which can be run from an SAP point of view. I can create automation which runs in any SAP environment that I want it to run in, as long as the UI is the same.

Their technical support is very good. They have different aspects of technical support:

• If you're standing up a project, they can be with you every step of the way. 
• When you just submit a ticket, they'll respond to you and get back to you within their service level agreement.

Worksoft Certify's codeless scripting appeals along with the use of the Capture feature which helps in those initial phases. It also helps to translate the business requirements to the automation team. If there's a separate team, this is a little better too.

There is recognition in the industry that automation capabilities, like Worksoft Certify, create value for companies, as things are only getting harder and larger. Companies are integrating systems to try to align their processes.

The initial setup is relatively straightforward. The team helps guide you. The technical support is there. It comes down to the training that's applied, and having the people who understand the use of the tool.

The typical range that we see is between 50 to 90 percent improvement in speed capabilities. Another aspect comes with the depth that they're able to test. It increases their capabilities overall to test at a broader level.

Our customers often evaluate Tricentis along with Worksoft.

From an Accenture point of view, our customer chose Worksoft because of our alliance with them. It's the capability that we've built and the knowledge base that we bring along with that.

Capture 2.0 is good for new implementations or new processes. It's not as useful when you get into more mature automation. Its documentation and automation are fairly straightforward.

There is a bit of a learning curve around the Worksoft Certify tool. The best practices, which are lined out by Worksoft, as long as those are followed, then this leads you to understanding the tool and using it in a proper way. If you don't get started off on the right foot, it will be hard to course correct. So, it is vital that they get started on the right foot and understand the best practices. The product's learning curve is relatively good.

Automation is only as good as the functional knowledge that is used in order to create it. This tool works extremely well. Manual testing and automation testing are two different animals. You have to look at automation in a different way. Simply taking manual scripts and automating them, and you're not going to get the full value out of a solution like Worksoft that you could if you were to rationalize the testing and come up with an automated approach.

When you're manually testing, it's about having the least number of clicks possible. Every click for a human is time. With automation, clicks don't cost anything. You might approach the testing in a different manner. It would take a human multiple times longer, but with automation, it makes sense not only from that specific test case point of view, but also from a reuse factor. When you're going to use a certain business process that you've created, then reuse it for different work streams.

We have been working on test automation for SAP-based platform for the last 15 years and leveraging Worksoft for SAP test automation in last 10 years. 

We have some fantastic use cases on how we can leverage Worksoft in large SAP transformation programs, especially for major global rollouts. If you look at some of the industries, like retail and logistics, we can see a lot of consolidation. When there is consolidation, we need to merge two SAP landscapes to a single landscape instance. It is a very complex transformation program. The testing is labor-intensive work in such a large scale IT landscape program. Worksoft has played a key role in delivering these complex integration programs on time, because we automated the core processes using Worksoft Certify. This helped us to automate the testing scenarios in new companies being integrated. At the same time, we test the existing company codes on the existing landscape to ensure that business as usual is not impacted.

If you look at the industry trend, the next six or seven years will be SAP S/4HANA migrations and adoptions of SAP cloud applications, whether it is SAP SuccessFactors, Ariba, or Fieldglass. We strongly believe that WorkSoft can play a pivotal role in delivering large-scale SAP S4 HANA programs since Worksoft can support the implementation of SAP cloud applications. We have delivered a couple of major SAP S/4HANA programs by leveraging Worksoft Certify. 

One of the key things that we have done is introduce test automation on day one in system integration testing (SIT). We don't want to wait until the program is over, then start the automation development. We start the automation right from the build phase, then we are able to deliver test automation on day one to the SIT.

If you look at the customer's experience, we have seen that if you take any testing engagement, whether it is large-scale system integration testing activities, UAT, or regression testing, and have seen that companies involve their business workforce to do lot of manual testing. Then, the business needs to take from the business critical activities and focus on testing. Worksoft significantly reduces the testing effort involved, so they can focus on their business critical activities. 

The company has been investing in a lot of new features. They are changing the trend in market demand, especially with their Worksoft Analyze features. These are important for customers when they have massive test case documentations. In addition, Worksoft is changing their direction towards Robotic Process Automation (RPA).

If you take any Robotic Process Automation, the key things you need are:

• The ability to automate SAP.
• The integrated software applications.
• Support for continuous testing or providing a platform to adopt DevOps.

I can see that Worksoft has been investing in these areas to make the product more relevant for SAP-centric platforms.

Another fantastic thing about Worksoft Certify is using it for end-to-end testing of packaged applications. We are able to automate, not just SAP, but the entire application ecosystem. If you take any company, SAP is the backbone, and if they use SAP ERP, then, there are multiple software applications, where some of them are SAP and some of them are non-SAP applications. Worksoft is one of the tools which can transcend across SAP and non-SAP applications. Non-SAP application include Java or .NET. Worksoft can seamlessly automate these applications.

If you take any ALM tool, Worksoft can be easily integrated, whether it is Micro Focus ALM or SAP Solution Manager. Another important thing is Worksoft's integration with SAP Solution Manager BPCA, this is helping customers to set up a risk-based testing platform. They can do impact assessment and identify test scripts to be tested based on the transports to be applied. Then, import that testing suite into Worksoft Certify and trigger the automation. We have seen that bi-directional integration between SAP solution manager and Worksoft Certify. This makes the product seamless, and if you take any company, they have their own ALM tool and the level of usage of Solution manager is varying. WIth the API that Worksoft provides, it can be easily integrated with the leading test management tools available in the market.

I would like Worksoft Certify to do automation at any layer (the UI layer, API layer, or database layer) and challenge competitors in the RPA industry, like UiPath and Automation Anywhere. 

A lot of customers are looking at testing, not just at the UI level, but testing the application or their ecosystem at the API layer. Worksoft could invest on testing on APIs. There are some open source tools available in market which do this, like SoapUI. 

The stability of the automation tool to run without any performance issues has improved with the use three-tier architecture. The three-tier architecture in Worksoft can help the customers access applications across globe. If global customers have geographically diverse teams, we recommend customers go with a three-tier architecture.

We have seen the customers starting their automation journey from zero test scripts to 8000 or 10,000 scripts. We have seen that Worksoft is able to scale up, then run thousands of test scripts concurrently and deliver value.

One of our customers has about 10 SAP applications with about 70 plus non-SAP applications. So, we automate approximately 80 SAP and non-SAP applications using Worksoft Certify.

A lot of our customers have used the technical support. Worksoft has a very good customer success team, as well as technical support team. When we started our first relationship with Worksoft, we had to build the integration between ALM and Worksoft Certify. So, Worksoft was agile enough to build that integration faster. We can see that the technical support team is fast enough to resolve some of the product bugs for any customer queries quickly. The team is open to build any type of new integrations to support the emerging tools available in the market.

As a partner, we help our customers to invest in the right tools and platform. So, we educate our customers to buy Worksoft Certify. We help them to build a business case or plan and do a joint PoC with Worksoft. We enable and empower our customers with enough details and help them to invest in the right automation tool for SAP, which is Worksoft Certify.

Some of our customers do not have any tools in their landscape. In that case, it is easy to position Worksoft Certify. Other customers already have other automation tools in their landscape in which they are experiencing a lot of pain points with their automation tool. They invested a lot of money in their automation tools and framework, but they did not realize the benefits. Therefore, we help those customers move away from their existing tools to Worksoft Certify. If they decide to build on their existing tool and use Worksoft Certify, it's a very complicated decision. We need to build a very strong business case and we also need to help the customers to migrate the automation test suite.

Worksoft installation is straightforward. They have a dedicated team to support the customers during the initial installation. It doesn't take more than couple of days to complete the installation. A lot of our customers are very happy with the way Worksoft completes the initial installations. 

Also, the professional services offered by Worksoft to help the customer to set up the test automation best practices using Worksoft has been helping the customers to roll out enterprise wide automation using Worksoft Certify.

Worksoft Capture 2.0 can help our customers to accelerate their automation development at least 40 percent faster than any other commercial tools available in the market.

We have seen that the initial Worksoft implementation has helped our customers reduce their testing cycle time by 50 percent. With further continuous improvement, we have seen cycle time reduced up to 75 percent. That is the level of productivity achieved using Worksoft Certify.

If I compare Worksoft Certify with other script-based automation tools, the test maintenance is relatively faster. We save about 60 to 70 percent of the maintenance effort by using it because it's model-based and scriptless. This helps the business team and the automation testers to quickly identify the changes needed to be introduced in the automation scripts.

The typical value realization takes about six to 12 months based on the number of scripts and how often the customer wants to execute.

Worksoft can help you to select the right automation platform, then deliver value quickly.

Our customers evaluate three or four major products along with Worksoft: Micro Focus UFT, Tricentis Tosca, Testimony from Basis Technologies, and then some of open source tools. Some of the customers evaluate all the tools, and some of the customers pick one of the other tools along with Worksoft Certify to make their decision.

There are a lot of tools available in the market. However, if you take any automation project, the lead time to implement an automation solution is typically, painfully longer. It may not give the right return on investments. 

We could have gone with any commercial tools available in the market or open source tools, but going with Worksoft gave us a few advantages. We were able to deliver and implement automation two and a half times faster than any other automation tools. Then, if you look at the total cost of effort to build, maintain, and run the scripts in the customer landscape, it is phenomenally low compared to other tools in the market.

Worksoft gives you return on investment faster than its competitors. It will make your QA organization more agile and nimble.

We recommend our customers be very serious about automation and not to experiment with too many tools. Start with a small PoC or pilot. Involve their business team to articulate the value of what Worksoft can deliver, not just within the IT department, but also the business. 

Worksoft Certify has ease of use, ease of maintenance, and value realization. Automation is not just completing testing faster, it's about reducing production support incidents, after go-live or ongoing production support incidents. It's a collective total cost of ownership and all about delivering value to the IT operations team, IT project team, and the business team.

With Capture 2.0, we have seen the product evolve. Worksoft Capture 2.0 is helping our customers to accelerate automation development. Introducing automation around day one is only possible because of Capture 2.0. When we are in the build phase, we can capture the important screens, whether it is in SAP Fiori apps, the SAP GUI, or SAP cloud apps. During the development phase, we can start building the automation scripts, then start leveraging automation on day one in SIT.

Worksoft can support modern UIs quite easily, such as SAP Fiori and Oracle Fusion. If you look at the modern UIs, it is all about the customer experience, and we have seen that Worksoft can evaluate that modern user interface and ensure that the customer experience is delivered as expected.

The primary use case is more on SAP for automating all our manual business work, as well as doing regression testing. We do end-to-end regression testing on SAP.

The product has worked very well with SAP.

Worksoft is used for manual processes that needed to be automated. This has helped us make our processes be faster and more efficient because there are no manual interruptions or errors.

For my processes, Worksoft makes them faster when creating scripts.

Capture 2.0 is a very good feature, on which we can record very easily and get documentation generation and testing acceleration. 

The product had some UI issues. In the next release I heard the UI issues will be lifted up (version 11), I am excited about it because the product will have more UI features. We are thinking of upgrading our existing Worksoft Certify from 9 to 11, when it is released, as this will be good for the company and help all our users.

The stability is pretty good what I have used sofar with SAP

The scalability is good.

The technical support is pretty good. If you open a ticket, they are able to answer it within 24 hours. They are able to support us.

We previously did manually testing and used other tools. With Worksoft, we can more see the productivity and benefits that it provides. We can also do more testing, making easier for all the users. 

We used the Worksoft for the deployment. They came and helped us. We have a good relationship with them.

Worksoft Certify is an investment. We see value in it more than the money. We see value in it because the user can look at different aspects of testing and what they can do.

This solution has enabled us to automate in order to tremendously save time. Only first time when you are recording and creating the script will you spend some time with it, the rest of the time, you are just executing. If we do one manual process, it could take approximately two hours. The same process using Worksoft probably takes ten minutes.

It help cut test maintenance time too. An eight hour manually project using Worksoft can be done in an hour.

Worksoft is good for SAP to do your automation and testing.

We are not using it for web UI testing.

The primary use case is regression testing and test automation.

We use this solution for end-to-end testing of packaged applications, like SAP, ServiceNow, and some in-house web apps (which we built).

We have already found some defects using the test automation. 

One of the bigger value-adds that we had was extracting data from our warning systems to be inputted into our new learning system.

We automated because of the value. Our business users do a lot of regression testing. They often don't have the time when we need to get the testing done, so this frees them up to do more value-added work for the company.

• Speed, as far as getting tests completed on time. 
• Its ease of use and the value of that. It is a lot easier to maintain test scripts on Worksoft Certify than on other testing tools that we have had in the past.

We have had run ins with some bugs on Business Process Procedure (BPP) and Execution Manager. 

We also did upgrade Worksoft Certify recently to clear up some issues with server fogging.

More than five years.

It scales pretty well.

I would rate technical support as a seven out of ten. Sometimes, we end up going back and forth with Level 1 support. Usually, if we go back and forth, it is email most of the time, and sometimes it is once a day with 24-hour turnaround. We usually understand what they require and trying to give as much information as possible. We still need to tell them to kick it up to Level 2 support or at least have a meeting with them, so we can show them what is going on. Sometimes, it does get a little frustrating, but once we either have the meeting or get Level 2 support, the issue gets resolved pretty quickly.

We had and still use QuickTest Professional from Micro Focus a little bit, but we were approached back in 2008 about getting into Worksoft Certify.

We went with Worksoft Certify because of its ease of use. You don't have to know a scripting language, like with QuickTest Professional. Also, it has ease of maintenance.

The initial setup was not really complex, just a little. We did not understand the full setup; it was just confusing. We really didn't get a full explanation of how things connected together until we actually started to use it. Once, we start using it, things became much clearer.

During the initial deployment where we received execution suite, the necessary information was not really provided. However, it was a fast deployment. We had to meet some deadlines, so people just came in and did the installs, or helped us do the installs over the phone, then afterwards we realized how things were connected.

We used Worksoft for the deployment, who was very good.

We have seen ROI by being able to free up and give time back to the business for other value-added work.

In 2008, we evaluated SAP TAO. The person who came in to do the proof on concept wasn't able to get the test scenario working at all. He had three and a half days to do it, and he couldn't do it. When Worksoft came in, they had the same length of time and were able to get that one plus another one partially working in the same time frame.

Worksoft generally seems to want to make sure you are successful at what you are trying to do. I haven't come across an employee from Worksoft that isn't willing to help. A lot of times you do get that from other salespeople, and that is just not the case from what I have seen from Worksoft. 

The product has a lot of benefits as far as getting testing done. It gives you some value back.

The company is using it across our IT landscape on different projects. We use it to test web applications, SAP mainly. Right now, we're doing a big conversion project. The one that I am working on is a big conversion from an old mainframe system over to SAP. Thus, we are using it to automate all the UAT test cases.

We have also used it in other areas of the company for ServiceNow upgrades and general web design stuff.

For this SAP upgrade or implementation, the business users are creating building blocks in their areas, then they're changing the building blocks into long end-to-end scripts to do a complete end-to-end testing to speed up the UAT. It definitely has a bigger coverage of UAT testing.

• It is user-friendly. We can give the tool to business users, and they're able to use the tool pretty efficiently without a whole lot of training. 
• The reporting features are nice. 
• It is easy to take someone else's work and manipulate it to use yourself. 
  
• It is not heavily code-based, so you can pick it up and automate very quickly.

For Execution Manager, I would like it to be more robust interface and be able to view the remote machines full screen instead of a little window. This would be a great upgrade for us.

I would also like more customized reports without having to print out big reports.

It is a very stable product. When we have had upgrades, it's been pretty seamless. Older versions of our scripts work in the newest update without a lot of rework.

It is very scalable with users. We have business users who are scripting. We have Worksoft developers who do more complicated work. Then, it moves over to the people who do execution and process through Execution Manager, so we have several different layers of users doing different tasks.

The technical support is very good. They are very responsive. When we have created tickets with them, there is usually a 24-hour turnaround time, then we are contacted back. Their interface is good for back and forth communication.

It is not practical to do manual testing anymore due to the volume of it. The amount of data variations is too great to test manually. If we were going to do it manually, we definitely would not have full coverage, where now we can get pretty close to full coverage on our tests.

When you are implementing Worksoft Certify, it is probably best to bring some Worksoft consultants in to help get you setup, and set it properly. Because if you don't get your initial systems, like your folder structures and your naming conventions, set at the beginning, there is a lot of rework to get it to work. We did have to go through that.

We save probably 50 percent of our time. The tool does what it's supposed to do, and we are able to actually work.

Worksoft Certify is a good product. The customer support is really helpful and supportive. They are always upgrading their products to new features, which we like. It is a pretty stable tool, which doesn't require a lot of maintenance.

Our environment has SAP Fiori. They are also doing a HANA implementation. As far as the web, I don't work on that side as much.

I haven't used the Capture 2.0 very much.

We use Worksoft Certify for testing and non-testing. 

• Under the testing umbrella, we use it for regression testing, disaster recovery, validations, and system refresh testing. This has been our main focus. 
• Recently, we have started getting more requests for more RPA-light type of work, which is not testing. It is using the same skills, process knowledge, and tool sets to do work that would replace manual repetitive tasks with automation.

We use Worksoft Certify for some basic ServiceNow functionality or Workday releases, weekly and quarterly releases. For SAP Hybris and ECC, we are also using it for our internal security protocol testing. So, we test Office 365 and Windows 10 compatibility. We test some Excel functionality and file sharing, as part of our security protocols. The most in-depth end-to-end testing that we have is in SAP.

We have done a lot of manual testing. We still do a lot of manual testing for our projects. We've eliminated a significant amount of manual testing with our system refresh, automation, and for technical upgrades where changes are known. However, for projects with new enhancements and functionality, we are having a slower time penetrating into them. With regression testing, we have completely replaced it with automation. 

Now, we are trying to shift as much as we can to start automating processes earlier in the project lifecycle, but it has still been a bit of a challenge. This is one of our stretch goals for this next year. The non-testing area is where we have had the most growth over the last six months.

The freshest example is some of the RPA-light activities that we have done. We have been able to save on a lot of manual work for some very highly skilled, expensive resources. This has been able to free up a significant amount of their time so they can spend more time on innovation and more creative, value-add activities. That's been one of the more rewarding things that we've done, and the most appreciated. 

We run our tests and our full suites every week. We have them scheduled in batches so certain sections of our scripts run every day, then we run them through the whole suite every week. That is how we maintain them by running and repairing them.

To run them every week and make repairs takes us maybe two hours. Because they are scheduled on batches to run, it's lights out. They are pretty hardened at this point, so there are not a lot of repairs for data, etc. If there is a break, it is usually because there has been a change to a process that we were not aware of. Our automated scripts are the company's best business process documentation, as we don't have a business process management tool. Therefore, if anybody wants to know about our business processes, they come to us for something like training or new production support resources. They will come and watch our automation if they want to understand how the systems work.

The ability to run multiple processes at the same time remotely or on a schedule. So, we have some testing that we do every day, and it is pretty much lights out. It is unmanned. We have some virtual machines that run on a schedule. Therefore, it's out of mind testing unless there is an issue. They are very hardened tests. If there is an issue, it means that there is something that we needed to catch, so it is always a good catch. This has given us a lot of flexibility because now we can use those resources in other ways. Besides the basic automation capability, it has been great having the ability to test multiple applications and multiple processes at the same time and overnight, then just receive the results.

I have always appreciated the Capture tool. I'm excited about the new enhancements that have been made to it. I think this will make adoption a lot easier because the tool is a lot easier to use and has more capability. I'm excited, because this is a good time for us as we are expanding into more RPA-light space. It will be easier if we could have some of our more technical resources doing a lot of this capture work. Then if they do something wrong, it's very intuitive to stop, delete that step, and start over. It will make adoption a lot better, and we'll get better requirements and processes documented that we can then convert into automation.

The time it takes users to document, then for us to automate, depends. We have a lot of end-to-end business processes, but they have to go across functional teams. They tend to get passed around a bit. Worksoft definitely saves time. We were literally getting processes on napkins. We didn't care how we got them as long as we got them. They would walk us through really fast, and they would try to show us. We were doing screenshots and trying to record them in WebEx. 

Since a lot of the business process owners are onshore and the developers for the automation are offshore. It was very challenging trying to find a time when they could get together. This product allows them to do the capture on their own time. It can be very quick. They just send it over, then they're done. They don't have to think about it anymore, and it's documented well enough that we then don't have to spend a lot of time coming back to them. It just makes the whole process more efficient.

I would like more reporting in analytics. There is a lot of manual work for us as program managers and test managers which has to do with supporting our value statements. E.g., if there is some way that we could systematically capture how long it is taking for automation processes to execute, then we could insert some notes as far as here is how long it took for them to do the manual capture. Then, we could calculate time saved and have a formula for savings. 

If they have some templates that we could all partner, there are a lot of customers who have created their own, but with the new companies coming onboard. Do they create them on their own or do they try to leverage the best practice within our customer community? There is more we can do here.

There are no issues so far. We haven't had any problems with the tool not being available for us when we need it. 

We had trouble with an upgrade once, but there was an immediate response on their side. We had a very technical resource who helped us get past it quickly. So, there has never been anything which has really stopped us from working.

Because we started early with the tool, it took us a while. I feel like we almost glued things together as our needs became more. The capability that was delivered by Worksoft was there. Therefore, there hasn't been a time when they have been behind me when I have needed something that wasn't available.

I am expecting it will the same in the RPA space. We will grow together because of our close partnership, and if there is a gap, I can work with them to figure out what the best approach is to close it. I think we will be able to stay with the tool for a long time because of its scalability.

I am aware that they have some challenges with some of their support resources, especially offshore which is very common. I don't think this is specifically a Worksoft issue. It always seems to be a software issue, and I know that Worksoft is aware of this and they are trying to make some improvements. 

If ever I need to escalate something, I never feel like I'm stopped. I always feel like there is another level where I can go and get support. We have never had an issue which has gone unresolved for a long time. We try to follow the process, but since our team is so experienced with the tool, if we can't fix it and their support can't fix it either. then it gets escalated up through the chain, getting somebody whose pretty senior with the tools to help us.

Our initial goal was regression testing. It was really expensive. It was throwaway work. We always had to outsource it. It overlapped other test cycles within a project. So, all the functional business folks were busy. It was something that if we left it up to the project resources, they didn't do a very good job with it. We would bring in manual testers almost literally off the street. They didn't know our processes. They ended up having to interact with our project resources anyway. It was just a mess. It was inefficient, clunky, expensive, and the quality was poor. 

We knew that we had a lot of SAP implementations coming up because we had acquired several new companies. So, we made the decision at that time that we needed to automate regression testing. That was our first initial goal, and we've hit that. During our last major SAP implementation project. Our regression coverage was at 90 percent which is pretty much the top you can ever really expect. Now, we are looking at other use cases.

The initial setup was a little rough in the beginning. It was so new for us. The whole idea of automation was new that trying to get the tool setup, internalizing all the best practice training and everything that came so quickly, was a lot to try to digest. Thus, we ended up asking if we could spread the mentoring out across a few months. This seemed to work better for us.

We purchased the software, then we found a vendor on our own to help us with the development.

It has saved us significant time. I have an entire dashboard that I use to showcase to everybody the amount of manual hours that we have saved and how that equates to dollars saving.

On our last big SAP implementation project, we inserted an automation resource into the beginning of the project. Between automating regression processes, data staging, and using our automation to help repair cutover and conversion issues, we saved the project about $1,700,000.

Worksoft has paid for itself fives times over.

We evaluated Micro Focus UFT, Selenium, SAP CBTA, and Worksoft.

My main focus was an SAP automation solution, and Worksoft was really good in that space. They were an SAP partner, but I recognized that I wanted something to be more scalable across other applications, and that ruled out the SAP solution. We liked the price point of Selenium and some of the open source tools, but there are risks to something like that. You don't have as much control, and there are always security concerns. Our internal teams weren't excited about that, as they are not great with SAP. We already had a lot of pain points with UFT. It took way too long to develop processes with UFT. It required more coding.

I would recommend taking a slower, organic approach to automation. It is hard to insert ourselves into the projects. The functional resources, business resources, and process owners don't have a lot of time for us. They don't see the value initially. It is overhead for them and more work. So, you have to bite off small chunks. Show the value, then build up the trust. If you try to be too aggressive and force something down everybody's throats, they will barf.

If you have super strong executive support and it's a top-down, e.g., the CIO says, "You will do this or else." You may be successful. However, in that scenario, your failures will be noticed and made very public. If you take a slow organic approach, where you're just trying to be really helpful and free up time, doing little favors here and there, you build up confidence. Then, people support you more for your success.

Start with the low hanging fruit for the value. Build it up. Once you get a bit more expertise, then start tackling the more complex processes.

Worksoft is a great supplier to work with. They have never pushed back when we have had issues or questions. They have always been available to help us. They put us in touch with other customers that have done something similar to what we were looking to do. They set up user groups by region so we could get together. They facilitate a lot of good discussions. That's why I mentioned we continue to grow together: customers and suppliers. It's just been a great relationship. We don't get that with every vendor. So, when we have it we appreciate it.

It has been very easy to use, but I don't think every automation tool is for everyone. I don't think just anybody off the street can come in and use it. Maybe for some basic stuff, but if you really want to maximize the use of the tool, you need some folks who are really experts in it. 

We were able to really grow when we hit that inflection point: When we transitioned to a different vendor that we had doing our automation development. They were experts in the tool. That was when we started being able to deliver these creative solutions. That was when we were able to see the cost per automated script go down, because they were able to develop so much faster.

While it can be used by everyone on the surface level or to capture the business processes, to get more return on your investment, you have experienced resources using it.

My company brought on Worksoft to assist us with an SAP upgrade in order to make it so the testing was less stressful for business users through automated testing.

We configured Worksoft to setup our SAP testing environment after a refresh by regenerating all of our testing users. This sets them up with the right access, the right approvals, and everything on the back-end so we can do our scripts the next night.

The Capture 2.0 feature recording the process automatically as you go through. It saves everyone a lot of time. It allows the business to give IT the process they go through without having to spell it out so it can be recreated as an ongoing test.

There is a learn functionality where Worksoft learns applications that would be nice if Worksoft expanded its support for other applications that aren't web-oriented.

We have had some stability issues just getting stuff setup. It seems to be pretty stable outside of the web UI. We have had some issues with our testing running into nightly backups of database backups, etc. That has caused some issues, but when we get everything ironed out in a nice, controlled environment, it seems to be pretty good. The web can be a bit finicky sometimes, but it's just that the response times aren't always the same. So, it's a little harder for it to be resilient.

The scalability is really good. We had just talked with Worksoft about if there were any limits in terms of how long your scripts can run for or how many can be running at one time on the database. It seems the only concern is the hardware that you are running it on.

We ran into some issues with the version that we were on during the initial setup. We ran into a bug on one version, then they upgraded us to a new version, and we got hit with another bug. So, they had to put us in a beta. That was a little frustrating. However, besides the bugs that we ran into, the install was pretty straightforward.

It saves us time. It makes it so we are not as worried about changes which are going into the system. We know that we have nightly runs to ensure that things are working. In general, with upgrades, we can always rely on the testing to make sure that certain business processes are working. If they do stop working, we know when/where and can tie that back to changes in the environment easier.

Our eCommerce platform is Hybris. We run end-to-end tests where we place orders in Hybris, then we validate the order in ECC. Additionally, when an order is placed on Hybris, our QA environment has a lot of things which the SAP analysts have to prepare to get an order ready, so it doesn't clutter up the system, such as creating deliveries. Worksoft can do this for us as well.

Hybris is out most modern application. Our point of sale system is web-based, and it is in web form. We are on Azure. One of the things that we've been able to do is use Jenkins to put our Azure machines on business hours. We tell them to turn it off at 5 PM, then we tell them to turn it on at 7 AM. This has saved us about 62 percent of computer operations.

Prior to Worksoft, there were three different individuals within the company who worked collectively for about three years trying to automate just one of our smoke tests for our point of sale system. A lot of them got pretty far, but they weren't able to finish. However, with Worksoft, we have been able to automate six of our smoke tests in four months.

The most valuable feature is having a no-code solution for automation, so our QA team and some of our business users can work on automation. Then, they don't have to be developers.

Most of our SAP analysts use LiveTouch. They use LiveTouch along with prebuilt components. Our QA team uses LiveTouch when they need to add things.

I would like to learn how to get better logs for their support team.

It is fairly stable. We have run into some intermittent bugs off and on that we can't explain. Since they are typical Window's stuff, you just kill them.

When it is unstable, there will be times when a test that we are running in Certify will just stop, and it will say, "Aborted." There will be errors. There will be no explanation as to why it happens. It has now happened maybe one out of 20 times. When it happens, I just tell our QA team to stop Certify and restart it, hoping we don't see it again.

Scalability is throttled in part by the system that you are testing. So, how much testing can your system handle? 

I think they came out with a different type of licensing specifically for testing. Therefore, you don't have to use a more expensive user license, you can use an automation license. So potentially, if we had 100 use cases, we could spin up a 100 different machines, have them all run and be done in five minutes. That would be the goal, but I don't know if that would actually succeed or not.

For the couple of the issues that we were really scratching our heads over, we were in communication with the technical support several times, but they never got back to us. The issues are not critical because they're not really blocking anything. They're just annoying.

We needed a first step in order to get into DevOps. The first step was being able to automate our smoke tests and regression tests. They are tests that we use to make sure that our SAP environments are viable and our point of sale system. We chose Worksoft because they were the only people who we could find which were capable of automating SAP right out-of-the-box.

We needed a faster feedback loop. We have a third-party who develops our Hybris application for us and wanted to be able to hook into their Git repository, so when they push a new version, it would automatically deploy and run our smoke tests. Then, I can know within ten minutes if it works.

The new environment was pretty straightforward to set up. There were four servers, and maybe a fifth one, if you wanted to have a separate server for automation testing.

Some of the integration depends on the subject matter expertise on your team. How well do they know ECC and their processes? Then teaching them how to use Certify to build out their processes. So, on a scale of one to ten, it is probably a seven if you are not familiar with some of the development principles, like looping. If you are not familiar with them, then it will become more difficult to build out processes needed. This is just understanding the methodology of doing certain things, not Worksoft specifically.

The IT department and I worked with a Worksoft deployment engineer because of all of our Worksoft infrastructure. She walked us through setting up the database in SQL and the MongoDB with Worksoft Analyze. It was a pleasant experience. Most of the issues that we ran into were because I did not know something.

Because we haven't built out our suite of tests yet, we haven't saved that much time. However, we know that it will allow us to save a lot of time and money, because once we are fully DevOps, we'll be able to spin up and spin down our systems on demand. Then, we will know within 30 minutes whether the system deployed successfully or not.

We did not evaluate any other vendors. We didn't find anything else that did what we wanted.

if you can use Azure or AWS for your Worksoft infrastructure, then use that for ease of deployment. Once you have your environment, then you can save it using Infrastructure as Code. Thus, if you needed to rebuild or repurpose it, you would be able to do it.

We haven't taken advantage of all the current functionality.

We hardly use the Capture 2.0 feature at all.

Our primary use case is test automation. We have an SRP solution called One1ERP on an ERP platform. We started the automation in Worksoft Certify, getting more than a thousand test cases automated.

Nowadays, we also have automation for a web application in our HR area, so all our HR processes on the platform, Pega. This started to be automated since November last year. We have 40 test cases automated on this application in one year. 

Improvement means for us that we have to be better in quality. Due to automation, you can run every automated test case twice a week. If you do it manually, you do it once per release. This is a quality improvement.

We always have to be more efficient. E.g., if we can directly automate test cases, so if we are using older test cases, then by just switch the company code (national company), we don't have to do it manually. We are able to directly automate the test. This saves money, which is always important for our top management.

We are creating a team an automation team, which will have up to 10 people/colleagues. This will be set up in Bratislava, and those 10 colleagues will work 100 percent on automation. Usually our SMEs do the test execution and try as hard as possible to have everything directly automated, e.g., 40 percent test execution and 60 percent automation. In the end, we want to have a pure automation team who is just doing automation, and have the testers on another team. After the tests are finalized, then they can give them directly to the automation team, or work on them together. That is a mixture. For those colleagues, it's really 100 percent automation

We are interesting to do better, year-by-year. At the moment, we are doing automated regression tests. The next step would be DevOps or artificial intelligence. Our programs should also develop in this way. We want to have automation everywhere where it is possible. Therefore, we need more options for these next steps.

We have used Capture, and it works with Worksoft Analyze. We had some experience last year in August because we started our schedule 1.0 and used Analyze in our tests. We were using the central site for one year with our ERP testing and were quite successful, but this was with Capture 1.0. Then, there was a change to Capture 2.0. In the end, there have been some challenges with the newer version. Therefore, the company testers, the local ones, do not want to use Capture 2.0.

40 percent of the test cases were finalized with automated capture and automated documentation, then the others were done manually. Because we have to create test nodes, we were asked to create a tool that automates documentation, which was Worksoft Analyze. However, with the switch to Capture 2.0, we had some challenges in the beginning. What we did afterwards, together with Worksoft, was we sorted through all the known bugs. So, at the moment, we don't have any known bugs open. We will retry this year in our central test first to find out if it Work Analyze is fine, then if it is okay, we will continue with the local test teams, as well.

On the Capture 2.0 topic, we were not satisfied, because we had a version that wasn't really tested from my perspective. Of course, Worksoft said it was tested, but we found a lot of bugs. In the end, our national company and local test teams did not use Worksoft Analyze because it stopped working. We have ten steps, then on the ninth step, it stopped working and we would have to do it again. So, they stopped using Worksoft Analyze and Capture 2.0. However, this is solved. Worksoft directly helped us to find out what the bugs are, and solved them. Then, we retested it. At the moment, we don't have any open bugs.

On average, it takes one day for Capture 2.0 users to create document. We expect it is faster, but you have to do it several times sometimes. You have to check the documentation that everything is fine. 

There have not been many issues at all, maybe one or two during the last three years. However, it has been quite stable.

Worksoft has good scalability. This is also the reason why we easily can automate for our new laptops.

Usually, we have one contact person for support, who then will contact the Worksoft support. Our contact organizes the people around her. While we have some direct contacts, and most of the time, we receive answers from them. I don't know if there is somebody behind them from the second or third level. 

We are forced to have efficiencies every year. We always get less budget and having to do more. So, we had to have some ideas, and our idea in 2017 was to increase automation. We had automation in place beforehand with eCut. However, in the end, we cut rates 10 to 15 percent. With Worksoft Certify, we had this campaign year with company codes up to 80 percent of the automation rate. This is much faster, and we are finding the defects earlier. In the end, you can save money and have better quality.

In three months, we created 1000 scripts with Worksoft. When the three years before with eCut, we did 450 scripts. This is where we saw a difference.

The initial setup was straightforward. 

In the beginning, it took a long time to integrate Worksoft in our landscape because we have a lot of security levels to fulfill. Therefore, it was not permitted to be install on the security server. It had to be installed through our data center. We learned step-by-step.

Due to security issues we could not use VPN tunnels, we had to have a jump server. This cost us in the end up to eight months. We had four solutions beforehand that were not successful. The fifth one was the jump server. This was the final one and is our solution at the moment.

In Germany, we have a higher security level. Therefore, it costs us by having a longer time to integrate. This is due to our requirements, not Worksoft. The reason why it took that long was on our side. We have a lot of IT departments. With security, social partners, and data privacy, there are a lot of requirements to fulfill.

After that, we agreed to have an automation manager on our team from Worksoft. He sat in Bratislava and all the open issues or questions sent to him, and he answered them, either directly or he contacted the support teams. Then, he assured that those Worksoft issues were solved. He also gave us hints how to use Worksoft, such as naming conventions and how to use it so you don't have a mess in the system.

After we had the automation manager for eight months, then he left. We decided to go with Cognizant, but that was also not successful. In the end, we decided to to have a Worksoft expert from Worksoft, if we need it. For example, we now need an for an expert for two weeks, who will tidy up our system. 

In the beginning, we had an integrator, Cognizant, who created a lot of scripts, but we stopped because we had a contract with them that they would install Worksoft, but it was not allowed. In the end, they just did some automated scripts for us. Our automation team in Bratislava was not satisfied with the quality in the end. We want to have a flexible style with quality, and this wasn't done by our Cognizant colleagues. Therefore, we decided to do it on our own.

From Cognizant, they were sold as SAP and Worksoft experts. Concerning Worksoft, I cannot say if they're expert or not. Concerning SAP, not all of them were experts, maybe one. However, this is always the same in this business. We also had cooperation with Worksoft. Concerning the flexibility of the scripts, we decided to work on our own.

We expect partners too be really good. Otherwise, the company does not see why we should pay for them.

Saving money and better quality, these are the benefits of Certify.

We have seen ROI. This was one of the goals from our top management when investing in automation. They want to see savings in the following year.

We did an RFP in 2016 and 2017. We found that Worksoft is easier to use because our business experts can do the tests. We didn't have to have IT experts, like with eCut.

If you use SAP, you can use this. It is easy.

I am really satisfied with the product. If I ask for support, I get support. I have direct contacts and every issue will be discussed. If we need something, they help us directly.

We did not automate our test maintenance. 

We don't have experience using it with apps and mobile testing, but are looking to add this to our portfolio in the future.

We have been using Worksoft Certify for ten years.

We do automation of both SAP and other applications: One of them is our new grants management system. Originally, they wanted to do everything manually. Now, they highly rely on us to do testing in very short periods. 

We do web UI testing of modern applications. In our environment, we have SuccessFactors, plus SRM through Fiori. 

We also do performance testing and end-to-end testing of packaged applications, like SAP, InfoEd, and OnCore.

The most valuable features right now are the Capture capability and versioning. The Capture 2.0 has a lot of flexibility.

The easy of use and ease of integration of Worksoft Certify are very good.

We are looking for some enhancements on the Capture 2.0 tool. This would give us the ability to control it directly, like we could with Capture 1.0. Right now, Capture 2.0 doesn't really work for our Business Analysts. Because they don't work with it all the time, it's a little complicated for them to stay up to speed on it. With Capture 1.0, we wrote a wrapper to make it easier for them to use, but we can't use that wrapper with Capture 2.0. So, if Capture 2.0 gets enhanced, we'll start using it. For an enhancement, we want to be able to start and stop recording through an API. Then, we want to see how many steps have been recorded through the API.

We do a lot of test maintenance because they are constantly changing the applications. This is one of our biggest problems that are constantly making changes and switching products. For example, we used to use the Supply Relationship Manager. Now, all those tests that we build there will be replaced when we go to Ariba. All the old SAP GUI stuff for HCM, when we got SuccessFactors was thrown out, then had to be redone with SuccessFactors.

It gives you the ability to have a more structured environment for tests. It is not just recording of key strokes. It is more systematized, more like a programming language. That is the biggest advantage for us. Because of its consistency, once the developers know one skill set, they can use it to automate any application.

It scales well. We have eight offshore people and four developers in New York. We can add developers as needed. With the offshore team, we've have at least 20 people trained on Worksoft, but eight people actively on the team now.

We have about a 1000 active users with about 5000 users total. However, this includes all of the employees and their self-service.

The technical support is very good. They have always been a great help. With a lot of companies, you can't even really talk to the developers that someones provides you. With Worksoft, you can get a hold of the developers and work with them. It is all very timely.

The initial setup was very straightforward. We just build the database and installed the client application on the system. There was nothing out of the ordinary about it. It is not like setting up a lot of the IBM products or the Quality Manager, where there were so many different modules. There was just one module to set up.

During our yearly upgrades, we have now gotten them down to ten days or less. We have Worksoft run all our integration tests, where it used to take probably six weeks to do that manually.

As far as maintenance testing, it has saved us time. We find problems a lot of the time that they aren't aware of. This is because we run the tests even though they don't ask us.

We evaluated Micro Focus QuickTest Professional (QTP).

Properly staff the testing team before they attempt to do automation. Be aware that this will not be a one-time overnight process. We tried to automate everything in two months with eight people, and it was impossible.

We have been using it for so many years that we are really very happy with it.

We will be converting to Ariba for purchasing, so that will have to be automated.

We have automated quite a number of test cases in last year.

We have a weekly release. A weekly release means every week that we have testing going on. The particular year where we started is called the asset management area. We could never run our Intuit testing, and we have a whole lot of Intuit testing. This one product helped us pass the testing with the Nighthawk testing, which is working on the Nighthawk manager. That's the one that we use. We can switch it on in the night and run the testing, then come back in the morning and see what has been completed. If there is any fail, we can even analyze it. We can use the evidence document to pass it on to development team to tell them, "This is where we failed."

It has improved on our defect management time. It has improved our test execution time. I don't need to manage these things, just sit somebody down to look at how the script runs. There are a lot of ways that it has helped us.

It is very easy to maintain. With scripts, I can change one line and in one step. Whatever I want, I can do. I don't need to be an expert to use it. Being a manager without a whole lot of technical knowledge, like an automation person, I can change Worksoft using what I learned during the training. That's what I like about it. Anybody can do automation.

I love the Capture 2.0 feature. When you are doing a normal manual testing, go ahead and switch on the Capture 2.0 feature, then capture everything and pass it on to your teams who can convert them quickly into test automation. With this feature, it is saving our automation creation time by about 60 to 70 percent. It is also helping our manual testing time in terms of catching all the evidence documents. 20 to 25 percent time is being saved because of this product and Capture 2.0 feature and what we are receiving with the good documentation.

I would expect more opportunities to automate Java.

I would like it to analyze what we are not using. 

I would like to see the impact analysis integrated with the performance testing tool. We have multiple tools doing multiple items. I would like to have one common tool. 

I'm also interested in load testing automation and whether we can create a script for it, then can we use the same script for my performance testing? 

Stability is quite good when compared to other things. I don't want to say it gets 100 percent rate in terms of stability, because I'm using this for about ten to ten and a half months right now. It has been close to a year, but I'm really amazed when comparing it to any other tool that I have used in the past.

I don't have the words to express its scalability besides awesome. The amount of changes that we have seen are tremendous.

We don't use them frequently, just when we have an issue. I would rate the technical support a seven out of ten. Most of the team is good and helpful. However, I would like them to evaluate the issues a little more sometimes before escalating them to engineering.

We were previously using Micro Focus UFT. The tool was good, and we did not have a lot of problems with it. The only problem was SAP changes a lot of things every time. The frequent changes were causing a lot of issues for us in terms of automation. We were able to automate many things, but the maintenance was a big problem for us. 

1. You needed to have a person who had the coding knowledge to do it.
2. The frequent changes made the scripts useless. Then, we would have to come back and redo a lot of things. 

This is where we were looking for a product where we could have minimum maintenance that anybody can automate. This is the concept why we came into Worksoft.

We used Worksoft team initially. That helped us through the setup and other things. They did a great job. We probably automated about 275 tests in less than five weeks, including the setup. 

I suggest people go head and use Worksoft, along with their services, when you buy the tool. They will help you to onboard it quickly and set it up for you. They will do lot of the automations. They will help you with lot of these practices, then you can take it over from there.

We have seen ROI.

By using automation, it reduced about 75 percent of the time when compared to any other tool. The changes for Worksoft used the same script as the script automating UFT. Manually, running our tool takes about 4 hours, but with Worksoft, we were able to do it in less than 30 minutes. Whereas, the same thing that you had to do would take you almost 55 minutes to an hour. There is now a 50 percent savings in terms of other automation tools and an 85 percent savings in terms of manual to automation.

We have seen more than 40 to 50 percent reduction, in terms of all around time, where we were doing five days a week for a major maintenance testing of our first cycle. We have reduced it down to three days now. So, that is a 40 percent savings which we are seeing. We are not completely automated because we are still doing sampling. We have just automated a good 40 percent of our things. When we go to 80 to 100 percent, it will reduce 60 to 80 percent of our time, which is what we are looking for.

We looked at Panaya, but Panaya is not a desk automation tool. We are using Panaya for some of our impact analysis.

We did try with Selenium and many other vendors. A lot of other tools have a small test director that we tried. The ease in using this tool is very good because our business team can go in and use it. So, anybody can record for me and capture. Then, we have a very small team of automation testers who can convert the information immediately into a reusable component, parameterize it, and do the records sets. In that way, with a very small set of test automated guys, we can do much more.

I recommend this solution already to my colleagues worldwide.

We run this on seven different multiple applications. It starts from SAP, goes to the UI, comes back to SAP for violations, and then goes to mainframe for validation. Then, we use Java Web as a Java. After that, there is another HDM which we try to validate. Also, we are trying to validate a third-party application using it, because we have used a lot of their components trying to do a mock type of filing import/export option with the tool. 

We have used this solution for web UI testing, as we are on SAP Web UI 5.0 right now. We use this very heavily right now in our asset management area. It is very easy to use. The Capture 2.0 together with it is helping us, because we are now able to recognize some objects through Capture 2.0. We also have LiveTouch. This is another advantage where you can use this to capture multiple items at a time.

We use Worksoft Certify to test our SAP System. We have a global instance of SAP, which we started implementing in 2012, and we are still in the process of implementing. We have rolled out SAP to about 80 percent of our manufacturing and distribution. Right now, the remaining projects are a small distribution center and sales offices. We have ongoing projects, and three times a year, we release a new version of SAP. We rolled out SAP to a new geography, and we also added new features for our business users. Thus, as part of those projects, we use Certify to do regression testing of our existing business processes, and we also use it in the project to test new functionality.

When we are rolling out in a new country, we do a configuration for that new country. We use the automation test to test the business processes and prices of that new country. It is sort of semi-automated. Our business analysts generate sales orders from the new country, and we will run them through the shipping orders to cash, the shipping steps, and the concrete steps. Then, we get a set of documents to review. The business analysts review those documents to make sure the order is processed correctly. So, it's not fully automated, but it does help cut down on testing a lot when we roll out to new countries. 

For regression testing, that is fully automated. We have tests where the software checks the results and either returns a pass or fail. These are run as a regression suite anytime we push a change to production.

We do use it for the end-to-end testing of packaged applications, primarily SAP. We do have some plugin applications that we use it to test, which are part of the business process. We use Salesforce for CRM, and we have a custom built eBusiness application. While we don't do extensive testing of those applications using Certify, when the business processes touch one of those applications, we do cover those application with another certified test.

It has cut down on the amount of low level, grunt work that business analysts have to do and can free them up to do more critical thinking. Before we had test automation, we were running tests and relied on people, which was very time consuming. A business process test might have 100 to 150 steps across different applications, and we don't have a single person who has expertise in all those applications. When executing a manual test, we have to balance the test between different people to do their steps. In a typical project, we might have 100 to 150 of these types of tests running. The coordination of the testing process where you have to have different people available at different times is very time consuming and inefficient. What automation has done is cut that cycle dramatically because automation does not have to worry about having to find the right order management or warehouse person to do their steps. The automation just runs through, then the business analyst can review the results afterwards. Therefore, it has been more efficient, cutting our testing part down by almost two-thirds to 75 percent.

Our automation tests are more robust than our manual tests. We found our test lab would grow over time because we didn't have a lot of discipline within the team for manual testing to have a master test which could be used repeatedly and revise as necessary. So, they were creating a new test for every specific little thing that they wanted to test. They were  setting up these manual tests where they had ten to 15 tests which tested the same thing, but not quite. Therefore, it became a bear to manage. Whereas, with automation, because it is more controlled, we have a core set of about 125 automation tests entered into our library. That's in our change control. Therefore, we know exactly what the state of our tests are. 

If there is a new business process or new wrinkle in a business process, we didn't have a defined process, so now we are updating automation tests. The quality of the data that we're getting out of test from automation is much higher than we received out of manual testing. If we know the automation suite is parsing, then the application is working properly. With automation, we have more confidence that if the test is parsing than the application under test is working correctly.

It is fairly straightforward. We have some deep expertise after using it for five years. We have some people who know it now very well. 

This type of marginalization of the code inside Worksoft Certify has been very valuable to us. The ability to capture documentation. We are a technology company and are regulated, so we have pretty stringent requirements. We use Certify to capture screenshots and evidence during testing. We can capture every screenshot during the business process including a document and hand it off to the auditors. It makes defending an audit very simple. We can, if they ask for evidence, produce a document that shows the business process of every step and the screenshots showing all the pertinent data, which has been pretty useful as well. This is the report feature with Certify. When you run a test, you can either have it generate documentation or run it in the background. Most of the time when we were running regression tests, we just run them in the background. 

We don't generate documentation, but we could turn Capture 2.0 on, where it creates a screenshot. As we're starting the test, it creates a screenshot of the application it is testing every step of the way and produces a word document or a pdf at the end that you can then hand off to auditors and show them the actual flow of the process that you're testing. However, we do not use this feature.

One feature that we have been asking for has been to treat tests as code and store the source code for tests in a configuration management tool. Right now, for version control of testing, it's all internally within the tool. If we have a test of a business process and want to revive that test, our methodology now is purely manual work. We go into the tool, create a copy of the existing test, and call the next one: v2. Now, we have two of them and the only way you can tell them apart is by its naming convention.

This is not an efficient way in terms of how modern applications do version control. If this was code, we could plug it into a tool like Git or GitHub to manage of our versioning and branching. The reason why we want to do this is that the application which we are testing branches. When we branch the code, we put a bunch of new functionality on the new version while our production version stays unchanged. Then, at the end, we merge the two together.

From an automation testing perspective, we have to run tests on both. Then, we have two current versions of our test. So, it's a bit hard to manage in the tool right now because you can only have this manual approach where we are tracking it via the name convention. Whereas, a modern way of doing it would be to have our application plug it into a version management tool, like GitHub, where we would store the code and could just pull in the version of the test that was applicable to the version of the software that we were testing.

This is something we have been asking for for a while now. I understand that it's in the pipeline, and it may be in their latest version (version 11). This is something that we will be looking into this quarter.

The challenge that we face everyday for test automation are more internal (people issues). We need change management and getting people to accept automation instead of the technical limitations of the tool. The tool does what we need it to do from an SAP testing perspective.

We have no stability issues. 

Maintenance-wise, we have one system administrator who is not full-time, since it has been pretty stable. We don't change much compared to other applications. This application is pretty hands-off.

We should be upgrading to the latest version in next couple of months.

At our peak usage, we have had seven people working on it, and have had no issues with that. Now, with our current work load, we only have three people. We only run our test suite. It was one of my goals on this project that we had the infrastructure setup, so we could always run our entire test suite overnight. As we built out our library, this meant expanding our infrastructure. Right now, we have 100 to 150 integration tests, and some of them can take ten to 20 minutes to run. A single instance of Certify can only run one at a time. Thus, we have had to think about how we set up our infrastructure in such a way that we can run the entire suite of 150 tests in six hours.

The way that we have done this is to split it up amongst servers. Therefore, we still have extra servers for execution. We have four servers now and run the tests in batches of about six queued up at a time. In this way, we can run our suite of 150 in parallel across four machines and get it done in about six hours. Right now, we do this manually. We do the manual breaking up and monitoring. I know Worksoft has some tools which automate this. This is something that is on our radar to look at as we grow. However, right now, we just manually manage the process.

We have three test developers using it. These are the people actually building tests. In terms of consumers of the test automation, we have probably 35 to 40 business analyst.

The technical support is pretty responsive. We haven't had many issues with it. When we were doing an investigation into doing web testing, we ran into some roadblocks. The team at Worksoft was very responsive. At the end of the day, it came back to technical limitations of a tool. I have been pretty impressed with how responsive the team. They were always able to answer our questions to the extent that the tool was able to do what we needed to do.

It was all manual. For convenience, we used Micro Focus ALM for tracking our manual tests. We still use that as our central hub for our test documentation. We weren't using any test automation tools in IT. Within the organization, we have R&D groups that develop software for various systems and medical devices. Those teams are running tests and code. They are in automation test suites, and I was part of one of those teams before joining IT. However, in IT, before we started using Certify, we weren't using any test automation.

Manual testing was costing us a huge amount of money. We did a double rollout of SAP. We split it over three deployments:

• With deployment one, it was just one division in North America. We had over a 120 people doing manual testing for a period of about sixteen weeks. Add up the cost of that. 
• As we moved into deployment two, we were going to have to test new functionality and also regression test what we'd already booked. If you took the amount of testing that we'd done in deployment one, even if we weren't going to redo all of that, we're going to have to do 50 percent of that. It was going to be a huge manual effort and a sunk cost. We'd put all that money into manual testing and wouldn't have an asset. It would be money that we are basically suspending with no reuseability.

It was a pretty easy decision to convince the team to move to automation because it would be an asset that we could reuse again. Over the last five years, we've shown that we've had a positive ROI on it. The initial upfront cost in terms of licenses, plus all the money that we spent developing tests, has proven it's worth. Now, we can do a regression test suite in ten days as opposed to sixteen weeks.

The setup was very straightforward. We did a proof of concept with Worksoft. They came in and had an engineer onsite. We set them up on a server and pointed them at our test SAP system. They built a couple of prototype tests for us. When it came to implementation, we had an existing prototype that we looked back on. I have a systems administrator on my team, and he was able to pick it up pretty quickly. 

The documentation was good. We did the install on our production system, copying over our prototype tests. We used that as our starting point for building out our library. We also sent out a couple of guys for training.

We were up and running with a functional system within a couple of weeks. The challenge, at that point, came down to training our business analysts on how to use the tool. This took longer than getting the system up and running, which was pretty straightforward.

We did the deployment ourselves. It took less than a week. Internally, we had one system administrator do the bulk of the work.

We ran the deployment on Windows Server. We have two machines: a database server and an application server. Our test developers can logon via Windows Remote Desktop to access those machines. They built all their tests out on that system. Architecture-wise, it is hosted all behind our firewall, but it is all server-based. No one is building tests on their local desktops. It's all server-based, and we can share some of our scripts amongst our team members.

My primary team is offshore. They are in India and Bangalore. Therefore, all of the test development is done there. However, we can access the central test library seamlessly, and  the test strategy for setting up and standing up servers and installing the software was pretty straightforward.

Our ROI is primarily a reduction in testing time. The testing, when we were doing it manually, was 30 to 40 percent of the project's cost. This was a $450 million USD deployment of SAP, and testing is 30 to 40 percent of that cost. We spent probably about a million and a half in test automation, but managed to reduce our testing times from weeks to days. There is a clear cut return.

If we write a new test that's 80 percent the same as an existing test, it is pretty straightforward to reuse the steps from existing tests for our new tests and build upon them. We found that there has been increasing ROI automation as we built up our library. When we write new tests now very seldom is a new test build from scratch. It is normally a variation of something that we already have, so we can turn those around pretty quickly within a couple of days to two weeks.

We ended up buying too many licenses. They were very good at selling it to us, and probably oversold it a little. We bought 45 licenses and have never used more than twenty. However, they gave us a pretty significant discount on the bigger license, so it made sense for us to buy enough that we wouldn't have to go back and ask for more. 

At that time, we had budget to do that. The licensing is pretty straightforward. We have considered using them to do robotic process automation and may still do that. Initially, we were worried that our license might preclude us from using the tool for something other than testing, but when we checked into that, there is no limitation. 

We could use Certify to do robotic process automation, which is basically running a process on your correction system instead of your test system. Therefore, we may do that in the future. 

We also looked at HPE UFT (the HPE automated testing tools) and SAP TAO (SAP's own internal test automation). The reason we pretty quickly went with Worksoft was primarily the responsiveness of the team. The evaluation happened between deployment one and deployment two.

When implementing SAP, we had IBM as our system integrator. We went to both SAP and HPE asked them to show us what they could do for test automation. We also looked around and found Certify as a third candidate. The response from the Worksoft team was far higher than the other two. IBM wasn't able to produce sufficient expertise to demonstrate the SAP test automation tool and same with HPE. I also didn't have a good response from them. We felt, " If this is the level of support that we were getting during the sales cycle, how will it be after the sale has occurred and we have to go to them for support?" Whereas, Worksoft was very responsive. They sent people onsite. They did a proof of concept using our system and data. There was a pretty clear cut night and day difference in teams and companies involved. I didn't get a chance to evaluate the technology of the SAP or HPE solution because their sales teams weren't responsive.

We have a dedicated team of what we call test developers who are specialists in this application. While I don't use the application myself, but they're pretty productive with it. We have a team using Certify for SAP Test Automation and a team using Selenium for web application development. The SAP test development is more efficient than the web test development. For a similar sized test development project where they have to test and develop five automated tests of a certain method, we can turn them around in SAP faster than we can turn them around in Selenium.

Now, it might be Selenium has a higher learning curve than Certify. Or, it is easier for test developers to get good at developing test units using Certify. Selenium is far more technical. Of the two tools that we use, Worksoft is more user-friendly than Selenium.

The technical instrumentation was pretty straightforward. The tool does what we need it to do. The primary challenges that we have had with test automation have been change management, getting the old, greater IT organization to accept automation as a substitute for manual testing. Culturally, within our organization, we put a lot of pressure on our business analysts to thoroughly test the application, and if they have never used automation before, there is a fear factor there saying, "I'm responsible. Then, I want to see it with my own two eyes." 

I recommend expanding, training, and coaching people that automation is just as good, if not better, than manual testing in terms of finding bugs and proving that the system is working correctly. It is far faster, and you will get a lot of your life back. That has been the biggest challenge for us: Telling that story and expanding the use of automation throughout our organization. Now, automation is pretty mainstream and accepted, but that was the biggest challenge for us. It certainly wasn't technical challenges.

We don't use Capture 2.0. We found it easier because we have a large pool of business analysts who are not certified users. Our process for capturing the business process which needs to be automated, therefore we use Zoom Recorders. It is like a WebEx tool. It has a screen sharing device and a record feature with audio. We find the audio is quite beneficial. When we capture the business process, we will have people record in Zoom, annotating with their voice (doing a voice over of what they're doing). Then, we handed it off to the test engineers to build up the automation. We look at Capture some time ago and felt it wasn't as efficient. Capture 2.0 is the newest version, and we haven't really looked at it in-depth. We will certainly reconsider it, but right now, we are not using Capture 2.0 to do business processing.

We use web UI testing to a smaller extent as part of the SAP business process. For a business process which incorporate Salesforce, a field service engineer might order a spare part. This is a post process that spans both Salesforce and SAP. For the first half of the processes, we use Certify. We did attempt to use an in-depth testing of web applications sometime ago. At that point, we felt there were some technical limitations. The project was to use Certify to do comprehensive testing of our Salesforce application. However, we found when we did a deep dive that there were some aspects of Salesforce and proprietary screens which Certify already struggle with. At that point, we decided to switch to Selenium which is the industry standard for web testing. Now, we do most of our tests on Salesforce in Selenium. While Certify has become a lot more capable with web testing since then and the newer versions are better at it, at the time we investigated it, we felt that Certify probably wasn't up to scratch as a web testing application.

Going forward, we will look at Certify again as a web testing application tool since it is more efficient than Selenium. We are finding that it's costing us more to develop a test for a web application than it does to develop a test for a SAP based application. We want to take a look at them again as a solution because it might help increase our efficiency as most our applications from this point forward will probably be web applications. So, there's a lot of work to do in that arena.

With our eBusiness and Salesforce suite, we are not even close to full test automation coverage. We still have a lot of work to do. So, it's worth us looking at Certify again. We're expanding into big data and big data analytics. There are a whole slew of terms around that with regard to testing. E.g., how do you verify that your data's accurate? We are just dipping our toes into it, as we haven't done any model testing yet. That is something that we have to look into. There are a lot of areas where we could use it.

In the last couple of years, we have become an established and accepted part of the SAP testing in the organization. We are a fairly conservative group. Now that we've done the SAP testing, we need to start looking at different horizons of mobile, big data, and web testing where we still have a lot of work to do in terms of building up our automation.

Worksoft Certify is being used to run automated weekly regression tests across some of our primary SAP systems in line with our Change and Release management strategy. These tests run every weekend without fail. The results are reviewed on every Monday morning to check for failures and to analyse if any failures are associated with the changes scheduled to be transported to the production environment that week. Failures (if any) are fixed and the tests re-run before transporting the associated changes into the Production environment.

We also utilize it for projects that need extensive business-user testing and functional testing. There can be testing requirements which come at short notice which can take three to four weeks of manual testing effort. By using Certify, we have been able to bring timescales down to a few hours of automated testing effort.

Our final goal is to utilise this for 'Unify', our new global solution which is currently being deployed, which will deliver common processes and systems to all sites and sectors, replacing all our existing legacy systems which will demand extensive regression testing. 

We have never had any systematic regression testing regime in the organisation. This has helped in building an automation framework across our SAP application landscape, thereby introducing mandatory regression testing across all our key systems and improving the overall quality across our production systems.

From an audit perspective, results generated from Certify (BPP reports) provides detailed test evidence which is also being utilized for internal training purposes/training guides, etc. The BPP reports also provide details on failures along with screenshots.

We have a variety of complex systems in our landscape, one of them being the Openlink Endur which is a commodity trading and risk management system. We are currently building an automated regression test suite to support application testing for Endur.

Our weekend regression tests are performed in 'lights-out; mode. Tests are scheduled to run at a certain time over the weekend using the Execution manager functionality. Usage of Certify has also prevented some major defects going into Production and we have seen significant savings in all manual testing activities as the business users/functional teams are getting more time to perform 'value- adding' activities.

Post our recent upgrade to Solution Manager 7.2, we are currently in the process of implementing the Test suite functionality and the integration of the same with Certify. We expect Solution manager to be the single source of truth bringing out all the results from Certify which is going to be extremely beneficial from an audit perspective. We have already implemented the integration of Certify with HP ALM in our landscape.

Moreover, we have this reusable asset now which can be run frequently to support all our projects and change requests across our legacy SAP systems. Even last-minute testing requests are being accommodated by utilising the automated regression suite without any dependency on business users/functional users for their efforts. We use it across the multiple projects which need immediate assistance and for our weekly regression cycles. To give an example of a recent project which was a major platform migration from a Data Centre in Asia to Europe which needed extensive Disaster recovery testing and Functional testing/User acceptance testing. The initial testing estimate was approximately five to six weeks, however with the use of Certify we could do extensive testing in less than three hours saving many weeks of manual testing effort.

It is very user-friendly with an appealing UI, unlike a lot of other automation tools that we have evaluated. With sufficient training and adoption of best practices, the tool will certainly help organisations to successfully implement an automated testing framework and eliminate manual testing activities.

The fact that it can be used across SAP and non-SAP applications (including web-based apps like Web Dynpro) is a big advantage for us because we have a variety of SAP and non-SAP applications across the Johnson Matthey IT landscape. Being a 200-year-old organisation, our variety of legacy systems have a lot to benefit from the use of automated testing.

Certify has many interesting features, e.g.: 'PRIMO' which is the image recognition functionality is a life saver in instances where Certify standard functionality cannot identify and learn objects within certain legacy applications.

Regarding end-to-end testing of packaged applications, Certify is primarily used across our SAP application landscape and the Openlink Endur (commodity trading and risk management system). We hope to realise more benefits by implementing Certify across our wider application landscape over the next few months.

We have been using the Capture feature, although not the latest version, the initial version, for process captures was used to create our test designs. It has been a life saver in many instances, without the need to spend any extra effort to create test designs and captures. The test steps get captured in the background which generates an XML file which can be easily imported into Certify, creating the basic test structure which can be improvised/modified to make it a repeatable reusable test. In terms of the amount of time it takes users to create documentation automation using this feature, it is the same amount one would spend to do a manual test. While a person is performing a manual test, Captures are automatically generated in the background. We have used it extensively to build our test designs.

We have requested for some minor new features which Worksoft is considering.

The PRIMO image recognition functionality has room for improvement, especially around its ability to work with java interfaces, Execution manager scheduling, etc. as we have observed. As we explore more of our legacy systems, I am certain there will be a need to use more of the PRIMO features to learn the objects.

Overall from a SAP perspective, it works almost seamlessly.

One to three years.

The version of Certify that we are using has been mostly stable and we have rarely encountered any problems. Our weekend regression test failures are often associated with environmental/system performance issues and not related to the stability of Certify. I have been happy with the overall performance of Certify and how it has helped to optimise our tests.

I am confident that Certify can scale to fit our automation testing needs as we expand the current automation testing framework across the wider Johnson Matthey application landscape. We are also exploring options to identify potential areas where Certify can help support mass data uploads, etc. to benefit other teams in their day to day operations.

We have several concurrent users accessing Certify in our environment, primarily automation engineers, test engineers and tech managers.

Our interactions with technical support has not been the best always and there is room for improvement especially with respect to the time taken to respond to cases. However, with the right contacts and reasonable escalations we have always managed to get quick attention on our issues.

We have always been heavily reliant on manual testing and as a result, regression testing was not systematic and we could never think of implementing frequent weekly regression test cycles which was challenging. We decided to go ahead with automated testing and use Certify because:

1. Manual regression testing takes a lot of time and resourcing is always a challenge.
2. Regression testing not being systematic, the quality was very difficult to measure as we did not have a standard set of manual regression test scripts/sufficient documentation.
3. There was a delay in our time-to-market because all the testing was being done manually and there was no way we could accommodate frequent, weekly, regression test cycles. That meant high business risk, that we would have more defects in the production environment/ more associated costs.

We had all these challenges and we started exploring options to mitigate these risks and automation was identified as the way forward, nearly two years ago. We evaluated various automation tools in the market. It was critical that we had to identify a strategic tool which would cater to our SAP and non-SAP application landscape. Worksoft Certify came in as a big winner ticking most of our requirements.

We went through a lot of initial challenges, mostly around internal resourcing issues. Looking back, I am happy to say that we could overcome these challenges and have managed to successfully implement an automation framework using Certify.

Early in 2017, we decided to go ahead with Worksoft Certify post evaluation of multiple automation tools. Our initial engagements with Worksoft consisted of several onsite workshops to explore the tool in detail along with technical feasibility assessments across our application landscape. These engagements were extremely beneficial and it gave us the overall confidence to adopt Worksoft Certify as our strategic test automation tool.  

We did a pilot implementation with Worksoft to see if we could take this ahead on a large scale before embarking on the major project to build the automated tests. Some key processes across our critical SAP systems were identified as candidates for this exercise. Test designs were created with support from the functional teams and taken ahead for automation build with Senior Worksoft consultants and our internal resources. This 7-week Automation Roadmap Engagement exercise was extremely successful and we learned a lot of lessons from it which helped us plan the next big phase of the automation roll out. It gave us overall confidence across the functional and management teams which subsequently led to securing the appropriate budget, etc.

One of the biggest lessons learned from this engagement was around the ways to structure our teams. This led to us going ahead with a Managed Services model with Worksoft. We have an offshore based Worksoft Automation Services Factory team who helps build our automated tests. The team can scale up/down based on our automation forecasts.

The automation deployment is still ongoing. The initial phase was completed across a five-month span. Currently we are rolling out the second phase of the automation build focusing primarily on our global Unify solution and the Openlink Endur application.

Regarding implementation strategy, we followed an agile two-week sprint approach. Our functional teams continuously created test designs and these were fed to the Automation Factory every two weeks, who in turn developed the automated tests. This was the most practical model, which worked well in our environment.

At its maximum capacity, we have had approx. 10 to 12 automation engineers in the Factory team. Our functional teams are spread across multiple global locations and we had between 3 to 6 resources working on test designs liaising with the business users as required.

From a script maintenance perspective, we spend an average of 4 to 5 hours every week with the current asset of nearly 800+ tests.

We have always worked directly with Worksoft, along with support from our internal resources. Worksoft has been delivering excellent services through their managed services model.

We have a res-usable re-runnable asset built which is saving a lot of time across the functional teams/business user community.

Our final goal is to utilise this for 'Unify', our new global solution which is currently being deployed, which will deliver common processes and systems to all sites and sectors, replacing all our existing legacy systems, which will demand extensive regression testing. 

It is expensive compared to some of the other automation tools in the market. However, the benefits and ROI has proved that it has been a good investment.

We have concurrent licenses.

SAP TAO and Micro Focus UFT.

It was critical to identify a strategic tool which would cater to the testing requirements across our SAP and non-SAP application (including web based apps like Web Dynpro) landscape. Worksoft Certify came in as a big winner ticking most of our requirements.

It is a great product and we have not seen anything which cannot be automated till date in our application landscape.

It is important to do sufficient technical feasibility assessments before deciding to go ahead with Certify and equally important to determine the best implementation approach which will work for your organisation. Functional teams/business users' buy in is critical as the test designs cannot be created without their continued support. Adoption of best practices around naming conventions/folder structures etc. will help in easy overall maintenance of the test assets, which will also help with the generation of development and execution dashboards/overall reporting.

I would rate Certify at eight out of ten. Worksoft has always been very supportive and responsive to our needs and this has certainly helped us achieve our initial milestones successfully. I am extremely proud of what has been achieved so far and looking forward to expanding the automation framework across our wider IT application landscape over the months ahead.

We have developed some end-to-end regression testing scenarios that we've found pretty valuable, so we have created a bunch of processes in Certify, linked them together, and we use them every week - sometimes more than once a week - in regression testing.

We used another tool for many years. It became unworkable because of the length of our scripts and how many of them there were, and how they were linked together. They became cumbersome in the other tool. It's much easier in Certify, and Certify can handle them, no problem.

In addition, our organization is implementing Agile, we're moving towards continuous development, and I don't see how we could do that, in any imaginable way, without Certify. We're able to import our changes weekly, based on the results in Certify. And we're confident that because of having tested the main business processes, fairly rapidly, within one day, we can tell whether the imports are going to break anything.

It has absolutely enabled us to automate and save time. The weekly imports of the changes allow the developers to plan on a weekly cycle, which increases the speed of their development. They don't have to wait for a release or anything else, they can test their changes quickly and get the results the next day. They know that they're able to import with no problem.

Finally, it has helped us cut test maintenance time. 

The idea that it's not language-specific is really nice. Keywords and the drag-and-drop functionality are great. The ease of use is superior to anything on the market. It's very easy to integrate. We've been very impressed with the tool. Because we primarily use the configuration with SAP, the integration is pretty seamless. But we have used our own in-house VB app as well, and it's worked very well with that. We've really not seen any problems whatsoever with integrating.

We love the Capture 2.0 feature. It seems to work very well. As for how long it takes to create documentation using it, we do not get into the documentation so much. That end is not as useful to us. But it's built-in if we ever needed it. We're not USDA or anything like that so we don't have a super need for documentation right now.

Also, the ability to work with the data, with recordsets, and plug those into the scripts is very easy and very powerful. We use it extensively.

In the past, when we've tried to automate some of our web apps, it has not been as robust. If there were one thing that could be improved, it's interaction with web applications. The issue we were running into is that it was harder to identify the objects than it is with some of the other architectured applications. That's the nature of the beast with the web as well.

It's very stable.

We have had some issues. We would be unable to log in, in certain situations. But they've all been self-inflicted, changes that we've made on our side that have prevented us from being able to use the tool at times. Once we got those resolved, we were fine.

It's fairly easy to scale, which is a nice thing. Once you create what I will call a sub-process, if you want to use that sub-process in many other processes, it's really easy to use. For us, that's what makes it scalable. You can use that same process wherever you need it. The use of the recordsets just allows us to be able to change the data that make it unique and that make it easily maintained. It's very easy to scale. It simplifies our workflow.

Worksoft technical support is very helpful, very knowledgeable. Whenever we've had an issue, they've responded very quickly. We don't actually have very many tickets, but whenever we've had them in the past, I've just gone into the portal and I get an email back, usually the very next day. I've never had to escalate an issue.

We were using Micro Focus UFT. It became completely unworkable for us. Our end-to-end processes were just too cumbersome for the tool to handle. It got worse and worse to the point where we had to say, "You know what? We have to change tools, this is not helping us." That's when we investigated Worksoft, and we were very pleased with how it worked.

The initial setup was pretty straightforward.

The last time we did it, for the upgrade into version 10, it just required me and one other person on the database side, and then the technical person from Worksoft. It was fairly easy. It took just a couple of hours. At that time, we were just upgrading. The basic architecture was already there so it didn't really require a project plan or anything like that. Once we got it set up, it was just a matter of migrating what we already had in UFT.

There have been several times where it has highlighted an important issue. Some of the defects we've found have been high-impact defects that would've really been costly had they made it to production. There are other times where, because we were able to test with Certify, we knew within a day whether there were gaps in the way we configured a change, things that we had missed that we wouldn't have been able to find if we didn't have the ability to test quickly.

That one defect we found easily saved us $1,000,000. That was just one. Over the years, the amount of money that it has saved us is certainly in that range.

The ability to test quickly has enabled us to develop quickly. We've been able to capture lots of savings in terms of projects that have been delivered faster because we can test faster.

There are savings on a lot of fronts because of this solution.

We would purchase more licenses right now if they were cheaper. Pricing is a little bit of a hindrance.

We didn't look into any alternatives.

It's a highly powerful tool. It's very customizable. It's not a cure-all for everything, but if you want to do end-to-end testing, regression testing, it's a great investment.

We use Certify for end-to-end testing of packaged applications. We have implemented almost anything that touches SAP, using Certify. When C4C came out, the customer application, we regression tested our existing suite to make sure that nothing would break. We anticipate doing the same thing with Success Factor. At the moment, we don't use Certify for web-UI testing, but we're planning on implementing some of that, coming up.

Since it has been up and running, we've had three people maintain it: Myself, I'm the principal QA person, and we have two offshore partners whom I've trained on Certify and they are now helping us execute and maintain the tests. It requires full-time maintenance. We have plans to expand the reach of our automated testing, so we plan on adding more people. We are the only three using Certify in our organization at the moment.

It tests our core business processes but we still have many core business processes that we would like to add to that, to validate if they work, before we send changes through every week. And we would also like to increase the speed at which we can add changes; not just once a week, but eventually daily. We plan on increasing our resources from a manpower standpoint and also from a technological standpoint. We're just going to try to do that as fast as we can. There are a lot of business processes that we would like to add, a lot of apps that we would like to add. The business side has continual, increased demand in terms of things that they are working on and they would like to automate and not test manually, so there's a lot of demand on us right now.

I would rate Certify at nine out of ten. I rely on it every day. It's a great tool, and any problems that we have are hardly ever attributable to the tool itself. It's always some other factor; the way we're using it, or some external factor, which is the problem. It's nice not to have to worry about the tool being the issue. We're very enthusiastic users.

We are using Worksoft Certify to enable our stakeholders to do test automation on the UI level.

We use it for end-to-end testing of packaged applications. We are part of the internal IT department within our company. Most of the time we are using it on our own products. The products and systems that we get are usually preconfigured and prepackaged and we do additional testing, not just for the functionality for the coding that we add to the product, but also on the prepackaged solutions.

We use it for all kinds of applications. Our focus is the web area, including web-UI testing of modern applications. We have two tools in place: our own internal corporate test automation tool, and Worksoft Certify. The latter is a complementary tool, especially in the web area where there are some white spots for our corporate tool which it cannot cover. That was the main reason why we brought in another tool. And for that, it fits perfectly.

We are able to run our test phases faster. Once the scripts, the test cases, are ready and automated, not only are we able to check our systems or landscapes during the test phases, but we can proactively monitor our development and test systems. Proactive monitoring of our systems is very important for us and was not possible before because manual testing is just too time intensive.

Worksoft Certify helped us to increase time savings. We didn't start test automation in general with Worksoft Certify. We did automation before with our own tool, but it helped us to increase the coverage of test automation and to increase the time savings.

We had a success story with two teams. For the execution of the scripts, we had time savings of 82, 88, 95, and 90 percent. And for the speed, it was between nine and 21 times faster than manual execution.

It's not necessarily saving us money, but it's helping us to free up the capacities of our end users to work on other stuff. Instead of doing testing for two days, they can work on bug fixing, developing new features, etc. That person still gets the same paycheck at the end of the month, so it's not saving us money, but it increases the value of our products. It increases the quality of our products. The reason for that is we are not customer-facing. We are dealing with internal teams and internal products. We are not selling anything to the outside. We are with the internal IT department. For the development teams and the sales team or the consulting team it might be different. But we are not really going out, selling our products and getting the revenue for the company. This is done by other teams.

It's script-free, which is really important for our end users because we are usually dealing with colleagues who are not developers and who do not always have the technical background of developing and scripting. It's very useful that there is a nice UI and the tool is script-free.

It covers all of the technologies we need to cover.

And one big advantage of Worksoft Certify is its integration with SAP Solution Manager, the test suite of Solution Manager, with the certification. That is crucial for us since Solution Manager is our test management tool of choice.

There are a couple of small things, technically, that could be improved.

Features we have asked for include single sign-on. It's a bigger project to make sure that our end users do not have to store passwords, usernames, and the like, for the different tools we have. 

We are also working on an additional integration with another tool that we have in place for lights-out testing. That's ongoing at the moment.

Another idea we brought is that the definitions for the objects need to be automated. They need to be recognized automatically by Worksoft Certify instead of changing them back and forth manually. This is also something that Worksoft is currently working on. 

Updates, in general, is a topic that we are working on with Worksoft on a regular basis. For new products, for new UI technologies when they come out, the test-automation providers need to update their definitions to make sure that the objects are recognized correctly.

It is pretty stable. After upgrades, we usually don't experience any big issues. Of course, it's software, so here and there we find bugs, but nothing crazy, to be honest. The availability of the system is pretty good, almost 100 percent. I don't see an issue here.

At the moment we don't have any issues with scalability. We have about 300 end users working with Worksoft. On infrastructure, it's split. We have a WTS environment, a Citrix environment, for those colleagues who want to use the prepared environment. We have other colleagues who are using the client on their own machines, on their own laptops or desktops. The only thing that we saw at the beginning which we need to change in the future is that, due to the latency, we cannot use clients in the US, for example, while having the server in Germany.

If the latency is over a certain number of milliseconds then it is basically impossible to do automation. That was one of the main reasons why we set up the Citrix environment at the very beginning.

We are still in the phase within our company, or within IT services, of training and spreading the topic of test automation, overall. So our coverage, at the moment, is not the entire organization, it's only the IT department. Once we have done this - and it will take at least another year - we will see if we spread using Worksoft and our internal corporate tool as a combination, or tool ecosystem, further into the organization. But this is not our not our team's responsibility so it's not really in focus at the moment. We are pushing for test automation in our teams and there is still a huge demand for training and new teams coming into the topic of test automation.

In general, technical support is good. They are collaborative and responsive. The only thing I don't like - and this is the only complaint I usually have for Worksoft - is that the first-level support is not always the best for working on topics. We sometimes need to escalate to second-level support and then we know that we are getting a colleague who is aware of the issue and is not just playing for time.

We already reported this to Worksoft and asked them to find another way or to educate the first-level support or to make sure that the tickets go directly to second-level support if they come from us. The guys on our end who are reporting the issues sometimes know more than the first-level support.

When it comes to second-level support, we are happy. There, we know we will get the help that we need. The colleagues are responsive and very helpful and, from a quality perspective, they are very good.

In the beginning, there were some issues with the integration, it didn't work the way we wanted. We spent some time with the Worksoft team, with the support and engineering team, in adding some enhancements to adapt the solution to our three-tier Solution Manager landscape. But that worked very well.

We have a very good collaboration and relationship to Worksoft. For example, every two weeks we have calls with them. We'll provide feedback and they take it seriously. They usually provide us with updates, with enhancements, with new functionalities that we need. That's working pretty well.

We started with manual testing and then we started the test-automation initiative. We started with our internal corporate tool and then, as I mentioned earlier, we figured out that we could not cover everything with that. At the end of 2015, we started to check the market. We did some PoCs and we decided to go forward with Worksoft Certify.

There were a number of reasons we went with Worksoft Certify. The Worksoft team did a great job. They came to our headquarters and did the PoC, showing that the tool is suitable for our needs. They did another PoC with our operations colleagues who were running the regressions testing in Singapore. And then there were the technical requirements that we had. Worksoft Certify was able to cover all of them, some of which I have mentioned already: Being script-free, being fully integrated in Solution Manager, and being able to script in a modular way. And finally, the integration between our own internal tool and Worksoft Certify was also important.

The initial setup was pretty much straightforward. I cannot give you too many details because I did not take care of the implementation. But I know that it took us about two weeks to set up the whole infrastructure. It was not really difficult. And we had very good support from Worksoft, from the support and engineering team. They helped us in setting up the database, setting up the connections. It was not a big deal. In total, within two to four weeks, everything was working fine.

On our side, we had a couple of staff members involved in the implementation because our team is the application owner. We had to involve two more colleagues from the database team because we don't have all the authorization stuff, for the databases area, for the servers etc. In total we had about three staff members involved, but not full-time. It was about one work-week for each of them.

Maintenance is done by myself and one of my colleagues, with the help of our database and server/infrastructure team. We don't have authorizations for everything and we are not database experts. There are three or four staff members taking care of maintenance, as part of our job; it's not a full-time job, obviously.

Whenever we need to do a full upgrade, when we need to plan the downtime for the production system, we try to make it on the weekend. I also already recommended to Worksoft that it would be nice to have something like an offline update where the system can be upgraded or smaller changes and fixes can be included without having full downtime. For an upgrade we usually need two to three hours. Afterward, we do a bit of testing, so upgrading takes about half a day.

It was just one or two people from Worksoft and three guys on our side.

We get feedback from all areas that the return on investment is there. Not just regarding time savings, but also cost-reduction. The return on investment in one case was reached at something between five and six runs, which is pretty fast, especially in an Agile environment.

What is also very important for us here is the avoidance of human error during the execution of tests. Usually, if someone is sitting in front of a laptop and doing testing eight hours a day, he or she will make some mistakes. This does not happen with a tool. Another important factor for us is the availability for testing. Usually, it's pretty hard to plan a test phase to bring all the testers to the table and get the time blocked off for the test phases. For the tools, we just need the systems up and running and then it's a matter of minutes to set up the test plans and to run the tests.

At the moment we are rolling out Execution Manager.

At the time we decided to bring in Worksoft Certify, we looked into two other tools. The key difference was that Worksoft was script-free. That was not the case for other tools. And the full integration to Solution Manager was one of the key differentiators between the tools.

If you have done a market evaluation and have decided to go for Worksoft, my advice is to go for it. I would definitely recommend Worksoft Certify as a test automation tool.

The feedback that I get from our stakeholders is that the tool is pretty simple to use. What we usually do is a two-week training, not full time, where the total is about three to four business days, 20 to 25 work hours. From there, most of our colleagues can start working with the tool. Of course, they have questions later on, some difficulties when it gets into special activities. But overall, the tool is easy to use. It's generally found to be intuitive.

In terms of cutting test maintenance time with respect to the scripts, that has not happened. If you need to adapt your scripts, automated test scripts are much more complex and more effort-intensive than manual test cases. But this is the nature of the beast. It will happen with every tool. If a screen changes, if a system changes, then you have to adapt your script for manual testing. For a manual script, you just adapt a Word document or an Excel sheet or the like. But if the process flow changes, you have new windows, new options, then you have to adjust your script for each and every provider that you're selecting. The maintenance of scripts is something that I always discuss with my end users and should never be underestimated.

We are not using the Capture 2.0 feature at the moment. We are planning to use it in the future. But at the moment, due to the heavy workload on our plate, we haven't had the chance to look into and to roll it out. We are familiar with the concept of Capture and it's a very nice feature because it makes the collaboration between business and IT much easier, and business can be involved in test-automation topics and activities as well.

We have three roles in our environment. We have the key players, who are the project managers, the persons responsible for test automation overall in the respective teams.
Then we have the test automation engineers who are responsible for creating test scripts and to maintain them; sometimes they run them as well. And finally, we have the executors, the ones who are running the scripts, checking the details and, if something is not working fine, going back to the test-automation engineers and asking for support and help.

I rate Worksoft Certify at nine out of ten. I'm happy with the tool, I'm happy with our colleagues at Worksoft. We have a very good relationship, we can bring up everything. There isn't much I can complain about. I'm happy at the moment with Worksoft.

We use it for SAP and ERP. I think from the SAP and ERP point of view, it is very good.

When we used it for web based application, so it was quite poor.

One, it's module based and it's giving functionality. That is, first of all we can really implement it as an ERP (Enterprise Resource Planning). We can utilize our concept and our functional people to work with the tool and without having a lot of technical knowledge, so they can really work with the tool with small technical knowledge. So that's really a big thing for us. We can use the tool with our existing people so that we don't need to have a new technical expert using the tool.

It is poor for a web based application. We are living in very integrated organization are most of the companies out there. There's the big companies we have these organizations that we work with, where we have plenty of different services which are very much connected. So, we are really looking solution which really can support all the different services. We really need to focus end to end instead of stand alone case. 

Whatever reportings are there, so they're really not very user-friendly. So there is a lot of technical data instead of user-friendly data. It would be wonderful to add web support, I would really like to see this in the future.

If I would evaluate it on a scale of one to seven, I would give it a seven. 

It's not very scalable. 

They struggle to provide an accurate solution.

I think it was not really very difficult. It was quite easy to set up the tool. The architectural tool was really easy.

We define certain different criteria. We check if the vendor is giving support for the tool, and we also evaluate the price. Of course, we also take note of the ROI of the solution.

This product is a good one, if you are really focusing as an ERP. We really need to have a solution which is really end to end rather than just focusing on key product. It's good to be aware if there something that can support you end to end rather than one specific product.

When talking about continuous testing, so then the tool is not really good at all because you need to purchase some additional tools. For example, the Execution Manager and  other tools, which are an additional cost. 

Another added cost is the payment necessity for reporting.

We are using it so far for regression testing. We have reduced 5% to 10% of regression related issues. This is a huge impact on our organization. So that's really quite good improvement in our eyes.

Speed at which it processes our testing scripts, detail of testing results for audit validation, documentation can serve as training document for new employees.

Prior to implementing Worksoft for our testing, the scenarios required eight employees over two weeks to complete (waiting for hand off of next steps, employee availability due to meetings, etc.

After implementation of Worksoft, the same test scenarios currently run in approximately 45 minutes – hands free! We have also implemented Worksoft into our SOX Control documentation processes.

Approximately seven years.

No.

No.

Excellent. They always went above and beyond to timely support our needs

Setup of the software was extremely easy. Setting up the scripts was complicated at times (due to our customization of our SAP clients).

However, most routine transactions were handled with ease.

I am not familiar with the pricing or licensing fees.

No.

Make sure to be prepared for the amount of time it will take you to complete the testing scripts.

Have a dedicated team that performs your test script setup and execution of these tests.

Include your business users to ensure the processes are clear and accurately reflect their business processes, not what you think their processes are.

Well, obviously it's the test automation piece of it. Creating our scripts and being able to execute our SAP scripts in order for us to be able to do comprehensive regression testing of our SAP system.

Well, it's definitely reduced the number of defects that we have. Before, when we were manually testing everything, it was unclear to us where changes that we were making, where the impacts were in our system. I mean, we've got very smart people that work here, but they were not 100% on where the impacts were going to occur with their changes.

By automating our scripts, we were able to do more comprehensive testing and therefore eliminate the number of defects that were be introduced into our environment.

There is an impact assessment piece to Worksoft. I think that's one of the pieces that we still struggle to use.

What the tool does is, you can analyze a transport to know where your impacts are. I think that's one of the challenges that we still have -- is just getting a better handle on how to use the Impact Assessment better for us.

In our regression testing, all that, we've got a pretty good handle on that. We identified things. When we run the analyze of a transport, we're not...we've found other ways to do things better.

Four years now.

No. It's been very, very reliable. The only issue, we've virtualized everything. I think the only thing, the only real issue we encountered was with our virtual machines. It was not a Worksoft issue.

It was on our side -- once we worked with our sys-admins to investigate what was going on, they were able to resurrect the issue and we were fine. It was not a Worksoft issue at all. It was something with our virtual environment.

It's been very scalable for us;

Depending on how many licenses, it's all seat licensed. It's not a concurrent license. If you don't have people logged in, more people can't use your licenses from your pool.

You assign those licenses out to people and as you continue to grow, like I was talking about with our virtual machines, we do everything in a virtual environment for all of our Worksoft work.

As we've needed more virtual machines, we've got an image of our Worksoft clients and we just add them into our environment.

3 out of 10 -- You submit an email, so there's not really a hotline that you can call to talk to somebody on the other end.

I had familiarity with several tools out there. Turnkey is probably one that you guys have looked at. I've done a lot with the quality center products in the past...Quicktest Pro.

There's other test automation tools out there. I've got a background with test automation and Worksoft...really it was between Worksoft and Turnkey. Worksoft, we just found ,was going to better meet our needs.

Yeah. That was actually pretty straightforward. When we set up our environment, they gave us a technical support person that really walked through with our system administration team to get everything set up in our environment.

As far as training, they have a very comprehensive training program. I'm sure it's all ... I'm sure it's very good. What we did and what helped us to be very successful is we went through and did live test cases as part of our training.

Instead of spending all the time going through the books and learning the tool in and out, we went through and we got right into creating our test automation with the trainer that they sent out. That hands on experience was, it was priceless. It helped us to be successful and get going with creating all of our test automation.

Make sure...before you start any of your test automation, make sure you have the appropriate support from your business.

What I mean by that is, you have all of your business processes documented. There is a lot of prep work that goes into creating your test automation to make sure you're creating...you're creating your scripts in the most efficient manner. A lot of this testing is done based on your business process. You need to understand your business processes.

For my team the most valuable feature of Certify is its flexibility and ability to include logic in the test steps. Our previous test tool could only handle linear test processes.

The build/maintenance times for testing are much faster. It also allows us to build tests for applications our previous tool could not do.

The faster build times allow us to respond quickly to ad hoc requests from customers that were not possible previously.

The user interface isn’t the best. It is functional, but the different areas look too similar so it is not immediately obvious which module you are in. It does get easier as you become more familiar with the application.

We have been using the solution for seven months.

We have seen one issue, but it is minor and reproducible so it is easy to work around and only appears in a very specific environment.

We did not encounter any issues with scalability.

The technical support is great. It is very fast with a friendly response. Everything from the initial install to tickets is handled quickly.

Our previous tool couldn’t automate many of the applications we needed to test. Certify also allowed us to increase the number of tests we could maintain with our current team.

The setup was straightforward. We are running fine on a single applications server that houses the database and license server. The clients themselves have almost no configuration.

I was not involved in the pricing negotiation.

We evaluated Tricentis and Selenium.

Expect to include some funds for training or have someone in the QA group who is familiar with the product.

You can start making simple tests immediately, but to fully utilize the tool you will need someone with experience. It also requires some organizational discipline.

The tool is extremely flexible and can fit whatever process your team wants, but everyone needs to know and follow the process or you will end up with a confusing tangle of tests and data.

All the different kinds of recording features (learn) on the different technology platforms such as Java, SAP, and HTML are its most valuable features.

It facilitates 80% of our regression tests.

We have had problems with an external cloud service (technology is Adobe Flex).

I have used this solution for two years.

There were no stability issues.

There were no scalability issues.

The technical support level is quite high. We are very pleased with its flexibility of services and response time on issues.

Previously, we just had manual testings.

We had quite a slow implementation. The process has given us time to learn and do things in the right way. We have also received good support when we experienced any issues.

The consultant's rate is very high but Worksoft is quite flexible.

We did do a PoC and compared it with other products from SAP and HPE.

Take your time, do a business case and plan from the beginning as to how your organization will work with the test automation.

In the beginning, there will be different technical issues between Worksoft and your business application, which will take some time to fix. You also need good internal support from the people who know the business application. Support from the managers is a must.

Provides capabilities in testing SAP ERP. It’s easy to create test cases/processes using Capture/LiveTouch functionality.

We have been able to increase our overall testing maturity and test automation coverage. We have a large ERP program ongoing which requires a lot of testing. Our goal is to automate the majority of system integration test cases with Certify. This brings efficiency and quality to the program and also supports the continuous services in the future (less time needed for regression testing).

Testing of web based applications can be really cumbersome at times. But this is partially due to the way the applications themselves are developed.

We have been using this solution for one year.

In terms of stability, the new version (9.0.3), especially the client and its UI, seem more sluggish compared to the earlier version (9.0.2).

There were no scalability issues.

We have not used Worksoft’s technical support so far. We have been in “project mode” and received direct support from Worksoft team related to configurations and also for our version upgrade.

We didn’t have any other tool in use.

The initial setup was straightforward. Afterwards, we also setup Certify on the Citrix environment to be used by our partners.

What do you advise others about pricing and licensing? We will have a large number of users so we chose an enterprise license.

Plan your training schedule well. The tool is pretty easy to learn, but you will need some weeks of hands-on usage before you learn to use it efficiently. We had a too long break between the first trainings and the actual start of using the tool.

A lot of the features in Certify are valuable, but a few of those are very useful and make automation easier.

The Content Merge feature allows us to copy the Certify items from one project to another, either in the same database or in a different database. The feature set of Content Merge allows several abilities, so as to control the behavior of copying these items from one project to another.

The Recordset and Layout features help us in increasing the test coverage and reducing the test cases count in a simpler way.

Other features such as Certify learn, HPE QC integration also come in handy.

SAP automation was very easy with Certify; it reduced the automation time and also, the creation of the test cases was easy.

Web automation can get a little better.

I have used this solution for two years.

As of now, there are no stability issues.

As of now, there are no scalability issues.

I would give the technical support a 3/5 rating.

I have used Tosca Testsuite before moving over to Worksoft Certify. It depends on the client's requirements that we move over to different tools. So, nothing is specific.

It was a little difficult for installing, but the setup was not complex.

I am sure on the pricing part for Certify. One will need to connect with the Worksoft team for that.

If you are working on SAP automation with Certify, then it’s the best tool to work with. Other technologies too work well but are not that stable.

In the near future, I guess everything will be sorted out and this will be the best automation tool.

Certify and Analyze are the most valuable features.

It has greatly improved testing time and has found defects before production.

• Better Silverlight automation

I have used it for a little over a year.

I have not encountered any deployment issues.

I have not encountered any stability issues.

I have not encountered any scalability issues.

Customer service is good.

Technical support is good.

We did not previously use a different solution.

Initial setup was straightforward.

We worked with a consultant for implementation.

• Expanded test coverage/fewer defects

It is good for it's price and functionality.

Before choosing this product, we did not evaluate other options.

It is a great product with a good team behind it.

Parameter-driven action steps (no programming): Able to interact with all application objects, as well as execute all logical, mathematical and text manipulations, with clearly defined parameter-driven action steps. These features, and the basic structure and framework, eliminate the confusion of building a framework for every test. The framework is inherent with Certify.

Certify is built on a database where all components are indexed for easy usability and reusability. This concept is difficult to manage with most other tools where tests and components of tests are very possibly on different servers, workstations, and folders.

Process and data RecordSet coupling feature: You can import lengthy spreadsheets and, with the framework of Certify, then extract only test data needed by attaching that one imported spreadsheet RecordSet to your executable process and, with just one action step in the process, write to the RecordSet, and build only the test data you need. This feature of a Certify process does automatically read and process the next row of data in the spreadsheet RecordSet without any programming. And the feature of a Write action will write all the variables associated with a RecordSet with just one write action step.

Point and click on application objects will automatically create an action step referencing the Object's Attributes and with a variety of actions automatically promoted for selection based on your application object type. This means minimal guess work on how to interact with an object.

Easy transition during execution between application development disciplines. Go to SAP, Web, Java, mainframe, or any discipline, all in the same test execution.

Easy interaction with DOS files during execution.

Of course, one of the benefits is, we are able to complete and maintain more automation.

The biggest benefit for me has been that, because the automation is basically 'event-driven-executable-documentation', it has been easy to enhance my lengthy end-to-end tests with restart capability.

It has also been easy to create automation as part of the test to find, or create data as part of the test execution. This has enabled us to instantly run a test again without the delay of manually finding and populating input RecordSet data. When you are on the clock to complete regression testing for production implementation, this is critical; that you don't have any delay to rerun.

Certify has been in perpetual improvement mode since it's inception.

Although I don't think it would be easy, being able to edit and change an action and logic flow in mid-test execution would be nice. Of course, with event-driven execution, this would be a bit of a reach.

I’m not an expert on compliers however I do have a background as an Assembler coder. I believe most applications now are event driven and most, if not all, automation tools are Interpretive, compiled at execution time. Certify is Interpretive as well as having only parameter driven source (no programming syntax). During execution, the processer is using the Interpretive version of each step, not the actual source created by the developer. Linking back to that original source and maintaining executable sync and content would be a stretch. This is not a restriction of Certify but of any Interpretive complied automation tool. Certify does have a feature to execute just portions of its source, during editing; however, I believe that is more manageable than interrupting execution to then edit the source.

I found it difficult to conjure up enhancements to this tool possibly because I have intimate experience with Certify and have always found success using one or more features. However, there is one enhancement request that might have merit. Allow a Variable to be used in the Label parameter of the Execute Process Action. Certify has an Action step ‘Execute Process’ that has a Parameter for ‘Process Name’ and a Parameter for ‘Label’. This Action allows you to execute a sub-Process and begin that execution at any Label in the sub-Process. The Parameter for ‘Process Name’ can be a Variable containing the name of a sub-Process which can be changed based on application and logical response during execution. However, the Parameter for Label cannot be a Variable. If it could, it would provide yet another nice structural feature of Certify.

I have been a test automation developer since about 1992, but with Worksoft Certify since Jan 2000, just over 17 years.

Because of the unique framework of Certify and the extensive object, math, text and logical parameter-driven actions, there is nothing I haven't been able to accomplish with Certify and all without traditional programming or scripting.

Of course, with experience, you learn that most of the stability issues are environmental; meaning, it is usually the environment that has had to be tweaked.

I have had once where a threshold has been reached in the database for updating a component, but that basically took a phone call.

Worksoft has done a good job keeping abreast with browsers and various different development disciplines. In addition, database size has never been an issue. In fact, as test automation requirements have grown, being built on a database has certainly enabled better management of the automation reservoir of tests, as well as all the components, discipline interfaces, requirements, results, users, security, etc.

Customer support is always cordial and helpful. There is an annual Certify User Conference, where you get to actually hear presentations from customer support, technical support, and experienced users. You can actually meet and speak with the members of technical support, engineering and management on any issues.

Early on, I needed math capability and literally within a half hour, I was sent an update to allow math using a specific Set action. Now Certify has extensive math, data math, and many other numeric and text manipulation actions. Of course, since Worksoft has grown, they have instituted a formal way of communicating, which has been very successful. I am able to report any issue with an immediate response that it is being forwarded to the appropriate engineer. If I have a high priority, I get immediate attention. That has happened once during a release update, which was addressed and handled within 24 hours, and it was caused by an in-house, user-developed map.

As mentioned, I evaluated Certify against a tool I was using, AutoTester. I was also using WinRunner, Rational Robot, and Seque's Silk at the time. Those names have changed over the years. However, it was immediately evident that none of them could compare even closely with the productivity I could get with Certify. Also, because there is no syntax involved with Certify, process and actions are more like executable documentation, making maintenance much easier.

Initial setup is pretty straightforward now. Early on, I used to write my own .bat files for the install. Now, they have good documentation and you can either use a .exe or .mis file for the install and upgrade. And, I have found that support is available immediately during any installs or upgrades.

Because I have been using Certify for so long, I do my own implementation, except for local DBA support for the database. However, for brand new implementations, Worksoft has an excellent professional team for just that purpose.

I can't give you any specific dollar amount on the fly but, for instance, I was at the Certify Users Conference in the fall of 2016 and the manager from AAA Southern California mentioned that the system I architected and developed for them while training two of there business analysts back in 2006 is still being used. The system is a mainframe system for applying for insurance with quite involved requirements.

ROI tends to be substantial because business analysts can be trained in just days to be effective with Certify. With almost all other tools, you need programming experience.

Cost is always based on need. I haven't found a vendor that could give me a straight answer unless I just wanted one or two copies of a software. And, of course, the more licences you need, the more cost reduction per seat you can get.

Before choosing this product, we evaluated other options, as mentioned. We were using more than one other tool at the time at Charles Schwab and we chose Certify for it's flexibility, quickness in development, and easier maintenance.

During an R&D project to find a better automation solution at Charles Schwab (a major investment company) in January 2000, I evaluated Worksoft Certify. At the time, I had about 10 years’ experience using popular test automation tools. For the evaluation, among the considerations, I measured Certify against an automation project I recently completed. I took an IBM mainframe mutual funds order entry system I had developed with a different tool that consisted of about 10,000 syntax instructions to complete. I had been perfecting this for about one year. I used this in my R&D evaluation as a comparison. With Certify, I developed the same automation features in about 1/3 of the time and which consisted of only about 450 Certify parameter-driven actions steps. It was clear, and obvious, that going forward, development would be easier, much faster, and that maintaining 450 Certify action steps would always be easier than maintaining 10,000 syntax instructions typical with other tools.

I have been using Worksoft Certify now for over 17 years. I have Certify automation developed for HTML web, .NET, Java, PDF, mainframe, and the most beneficial, SAP. I have many test automation processes for testing SAP, including testing end to end SAP Material Management from RFQs, PR, role approvals, PO, goods receipt, payment, return product, to inventory validation. Since 2004, I continue to primarily focus on SAP test automation with Certify for easier and faster development, and easier and faster maintenance.

Is it easy to learn? Think of the concept that, with other tools, you need to develop a framework and architecture to then develop your automation in. With Certify, it is already a development framework. It is built on a database with all the features for test automation. It’s not a programming solution (no programming) just action steps with parameters against application objects and action steps with parameters for math, logic and text manipulation, etc.

And, yes it is easy to use and easy to learn. In 2016, I went to Mexico for Yazaki, the company I consult for (a major automation supplier). I went there to train two business analysts on using Certify for developing test automation for several Yazaki Web business applications.

One of the biggest benefits: I was there for only two (2) weeks and completed training of two (2) business analysts that have no programming background. Since then, they have been pretty much developing with Certify on their own, almost one year now.

It has ability to execute the scripts without any manual intervention. It has ability to run scripts overnight without any human intervention (lights-out testing), which proves to be vital in providing wider coverage in a short time.

We are able to scale up the testing and to address more business-driven projects using Worksoft, which would not be possible with any other tools. The testing organization can cut short its budget, as it is not required to hire more testers for regression testing.

Upgrading Certify takes a lot of time and effort, which is like a small project by itself.

I have used it for two years.

We have not encountered any deployment issues.

We have not encountered any stability issues.

We have not encountered any scalability issues.

Customer service is 4.5 out of 5.

Technical support is 4.5 out of 5.

We used to use QTP, but the ability of Worksoft in the SAP automation space compelled us to make the transition.

Most important feature is the ability to automatically perform test without the need of a manual script, it's very fast and can perform overnight testing without human intervention (Lights-out testing). It can perform automated end-to-end validation across application landscapes.

Organizations does not need to hire numerous resources to perform regression testing, therefore, cutting overhead cost at a substantial level.

It uses a sizeable space for its database, therefore requires a regular cleanup in the server's space. Secondly, upgrading of its version requires enormous time and effort, that it's like a mini project. It could impact testing phase if not managed well. Technical support sometimes requires access to client's server, which is not allowed for some clients.

Three years.

No issues encountered.

No issues encountered.

No issues encountered.

9 out of 10

8.5 out of 10, it is quick to respond, but takes time to resolve on some issues (related to #6).

There was no established solution and testing was being done manually.

It supports Siebel, SAP, Web based, and Terminal-based applications.

It is good for automating SAP applications.

Scripts with a large number of lines have performance issues. As it is script-less, understanding loops in script is a bit hard. It also needs improvement to the UI.

Four years.

Installation is straight forward.

Quite stable.

Able to handle large volumes.

Haven't contacted customer service so far.

Good.

We replaced our existing tool, QTP. Jenkins has script-less automation, and is easy for non-technical people to use.

Installation is straight forward.

In-house installation was done.

It was an organizational decision and we were replacing QTP.

I would recommend it, though it's a costlier solution, but the script-less automation is very useful.

• Certify Data
• Layout
• Remote Control

It allows us to do end to end script executions.

Fetching data from CSV files

For 2.5+ years,

None encountered.

We have issues with database response during execution.

7/10

7/10

We used QTP on BPT framework. We switched because Worksoft Certify gives us a better regression execution & better results for SAP business. Additionally, it ha the Certify Data feature.

We used a vendor, I would rate them 7/10.

Its best for SAP automation. Questions on its web automation.