Worksoft Certify Reviews

We mainly use Worksoft Certify for SAP automation, whereas we have web-based SAP and normal SAP background applications.

Worksoft Certify has helped organizations save time by using automation.

The most valuable features of Worksoft Certify are the way we can maintain the processes and sub-processes inside. We can immediately identify and replicate multiple objects in the application without having a major issue with it. We are able to do a lot of operations even with the solution being completely scriptless. That is a large advantage compared with other automation tools.

There are some other more complete tools than Worksoft Certify, such as Tricentis Tosca. It has a quicker way of taking in a customer's feedback with more efficiency. I do not see Worksoft Certify having a lot of progress over the years that we have used the tool in this area.

From 2014 to 2022, there were fewer improvements in the UI looks and feel. Overall there were fewer improvements in the solution. There were some challenges that we faced with our customers, and Worksoft Certify provides customer-specific call announcements. For example, customer A is trying to work on Worksoft Certify, they will face a specific challenge in utilizing the tool, and Worksoft Certify provides a custom solution to them. With customer B, they are faced with other challenges, and Worksoft Certify provided them with a customized solution. There is a chance that the same issue that was faced with customer A, customer B had also. Instead of Worksoft Certify providing a dedicated solution for everyone, they were providing solutions on a customized level. They can improve on this process.

There were some challenges that we faced with respect to automation. For example, there were some areas where we had to do a drag and drop of some of the objects from one place to another. In some of the areas in SAP where you have to perform a drag and drop, that feature was not available in Worksoft Certify. We had to find some alternate ways of doing those things.

I have been using Worksoft Certify for approximately three years.

The performance and stability of Worksoft Certify were good. We don't see a lot of issues or challenges that we're facing in this area. We have a 98 percent stability rate. We don't expect a lot of issues to happen in the application. On the SAP side, it was completely normal, it was working as expected, but coming to the web-based applications, when there are major changes. We have to remap the controls sometimes to ensure that the controls are working as expected.

Worksoft Certify is a highly scalable solution. However, it was a bit slower compared to the other automation tools. For example, Tricentis Tosca is providing major competition to Worksoft Certify and they are slightly a better tool. Whenever Worksoft Certify faces any challenges they're providing some patches to fix them, and that is helping the automation to grow faster and more efficiently. We have the layouts and plenty of recordsets that make Worksoft Certify more scalable.  

As an architect, I have to compare the support to other automation tools, there are some free tools, such as Selenium, where there is no support. However, there are some tools, such as Tricentis Tosca, where there is a dedicated technical support team who are available 24 hows a day seven days a week in the chat. We can directly chat and communicate with them. With Worksoft Certify we have to contact them directly and sometimes it can take some time, and the solutions that they provide are customer-specific. That is another challenge because when we have to work with multiple customers, this is another challenge that we face.

Neutral

I have been using multiple automation tools, such as Worksoft Certify, UFT, and Tricentis Tosca. Additionally, I have worked on other automation tools, such as Selenium and AccelQ. I provide solutions to the clients based on their requirements. I work on different automation tools.

We have a set of regression test cases that are already identified with the customer. They used to provide those sets of regression test cases to be used in their projects. We had to create some processes and sub-processes, such as our layouts and recordsets for the projects. Once we create all our sets, we create our regression suites using these areas and start working on them. We used to do a time-bound process where we needed to automate all these elements in a specific set of time that we had. Once we are done with the automation, we performed the activation of the sets in a cycle.

For the understanding of the requirements of Worksoft Certify, it did not take us a lot of time because we understand the process which is very important. Due to the specific functional area in SAP initial took the automation developers some time until they understood the functional requirement. However, once we understood how we were going to handle or manage the static and dynamic data, then the automation was simple using Worksoft Certify.

Once we complete the automation, our SMEs used to start utilizing this solution whenever there is any deployment. Instead of triggering the script manually, our SME triggers the automation scripts that they have the regression suite executed, and for other new automation scripts. Whatever the testing team tests the functional they started using Worksoft Certify for creating those automation scripts and started using them. There are different clients requirements based on the deployments. The deployments used to happen every two months, we have some regression suite to be executed, and whatever the new development that is going to happen, we used to utilize the dev environment to automate them and start executing them in the QA. This way they were faster and we can do instant automation.

We typically have a team of six people who were involved in the development of our particular automation scripts, and the exact number of people depends on the project that needs these scripts. For example, in my initial project where I worked on Worksoft Certify, we had a team of 13 people who were working for different process areas. We have different areas, such as production planning, retail management, and quality management service. Each area was handled by a specific developer and we used to have some teammates who were helping us and assisting in the automation of these scripts.

Normally it can take clients a lot of time for doing the regression. Using Worksoft Certify has reduced the time needed for this process. They don't need to replace all their SMEs for performing the regression because normally it used to take a lot of time for doing the initial regression activation. There were a lot of bugs that were reported and were reduced when we started using this tool. They used to use 10 people for doing the regression previously, for a span of a week. By using Worksoft Certify with using the automation scripts and 10 different systems activations happening, we saved a lot of time. We have seen a return on investment by using Worksoft Certify.

Worksoft Certify is slightly more expensive compared to other automation tools that are available on the market. However, the majority of the clients who are utilizing this solution, don't care much about the price. The main aspect the clients look for is its stability and reliability on the automation and whether they are able to rely on the automation scripts.

The solution can be purchased in different packages, some can include support.

I have evaluated lots of other solutions.

I have approximately eleven years of experience in this industry and over the past few years, I have been working as a senior architect.

Worksoft Certify is a good tool, but any tool can perform efficiently if you follow the best practices. If you're not following or utilizing the tool properly, there are chances that you might miss the opportunity. The main challenge with script less automation tools, such as Worksoft Certify or Tricentis Tosca is that you have to implement it along with best practices, and if proper best practices are implemented for these tools from the beginning, they can be extremely efficient and beneficial tools that can help to reduce efforts.

I rate Worksoft Certify a seven out of ten.

We used the Worksoft director access to test automation.

The tool is easy to use. It is a drag and drop Microsoft type of solution.

Worksoft Certify needs improvement on customization of reporting and how you report final outcomes. It needs more customization.

I have been using Worksoft Certify for  years.

The solution is very stable.

Worksoft Certify is scalable. However, if the final output were more customizable, it would be even more scalable.

Worksoft Certify's technical support is a 3 out of 5.

Positive

The initial setup is straightforward. 

The implementation depends on the client's business. Overall, it is not very complex. It is simple and easy to implement. There is no need for a third party integrator.

We have an annual license for this solution. The product is very expensive.

Choosing this solution is entirely dependent on the specific business requirements, the time required and the budget available. If you are looking for an easy to implement scalable solution, Worksoft Certify is recommended if it works within your budget.

I would rate Worksoft Certify an 8 out of 10.

We have an SAP application, and for the automation of this SAP application, we are using Worksoft Certify. This project is in the healthcare domain. I get the requirements from the client. They tell me about the new changes that are going to happen in the system, and based on that, we do the automation. The client tells me about the scripts and new features in the application, and I create an estimation plan, which depends on the size and the complexity of the scripts and features, and share it with the client. We do the development activities accordingly.

We have two teams. Being an automation team, we are not very much on the functional side. So, we need to take some support from the functionality team. for test case writing. They are usually able to give us steps in Excel or something like that, and we are able to start with that. Otherwise, we have some sessions with the client or the functional team to discuss any issues or doubts we have. We also need data for developing the script. We get the required information about the data, such as if data is consumable and how to find new data, from the client or the functional team. Once the automation is done, we inform the client, and we are ready for the sign-up for the first set of the script. They do a functional review, and after they okay it, we do the final sign-off. We then move our scripts from the local sandbox folder to a production folder created in Worksoft Certify. We then use the script for regression.

In terms of its deployment, they provide the updates, and we install them on our VPS.

We are mainly using it for the SAP application, and for the SAP application, if you don't have any experience with automation tools, after a few training sessions, you can easily automate the scripts. That's because no specific programming language is used. All resources that I have are specifically SAP resources. They are not from the automation background, but after gaining the knowledge, they are able to develop a script, or when there is any issue while doing regression testing, they are at least able to understand the issue, such as whether the issue is in the code or data. 

We can use it for the web application, but we are facing stability-related challenges. The properties are getting changed. For example, when I am performing any operation on the text box but the development team has done some changes, our Worksoft scripts are getting changed. This is the main challenge that we are facing while developing tests for the web application in Worksoft Certify, where any changes in the backend are indirectly impacting our scripts. For the web application, there is a scarcity of resources. Unlike an SAP application that doesn't require much experience, for the web application, you require experienced people.

We also use XF definitions for the web application, but we can't create them on our own. So, we need to take support from the Worksoft team, and we have a dependency on them.

I have been working with this tool for the last five years.

It is completely stable for our SAP application. There is no issue. Once you have developed a script, you don't need to do any changes.

We are using it on quite a large scale. If we have a requirement for the new development, we have to use Certify. My project involves maintenance and new development. In the maintenance phase, we have regression testing. We have created a batch file, and when we are doing the regression testing, we are executing this batch file. So, we are using it for day-to-day activities. We are also using another tool from Worksoft called Worksoft Execution Manager.

The number of users working on this solution varies based on the project timeline and the number of scripts. For example, if the client's requirements need to be delivered in a very short period of time, then instead of four resources, I will take eight resources.

For any issues, we have to go to the Certify support team. They have a customer portal, and you just need to raise a request on the portal. Based on the priority, you get a response from them. Initially, they provide the solution through the ticket, but if the solution doesn't work, they also schedule a call. If the issue is not resolved in one call, they schedule more calls till they have resolved our issue.

I have used UFT in my previous company. Companies that have an SAP application or SAP Fiori prefer Worksoft Certify. For my current client, initially, we were using UFT, but because any non-automation resource can do automation in Worksoft Certify, we brought Worksoft Certify. Now, instead of UFT, we are developing everything in the Worksoft Certify. 

I am also familiar with Selenium. For SAP applications, Worksoft Certify is a very good tool. With UFT and Selenium, you need to create a framework, which is not required with Worksoft Certify. You can just go and develop an automation script. 

So far, I have used Worksoft Certify for the SAP application, the mainframe application, and the web application. For the SAP application, it is working well, but for the web application, instead of Worksoft Certify, it is better to go for Selenium. With the web application, we have the issue of values getting changed when developers make any change in the backend, and we also have a dependency on Worksoft for XF definitions. So, Selenium would be a better solution for web applications.

I don't have much idea about the pricing, but it is a licensed tool, and it is somewhat costly.

If you are considering this tool for your project or company, the Worksoft sales team would connect with you and explain what are the things that you need to perform. Initially, they give a demo, and after that, they discuss the licensing, etc.

It can be easily used by our non-automation resources with some training, but your team should ideally also have experienced people to be able to deliver within your deadline. For some of the development, you can also hire Worksoft resources on a contract basis. They would come to your location and help you with the development of a script in a short duration.

I would rate Worksoft Certify a six out of ten.

The reporting feature is something we look forward to. Also, the Worksoft training that we have done was mainly on the web application along with SAP, SNOW, and Silverlight integrations with the tool. It works efficiently with any tool. The projects we mainly used were for the testing of SAP applications over the web. 

Automating the User Acceptance Testing (UAT) process is the most valuable feature of this tool which helps us save loads of time for our clients on the projects and also generating testing and other reports.

We are doing Worksoft Online Training and also take up various Worksoft automation testing projects. I have totally trained about 50 participants on this tool. Most of the trainees are happy to learn and use this product in their jobs now. We have also done two projects, one for retail sales chain and another for a banking project. Both of the projects were delivered on time and the clients are happy to generate some automated OOB reports and also create their own Receipts. The clients were happy with our timelines and also with the training we offered at the end of the implementation.

Most of the features are good. However, being codeless is the best feature that makes it easy to use for those who have done any coding but understand simple English like commands. Anyone with basic knowledge of using a basic tool can learn and implement the solutions with Worksoft. This is why we have been able to train so many consultants on this tool and they use the same in their jobs currently and keep referring us for more candidates. We recommend using this tool for your automation testing projects.

Pricing is a bit high and we would like to have the availability of a trail environment for beginners and training would be great to have and easier to expand and use by more and more consultants.

The bigger reports needs to be more customized by the tool for better use, also tool also can be further simplified by a better Graphical user interface (GUI) will help us a great deal while taking up projects. 

We look forward for these upgrades so we can enjoy using the tool more and help others learn this tool.

I have been using this for the past three years. It has evolved as a codeless automation tool for testing. We used sed 10.0.1 and are currently using 11. 

We have also used few other tools for our trainings. 

ROI is pretty good so far.

We would like to have a cheaper version for a single user/trail version of the same for our trainings. The cloud based environment should be available free for the trainings for 15 days or so.

We do trainings on a needed basis so we understand the demand of various tools in the market. We strongly believe while Worksoft is a good tool it is losing its potential a bit to other tools in the market. The demand for training has gone down in the last two years.

We use it to do end-to-end testing for the business. After development has occurred and once we're into verifying that no regression has been broken, it's at that stage of testing that we deploy it.

We are a subsidiary of a larger company and we are focused on rolling it out, at the moment, to our larger company. With the tool's simplicity of use, where we are able to have a code review occurring, in that sense it will be useful in being able to roll it out to the greater company. We will be able to give it to the people who are experts in their areas, rather than trying to pass off test cases to one centralized location. It will be centralized automation and we'll just have one central COE.

Automation using the solution has saved testing time. I couldn't give you a number of hours or days because we're still in the beginning stages of trying to roll it out globally. We haven't been able to use the product and reuse automation. The whole point of automation is that the upfront cost to automate something is heavier and then, as you reuse it, it reduces the testing cycle. We're still investing in the earlier stages where perhaps we have spent equal parts right now, but we intend to see a reduction as we capture more and more.

Certify has also enabled us to find more defects. While I'm focused solely on automating and testing, so I don't have access to the defect count number, I know we have found defects, which tells me that we are finding defects that wouldn't have been found otherwise, or defects that wouldn't have been found as quickly.

• The dataset.
• The reusability.

It does allow for good reusability. When it's designed properly and utilized properly, we can put things in a way that allows for reusability, meaning a lot of reuse of VA01, if they're very similar flows, to keep it simple. And if we do have problems with a more complex flow, we'll make another version of VA01 that targets edge cases.

In terms of web UI testing, we've done very limited Fiori testing, but we have done Salesforce and a few others. Our experience is that when we get that stuff applied properly and working properly, it works very well. They're usually built well and if we do have problems with them we can get Worksoft to fix them. A lot of the times, if we're running on something that doesn't have an XF definition for it, by understanding how it's building objects, we are able to easily map objects fairly well and quickly.

The solution's ability to automate testing for packaged applications like SAP and Salesforce is related to when they do have that XF definition, but I do think it works very well. That's especially true for the SAP integration. That interface is very solid and objects are just about always discovered properly.

Since they updated the Capture feature to a more "Snagit" look and feel, it has become our primary tool. We've moved off of the old LiveTouch functionality. We will use it occasionally, but with Capture being built-in, it's easier for users to be trained on one tool. That tool has enough capability to be able to do both verifying the properties and recording the playback. It works well for us.

Looking at it as a product fully packaged, I would like to see more documentation or ease of use of the documentation. Sometimes documentation does exist but we have to search three different sites to find the proper way to do things or track down the technical document that explains certain fields. 

That, in turn, relates to the ease of use and how objects interact with each other. The application could lend itself to be simpler.

Another area that I would like to see improved is how the permissions are applied. If you're applying permissions groups to a user, one of the options is to delete the group entirely and lose the entire permission group, rather than just deleting the permission from the user, which seems a little silly. In my opinion, that whole module of permissions is very confusing and lends itself to common errors. We have to rebuild permissions occasionally.

The functionality is all there. I just think the way it's packaged can be confusing. We are successful and we can get things working the way they're intended to in Worksoft. It's just that sometimes finding how to do that, or where it is described, can be difficult.

I've been using Worksoft Certify for about the last year-and-a-half.

It's a pretty stable application. When it works, it works well, and it seems to work consistently. And when it doesn't work, it does not work — if that makes sense. When we see it functioning, we've got everything just right, it frequently seems to function solidly. And then, when we seem to have problems, it seems to not function at all, meaning tests will not run, or we cannot get a script to work in this or that particular way at all. But we've been able to work through all of our non-functioning issues through their support.

The solution will enable us to scale up our testing. With our focus being more on regression testing, it increases the testing of existing functionality first, and then we'll bring in that new functionality.

We are planning on rolling this out to more people, multiples of the number we have using it today. We think that it should be scalable but we haven't done it yet on that scale so we don't know for sure. But we do feel it will be scalable and that it will scale well.

Our extent of usage is pretty narrow at the moment. Approximately 10 people are using it right now and they are mainly automation engineers. There are a few directors using it to understand what the product is. People who we would consider to be "automation champions," who will help champion the product at our global headquarters, are being trained on it right now. They're not actually going to use the application, they're just going to understand it so they can help champion it and bring it on, full-scale, with user acceptance. 

Our main users in the future will be those information business analysts who know their respective products very well, the ones who are making the changes in targeted areas and who can easily reach out. They will be able to quickly test and record whatever they need to record for testing. We're looking at anywhere between 20 and 50 additional users within the next year, depending on how well user-acceptance goes, and expansion will continue from there.

I'll start with our positive experience. We always end up with some kind of resolution whenever we do submit something through support. 

There have been times though where their support has been very slow or difficult, where we end up with a level-one support for what feels like much longer than we should have a level-one for the issues we have. These are high-end issues that mean we can't function. That's been a frustration point for us. We've had to meet with Worksoft to talk about the support that we're getting.

As we start to build better in-house knowledge of some of the caveats of Worksoft though, that support has been needed less. That has made things a little better for us and that's why we focus heavily on training and having supporting documents on what we're doing.

I have used Winshuttle as well as DataLoad, which is an open-source and much more simplistic. Winshuttle is used more for something like an RPA function.

Certify has a much deeper bench in terms of what it can actually do. Winshuttle is only functional, to my knowledge, with SAP applications because it's built on the scripting portion of SAP. Its focus isn't for testing, so it's not a good tool for testing. But it is more simplistic in the sense that it looks like a spreadsheet and the result is provided in the last column of what the status bar gave you. It is really designed for one Pcode at a time, in my opinion. Whereas with Certify, you can run a larger-scale test or function or even a larger-scale RPA function, compared to what Winshuttle can support. The complexity involved in that is much harder. It's something of Catch-22, but Certify does enable you to do much more.

I can't speak to about the installation process, as we have a different person who manages installation. As far as setting up users goes, it's fairly simple within the application, once it's installed and functioning on the servers.

We started out with one model of being centralized and we're rotating to a decentralized model of sharing this out with more users and increasing usage. It's almost like we're in a second deployment of the product, and using more of the tools.

We're rolling it out to the specialists in each business area, on the information systems side. These are the people who are producing changes and who understand the changes and updates quite well. We'll have them write the scripts themselves, with our support as the center-of-excellence team. The idea is that they will be submitting the scripts that they've written back to us for code approval and then promotion to gold, to be able to be run regularly, as a script that's been validated. It should work well and be successful for them. We'll give them help with training, etc., in the Worksoft product itself. We're trying to focus on somebody becoming an application expert, for each application we're testing, and to be an application expert for the automation product, allowing them to function well enough within the Worksoft application.

The person who is responsible for installation is also responsible for maintenance of the solution. Like me, he is an automation engineer, but we have different focuses.

My advice would be to develop a very good training program to go with it. Also, understand how to build a good structure to allow for success and to limit exposure where people are editing things that they shouldn't be editing. You should also partner or work with other businesses that have used the solution successful. Build up industry contacts who can help you understand where they're going and where they're having problems, as well, with the model they're implementing.

The biggest lesson I have learned from using Certify is that you can design it to be way more complex than you need to, and you need to be very careful, when you're designing the solution, to design it in a very simplistic manner. It's almost like code in that it enables you to do things that are very complex, but you need to be very cognizant that you shouldn't always do the most complex flow, and that you shouldn't overly design logic out of any one script. They should be relatively simple.

Regarding ease of use, once you understand how to use it you can use it very effectively. But at times it's difficult to understand what the application is doing, what you are actually editing, within the application. So at times, when it comes to certain objects, you might not realize you're editing another object, in a way, unless you've used the application and understand how it actually builds together. It is simple once you know what you're doing, once you understand how all the objects work together, but leading up to that it can be more complex. We overcome that with training, reference documents, and a lot of training documents. We did an intro training with our team just yesterday. We're rolling out more globally, so we're training and trying to have a center of excellence team that can help out with these concepts. For example, they can help design better training to understand, "Hey, when you're editing here, you're doing this." We're trying to do more targeted training to the things we do with our standards inside of Worksoft.

As far as the Capture documentation goes, for us, it's almost too detailed. We've actually implemented a custom solution for documenting, because we need something that's simple, almost like what users would experience for test cases for manual testing. We also designed our own solution for that, in part, because we utilize a lot of Selenium-style code and we need to be able to record results that are occurring in that application. We'll call Selenium and Worksoft and we need to have a consolidated results report. We don't utilize, and, just to be clear, we've never purchased, BPP (Business Process Procedure) so I don't know any of that functionality. But with our unique set up, it did not make sense to utilize those reports. The reporting that is built into Worksoft is good for development cycles, developing scripts, but we don't use it for result-reporting, in the sense of whether the test passed or failed. We've narrowed it down into a custom application.

While it does allow for good reusability, even if best practices are followed, at times it's hard to identify if you have the same components or processes being built. That can be hard to recognize. For example, there will be duplicate login scripts. The application doesn't seem to lend itself to being easy to manage for duplication of processes. We are trying to put workflows in place on our team to help identify duplication and to reduce it. We do intend to use Analyze as a way to help catch duplicate workflows.

We are working towards use of the solution for RPA testing, but our primary charter is to industrialize our testing cycle, and then we can move into something like that.

We get user stories from the DevOps teams, in conjunction with a recording they make with another Worksoft tool. Then we will investigate if it has already been automated and, if not, to automate the process which has been delivered to us.

This is all set up in a Citrix environment. We have SAP being used at the moment and we still have the old SAP ECC up and running. I'm not sure which part of this is cloud-based, but the Certify solution is installed on Citrix.

Because of the fact that we started just six months ago and we have a small team. But slowly but surely we are gaining more audience; more people are starting to get interested. That should lead us to be able to start implementing it the way it should be done. We have done some regression testing and, when doing so, we found real issues. So it has proven itself to be useful during regression testing at least.

We have definitely seen savings in testing time. Scripts are executed five or even 10 times faster than any one of us could do by hand. While we don't do so at the moment, we are going to start executing them in a lights-out environment. We will run tests during the night and get more numbers, execute more tests. That should also help us save time. We have to get the experience and the numbers for this, but I think it will save us a lot of time.

The biggest feature is the fact that it's codeless. It takes away the problem of finding people with the correct programming language, since there are multiple such languages. It saves time in introducing people to the solution because they don't need programming knowledge, they just need to be able to think logically. This makes it vastly usable by more people who are not even acquainted with IT at all.

Also, the solution's web UI testing abilities for testing of modern applications is pretty awesome. Like with every product there are some parts which can be improved, but overall it's great.

It's very easy to use and to install. You have to know, as a user, what your exact application is on the test; you need to know which object recognition files you need to use.

You use the tool to do your automated testing. As far as I know at this moment, it can do a lot of stuff. It's usable in DevOps, so with regards to packaged and non-packaged software, it's good.

I use Capture from within Certify. I also have a stand-alone capture that I have up and running. If you look at the whole cycle, it takes the user a lot of time to create the records. During the capture, the responsiveness of the system is really slow. But after that, when you send it to Analyze, the documentation is really easy. Just click the button and choose the format. Automate is the same. You just create automation and choose a file name. Then, when you need it, you just download it into Certify and start using it. We've been doing it for some time now.

The Capture feature helps find the actual processes to test for and to create end-to-end testing. We ask the users, when we are making the recordings, not only to enter the proper data but also to provide us with comments or LiveTouch images of messages that need to be recorded. They know, "Okay, when I see this message then it's up and running." Because they take the end-to-end as a whole in the recording, we can use that as process knowledge as well. So the process is, in fact, being captured in the Analyze software.

In terms of the solution's ability to build tests and reuse them, I would rate it at eight out of 10. We record it on one environment. We make it completely environment-agnostic, data driven. Once recorded, we can reuse it on every single environment in the development cycle, which is awesome.

Performance on the web UI part, especially with some of the more comprehensive Fiori features, like the complex tables that are being used, could be improved. In those cases we have noticed a lot of execution-time increase with regards to the Certify solution.

I have been using Worksoft Certify for six months now. I started using it in September of 2019.

We've had no Certified-related stability problems. We have issues with things like Execution Manager and Analyze. I'm not sure if those problems are infrastructure-related or due to the Worksoft setup, but with regards to Certify it's stable. Sometimes there is a crash, but I think it's more related to the fact we're doing a lot of complex stuff in a Citrix environment with low resources.

The solution hasn't enabled us to scale up testing yet. We're on the way forward, but because of some issues in our own architecture we are not able to execute those tests. But I know how the setup is working, and I think you can scale up really easily; just add more machines, add more users, and have a go.

When I started within the program itself, no one else was using it. There were two users on the Railnova team. At this moment, about 10 or 12 people are using but within a couple of months we will be around 50 users in total.

I would rate Worksoft's technical support somewhere between a six or seven out of 10. In two-thirds of the cases we get a proper technical support member who has the knowledge to help us with our problem. But at other times we get someone who doesn't really know what he or she is doing or doesn't really understand the issue.

Another big part of the grade I gave is the fact that when you are in contact with the call center, a lot of times there really is a lot of background noise. With the accent, it's already really tough for me in fact to understand them, and with the background noise the problem gets bigger. But I've had also a lot of support from the Germans and from all over the world. Most of them are really capable.

It's an easy setup. There are some small configuration settings and then you can have a go. It's all up to the user to do the updates on the definition files. That's also easily available to us.

I started using the software without hearing from Worksoft. I only had to do some courses on the Worksoft University web page they provide. I didn't have any real, proper training, and I was up and running within two or three weeks. And within two months, I was able to provide enough support to get multiple teams within our company up and running with Worksoft. It's really straightforward.

I wasn't part of it, but I believe the initial setup and further configurations took two or three months in total.

Because of the fact that it's also able to do orchestration and because of the fact that our company is moving from the old SAP towards SAP Fiori — they wanted to have the main focus on Fiori for the UI part, in conjunction with the orchestration which Worksoft is able to do — at first it was only UI-driven. But we will expand into more and more Worksoft uses.

The company used a Worksoft consultant for the deployment but I don't know her name.

We, as a company, have good contact with one of the Dutch Worksoft managers and he introduced the integrator to us, as that manager is from the United States. The consultant came over for a week to give us some training on a number of things because we are not only using Certify.

They were really happy with her. A lot of questions were answered, a lot of issues were resolved. She was able to contact Worksoft support really fast. They had a blast while she was here.

We haven't seen ROI yet because we are in start-up mode with Certify. At this time we are only investing in the solution. Hopefully, we will be able to have some insights into ROI within a half-year from now.

The licensing is yearly.

My advice would be to think outside of the box. If you've chosen to work with Worksoft, you have to embrace the product as a whole. You will find, as with any other product, that some things that don't operate the way you want them to or would have expected them to. But if you teach yourself to view your problems from different angles using the software, then you will be able to come up with the most brilliant solutions. You can do much more with a codeless tool than you might think upfront.

The biggest lesson I have learned using comes back to the codeless part. I view myself as a smart guy, but I don't have the proper coding language knowledge. I was working for myself over the course eight years, before working here, and oftentimes the jobs were really cool, but most of the time I had to do Python and this and that. That was always a struggle because sometimes, when you've learned a language but you're not using it for a year or two years and you want to go back, you have to start remembering it. So I was turned down for those jobs. In this case, and we can show the world that it can be done codeless, if you have the proper tools.

When I was first introduced to Worksoft and they told me it was codeless, I was really skeptical. I said, "I don't see that happening," because I had been doing this for quite a while and was used to doing some coding. But the tool convinced me otherwise, which is really nice.

Overall, it's capable of being used in modern technology environments. I have been using it for six months now and I still have a lot of learning to do. And as a company, we need to start using more of the Certify features, not only scripting and rerunning those scripts.

Most of the people who are using it right now in our company already have some testing experience, but it's our goal to have business and IT people use the Capture feature as part of the process for DevOps.

We don't do test maintenance at the moment. We started out with test automation. We had to set up a base for the DevOps teams and then support them from that point onwards. So we are slowly moving into the maintenance part. Because we have split the data from the script itself — everything is data-driven — so it should be fairly easy for us to make the necessary changes. I think execution is faster when compared to human hand movements. But for changing or maintenance, I don't know.

The solution hasn't enabled us to find more defects at the moment, because we have been focusing on "happy path" testing. We need to get to the end-point of the end-to-end testing. But I believe, and I'm rather positive about this, that if defects are entered into the system, given that our regression test set covers a big percentage of the complete solution, it should be able to find defects really fast. Faster than we can.

The Certify users within our company are all in scripting. We're developers. And because we are in a scrum team, we don't have different roles in our team for test automation. A lot of things are being delivered by DevOps the teams, which you can view as functional consultants. As for the deployment and maintenance, a lot of it is outsourced to one of our partners. We do have functional and technical maintenance or support. I'm the technical guy and then we have two functional guys as well.

We use Worksoft Certify for testing and non-testing. 

• Under the testing umbrella, we use it for regression testing, disaster recovery, validations, and system refresh testing. This has been our main focus. 
• Recently, we have started getting more requests for more RPA-light type of work, which is not testing. It is using the same skills, process knowledge, and tool sets to do work that would replace manual repetitive tasks with automation.

We use Worksoft Certify for some basic ServiceNow functionality or Workday releases, weekly and quarterly releases. For SAP Hybris and ECC, we are also using it for our internal security protocol testing. So, we test Office 365 and Windows 10 compatibility. We test some Excel functionality and file sharing, as part of our security protocols. The most in-depth end-to-end testing that we have is in SAP.

We have done a lot of manual testing. We still do a lot of manual testing for our projects. We've eliminated a significant amount of manual testing with our system refresh, automation, and for technical upgrades where changes are known. However, for projects with new enhancements and functionality, we are having a slower time penetrating into them. With regression testing, we have completely replaced it with automation. 

Now, we are trying to shift as much as we can to start automating processes earlier in the project lifecycle, but it has still been a bit of a challenge. This is one of our stretch goals for this next year. The non-testing area is where we have had the most growth over the last six months.

The freshest example is some of the RPA-light activities that we have done. We have been able to save on a lot of manual work for some very highly skilled, expensive resources. This has been able to free up a significant amount of their time so they can spend more time on innovation and more creative, value-add activities. That's been one of the more rewarding things that we've done, and the most appreciated. 

We run our tests and our full suites every week. We have them scheduled in batches so certain sections of our scripts run every day, then we run them through the whole suite every week. That is how we maintain them by running and repairing them.

To run them every week and make repairs takes us maybe two hours. Because they are scheduled on batches to run, it's lights out. They are pretty hardened at this point, so there are not a lot of repairs for data, etc. If there is a break, it is usually because there has been a change to a process that we were not aware of. Our automated scripts are the company's best business process documentation, as we don't have a business process management tool. Therefore, if anybody wants to know about our business processes, they come to us for something like training or new production support resources. They will come and watch our automation if they want to understand how the systems work.

The ability to run multiple processes at the same time remotely or on a schedule. So, we have some testing that we do every day, and it is pretty much lights out. It is unmanned. We have some virtual machines that run on a schedule. Therefore, it's out of mind testing unless there is an issue. They are very hardened tests. If there is an issue, it means that there is something that we needed to catch, so it is always a good catch. This has given us a lot of flexibility because now we can use those resources in other ways. Besides the basic automation capability, it has been great having the ability to test multiple applications and multiple processes at the same time and overnight, then just receive the results.

I have always appreciated the Capture tool. I'm excited about the new enhancements that have been made to it. I think this will make adoption a lot easier because the tool is a lot easier to use and has more capability. I'm excited, because this is a good time for us as we are expanding into more RPA-light space. It will be easier if we could have some of our more technical resources doing a lot of this capture work. Then if they do something wrong, it's very intuitive to stop, delete that step, and start over. It will make adoption a lot better, and we'll get better requirements and processes documented that we can then convert into automation.

The time it takes users to document, then for us to automate, depends. We have a lot of end-to-end business processes, but they have to go across functional teams. They tend to get passed around a bit. Worksoft definitely saves time. We were literally getting processes on napkins. We didn't care how we got them as long as we got them. They would walk us through really fast, and they would try to show us. We were doing screenshots and trying to record them in WebEx. 

Since a lot of the business process owners are onshore and the developers for the automation are offshore. It was very challenging trying to find a time when they could get together. This product allows them to do the capture on their own time. It can be very quick. They just send it over, then they're done. They don't have to think about it anymore, and it's documented well enough that we then don't have to spend a lot of time coming back to them. It just makes the whole process more efficient.

I would like more reporting in analytics. There is a lot of manual work for us as program managers and test managers which has to do with supporting our value statements. E.g., if there is some way that we could systematically capture how long it is taking for automation processes to execute, then we could insert some notes as far as here is how long it took for them to do the manual capture. Then, we could calculate time saved and have a formula for savings. 

If they have some templates that we could all partner, there are a lot of customers who have created their own, but with the new companies coming onboard. Do they create them on their own or do they try to leverage the best practice within our customer community? There is more we can do here.

There are no issues so far. We haven't had any problems with the tool not being available for us when we need it. 

We had trouble with an upgrade once, but there was an immediate response on their side. We had a very technical resource who helped us get past it quickly. So, there has never been anything which has really stopped us from working.

Because we started early with the tool, it took us a while. I feel like we almost glued things together as our needs became more. The capability that was delivered by Worksoft was there. Therefore, there hasn't been a time when they have been behind me when I have needed something that wasn't available.

I am expecting it will the same in the RPA space. We will grow together because of our close partnership, and if there is a gap, I can work with them to figure out what the best approach is to close it. I think we will be able to stay with the tool for a long time because of its scalability.

I am aware that they have some challenges with some of their support resources, especially offshore which is very common. I don't think this is specifically a Worksoft issue. It always seems to be a software issue, and I know that Worksoft is aware of this and they are trying to make some improvements. 

If ever I need to escalate something, I never feel like I'm stopped. I always feel like there is another level where I can go and get support. We have never had an issue which has gone unresolved for a long time. We try to follow the process, but since our team is so experienced with the tool, if we can't fix it and their support can't fix it either. then it gets escalated up through the chain, getting somebody whose pretty senior with the tools to help us.

Our initial goal was regression testing. It was really expensive. It was throwaway work. We always had to outsource it. It overlapped other test cycles within a project. So, all the functional business folks were busy. It was something that if we left it up to the project resources, they didn't do a very good job with it. We would bring in manual testers almost literally off the street. They didn't know our processes. They ended up having to interact with our project resources anyway. It was just a mess. It was inefficient, clunky, expensive, and the quality was poor. 

We knew that we had a lot of SAP implementations coming up because we had acquired several new companies. So, we made the decision at that time that we needed to automate regression testing. That was our first initial goal, and we've hit that. During our last major SAP implementation project. Our regression coverage was at 90 percent which is pretty much the top you can ever really expect. Now, we are looking at other use cases.

The initial setup was a little rough in the beginning. It was so new for us. The whole idea of automation was new that trying to get the tool setup, internalizing all the best practice training and everything that came so quickly, was a lot to try to digest. Thus, we ended up asking if we could spread the mentoring out across a few months. This seemed to work better for us.

We purchased the software, then we found a vendor on our own to help us with the development.

It has saved us significant time. I have an entire dashboard that I use to showcase to everybody the amount of manual hours that we have saved and how that equates to dollars saving.

On our last big SAP implementation project, we inserted an automation resource into the beginning of the project. Between automating regression processes, data staging, and using our automation to help repair cutover and conversion issues, we saved the project about $1,700,000.

Worksoft has paid for itself fives times over.

We evaluated Micro Focus UFT, Selenium, SAP CBTA, and Worksoft.

My main focus was an SAP automation solution, and Worksoft was really good in that space. They were an SAP partner, but I recognized that I wanted something to be more scalable across other applications, and that ruled out the SAP solution. We liked the price point of Selenium and some of the open source tools, but there are risks to something like that. You don't have as much control, and there are always security concerns. Our internal teams weren't excited about that, as they are not great with SAP. We already had a lot of pain points with UFT. It took way too long to develop processes with UFT. It required more coding.

I would recommend taking a slower, organic approach to automation. It is hard to insert ourselves into the projects. The functional resources, business resources, and process owners don't have a lot of time for us. They don't see the value initially. It is overhead for them and more work. So, you have to bite off small chunks. Show the value, then build up the trust. If you try to be too aggressive and force something down everybody's throats, they will barf.

If you have super strong executive support and it's a top-down, e.g., the CIO says, "You will do this or else." You may be successful. However, in that scenario, your failures will be noticed and made very public. If you take a slow organic approach, where you're just trying to be really helpful and free up time, doing little favors here and there, you build up confidence. Then, people support you more for your success.

Start with the low hanging fruit for the value. Build it up. Once you get a bit more expertise, then start tackling the more complex processes.

Worksoft is a great supplier to work with. They have never pushed back when we have had issues or questions. They have always been available to help us. They put us in touch with other customers that have done something similar to what we were looking to do. They set up user groups by region so we could get together. They facilitate a lot of good discussions. That's why I mentioned we continue to grow together: customers and suppliers. It's just been a great relationship. We don't get that with every vendor. So, when we have it we appreciate it.

It has been very easy to use, but I don't think every automation tool is for everyone. I don't think just anybody off the street can come in and use it. Maybe for some basic stuff, but if you really want to maximize the use of the tool, you need some folks who are really experts in it. 

We were able to really grow when we hit that inflection point: When we transitioned to a different vendor that we had doing our automation development. They were experts in the tool. That was when we started being able to deliver these creative solutions. That was when we were able to see the cost per automated script go down, because they were able to develop so much faster.

While it can be used by everyone on the surface level or to capture the business processes, to get more return on your investment, you have experienced resources using it.

Our primary use case is test automation. We have an SRP solution called One1ERP on an ERP platform. We started the automation in Worksoft Certify, getting more than a thousand test cases automated.

Nowadays, we also have automation for a web application in our HR area, so all our HR processes on the platform, Pega. This started to be automated since November last year. We have 40 test cases automated on this application in one year. 

Improvement means for us that we have to be better in quality. Due to automation, you can run every automated test case twice a week. If you do it manually, you do it once per release. This is a quality improvement.

We always have to be more efficient. E.g., if we can directly automate test cases, so if we are using older test cases, then by just switch the company code (national company), we don't have to do it manually. We are able to directly automate the test. This saves money, which is always important for our top management.

We are creating a team an automation team, which will have up to 10 people/colleagues. This will be set up in Bratislava, and those 10 colleagues will work 100 percent on automation. Usually our SMEs do the test execution and try as hard as possible to have everything directly automated, e.g., 40 percent test execution and 60 percent automation. In the end, we want to have a pure automation team who is just doing automation, and have the testers on another team. After the tests are finalized, then they can give them directly to the automation team, or work on them together. That is a mixture. For those colleagues, it's really 100 percent automation

We are interesting to do better, year-by-year. At the moment, we are doing automated regression tests. The next step would be DevOps or artificial intelligence. Our programs should also develop in this way. We want to have automation everywhere where it is possible. Therefore, we need more options for these next steps.

We have used Capture, and it works with Worksoft Analyze. We had some experience last year in August because we started our schedule 1.0 and used Analyze in our tests. We were using the central site for one year with our ERP testing and were quite successful, but this was with Capture 1.0. Then, there was a change to Capture 2.0. In the end, there have been some challenges with the newer version. Therefore, the company testers, the local ones, do not want to use Capture 2.0.

40 percent of the test cases were finalized with automated capture and automated documentation, then the others were done manually. Because we have to create test nodes, we were asked to create a tool that automates documentation, which was Worksoft Analyze. However, with the switch to Capture 2.0, we had some challenges in the beginning. What we did afterwards, together with Worksoft, was we sorted through all the known bugs. So, at the moment, we don't have any known bugs open. We will retry this year in our central test first to find out if it Work Analyze is fine, then if it is okay, we will continue with the local test teams, as well.

On the Capture 2.0 topic, we were not satisfied, because we had a version that wasn't really tested from my perspective. Of course, Worksoft said it was tested, but we found a lot of bugs. In the end, our national company and local test teams did not use Worksoft Analyze because it stopped working. We have ten steps, then on the ninth step, it stopped working and we would have to do it again. So, they stopped using Worksoft Analyze and Capture 2.0. However, this is solved. Worksoft directly helped us to find out what the bugs are, and solved them. Then, we retested it. At the moment, we don't have any open bugs.

On average, it takes one day for Capture 2.0 users to create document. We expect it is faster, but you have to do it several times sometimes. You have to check the documentation that everything is fine. 

There have not been many issues at all, maybe one or two during the last three years. However, it has been quite stable.

Worksoft has good scalability. This is also the reason why we easily can automate for our new laptops.

Usually, we have one contact person for support, who then will contact the Worksoft support. Our contact organizes the people around her. While we have some direct contacts, and most of the time, we receive answers from them. I don't know if there is somebody behind them from the second or third level. 

We are forced to have efficiencies every year. We always get less budget and having to do more. So, we had to have some ideas, and our idea in 2017 was to increase automation. We had automation in place beforehand with eCut. However, in the end, we cut rates 10 to 15 percent. With Worksoft Certify, we had this campaign year with company codes up to 80 percent of the automation rate. This is much faster, and we are finding the defects earlier. In the end, you can save money and have better quality.

In three months, we created 1000 scripts with Worksoft. When the three years before with eCut, we did 450 scripts. This is where we saw a difference.

The initial setup was straightforward. 

In the beginning, it took a long time to integrate Worksoft in our landscape because we have a lot of security levels to fulfill. Therefore, it was not permitted to be install on the security server. It had to be installed through our data center. We learned step-by-step.

Due to security issues we could not use VPN tunnels, we had to have a jump server. This cost us in the end up to eight months. We had four solutions beforehand that were not successful. The fifth one was the jump server. This was the final one and is our solution at the moment.

In Germany, we have a higher security level. Therefore, it costs us by having a longer time to integrate. This is due to our requirements, not Worksoft. The reason why it took that long was on our side. We have a lot of IT departments. With security, social partners, and data privacy, there are a lot of requirements to fulfill.

After that, we agreed to have an automation manager on our team from Worksoft. He sat in Bratislava and all the open issues or questions sent to him, and he answered them, either directly or he contacted the support teams. Then, he assured that those Worksoft issues were solved. He also gave us hints how to use Worksoft, such as naming conventions and how to use it so you don't have a mess in the system.

After we had the automation manager for eight months, then he left. We decided to go with Cognizant, but that was also not successful. In the end, we decided to to have a Worksoft expert from Worksoft, if we need it. For example, we now need an for an expert for two weeks, who will tidy up our system. 

In the beginning, we had an integrator, Cognizant, who created a lot of scripts, but we stopped because we had a contract with them that they would install Worksoft, but it was not allowed. In the end, they just did some automated scripts for us. Our automation team in Bratislava was not satisfied with the quality in the end. We want to have a flexible style with quality, and this wasn't done by our Cognizant colleagues. Therefore, we decided to do it on our own.

From Cognizant, they were sold as SAP and Worksoft experts. Concerning Worksoft, I cannot say if they're expert or not. Concerning SAP, not all of them were experts, maybe one. However, this is always the same in this business. We also had cooperation with Worksoft. Concerning the flexibility of the scripts, we decided to work on our own.

We expect partners too be really good. Otherwise, the company does not see why we should pay for them.

Saving money and better quality, these are the benefits of Certify.

We have seen ROI. This was one of the goals from our top management when investing in automation. They want to see savings in the following year.

We did an RFP in 2016 and 2017. We found that Worksoft is easier to use because our business experts can do the tests. We didn't have to have IT experts, like with eCut.

If you use SAP, you can use this. It is easy.

I am really satisfied with the product. If I ask for support, I get support. I have direct contacts and every issue will be discussed. If we need something, they help us directly.

We did not automate our test maintenance. 

We don't have experience using it with apps and mobile testing, but are looking to add this to our portfolio in the future.