Worksoft Certify Reviews

We are using Worksoft Certify to enable our stakeholders to do test automation on the UI level.

We use it for end-to-end testing of packaged applications. We are part of the internal IT department within our company. Most of the time we are using it on our own products. The products and systems that we get are usually preconfigured and prepackaged and we do additional testing, not just for the functionality for the coding that we add to the product, but also on the prepackaged solutions.

We use it for all kinds of applications. Our focus is the web area, including web-UI testing of modern applications. We have two tools in place: our own internal corporate test automation tool, and Worksoft Certify. The latter is a complementary tool, especially in the web area where there are some white spots for our corporate tool which it cannot cover. That was the main reason why we brought in another tool. And for that, it fits perfectly.

We are able to run our test phases faster. Once the scripts, the test cases, are ready and automated, not only are we able to check our systems or landscapes during the test phases, but we can proactively monitor our development and test systems. Proactive monitoring of our systems is very important for us and was not possible before because manual testing is just too time intensive.

Worksoft Certify helped us to increase time savings. We didn't start test automation in general with Worksoft Certify. We did automation before with our own tool, but it helped us to increase the coverage of test automation and to increase the time savings.

We had a success story with two teams. For the execution of the scripts, we had time savings of 82, 88, 95, and 90 percent. And for the speed, it was between nine and 21 times faster than manual execution.

It's not necessarily saving us money, but it's helping us to free up the capacities of our end users to work on other stuff. Instead of doing testing for two days, they can work on bug fixing, developing new features, etc. That person still gets the same paycheck at the end of the month, so it's not saving us money, but it increases the value of our products. It increases the quality of our products. The reason for that is we are not customer-facing. We are dealing with internal teams and internal products. We are not selling anything to the outside. We are with the internal IT department. For the development teams and the sales team or the consulting team it might be different. But we are not really going out, selling our products and getting the revenue for the company. This is done by other teams.

It's script-free, which is really important for our end users because we are usually dealing with colleagues who are not developers and who do not always have the technical background of developing and scripting. It's very useful that there is a nice UI and the tool is script-free.

It covers all of the technologies we need to cover.

And one big advantage of Worksoft Certify is its integration with SAP Solution Manager, the test suite of Solution Manager, with the certification. That is crucial for us since Solution Manager is our test management tool of choice.

There are a couple of small things, technically, that could be improved.

Features we have asked for include single sign-on. It's a bigger project to make sure that our end users do not have to store passwords, usernames, and the like, for the different tools we have. 

We are also working on an additional integration with another tool that we have in place for lights-out testing. That's ongoing at the moment.

Another idea we brought is that the definitions for the objects need to be automated. They need to be recognized automatically by Worksoft Certify instead of changing them back and forth manually. This is also something that Worksoft is currently working on. 

Updates, in general, is a topic that we are working on with Worksoft on a regular basis. For new products, for new UI technologies when they come out, the test-automation providers need to update their definitions to make sure that the objects are recognized correctly.

It is pretty stable. After upgrades, we usually don't experience any big issues. Of course, it's software, so here and there we find bugs, but nothing crazy, to be honest. The availability of the system is pretty good, almost 100 percent. I don't see an issue here.

At the moment we don't have any issues with scalability. We have about 300 end users working with Worksoft. On infrastructure, it's split. We have a WTS environment, a Citrix environment, for those colleagues who want to use the prepared environment. We have other colleagues who are using the client on their own machines, on their own laptops or desktops. The only thing that we saw at the beginning which we need to change in the future is that, due to the latency, we cannot use clients in the US, for example, while having the server in Germany.

If the latency is over a certain number of milliseconds then it is basically impossible to do automation. That was one of the main reasons why we set up the Citrix environment at the very beginning.

We are still in the phase within our company, or within IT services, of training and spreading the topic of test automation, overall. So our coverage, at the moment, is not the entire organization, it's only the IT department. Once we have done this - and it will take at least another year - we will see if we spread using Worksoft and our internal corporate tool as a combination, or tool ecosystem, further into the organization. But this is not our not our team's responsibility so it's not really in focus at the moment. We are pushing for test automation in our teams and there is still a huge demand for training and new teams coming into the topic of test automation.

In general, technical support is good. They are collaborative and responsive. The only thing I don't like - and this is the only complaint I usually have for Worksoft - is that the first-level support is not always the best for working on topics. We sometimes need to escalate to second-level support and then we know that we are getting a colleague who is aware of the issue and is not just playing for time.

We already reported this to Worksoft and asked them to find another way or to educate the first-level support or to make sure that the tickets go directly to second-level support if they come from us. The guys on our end who are reporting the issues sometimes know more than the first-level support.

When it comes to second-level support, we are happy. There, we know we will get the help that we need. The colleagues are responsive and very helpful and, from a quality perspective, they are very good.

In the beginning, there were some issues with the integration, it didn't work the way we wanted. We spent some time with the Worksoft team, with the support and engineering team, in adding some enhancements to adapt the solution to our three-tier Solution Manager landscape. But that worked very well.

We have a very good collaboration and relationship to Worksoft. For example, every two weeks we have calls with them. We'll provide feedback and they take it seriously. They usually provide us with updates, with enhancements, with new functionalities that we need. That's working pretty well.

We started with manual testing and then we started the test-automation initiative. We started with our internal corporate tool and then, as I mentioned earlier, we figured out that we could not cover everything with that. At the end of 2015, we started to check the market. We did some PoCs and we decided to go forward with Worksoft Certify.

There were a number of reasons we went with Worksoft Certify. The Worksoft team did a great job. They came to our headquarters and did the PoC, showing that the tool is suitable for our needs. They did another PoC with our operations colleagues who were running the regressions testing in Singapore. And then there were the technical requirements that we had. Worksoft Certify was able to cover all of them, some of which I have mentioned already: Being script-free, being fully integrated in Solution Manager, and being able to script in a modular way. And finally, the integration between our own internal tool and Worksoft Certify was also important.

The initial setup was pretty much straightforward. I cannot give you too many details because I did not take care of the implementation. But I know that it took us about two weeks to set up the whole infrastructure. It was not really difficult. And we had very good support from Worksoft, from the support and engineering team. They helped us in setting up the database, setting up the connections. It was not a big deal. In total, within two to four weeks, everything was working fine.

On our side, we had a couple of staff members involved in the implementation because our team is the application owner. We had to involve two more colleagues from the database team because we don't have all the authorization stuff, for the databases area, for the servers etc. In total we had about three staff members involved, but not full-time. It was about one work-week for each of them.

Maintenance is done by myself and one of my colleagues, with the help of our database and server/infrastructure team. We don't have authorizations for everything and we are not database experts. There are three or four staff members taking care of maintenance, as part of our job; it's not a full-time job, obviously.

Whenever we need to do a full upgrade, when we need to plan the downtime for the production system, we try to make it on the weekend. I also already recommended to Worksoft that it would be nice to have something like an offline update where the system can be upgraded or smaller changes and fixes can be included without having full downtime. For an upgrade we usually need two to three hours. Afterward, we do a bit of testing, so upgrading takes about half a day.

It was just one or two people from Worksoft and three guys on our side.

We get feedback from all areas that the return on investment is there. Not just regarding time savings, but also cost-reduction. The return on investment in one case was reached at something between five and six runs, which is pretty fast, especially in an Agile environment.

What is also very important for us here is the avoidance of human error during the execution of tests. Usually, if someone is sitting in front of a laptop and doing testing eight hours a day, he or she will make some mistakes. This does not happen with a tool. Another important factor for us is the availability for testing. Usually, it's pretty hard to plan a test phase to bring all the testers to the table and get the time blocked off for the test phases. For the tools, we just need the systems up and running and then it's a matter of minutes to set up the test plans and to run the tests.

At the moment we are rolling out Execution Manager.

At the time we decided to bring in Worksoft Certify, we looked into two other tools. The key difference was that Worksoft was script-free. That was not the case for other tools. And the full integration to Solution Manager was one of the key differentiators between the tools.

If you have done a market evaluation and have decided to go for Worksoft, my advice is to go for it. I would definitely recommend Worksoft Certify as a test automation tool.

The feedback that I get from our stakeholders is that the tool is pretty simple to use. What we usually do is a two-week training, not full time, where the total is about three to four business days, 20 to 25 work hours. From there, most of our colleagues can start working with the tool. Of course, they have questions later on, some difficulties when it gets into special activities. But overall, the tool is easy to use. It's generally found to be intuitive.

In terms of cutting test maintenance time with respect to the scripts, that has not happened. If you need to adapt your scripts, automated test scripts are much more complex and more effort-intensive than manual test cases. But this is the nature of the beast. It will happen with every tool. If a screen changes, if a system changes, then you have to adapt your script for manual testing. For a manual script, you just adapt a Word document or an Excel sheet or the like. But if the process flow changes, you have new windows, new options, then you have to adjust your script for each and every provider that you're selecting. The maintenance of scripts is something that I always discuss with my end users and should never be underestimated.

We are not using the Capture 2.0 feature at the moment. We are planning to use it in the future. But at the moment, due to the heavy workload on our plate, we haven't had the chance to look into and to roll it out. We are familiar with the concept of Capture and it's a very nice feature because it makes the collaboration between business and IT much easier, and business can be involved in test-automation topics and activities as well.

We have three roles in our environment. We have the key players, who are the project managers, the persons responsible for test automation overall in the respective teams.
Then we have the test automation engineers who are responsible for creating test scripts and to maintain them; sometimes they run them as well. And finally, we have the executors, the ones who are running the scripts, checking the details and, if something is not working fine, going back to the test-automation engineers and asking for support and help.

I rate Worksoft Certify at nine out of ten. I'm happy with the tool, I'm happy with our colleagues at Worksoft. We have a very good relationship, we can bring up everything. There isn't much I can complain about. I'm happy at the moment with Worksoft.

We use it for SAP and ERP. I think from the SAP and ERP point of view, it is very good.

When we used it for web based application, so it was quite poor.

One, it's module based and it's giving functionality. That is, first of all we can really implement it as an ERP (Enterprise Resource Planning). We can utilize our concept and our functional people to work with the tool and without having a lot of technical knowledge, so they can really work with the tool with small technical knowledge. So that's really a big thing for us. We can use the tool with our existing people so that we don't need to have a new technical expert using the tool.

It is poor for a web based application. We are living in very integrated organization are most of the companies out there. There's the big companies we have these organizations that we work with, where we have plenty of different services which are very much connected. So, we are really looking solution which really can support all the different services. We really need to focus end to end instead of stand alone case. 

Whatever reportings are there, so they're really not very user-friendly. So there is a lot of technical data instead of user-friendly data. It would be wonderful to add web support, I would really like to see this in the future.

If I would evaluate it on a scale of one to seven, I would give it a seven. 

It's not very scalable. 

They struggle to provide an accurate solution.

I think it was not really very difficult. It was quite easy to set up the tool. The architectural tool was really easy.

We define certain different criteria. We check if the vendor is giving support for the tool, and we also evaluate the price. Of course, we also take note of the ROI of the solution.

This product is a good one, if you are really focusing as an ERP. We really need to have a solution which is really end to end rather than just focusing on key product. It's good to be aware if there something that can support you end to end rather than one specific product.

When talking about continuous testing, so then the tool is not really good at all because you need to purchase some additional tools. For example, the Execution Manager and  other tools, which are an additional cost. 

Another added cost is the payment necessity for reporting.

We are using it so far for regression testing. We have reduced 5% to 10% of regression related issues. This is a huge impact on our organization. So that's really quite good improvement in our eyes.

Speed at which it processes our testing scripts, detail of testing results for audit validation, documentation can serve as training document for new employees.

Prior to implementing Worksoft for our testing, the scenarios required eight employees over two weeks to complete (waiting for hand off of next steps, employee availability due to meetings, etc.

After implementation of Worksoft, the same test scenarios currently run in approximately 45 minutes – hands free! We have also implemented Worksoft into our SOX Control documentation processes.

Approximately seven years.

No.

No.

Excellent. They always went above and beyond to timely support our needs

Setup of the software was extremely easy. Setting up the scripts was complicated at times (due to our customization of our SAP clients).

However, most routine transactions were handled with ease.

I am not familiar with the pricing or licensing fees.

No.

Make sure to be prepared for the amount of time it will take you to complete the testing scripts.

Have a dedicated team that performs your test script setup and execution of these tests.

Include your business users to ensure the processes are clear and accurately reflect their business processes, not what you think their processes are.

Well, obviously it's the test automation piece of it. Creating our scripts and being able to execute our SAP scripts in order for us to be able to do comprehensive regression testing of our SAP system.

Well, it's definitely reduced the number of defects that we have. Before, when we were manually testing everything, it was unclear to us where changes that we were making, where the impacts were in our system. I mean, we've got very smart people that work here, but they were not 100% on where the impacts were going to occur with their changes.

By automating our scripts, we were able to do more comprehensive testing and therefore eliminate the number of defects that were be introduced into our environment.

There is an impact assessment piece to Worksoft. I think that's one of the pieces that we still struggle to use.

What the tool does is, you can analyze a transport to know where your impacts are. I think that's one of the challenges that we still have -- is just getting a better handle on how to use the Impact Assessment better for us.

In our regression testing, all that, we've got a pretty good handle on that. We identified things. When we run the analyze of a transport, we're not...we've found other ways to do things better.

Four years now.

No. It's been very, very reliable. The only issue, we've virtualized everything. I think the only thing, the only real issue we encountered was with our virtual machines. It was not a Worksoft issue.

It was on our side -- once we worked with our sys-admins to investigate what was going on, they were able to resurrect the issue and we were fine. It was not a Worksoft issue at all. It was something with our virtual environment.

It's been very scalable for us;

Depending on how many licenses, it's all seat licensed. It's not a concurrent license. If you don't have people logged in, more people can't use your licenses from your pool.

You assign those licenses out to people and as you continue to grow, like I was talking about with our virtual machines, we do everything in a virtual environment for all of our Worksoft work.

As we've needed more virtual machines, we've got an image of our Worksoft clients and we just add them into our environment.

3 out of 10 -- You submit an email, so there's not really a hotline that you can call to talk to somebody on the other end.

I had familiarity with several tools out there. Turnkey is probably one that you guys have looked at. I've done a lot with the quality center products in the past...Quicktest Pro.

There's other test automation tools out there. I've got a background with test automation and Worksoft...really it was between Worksoft and Turnkey. Worksoft, we just found ,was going to better meet our needs.

Yeah. That was actually pretty straightforward. When we set up our environment, they gave us a technical support person that really walked through with our system administration team to get everything set up in our environment.

As far as training, they have a very comprehensive training program. I'm sure it's all ... I'm sure it's very good. What we did and what helped us to be very successful is we went through and did live test cases as part of our training.

Instead of spending all the time going through the books and learning the tool in and out, we went through and we got right into creating our test automation with the trainer that they sent out. That hands on experience was, it was priceless. It helped us to be successful and get going with creating all of our test automation.

Make sure...before you start any of your test automation, make sure you have the appropriate support from your business.

What I mean by that is, you have all of your business processes documented. There is a lot of prep work that goes into creating your test automation to make sure you're creating...you're creating your scripts in the most efficient manner. A lot of this testing is done based on your business process. You need to understand your business processes.

For my team the most valuable feature of Certify is its flexibility and ability to include logic in the test steps. Our previous test tool could only handle linear test processes.

The build/maintenance times for testing are much faster. It also allows us to build tests for applications our previous tool could not do.

The faster build times allow us to respond quickly to ad hoc requests from customers that were not possible previously.

The user interface isn’t the best. It is functional, but the different areas look too similar so it is not immediately obvious which module you are in. It does get easier as you become more familiar with the application.

We have been using the solution for seven months.

We have seen one issue, but it is minor and reproducible so it is easy to work around and only appears in a very specific environment.

We did not encounter any issues with scalability.

The technical support is great. It is very fast with a friendly response. Everything from the initial install to tickets is handled quickly.

Our previous tool couldn’t automate many of the applications we needed to test. Certify also allowed us to increase the number of tests we could maintain with our current team.

The setup was straightforward. We are running fine on a single applications server that houses the database and license server. The clients themselves have almost no configuration.

I was not involved in the pricing negotiation.

We evaluated Tricentis and Selenium.

Expect to include some funds for training or have someone in the QA group who is familiar with the product.

You can start making simple tests immediately, but to fully utilize the tool you will need someone with experience. It also requires some organizational discipline.

The tool is extremely flexible and can fit whatever process your team wants, but everyone needs to know and follow the process or you will end up with a confusing tangle of tests and data.

All the different kinds of recording features (learn) on the different technology platforms such as Java, SAP, and HTML are its most valuable features.

It facilitates 80% of our regression tests.

We have had problems with an external cloud service (technology is Adobe Flex).

I have used this solution for two years.

There were no stability issues.

There were no scalability issues.

The technical support level is quite high. We are very pleased with its flexibility of services and response time on issues.

Previously, we just had manual testings.

We had quite a slow implementation. The process has given us time to learn and do things in the right way. We have also received good support when we experienced any issues.

The consultant's rate is very high but Worksoft is quite flexible.

We did do a PoC and compared it with other products from SAP and HPE.

Take your time, do a business case and plan from the beginning as to how your organization will work with the test automation.

In the beginning, there will be different technical issues between Worksoft and your business application, which will take some time to fix. You also need good internal support from the people who know the business application. Support from the managers is a must.

Provides capabilities in testing SAP ERP. It’s easy to create test cases/processes using Capture/LiveTouch functionality.

We have been able to increase our overall testing maturity and test automation coverage. We have a large ERP program ongoing which requires a lot of testing. Our goal is to automate the majority of system integration test cases with Certify. This brings efficiency and quality to the program and also supports the continuous services in the future (less time needed for regression testing).

Testing of web based applications can be really cumbersome at times. But this is partially due to the way the applications themselves are developed.

We have been using this solution for one year.

In terms of stability, the new version (9.0.3), especially the client and its UI, seem more sluggish compared to the earlier version (9.0.2).

There were no scalability issues.

We have not used Worksoft’s technical support so far. We have been in “project mode” and received direct support from Worksoft team related to configurations and also for our version upgrade.

We didn’t have any other tool in use.

The initial setup was straightforward. Afterwards, we also setup Certify on the Citrix environment to be used by our partners.

What do you advise others about pricing and licensing? We will have a large number of users so we chose an enterprise license.

Plan your training schedule well. The tool is pretty easy to learn, but you will need some weeks of hands-on usage before you learn to use it efficiently. We had a too long break between the first trainings and the actual start of using the tool.

A lot of the features in Certify are valuable, but a few of those are very useful and make automation easier.

The Content Merge feature allows us to copy the Certify items from one project to another, either in the same database or in a different database. The feature set of Content Merge allows several abilities, so as to control the behavior of copying these items from one project to another.

The Recordset and Layout features help us in increasing the test coverage and reducing the test cases count in a simpler way.

Other features such as Certify learn, HPE QC integration also come in handy.

SAP automation was very easy with Certify; it reduced the automation time and also, the creation of the test cases was easy.

Web automation can get a little better.

I have used this solution for two years.

As of now, there are no stability issues.

As of now, there are no scalability issues.

I would give the technical support a 3/5 rating.

I have used Tosca Testsuite before moving over to Worksoft Certify. It depends on the client's requirements that we move over to different tools. So, nothing is specific.

It was a little difficult for installing, but the setup was not complex.

I am sure on the pricing part for Certify. One will need to connect with the Worksoft team for that.

If you are working on SAP automation with Certify, then it’s the best tool to work with. Other technologies too work well but are not that stable.

In the near future, I guess everything will be sorted out and this will be the best automation tool.