Worksoft Certify Reviews

We have automated quite a number of test cases in last year.

We have a weekly release. A weekly release means every week that we have testing going on. The particular year where we started is called the asset management area. We could never run our Intuit testing, and we have a whole lot of Intuit testing. This one product helped us pass the testing with the Nighthawk testing, which is working on the Nighthawk manager. That's the one that we use. We can switch it on in the night and run the testing, then come back in the morning and see what has been completed. If there is any fail, we can even analyze it. We can use the evidence document to pass it on to development team to tell them, "This is where we failed."

It has improved on our defect management time. It has improved our test execution time. I don't need to manage these things, just sit somebody down to look at how the script runs. There are a lot of ways that it has helped us.

It is very easy to maintain. With scripts, I can change one line and in one step. Whatever I want, I can do. I don't need to be an expert to use it. Being a manager without a whole lot of technical knowledge, like an automation person, I can change Worksoft using what I learned during the training. That's what I like about it. Anybody can do automation.

I love the Capture 2.0 feature. When you are doing a normal manual testing, go ahead and switch on the Capture 2.0 feature, then capture everything and pass it on to your teams who can convert them quickly into test automation. With this feature, it is saving our automation creation time by about 60 to 70 percent. It is also helping our manual testing time in terms of catching all the evidence documents. 20 to 25 percent time is being saved because of this product and Capture 2.0 feature and what we are receiving with the good documentation.

I would expect more opportunities to automate Java.

I would like it to analyze what we are not using. 

I would like to see the impact analysis integrated with the performance testing tool. We have multiple tools doing multiple items. I would like to have one common tool. 

I'm also interested in load testing automation and whether we can create a script for it, then can we use the same script for my performance testing? 

Stability is quite good when compared to other things. I don't want to say it gets 100 percent rate in terms of stability, because I'm using this for about ten to ten and a half months right now. It has been close to a year, but I'm really amazed when comparing it to any other tool that I have used in the past.

I don't have the words to express its scalability besides awesome. The amount of changes that we have seen are tremendous.

We don't use them frequently, just when we have an issue. I would rate the technical support a seven out of ten. Most of the team is good and helpful. However, I would like them to evaluate the issues a little more sometimes before escalating them to engineering.

We were previously using Micro Focus UFT. The tool was good, and we did not have a lot of problems with it. The only problem was SAP changes a lot of things every time. The frequent changes were causing a lot of issues for us in terms of automation. We were able to automate many things, but the maintenance was a big problem for us. 

1. You needed to have a person who had the coding knowledge to do it.
2. The frequent changes made the scripts useless. Then, we would have to come back and redo a lot of things. 

This is where we were looking for a product where we could have minimum maintenance that anybody can automate. This is the concept why we came into Worksoft.

We used Worksoft team initially. That helped us through the setup and other things. They did a great job. We probably automated about 275 tests in less than five weeks, including the setup. 

I suggest people go head and use Worksoft, along with their services, when you buy the tool. They will help you to onboard it quickly and set it up for you. They will do lot of the automations. They will help you with lot of these practices, then you can take it over from there.

We have seen ROI.

By using automation, it reduced about 75 percent of the time when compared to any other tool. The changes for Worksoft used the same script as the script automating UFT. Manually, running our tool takes about 4 hours, but with Worksoft, we were able to do it in less than 30 minutes. Whereas, the same thing that you had to do would take you almost 55 minutes to an hour. There is now a 50 percent savings in terms of other automation tools and an 85 percent savings in terms of manual to automation.

We have seen more than 40 to 50 percent reduction, in terms of all around time, where we were doing five days a week for a major maintenance testing of our first cycle. We have reduced it down to three days now. So, that is a 40 percent savings which we are seeing. We are not completely automated because we are still doing sampling. We have just automated a good 40 percent of our things. When we go to 80 to 100 percent, it will reduce 60 to 80 percent of our time, which is what we are looking for.

We looked at Panaya, but Panaya is not a desk automation tool. We are using Panaya for some of our impact analysis.

We did try with Selenium and many other vendors. A lot of other tools have a small test director that we tried. The ease in using this tool is very good because our business team can go in and use it. So, anybody can record for me and capture. Then, we have a very small team of automation testers who can convert the information immediately into a reusable component, parameterize it, and do the records sets. In that way, with a very small set of test automated guys, we can do much more.

I recommend this solution already to my colleagues worldwide.

We run this on seven different multiple applications. It starts from SAP, goes to the UI, comes back to SAP for violations, and then goes to mainframe for validation. Then, we use Java Web as a Java. After that, there is another HDM which we try to validate. Also, we are trying to validate a third-party application using it, because we have used a lot of their components trying to do a mock type of filing import/export option with the tool. 

We have used this solution for web UI testing, as we are on SAP Web UI 5.0 right now. We use this very heavily right now in our asset management area. It is very easy to use. The Capture 2.0 together with it is helping us, because we are now able to recognize some objects through Capture 2.0. We also have LiveTouch. This is another advantage where you can use this to capture multiple items at a time.

We use Worksoft Certify to test our SAP System. We have a global instance of SAP, which we started implementing in 2012, and we are still in the process of implementing. We have rolled out SAP to about 80 percent of our manufacturing and distribution. Right now, the remaining projects are a small distribution center and sales offices. We have ongoing projects, and three times a year, we release a new version of SAP. We rolled out SAP to a new geography, and we also added new features for our business users. Thus, as part of those projects, we use Certify to do regression testing of our existing business processes, and we also use it in the project to test new functionality.

When we are rolling out in a new country, we do a configuration for that new country. We use the automation test to test the business processes and prices of that new country. It is sort of semi-automated. Our business analysts generate sales orders from the new country, and we will run them through the shipping orders to cash, the shipping steps, and the concrete steps. Then, we get a set of documents to review. The business analysts review those documents to make sure the order is processed correctly. So, it's not fully automated, but it does help cut down on testing a lot when we roll out to new countries. 

For regression testing, that is fully automated. We have tests where the software checks the results and either returns a pass or fail. These are run as a regression suite anytime we push a change to production.

We do use it for the end-to-end testing of packaged applications, primarily SAP. We do have some plugin applications that we use it to test, which are part of the business process. We use Salesforce for CRM, and we have a custom built eBusiness application. While we don't do extensive testing of those applications using Certify, when the business processes touch one of those applications, we do cover those application with another certified test.

It has cut down on the amount of low level, grunt work that business analysts have to do and can free them up to do more critical thinking. Before we had test automation, we were running tests and relied on people, which was very time consuming. A business process test might have 100 to 150 steps across different applications, and we don't have a single person who has expertise in all those applications. When executing a manual test, we have to balance the test between different people to do their steps. In a typical project, we might have 100 to 150 of these types of tests running. The coordination of the testing process where you have to have different people available at different times is very time consuming and inefficient. What automation has done is cut that cycle dramatically because automation does not have to worry about having to find the right order management or warehouse person to do their steps. The automation just runs through, then the business analyst can review the results afterwards. Therefore, it has been more efficient, cutting our testing part down by almost two-thirds to 75 percent.

Our automation tests are more robust than our manual tests. We found our test lab would grow over time because we didn't have a lot of discipline within the team for manual testing to have a master test which could be used repeatedly and revise as necessary. So, they were creating a new test for every specific little thing that they wanted to test. They were  setting up these manual tests where they had ten to 15 tests which tested the same thing, but not quite. Therefore, it became a bear to manage. Whereas, with automation, because it is more controlled, we have a core set of about 125 automation tests entered into our library. That's in our change control. Therefore, we know exactly what the state of our tests are. 

If there is a new business process or new wrinkle in a business process, we didn't have a defined process, so now we are updating automation tests. The quality of the data that we're getting out of test from automation is much higher than we received out of manual testing. If we know the automation suite is parsing, then the application is working properly. With automation, we have more confidence that if the test is parsing than the application under test is working correctly.

It is fairly straightforward. We have some deep expertise after using it for five years. We have some people who know it now very well. 

This type of marginalization of the code inside Worksoft Certify has been very valuable to us. The ability to capture documentation. We are a technology company and are regulated, so we have pretty stringent requirements. We use Certify to capture screenshots and evidence during testing. We can capture every screenshot during the business process including a document and hand it off to the auditors. It makes defending an audit very simple. We can, if they ask for evidence, produce a document that shows the business process of every step and the screenshots showing all the pertinent data, which has been pretty useful as well. This is the report feature with Certify. When you run a test, you can either have it generate documentation or run it in the background. Most of the time when we were running regression tests, we just run them in the background. 

We don't generate documentation, but we could turn Capture 2.0 on, where it creates a screenshot. As we're starting the test, it creates a screenshot of the application it is testing every step of the way and produces a word document or a pdf at the end that you can then hand off to auditors and show them the actual flow of the process that you're testing. However, we do not use this feature.

One feature that we have been asking for has been to treat tests as code and store the source code for tests in a configuration management tool. Right now, for version control of testing, it's all internally within the tool. If we have a test of a business process and want to revive that test, our methodology now is purely manual work. We go into the tool, create a copy of the existing test, and call the next one: v2. Now, we have two of them and the only way you can tell them apart is by its naming convention.

This is not an efficient way in terms of how modern applications do version control. If this was code, we could plug it into a tool like Git or GitHub to manage of our versioning and branching. The reason why we want to do this is that the application which we are testing branches. When we branch the code, we put a bunch of new functionality on the new version while our production version stays unchanged. Then, at the end, we merge the two together.

From an automation testing perspective, we have to run tests on both. Then, we have two current versions of our test. So, it's a bit hard to manage in the tool right now because you can only have this manual approach where we are tracking it via the name convention. Whereas, a modern way of doing it would be to have our application plug it into a version management tool, like GitHub, where we would store the code and could just pull in the version of the test that was applicable to the version of the software that we were testing.

This is something we have been asking for for a while now. I understand that it's in the pipeline, and it may be in their latest version (version 11). This is something that we will be looking into this quarter.

The challenge that we face everyday for test automation are more internal (people issues). We need change management and getting people to accept automation instead of the technical limitations of the tool. The tool does what we need it to do from an SAP testing perspective.

We have no stability issues. 

Maintenance-wise, we have one system administrator who is not full-time, since it has been pretty stable. We don't change much compared to other applications. This application is pretty hands-off.

We should be upgrading to the latest version in next couple of months.

At our peak usage, we have had seven people working on it, and have had no issues with that. Now, with our current work load, we only have three people. We only run our test suite. It was one of my goals on this project that we had the infrastructure setup, so we could always run our entire test suite overnight. As we built out our library, this meant expanding our infrastructure. Right now, we have 100 to 150 integration tests, and some of them can take ten to 20 minutes to run. A single instance of Certify can only run one at a time. Thus, we have had to think about how we set up our infrastructure in such a way that we can run the entire suite of 150 tests in six hours.

The way that we have done this is to split it up amongst servers. Therefore, we still have extra servers for execution. We have four servers now and run the tests in batches of about six queued up at a time. In this way, we can run our suite of 150 in parallel across four machines and get it done in about six hours. Right now, we do this manually. We do the manual breaking up and monitoring. I know Worksoft has some tools which automate this. This is something that is on our radar to look at as we grow. However, right now, we just manually manage the process.

We have three test developers using it. These are the people actually building tests. In terms of consumers of the test automation, we have probably 35 to 40 business analyst.

The technical support is pretty responsive. We haven't had many issues with it. When we were doing an investigation into doing web testing, we ran into some roadblocks. The team at Worksoft was very responsive. At the end of the day, it came back to technical limitations of a tool. I have been pretty impressed with how responsive the team. They were always able to answer our questions to the extent that the tool was able to do what we needed to do.

It was all manual. For convenience, we used Micro Focus ALM for tracking our manual tests. We still use that as our central hub for our test documentation. We weren't using any test automation tools in IT. Within the organization, we have R&D groups that develop software for various systems and medical devices. Those teams are running tests and code. They are in automation test suites, and I was part of one of those teams before joining IT. However, in IT, before we started using Certify, we weren't using any test automation.

Manual testing was costing us a huge amount of money. We did a double rollout of SAP. We split it over three deployments:

• With deployment one, it was just one division in North America. We had over a 120 people doing manual testing for a period of about sixteen weeks. Add up the cost of that. 
• As we moved into deployment two, we were going to have to test new functionality and also regression test what we'd already booked. If you took the amount of testing that we'd done in deployment one, even if we weren't going to redo all of that, we're going to have to do 50 percent of that. It was going to be a huge manual effort and a sunk cost. We'd put all that money into manual testing and wouldn't have an asset. It would be money that we are basically suspending with no reuseability.

It was a pretty easy decision to convince the team to move to automation because it would be an asset that we could reuse again. Over the last five years, we've shown that we've had a positive ROI on it. The initial upfront cost in terms of licenses, plus all the money that we spent developing tests, has proven it's worth. Now, we can do a regression test suite in ten days as opposed to sixteen weeks.

The setup was very straightforward. We did a proof of concept with Worksoft. They came in and had an engineer onsite. We set them up on a server and pointed them at our test SAP system. They built a couple of prototype tests for us. When it came to implementation, we had an existing prototype that we looked back on. I have a systems administrator on my team, and he was able to pick it up pretty quickly. 

The documentation was good. We did the install on our production system, copying over our prototype tests. We used that as our starting point for building out our library. We also sent out a couple of guys for training.

We were up and running with a functional system within a couple of weeks. The challenge, at that point, came down to training our business analysts on how to use the tool. This took longer than getting the system up and running, which was pretty straightforward.

We did the deployment ourselves. It took less than a week. Internally, we had one system administrator do the bulk of the work.

We ran the deployment on Windows Server. We have two machines: a database server and an application server. Our test developers can logon via Windows Remote Desktop to access those machines. They built all their tests out on that system. Architecture-wise, it is hosted all behind our firewall, but it is all server-based. No one is building tests on their local desktops. It's all server-based, and we can share some of our scripts amongst our team members.

My primary team is offshore. They are in India and Bangalore. Therefore, all of the test development is done there. However, we can access the central test library seamlessly, and  the test strategy for setting up and standing up servers and installing the software was pretty straightforward.

Our ROI is primarily a reduction in testing time. The testing, when we were doing it manually, was 30 to 40 percent of the project's cost. This was a $450 million USD deployment of SAP, and testing is 30 to 40 percent of that cost. We spent probably about a million and a half in test automation, but managed to reduce our testing times from weeks to days. There is a clear cut return.

If we write a new test that's 80 percent the same as an existing test, it is pretty straightforward to reuse the steps from existing tests for our new tests and build upon them. We found that there has been increasing ROI automation as we built up our library. When we write new tests now very seldom is a new test build from scratch. It is normally a variation of something that we already have, so we can turn those around pretty quickly within a couple of days to two weeks.

We ended up buying too many licenses. They were very good at selling it to us, and probably oversold it a little. We bought 45 licenses and have never used more than twenty. However, they gave us a pretty significant discount on the bigger license, so it made sense for us to buy enough that we wouldn't have to go back and ask for more. 

At that time, we had budget to do that. The licensing is pretty straightforward. We have considered using them to do robotic process automation and may still do that. Initially, we were worried that our license might preclude us from using the tool for something other than testing, but when we checked into that, there is no limitation. 

We could use Certify to do robotic process automation, which is basically running a process on your correction system instead of your test system. Therefore, we may do that in the future. 

We also looked at HPE UFT (the HPE automated testing tools) and SAP TAO (SAP's own internal test automation). The reason we pretty quickly went with Worksoft was primarily the responsiveness of the team. The evaluation happened between deployment one and deployment two.

When implementing SAP, we had IBM as our system integrator. We went to both SAP and HPE asked them to show us what they could do for test automation. We also looked around and found Certify as a third candidate. The response from the Worksoft team was far higher than the other two. IBM wasn't able to produce sufficient expertise to demonstrate the SAP test automation tool and same with HPE. I also didn't have a good response from them. We felt, " If this is the level of support that we were getting during the sales cycle, how will it be after the sale has occurred and we have to go to them for support?" Whereas, Worksoft was very responsive. They sent people onsite. They did a proof of concept using our system and data. There was a pretty clear cut night and day difference in teams and companies involved. I didn't get a chance to evaluate the technology of the SAP or HPE solution because their sales teams weren't responsive.

We have a dedicated team of what we call test developers who are specialists in this application. While I don't use the application myself, but they're pretty productive with it. We have a team using Certify for SAP Test Automation and a team using Selenium for web application development. The SAP test development is more efficient than the web test development. For a similar sized test development project where they have to test and develop five automated tests of a certain method, we can turn them around in SAP faster than we can turn them around in Selenium.

Now, it might be Selenium has a higher learning curve than Certify. Or, it is easier for test developers to get good at developing test units using Certify. Selenium is far more technical. Of the two tools that we use, Worksoft is more user-friendly than Selenium.

The technical instrumentation was pretty straightforward. The tool does what we need it to do. The primary challenges that we have had with test automation have been change management, getting the old, greater IT organization to accept automation as a substitute for manual testing. Culturally, within our organization, we put a lot of pressure on our business analysts to thoroughly test the application, and if they have never used automation before, there is a fear factor there saying, "I'm responsible. Then, I want to see it with my own two eyes." 

I recommend expanding, training, and coaching people that automation is just as good, if not better, than manual testing in terms of finding bugs and proving that the system is working correctly. It is far faster, and you will get a lot of your life back. That has been the biggest challenge for us: Telling that story and expanding the use of automation throughout our organization. Now, automation is pretty mainstream and accepted, but that was the biggest challenge for us. It certainly wasn't technical challenges.

We don't use Capture 2.0. We found it easier because we have a large pool of business analysts who are not certified users. Our process for capturing the business process which needs to be automated, therefore we use Zoom Recorders. It is like a WebEx tool. It has a screen sharing device and a record feature with audio. We find the audio is quite beneficial. When we capture the business process, we will have people record in Zoom, annotating with their voice (doing a voice over of what they're doing). Then, we handed it off to the test engineers to build up the automation. We look at Capture some time ago and felt it wasn't as efficient. Capture 2.0 is the newest version, and we haven't really looked at it in-depth. We will certainly reconsider it, but right now, we are not using Capture 2.0 to do business processing.

We use web UI testing to a smaller extent as part of the SAP business process. For a business process which incorporate Salesforce, a field service engineer might order a spare part. This is a post process that spans both Salesforce and SAP. For the first half of the processes, we use Certify. We did attempt to use an in-depth testing of web applications sometime ago. At that point, we felt there were some technical limitations. The project was to use Certify to do comprehensive testing of our Salesforce application. However, we found when we did a deep dive that there were some aspects of Salesforce and proprietary screens which Certify already struggle with. At that point, we decided to switch to Selenium which is the industry standard for web testing. Now, we do most of our tests on Salesforce in Selenium. While Certify has become a lot more capable with web testing since then and the newer versions are better at it, at the time we investigated it, we felt that Certify probably wasn't up to scratch as a web testing application.

Going forward, we will look at Certify again as a web testing application tool since it is more efficient than Selenium. We are finding that it's costing us more to develop a test for a web application than it does to develop a test for a SAP based application. We want to take a look at them again as a solution because it might help increase our efficiency as most our applications from this point forward will probably be web applications. So, there's a lot of work to do in that arena.

With our eBusiness and Salesforce suite, we are not even close to full test automation coverage. We still have a lot of work to do. So, it's worth us looking at Certify again. We're expanding into big data and big data analytics. There are a whole slew of terms around that with regard to testing. E.g., how do you verify that your data's accurate? We are just dipping our toes into it, as we haven't done any model testing yet. That is something that we have to look into. There are a lot of areas where we could use it.

In the last couple of years, we have become an established and accepted part of the SAP testing in the organization. We are a fairly conservative group. Now that we've done the SAP testing, we need to start looking at different horizons of mobile, big data, and web testing where we still have a lot of work to do in terms of building up our automation.

We are in the retail business, so our primary use case is automated regression testing on our supply chain. It helps us oversee our internal economy, including the warehouse, HR, and the stores. 

We prefer Worksoft over other platforms because it's a low-code solution

We can't get the process intelligence module to work properly. We can't get the impact comment that analyzes the incoming development code to run, either. We've also had bugs in the CTM and execution manager in the past year. It took technical support a long time to resolve this issue. We escalated it so that the vice president of the company was included as well. 

I have used Certify for a few years, but the company has been working with it for five years. 

I rate Worksoft support five out of 10. They have some general issues that they're they're not testing, but when we open a ticket, they give us a hot-patch fix. It seems like they're not testing their products enough. On the other hand, Worksoft support has been responsive to our requests. They've addressed the whole list of issues, so we have no more open tickets. Once they escalate something, they're efficient. 

Neutral

The vendor installed it for us, but there were still many issues. We've reported over a hundred bugs and defects. In that time, we could run stuff, but sometimes it has been standing still. 

Worksoft is reasonably priced. 

I rate Worksoft Certify six out of 10. Before implementing the solution, you need to think about how you will use it because it has so many modules. You should consider how you will use those and do a proof of concept. Worksoft hyped up the process intelligence and impact modules for two years, and they still aren't working for us. Before you buy it, make sure it works in your environment. In the future, I hope they have a more stable product with more internal testing and quality control. 

Our primary use case for Worksoft Certify is the automation of test cases. Performance-wise Worksoft Certify is very good. 

The most valuable feature of Worksoft Certify is that it has a codeless environment, so you do not have to be a programmer to use it. 

Worksoft Certify supports multiple interfaces and applications like SAP, Web, or Silverlight Java, and Mainframe. It is easily integrated. 

I have used the tool for a long time and much has improved over the years. Many new features have been added and the tool improves on a daily basis. The UI is much better now and it looks much nicer as well. 

Worksoft Certify's tech support's response time could be improved. 

I have been using Worksoft Certify for nine years. I use the product on a daily basis. 

Worksoft Certify is easy to scale. We have somewhere between 60 and 80 people using it at the moment. 

We do have technical support from Worksoft Certify. When we do run into issues, we first see if we can sort them out in-house. If we are not able to, we reach out to them and they look into the issue and sort it out. 

Positive

I previously used Selenium. Comparing these two products, I can say that you need much more IT knowledge to use Selenium. Worksoft Certify is quite easy to debug. 

The initial setup was easy. On a scale of one to five, with one being very complex and five being very easy, I would give the Worksoft Certify deployment process a five. It also does not need very much maintenance at all. 

We deployed Worksoft Certify in-house. The deployment did not take much time at all. It took just a few hours. 

Worksoft Certify is a bit costly. It is a good product so the cost is a bit high. But I'm not sure about how much we pay for it exactly. 

The advice I would give to someone looking to implement Worksoft Certify is this: contact the provider and request what you need to be installed. Once the solution is installed, you can open up the user manuals and start learning it. It is easy to learn and implement a project in Worksoft Certify.

On a scale from one to ten, with one being the worst and ten the best, I would rate this product a ten overall. 

The primary use is developing automation on SAP at a medical device company moving from Micro Focus UFT scripts.

We were able to increase the percentage of automation from between 40% and 50% to 80% within a year.

The most valuable feature is the ability to automate quickly and to maintain and update scripts.

We would like this to be able to be used outside of SAP applications, as it would be good for other types of products.

I have been using Worksoft Certify for three years.

We expanded licenses after a year.

Prior to this product, we used Micro Focus UFT.

We evaluated an SAP escalator from SAP in UFT.

My advice for anybody who is implementing this product is to get consulting and on-site training.

We are using the on-premise version and testing an eCommerce platform: SAP Hybris. This would take in all of our vaccine orders over the Internet. That is primarily what we are testing it on.

The autotesting piece has allowed us to complete testing more quickly, which helps us with our agile sprint. We are getting our releases out at the end of every four to six weeks, then pushing them out. Thus, we're delivering value in terms of our website being faster. The whole autotesting piece has helped us get there and deliver that. This has saved us time.

We are able to take about 1500 manual test cases, working with the Worksoft resources and system, and clobbered them down to 600 to 700 test cases. Our testing has gone from what would have been a course of six weeks down to a little over a week's worth of testing on the autotesting. This is possibly because our systems aren't all in sync at the moment, and we're still in the process of fine tuning this. When we finish fine tuning them, the testing may even be quicker.

For the tool, its valuable features are:

• The ability to build objects. E.g., if you have 50 test cases, but in those fifty test cases, they all had embedded a place order flow. You could build an object of place order, then all 50 of those test cases could use those steps in that object. This has been a real sharp feature, because you don't have to do those order steps 50 times like we used to do on our manual tests. You now have an object sitting out there where you can just reference the object, if you will. That has been a sharp feature.
• The general Capture tool, where you can walk through and mimic what you think is a test case, and the whole time it's capturing your steps. You can then use that to fine tune it and pull a test case out of it. We have it hooked up with Micro Focus ALM. This is where we have all of our test cases living, and it's sitting in that repository for us.

It is a pretty easy tool to use as far as automated testing tools go.

We went into this with the thought that we wanted to be able to hand this off to a business user, so the business user could develop their own test cases automatically through automation. We are not seeing that. We still have it assigned to an IT professional, someone who is certified in Certify. We constantly have to have that type of person around who can build these test cases for us. At the moment, there is not an automated testing tool out there that will allow a business user to develop their own test cases, and certainly not at the level that we want it to be it. So, this may not have been a realistic goal on our side to expect that one of our business people, who has their real job, could spend a couple hours here and there developing test cases on an automated testing tool, like Worksoft or any other.

It's a software package, and you have to know the software to be good at it. You have to have a certification in the tool to be able to be really good at it.

It would be great if our business testers could develop their own automated test cases. However, we either have to bring them up to a level of certification on the software or go hire somebody to do it. Worksoft, in essence, is the Mercedes-Benz of testing tools. If you want a Mercedes-Benz, you have to pay a bit more money. 

With every release you do, you have to go back and touch your old test cases and bring them up to speed, or develop new test cases. In the beginning, that is a challenge because you have to have someone who is certified in the tool to help you develop these test cases.

It is a little complex for someone who is not in the autotesting space to learn it. Like any software, you don't show up to use Oracle Database on day one and think you know it. You have to learn it, get certified in it, and understand it. This tool is similar in that sense. You have to have someone who knows the tool and knows how to use it. It's not something that your business users are gonna pick up, especially if they have a day job. It will take a long time for them to pick it up without full dedication and going to get certified.

We have had no issues with stability. It's been stable.

We haven't jumped yet to other systems. We are definitely looking into that. There should be no scalability issues.

We came to Worksoft with the hope of doing end-to-end testing. When Worksoft came in, we challenged them for a week to show us that it worked where you jump systems. They literally put people on our site for a week. It was a quick little RFP. We did see it work, so we know it works. We saw it jump back to our CARS system. Literally, we pulled Revitas CARS system up, and it logged into our CARS system, that's a web-based system as well. It started going through the steps that we needed to go through in CARS, then passed the status back to our eCommerce system. So, we know it works for us.

We know that the systems that we have involved will work with the end-to-end testing, but we haven't gone there yet. It's mostly on our side, not the software nor Worksoft. We are just doing other projects right now.

We currently have ten people (tops) using it in our organization. 

The technical support was pretty sharp, friendly, and responsive. There wasn't anything glaringly wrong with them.

We have a guy from Worksoft sitting onsite. If I have a problem, I talk to him first.

Long ago, we used HPE QuickTest Professional (UFT). The reason that we had to get off it, because it was liked by our business people, was when we went to Hybris in 2015, they didn't work together. So, we jumped ship at that point and went back to manual testing.

While we already had manual test cases, we wanted to move to autotesting because we are doing agile sprints. Our sprints were down somewhere between four to six weeks, depending on what is in that particular sprint and various conditions of trying to get that sprint out the door. We are trying to get down consistently to four weeks. Consequently, we had these test cases, which was up at around 1500 before, and also manual. We needed to get them to run in quicker, shorter periods. That's where autotesting came in.

The initial setup was pretty straightforward. We had manual test cases already in place. Before you even got into the entire tool, you would have to do basically a cleansing of your manual test cases. We worked with Worksoft on an analysis period for about a month upfront, where they run through your test cases and make sure they understand what you're trying to test. Then, they try to map out a way forward as the best way to automate. Therefore, the gear up is the homework you do with them in that month before you even touch the tool.  That is the setup piece. 

The setup for us was wanting to know our test environments and putting together solid test cases. In our case, the account setup with customer emails, names, and addresses with all that testing data that you need. We spent that month getting the tests in good shape and all the prerequisites needed to run our tests lined up. Doing homework ahead of time then makes the autotesting run smoothly.

We are still deploying. We started with what we called a 'pilot'. We gave them around 300 manual test cases, which we got down to 141 test cases as part of our pilot. We ended up reskinning out website, and consequently, we had to go back and touch all 141 test cases. Some of them have now become irrelevant with the reskinning of our site. Therefore, we had to go back and reanalyze all of them, and find the ones which were still relevant, which were 90 to 95 percent of them. Then, we had to touch them up. We are in the process of doing that now, touching up our old test cases and building new ones on top of them. This took us a good six months, but we are starting to run regression tests now, though they are not formally in place.

We hired resources from Worksoft, but we do have our own testing groups. We have also hired some Worksoft certified people. We have been using Wipro for Certify testing and development, but our experience with them is not so good. 

Wipro was our testing vendor for manual test cases before we brought on Certify. My guess is that they did not want to lose our business because they told us that they knew how to work with autotest cases, and they didn't. They told us they knew Worksoft Certify, and they didn't. They were given a second chance and hired some Worksoft certified people, but it was a really big headache.

For deployment, we have two people from Worksoft and five or six Wipro people. The Worksoft people are far more productive, since they know the tool better than anyone, but they are more expensive as well. The Worksoft people are sharp. They notice things in our testing and point things out. Their understanding of test cases is off the charts. They picked up our systems very quickly. Wipro has been a bit more of a drag. It is because they're learning Worksoft Certify and don't really know the tool. We also have one or two business people involved who are not developing test cases, but they're project managing.

We are still building test cases, but we are running the testbed that we have, which is a few hundred test cases. However, we only need one person from Worksoft to maintain this.

What autotesting has helped us do is consolidate our test cases because our company departments were testing in their own individual silos, running their own test cases manually. Now, with autotesting, we have been able to eliminate duplication of test cases across those departmental areas. This has helped us knock down our number of test cases. Our test cases are also running more optimally. Therefore, it has very much helped in that sense, so we were able to eliminate a lot of test cases and get out of manual silos by running on autotesting, which is more efficient.

Autotesting can runs overnight. It can run faster than someone doing it manually. The assumption is the system is not making any errors, but someone who is manually doing all the testing could miss something, get tired, etc. Be human, basically. Consequently, the autotesting eliminates some of those types of errors. Put all that together, and we are able to run autotesting and get our whole testing cycle done along with regression testing for an upcoming release, which is being done on these agile four to six-week sprints.

Overall testing has gone from six weeks down to pushing a button on a Friday. We may come in a couple days later, and the testing is done. At the moment, it takes less than a week for the testing, as opposed to six weeks in the past.

We haven't really seen the cost saving come in yet.

The price is in line with everyone else's in the market. They are not cheaper nor more expensive than anyone else who was in our RFP. 

There is a cost involved to doing it, but once you get over the initial cost, then you'll start reaping the benefits and seeing that testing is getting done more quickly and efficiently. We are still early on with it, but the expectation and what we're seeing is that we will start seeing some savings coming out on the back-end once we have this done. 

We looked into TCS and Cognizant.

There is an initial mountain to climb, where you have to get all your test cases in order and have the data ready. This will make it a much smoother setup when it comes to having Certify people coming in. I recommend hiring Certify people who really know the software. Once you get it humming, this is where you will see everything you are dreaming of, where you start a testbed one day and within a week your whole testbed is running, then you have figured out all the issues and can rerun it again. This is where you start seeing the benefits of autotesting.

We have the Capture tool, but I don't know the version that we have.

We are not doing web UI testing for modern applications, as we have SAP ERP, SAP Hybris, and Revitas CARS.

SAP HANA S/4 HCM, FI, BI, Payroll, SuccessFactors, Fiori, other web HTML interface applications.

Our company automates a lot of functional tests for SAP. The automated tests are included in larger integration tests of SAP transaction, workflows and end-to-end processes. Successful implementation and execution of Worksoft Certify tests has been faster than execution of the tests manually. With other tools, it was hard to get successful technical execution right quickly. Automation is not easy. Worksoft Certify makes it easier yet flexible enough to handle some of the most complex automation tasks. Every time steps are automated, test execution times are reduced and application problems are identified faster. Every time that the automated script runs (many times if developed early enough and applied with each change of the application being tested - in the current environment and leveraged in another environment), the investment of work made to develop the script is realized. The more times that the test is executed, the risk of problems going undiscovered is reduced. Testing time is shortened. Avoiding the tedious tasks of manual testing easily repeatable steps saves money.

Worksoft Certify provides the ability to develop test scripts with a low bar of entry among automation tools. It’s very simple to develop. However, great care must be taken to use the tool as intended, otherwise efficiencies and effort savings will not be realized. Like all automation projects, well-designed automation, with thoughtful planning and application, is necessary or you will unable to get back the return after spending the cost in investing time and effort with the tool.

Worksoft Certify provided outstanding feature support for SAP test automation which is unrivaled by other tools in the market. If implementing and need to automate SAP, first pilot the tool with a representative scenario, and then evaluate that retesting the scenario multiple times will ultimately provide compelling overall value for SAP test automation effort.

Worksoft Certify can better identify web test objects by providing libraries through its Extensibility framework. It has improved with it's latest version through Extensibility Tools support and additional test object attributes. Object identification is a necessary barrier for the technical execution of a test script, aside from the script's needs to verify business rules. Automation engineers must affirmatively answer the question: Is the application automatable?

If Worksoft Certify or any other test automation tool can get object identification down consistently and flexibly so that maintenance of web object identification (and hence the web test script) is easy and done with no execution problems, it will allow more scripts to be developed and applied faster with the human test automation staff resources on hand.

A good part of the technical test automation work is to make the script run and affect script changes when the application under test changes. When done right, the business part of automating the test application functions remains the same - leveraging your test investment.

Certify may not the best solution for your needs, but among the commercial and open source offerings in the automation tools space, it should rank very high. It is also a leader in rapid SAP test automation, and can be used as general all-purpose test automation tool for varied clients like PC, SAP, and especially Web.

Only one or two tools I have experienced could be suitable in place of Worksoft Certify. However those tools also carry their own set of disadvantages.

In terms of stability, Worksoft Certify is a polished and mature commercial offering. We rarely encounter problems with the tool. When we do, Worksoft is very responsive to the issue.

We also find the new features in version 11 of the tool to be innovative. The more friendly and fast UI, greater support for test object Extensibility support, Capture 2.0 functionality and global search are all market-driven features that benefit the bottom-line in creating and maintaining Worksoft Certify automation script assets. The improvements are welcomed and we look forward to even more such improvements to make Certify a top-tier competitor in the automation market.

In terms of scalability, we only have a few licenses. We manage and maintain the underlying database actively to avoid scalability issues. We are confident that Worksoft Certify can scale to fit our automation testing needs as we:

• Scale-up with many more automated tests of varying length
• Scale-out with unattended lights-out execution.

I would give technical support a rating of 8/10, 10 being the highest. Worksoft is very quick to respond and address customer problems identified with Certify.

My organization has been using Certify since 2007. We are committed and very happy with Worksoft. I picked up experience with Worksoft Certify when I joined my organization and leverage my existing automation skills to quickly become effective with the tool. I still use Selenium/Java/Python and was expert-level proficient several market leading enterprise QA automation tools.

The setup was done prior to my joining the organization. The existing work helps make my drop-in experience very smooth. It would be harder from the ground up, since there is a database and application server that needs to be installed and administered, and procedures and policies to define so that development is standardized (establishing naming conventions, specific ways to do things when there are multiple options available, etc.)

We implemented and developed in-house expertise. A vendor team would have accelerated and made adoption smoother.

We get at least a 2x return on investment (time, cost, labor) over manual efforts. When tests repeat execution the ROI can go higher. Tests often repeat when executed annually with planned annual system upgrades and additional time, labor and cost to maintain factored in. Including and reusing the tests in other testing activities such as sanity checks and regression cycles increases the overall ROI and investment value.

Conduct a proof of concept (POC) so that you understand what needs to be invested with Worksoft Certify. You will need to understand what sort of time you need to initially set aside to learn and apply the tool, and how that time can be leveraged to shorten continuing development of Worksoft Certify test scripts (processes) as facility in the tool increases.

• Someone needs to know to administer it.
• Someone needs to know how to develop the tests.
• Someone needs to develop a representative automated test script for a candidate application in your organization.
• That person also needs to execute the scripts and then report results of the scripts.

That script needs to execute several times. Now a comparison needs to be done with the time it takes to execute the multiple test runs against the same time it would take to do the same thing manually. In this way, you can now quantify your savings.

Now apply the need over the instances in your organization and compare that to the pricing and licensing to see how many instances of this effort would give you a return on your investment.

You will find that you will have many instances that would justify the price and licensing costs.

We evaluated QTP and Selenium. Worksoft Certify was chosen because of:

• The comprehensive coverage
• Extensive support with SAP
• Additional web test automation support
• Ability to develop tests fast and execute them reliably
• Technical support of the tool

The combination of all narrowed down our choice to Worksoft Certify. It was a great choice in 2007, and if evaluated today instead of 2007, it still would have to be Worksoft Certify since the importance of SAP support is priority and is significant over any of the other criteria and support provided by the other competitors.

Perform a pilot. Apply it to your app under a test with representative workflow. Understand the process of developing Worksoft Certify tests by doing it. Execute the develop tests multiple times. Can you live with developing the tests for Worksoft Certify? Is there value in the execution over multiple times quickly? Can you justify this effort over the cost of your current practices? If you can, then this incremental step can be leveraged and then take you further into more automation successes and favorable outcomes with Worksoft Certify.

Our organization usually tests on beta and production releases of Worksoft Certify so we are privileged to be able to work with the tool ahead of the general market. In that, we experience more issues that would not get out to production releases. Also even after release we work closely with Worksoft Certify support to identify and resolve Worksoft Certify functionality and may receive such priority support than a normal Worksoft Certify client would experience since we have such intimate technical knowledge, experience and relationship with Worksoft and the Worksoft Certify automation tool.

Because we are an active participant in a robust enterprise commercial process to help improve the stability of Worksoft Certify (staged releases to select customers to improve quality prior to some production release), I would say we’re contributing towards a good job of keeping up the stability of the automation tool by testing it first hand in our complex situation and environments and responding back with realistic and practical feedback encountered. We accomplish our organization's mission to develop and run diverse, varied and large amounts of Worksoft Certify processes automation and also assist Worksoft mission to develop and improve the Worksoft Certify automation software.

When speaking on stability, I refer to the stability of the Worksoft Certify software tool. Execution stability of the developed test scripts (Worksoft Certify processes) is a whole separate concern. Even so, Worksoft Certify also shines here in comparison with other tools. To really address this properly and clearly, you need to automate a small test scenario in an application that is to be tested and compare the tool script execution with another tool to see that Worksoft Certify achieves a high-level of repeatability and resiliency in script execution.

Even so, we do work with an occasional problem that comes out in production releases and experienced that the interaction and speed to which the problems are addressed and resolved by Worksoft Technical Support to be extremely satisfactory. I am not sure if this is representative of all of Worksoft’s clients, but we are very happy with their speed and service with which they handle our issues. This contrasts with the unsatisfactory paid subscription support experienced with commercial HP QTP/UFT and IBM Rational tools, and the lack of any support (other than searching the internet and relying on the user and development community) for open-source tools like Selenium and JMeter.

Without the interactive and engaging support, some of these problems would be extremely difficult to solve on our own. However Worksoft solutions were technical in nature beyond what a normal client user could identify and resolve on their own. Using the available and comprehensive technical support and educational services increases the likelihood of success and a positive outcome and benefit with Worksoft Certify. Successes with Worksoft Certify will come from meaningful collaboration to address automation issues rather than handling problems through independent or isolated effort (e.g. the problem could have been addressed with another customer and the resolution could have been worked out already. Worksoft support would be in the best position to know the possibility that a solution exists). 

This is typical of automation tools and the experience of other tools in the automation tool trade. The knowledge and experience is specific to the tools. As such, going to the source who has the most experience is the recommended strategy.

We automated a big end-to-end process: Hybrid to SAP to Manhattan, then back to SAP.

We have web UI automation testing. Hybrid is a type of web UI testing. We have SaaS automation testing, along with CRM testing. We also have Manhattan, which is a third-party application.

We are now trying to automate all our applications: SAP, web, third-party, and legacy. Instead of multiple tools, we have been able to have Worksoft handle a lot of our applications. This has saved us a lot of time and effort. 

The test automation for SAP is very good. It works with the automated dial up menu, which helps us compared to the other marketing tools in SAP.

People who don't have coding knowledge are capable of doing automation with Certify. It reduces coding and scripting dependencies.

We would like it to support mobile automation going forward.

The stability is pretty good.

The technical support is pretty good. However, their first response is usually late. Once they start responding, it will be on time.

Our upgrade last year was straightforward.

We no longer have any manual efforts for our regression testing, which we run on a weekly basis. So, we are saving two resources at 80 hours a week.

I would recommend Worksoft Certify. It is a library for everything. It supports cloud applications, and the market is moving into the cloud.

The Capture 2.0 feature worked very well with the maintenance testing versus the regular development.