IBM Rational Functional Tester alternatives and competitors

We are using Qualibrate for our testing strategy, as well as for standard change requests.

We began working with Qualibrate for testing our on-premises SAP solution. After that, we started implementing the cloud application, SAP SuccessFactors, so we started using it to test that.

Regression testing is very important for this application, and we use Qualibrate for this, as well as to perform user acceptance testing.

Another activity that we use Qualibrate for is the automation of standard changes. These types of changes are pre-approved, and via Qualibrate, the configuration is done. The flow receives requests for changing an item in our ticketing system, ServiceNow, via our middleware system, which is connected to Qualibrate. In Qualibrate, existing recorded scripts will be started and then Transport in the SAP development system will be created automatically.

In the SAP landscape itself, the Transport will be moved automatically with all checks through the landscape. Of course, at the end when the testing is completed in the production environment, there will be a message sent back to the ticketing system via the middleware and the ticket will be closed.

I recently created a business case for Qualibrate and it has to do with regression testing. We have approximately 300 to 400 SAP end-users on-premises in the organization, across different business units. If you have to organize a regression test, it is because you have changed something and you want to know if it will have an impact on the processes in the other regions. In cases like this, you want to check to ensure that nothing was touched.

Normally you invite maybe 50 key users to test in the different functional domains. This will perhaps cost an average of four hours per person. Instead, you are running automated test scripts that have been pre-recorded. Even if the length of time per test remains at four hours, you are saving that much in terms of what it would cost to pay testers for the same work.

It is not possible to test everything, especially if you have interfaces with other external systems, but you do win a lot of the time. Also, it's not only for testing when your project is going live, or when you're bringing changes to an environment but also checking your daily operational support. It brings a really big benefit in terms of keeping your system stabilized. In the end, you have fewer disruptions for the business.

The low-code approach enables users to onboard quickly and then use it in their daily test creation, but I would always suggest centralizing it. The users have the scripts and you only need one or two people on a central level to keep control of it. 

People can make use of Qualibrate for training because that's also a benefit. If you have a new employee, you can show them in the system how one of the end-to-end processes looks. You can also explain the UFT in the same way.

The most valuable feature is that it's user-friendly. It is really easy to use and if you have your adapter downloaded, you can remotely explain to your key users, the testers, how to use it.

We have all of the biggest parts centralized, which means that we can run all of the scripts at the moment we bring changes into the UFT environment. They can easily be run in the background. 

You have the ability to reduce all of the recorded transactions so that you can build your own consolidated end-to-end process. If something in the end-to-end process changes then it is very simple to insert a new part into it.

We have just finished all of the scripts for SuccessFactors, which is an HR application. Every quarter, you have a new release, and we are dealing with scrambled data that needs to be reduced. Using Qualibrate makes it easy.

What I would really like to see is if you are running scripts in Qualibrate, and there is a defect, then you can have it automatically raise a defect in your own ticketing system. We do not use the defect system in Qualibrate, which is why this would be helpful. We won't want to have to use the ticketing system in Qualibrate in addition to the one we use now.

The reporting can be improved. For example, the reports can be better and show details like how many of your scripts have been executed, what the history is, which scripts have not been executed for more than a month, and other types of statistics.

It would be nice if when you add or change some specific script in your SAP system, it would automatically check to see what it implied with respect to Qualibrate. For example, it would link to the appropriate Qualibrate script if it needed to be changed. In the context of an end-to-end process, which has already been recorded, if next week I change something in the process then I would like to get an alert that we need to make the relevant change in the process.

My company has been working with Qualibrate for two years.

I worked with Qualibrate for another company approximately six years ago.

Stability-wise, so far so good. Last week, there were two times that the system was unavailable but that really does not happen very often. In case it does happen, the communication from their support desk is very fast. They inform you directly.

Scalability-wise, this solution is fine.

We don't have very many people who are working on it. There are two external, my test manager, and myself, so four in total. 

We have no issues with the technical support. When I have a problem and the system is not available, I want to know how soon I can get it back up and running. I do not run these kinds of systems every hour; even when your SAP is running 24/7, your testing does not. However, when you want to test something, it needs to be available, and support takes good care of this. When I have called they pick up the problem and they resolve it.

Approximately six years ago, I was with another company and I used several different test systems. I was involved in a pilot at the time and helped to define the requirements. One of the important ones was to keep it fair user-friendly, which Qualibrate is.

Over the period of a few years, I looked at test automation. At one point, in 2018, I compared the features in Qualibrate with what it had four years prior, and there had been new guidelines for things like data introduced during that time. A lot of improvements had been made to the product.

I have experience with HP QTP, but it is an expensive tool and I explained that I didn't need a Rolls Royce for testing. Rather, I just wanted a simple tool. I didn't want to have a very expensive tool where I only use 50% of the functionality when instead I can have one that's user-friendly.

Comparing Qualibrate with my other tool in terms of manual testing and the process of documentation, I would rate Qualibrate a nine and the other tool a five. The other tool was very technical and not user-friendly. Every time you need to change something, you have to contact technical support. It is also very complex when you want to explain it to your key users.

I have my own test manager who is responsible for the SAP on-premises solution, which includes testing it with Qualibrate. We had an external team help us with supporting the functionality of our cloud system, and we also asked them to take care of Qualibrate. Unfortunately, it took a really long time and the person responsible for it has now been replaced. Together, the new person, working with my test manager, has it under control.

If you have it under control, you just have to do the maintenance.

We have seen a return on our investment. There are a lot fewer people involved in our testing and the system is more stable. In addition, more people compliment you and less complain if the system is not available, which is a really big benefit.

We signed a three-year contract and the pricing is in line with our expectations.

If you look at the cost of your license and what is required, you will see that it is better to keep everything centralized and pay for perhaps five user licenses. This is more cost-effective than giving one to everybody in the business.

When I proposed to use Qualibrate, they were hesitant because it was not in the top four or five of Gartner. That necessitated some more evaluation which included a tool by IBM and at least one other one.

My advice is that if you know how an application is working then reduce it on a weekly or perhaps daily basis. If you are doing this then you will very quickly understand how the system is working.

The first time you record a regression test, I would advise doing it in a classroom session. In this scenario, people are doing the regression test manually and you ask them to execute the same script, but this time, start the Qualibrate recording function. People can make use of the backspace functionality and with it being recorded, it can be used the next time a regression test is needed.

I suggest that you not only look at the price but also the quality. This vendor listens to you and if you have requirements or need help, they are always available and not only by phone. If required, they will visit the customer from time to time. The application is user-friendly and there is documentation available. Overall, we are really happy with it.

I would rate this solution a nine out of ten.

I have led regression testing projects involving CRM, employee registration, and eCommerce. In each of these test cases, we have used the automation tool which has brought us benefits in time savings, shortening schedules, or recovering lost time - all of this leading to money savings. 

It's very useful when there are constant changes in the back-end. If the changes are constant in the source end it is not a good idea to use it. It is very important to previously evaluate the use of the tool depending on the characteristics.

The main improvement is the lower cost of regression tests. It will probably be more expensive in the first iteration (more or less 30%), however, we can save up to 40% or 50% in the next runs. 

Another benefit is that, once the test scripts have been built, it is not necessary for the person executing the test processes to be an expert, allowing resources and costs to be optimized with lower costs in human talent. 

In addition, the barrier of functional knowledge is eliminated; specialization is not necessary during the life cycle of the tests.

The main characteristic that is useful is that the tool is completely free.

Additionally, currently, you can find resources that handle the characteristics of the tool very well. 

The range of web platforms it supports is also wide, including the operating system platforms. It is possible to reuse the test scenarios that have been built in iterations after the first one or in the improvements of the systems that are part of the normal maintenance and updating that must be done periodically and that imply an investment in tests.

A great improvement could be in the user experience which is necessary as it does not have much. The use is too technical; it is not designed for the end-user. The best way to maintain the test scenarios over time should be considered. If the test scenarios are not subdivided correctly, it is very likely that maintenance will become very expensive and re-use is unlikely. This would ultimately mean that it would reflect the possible benefits in the projects and, on the contrary, the use of the solution ends up being more expensive.

I have been working with Selenium for five years.

The latest version should always be treated with care. Being open-source it can be unstable.

The scalability will depend on where the tool is installed.

Until now, I have not used the support service. Everything has been done using the documentation that is registered in the forums.

Neutral

In the past, I used a very good solution. However, it had a high cost and had to be used by more than one resource.

The initial setup can be very complex.

In my case, the initial setup was done internally.

The ROI can be between 20% to 50% in the testing process in the medium and long term. In the short term, it can be 10% or less.

As for the license, there is not much to say. It is free, however, the configuration must be done correctly or it can be very expensive.

Other options that I evaluated were TestComplete, Kathalon, and IBM.

It must be evaluated very well before being used. In my case, we implemented a front-end that facilitates the use. If you want to use it in the future I can give you some access.