Qualibrate alternatives and competitors

With Qualibrate, the accent for us is on SAP, because that's our biggest application and a critical application for our company. Web was also an opportunity.

If you do a SAP regression test and you have to do it via manual testing, you need to organize a lot of people. The difference with automation testing is I can do it every day. What we have with SAP are the HR Support Packages, the service packages. We do them every half year, but if I want to check it, I can do it every day. That is a great advantage when using Qualibrate for automated testing.

Qualibrate has enabled our business users to create automated test scripts without requiring coding knowledge. That was a main part of the decision for the business, that it is easy to use. You don't have to be educated and have experience in Java or HTML. They have to know what kinds of flows they want. I have explained that they need to make their flows in building blocks. We are now planning to bring the Qualibrate solution to the business users.

Qualibrate enables us to do more frequent testing, without putting too much pressure on timelines and resources. I'm doing it every day and I'm showing management the differences between that and once a month or once every half a year. We now have continuous testing and that's what we also want to do with all the applications in the cloud. We need to be able to test development or when a change is planned for an application. We need automated testing in the continuous delivery pipeline.

The more frequent testing has affected the number of production incidents and software quality a lot. If you do testing every day, after a weekend of development changes in some formulas we use for the desktop application, running the scripts automatically means they see the changes in what they have deployed. That is a big advantage. I can immediately see, in the morning, what they have done over the weekend in deploying applications. I report things to the project leaders and the people responsible for functional maintenance of that application. They are surprised: "How do you know that so fast?" It's because by running Qualibrate we get the answer.

With Qualibrate, when we looked at the time and resources we are saving it was €1.5 million per year, mostly because of the move away from manual testing. Maybe that's a small amount if you compare it with my last employer, when I worked for ING Bank, but we are a small company. We only have 1,800 employees. Using Qualibrate we are saving a lot of time for the regular users because people on the business side are not testers. They have to take care of daily business as a priority. I am able to take work from them and that makes it easier for them to do their regular work.

Also, where it might have taken a day to create a test document, now, if you do work with a flow, it takes you four or five minutes and you have your automated test flow and your documentation for your process. That makes Qualibrate a fine application to work with.

What Qualibrate makes very easy to do is to record a process flow. Within five minutes you have a clear document produced by Qualibrate. Instead of using Word, and copying and pasting pictures into it from printscreens, within five minutes what you have was easy to make and it's easy for users to use.

We had an issue with SAP when using PDF forms. That was something that was not supported by Qualibrate, but we solved that issue by choosing another solution. That was the only wish we had with Qualibrate. All the other functionality provided is all the functionality we need.

We have been using Qualibrate for three years.

It's very stable. I'm working in Qualibrate almost every day and sometimes on weekends. I haven't experienced any issues. It's running smoothly.

Our use of the solution is still growing. We started using it on sub-solutions but nowadays we are getting a lot of requests from the business to do dev projects or help them with their applications. Instead of their manual testing they would like automated testing. It's easy for me, as an admin user, to create a new project and guide them with my knowledge in creating flows and scenarios. The number of projects and business partners requesting use of it is growing in our company.

We now have 301 flows using Qualibrate and in a year we did about a half million test steps.

We have five licenses. I use it as a test engineer. On the business side there are functional maintainers, and some of the business users, or the people around them, are guiding the process flow by first creating the test steps we have to record. So we now have 10 people directly involved with Qualibrate.

We have weekly contact with Qualibrate. If there are some wishes on our side, we can mention them there and they will be put into review for deployment. Our last request was that when you record a flow, that it would automatically create screenshots. Screenshots are much better than they were before for the documentation and for use as a user manual. That has improved now.

I'm very happy with the service level they give us, especially from Alan, Marina, and Martina. They are always available for me if I'm calling, or by email or by Teams. Their technical support is a nine out of 10. 

We previously used a manual testing system that I had adopted from ING Bank. Phase one was to collect the information and phase two was to create your manual testing.

We started using Qualibrate in 2017 with a pilot version because our organizations had a big need for automating our process flows. We would usually do it manually and part of it is still manual. But our organizations had a question: "Can we go for automated testing?"

I was the person who introduced Qualibrate as a choice in our company. The parent company is IOVIO, and I worked with IOVIO when I was at ING Bank.

The initial setup was very easy. It is a SaaS application. It was easy to install and easy to use. There was no problem installing it in our environment. Within an hour it was operational.

The only thing we needed to do to use Qualibrate was more due to our technical maintenance requirements. We weren't allowed to have local admin rights but because Qualibrate is using an updater. The updater has to be clear of admin rights because otherwise you can't install new features. That made for a little discussion, but happily that's over now. But there were no problems installing or setting up the whole thing. However, I have to mention that I'm experienced in test engineering, so that also enabled us to do it very quickly and easily.

We deployed it with help from Qualibrate consultants because it was our first time using Qualibrate. They were onsite and helped us to install it on each of the laptops, easily. We had three licenses to start.

The  consultants were great. Qualibrate is a young and dynamic team, and they were willing and able to help us every time I called.

Automated testing is not cheap. But other companies, for example, Panaya, required a minimum of 10 licenses. Qualibrate allowed us to start small, with three licenses, with a price that was competitive within the market.

We invited a lot of companies to do a presentation. In the end, our stakeholders were convinced that Qualibrate was the best solution for us. We looked at Tosca, Solution Manager from SAP, Panaya, and Testbirds. Because we are government organizations, and we need to be very transparent in choosing a company to provide us with automated testing, we had to invite several companies.

In the past I was a test engineer with Micro Focus LoadRunner and Solution Manager. Qualibrate is a very simple solution. You don't have to be a person who has skills in coding or creating or something like that. It makes for a very easy tool to use in Agile sprints. The difference with LoadRunner was that you had to be a coding specialist, otherwise you could not run the application. That makes a great difference. People without those skills can use Qualibrate as a tool for flow documentations, and it's easy to maintain while providing test automation as well.

While you don't need to have coding skills, if you have the skills it makes some things easier to use. There are some options within the data, in the advanced options, that you can use. There are building blocks created by Qualibrate and some code that you can use. For example, for a date field you use code they have already made for you. You can specify today plus seven days so that your script is always running smoothly.

I would advise using Qualibrate as a testing tool in a DevOps organization. It's a fine product. Other solutions are also okay. Tosca is also a great solution, but in the end, Qualibrate makes us better.

We don't use Qualibrate's Test Planning and test Execution Scheduling features yet, but we plan to do so this year. We are now growing from three to five licenses and management wants to know what the options are for test management and test planning so that we can schedule things for when the business wants to run the scripts. It can be every day or once a week. That's the way we plan to use Test Planning.

We don't use defects at the moment because we are now organizing API management to get JIRA onboard. Our process is still growing in production.

We are totally satisfied with what we now have in our possession with Qualibrate. We are very happy with the solution as it stands now.

We're using it for testing, mostly regression testing.

Before Qualibrate, we did a release in about three months, and the regression testing would take us about three weeks with 15 people. Now, we do it in one night. The time savings are really important because we can now release things more quickly.

When it comes to problems that require changes, we can now do them in one week. Before, we would save all such changes and do them together over three months. The business is very pleased with the way we work now.

Qualibrate's low-code approach also enables users to get on board quickly and use it. We work with SAP and we know everything about SAP, but when we use Qualibrate, we can easily explain to our new people also how it works. Even business users can create automated test scripts without coding knowledge. Generally, getting them to do so would require about a month of training but now it is done in a day.

We're also able to do more frequent testing, without putting too much pressure on timelines and resources. In previous times, we didn't test security notes because there were too many. Now, every time we have a security note in the system, we do a test, and that's great. That has helped make our risk very low.

The most valuable feature is the way it works, the usability. From the first setup, the user interface is the most beautiful. It's very nice the way you can recover things and make it work.

We use the solution’s Test Planning & test Execution Scheduling features, and they are very important. They are easy to work with. We use SAP Solution Manager, and Qualibrate works with it, enabling us to manage all our tests, taking them from Solution Manager directly into Qualibrate. Because everything we do is in SAP Solution Manager, we are now able to do test automatization, combined with Qualibrate. All the defects are available in SAP Solution Manager and all the changes will be in Qualibrate, immediately.

From Qualibrate, we see whether tests are good or not. We can create a test-related incident in Solution Manager for those that are not good, and Test Suite in Solution Manager will notify the person who has to rebuild it. That works very nicely.

Not everything in SAP works well with Qualibrate. There is a development tool called xpath and you have to program it. We always thought it wouldn't be necessary to program it with Qualibrate, that everything could be solved by Qualibrate without programming, but you have to program some things. Using xpath is more complicated, and not easy for everybody. It would be helpful if there were a no-code solution for this.

We have been using Qualibrate for more than three years now. Our first version was version 2.0, and now we are moving to version 3.0.

It's very stable. We never have problems with it.

We haven't had any problems with scalability. Currently, we have more than 100 scripts and some are big scripts with 400 to 500 steps.

We have plans to increase our usage of Qualibrate because we are going to S/4HANA, and that's a big step for us. 

Qualibrate's technical support is great. They always work together with us. There are people from India who help us and there is no problem with support when there is a problem we need to resolve.

Positive

We did not have a previous solution. We did everything manually.

The initial setup of Qualibrate was not complex at all.

The group we bought it from, Fiori, was very willing to work with us. It was very good because there was very little waiting. I would call the manager and he would call me back and we would immediately have someone on the line who knows the business. That was really great. And they come onsite, making it easy, and explain the solution to you.

Our deployment took about two weeks.

We had 32 big scripts already and we wanted to put them in Qualibrate. The consultant gave us lessons and, in about two weeks, we were ready. It was easy. We had 15 people on the floor, and two of them were from the consultant, and they helped us with the scripts so that everybody understood everything. That was great.

We have seen ROI in our ability to release things a lot faster.

I compared the prices of the 15 solutions we looked at. Qualibrate was the most valuable because it could be integrated with SAP Solution Manager. Tosca had a big price, but it does everything, including defects. But because we already had defects taken care of with SAP Solution Manager, we just needed one module to work with and that was Qualibrate. It was easy to pick it and not so expensive.

We looked at Tosca and about 15 other solutions. Because we have a lot of things in SAP Solution Manager we tried an SAP solution, but that didn't work for us, so we looked further. Qualibrate was the best for Solution Manager. The big pro was the user interface. That was really the most important thing I saw.

If we didn't have SAP Solution Manager, I probably would have bought Tosca, but our other tools already covered a lot of its functionality.

When you are considering buying something, you have to see the user interface. If you buy something that is complex, people won't work with it. Qualibrate is so easy to work with.

I would probably rate the solution higher than eight out of 10 if we weren't using the on-premises version. With the cloud version, Qualibrate can look at things with you, but with the on-premises version they can't. That makes some things a little bit difficult, but because we work with defense material, it's not possible. The on-premises version is more difficult than the cloud version.