What is our primary use case?
We have licenses for qTest Manager and Flood. We use Flood for performance testing. We use the Manager on a day-to-day basis for storing the test cases and linking them with the NPM, with the Selenium automation test cases, and we schedule runs through qTest.
We also have Jira Cloud and connectivity using the CI/CD pipeline. We connect qTest with Jira and set up our runtime and regression automation. Manual is done using just the Manager and the automation is done using Selenium and Selenide.
All the user stories are done in JIRA. We take those user stories from JIRA and input them into qTest. From there, people write the test cases related to each and every user story and these test cases reside in qTest. qTest is then connected to a Linux box and Selenium. Since we have connected the qTest automation, Selenium runs the suite. We create defects in JIRA and connect that to qTest. This is how we link the entire package.
How has it helped my organization?
The solution's reporting enables test team members to research errors from the run results.
Our executives have started to review results provided by qTest, but that process is not completely done. We are in the process of implementing it for the higher officials and showing it on their screens. Everything is in the cloud and they can just click on things and it says, "Okay, these passed and these failed."
The speed with which our team understood the tool and started implementing and using it has drastically improved things. I'm sure we will improve our use of it over the next couple of years and use the tool to the maximum.
The solution is helping increase testing efficiency. We spend less time trying to find defects and doing manual testing.
qTest is definitely doing a good job of meeting our requirements and meeting the needs of our higher officials for understanding how the tests are being run.
What is most valuable?
The most important feature which I like in qTest manager is the user-friendliness, especially the tabs. Since I'm the admin, I use the configuration field settings and allocate the use cases to the different QA people. It is not difficult, as a QA person, for me to understand what is happening behind the scenes. Looking at the code and looking at the Node.js or the NPM connection, it is so easy for anyone to understand the CI/CD pipeline.
The terminology used on the tabs, like "test plan" or "test cases" or "regression" or "requirements," make it easy for any layman to understand: "Okay, this is the model. This is how QA works. This is how the lifecycle of a product moves." There isn't any tough terminology which might make a new user say, "Okay, what is this? I don't understand."
It also provides the easiest way in which I can set up automation. It is really easy to do, compared with other ALM tools.
The integration with JIRA is superb. It was easy for my DevOps manager to go ahead and create integration between JIRA and qTest.
What needs improvement?
As an admin, I'm unable to delete users. I'm only able to make a user inactive. This is a scenario about which I've already made a suggestion to qTest. When people leave the company, I should be able to delete them from qTest. I shouldn't have to have so many users.
There are more improvements, which can be made, such as giving users an easier way to access the tool.
For how long have I used the solution?
We have been using qTest for just under a year.
We have not completely implemented everything, all the features. Although we know what qTest has, we have not explored the data and the dashboard and the tabs. So we are just using 60 percent of the tool's assets. We are still waiting for our own stable releases to happen and then we can say, "Okay, automation is done, manual is done."
What do I think about the stability of the solution?
Initially, we had a few issues, but now it's stable. It doesn't give us any problems. For the past six months at least, I haven't had to create support tickets as often as I used to in the six months before.
The tool was new for us and it was pretty difficult for us to understand certain things. But now, we know what it is and how to implement it. We know how to integrate with JIRA, with Selenium, etc. Everything has settled down.
What do I think about the scalability of the solution?
The solution is scalable.
We currently have ten users using licenses out of our total of 12 licenses, and they use it on a daily basis. It's used extensively to create the test cases, run automations, and create defects in JIRA.
Currently, we don't have any plans to increase our usage. Five staff members are required for the deployment and maintenance. They are the people who schedule the automation runs and who do all the other jobs on a daily basis.
How are customer service and support?
Technical support is pretty good. During the first six months I was creating tickets and tried to get the answers immediately through email. If it was not possible for me to understand their answer, they immediately scheduled a meeting. So at the maximum, my problem would be resolved over the course of a week. The support is really good.
Which solution did I use previously and why did I switch?
Previously, we used Micro Focus ALM. Now, we have divided our products internally: an old product where we use Micro Focus, and a new product for which we wanted a newer tool to be implemented, which is qTest.
How was the initial setup?
The initial setup was a little bit difficult. Once we started with qTest, we had to migrate all our test cases from Micro Focus ALM. That's where we had a few difficulties in implementing this.
We had the help of a migration manager from Tricentis who really helped us out. At that particular stage, I had difficulty setting this up. Once it was done I was so relieved. It did take time. We thought it would take a week's time, but it took a month to finish the entire task.
The code didn't work as it was supposed to in the wizard for the migration. It's true that our company's repository in Micro Focus ALM was very large, so it was difficult for us to take everything from there. We had to break the repository in half, and we had a lot of issues with IT here, and with Tricentis there. Everything got settled, but it was not quick.
What about the implementation team?
Our experience with the Tricentis consultant was good.
It's just that our setup took a lot of time. We had a lot of difficulty, initially, in migrating the entire project. We needed to activate the product in ALM, and then deactivate back. It was kind of a mess. But the support engineer would coordinate with me, even outside of office hours. We sat together in meetings and tried to clear things up. He was a pretty good guy who really helped us out to set this up.
Now, when it is so user-friendly and so easy to work on, that's only because he gave us the initial foundation for the product. We're really thankful for that.
What was our ROI?
It's too soon for us to see return on investment.
What's my experience with pricing, setup cost, and licensing?
We signed for a year and I believe we paid $24,000 for Flood, Manager, and qTest Insights. We paid an extra for $4,000 for the migration support.
Which other solutions did I evaluate?
Being a lead manager, I shopped around among many ALM tools and tried to understand which would really suffice the needs of our newer tool. We found qTest was the most user-friendly, and I can even say the most popular. The cost-effectiveness was also part of it. All of that helped us choose this.
Comparing Micro Focus and qTest, the cost of qTest is far less. Secondly, the cloud base and the fact that I am able to see everything on one screen is helpful. Although Micro Focus is updating as the time goes by, it's not as easy and as user-friendly as qTest.
There's reporting in both solutions. qTest Insights has more customization. Although ALM has some customization, it's not so easy to set up. You need to write a type of VBScript code to do more customization. But in Insights, it's easier for me to customize my reports.
We use both solutions, but the team that started using qTest is entirely different. The team is new, the product is new, so they didn't find any difficulty in adopting this tool. The other team, which was using Micro Focus ALM is still using it. We have not changed any team's structure because qTest is used by the newer team and Micro Focus ALM is used by the older people.
We looked at SpiraTest Inflectra and TestRail. SpiraTest is definitely competitive with qTest. We found everything that was in qTest was in SpiraTest as well. But there were flaws in terms of the terminologies used by Inflectra. It would not be easy for any QA to really understand. That was one of the differences we found. And the initial support which I needed from SpiraTest — I did have to mail them every day — was not what I wanted. I was not getting immediate answers to my questions.
As for TestRail, its integration with JIRA was not as easy as we thought it would be. That was one of the flaws in TestRail which caused us to give up on it and we moved to qTest.
What other advice do I have?
Go for it, take a shot at it. Try it out with the 30-day free trial. If you really find it to be a good fit for your company, the productivity and the cost, go ahead and choose it. It's definitely a good tool.
The biggest thing we've learned from this tool is the ease of using it. It is easier. There is a possibility of creating the entire application lifecycle management by moving around different tabs and moving around different options. With one screen it is easy for a QA person to get into it.
We have not used Insights that much. We have used it to some extent but we haven't gone into the details in the graphics and the reporting. Because our own product is changing so often — the versions and the management and the configuration of the product are changing — we do not have a stable release for our product. So we are not set up completely with Insights. We are in the process of doing so.
About 40 percent of what we do is still manual testing; only 60 percent is automated. The basic aim is for at least 80 percent automation.
Our team which is working on qTest Manager is located in Ukraine, so a team leader there could provide more elaborate answers than me. I'm leading it from our head office.
The team in Ukraine are the people who are using it on a day-to-day basis.
I would rate qTest at seven out of ten. To make it a ten there are a few things here and there which could be easier for the user, like giving popups in between operations. When I want to delete something it asks me, "Are you sure you want to delete?" But it does not do that everywhere. So there are some small things, here and there, which could really improve the tool. It is supported in Chrome, Firefox, Safari and IE11. I would like to see more browser compatibility options, like using it in Edge. And when I move to different browsers, the format of the tool is not consistent.
Which deployment model are you using for this solution?
Public Cloud
If public cloud, private cloud, or hybrid cloud, which cloud provider do you use?
Amazon Web Services (AWS)
*Disclosure: PeerSpot contacted the reviewer to collect the review and to validate authenticity. The reviewer was referred by the vendor, but the review is not subject to editing or approval by the vendor.
