Tricentis qTest Reviews

We use Tricentis qTest for capturing all the user stories, data, requirements, and dashboards. We also have integrations with Tosca to capture all the execution runs. It is a test management tool, and we can generate everything from dashboards out of Tricentis qTest.

The return on investment is quicker with low-code, no-code solutions. In my client landscape, more than 80% of clients have moved to Tosca already, and they have been using it for at least two to three years minimum. Because of such faster adoption, I am facing challenges in getting people with Tosca skills because every client wants Tosca skill sets. While this is good, it is also challenging because every vendor will seek Tosca profiles. Since the adoption is faster, the ROI is quicker, and with Tosca AI, the maintenance is better. The self-healing aspects and maintenance of scripts are much faster and quicker. We are able to find better avenues and better productivity in terms of maintenance, which we can pass on to the customer.

Most clients do not use these products in standalone mode as they always want to have integrations done. From the customer standpoint, they buy the products but do not know how to integrate them. As a service vendor and given our strategic partnership with Tricentis, we bring in our capabilities and leverage our knowledge of these products. We have built many accelerators on top of it. For example, if migration has to happen from ALM to Tricentis qTest, we have accelerators and specific tools which can aid that migration in terms of moving user stories, test artifacts, attachments, and everything. We have many other enabler solutions built around Tosca that help us in faster migration and ensuring customers are up to speed.

The tracking capabilities and dashboard features are particularly valuable. We have single-view dashboards where we can bring in data across multiple projects. Due to the integrations, we are capable of pulling all statuses and current execution status in real-time. These tracking mechanisms at both project and program levels are really useful for capturing relevant areas and highlighting risks immediately as part of the program test management.

There are also AI insights available based on the data and defects raised, which gives a view of functionalities where there are challenges and where more regression needs to happen. I am currently working to mature customers slowly, moving from QA to QE, and now towards QI (intelligent quality).

Customers are moving towards Tricentis due to their association with SAP. There is interest in understanding if there are connectors for converting UFT scripts to Tosca, as many customers are looking to make this transition.

We have developed capabilities for automated migration from ALM to Tricentis qTest without any loss of data. However, for UFT to Tosca migration, scripts need to be rewritten as there are no automatic converters available.

From a project perspective, there have not been many challenges with Tricentis products. The main improvement area would be developing a connector to move UFT scripts to Tosca, which would enable quicker and easier movement for customers. This would aid faster adoption of Tosca and ease the financial pressure on clients who currently need to invest in rewriting scripts.

Tricentis qTest is a test management tool. When clients purchase it, they buy the complete suite including Tosca for test automation. We have strong capabilities in integrating Tricentis qTest with Tosca for automation, plus NeoLoad and LiveCompare. This is particularly important as most projects and programs across the globe are now transformation programs in SAP, moving from ECC platforms to SAP S/4HANA and Fiori applications.

The deployment process is straightforward. All Tricentis product integrations can be completed in less than two to three weeks, including server setup and configurations. It has not presented any practical challenges.

The product is stable and the project teams have not reported any specific issues.

The product can be easily scaled without any problems.

Support is absolutely fine. We have a dedicated SPOC because we deal with Tricentis directly. Support has never been an issue, except for consulting which Tricentis charges for. Any issue resolution is fairly quick.

Positive

Return on investment is quicker. The adoption is quicker, and the ROI also quickens with a low-code, no-code sort of solutions. Many people are using Tosca, and more than 80% of my clients have moved to Tosca. They have been using it for at least two to three years. Due to the fast adoption, there is a quick ROI along with faster and better maintenance due to Tosca's AI and self-healing aspects. This quicker adoption and ROI also come with challenges, including finding Tosca skill sets in the market.

The solution is expensive. For the features that are available, depending on the volumes of licenses we get, we are able to get better discounts as strategic partners of Tosca. We can pass some benefits to the customer. The pricing is based on the available features, and while it is slightly on the higher side, we can often secure discounts for customers through our strategic partnership.

As a portfolio practice head, I understand the general and most-used features by customers. I have a competency specifically which caters to all requirements, and I typically consult my competency team for any specific client requirements. The reviewer rated this solution 9 out of 10.

The main use cases for Tricentis qTest are for test management, to keep test cases and execute those. Overall, centralized test management with Tricentis qTest was beneficial for the team's productivity. As far as test case management is concerned, it was a really nice tool where users can perform copy-paste operations with all the folders. In that sense, it was really good, but when it comes to reporting, that was the pain area.

Tricentis qTest brought benefits by providing all test cases in one central repository with easy access. The whole team can access those, and it also maintains the history, so users can go back and check what has changed over time.

One good feature of Tricentis qTest was that users can directly upload test cases in the tool itself. Unlike ALM, where we had a plugin in Excel and it used to hang very much, this was a really cool feature where users can upload test cases directly into the tool.

I have used Tricentis qTest's integration with Jira, but that's why we migrated to Octane tool. There were some hiccups, such as when we updated anything in Tricentis qTest, it wasn't really reflecting in our release container or release manager.

I have utilized the customizable dashboards with Tricentis qTest, though they weren't very interactive.

I have seen measurable benefits with Tricentis qTest in terms of time-saving and resource-saving. It's definitely time-saving, and also in terms of space, users can have everything centralized instead of having it all in Excel locally, which can be maintained in the repository of Tricentis qTest.

One of the things I noticed was the reporting part wasn't very good.

It was hard to customize the dashboards in Tricentis qTest, and there were limitations in the dashboard.

There may be missing features that should be included in the next release of Tricentis qTest, such as integration with other tools, specifically integration with Parasoft. I'm certain these features might exist, but they may not be available in the public domain or widely known. Integrating with the automation tool would really help the teams.

I have been working with Tricentis qTest for two and a half years.

As for stability and performance, apart from a few hiccups and glitches, Tricentis qTest worked fine. For example, in a quarter, for one or two days, we faced issues accessing Tricentis qTest, but other than that, it was stable.

When it comes to scalability, Tricentis qTest is indeed scalable.

I do not have much information about the technical support from Tricentis, as I haven't escalated any questions to them.

Neutral

We migrated from Tricentis qTest to another product because our tools and support team evaluates and migrates from time to time.

My experience with the initial setup of Tricentis qTest and the deployment part was really good; we didn't find any issues.

The deployment model for Tricentis qTest was deployed on a centralized server and on our in-house servers only.

I am currently working with Tricentis qTest, but we just migrated from it to another tool called Octane about four or five months ago.

I prefer the current solution over Tricentis qTest. One main difference is that in Tricentis qTest, users have two different tabs, one for test case and one for execution, but in Octane, there is just one tab. In this regard, Tricentis qTest was better, but from a reporting perspective, Octane is doing much better.

My advice for users who would want to start using Tricentis qTest is that test case upload is very easy, and users can also do copy-paste very easily. If users need to clone or modify some test cases, it can be done with very minimal effort.

I rate Tricentis qTest seven out of ten.

The primary use case involves efficiently managing integration testing in conjunction with Jira and scheduling test cases through TOSSCA.

The primary advantage for us was the enhanced visibility into people's utilization. While we also utilized expert testing features on the platform, we encountered some early issues with the recordings. However, overall, the tool seems highly promising. On the manual testing front, qTest allowed us to conduct exploratory testing, enabling us to record and capture events during our expert testing sessions.

I found the reporting aspect to be the most valuable as it provided a comprehensive overview of the efforts needed and the workload for individual tests.

The user interface has a somewhat outdated design, which is certainly an area that could be improved. Some of the modules appear to be loosely connected, but despite these aspects, our overall experience with the tool was positive.
When you begin integrating your testing tools with qTest, the available examples may not be very clear, and I believe this is an area that could be enhanced, particularly in terms of providing clearer integration guidance. While the tool's integration with various testing tools is impressive, there is room for improvement in showcasing more cases and benefits, especially through additional videos and documentation.

I have been working with it for six months.

While stability is present in the expected testing aspect, it's not as pronounced in the main product.

There were no problems with scalability, but the installation process wasn't extensive. Currently, around ten individuals use it.

I previously worked with Micro Focus Quality Center, which may now be under OpenText or another entity. The reporting functionality and the seamless integration with various solutions out of the box are noteworthy aspects. Quality Center is a comprehensive ALM tool, that enables requirements management within the platform. In contrast, in qTest within Cisco, there's a reliance on importing features from external sources, especially in agile projects where qTest appears to be superior. The key difference lies in reporting preferences, with a personal inclination towards liking the key test reports better, although in large organizations, these reports might necessitate some filtering.

The initial setup was straightforward.

When compared to other test management tools, it is advisable to thoroughly explore all the features of qTest. Many users might mistakenly focus solely on specific modules or features, but it would be wise to integrate them into the CI/CD project and test out the generative features. Additionally, the manual regression testing capabilities, including parameterization, are notably impressive. To truly understand and appreciate all the benefits, it's essential to invest time in exploring the product comprehensively. Overall, I would rate it nine out of ten.

My company uses Tricentis qTest for test management, so I use it for documenting tests and test executions. I also use the tool for defect management.

What I found most valuable in Tricentis qTest is that it doesn't require installation. You use it through the URL.

It also has an excellent reporting feature.

Tricentis qTest has good integration with different tools, so you can easily integrate it with Azure DevOps, for example.

Another valuable feature of the tool is that it's user-friendly. Tricentis qTest is easy to use, and my company finds it useful.

The support for Tricentis qTest has room for improvement. The response could be better.

There's a feature I want to document on the Tricentis Idea Portal for Tricentis qTest, which I hope to see in the next version of the tool. It's a feature available in Micro Focus where you execute a test, and then on a spec level, you mark it as pass or fail. Then at the overall level, Micro Focus will automatically mark the test as a pass if all steps passed or failed, even if one step failed. However, here in Tricentis qTest, you still need to mark the overall level of the test cases. It's not automated, unlike what you have in Micro Focus. If Tricentis adds that feature in Tricentis qTest, it will make life easier for testers.

My experience with Tricentis qTest is almost one and a half years.

Tricentis qTest is stable, so stability-wise, my rating for it is a nine out of ten.

Tricentis qTest is a scalable tool, so scalability-wise, I'd give it an eight out of ten.

Whenever my team reaches out to the Tricentis qTest support team, the response time isn't as fast. It's the area to improve on in terms of support.

My rating for the Tricentis qTest technical support is six out of ten.

Neutral

We used Micro Focus ALM but switched to Tricentis qTest because upgrading Micro Focus ALM requires reinstalling and setting up all the users again, which requires admin rights. In our company, providing admin rights was challenging because of the number of users.

Performance-wise, we also found Micro Focus ALM lacking because it's slow, and users complained a lot about waiting for some time before they get to the next page. We also didn't find Micro Focus ALM user-friendly.

Those were the critical factors that made us switch to Tricentis qTest.

The initial setup for Tricentis qTest is an eight out of ten. Deployment and access could be completed over one day after the contract is signed.

Deploying Tricentis qTest wasn't a big deal to my company in terms of complexity. Still, my company has many users, and the company previously used Micro Focus ALM, which means there was a need to migrate from the existing project and train users in Tricentis qTest. My company formed a team of four responsible for handling the migration and user training.

Tricentis qTest gave my company twenty percent more savings in the first year. The tool is good money-wise and resource-wise. It's also seamless in terms of upgrading, immediately giving my company all features.

For me, pricing for Tricentis qTest is moderate, so that's a five out of ten. It's more affordable than my company's previous solution, which was Micro Focus ALM.

My company uses Tricentis qTest.

Tricentis qTest is on the cloud, so it's always up-to-date, version-wise.

Between one thousand to one thousand five hundred people use Tricentis qTest within my company.

I'd suggest that others planning to use this tool have a deeper look into it. For example, you should contact other licensees and then get a trial or evaluation license to use it first and evaluate whether Tricentis qTest fits your purpose. If it does, do a short POC on one or two projects. Once that's successful, that's the time you can go for a large implementation.

My rating for Tricentis qTest overall is a nine out of ten.

My company is a Tricentis customer with a gold membership.

On a day-to-day basis, my company works on an in-sprint Agile model. In our company, whatever testing needs to be done in sprints is instead done on Tricentis qTest, and then those are linked to a particular story in Jira, meaning integration with Jira also exists.

Tricentis qTest helps the employees in our company at the workplace since it provides the functionalities of a test case management tool. Initially, in my company, instead of Tricentis qTest, we used to write in Excel. When on Tricentis qTest, my company has seen that it is much more visible to various stakeholders while also observing that the executions we carry out become much more manageable. Whenever in our company, we execute test cases and mark them as pass or fail straight away in Tricentis qTest, which then provides us with an overall report indicating things like the pass percentage, etc. Regarding one-to-one mapping, when in our company, we automate test cases, Tricentis qTest is a very effective tool since we don't need to go to Excel sheets to look for and make markings. One-to-one mapping with an automated test suite is very effective in Tricentis qTest.

Our company still hasn't found a way to mark in Tricentis qTest using APIs when our automation strips run, which is a process that, if introduced in the future, can probably improve the tool. The aforementioned process happens in other tools like Azure DevOps, Zephyr, or Xray but not in Tricentis qTest.

In the future, an automated test suite, triggering functionality in the test suite, and execution of automated tests, which could be marked in Tricentis qTest, would be a great addition to the tool.

Tricentis qTest's technical support team needs to improve its ability to respond to queries from users.

I have been using Tricentis qTest for six to eight months. I am a user of Tricentis qTest.

Stability-wise, I rate the solution a seven out of ten.

Scalability-wise, I rate the solution a six or seven out of ten.

More than 15 people in my company use the tool.

My company plans to increase the usage of the tool in the future. My company is in the process of expanding it to other LOBs as well.

There weren't frequent responses from Tricentis qTest's technical support team whenever our company needed some help. When our company was struggling with the setup phase of the solution, during which we wanted to integrate qTest with Jira, we tried to reach out to Tricentis qTest's technical support team, but we didn't receive any substantial response from their end. I rate the technical support a six out of ten.

Neutral

I rate the initial setup phase of the product a five on a scale from one to ten, where one is difficult, and ten is easy, since in our company, we did say some hiccups in the beginning.

Considering that my company wanted Tricentis qTest to be integrated with Jira, the initial setup phase took around a week or two to complete.

The solution is deployed on the cloud.

During the initial setup phase, some users need to be created, and then there are different access levels to be done once you buy the product. There is a two-way linkage in which, in Jira, there is a qTest extension that you need to install and can be done very easily. But the two-way linkage for qTest to Jira was something which, in our company, we had to figure out in the setting, a process that only an admin can handle and no other users can do. The aforementioned steps need to be followed to be able to start using Tricentis qTest.

One person having full admin access should be enough for deploying the tool.

Based on whatever I heard, I can say that Tricentis qTest is a little costlier than other test management tools, like Jira, Zephyr, or Xray.

I would recommend qTest to others since its UI and UX are pretty easy to understand without much of a problem. With Tricentis qTest, Folder structures are easy to do, but the only area of concern comes into the picture when you try for its integration with your other project management tools. Integration is an area you need to look at to ensure that it allows for easy integrations. When trying to integrate Tricentis qTest with Jira, my company faced a lot of problems. If I compare it with other test management tools like Zephyr, which is built into Jira, you just need to enable it, and it is linked. With other project management tools, like Jira or maybe Azure DevOps, you will have to figure out how the integration between the aforementioned tools and Tricentis qTest can be made possible. Linkage with other project management tools, other automated tools, or other automation suites you have developed is an area you need to take care of when it comes to Tricentis qTest.

I rate the overall tool an eight out of ten.

The solution works very well for test management and it also automates a lot of the testing functions so that you don't have to manage them in Excel spreadsheets. It doesn't go all the way to automated testing, but it becomes a good testing repository. If an organization is not fully ready to take advantage of automated testing, qTest is a good first step. The value with qTest is that it has nice hooks with the GR. When it comes to test management, it has good integration. 

I'd like to see better integration in the platform so that there is a testing automation continuum, where customers can easily mature through qTest and Tosca functionalities.

It took a little bit of back and forth to get started but since then, it's working very well for us. We had to work with our support team and it took a little longer than expected. 

I rate this solution eight out of 10. 

When I first started here, my goal was to get a test case management tool. The testers were using spreadsheets, so the idea was to set up a strategy and plan to not only create a testing process, but to provide a way to improve that testing process. One of my suggestions was that we get a tool that allows us to be ready for the future, as we get to automation. Now after all the manual testing we have moved to Cloud Application testing for 3rd party Cloud solutions and also complete testing for our Electronic Medical Records system and integrations. It has allowed use to expand the Quality Assurance of all software used at Parkview Health .

We can actually track without having to have spreadsheets, which really improved our process by, probably, 180 percent. That was a biggie. We were able to put test cases into one area so that everybody can see them, for every module and every application that we use.

The solution's reporting does a good job of enabling test team members to research errors from the run results. You can query down, make it smaller. Since we are not a Dev shop, our releases are projects that are opening offices, or changing a piece of functionality, or if we get a release from our vendor. We're able to see a report of the execution, and that too is getting better. It has improved our productivity but has not totally gotten rid of some of the manual work that we do.

qTest has also helped us to quickly solve issues when they occur. I've seen demos of the automation and the like for Tosca. That's going to be interesting and an eye-opener for my company, given that "medical" is very slow at adopting new technology. 

We have also seen a decrease in critical defects in releases. When we started off we probably had close to a 60 percent defect rate, and the last time I did my metrics, a couple of months ago, it had a gone down to 36 percent. It's because everything is right there. It's visible. You have to be accountable for what you do and what you mark up. In our company, it's been a huge culture change that they actually have to keep track of what's not working, in one location.

Overall, the solution has increased testing efficiency by a good 95 percent.

The most valuable features are the execution side of it and the tracking of defects. I like the way it structures a project. We have a PMO that does project management. That project management then triggers a process that tells us that it's going to be tested at a certain date. We're able to put the test cases into qTest or modify something that's already there, so it's a reusable-type pf environment. It is very important that we can do that and change our test data as needed, especially for our EHR system. I also like the flexibility in how it can used for DevOps or non-DevOps operations of Quality Assurance and Testing.

We have used the Insights reporting engine but, within the last six months or so, since Tricentis took it over, they've started to improve that. We had some custom fields to match our process dates, and to track who is the project manager of the release, and who the test coordinator is. That way, we can keep track of what kind of testing is being done for that particular project. The Insights engine would not show us any of the custom fields when we first started using it. I've been working with them to improve that factor for Insights.

The next phase is that by the end of the year, they're supposed to release a new analytical tool within Insights or change Insights to be that analytics tool. I'm looking forward to that because I do all my analytics with exports from qTest and exports from our ITSM/ITIL system, Cherwell. I then make my reports out of them, so it will be very welcome to have that functionality.

I do some reporting for executives and business users from qTest. I go to Insights, do a query on the fields I want them to see, and then export that into Excel. I get the graphs, and then do a screen print, put it into a report, and send it off in a PowerPoint presentation. The quality of that data needs help. I use it fairly regularly for defect reporting because it does show an excellent view of the defects that are associated with the project and whether they're open or closed—looking forward to the new Analysis tool that is coming to Cloud customers soon.

Reporting shouldn't be so difficult. I shouldn't have to write so many queries to get the data I'm looking for, for a set of metrics about how many releases we had. I still have to break those spreadsheets out of there to get the data I need.

Also, qTest doesn't have any workflow engine. The only one they have a workflow engine for is the defects. I'd like to see more of something of that nature. It might help improve efficiency as we move into the future, especially when automation comes in.

We have been using qTest for almost five years. We are able to track all types of testing Application, Unit, Functional, Integrated, and or the combination of types of releases.

We just had an outage on Monday. They had a data center outage. Outages happen a couple of times a year, so it's not bad. It's up about 97 percent of the time.

I don't think I have any scalability issues with it. Right now, we have concurrent licenses, which seem to be plenty. We've not had a problem with that. It has only happened once or twice where there were 35 licenses used at the same time. The tool tells you, and then you have to wait until somebody signs off. That's easy to manage. We don't have any issues yet. I'm not saying that we won't, once automation is there.

We have 200 users and a total of 35 concurrent licenses. Generally, the users are analysts or Epic analysts, as well as managers, directors, and people involved in network validation. We have a tester lab, project administration, project manager, quality assurance, and quality assurance leads, as well as some people who have report-read-only access. Some of our vendors also have access. They have their user profiles because I limit their access in terms of what they can see and what they can do.

There are two of us involved in deployment, upgrading, and maintenance of qTest. I'm the lead, and I have a test coordinator who helps me.

We plan on increasing usage as we add more systems to it, and once we add automation. I will analyze how many licenses we have versus what we will be running at that time and will determine if we need any more. Make sure you set it up the way you do your business. The process is essential, not just the tool that you're using to manage it. The biggest lesson I have learned from using qTest is that I should have used it years ago. We should have had this a long time ago, not just five years ago.

I send out periodical reports of all the metrics that we do, usually twice a year. We use other tools for keeping track of tasks that have to be done on each one of the projects. We use Microsoft Planner. It makes it easier for people to actually do their assignments and then let us know that the tasks are completed. If we had the JIRA tool or something of that nature, that would help the process. But, at this time, we don't use that functionality.

We've used Tricentis technical support quite often. The way I understand it, they have tier-one, tier-two, and tier-three. Their development people are probably their third level.

They answer quickly, but sometimes they ask questions that I cannot answer because it's part of their tool. Last week I finally told them, "Just go out and look at our system. Follow my instructions and just go out and look. You have it. It's your cloud."

I have used Micro Focus LoadRunner at a couple of locations. The Micro Focus tool is very complex and not as user-friendly as qTest is. I knew that implementing HP, plus the price — it is much more expensive than qTest — would be more difficult.

So the factors were both price and usability of the tool. Because some of the people who do our testing are not IS people. They don't understand the software development lifecycle. You have to make it simple for them to use, and I can do that within qTest.

The initial setup was easy to use, but you have to make sure that you follow the process that is associated with how you manage your testing overall.

Our deployment took about six months. That was because of the data that we had to load. The strategy was to make sure it was working for all of Epic, which is our EHR system. We wanted to get that part done first. We then started making the third-party applications and got all that data into it. We waited about a year to make sure that the third-party applications had their regression test cases in the system, and we still add new application data in, as we go forward, separating the implementation of our EHR. In terms of the adoption of the solution within our company, it's much more user-friendly. It allows everything to be in a central location. The data management becomes more critical because you have everything right there at your fingertips, versus a spreadsheet which could be located anywhere.

I did it myself.

We have absolutely seen return on our investment.

For the 35 concurrent licenses, we pay something like $35,000 a year. There are no additional costs to the standard licensing fees, until we get into Tosca. 

We have the Elite version, which allows us to have Insight, Parameters, Explorer Sessions, Pulse, Launch, and Flood.

We evaluated five tools, narrowed it down to three, and qTest was ranked as number one. SmartBear, the HP tool, and Tricentis were the top-three. Back then, it was QASymphony. It was before Tricentis bought them out.

The solution had to manage test plans, requirements, and test design. We wanted to make sure we could revise test cases as we moved forward with releases. Because we're not centralized as a testing organization — we have other groups that do our testing — it had to be able to get them involved cohesively. It had to track defects. Also, we do not have a project management tool, nor do we do DevOps projects. But we are continually doing different releases of all types of medical software. So we wanted to be able to manage our releases for all of our software. Ninety percent of our software is medical, but we also have things such as supplier management. We wanted to be able to do it all, all of our test cases, in one location. We wanted it to be easy to share. We also wanted it to have a good road-map toward the future. It needed to be integrative and have the ability for single-sign-on.

Make sure you set it up the way you do your business. The process is essential, not just the tool that you're using to manage it. The biggest lesson I have learned from using qTest is that I should have used it years ago. We should have had this a long time ago, not just five years ago.

I send out periodical reports of all the metrics that we do, usually twice a year. We use other tools for keeping track of tasks that have to be done on each one of the projects. We use Microsoft Planner. It makes it easier for people to actually do their assignments and then let us know that the tasks are completed. If we had the JIRA tool or something of that nature, that would help the process. But, at this time, we don't use that functionality.

It's our primary tool for managing for testing across the Guardian enterprise.

It's helped us in having a web interface and being intuitive for how testing is done. If you're a tester, it makes a lot of sense. Instead of an application that we try to modify to make use of in QA, this is very much a QA-centric application. How to use this and what it's referring to are pretty seamless if you're a QA engineer. To that end, it has really increased the productivity of my team. In an agile world, being able to create suites of test cases, and copy them from one project to another project, is really important.

The on-demand reporting has also helped. To be able to just look at defect counts, and how much progress was made for the day, or where we stand overall with the project, is really important. All of that has really simplified things for us quite a bit.

On a weekly basis, for reporting it has definitely saved at least 50 percent of our time, if not more.

In terms of it helping to resolve issues when they occur, being able to log into Defects, go right into JIRA, add that defect to the user story, right there at that point, means we connect all of that. That is functionality we haven't had in the past. As a communication hub, it works really well. It's pretty much a closed loop; it's all contained right there. There's no delay. You're getting from the defect to the system to JIRA to the developer.

It very much provides our team with clear demarcations for which steps live in JIRA and which steps live in qTest. That takes away the discussions like, "What are we going to use? How are we going to communicate? Where will the data be?" Any of that preparation time is behind us. It's now the default for how the teams function and what they do. That's a really powerful process. The fact that it's a reusable, repeatable process makes everybody much more comfortable and trusting of the data that they're getting. They can then focus on the issues at hand. qTest really becomes a tool, and the best thing about a tool is not knowing you're using it. To that end, it's doing a really great job.

I can't say that we've seen a decrease in critical defects in releases since we started using qTest, but we have more visibility into our test coverage, block test cases, daily activities, etc. But I can't say that it's done anything to necessarily improve the quality of code.

Overall, it has helped to increase testing efficiency by around 30 percent. A lot of that, again, is due to being able to reuse things and being able to get to the metrics quickly. I can't overemphasize how easy it makes things.

We are using qTest Manager and qTest Insights, primarily.

We have a global testing team, so we needed a centralized, easy, web-based interface for accessing all of our testing and being able to manage test cases. We ship over 400 projects a year, so we needed something that was going to scale to that. That's what the Manager piece is for.

The Insights piece is for obtaining where we are in terms of status and finding out about metrics, etc. It provides insight into the status of my testing.

Also, we are fully integrated with JIRA, so back and forth, we use JIRA as an agile shop. JIRA does all of our user stories, etc., and is the main source for defects. qTest Manager is the testing hub. The integration between the two has been great, pretty much seamless. We did run into one defect with volume, but the 9.7.1 release fixed that.

They're coming out with a new feature now, an analytics module. I, personally, slide more toward the metrics/analytics side of things, so anything they can do to come up with a reliable template where I can look at all of my metrics at a project-, quarter-, or enterprise-level, would be fantastic. So that's my "nirvana" goal. I don't want to have to go to Tableau. I have a lot of hopes in their analytics module.

And I would really love to find a way to get the results, into qTest Manager, of Jenkins' executing my Selenium scripts, so that when I look at everything I can look at the whole rather than the parts. Right now, I can only see what happens manually. Automation-wise, we track it in bulk, as opposed to the discrete test cases that are performed. So that connection point would be really interesting for me.

We have between 150 and 200 users who are all QA. Project managers might cycle in sometimes for metrics, but we publish our metrics. You can embed scripts that come out of Insights, which is a really great feature. It's a feature I would really like to see them work on more, to make sure their APIs are bi-directional or timely. It's a little unclear if they refresh at a certain point in time or when I click it. That is one area that is a little murky.

We hit a wall before the 9.7.1 upgrade — we waited too long to upgrade. We hit a volume where we started seeing that JIRA and qTest were out of sync a little bit. It seemed to be a timing thing. But once we upgraded, that all went away. 

In terms of stability, I don't think it's ever crashed. Sometimes the Insights module is slow to load up but I think that is a timeout issue.

I have a better feeling about scalability with 9.7.1 than I did prior to that. We should be okay. There will become a time, though, where we're going to have to consider archiving data, and how we want to do that. That would be a great feature for them to have over time, to be able to go back and archive.

We continue to bring a lot of projects into the department, so the volume of projects that qTest will manage for us continues to increase. We're starting to integrate it a lot with other products. An example would be SmartBear — we do a lot of API testing there. Anything that Tricentis would build, API-wise, along those lines would be really helpful.

We use NeoLoad for all our performance testing and that integrates with AppDynamics, so I don't know that we would need to integrate them, but it would be nice if it were an option. We definitely continue to use JIRA. We'll continue to expand on that platform. There's a lot of potential.

I speak very highly of the company, especially the QASymphony folks who were merged into Tricentis. There was some merger pain in terms of availability. We found that our calls were cycling. But they recognized that pretty quickly and definitely helped us get on the right path. 

When we were doing the upgrade, we were able to get slots scheduled fairly easily. 

So tech support is as I expect it to be, at this point. 

I have names of people whom I can call. That's always nice. It's not just "1-800-qTest." As a vendor they're attentive. They've been up here a few times and we definitely have a view into their roadmap. I find that as much as you're willing to give, you'll get.

We were using the HP suite. We switched because of price point and ease of use. We went into agile quickly, as an enterprise, and HP wasn't at an agile point at that time. We needed to make a switch.

qTest is much more intuitive and straightforward. There's not a lot of complexity to it. HP opened up the world so there were far too many features than we needed. That became a burden over time. HP's integration with JIRA was difficult. It was a thick client and it was very difficult to use the web interface and have good response times. I could go on and on, but you get the gist of it.

We did a prototype two years ago and demo'ed it. It definitely played strong. The price point was right and then we started road-mapping it in 2018. We started implementing in October of 2018. We have a lot to do here so it took us until June of 2019 to get us all to steady-state. But it went without hitches, and that's probably due to a combination of how much planning we put into it and its ease of use.

You need to plan it. You need to know what your JIRA templates look like. You need to know what your JIRA workflow is, and then you need to understand what you want qTest Manager to look like. If you're integrating with JIRA, that will be the defining piece in how all of that structure will look. Once you understand that — and fortunately, I have control over both in my department, so we are really intimate with what our Jira template looks like — it really maximizes how it integrates and the efficiency of how to get to where we wanted to go. It sounds like it took a long time, but it was really a lot of planning time and then we did the cutover. We also had a lot of training that we put into it. Having done this before, qTest was, by far, one of the easiest ones I've done.

We do internal audits on our own. We look back quarterly and say, "Are we meeting our own processes? Do we have reliable, reputable standards with our projects and the metrics, the way we count things? Are we consistent?" I do you think you have to measure yourself, in addition to measuring your projects. That's really helped us significantly.

As for adoption of the tool, it's been really easy. It has simplified a lot of things. Things are right there. You can quickly drill through and it's pretty intuitive to pick up. There's not a lot of complexity around it. There are not a lot of unnecessary fields. The training on it and the adoption of it have been a lot easier than with HP.

The deployment was all my department. We have a third-party vendor, Cognizant, that we work with. We have an 80/20 split: 80 percent of the department is Cognizant, 20 percent is Guardian. This touched everybody in the department, and we're somewhere between 150 and 200 people. But we had a core team of about a dozen people who mapped and planned it all out, and then they touched the rest of the department as their projects migrated over.

I have two to two-and-a-half people maintaining it.

I definitely see ROI in that I have testers who are focused more on doing really complex testing, rather than writing test cases. The reusable regression suite is always a good thing; to be able to copy and move tough cases from one project to another. I don't want my testers to be rewriting things.

I have not looked at it recently, but our license price point is somewhere between $1,000 and $2,000 a year. It's pretty low when you think about what we used to have. We haven't had any additional costs from Tricentis. We do the hosting on Amazon, so that's our cost.

We actually did a bake-off between Tricentis and QASymphony. And then we got the best of both worlds when Tricentis acquired QASymphony.

We looked at Zephyr and Xray but they were really too small-scale for the enterprise that we have. They probably would have saved us a lot of money, but our efficiency would have really scaled off.

What I've learned from using the solution is "don't be afraid of change." HP was the blockbuster of our industry. There are a lot of great options out there. Do your due diligence and be brave.

Also, have a plan. It's not something that you want to go into and figure out as you're going. You need to really sit down and consider where you are, where you want to go, what variables are going to help you figure out how to implement this. It's just like any other software package. You need to need to have a plan. You need to have a training plan. You need to make sure your team understands the opportunity and what they're going to get out of it. It can be scary, so you have to manage change as much as you have to manage implementation.

In terms of using qTest to look into failures, we haven't really enabled that part of it, yet. We use Selenium to do all of our automation, and that's a little different than using the Tricentis application. We're a Java shop and .NET shop, so we wanted to go with an open-source tool so we could hire Java developers for automation. We also use Selenium for open-source test automation. We know there are some exploratory options in qTest, and we will start setting the roadmap for 2020 in that direction. We definitely want to expand what we're using within the product, now.

Getting our upgrade to 9.7.1 was really significant for us. This past year has been a migration year. We got to steady-state around June. I wanted to get everyone to steady-state, spend some time with it, get our upgrade behind us, and then start to expand out to use some other pieces of functionality in 2020.

We use qTest to provide results to executives, but it's usually sanitized through my team a little bit, just because we're still getting used to the cycles and execution. We can do multiple runs and we have to get down to what the actual results are, not the overall multiple runs' results. I use qTest for that and that information gets cleaned up before it goes out to executives. The information it provides them is accurate.

qTest is an eight out of ten at this point. For me, it's been the metrics. Every company counts things differently. To understand their reports, out-of-the-box, and to align the solution to where I want it to be — do I massage it to maintain the metric that I have or do I have to wait for a breaking point or do I redefine my calculation — is where those two points go. That has taken a little more than I would've expected. All the data is there. It's just a matter of how you're layering it out for your company.

Knowing what our calcs are and knowing what the qTest calcs are, and where they diverge, would've been really great. We were a little more naive than we had planned for.

I hope Tricentis keeps it alive and well. It's a great little product that is going to quickly grow to be something that gets out there with the big boys.

We are using qTest to store and house test scripts and test design. We're using the test execution to execute and we're using the Defects module to track defects.

This is an SAP implementation and until we brought in qTest the team had no tool. They were doing everything manually in Excel, all their tests and execution. I came on board to help lead. We've done multiple test cycles. We're in UAT right now. They did one integration test cycle without the tool and we've done two with the tool. It's helped with productivity when you compare it to doing it manually in Excel.

The solution's reporting has enabled test team members to research errors from the run results, for the most part. In terms of its effect on their productivity, they went from a complete manual Excel testing solution to this tool. We're able to execute 550 test scripts within a six-week period. Some of these are simple tests but some of them are robust integration scenarios.

Our company had an assessment done by an outside organization before the tool was in use. When they did their assessment they said they were not seeing good metrics. When they did the second assessment, they asked, "What's changed since our last assessment? We're seeing numbers, we're seeing reports, we're seeing data. Everything looks stable and not all over the place." They definitely noticed the change, and that's an outside organization.

qTest helps us compile issues and have one place to look for them. We're not chasing down emails and other sources. So in the grand scheme of things, it does help to resolve issues faster because everyone's working off of the same information in one location.

Overall, the solution has increased testing efficiency by about 60 percent, compared to what they were doing before.

We get the most benefit from the

• test library
• test execution.

There is also a high level of reporting provided in the tool and you can extract reporting out of the tool. That has been very beneficial.

Ease of use is another helpful aspect. Right now, during the UAT, we have a lot of business users using the tool. They test the UAT scripts. The team quickly adapted to the tool. I was able to roll it out and I was actually commended by several internal people, as well as external consultants who were doing assessments, as to how things had changed. They were shocked that I was able to implement the tool so quickly. I had never used the tool before either. I received a trial copy of the tool and was able to quickly implement it and have it up and running for the team.

The Insights reporting engine is a little challenging to use in terms of setting up a report and getting the data. It took me a while to understand how to use the tool. 

I'm mainly extracting the data out of the tool. I'm not necessarily using any of the dashboards in the tool. There are some fields that I did not make site-specific because I had to get things up and running quickly. The fields are in both the Test Run area and Defects. If you do a project via site-specific, you can't get any of those fields out of Insights. That's a limitation that they need to figure out. They shouldn't have that limitation on the tool.

In addition, I really can't stand the Defects module. It's not easy to use. ALM Micro Focus used to be called QC. That solution's Defects Module is really robust. For example, let's say you have a defect and you have a query. You can actually walk through each defect by just clicking an arrow. You go through that defect, add your updates, click the "next" arrow, and walk down through them. But with the qTest Defects module you can't do that. You have to run a query. You're pretty much just querying a database. It's not really a module, or at least a robust module. Everything is very manual. By contrast, qTest's test design and test execution modules are very robust. They just missed the boat on the Defects module. From what I've heard and from what I can understand, other people are using JIRA or something else to do their defects tracking, and we're not. I needed a tool to do everything. That's their weakest link.

Overall, qTest is stable. Sometimes we see some performance slowdowns, a hiccup or glitch-type of pause. But for the most part, it has been operating. We haven't felt any pain yet.

Right now, it's handling everything we're throwing at it. Since we're in UAT, this will be the highest number of people in the tool and probably the most activity in the tool, and it's been supporting things without any issues.

We went from 30 licenses to 60 licenses during this four-month period of time. I don't think that number will be increased. Once this project is over, the number of consultants will be reduced and the number of people involved will be reduced.

Technical support has been fine, acceptable. Their responses have been in an inappropriate amount of time for the most part. 

There are just those two limitations that I've uncovered, as compared to other tools that I've used. So a lot of my interactions are like, "Hey, I want to do this," and they say, "Oh, you can't do that," or "the tool doesn't support that." That's the thing I have run into the most. It's not a support issue, it's just a tool issue. Functionality.

Cost and time were the main reasons I went with qTest. If I were to have my choice, I probably would have implemented the Micro Focus product because I am familiar with it and know it can do everything I wanted to do. But that would likely have been overkill; way more than this project needed, and it was much more costly. 

I was looking at another tool, the SmartBear QAComplete tool that I had used on a previous project. I didn't necessarily like that tool, but its cost was less than either qTest or HP QC/ALM. But once I get my hands on qTest, I definitely liked it better than the QAComplete product.

The ease of use and the interface helped push me toward qTest. I had also called a friend and he said. You have to look at QASymphony or Tricentis. This qTest is good." I said, "Are you sure?" He said, "Yes, it's good. Trust me." That helped push me over the top.

The initial setup was challenging. There are certain areas where it's very strict in how you have to set up your project. There are some strict guidelines that you have to follow. You have to have a release and a test plan. You can't do certain things within the test design module or test execution module. There are only certain ways that you can set up a folder structure, whether it's related to a cycle or a test suite or a module. I would prefer fewer restrictions. The restrictions are what made it complicated.

Deployment itself was done over a weekend. This is not on-prem, it's in the cloud. I set up the structure and then had to understand how to load the test scripts. It was very fast.

Our implementation strategy was to get it done as soon as possible. It was very off-the-cuff. There was no time to plan. I landed here right before this test cycle was supposed to start and I knew that if we left it as a manual execution we would fail miserably. For me, the plan was to identify, learn, and implement a tool, all within less than a week. It took me two weeks, including training myself. There was no plan other than "we need a tool."

I used Justin, I used one of the support people, and I had one meeting with one of their people. I had no more than four hours of support and a couple of emails. 

Overall, my experience with their support during deployment was good. I was asking some questions and needed to take an approach that either they didn't agree to or didn't understand why I was doing it that way. The one person I remember talking to was so tied up in the Agile methodology that she couldn't see outside the Agile box, and that's what I needed. We weren't coming from an Agile methodology.

Over the four months there has been ROI. If I crunched the numbers I would probably find it has paid for itself already.

The price I was quoted is just under $60,000 for 30 licenses, annually, and that's with a 26.5 percent discount.

I've used QAComplete from SmartBear. I've used HP QC or ALM from Micro Focus. I also used an old IBM rational test manager which I think was called SQA.

I think qTest was really built to support Agile, where the other tools were built to support traditional Waterfall but were easily adaptable to Agile. qTest is probably going to struggle a bit before they can truly support non-Agile implementations.

The biggest lesson I have learned from using qTest is that every tool has limitations and you need to be able to adapt and overcome and not be stuck with one way of doing things. You have to find out where the tool shines and where it doesn't and make sure that the team feels the least amount of pain when you do implement it.

This solution has been implemented for one particular project. We have 60 concurrent licenses available and we have about 120 users who have been given access. Their roles in the project are either business analysts or quality testers. But these people also have their roles within the business. Some are managers within finance, some are directors, some are AP specialists, some are AR specialists. The project is a financial system implementation so we have a sampling of users from all departments executing scripts.

Since implementing the tool, we've seen a decrease in critical defects but I don't know if I can attribute it to the tool. I don't know if that's possible. It might be a stretch. But we definitely have seen a drop in critical defects. Over the last four months we have seen a 40 to 60 percent drop.

For deployment and maintenance of the solution, it's just me. I'm the one who picked the tool, I'm the one who implemented the tool, I'm the main administrator of the tool, and I am leading all of the testing efforts.

Setting up the users is pretty simple. I would recommend it. If you're looking for something quick, easy to use, and robust, it's definitely a very good tool. If I could get them upgrade the Defects module, I would be very happy.

I do love it. I'm giving it a nine out ten just because I don't think any tool out there is a ten, but Tricentis is close.