Try our new research platform with insights from 80,000+ expert users

SmartBear TestComplete vs TFS vs TestRail comparison

 

Comparison Buyer's Guide

Executive Summary

Review summaries and opinions

We asked business professionals to review the solutions they use. Here are some excerpts of what they said:
 

ROI

Sentiment score
6.8
SmartBear TestComplete automation saves time, enhances client satisfaction, and boosts efficiency, with annual savings of approximately $10,000.
Sentiment score
4.5
TestRail improved productivity, collaboration, and accuracy in test management, providing valuable reporting, integrations, and an intuitive interface for users.
Sentiment score
7.2
TFS enhances productivity and cost-efficiency by automating processes, integrating with Microsoft products, and improving software development for oil and gas.
Once set up, only one person is needed to handle all tasks, reducing the requirement for multiple personnel.
Time-wise, it saves about fifteen to twenty percent compared to Excel, and money-wise, it's around ten percent.
Integrating TFS with Visual Studio and Azure Cloud has improved our development processes by providing better integration and reducing errors.
 

Customer Service

Sentiment score
6.9
SmartBear TestComplete's customer support is knowledgeable but inconsistent, with delays and unresolved issues needing faster escalation and responses.
Sentiment score
2.5
TestRail's self-sufficient support has mixed reviews on communication and responsiveness; some seek improvements for complex issues.
Sentiment score
6.9
Users praise TFS support for its excellence, quick responses, and resources, although some suggest improvements in response time and SLAs.
Sometimes, you really need to speak to a person, and arranging such calls is not easy.
I personally escalate Leapwork issues with the Leapwork support tech team, and they are very quick to solve problems.
as a Microsoft product, it might have limited global documentation or support options compared to GitLab.
 

Scalability Issues

Sentiment score
7.4
SmartBear TestComplete is scalable and adaptable, with flexible scripting, but may require licensing for wider deployment.
Sentiment score
9.2
TestRail remains stable and scalable with minimal maintenance, effectively managing increasing users and data loads for organizations of all sizes.
Sentiment score
7.4
TFS is scalable, suitable for enterprises, but larger organizations may encounter challenges; rated 6 to 8 out of 10.
We achieved this scaling without any significant issues.
I have never experienced any issues with its scalability.
 

Stability Issues

Sentiment score
6.9
SmartBear TestComplete generally stable, but users report crashes, memory leaks, and HTML5 testing delays in certain scenarios.
Sentiment score
5.3
TestRail is praised for its strong stability and reliability, with users consistently rating it highly for performance.
Sentiment score
7.7
TFS is reliable and robust, though minor issues under load or specific setups can impact performance, requiring proper hardware.
It has never had any issues.
Its stability is lacking as we have encountered security leaks and glitches.
 

Room For Improvement

SmartBear TestComplete faces challenges in object recognition, integrations, licensing, performance, and support across browsers and mobile devices.
TestRail users seek enhanced reporting, better integrations, built-in defect management, and improved usability, performance, and automation features.
TFS requires improved interface, tool integration, agile features, merging, reporting, affordability, stability, cloud synergy, and performance enhancements.
While using SmartBear TestComplete, we are fine with the current capabilities, however, it would be beneficial to improve some performance aspects, especially the image comparison feature.
TestRail lacks the functionality to map test cases to requirements and risks.
TFS is not as fast, easy to use, or configurable as GitLab, despite moving into the cloud.
I am content with how TFS is structured now, particularly the Azure version.
 

Setup Cost

SmartBear TestComplete's pricing and licensing receive mixed reviews, seen as both reasonable and costly depending on usage and modules.
Enterprise users have mixed opinions on TestRail pricing, recommending shopping around and negotiating for the best deals.
Enterprise TFS users experience varied costs; many manage expenses with agreements, praising integration despite fewer features than HPE ALM.
 

Valuable Features

SmartBear TestComplete excels in cross-platform automation, integration, and support for multiple languages, enhancing automated testing efficiency and maintenance.
TestRail excels in test management with intuitive organization, JIRA integration, user-friendly interface, real-time dashboards, and global accessibility.
TFS integrates with Visual Studio, enhancing version control, lifecycle management, and Agile support, valuable for diverse development needs.
The most valuable feature of SmartBear TestComplete for me is the image comparison functionality.
The reporting capabilities and the simplicity of it make it user-friendly, as new users can easily understand the tool.
It significantly saves effort in managing test execution and managing all test cases.
The integration with Azure DevOps also offers seamless functionality for CI/CD processes.
Makes it easier for me to create builds and release pipelines without needing to program YAML files.
 

Mindshare comparison

Test Automation Tools
Test Management Tools
Test Management Tools
 

Featured Reviews

Prakhar Goel - PeerSpot reviewer
Used for integration automation, user-based automation, and web automation
The solution's most valuable features are the drag-and-drop feature, keyword-driven approach, and reusability of the scripts. The solution has introduced a new feature that helps us identify objects we cannot normally identify. It gives you a fair idea of objects, resolving the object recognition issue. The solution can be used to perform different tests on different machines.
StuartBarker - PeerSpot reviewer
A tool that provides effective test management and real-time reporting capabilities
I have faced some issues with the integration between TestRail and Jira where the status of tests is not displayed (in Jira) due to I suspect security settings on the browser. In a large corporate environment, it is not easily changed. The support wasn't particularly helpful. It would be great if I could create custom reports, ideally with a tool designed specifically for that.
Pmurki@Micron.Com Praveen - PeerSpot reviewer
Version control is excellent and good for code review, branching, merging strategies and more
I've worked with TFS for source control and Agile project management. We also used TFS for seamless team collaboration and tracking.  I used TFS for a couple of years. Now, we use Azure DevOps. It's a wonderful tool for source control and CI/CD pipelines It's a really valuable tool for…
report
Use our free recommendation engine to learn which Test Automation Tools solutions are best for your needs.
862,077 professionals have used our research since 2012.
 

Top Industries

By visitors reading reviews
Computer Software Company
18%
Manufacturing Company
15%
Financial Services Firm
13%
Government
6%
Computer Software Company
18%
Financial Services Firm
11%
Manufacturing Company
11%
Healthcare Company
6%
Educational Organization
24%
Computer Software Company
10%
Manufacturing Company
9%
Financial Services Firm
8%
 

Company Size

By reviewers
Large Enterprise
Midsize Enterprise
Small Business
 

Questions from the Community

What do you like most about SmartBear TestComplete?
TestComplete has strong reporting capabilities. The reports they generate are really good.
What is your experience regarding pricing and costs for SmartBear TestComplete?
I am not involved in pricing or licensing; our management team handles these aspects.
What needs improvement with SmartBear TestComplete?
While using SmartBear TestComplete, we are fine with the current capabilities, however, it would be beneficial to imp...
What is your experience regarding pricing and costs for TestRail by Gurock?
Pricing is reasonable for TestRail. It offers good value for money.
What needs improvement with TestRail by Gurock?
TestRail lacks the functionality to map test cases to requirements and risks. While integration with another tool is ...
Which is better - TFS or Azure DevOps?
TFS and Azure DevOps are different in many ways. TFS was designed for admins, and only offers incremental improvement...
What do you like most about TFS?
Microsoft's technical team is supportive.
What is your experience regarding pricing and costs for TFS?
While I do not know the exact pricing, TFS is likely more expensive than GitLab.
 

Also Known As

No data available
TestRail by Gurock
Team Foundation Server
 

Overview

 

Sample Customers

Cisco, J.P. Morgan, Boeing, McAfee, EMC, Intuit, and Thomson Reuters.
Apple, Microsoft, Boeing, Intel, NASA, Amazon, HP, Samsung
Vendex KBB IT Services, Info Support, Fujitsu Consulting, TCSC, Airways New Zealand, HP
Find out what your peers are saying about Tricentis, UiPath, OpenText and others in Test Automation Tools. Updated: June 2025.
862,077 professionals have used our research since 2012.