Try our new research platform with insights from 80,000+ expert users

SmartBear TestComplete vs TFS vs TestRail comparison

 

Comparison Buyer's Guide

Executive Summary

Review summaries and opinions

We asked business professionals to review the solutions they use. Here are some excerpts of what they said:
 

ROI

Sentiment score
6.7
SmartBear TestComplete automation saves time, enhances client satisfaction, and boosts efficiency, with annual savings of approximately $10,000.
Sentiment score
8.0
TestRail improved productivity, collaboration, and accuracy in test management, providing valuable reporting, integrations, and an intuitive interface for users.
Sentiment score
7.8
TFS enhances productivity, reduces costs, and integrates well with Microsoft tools, proving valuable for efficient software development.
Once set up, only one person is needed to handle all tasks, reducing the requirement for multiple personnel.
Time-wise, it saves about fifteen to twenty percent compared to Excel, and money-wise, it's around ten percent.
Integrating TFS with Visual Studio and Azure Cloud has improved our development processes by providing better integration and reducing errors.
 

Customer Service

Sentiment score
6.9
SmartBear TestComplete's customer support is knowledgeable but inconsistent, with delays and unresolved issues needing faster escalation and responses.
Sentiment score
4.0
TestRail's self-sufficient support has mixed reviews on communication and responsiveness; some seek improvements for complex issues.
Sentiment score
7.1
Most users find TFS support efficient, despite some wanting faster responses, with scores generally between 8 and 10.
Sometimes, you really need to speak to a person, and arranging such calls is not easy.
as a Microsoft product, it might have limited global documentation or support options compared to GitLab.
 

Scalability Issues

Sentiment score
7.3
SmartBear TestComplete is scalable and adaptable, with flexible scripting, but may require licensing for wider deployment.
Sentiment score
9.5
TestRail remains stable and scalable with minimal maintenance, effectively managing increasing users and data loads for organizations of all sizes.
Sentiment score
7.5
TFS is scalable and integrates well with Microsoft apps, but faces challenges in large deployments with high user loads.
 

Stability Issues

Sentiment score
6.9
SmartBear TestComplete generally stable, but users report crashes, memory leaks, and HTML5 testing delays in certain scenarios.
Sentiment score
9.7
TestRail is praised for its strong stability and reliability, with users consistently rating it highly for performance.
Sentiment score
7.8
TFS is stable and dependable, with minor concerns addressed through updates, but lacks agility compared to Jira.
It has never had any issues.
Its stability is lacking as we have encountered security leaks and glitches.
 

Room For Improvement

SmartBear TestComplete faces challenges in object recognition, integrations, licensing, performance, and support across browsers and mobile devices.
TestRail users seek enhanced reporting, better integrations, built-in defect management, and improved usability, performance, and automation features.
TFS needs stability, interface, merging improvements, better integration, lower costs, simplified features, and enhanced agile and cloud support.
While using SmartBear TestComplete, we are fine with the current capabilities, however, it would be beneficial to improve some performance aspects, especially the image comparison feature.
TestRail lacks the functionality to map test cases to requirements and risks.
TFS is not as fast, easy to use, or configurable as GitLab, despite moving into the cloud.
I am content with how TFS is structured now, particularly the Azure version.
 

Setup Cost

SmartBear TestComplete's pricing and licensing receive mixed reviews, seen as both reasonable and costly depending on usage and modules.
Enterprise users have mixed opinions on TestRail pricing, recommending shopping around and negotiating for the best deals.
TFS pricing is competitive yet complex, favoring Microsoft's subscription for cost efficiency, especially beneficial for existing Microsoft users.
 

Valuable Features

SmartBear TestComplete excels in cross-platform automation, integration, and support for multiple languages, enhancing automated testing efficiency and maintenance.
TestRail excels in test management with intuitive organization, JIRA integration, user-friendly interface, real-time dashboards, and global accessibility.
TFS offers versatile version control, seamless Visual Studio integration, robust lifecycle management, and efficient project and code management.
The most valuable feature of SmartBear TestComplete for me is the image comparison functionality.
It significantly saves effort in managing test execution and managing all test cases.
The integration with Azure DevOps also offers seamless functionality for CI/CD processes.
Makes it easier for me to create builds and release pipelines without needing to program YAML files.
 

Mindshare comparison

Test Automation Tools
Test Management Tools
Test Management Tools
 

Featured Reviews

Prakhar Goel - PeerSpot reviewer
Used for integration automation, user-based automation, and web automation
The solution's most valuable features are the drag-and-drop feature, keyword-driven approach, and reusability of the scripts. The solution has introduced a new feature that helps us identify objects we cannot normally identify. It gives you a fair idea of objects, resolving the object recognition issue. The solution can be used to perform different tests on different machines.
StuartBarker - PeerSpot reviewer
A tool that provides effective test management and real-time reporting capabilities
I have faced some issues with the integration between TestRail and Jira where the status of tests is not displayed (in Jira) due to I suspect security settings on the browser. In a large corporate environment, it is not easily changed. The support wasn't particularly helpful. It would be great if I could create custom reports, ideally with a tool designed specifically for that.
Pmurki@Micron.Com Praveen - PeerSpot reviewer
Version control is excellent and good for code review, branching, merging strategies and more
I've worked with TFS for source control and Agile project management. We also used TFS for seamless team collaboration and tracking.  I used TFS for a couple of years. Now, we use Azure DevOps. It's a wonderful tool for source control and CI/CD pipelines It's a really valuable tool for…
report
Use our free recommendation engine to learn which Test Automation Tools solutions are best for your needs.
850,349 professionals have used our research since 2012.
 

Top Industries

By visitors reading reviews
Computer Software Company
20%
Manufacturing Company
14%
Financial Services Firm
13%
Government
7%
Computer Software Company
20%
Manufacturing Company
10%
Financial Services Firm
10%
Insurance Company
6%
Educational Organization
67%
Computer Software Company
5%
Financial Services Firm
4%
Manufacturing Company
3%
 

Company Size

By reviewers
Large Enterprise
Midsize Enterprise
Small Business
 

Questions from the Community

What do you like most about SmartBear TestComplete?
TestComplete has strong reporting capabilities. The reports they generate are really good.
What is your experience regarding pricing and costs for SmartBear TestComplete?
I am not involved in pricing or licensing; our management team handles these aspects.
What needs improvement with SmartBear TestComplete?
While using SmartBear TestComplete, we are fine with the current capabilities, however, it would be beneficial to imp...
What is your experience regarding pricing and costs for TestRail by Gurock?
Pricing is reasonable for TestRail. It offers good value for money.
What needs improvement with TestRail by Gurock?
TestRail lacks the functionality to map test cases to requirements and risks. While integration with another tool is ...
Which is better - TFS or Azure DevOps?
TFS and Azure DevOps are different in many ways. TFS was designed for admins, and only offers incremental improvement...
What do you like most about TFS?
Microsoft's technical team is supportive.
What is your experience regarding pricing and costs for TFS?
While I do not know the exact pricing, TFS is likely more expensive than GitLab.
 

Comparisons

 

Also Known As

No data available
TestRail by Gurock
Team Foundation Server
 

Overview

 

Sample Customers

Cisco, J.P. Morgan, Boeing, McAfee, EMC, Intuit, and Thomson Reuters.
Apple, Microsoft, Boeing, Intel, NASA, Amazon, HP, Samsung
Vendex KBB IT Services, Info Support, Fujitsu Consulting, TCSC, Airways New Zealand, HP
Find out what your peers are saying about Tricentis, Katalon Studio, OpenText and others in Test Automation Tools. Updated: April 2025.
850,349 professionals have used our research since 2012.