SmartBear TestComplete Reviews

TestComplete is simple, it's a very easy-to-use tool. It offers credibility and value for the money. A basic license is roughly $4,000 and you can add flexible components on top of it. 

If you want desktop testing, you can add-on a desktop license. If you want web testing, you're going to want to add-on a web license — there's a lot of flexibility, you just need to pay for what you use. There are no irritating subscription models.

People are able to quickly use the platform and with a variety of scripting languages, including Python and JavaScript — all the modern scripting languages are supported. It doesn't just rely on VB script like UFT.

It's very flexible and robust in that way. I have seen many of our clients quickly adopting the tool with all the scripting languages. 

Recently, they've been building a lot of futuristic features, for example, AI Self-healing is one of the interesting features where they try to improve and cut down on maintenance by automatically correcting the arbiter. That's a really cool feature for keeping your object repositories up to date, and it can considerably bring down or control your maintenance costs to some extent, at least as far as the object repository goes.

They also have some intelligent OCR features. They have a mini device cloud, for example, which allows you to run a testing tool and recently started supporting X spot. That actually goes well with selenium. You can reuse some of the scripts with other frameworks. They also acquired CucumberStudio a few years back. They have combined HipTest and Cucumber into one capability — CucumberStudio —, which is a great integration to TestComplete, that really becomes very seamless.

They're working on many features. Of course, the roadmap is not news to me, but yes, of course, they are working on different features.

I have been using this solution for many years.

We have not faced any concerns with respect to the stability of the tool; at least we have not seen any major issues with the tool where it malfunctions or anything, never.

It is a scalable solution. The beauty of TestComplete is that it's not tied to one scripting language. All the different scripting languages come with different abilities. You have different types of metrics and controls available with different scripting languages. That way we can drive our own framework. You can create many custom frameworks using TestComplete that will suit your organization. That's what we have been doing with many companies — it's an all-in-one solution.

The technical support is excellent. We don't deal with them directly, but the feedback that I have received from different clients is that it's really excellent. They are always attentive —  That's a feeling I've always got from different clients.

The initial setup is pretty straightforward. There's a good license server map for all those things — it's a fairly straightforward solution.

The licensing costs are in the range of $1,000 to $3,000.

We do a lot of consulting and training services for SmartBear. We try to educate the users in terms of the new features available in TestComplete so that they can do some smart automation. It's not just for automating some scenarios, you can optimize a lot of your effort.

On a scale from one to ten, I would give this solution a rating of nine.

I built it for a demonstration for my team and it's currently dealing with procurement as far as cost, et cetera. The demonstration deals with how the tool is easy to use for desktop and web applications and, of course, web applications on mobile devices as well. No mobile device applications yet. I was able to spend that also using the TestComplete cloud. Instead of building my own grid, I used the test Selenium cloud, which you could use selecting different platforms and different web browsers per platform and you could span them across multiple devices and that was extremely helpful. Building is the hardest part about the frameworks because there's really no right way to do it.

In TestComplete, I saw a conformed package of a tool that kept everybody in consistency. The team was able to regenerate further tests without having to manipulate more code because the record feature is great. Because we use JavaScript, we convert to JavaScript and then we just expand it from there and that was wonderful. The easiest way to start the test case or test scenario for design purposes is to put it on record, click it and I start messing with the automation. I was very impressed because I didn't have to step out of bounds of TestComplete and that's the kind of tool I've been looking for since I've been automating.

The most useful feature is the name mapping. It allows me to not have to worry about the object's name; I just use my alias.

This affected the way my client's organization functions in several ways. Number one is the consistency of the versioning. Everyone's using the same tools, same kind of code, same kind of coding methods. In case somebody's sick or out, another team member can pick up where they left off and keep going. 

It's also great because that record feature is not for the mindless. The recording plate is not only there to help you start writing the test, it helps you as a tester or an engineer understand the business mindset of how they built the application. You have to go in yourself and click around. Then, it puts everything in code and then helps you understand. We were testing sites like pharmacies, hotels, insurance providers, and other commercial companies like PetSmart. That's where I was using it and that's where the conformity got a lot better.

I was going to say load and performance but I really didn't get a chance to get into the load and performance part of TestComplete.

I can sense there's still a little bit more room for improvement in terms of making the whole testing process easier. Because it was desktop first, it kind of focused on COTS-based testing and API's because everyone's using it at the moment.

To bring it up to a 10, I would be looking for the addition of some key functional API testing. What I mean by key is like the video streaming or a consistent flow of data being used and then how much can it handle and how much API testing can it do because that's where the load and performance comes in. The API testing is where I would see a lot more improvement.

I've been using this solution on and off for about four years. About a year in total.

I have to say that scalability is pretty sufficient for now. I'm the test architect for the company and I have eight employees under me and we are all using the solution.

I would rate my experience with tech support as 10 out of 10. I had two emails and two calls with tech support. Within five minutes, my problem was solved. They were immediately aware of what I was having difficulty with and then showed me the right way to implement the solution.

We've been using Selenium with Java and it's too hard to manage the framework because everyone's changing it because they're running it locally and then they'll put it off the grid, but locally, they're building on the machine. I incorporated a Docker and I containerized the test framework so that now, new employees can go and just install the container but then there's still that engineering mindset. They want to go out and play, so it's just hard to manage. With TestComplete it's all encapsulated in one.

The initial setup was very straightforward. I didn't experience any bugs, glitches, or crashing. I was running 16 tests at one time using half of using my grid and half using the cloud and everything was running fine. I had to test and see if TestComplete could actually go on a Selenium grid and it did.

We're still kind of in the evaluation phase. The new requirements are still in development. So we're using this time to hang up our testing environment. 

We're looking for more COTS-based tools or tool so we don't have to do as much management. We want a tool that does the management as versions up and then we just lock it down.

My advice to others is: If you're looking for speed, configuration consistency, and accuracy of tests with fantastic results, go get TestComplete. If you're starting with it, you'll see how much more you can expand on it.

Desktop and web application support. TestComplete fits almost perfectly with a large amount of stacks, such as Delphi, C#, Java and web applications. It's an amazing feature for companies that want to automate UI tests on each application built in-house.

Before using TestComplete on our projects, we used to use a lot of tools to automate our applications, such as Sikuli or Java Robot to automate desktop apps, and Selenium WebDriver for web apps. After starting to use TestComplete, we were able to centralize all scripts in only one tool and technology.

TestComplete gives support to do requests to a SOAP web service but has no support to do HTTP requests on Restful services. In a microservice world, this is a big flaw. Another thing is that the cross-browser support has a lot of different traits between browsers. It should be improved.

Five years.

Sometimes the app crashes during test execution based on the amount of code that is running. It's recommended that you modularize scripts, but that is not a justification for not being more stable.

TestComplete has a test executor app that can be used in a distributed test execution environment. The problem is that this is a paid product. Thus, it was pretty expensive to scale this architecture.

Eight out of 10.

I used to use HPE QTP but TestComplete has the same features, low price, and support for handling a lot of stacks. In addition, I don't need to use hundreds of plugins.

To start working with TestComplete, we only needed to install (next, next, finish flow) and then start using it. There are some configurations to do to help increase efficiency, but I do not consider that more than a nice-to-have.

Buy modules on demand. If you have a four-person team and they will each automate tests only 25% of the time, it's better to buy a floating licence and share the tool during the work day. If they will each use it all the time, buy licences for all of them.

No, I did a PoC and discovered the pros and cons.

Do a PoC and try to understand if TestComplete fits your context and requirements. Use the script-driven approach instead keyword-driven, because the former is more efficient.

• SoapUI/ReadyAPI integration
• Selenium integration
• We run all of tests using TestExecute, this is just the executer for all TestComplete tests – command line version so can be integrated with any CI tools
• We used to these tests seamlessly on Jenkins (TestComplete has a plugin for Jenkins)
• The best part is TestComplete gives choice of selecting any of the six languages it supports
• List of checkpoints available in TestComplete is one of the strongest available in the market
• We also make heavy use of version control integration and JIRA integration
• Data driven testing feature of this tool is one of the best

We did a POC for number of tools in past and eventually decided on TestComplete. We started with one tester and one test which we used to trigger manually. As the product matured, it added cross browser, mobile testing functionality, integration with Selenium/ SoapUI and Jenkins plugins, so we started using it more and more. Currently, the automated test team size is 20 and we run hundreds of tests automatically at the end of build process without any human intervention. For us, the automated test farm of 20 virtual machines for execution, 20 TestComplete licenses and 20 automated testers are doing the job of 100 manual testers. The ROI has improved significantly and all the regression tests for our product are automated. We have plans of using TestComplete in the test driven development approach where developers can make use of our tests as part of their unit testing and I am sure we are going to achieve a lot more from this tool in coming years.

• Error handling features in the tool are a little limited
• The tool currently is Windows only, would like to see it being implemented for Mac and Linux
• Once the tests are created in one languages, you can’t change it use other languages

There have been no issues with the deployment.

The tool sometimes seems a little unstable and crashes sometimes on Windows 10.

There are no issues with the scalability.

Average

Good

No. We did a POC on multiple tools and TestComplete was our first selection

StraightForward

In-house

QTP, TOSCA, RATIONAL, RANOREX

The tool has great capability but also has great potential to demoralize if you implement it in incorrectly. Automated tests complement manual testing and doesn’t remove manual tests completely so I would suggest keep your manual tests, they will come handy as reference when the automated test fails or needs an update. I would suggest following before you start implementing the tool in you automated test environment

• Choose the language to be used and stick to it. TestComplete supports six languages (VBScript, JScript, Python, DelphiScript, C++Script, C#Script)
• Choose the naming convention for objects in the name-map and keyword tests so that you can make use of these tests in several projects
• Always try to reuse the tests with parameterised values wherever possible rather than duplicating the tests
• Version control the tests so that you can maintain multiple versions for multiple branches of your tests
• Define and document a standard process for automation and communication
• Execute the tests frequently so that you can get the best out of the tests and defects are identified as early as possible

TestComplete is used for testing Windows desktop products; specifically the Embarcadero VCL interfaces created by Delphi/C++. All debug flags must be enabled during compilation that generates an output file required by TestComplete to identify and interact with the UI objects in the application.

An important feature of TestComplete is the ability to modularize testing. A lot of effort has been put into breaking the test script into reusable functions/methods that can be called from any test. A number of function libraries were created. This enabled reuse of code and kept the projects and project suites small. This is important because the size of these test artifacts have an impact on project loading time.

Test automation with TestComplete significantly (estimated 80%) reduced the regression timeline for a complex AUT with a large number of test cases. Automation with TestComplete has significantly shortened the feedback loop and the timeline to get a release production ready. A secondary benefit is that manual testers have begun thinking more technically about writing tests cases.

There are two major areas for improvement:

1. Version control integration embeds information in the project suite, and project files that include a direct reference to the location of the project or project suite in Team Foundation Server. When branching a set of scripts for the next version of the AUT, TFS gets confused about where the file should be mapped into source control. A workaround is to replace the hardcoded paths with relative paths.
2. There was also an issue object mapping related to the need to modularize the test script code. With the implementation of other automation test suites such at HP UFT, it is possible to have one object map for each function library. TestComplete has a limitation of only one object map per project. In order to modularize the code it was necessary to have a single shared object map used across multiple projects and project suites.

I have used TestComplete for 20 months. It was chosen as an incumbent toolset could not interact with the product to be tested.

Integration with third-party products; specifically Microsoft Team Foundation Server and HP ALM could not be overcome. A custom integration to HP ALM was written using the HP ALM OTA API.

TestComplete has been a stable product.

Customer Service can be slow to respond to electronic forms of communication and they do not have a way for a customer to speak directly to customer support. You create a ticket online, and request a phone call.

The team seems to be very knowledgeable when communication is established.

TestComplete was added to the toolset. HP UFT is used for automated tests for other products.

Setup is straightforward unless third-party tool integration is required. Integration with Microsoft Team Foundation Server is a little complex for initial configuration. Once it is understood the process is repeatable.

Implemented in-house. Implementation is not difficult to implement or write tests especially if you have experience with other test automation tools.

We have not done a ROI calculation. However, automated testing with TestComplete has cut regression test time by months.

HP UFT was tried but the object recognition did not work with the implementation technology.

The most valuable features of the SmartBear TestComplete are self-healing, they reduce the maintenance required. The different languages SmartBear TestComplete supports are good because some of our libraries are written in Python, JavaScript, and C#. It's very easy to put them all under one project and use them. The are other features that SmartBear TestComplete has but the competition widely has them as well.

In SmartBear TestComplete the integration with Jenkins could be easier. Additionally, some of the controls could have better customization options. For example, if a grid is used and it contains multiple controls within it, it can be a checkbox, radio button, or any kind of control, the way the Object Spy is operating currently there is a lot of room for improvement.

There are a lot of improvements that are required on the Object Spy, such as how we find the locators. That is a feature which I would love to see. 

It is currently very tightly coupled to the SmartBear ID. There should be a way that they could support the integration with external libraries and make it flexible to integrate with open-source frameworks. This would be a good option to add.

I have been using SmartBear TestComplete for approximately eight years.

SmartBear TestComplete is a stable solution.

When it comes to customization controls the scalability of SmartBear TestComplete is limited. However, the assets are scalable. It's easy to scale across the solutions within the same limited functionality.

The technical support from SmartBear TestComplete is good.

The initial setup of SmartBear TestComplete is easy. The process is almost instant, it is very easy.

The price of SmartBear TestComplete could be less. The main challenge is when it comes to node-locked. They should use a subscription model, such as a monthly-based subscription or, a quarterly-based subscription. Their floating license is very expensive, and this high price should be reduced or provide, at a minimum, a subscription model.

When it comes to automation, the fundamental aspects are the stabilization factor, how easy it is to automate, how much is the coverage, and how much are you able to cover with respect to an application. There will be always areas that cannot be automated. The coverage, stabilization, and ease to automate are the areas that have to be improved. The better the pricing and improved pricing models, the more it will attract more customers. 

I rate SmartBear TestComplete a seven out of ten.

It allows us to test both desktop and web applications. This is a very important feature for system integration tests, as we develop desktop applications, web applications, and a Web API using the same database.

It is very hard to read the test log generated by TestComplete Executor if the log file is very big. TestComplete Executor is a small tool for just running the TestComplete test framework (not for developing).

In fact, this is an issue regarding how to read the test log in MHT format generated by TestComplete.

We know a test log is always generated automatically by test tool after executing a test. TestComplete provides a standard IDE (Integrated Development Environment) interface for user to review the test log. We don’t have issue reviewing the test log within TestComplete regardless how big the test log is (Our test log record the test running for more than two days without stopping. The test log extension name is .tcLogs and the size of total test log is very big over 2.5 GB).

TestComplete also provides a function for the user to export the log into MHT format file via manually selecting each test log after complete a test or dynamically exporting the test log during test executing via command in test script. The MHT log file can be opened by IE. However, if the MHT file is a little big, such as >200M, the user could experience obvious performance issues. It takes a very long time for IE to open this file because IE loads all the contents into one page which could finally eat up all your system memory. I tried to find another tool to open the big MHT log file, but no luck. Even executing a small test (running for one to two hours), the MHT log file size can easily reach to 200M, if a screenshot is included, for example. Normally, each one round of our regression test runs for 30-48 hours. It is impossible for us to open the big MHT log file as the system runs out of memory and an error appears definitely after hours of waiting to just open this file by IE.

TestExecute is a small tool to just execute the test developed by TestComplete. But TestExecute only generates MHT log file rather than TCLOGS file. Obviously, we cannot open the MHT log file if it is very big. So we gave up using TestExectue and export MHT log file function in TestCompelete at all.

We have to buy six individual full license of TestComplete to run and review the test.

We have used this for nine years.

Overall, TestComplete is quite stable. But the only issue affecting me is if TestComplete is terminated unexpectedly while test is still running (such as, terminating TestComplete without stopping the test first, window restarting, or shutting down unexpectedly), user could lose all the logs.

There were no issues with scalability, but it may have a performance issue with expansion.

Technical support is excellent.

The initial setup was straightforward.

The product is becoming more and more expensive. There are two types of licenses: locked and float. Locked license save more, but can only be used in a physical machine. Float licenses can be run in virtual machines.

QTP, was very expensive when compared to TestComplete, many years ago.

This tool is very easy to use and very powerful. It has a short test development circle and good technical support. There is an expensive license, but it could save more, as testers may spend more time achieving a certain test goal if using an Open Source tool. You need to consider the testers' script experience and the company’s budget to choose the right tool.

The name mapping function to manage the objects across different web pages and the support for different web browsers are the most valuable features for us.

We use TestComplete to do the automation test on our product’s web UI. In total, over 500 test cases are written and the automation coverage is over 80%. This helps to reduce manual effort dramatically, and by using the same script, we can run the test on both IE and Google Chrome.

The scripts are saved as binary files on disk. This makes it troublesome to perform version control and merge it with the work that is done by other team members. I am not sure what the reason is that SmartBear chose to save the files as binary, but if they can change to plain text for the scripts in the future, that would be helpful.

I've used it for one year.

Some of the objects/parent objects are dynamically created in the web page. During the test, TestComplete will have problem to identify these kind of objects. This will cause a lot of false rejection and block the following test cases. Thus it requires some rework when running all the test cases as a batch.

I have not used any customer service/tech support yet. I cannot provide more comments on that.

TestComplete is the first tool that I use for automation on UI and I have not used any other tools yet.

It is pretty straightforward. Just by following the instructions given in the document and do the corresponding configuration on the web browsers that you want to test. However, we also encounter some problems such as the Chrome blocking users from proceeding because of an unsafe SSL connection, but we found a work around eventually.

We implement the automation by ourselves, not through any other vendor team. My suggestion is that you need to decide how to divide the automation work into corresponding components carefully. Otherwise, it will be hard for you to share the common scripts with other team members. Also, the code architecture design is very important as well.

I am not responsible with the licensing quotation/renewal, but the benefit that TestComplete brings to us is, apparently, acknowledged.