SmartBear TestComplete Reviews

Object-based recording, and data-driven testing.

Separation of data into Excel files made tests modifiable by QA personnel with limited development experience, and object-based recording kept maintenance to a minimum.

TestComplete's environment exposes a significant portion of its functionality to the command line, where execution details can be left to .bat files or Windows Scripts.

This allowed us to schedule execution of lengthy tests for nighttime and non-core hours, and to synchronize tests with Jenkins build releases.

This freed up QA assets to perform more specialized testing and reduced redundancy.

Native test result reporting does not provide overview reporting methods for tests that span multiple project suites. Features that allow for flagging a test as dependent on the result of another in multi-project hierarchies while maintaining name-mapping segregation would be much esteemed.

I've used it for approximately eight months.

8/10 - SmartBear representatives are surprisingly responsive, and they go out of their way to offer assistance.

8/10 - SmartBear representatives are surprisingly responsive, and they go out of their way to offer assistance.

The initial set-up was exceedingly simple. There is a silent mode option available during set-up, which is very convenient when deploying to multiple machines or remotely.

We implemented it through an in-house team.

We did not maximize our ROI until we put somebody full-time on our TestComplete endeavours. The tool does have a learning curve, and it wasn't until we had an in-house expert on it that we began to see the benefits of automated testing over traditional QA roles.

The licensing options for TestComplete both running a licensing server. This prevents users from running more sessions simultaneously than purchased keys. This can prove problematic if you want multiple developers writing or running tests at once, and prevents you from using your key while a distributed test is running.

For pricing, carefully consider how many machines you want running the software, rather than the number of developers.

We also evaluated another SmartBear product called SoapUI. The change to TestComplete occurred because we changed our target from web applications to desktop.

My advice in regards to implementation would be to choose carefully which tests to automate, specifically focusing on lengthy procedures, tasks that require looping, or places where you want to test against multiple data sets.

Additionally, I found it beneficial to prefix my keyword tests with a character and number to provide logical ordering instead of alphabetic.

I also found it beneficial to record "undo" steps with each keyword test; this allows each test to be more stand-alone and prevents your test from being dependent on the state the previous test left the application in.

Finally, I would suggesting limiting the number of test applications per test suite to prevent bloated name-mapping schemes.

• Initially support for flex/flash technologies, but since that project ended my primary use is Functional Automation against a web front-end built on AngularJS.
• I use VBScript for a majority of my tests; it is one of many language choices available when writing scripts.
• Out of the box support for desktop, mobile and web.

I am the only test engineer on the project, so time is limited in terms of automation development. Using TestComplete I was able to immediately (after installation) begin coding regression tests for available functionality.

My current suite of tests numbers in the hundreds, and each test involves several UI elements. Depending on how your organization defines a test you may call this one test or many. An example to quantify this, is as follows - Automated Build Viability (smoke test) execution:

• Number of tests = 75
• Automated Time to execute = 15 minutes
• Manual Time to execute = 360 minutes (approx.)
• Automation time saved = 345 minutes (5+ hours)
• 2300% improvement

I have substantially reduced the amount of time it takes for regression and build viability activities.

I pay for support and maintenance; having used the “support” functions through online forums, I can say there is room for improvement. I would like to have access directly to TestComplete developers as opposed to “users” who have some level of expertise with the platform.

I've used it for two years.

No issues encountered.

No issues encountered.

No issues encountered.

5/10, there are knowledgeable users in the forums, but I would like to go straight to the TestComplete staff with solution questions. That said, I have had very little need for product or development specific support.

Other tools were evaluated which supported Flex/Flash web apps, but we chose TestComplete for its rich feature set as well as Smartbear’s available suite of tools outside of Automated Testing.

It was very straightforward! You just need to install and begin, no special plugins or out of the ordinary system requirements were necessary.

We implemented it in-house. As with any automation platform, carefully planning the intended usage, understanding the technology under test and determining which features will be used are critical to success.

Given the cost was minimal, the best representation of ROI from my perspective is time saved using automated tests vs. manually execution.

My current suite of tests numbers in the hundreds, and each test involves several UI elements. Depending on how your organization defines a test you may call this one test or many. An example to quantify this, is as follows - Automated Build Viability (smoke test) execution:

• Number of tests = 75
• Automated Time to execute = 15 minutes
• Manual Time to execute = 360 minutes (approx.)
• Automation time saved = 345 minutes (5+ hours)
• 2300% improvement

Our recent purchase was a single node locked license for one user which was very inexpensive. That fit our project needs and the automation strategy. The solution resides on inexpensive laptop for development and execution.

The most valuable feature is the ability for a non-developer to develop intelligent, robust, data-driven tests.

TestComplete has greatly improved our organization functions by allowing us to develop, manage, and execute many forms of testing in one central repository. Being able to develop, manage, and execute JUnit, NUnit, PHPUnit, PyUnit, Selenium, functional, and nonfunctional tests has given our company great insight as to the health of our testing efforts.

I have used this solution for four years.

No issues.

No issues.

The level of technical support leaves a lot to be desired. The only way to get support is via email, therefore problems that could take a matter of minutes to solve by phone or web conference can sometimes take several days.

We previously used IBM Rational Functional Tester, Segue/Borland/Micro Focus SilkTest, HPE Mercury Quick Test Professional, and IBM Rational Robot.

We switched because of the ability for a non-developer to develop intelligent, robust, data-driven tests.

The initial setup was very straightforward and took very little time.

No advice. I did not deal with the licensing and pricing.

I did compare the tool to IBM Rational Functional Tester, Segue/Borland/Micro Focus SilkTest, and HPE Mercury Quick Test Professional.

TestComplete is a great product. It is a perfect fit for an organization that has a hard time finding testers with a development background and want to centralize the development, management, and execution of their tests.

I found the Keyword Test feature very valuable. Even with my limited programming knowledge, I was able to create automation tests using the Keyword Test feature. Also, the ability to create image based tests was extremely helpful, especially when automating tests for mobile devices.

TestComplete has enabled us to automate a lot of our tests for Android devices.

I've used it for three months.

I would rate it 10/10. Every time I had a question or issue I couldn’t figure out, they were right there to help.

Set-up was pretty straightforward.

We implemented it through an in-house team.

The two most valuable features are the manual script library (projects) and Test Execute.

I set up a manual test for new functions and hand it off to a tester (who uses Test Execute). Once the new function test is completed, it is added to regression for all future versions. This has allowed us to make significant product changes, new server deployments, and hosting changes with high confidence.

From my understanding, the product has changed significantly since v5 was released. I don’t know that it has necessarily improved, for my needs. If I worked in a different shop, then I might need an entirely different feature set compared to what I use now.

We've used it since May 2007.

No issues encountered.

No issues encountered.

No issues encountered.

I can't recall the last time I needed support.

I can't recall the last time I needed support.

Typically, I used my own method for automation (scripting tool, test data stored in SQL, results stored in SQL, result comparison to baseline with custom script). At the time I purchased TestComplete, the script recording tool wasn’t that great and I didn’t have the skills to write my own scripts. Over time, I used a different automation program and picked up the syntax for its scripting language. The product is currently called WinAutomation, though it had a different name at the time, that I can’t recall.

The initial setup was very straightforward. I installed the main app on my local workstation and test execute on five workstations available for testers. We use a shared network drive to store tests.

I installed everything myself. It was very straightforward. The real key is knowing what you need for your environment and to adapt the available tools.

It’s hard to sell testing tools, especially in a company that is owned by someone who thinks of software as a mule to pull his cart. It was a very tough sell and it is hard to quantify in dollars the money saved. However, I can say, with confidence, that there have been over one hundred bugs caught through regression since the software was purchased, which is a great track record for a team this size.

To me, the Object Browser has proven to be the most valuable feature of this product. This tab (and its associated tools) allows the test developer to locate objects within the application to be tested, and retrieve properties and methods that can be used in scripting. This saves a significant amount of time over referring to the application source code for such information. It also makes use of the recording feature periodically to rough out scripts for new tests. The recorded scripts are very fragile, but they can provide a good base for writing stable scripts.

While our test automation initiative is still in its early stages, TestComplete is allowing us to script and run complex business process tests against our desktop applications. Being able to create smaller modules that can be referenced by other scripts or keyword tests allows for a flexible and easy to maintain test suite.

A major part of the product that needs improvement is the licenser. This piece of the application is known to have issues installing that requires manual, command-line modifications to correct. While this doesn’t occur with all installations, it has been a frequent problem for me. The larger issue with the licenser is that it is incompatible with virtual machines. The applications themselves will run on physical or virtual machines, but they must call out to a physical server which controls the licenses for them.

The latest version of TestComplete has introduced support for Git, but my initial impression is that it still needs some work before it can replace my VCS tool.

The product has been in use at the company for about two years and I have been actively using it for about 8 months.

I haven’t experienced issues that I would attribute directly to the application. There have been some issues with timing and other common problems, but many of those have been corrected through modifications to the tests and the environment.

My experience with their technical support has been less than satisfactory, but all of my issues were resolved. Of the technicians I have worked with, two of them were very knowledgeable and one of them did little more than email knowledge base articles that had been ruled out in the original issue report. The latter tech had the additional problem of being about 12 hours out of sync so a full day was lost with every email exchange. I received prompt responses and quick resolutions from both of the knowledgeable technicians I have dealt with although one of them was rather gruff and left me feeling like I was bothering her. Overall, SmartBear needs to work on their customer service and technical support.

This application did not replace a previous automation effort.

When deploying new installations of TestComplete, I ran into multiple issues with the licensing engine. The first time I experienced the problem, I contacted support and was directed to a knowledge base article (which I had been unable to find while searching for the error message). I have since bookmarked the article and reference each time I need to do a new install. Once the licenser has installed properly, the configuration and licensing goes smoothly.

The implementation was performed in-house, and was quite easy despite the issues with the licenser. The best advice I can offer would be to plan out your testing environment and requirements before you begin. If you plan to use virtual machines for your test systems, make sure you also have a physical box to run the licenser that is dedicated to the lab. Do not attempt to use someone’s PC as it may not always be available when the test machine are trying to run.

We have not performed an ROI analysis.

SmartBear’s product licenses are reasonably priced and generally in line with similar companies and products.

I reviewed several products before deciding to continue our automation efforts using TestComplete. Among the products reviewed were Telerik TestStudio, Testing Anywhere, and Ranorex.

The artificial intelligence aspect is very useful for us.

Scaling to a cloud environment is very easy.

Scriptless approaches are very flexible compared to other scriptless automation tools.

The object identification for legacy applications that are very old can still be automated by TestComplete. The object identification is very powerful.

The stability has been good in the latest versions.

Technical support is quite helpful and they have a good community you can look for answers in as well.

The initial setup is pretty easy and it's quick to deploy.

We have something called Name Mapping in TestComplete it is the only Name Mapping that is available for TestComplete. If they allowed two Name Mappings, or run Name Mapping in run time, that would be ideal.

TestComplete has its own XPath Identifiers. It auto-identifies the XPath of the application. If that engine could better identify more XPaths automatically and make the process more flexible, that would be better. 

Code analysis would be helpful. If they could give us a code analysis feature, similar to something called Ranorex Coach (which will give us auto-suggestions of how to use certain things), that would be ideal. In Tricentis, they have a feature whenever we identify an element or automate an element, they give a unique identification element that the tool notes when we map. Here, we don't have a feature that can mention the unique object identification kind of stuff. We need to highlight it on screen and then we need to check it. 

I've used this product for more than eight years or so.  It's been a while. 

The solution is now very stable. Four or five versions previous to the latest, if I recall, in version 10 or 11, it had some issues. However, now it is highly stable. Currently, there's nothing to worry about in terms of the stability of the product.

The solution can be scaled across the cloud. It can be scaled using the automation framework.

Technical support is good. Either we can raise a technical support ticket with them, which gets answered in 24 hours or we can use the very good community that they have. Whenever we post a query, we are able to get the answer easily from the community. It's very helpful.

The installation is very simple. In just five minutes we can install it and we don't need to do any settings for resetting - such as, for example, how we do it for Selenium or other tools. We don't require much time to deploy it. In no more than five minutes it is installed and ready to go.

The price is moderate. It's not the cheapest or the most expensive if you compare it with other competitors. It's okay. 

My company is a partner of TestComplete. I am a senior test architect in our organization.

If a company has an ecosystem and they need to automate the web, desktop, or mobile, then TestComplete is the right tool. Even if they need enterprise application automation, it's very easy. On top of that, using a scriptless approach will save a lot of time. However, users are not limited to that. They can use Python, JavaScript, VBScript, or other kinds of options. You can import external Python libraries inside it, which will give additional capabilities to the tool. It's been very useful.

Overall, I would rate it at a nine out of ten.

Mainly we use the solution for when we need to install or deploying in different applications, if necessary. We use it for testing for the most part.

It's a good solution. It works in Windows Operating System.

The automation is very good.

We can test an application across many programs with ease. We can test, for example, on Internet Explorer, or Mozilla, or Google Chrome, et cetera. 

It's very helpful that, when you have a recording in one application, in one navigator, you can replay every step in different applications. You don't need to do any different testing in different programs, you only have to do one test and replicate it in different programs.

The initial setup is easy.

The solution has a very nice interface.

The one issue we have is that the tool was installed in a local machine. Now, it's more popular when you use tools that work in online environments. It would be better if it was available on the cloud. We'd like to access it on remote desktops. I don't want to have to install it on every machine.

It would be ideal if the solution could offer one subscription for time and one for life. Right now, when you buy the solution, you need to pay for one solution. You receive one set up and you install it and it's just in that one machine. It would be ideal if they could offer one subscription where you can connect to different machines with a group subscription. Or maybe they could offer different pricing. They need to be more flexible.

I've only used the solution for maybe one or two months at this point. It hasn't been too long. It's relatively new.

The solution is stable. We didn't run into any issues. It has been good so far.

The technical support is good. They are very responsive and helpful. We are quite satisfied with eh level of support they give us.

The initials setup is not complex at all. It's pretty straightforward. It's easy. A company shouldn't have any problems with deployment.

That said, it might be hard for a normal user. You need to do a few technical things. It helps to have a bit of knowledge on your side. Otherwise, you may run into issues.

You only need one person to set everything up.

We handled the deployment ourselves. We didn't need the assistance of an integrator or consultant.

The solution doesn't have a very flexible pricing model.

I didn't look at too many other options, however, this one had a special advantage in that it works on the Windows Operating System.

We're just customers and end-users.

I'd advise new users to take a look at what they need. They need to know if they need something that can test on desktop or on mobile, et cetera. This is a good solution if you need something that works with Windows, for example.

I would rate the solution eight out of ten.