SmartBear TestComplete Reviews

• High UI technologies coverage
• Direct access to object internal properties and methods

Faster feedback on any product changes and uncovered problems before full regression starts. That can save a lot of time.

Scripting languages should be extended to support more popular programming languages like Java, C#, Ruby, Python. Many other vendors have such support or moved into such a direction. e.g. SilkTest has OpenAgent since 2008 with support of Java, C#. HP recently published LeanFT which integrates with popular development environments (Eclipse, Visual Studio). IBM solutions were initially designed for Java. Squish supports multiple languages. So, TestComplete should have support of such programming languages apart from VBScript and JScript modifications.

About three years onwards.

TestComplete had problems during long hours runs. In some cases it could crash without leaving any logs. Also, if there were some unhanded exception it could drop the error message waiting for user input. This is not really acceptable for nightly runs and required human assistance. But this problem is rather language-specific

Customer Service:

I never had to speak to them.

Technical Support:

I never had to speak to them.

The setup is pretty easy.

We used a vendor team. My advice for implementation is only one, follow the best practices (some of them are available on the vendor site). They were formulated based on real experiences.

ROI was never calculated explicitly, but normally the ROI point was expected to be reached after three to five months of the project.

Mainly, when we did an evaluation and comparison, TestComplete had two big advantages -

• Good technology support
• Relatively low price in comparison to other vendor tools

1. Object Browser is the most powerful feature. This has helped me tear apart every single component of the application under test. It's an awesome feature.
2. Test Editor and Debugger: This has really made my life easy as I can trace memory and stack values very easily.
3. Data driven testing has been made very easy. I am comfortable using any sort of backend to support data driven testing such as Excel/databases/etc.
4. It supports using third party DLLs has given me an edge on several occasions.

Previously, we had to wait for the end-of-sprint to make a stable release. Now, rolling out a release is not a problem. As soon as a build is deployed, the scheduler quickly executes User Acceptance Tests, and verifies that the build is fit to be delivered to the clients.

1. Support for working with COM and ActiveX objects should be more powerful. Currently we have to drill down a lot to find solutions for certain tasks.
2. Cross browser testing is a bit complex when it comes to scripting. Developers have to take care with it care, or else issues may arise at deployment time.

Overall, I have used this for more than two years, including v1040. Currently I have been using it hands on for the last year.

There often are issues with web test execution. Web pages often do not load on the first attempt.

Customer Service:

Customer server is absolutely robust. The team respond within 12 hours.

Technical Support:

I never needed to use paid technical support as their forum is extremely helpful. The community members respond to issues within 24 hours. It’s really amazing that within the past twelve months, I have never had a technical question I asked remain unanswered.

I am an expert on the following:

• Selenium RC and Servers/Java - but this has multiple limitations
• IBM Rational Functional Tester - it's expensive, and not as powerful as TestComplete
• AppPerfect - is not in competition with any of the above, it’s pathetic

It was straightforward. Both the node locked and floating licenses were easily deployed, and now the whole team is using it.

At one point, we had to update the hardware of the servers where the licenses were deployed, and this was very easy. We just disabled the licenses from the account and deployed them onto the upgraded servers.

I did it myself in-house. It’s very simple. No rocket science needed. Just read the manuals and you can do it.

We are saving a lot in terms of client satisfaction. We are not yet in a phase where we have savings in terms of a reduced QA team, but our clients are happy that they get bug free software. Moreover, our testing team spends more time testing new features, and we are confident enough at build roll out time that all previously developed modules are stable.

To me, the Object Browser has proven to be the most valuable feature of this product. This tab (and its associated tools) allows the test developer to locate objects within the application to be tested, and retrieve properties and methods that can be used in scripting. This saves a significant amount of time over referring to the application source code for such information. It also makes use of the recording feature periodically to rough out scripts for new tests. The recorded scripts are very fragile, but they can provide a good base for writing stable scripts.

While our test automation initiative is still in its early stages, TestComplete is allowing us to script and run complex business process tests against our desktop applications. Being able to create smaller modules that can be referenced by other scripts or keyword tests allows for a flexible and easy to maintain test suite.

A major part of the product that needs improvement is the licenser. This piece of the application is known to have issues installing that requires manual, command-line modifications to correct. While this doesn’t occur with all installations, it has been a frequent problem for me. The larger issue with the licenser is that it is incompatible with virtual machines. The applications themselves will run on physical or virtual machines, but they must call out to a physical server which controls the licenses for them.

The latest version of TestComplete has introduced support for Git, but my initial impression is that it still needs some work before it can replace my VCS tool.

The product has been in use at the company for about two years and I have been actively using it for about 8 months.

I haven’t experienced issues that I would attribute directly to the application. There have been some issues with timing and other common problems, but many of those have been corrected through modifications to the tests and the environment.

My experience with their technical support has been less than satisfactory, but all of my issues were resolved. Of the technicians I have worked with, two of them were very knowledgeable and one of them did little more than email knowledge base articles that had been ruled out in the original issue report. The latter tech had the additional problem of being about 12 hours out of sync so a full day was lost with every email exchange. I received prompt responses and quick resolutions from both of the knowledgeable technicians I have dealt with although one of them was rather gruff and left me feeling like I was bothering her. Overall, SmartBear needs to work on their customer service and technical support.

This application did not replace a previous automation effort.

When deploying new installations of TestComplete, I ran into multiple issues with the licensing engine. The first time I experienced the problem, I contacted support and was directed to a knowledge base article (which I had been unable to find while searching for the error message). I have since bookmarked the article and reference each time I need to do a new install. Once the licenser has installed properly, the configuration and licensing goes smoothly.

The implementation was performed in-house, and was quite easy despite the issues with the licenser. The best advice I can offer would be to plan out your testing environment and requirements before you begin. If you plan to use virtual machines for your test systems, make sure you also have a physical box to run the licenser that is dedicated to the lab. Do not attempt to use someone’s PC as it may not always be available when the test machine are trying to run.

We have not performed an ROI analysis.

SmartBear’s product licenses are reasonably priced and generally in line with similar companies and products.

I reviewed several products before deciding to continue our automation efforts using TestComplete. Among the products reviewed were Telerik TestStudio, Testing Anywhere, and Ranorex.

• Dynamic Find methods - FindChild, FindAllChildren, and FindAll
• Keys method
• Project variables
• Time delay methods
• aqString methods
• Intelisense

Regressions and Smoke was done manually in the past, which has been replaced by Automated Testing.

The web testing framework of TestComplete is not very helpful for an Automation Engineer. It requires the same effort as Selenium, and. in most cases, Selenium proves to be a better testing tool for web-based testing.

Web Application Testing has the following issues:

• Events do not work on Modal Dialog
• Dynamic Object Capturing is too hard and takes too much time in finding the correct object
• Intellisense demos do not work on methods in the classes

Similarly, the mobile testing framework of TestComplete is not of much use either. I do not recommend anyone use this tool for web or mobile testing.

I have been a user of this tool for five years, including previous versions, on desktop, mobile and the web. I have completed a few major projects with thsi tool and have enhanced numerous test suites created with this tool.

There have been quite a few issues. Delay methods are not so dynamic in TestComplete, so if a screen hasn't loaded, and the delay is a static delay, then the test moves forward in script, but the previous operation has not been done, resulting in failure of the whole test one after another.

This also means that TestComplete and TestExecute work in a not so similar manner, causing TestExecute to be practically useless. This happens in the case of HTML5 based web application.

Customer Service: Level of customer services is satisfactory. Replies are swift and prompt.
Technical Support: Solutions are scarce. If there is no solution to a problem, the support people will tell you there is no way around it, and then you are stuck. The issues are not resolved in the next releases, mainly because these are not escalated on the backend.

SilkTest was being used previously with my previous employer. It was abandoned because it was hard to script and advanced features were not being added.

The previous solution I used was SilkTest, which was being used by my previous employer. It was abandoned because it was hard to script, and advanced features were not being added.

It was done through a vendor. It's better to get it done through vendors, as they are then responsible in the future.

It is not a good idea to choose this tool only if it is being used by others. In our case it is an expensive tool, and we should have exerted the same effort by using Selenium, and would have had better results. It appears that if you have a bif web application built in HTML5 and Jquery, and uses Divs and Modal dialogs, then avoid using TestComplete.