SmartBear TestComplete OverviewUNIXBusinessApplication

SmartBear TestComplete is the #4 ranked solution in top Regression Testing Tools, #7 ranked solution in top Test Automation Tools, and #8 ranked solution in top Functional Testing Tools. PeerSpot users give SmartBear TestComplete an average rating of 7.8 out of 10. SmartBear TestComplete is most commonly compared to Tricentis Tosca: SmartBear TestComplete vs Tricentis Tosca. SmartBear TestComplete is popular among the large enterprise segment, accounting for 64% of users researching this solution on PeerSpot. The top industry researching this solution are professionals from a computer software company, accounting for 22% of all views.
SmartBear TestComplete Buyer's Guide

Download the SmartBear TestComplete Buyer's Guide including reviews and more. Updated: November 2022

What is SmartBear TestComplete?

What is SmartBear TestComplete?

TestComplete is a reliable, sturdy automated testing platform created by SmartBear Software. SmartBear Software is a worldwide technological leader known for developing quality enterprise-class development and testing solutions.

TestComplete simplifies the process of creating tests for numerous applications, including, but not limited to; Desktop, Android, IOS, Web browsers, and Windows. Application tests can be manual, scripted, and even recorded by using keyword-driven or data-driven functionality. There are even additional options for error reporting and automated playback. The object repository is extremely accurate and is fully customizable. TestComplete can easily be used by experienced developers and even by manual novice testers to develop quality automated UI tests quickly.

TestComplete offers three different testing scenarios:

  • Desktop: Users can easily and quickly automate UI tests using today’s most popular desktop applications, such as; Windows, Java, Python,.Net, VBScript, and more.

  • Web: Users can effortlessly create renewable tests for all of today’s popular web applications, including JavaScript frameworks on 2000+ trusted browser and platform integrations.

  • Mobile: Users can safely build and automate serviceable UI tests on actual or virtual android or IOS devices, locally or in the cloud. Users can create code or codeless tests. TestComplete seamlessly integrates with many of today’s popular frameworks.

Key Features

  • Easily create automated UI tests: TestComplete offers scriptless Record and Replay or simple keyword-driven tests to quickly develop any type of UI test users may require. Tests can be recorded once, then replayed when needed across various applications on mobile, web, or desktop environments. TestComplete integrates with many different languages, such as Java, Python, C+, and more.

  • Keyword driven tests: Users can easily divide testing steps, actions, objects, and data with an integrated keyword-driven test structure. This makes it easy for every user to participate in the test automation process; there is no programming experience needed. Everything is made simple with easy-to-use point-and-click options.

  • Data driven tests: Easily distinguish data from test commands to keep administrative efforts simple. Users can improve overall coverage by running various automated mobile, desktop, or web UI tests.

  • Record and Replay: Users can reuse created automated tests across every environment as often as desired. This helps to expand overall test coverage and represents a huge cost and time savings.

Reviews from Real Users

Sandhiya T S., Sr Solutions Engineer at Lexington Soft, relates, “The record and replay aspects of the solution are quite useful for people. With them, you don't have to write any scripts. Basically, you can record your actions and play them back later. The initial setup is also very easy.”

Sai S R., Staff Test Architect at a tech services company, says, "The most valuable features of the SmartBear TestComplete are self-healing, they reduce the maintenance required. The different languages SmartBear TestComplete supports are good because some of our libraries are written in Python, JavaScript, and C#. It's very easy to put them all under one project and use them."

SmartBear TestComplete Customers

Cisco, J.P. Morgan, Boeing, McAfee, EMC, Intuit, and Thomson Reuters.

SmartBear TestComplete Video

Archived SmartBear TestComplete Reviews (more than two years old)

Filter by:
Filter Reviews
Industry
Loading...
Filter Unavailable
Company Size
Loading...
Filter Unavailable
Job Level
Loading...
Filter Unavailable
Rating
Loading...
Filter Unavailable
Considered
Loading...
Filter Unavailable
Order by:
Loading...
  • Date
  • Highest Rating
  • Lowest Rating
  • Review Length
Search:
Showingreviews based on the current filters. Reset all filters
PeerSpot user
QA Team Manager at MTS - Mer Telemanagement Solutions
Real User
Saves a lot of human resources by running full regression tests prior to each release.
Pros and Cons
  • "The database checkpoints detect problems which are difficult for a human resource to find."
  • "Customer service and technical support responsiveness are high. Everyone is very professional."
  • "Name Mapping feature should be clearer. Whenever I use it, I do not really know what will work and what will not work."
  • "Stability issues occurred only when connecting to the SourceSafe. Sometimes, after getting the latest version, the tool hangs and it should be reopened in order to recover."

What is our primary use case?

The primary use is to run regression tests on a call accounting system as a web application, which is running on a Windows operating system. The tests simulate user actions using UI and database verification.

How has it helped my organization?

The solution has saved a lot of human resources by running full regression tests prior to each release. In addition, during the years, many major bugs have been detected by the tool, and it has saved us the big expense of fixing problems after a release. Our confidence with the tool increased as releases were performed successfully, without any rejections from the field.

What is most valuable?

The database checkpoints detect problems which are difficult for a human resource to find. In addition, verification of UI items in all screens is also important task that consumes too
much manual resources.

What needs improvement?

Name Mapping feature should be clearer. Whenever I use it, I do not really know what will work and what will not work.

Buyer's Guide
SmartBear TestComplete
November 2022
Learn what your peers think about SmartBear TestComplete. Get advice and tips from experienced pros sharing their opinions. Updated: November 2022.
657,397 professionals have used our research since 2012.

For how long have I used the solution?

Eleven years.

What do I think about the stability of the solution?

Stability issues occurred only when connecting to the SourceSafe. Sometimes, after getting the latest version, the tool hangs and it should be reopened in order to recover.

What do I think about the scalability of the solution?

No issues.

How are customer service and support?

Customer service and technical support responsiveness are high. Everyone is very professional.

Which solution did I use previously and why did I switch?

We did not have a previous solution.

How was the initial setup?

The initial setup was straight forward. No issues at all during the setup stage.

What about the implementation team?

The implementation was in-house.

What was our ROI?

Our ROI is about $10,000 a year.

What's my experience with pricing, setup cost, and licensing?

The license price for a physical machine is cheap, and for virtual machine, it is very expensive.

Which other solutions did I evaluate?

We did not evaluate other solutions.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
PeerSpot user
Sr. Technical Manager - Testing Solutions at a tech services company with 1,001-5,000 employees
Real User
A flexible all-in-one testing solution
Pros and Cons
  • "TestComplete is simple, it's a very easy-to-use tool."

    What is most valuable?

    TestComplete is simple, it's a very easy-to-use tool. It offers credibility and value for the money. A basic license is roughly $4,000 and you can add flexible components on top of it. 

    If you want desktop testing, you can add-on a desktop license. If you want web testing, you're going to want to add-on a web license — there's a lot of flexibility, you just need to pay for what you use. There are no irritating subscription models.

    People are able to quickly use the platform and with a variety of scripting languages, including Python and JavaScript — all the modern scripting languages are supported. It doesn't just rely on VB script like UFT.

    It's very flexible and robust in that way. I have seen many of our clients quickly adopting the tool with all the scripting languages. 

    Recently, they've been building a lot of futuristic features, for example, AI Self-healing is one of the interesting features where they try to improve and cut down on maintenance by automatically correcting the arbiter. That's a really cool feature for keeping your object repositories up to date, and it can considerably bring down or control your maintenance costs to some extent, at least as far as the object repository goes.

    They also have some intelligent OCR features. They have a mini device cloud, for example, which allows you to run a testing tool and recently started supporting X spot. That actually goes well with selenium. You can reuse some of the scripts with other frameworks. They also acquired CucumberStudio a few years back. They have combined HipTest and Cucumber into one capability — CucumberStudio —, which is a great integration to TestComplete, that really becomes very seamless.

    What needs improvement?

    They're working on many features. Of course, the roadmap is not news to me, but yes, of course, they are working on different features.

    For how long have I used the solution?

    I have been using this solution for many years.

    What do I think about the stability of the solution?

    We have not faced any concerns with respect to the stability of the tool; at least we have not seen any major issues with the tool where it malfunctions or anything, never.

    What do I think about the scalability of the solution?

    It is a scalable solution. The beauty of TestComplete is that it's not tied to one scripting language. All the different scripting languages come with different abilities. You have different types of metrics and controls available with different scripting languages. That way we can drive our own framework. You can create many custom frameworks using TestComplete that will suit your organization. That's what we have been doing with many companies — it's an all-in-one solution.

    How are customer service and technical support?

    The technical support is excellent. We don't deal with them directly, but the feedback that I have received from different clients is that it's really excellent. They are always attentive —  That's a feeling I've always got from different clients.

    How was the initial setup?

    The initial setup is pretty straightforward. There's a good license server map for all those things — it's a fairly straightforward solution.

    What's my experience with pricing, setup cost, and licensing?

    The licensing costs are in the range of $1,000 to $3,000.

    What other advice do I have?

    We do a lot of consulting and training services for SmartBear. We try to educate the users in terms of the new features available in TestComplete so that they can do some smart automation. It's not just for automating some scenarios, you can optimize a lot of your effort.

    On a scale from one to ten, I would give this solution a rating of nine.

    Disclosure: My company has a business relationship with this vendor other than being a customer: Partner
    PeerSpot user
    Don Ingerson - PeerSpot reviewer
    Don IngersonQA Automation Engineer at Precise Circuits, Inc.
    ExpertTop 5Consultant

    Your review is well-written. Will TestComplete run on a locked computer?

    Buyer's Guide
    SmartBear TestComplete
    November 2022
    Learn what your peers think about SmartBear TestComplete. Get advice and tips from experienced pros sharing their opinions. Updated: November 2022.
    657,397 professionals have used our research since 2012.
    Sr. Consultant at a computer software company with 51-200 employees
    Real User
    Top 10
    Good technical support, but the cost of licensing is a little bit high
    Pros and Cons
    • "The most valuable features are the desktop and mobile modules."
    • "The licensing costs are a little bit high and should be reduced."

    What is most valuable?

    The most valuable features are the desktop and mobile modules.

    What needs improvement?

    The licensing costs are a little bit high and should be reduced.

    I would like the ability to automatically distribute web applications to all of the concurrent users.

    For how long have I used the solution?

    We have been using TestComplete for the past two years. 

    What do I think about the stability of the solution?

    The stability is good.

    How are customer service and technical support?

    The technical support is quite good.

    How was the initial setup?

    The initial setup is not too straightforward and not too complex. I would say that it's okay.

    What's my experience with pricing, setup cost, and licensing?

    The price varies depending on the plan that you choose. The option we chose was around $2,000 USD.

    Which other solutions did I evaluate?

    I am currently investigating the differences between TestComplete and TestLeft.

    What other advice do I have?

    TestComplete is a good product compared to other options that are on the market.

    I would rate this solution a seven out of ten.

    Which deployment model are you using for this solution?

    On-premises
    Disclosure: My company has a business relationship with this vendor other than being a customer: Partner
    PeerSpot user
    Raju YN - PeerSpot reviewer
    QA Architect at a computer software company with 5,001-10,000 employees
    Real User
    Easy set up and test creation but the test object repository needs improvement
    Pros and Cons
    • "The reporting is ready to use and doesn't require any setup."
    • "The test object repository needs to be improved. The hierarchy and the way we identify the objects in different applications, irrespective of technology, needs adjustments. The located and test objects are not as flexible compared to other commercial tools."

    What is our primary use case?

    We have what we call a UK tax related application. We are using the solution to automate some functional cases of the application. Use case wise, it's usually used more on the accounting side, which is the main purpose of the application. With the available functional testing of workflows, we are able to have accountability there.

    What is most valuable?

    The solution's most valuable aspect is how easy it is to create tests. For example, recording options.

    The reporting is ready to use and doesn't require any setup.

    We can easily create test suites without much trouble. For example, by using the UFP test, we can directly create a test suite. 

    What needs improvement?

    The test object repository needs to be improved. The hierarchy and the way we identify the objects in different applications, irrespective of technology, needs adjustments. The located and test objects are not as flexible compared to other commercial tools.

    For how long have I used the solution?

    I have three or four months of experience on the solution. I am an architect on technical optics. Different teams work on the project as well. I mentor them sometimes as part of my job. As far as hands-on experience goes, I can say I've only actually been using it for a few months. It hasn't been more than that.

    Our company as a whole, on the other hand, has been using the solution for the better part of four or five years now.

    What do I think about the stability of the solution?

    We haven't had any problems with stability. It's reliable. We haven't witnessed any bugs or glitches. It doesn't seem to crash on us or to freeze.

    What do I think about the scalability of the solution?

    The solution is scalable. There are no issues in that sense. If a company needs to expand out the solution they can do so easily. We can add as many artifacts as we want.

    I'm not sure if we have plans to increase usage in the future.

    How are customer service and technical support?

    I've never used technical support, so I can't speak to how good technical support is.

    How was the initial setup?

    The initial set up was not complex. I can say it's straightforward due to the fact that we have started out relatively small. We have gradually developed it. We continue to build it out all the time. That made it simple. This was important for us, as our team didn't have too much experience with the solution. 

    The deployment was handled by my IT team. I did not handle the deployment personally.

    What about the implementation team?

    The implementation was handled in house by our own teams.

    What's my experience with pricing, setup cost, and licensing?

    The pricing is reasonable. It's not too expensive as a solution. We're satisfied with the billing so far.

    What other advice do I have?

    I'd rate the solution seven out of ten.

    If you compare it with competitors, the object identification needs to be improved. It also needs to ensure that going forward it supports new technologies. We are seeing a lot of new technologies coming up. It's important that test companies support these new advancements. If they do, then it will give testers a reason to try them out and hopefully adopt the solution. It's a win-win.

    The solution is also great for language flexibility. It's important to support a number of languages to help increase user adoption.

    I'd recommend this solution to other organizations. It's one of three I'd recommend for sure based on the pricing and feature offering it has. Of course, this is just a recommendation n terms of a commercial tool. Open-source tool recommendations are a whole other question.

    Which deployment model are you using for this solution?

    On-premises
    Disclosure: I am a real user, and this review is based on my own experience and opinions.
    PeerSpot user
    VolodymyrNabok - PeerSpot reviewer
    Quality Assurance Engineer at a computer software company with 201-500 employees
    Real User
    Works perfectly with the CUTE application and great for Windows OS, but not other operating systems
    Pros and Cons
    • "Complete works perfectly with CUTE. That includes all dialogues, right-click menus, or system dialogues, etc., which are handled well."
    • "The solution needs to extend the possibilities so that we can test on other operating systems, platforms and publications for Android as well as iOS."

    What is our primary use case?

    I'm currently are trying to apply it to our software application.

    We use it for testing Windows applications for CUTE, based on the CUTE framework. For now, that's all we use it for because Complete does not suggest any other options that are appropriate for us. By that, I mean it doesn't seem to work with Mac OS, Android OS, and iOS.

    What is most valuable?

    I only have experience with Windows, so I find all the best features are for that operating system.

    Complete works perfectly with CUTE. That includes all dialogues, right-click menus, or system dialogues, etc., which are handled well.

    The solution has a great feature called macro recording. With it, I can make it into a macro in a few of the languages Complete suggests. This is really useful for me, personally.

    What needs improvement?

    The solution needs to extend the possibilities so that we can test on other operating systems, platforms and publications for Android as well as iOS.

    Right now, Complete can test only on native Android and native iOS applications.

    For how long have I used the solution?

    I've been using the solution for about a month. 

    What do I think about the stability of the solution?

    This tool is quite stable. I had only one crash, and I sent a report for this crash to technical support. Everything else seems to work perfectly. Aside from the one issue, we haven't had any other problems.

    What do I think about the scalability of the solution?

    I don't have any experience, unfortunately, with scalability. We use just one instance on one machine so I've never attempted to scale the solution. There's just one user on the platform right now.

    I am not sure if our organization will extend its usage in the future. We have a strong need to make this work on other platforms. We may switch to a different tool.

    How are customer service and technical support?

    I do have some experience with technical support. I've found a few answers to my issues on Smart BF5. I've also had assistance with the support engineers. So far, I've been satisfied with the level of support I have been able to receive. 

    Which solution did I use previously and why did I switch?

    I have used other solutions, including ALTA. It has good UI, but I don't know if you are really able to directly compare it to this solution. 

    How was the initial setup?

    The initial setup is straightforward. We found it simple and not too complex. Our team didn't run into any issues.

    For us, deployment took two to three days.

    What about the implementation team?

    We didn't use an integrator, reseller or consultant for the deployment. We handled the entire process ourselves.

    What's my experience with pricing, setup cost, and licensing?

    I'm not sure if there are licensing costs involved in the solution. We simply bought the product outright and started using it.

    What other advice do I have?

    I'm currently working with the latest version of the solution.

    My advice to others is to just decide if this tool is usable for your requirements. I spent a lot of time developing some tests, but then I understood that we actually needed more platforms. That's why we will switch to another tool. That's also why it's important to check your organization's requirements. Otherwise, like us, you may need to switch.

    I'd rate the solution seven out of ten.

    Disclosure: I am a real user, and this review is based on my own experience and opinions.
    PeerSpot user
    QA Head at a computer software company with 1,001-5,000 employees
    Real User
    Offers valuable regression testing and scalable with a straightforward setup
    Pros and Cons
    • "The solution is great as a record and playback tool. It also has valuable regression testing."
    • "The solution needs more training manuals or some form of online forum for learning. It needs more documentation."

    What is our primary use case?

    We primarily use the solution in our data center as an application.

    What is most valuable?

    The solution is great as a record and playback tool. It also has valuable regression testing.

    What needs improvement?

    The solution needs more training manuals or some form of online forum for learning. It needs more documentation. 

    Some sort of troubleshooting portal should be available for users.

    The solution could be faster. It would be good to increase the reusability of the code so that rerun time can be saved. It would be ideal to be able to run multiple scenarios at the same time.

    Right now, in terms of recording and scripting the solution is not user-friendly. They need to improve on this.

    For how long have I used the solution?

    I've been using the solution for one and a half years.

    What do I think about the stability of the solution?

    The stability needs a bit of improvement. On the open-source version, you're not really able to check stability like you can on the paid version.

    What do I think about the scalability of the solution?

    The solution is definitely scalable. We have about ten licenses. We may increase usage in the future.

    How are customer service and technical support?

    Technical support could be better. The exact solutions we need should be provided in a more timely manner.

    Which solution did I use previously and why did I switch?

    We didn't previously use a different solution.

    How was the initial setup?

    The initial setup was straightforward. It's a straightforward installation. Deployment took about half a day.

    What about the implementation team?

    We handled the implementation ourselves.

    What other advice do I have?

    The solution is pretty good. Other options to use are TestComplete, and, for negative scenarios, Katalon Studios.

    I'd rate the solution eight out of ten.

    Disclosure: I am a real user, and this review is based on my own experience and opinions.
    PeerSpot user
    Quality Assurance Engineer at a tech vendor with 201-500 employees
    Real User
    Easy initial setup, good stability and great script recording features
    Pros and Cons
    • "The solution helps improve the stability of our product. It also decreases the work of our manual quality assurance engineers."
    • "The solution needs Mac OS support. Right now, the solution has only been developed to accommodate Windows OS."

    What is our primary use case?

    We primarily use the solution to automate the user interface.

    How has it helped my organization?

    The solution helps improve the stability of our product. It also decreases the work of our manual quality assurance engineers.

    What is most valuable?

    The most valuable features are the script recording and the object spy. The solution has the ability to structure text well using different modules.

    What needs improvement?

    The solution needs Mac OS support. Right now, the solution has only been developed to accommodate Windows OS.

    For how long have I used the solution?

    I've been using the solution for one year.

    What do I think about the stability of the solution?

    The solution has impressive stability.

    Which solution did I use previously and why did I switch?

    There are some unsolved issues working with control drop downs that need to be resolved.

    How was the initial setup?

    The initial setup is easy. You just have to click and go. It was very fast for me on my desktop, however, I cannot speak to how fast it was on the production server.

    What's my experience with pricing, setup cost, and licensing?

    Currently, we're evaluating the product, so we have not yet bought a license.

    Which other solutions did I evaluate?

    Right now, we're in the process of evaluating this product and Squish.

    What other advice do I have?

    I'm using the desktop version of the solution.

    Right now, I'm working with TestComplete, and will compare it to another product called Squish. Once I've compared the two, I'll make recommendations to my customers and our company about whether or not to buy the complete solution.

    I'd rate the solution eight out of ten. I'd rate it higher if the solution offered the ability to test on Mac OS.

    Disclosure: I am a real user, and this review is based on my own experience and opinions.
    PeerSpot user
    Charles Netshivhera - PeerSpot reviewer
    Integration Specialist at ABSA
    Real User
    Good regressions tool, SoapUI tools, and cross-browser capabilities
    Pros and Cons
    • "The most valuable feature of this solution is regression testing tools."
    • "The artificial intelligence needs to be improved."

    What is our primary use case?

    I use this solution for functionality testing, API testing, and performance testing.

    We are moving away from this solution to use Eggplant Functional.

    What is most valuable?

    The most valuable feature of this solution is regression testing tools.

    The SoapUI tools are very good, as well as cross-browser capabilities.

    The GUI is very good.

    What needs improvement?

    There is a problem with usability because the speed decreases, which could be an issue with scalability because of too many hits on the site. It depends on how distributed their systems are, and how well they can handle multiple connections.

    The artificial intelligence needs to be improved.

    For how long have I used the solution?

    I have been using this solution for between six and twelve months.

    What do I think about the stability of the solution?

    This is a good product in terms of stability.

    How are customer service and technical support?

    The technical support for this solution is always available. They have an online chat that is quick.

    How was the initial setup?

    The initial setup of this solution was quite smooth. It was not complicated and within a week it was working. It did not take long.

    What about the implementation team?

    We performed the implementation in-house.

    What's my experience with pricing, setup cost, and licensing?

    This is a pay-per-use service that is not expensive, and cost-efficient if you have a small team.

    Which other solutions did I evaluate?

    We have also been using Eggplant Functional, and the decision has been made to move forward with that solution, instead of SmartBear. It is not for technical reasons. I also find Eggplant to be a little bit pricey.

    What other advice do I have?

    This product is quite mature, able to compete with other products in the market, and I would highly recommend it. Overall, we're very pleased with the implementation.

    My advice to anybody who is considering this solution is to do a PoC and try this solution out. They can always reach out to me for help, and I can assist.

    The biggest lesson that I have learned from using this solution is that there is a lot of hard work going on behind the scenes with this tool, to make each customer's journey easier. This tool is simple to implement and easy to use. If you don't have much time to do a lot of reading then it is still easy to take each case and adopt it quickly.

    I would rate this solution a nine out of ten.

    Disclosure: My company has a business relationship with this vendor other than being a customer: Partner.
    PeerSpot user
    Consultant at a tech services company with 1-10 employees
    Consultant
    Stable and Integrates well with Azure DevOps
    Pros and Cons
    • "The most valuable feature is the integration with Azure DevOps."
    • "What is currently missing from this solution is better support for mobile testing."

    What is our primary use case?

    We use this solution for test automation. A few months ago we began working on a project, and this solution is part of an evaluation phase. We are trying to include the whole suite from SmartBear including HipTest and SoapUI Pro.

    Our primary use case is a web solution that we are trying to get working on the mobile solution, based on Flutter. All of our apps are built on Flutter, which is still very new for us. There is no support right now, so we are working closely with SmartBear TestComplete to integrate it.

    We have an on-premises deployment.

    What is most valuable?

    The most valuable feature is the integration with Azure DevOps. That has been, by far, the best thing for us. We can use it to integrate everything that we are running in the test automation tool, which is linked directly to the test cases. This is the only tool on the marked at actually has this functionality, and it has only come about since 14.2, the latest version.

    What needs improvement?

    What is currently missing from this solution is better support for mobile testing. This is what we are currently negotiating with them for.

    Specifically, we would like to see Flutter support in the next release of this solution.

    For how long have I used the solution?

    We have been using this solution for a few months.

    What do I think about the stability of the solution?

    From my experience at this point, I would say that this solution is stable. Even with the earlier versions, as long as you set it up properly, it was stable.

    What do I think about the scalability of the solution?

    In terms of scalability, I think that it works pretty well if you have agents.

    How are customer service and technical support?

    We have not spoken directly with technical support. Currently, we are talking to the product manager.

    How was the initial setup?

    We are using the SmartBear suite, which meant that setting up TestComplete was pretty straightforward. As we are still evaluating, it has not been fully deployed.

    What other advice do I have?

    We haven't fully evaluated this solution, although I know that they have made a lot of updates since I worked on it the last time. The results of this evaluation will set the standard for other projects. We are creating the basis for everything. As things progress, it will be me creating the test, but it will be executed on several machines.

    I would rate this solution an eight out of ten.

    Disclosure: I am a real user, and this review is based on my own experience and opinions.
    PeerSpot user
    Deva Veluchamy - PeerSpot reviewer
    Test Automation Lead/Consultant at Aspire Systems
    Real User
    It supports various and different versions of browsers in web testing, but it's slightly unstable with longer executions of automation scripts.
    Pros and Cons
    • "Test items, project variables helps in managing automation suite and scheduling execution."
    • "Increased performance with less memory and CPU usage."

    What is our primary use case?

    Evaluated for automating regression suite for windows desktop application and found testcomplete suits well in the aspect of identifying most of the objects and in few other parameters.

    How has it helped my organization?

    Good test coverage through automation and provides unique solutions to the most of automation challenges (e.g. comparison of images).

    What is most valuable?

    - Object spy eases the object identification method.

    - TextObject recognition feature of this product exposes most of the objects to perform the actions.

    - Test items, project variables helps in managing automation suite and scheduling execution.

    - Jenkins integration to schedule executions.

    - Integrated with selenium webdriver.

    - Conversion of JScript project to Javascript project

    - Integrated to Environment manager

    - Support to read text from PDF

    What needs improvement?

    - Increased performance with less memory and cpu usage.

    - Supports various and different versions of browsers in web testing.

    - Support to Windows mobile application. (Native iOS and Android apps are supported currently.)

    For how long have I used the solution?

    More than five years.

    How is customer service and technical support?

    Customer Service:

    Customer service and technical support from SmartBear is more responsive. Active SmartBear forums are available to get clarification.

    Technical Support:

    Customer service and technical support from SmartBear is more responsive. Active SmartBear forums are available to get clarification.

    What about the implementation team?

    In-house team is used for implementing framework for automation through TestComplete. Tool must be explored completely and knowing its unique features is more important before implementation as it has lot of inbuilt features.

    What's my experience with pricing, setup cost, and licensing?

    TestComplete now have come up with three modules (Web, Desktop & Mobile), so based on the type of product for automation, it is adequate to purchase the required module. Node-locked and floating license are available which could be decided based on the team composition and work strategy. Current license pricing is available at http://smartbear.com/product/testcomplete/pricing/

    Disclosure: I am a real user, and this review is based on my own experience and opinions.
    PeerSpot user
    Marvin Reekie - PeerSpot reviewer
    Marvin ReekieQuality Assurance Analyst at a legal firm with 1,001-5,000 employees
    Real User

    In the process of evaluating tools to help improve the turn around for desktop testing. Looking at Testcomplete and Ranorex at this time. Still in the decision making process.

    See all 2 comments
    Test Automation Specialist at a tech vendor with 201-500 employees
    Real User
    The tool has the ability for a non-developer to develop intelligent, robust, data-driven tests

    What is most valuable?

    The most valuable feature is the ability for a non-developer to develop intelligent, robust, data-driven tests.

    How has it helped my organization?

    TestComplete has greatly improved our organization functions by allowing us to develop, manage, and execute many forms of testing in one central repository. Being able to develop, manage, and execute JUnit, NUnit, PHPUnit, PyUnit, Selenium, functional, and nonfunctional tests has given our company great insight as to the health of our testing efforts.

    For how long have I used the solution?

    I have used this solution for four years.

    What do I think about the stability of the solution?

    No issues.

    What do I think about the scalability of the solution?

    No issues.

    How are customer service and technical support?

    The level of technical support leaves a lot to be desired. The only way to get support is via email, therefore problems that could take a matter of minutes to solve by phone or web conference can sometimes take several days.

    Which solution did I use previously and why did I switch?

    We previously used IBM Rational Functional Tester, Segue/Borland/Micro Focus SilkTest, HPE Mercury Quick Test Professional, and IBM Rational Robot.

    We switched because of the ability for a non-developer to develop intelligent, robust, data-driven tests.

    How was the initial setup?

    The initial setup was very straightforward and took very little time.

    What's my experience with pricing, setup cost, and licensing?

    No advice. I did not deal with the licensing and pricing.

    Which other solutions did I evaluate?

    I did compare the tool to IBM Rational Functional Tester, Segue/Borland/Micro Focus SilkTest, and HPE Mercury Quick Test Professional.

    What other advice do I have?

    TestComplete is a great product. It is a perfect fit for an organization that has a hard time finding testers with a development background and want to centralize the development, management, and execution of their tests.

    Disclosure: I am a real user, and this review is based on my own experience and opinions.
    PeerSpot user
    it_user744705 - PeerSpot reviewer
    SQA Team Lead at a tech vendor with 51-200 employees
    Vendor
    Advantages include Name Mapping and finding objects at run time, although it has memory leaks

    What is most valuable?

    Name Mapping and finding objects at run time. I have no issue because its IDE is simple to use.

    How has it helped my organization?

    • License is less expensive than the other products available in the market.
    • I have automated the regression test suite of my company's product.

    What needs improvement?

    It has memory leaks. I have seven complete test projects, all of them loaded in one project suite. I don't know why memory increases when I just write the code and check-in/out files from the VSTS.

    For how long have I used the solution?

    For last five years.

    What do I think about the stability of the solution?

    Often crashes with large projects. File check-in takes too much time in TFS.

    What do I think about the scalability of the solution?

    No idea.

    How are customer service and technical support?

    Community is very helpful.

    Which solution did I use previously and why did I switch?

    I didn't switch from another solution.

    How was the initial setup?

    I would say straightforward, and really easy to work on TestComplete with its simple IDE.

    What's my experience with pricing, setup cost, and licensing?

    They are changing the licensing structure which is really bad. TestComplete 11 supports all TestComplete modules in one license. TestComplete 12 has separate licenses for each module, Web, Desktop, etc. I hate the new licensing structure.

    Which other solutions did I evaluate?

    I have no idea.

    What other advice do I have?

    It's a really good product with minimal issues.

    Disclosure: I am a real user, and this review is based on my own experience and opinions.
    PeerSpot user
    PeerSpot user
    Analista de TI Sênior/Teste de Software at a tech vendor with 1,001-5,000 employees
    Vendor
    Provides easy recording, understanding, and use without too much programming

    What is most valuable?

    It has easy recording, understanding, and use without too much programming.

    What needs improvement?

    Should improve parts for desktop products.

    For how long have I used the solution?

    I have used it for four years and still use it today.

    What do I think about the scalability of the solution?

    Yes, some.

    How are customer service and technical support?

    Good, I only needed it once.

    Which solution did I use previously and why did I switch?

    Yes, I had already used it. I changed because I did not have any solutions as close to the real thing as this. Free tools mostly do not resolve the problem.

    How was the initial setup?

    The first was difficult only in learning how to do it. Going forward, it has been easy and intuitive. We scale projects the best way for the company as we find that the simpler way to do it.

    What's my experience with pricing, setup cost, and licensing?

    It is expensive, and it is difficult to acquire living in Brazil. They should facilitate this. There are few representatives for this solution with little interest in selling it here.

    Which other solutions did I evaluate?

    Yes, JMeter.

    What other advice do I have?

    It is a great product. Just get advice before installation and everything will be much easier for you. I strongly recommend it.

    Disclosure: I am a real user, and this review is based on my own experience and opinions.
    PeerSpot user
    it_user796722 - PeerSpot reviewer
    it_user796722Senior Staff Engineer at a consultancy with 501-1,000 employees
    Real User

    May I know what scalability issues you've faced.

    it_user372528 - PeerSpot reviewer
    Senior QA & Test Manager, Head of Test Automation at a tech services company with 501-1,000 employees
    Consultant
    The automated test farm of 20 virtual machines for execution, 20 TestComplete licenses and 20 automated testers are doing the job of 100 manual testers.
    Pros and Cons
    • "Selenium integration."
    • "Error handling features in the tool are a little limited."

    What is most valuable?

    • SoapUI/ReadyAPI integration
    • Selenium integration
    • We run all of tests using TestExecute, this is just the executer for all TestComplete tests – command line version so can be integrated with any CI tools
    • We used to these tests seamlessly on Jenkins (TestComplete has a plugin for Jenkins)
    • The best part is TestComplete gives choice of selecting any of the six languages it supports
    • List of checkpoints available in TestComplete is one of the strongest available in the market
    • We also make heavy use of version control integration and JIRA integration
    • Data driven testing feature of this tool is one of the best

    How has it helped my organization?

    We did a POC for number of tools in past and eventually decided on TestComplete. We started with one tester and one test which we used to trigger manually. As the product matured, it added cross browser, mobile testing functionality, integration with Selenium/ SoapUI and Jenkins plugins, so we started using it more and more. Currently, the automated test team size is 20 and we run hundreds of tests automatically at the end of build process without any human intervention. For us, the automated test farm of 20 virtual machines for execution, 20 TestComplete licenses and 20 automated testers are doing the job of 100 manual testers. The ROI has improved significantly and all the regression tests for our product are automated. We have plans of using TestComplete in the test driven development approach where developers can make use of our tests as part of their unit testing and I am sure we are going to achieve a lot more from this tool in coming years.

    What needs improvement?

    • Error handling features in the tool are a little limited
    • The tool currently is Windows only, would like to see it being implemented for Mac and Linux
    • Once the tests are created in one languages, you can’t change it use other languages

    What was my experience with deployment of the solution?

    There have been no issues with the deployment.

    What do I think about the stability of the solution?

    The tool sometimes seems a little unstable and crashes sometimes on Windows 10.

    What do I think about the scalability of the solution?

    There are no issues with the scalability.

    How are customer service and technical support?

    Customer Service:

    Average

    Technical Support:

    Good

    Which solution did I use previously and why did I switch?

    No. We did a POC on multiple tools and TestComplete was our first selection

    How was the initial setup?

    StraightForward

    What about the implementation team?

    In-house

    Which other solutions did I evaluate?

    QTP, TOSCA, RATIONAL, RANOREX

    What other advice do I have?

    The tool has great capability but also has great potential to demoralize if you implement it in incorrectly. Automated tests complement manual testing and doesn’t remove manual tests completely so I would suggest keep your manual tests, they will come handy as reference when the automated test fails or needs an update. I would suggest following before you start implementing the tool in you automated test environment

    • Choose the language to be used and stick to it. TestComplete supports six languages (VBScript, JScript, Python, DelphiScript, C++Script, C#Script)
    • Choose the naming convention for objects in the name-map and keyword tests so that you can make use of these tests in several projects
    • Always try to reuse the tests with parameterised values wherever possible rather than duplicating the tests
    • Version control the tests so that you can maintain multiple versions for multiple branches of your tests
    • Define and document a standard process for automation and communication
    • Execute the tests frequently so that you can get the best out of the tests and defects are identified as early as possible
    Disclosure: I am a real user, and this review is based on my own experience and opinions.
    PeerSpot user
    it_user292632 - PeerSpot reviewer
    QA Automation Lead at a tech services company with 501-1,000 employees
    Consultant
    Our automation engineers can create tests more easily, but the code editor gives poorly formatted code.
    Pros and Cons
    • "It's cross platform automation capabilities specially ranging across web, UNIX (via putty), and other systems."
    • "The code editor, though following eclipse-style, is still a work in progress and gives a very poorly formatted code once viewed via other editing tools."

    What is most valuable?

    It's cross platform automation capabilities specially ranging across web, UNIX (via putty), and other systems.

    How has it helped my organization?

    TestComplete replaced QTP as the preferred choice of tool for the organization. It is much faster, works better across technologies (esp. Flex based UI) and is better compatible with newer technologies directly out of the box. We could have our automation engineers create tests more easily. Also, we were able to set-up lab machines to enable distributed runs for more applications in a shared environment.

    What needs improvement?

    The code editor, though following eclipse-style, is still a work in progress and gives a very poorly formatted code once viewed via other editing tools like Notepad++. Performance is another aspect which can be improved.

    For how long have I used the solution?

    I've used it for one and a half years.

    What do I think about the scalability of the solution?

    TestComplete was fairly easy to scale once we had the licenses in place for the organization.

    How are customer service and technical support?

    We used tech support for some specific third-party grids we had to automate. The support was average and we ended up creating our own automation solution for that piece of automation.

    Which solution did I use previously and why did I switch?

    We previously used QTP, and later moved to TestComplete as it provided more out of the box support for newer technologies like Flex and AIR.

    How was the initial setup?

    Initial setup was very straightforward. We did face some hiccups in license procurement however, once licenses were procured the process from there was quite smooth.

    What about the implementation team?

    We had a mixed team. The implementation was smooth overall and requires a few skilled automation experts to oversee the transition/initial implementation.

    What was our ROI?

    We achieved ROI in eight months from the start of implementation. Get a good automation architect to implement a good ROI directed framework. It is very easy to lose direction during a mmigration.

    Which other solutions did I evaluate?

    • QTP
    • Selenium
    Disclosure: I am a real user, and this review is based on my own experience and opinions.
    PeerSpot user
    it_user293895 - PeerSpot reviewer
    Software Developer at a retailer with 1,001-5,000 employees
    Real User
    Our biggest problem was the timing between input and responses, but a whole suite of tests could be run automatically.
    Pros and Cons
    • "The ability to run a whole suite of tests automatically (which we did overnight)."
    • "We were testing handheld barcode scanners running WindowsCE with many menus of warehouse functions, and our biggest problem was the timing between input and responses."

    What is most valuable?

    The features of TestComplete that were most valuable to me were:

    • The ability to run a whole suite of tests automatically (which we did overnight)
    • The ability to create test scripts that were easy to modify;
    • The ability to easily review a log of software errors, if any

    How has it helped my organization?

    It would have been a huge task to have to run the test scripts manually, probably 12 hours straight. We were able to run tests on daily promotions of software so all software was tested every night, making any bugs instantly visible the next morning. It gave us quick confirmation that software changes worked, and without breaking any existing software.

    What needs improvement?

    We were testing handheld barcode scanners running WindowsCE with many menus of warehouse functions, and our biggest problem was the timing between input and responses. This was because sometimes the Windows PC feeding the scanner script data was faster than the scanner could process it, so we had to constantly tweek the wait times so the script wouldn’t bomb out. I have since used software with a “wait for” function that would wait for a response, but if TestComplete did, we didn’t know about it.

    One of the main reasons I was hired was to use TestComplete with green screen applications, which was possible, but not easy. You had to know the exact column and row position of every text line on the screen so you could verify the string being tested and analyze exactly where and what the response would be.

    For how long have I used the solution?

    I used TestComplete as a QA Automation consultant for about six months.

    What was my experience with deployment of the solution?

    No

    What do I think about the stability of the solution?

    No

    What do I think about the scalability of the solution?

    No

    How are customer service and technical support?

    Customer Service:

    We never needed customer service.

    Technical Support:

    We never needed technical support.

    Which solution did I use previously and why did I switch?

    Nothing previous to TestComplete.

    How was the initial setup?

    That happened before I was hired, so I don't know.

    What about the implementation team?

    TestComplete was implemented with the help of contract software developers who created the test scripts, which was invaluable to be able to use it for testing, and to use them as templates so we could copy and modify to make new scripts.

    What was our ROI?

    unknown.

    What's my experience with pricing, setup cost, and licensing?

    unknown.

    Which other solutions did I evaluate?

    TestComplete was installed before I was hired, so I know nothing about this.

    Disclosure: I am a real user, and this review is based on my own experience and opinions.
    PeerSpot user
    Julio De Lima - PeerSpot reviewer
    Principal QA Engineer at Capco
    Real User
    Supports Desktop and application testing and enables centralizing all scripts in one tool
    Pros and Cons
    • "TestComplete fits almost perfectly with a large amount of stacks, such as Delphi, C#, Java and web applications."
    • "TestComplete gives support to do requests to a SOAP web service but has no support to do HTTP requests on Restful services."

    What is most valuable?

    Desktop and web application support. TestComplete fits almost perfectly with a large amount of stacks, such as Delphi, C#, Java and web applications. It's an amazing feature for companies that want to automate UI tests on each application built in-house.

    How has it helped my organization?

    Before using TestComplete on our projects, we used to use a lot of tools to automate our applications, such as Sikuli or Java Robot to automate desktop apps, and Selenium WebDriver for web apps. After starting to use TestComplete, we were able to centralize all scripts in only one tool and technology.

    What needs improvement?

    TestComplete gives support to do requests to a SOAP web service but has no support to do HTTP requests on Restful services. In a microservice world, this is a big flaw. Another thing is that the cross-browser support has a lot of different traits between browsers. It should be improved.

    For how long have I used the solution?

    Five years.

    What do I think about the stability of the solution?

    Sometimes the app crashes during test execution based on the amount of code that is running. It's recommended that you modularize scripts, but that is not a justification for not being more stable.

    What do I think about the scalability of the solution?

    TestComplete has a test executor app that can be used in a distributed test execution environment. The problem is that this is a paid product. Thus, it was pretty expensive to scale this architecture.

    How are customer service and technical support?

    Eight out of 10.

    Which solution did I use previously and why did I switch?

    I used to use HPE QTP but TestComplete has the same features, low price, and support for handling a lot of stacks. In addition, I don't need to use hundreds of plugins.

    How was the initial setup?

    To start working with TestComplete, we only needed to install (next, next, finish flow) and then start using it. There are some configurations to do to help increase efficiency, but I do not consider that more than a nice-to-have.

    What's my experience with pricing, setup cost, and licensing?

    Buy modules on demand. If you have a four-person team and they will each automate tests only 25% of the time, it's better to buy a floating licence and share the tool during the work day. If they will each use it all the time, buy licences for all of them.

    Which other solutions did I evaluate?

    No, I did a PoC and discovered the pros and cons.

    What other advice do I have?

    Do a PoC and try to understand if TestComplete fits your context and requirements. Use the script-driven approach instead keyword-driven, because the former is more efficient.

    Disclosure: I am a real user, and this review is based on my own experience and opinions.
    PeerSpot user
    it_user711891 - PeerSpot reviewer
    Test Analyst
    Vendor
    It has increased effective testing processes and reduced the cycle time. I would like to see recovery scenarios.

    What is most valuable?

    One of the valuable features is object identification. It provides the different options to users so they can uniquely identify the objects.

    How has it helped my organization?

    By automating the regression test cases, it has increased effective testing process and reduced the cycle time.

    What needs improvement?

    I would like to see recovery scenarios.

    For how long have I used the solution?

    I have been using this solution for over four years.

    What do I think about the stability of the solution?

    I did not encounter any issues with stability.

    What do I think about the scalability of the solution?

    I did not encounter any issues with scalability.

    How are customer service and technical support?

    I would give technical support a rating of 8/10.

    Which solution did I use previously and why did I switch?

    When we started using this tool, only TestComplete provided the support of the automation for WPF objects.

    How was the initial setup?

    The initial setup was easy.

    What's my experience with pricing, setup cost, and licensing?

    This is cheaper compared to other testing tools.

    Which other solutions did I evaluate?

    We evaluated other options QTP.

    What other advice do I have?

    This is the best tool for GUI and Web Automation.

    Disclosure: I am a real user, and this review is based on my own experience and opinions.
    PeerSpot user
    it_user710514 - PeerSpot reviewer
    Test Automation Engineer at a energy/utilities company with 10,001+ employees
    Real User
    Calls different operations through batch files. The product is not stable enough and it crashes often.
    Pros and Cons
    • "Runs in different remote machines. We have multiple versions of the software being tested."
    • "Product is not stable enough and it crashes often."

    What is most valuable?

    • Tests can be execute and run separately.
    • Runs in different remote machines. We have multiple versions of the software being tested.
    • Calls different operations through batch files.
    • Has the option for combining data and is keyword driven. It helps people with less knowledge in programming to work with it.

    How has it helped my organization?

    We already have the UI smoke test and have integrated to our build system, which runs each day for multiple version of the product. This saves us a lot of time.

    What needs improvement?

    * Product is not stable enough and it crashes often

    The application under test is a complex scientific application developed in C++ & C# and use different technologies. So when I try to do Name mapping, it sometimes hangs and have to restart Test complete

    * Checking from TestComplete to TFS has issues.

    I mostly try to checkin the changes with Visual studio as TestComplete hangs while you try to check in with many changes directly to TFS from Testcomplete.

    * Possibility to run a part of keyword tests through TestExecute

    Consider you have a keyword test with 10 individual tests. By using test execute, you can only run the main one, not the individual ones.
    So I like to have the possibility of running from the 3rd test or just running the 5th one.The option is not present in TestExecute (you can do it in Test complete).


    What do I think about the stability of the solution?

    It would help if it were more stable as it sometimes hangs and crashes.

    What do I think about the scalability of the solution?

    We have different version of the product and it framework/project was easily scalable and used by other projects.

    How are customer service and technical support?

    We had multiple technical training sessions which were helpful and almost all the answers are in the community.

    Which solution did I use previously and why did I switch?

    When I started at this company, they were using TestComplete.

    How was the initial setup?

    Getting used to product did take some time.

    What's my experience with pricing, setup cost, and licensing?

    The price is less, compared to other products, such as QTP.

    Which other solutions did I evaluate?

    We had a choice with Coded UI as we developed it using C#. However, TestComplete is the preferred choice one over coded UI, as it needed to grow more as a test tool.

    What other advice do I have?

    Get training and decide on a framework that suited for your application. It always depends on what you want to do with the tool.

    Disclosure: I am a real user, and this review is based on my own experience and opinions.
    PeerSpot user
    QA Manager at a computer software company with 201-500 employees
    Real User
    I like the cross browser compatibility. The way objects are added and used when utilizing descriptive programming could be improved.
    Pros and Cons
    • "I like the cross browser compatibility. It saves a lot of time re-writing scripts to accommodate different browsers."
    • "The way objects are added and used when utilizing descriptive programming could be improved. It is a little unwieldy, compared to UFT."

    What is most valuable?

    I like the cross browser compatibility. It saves a lot of time re-writing scripts to accommodate different browsers.

    How has it helped my organization?

    We are consultants. So we simply provide an automated solution to a client, then move on. We don’t use the product in our day to day work.

    What needs improvement?

    The way objects are added and used when utilizing descriptive programming could be improved. It is a little unwieldy, compared to UFT.

    In UFT, using descriptive programming for a web page you can use.

    Browser(description).WebList(description).Select anything.

    Regardless of how many panes, frames, panels etc are in the hierarchy before the Weblist object.

    But in Smartbear you have to store every frame, panel etc.

    So that if you didn’t use the ‘Alias’ functionality you would have an object description miles long.

    But even having to use the Alias, you still have to add each and every frame, panel, etc. whereas in UFT you can just use page.object and it will find the object on the page (as long as you’re using unique descriptions!!) without worrying about frames, panes, etc.


    For how long have I used the solution?

    We have used this solution for about two years.

    What do I think about the stability of the solution?

    I did not encounter any issues with stability.

    What do I think about the scalability of the solution?

    I did not encounter any issues with scalability.

    How are customer service and technical support?

    I didn’t require any technical support.


    Which solution did I use previously and why did I switch?

    I routinely use SmartBear, UFT, and SilkTest. I fit the application used to whatever my client requires.

    How was the initial setup?

    The initial setup was very straightforward. Even the mobile testing side was easy to setup

    What's my experience with pricing, setup cost, and licensing?

    My advice so far, is that while it’s not quite as powerful and easy to use as UFT, its price tag more than makes up for it. It makes it an excellent cost saving alternative.

    Which other solutions did I evaluate?

    I use several tools.

    What other advice do I have?

    As with all tools, verify that it will do what you need for a reasonable price.

    Disclosure: My company has a business relationship with this vendor other than being a customer: We are in the process of setting up a partner arrangement with SmartBear.
    PeerSpot user
    it_user710532 - PeerSpot reviewer
    Architecture Analyst at a tech company with 5,001-10,000 employees
    Real User
    Automated testing was used to validate security protocols for web applications

    What is most valuable?

    Automated testing was used to validate security protocols for web applications.

    How has it helped my organization?

    It is easier to store and execute test cases, track test results, and save time with automation.

    What needs improvement?

    LoadComplete could be a little more user-friendly, but is still better than LoginVSI. Network issues sometimes made for poor connections.

    For how long have I used the solution?

    We have used this solution for one year.

    What do I think about the stability of the solution?

    I did not encounter any issues with stability. The product was available within expected measures.

    What do I think about the scalability of the solution?

    I did not encounter any issues with scalability.

    How are customer service and technical support?

    I would rate the level of technical support as excellent.

    Which solution did I use previously and why did I switch?

    We switched from HPE QC/ALM. It did not offer automation on the same scale.

    How was the initial setup?

    The initial setup was made easier with professional services. Learning to code was harder.

    What's my experience with pricing, setup cost, and licensing?

    I tell people it is cheaper than ALM with the same features and better UI.

    Which other solutions did I evaluate?

    We did evaluate other options, but I can’t recall which ones.

    What other advice do I have?

    Make sure you have a large volume of testing to transfer. Otherwise, it is not worth the money.

    Disclosure: I am a real user, and this review is based on my own experience and opinions.
    PeerSpot user
    Senior Automation Test Analyst with 10,001+ employees
    Vendor
    Allows us to test both desktop and web applications
    Pros and Cons
    • "It allows us to test both desktop and web applications."
    • "It is very hard to read the test log generated by TestComplete Executor if the log file is very big. TestComplete Executor is a small tool for just running the TestComplete test framework (not for developing)."

    What is most valuable?

    It allows us to test both desktop and web applications. This is a very important feature for system integration tests, as we develop desktop applications, web applications, and a Web API using the same database.

    What needs improvement?

    It is very hard to read the test log generated by TestComplete Executor if the log file is very big. TestComplete Executor is a small tool for just running the TestComplete test framework (not for developing).

    In fact, this is an issue regarding how to read the test log in MHT format generated by TestComplete.

    We know a test log is always generated automatically by test tool after executing a test. TestComplete provides a standard IDE (Integrated Development Environment) interface for user to review the test log. We don’t have issue reviewing the test log within TestComplete regardless how big the test log is (Our test log record the test running for more than two days without stopping. The test log extension name is .tcLogs and the size of total test log is very big over 2.5 GB).

    TestComplete also provides a function for the user to export the log into MHT format file via manually selecting each test log after complete a test or dynamically exporting the test log during test executing via command in test script. The MHT log file can be opened by IE. However, if the MHT file is a little big, such as >200M, the user could experience obvious performance issues. It takes a very long time for IE to open this file because IE loads all the contents into one page which could finally eat up all your system memory. I tried to find another tool to open the big MHT log file, but no luck. Even executing a small test (running for one to two hours), the MHT log file size can easily reach to 200M, if a screenshot is included, for example. Normally, each one round of our regression test runs for 30-48 hours. It is impossible for us to open the big MHT log file as the system runs out of memory and an error appears definitely after hours of waiting to just open this file by IE.

    TestExecute is a small tool to just execute the test developed by TestComplete. But TestExecute only generates MHT log file rather than TCLOGS file. Obviously, we cannot open the MHT log file if it is very big. So we gave up using TestExectue and export MHT log file function in TestCompelete at all.

    We have to buy six individual full license of TestComplete to run and review the test.

    For how long have I used the solution?

    We have used this for nine years.

    What do I think about the stability of the solution?

    Overall, TestComplete is quite stable. But the only issue affecting me is if TestComplete is terminated unexpectedly while test is still running (such as, terminating TestComplete without stopping the test first, window restarting, or shutting down unexpectedly), user could lose all the logs.

    What do I think about the scalability of the solution?

    There were no issues with scalability, but it may have a performance issue with expansion.

    How is customer service and technical support?

    Technical support is excellent.

    How was the initial setup?

    The initial setup was straightforward.

    What's my experience with pricing, setup cost, and licensing?

    The product is becoming more and more expensive. There are two types of licenses: locked and float. Locked license save more, but can only be used in a physical machine. Float licenses can be run in virtual machines.

    Which other solutions did I evaluate?

    QTP, was very expensive when compared to TestComplete, many years ago.

    What other advice do I have?

    This tool is very easy to use and very powerful. It has a short test development circle and good technical support. There is an expensive license, but it could save more, as testers may spend more time achieving a certain test goal if using an Open Source tool. You need to consider the testers' script experience and the company’s budget to choose the right tool.

    Disclosure: I am a real user, and this review is based on my own experience and opinions.
    PeerSpot user
    PeerSpot user
    Senior Consultant at a tech services company with 10,001+ employees
    Consultant
    Using this solution I am able to process whole pages at once rather than doing everything one by one

    What is most valuable?

    Valuable for us is the ability to identify objects by using Find methods. I am able to process whole pages at once rather than doing everything one by one.

    What needs improvement?

    Using object spy can be slow sometimes and seems to require a lot of resources.

    For how long have I used the solution?

    We have been using the solution for six months.

    What do I think about the stability of the solution?

    The solution freezes sometimes, but not very often. Sometimes it doesn’t find any objects from the browser and needs to restart the browser or the whole machine. This happens with Internet Explorer.

    What do I think about the scalability of the solution?

    It is very difficult to have multiple developers when using Name Mapping. You can’t merge all the files and this creates conflicts.

    It scales better for multiple users when using pure script, which reduces the amount of files that cannot be merged.

    How are customer service and technical support?

    Support calls were helpful at the beginning, and I haven’t used the support since then.

    Which solution did I use previously and why did I switch?

    We used TestPlant eggPlant and are still using it for thick clients where objects are not available. The development speed for TestComplete is faster and more reliable when objects can be seen.

    How was the initial setup?

    It is easy to install and use.

    Which other solutions did I evaluate?

    We went through quite a few of them. This product seemed best for usability and recording features.

    What other advice do I have?

    Test the trial, then go for it. Keyword tests seem viable for straightforward test cases, but to create dynamic architecture you might want to stick only with scripts and drop Name Mapping.

    Keyword tests are easy and fast to record, but adding complex logic for them can be tricky and time consuming.

    Disclosure: My company has a business relationship with this vendor other than being a customer: We are partners with SmartBear.
    PeerSpot user
    it_user288375 - PeerSpot reviewer
    Quality Assurance Engineer with 501-1,000 employees
    Vendor
    We've now automated 90% of the testing we used to perform manually, but I would have preferred Python support in earlier versions.

    Valuable Features

    It's easy to work with and doesn’t take much to get it setup to start working with it.

    Improvements to My Organization

    In my previous three positions, there was not a test automation solution in place. I was given the opportunity to explore options. Once I chose a solution, we were able to implement TestComplete and were able to automate about 90% of the manual testing that was done prior to implementing TestComplete.

    Room for Improvement

    This product continually improves and in v11, they now have Python support. This was something that I wanted and they provided it the latest version.

    Use of Solution

    I've used it for 11 years.

    Deployment Issues

    I have not run into any issues with deployment.

    Stability Issues

    I have not run into any issues with stability.

    Scalability Issues

    I have not run into any issues with scalability.

    Customer Service and Technical Support

    Customer Service:

    Another one of the many reasons why I chose TestComplete. The level of customer service and technical support can’t be beat. SmartBear always answers my questions within 24-48 hours.

    Technical Support:

    They have a great website, help and forums that also help in finding the answers I need in a timely manner.

    Initial Setup

    It was very straightforward which was another reason why I went with TestComplete. I was able to use the demo version (when I was searching for options) to create tests easily, so when I had to demonstrate the product to my management teams, they could easily see why TestComplete was the product for us.

    Implementation Team

    The implementation was done by me. The best recommendation is to read the help guide, especially if you are using the product in different ways, like floating licenses. This is where the license server is on one machine and people have to access TestComplete that are not local. However, if you run into any issues, the customer support department is there to help in any way they can.

    ROI

    My ROI has been the fact that it takes less time to run all of the tests that were done previously. Prior to TestComplete, it would take over three weeks to run all of the tests that needed to be run for a release. After TestComplete, we have been able to reduce that time to less than one week.

    Pricing, Setup Cost and Licensing

    One of the main reasons I went with TestComplete, besides the information that I already provided, was the cost for TestComplete and TestExecute. They make it very easy for large to small companies to implement without large costs. The licenses are broken down the amount of users that need to use it and they also have the option of floating licenses.

    Other Solutions Considered

    I looked at various options like QuickTest Professional, Rational Functional Tester, and SilkTest.

    Disclosure: I am a real user, and this review is based on my own experience and opinions.
    PeerSpot user
    PeerSpot user
    Software Quality Analyst I at Bentley Systems Incorporated
    Consultant
    This application is helping a lot in regression testing. If something in the application breaks, we get to know about it quickly, and this saves us time.

    What is most valuable?

    Object Browser is one of very powerful tool TestComplete provide which can see what processes are running in the memory.

    How has it helped my organization?

    This application is helping a lot in regression testing. If something in the application breaks, we get to know about it quickly, and this saves us time.

    What needs improvement?

    There

    For how long have I used the solution?

    My team has been using it for three years, and I've been using it for one-and-a-half years.

    What was my experience with deployment of the solution?

    Not at all. The framework was already in place with my team.

    How are customer service and technical support?

    Customer Service:

    It's very good.

    Technical Support:

    It's very good.

    Which solution did I use previously and why did I switch?

    That is my first product of this type.

    How was the initial setup?

    Initial set-up was pretty straightforward, but that was basic. Complexity grows when you go deep in testing.

    What about the implementation team?

    We do get some training from SmartBear, but we implement tests on our own.

    What was our ROI?

    The ROI makes it worth buying this product.

    What other advice do I have?

    Disclosure: I am a real user, and this review is based on my own experience and opinions.
    PeerSpot user
    PeerSpot user
    Systems Engineer at a tech services company with 51-200 employees
    Consultant
    It is similar to other tools like Selenium IDE where you can record and play test scripts. There is no support for mobile testing automation.

    Valuable Features:

    For more than a year, I've been working on automation of functional testing, regression testing using TestComplete. It is a great tool and hard to learn and use. 
     
    Pros:
    - It is similar to other tools like Selenium IDE you can record and play test scripts 

    - Developer needed to create test scripts using VB Script, Jscript, Delphiscript etc - 
    - QA perspective, helps test results logs helps in finding precisely the issue where test failed
    - Tests can be recorded in keyword driven or screen capture modes on some screens.
    - Tests can be created for regression, functional, DDT (data driven testing), ODT(object driven testing), and so on and it is good to use it for Cross browser testing.
     
    Cons:
    - There is no support for mobile testing automation.

    - QA need to depend upon developer to correct the scripts any change happens. Automated

    Improvements to My Organization:

    Recording the scripts and running the scripts should be available to all users.

    As a QA, I would like to use it by myself using the application recording and replaying the tests.

    Room for Improvement:

    On some of the UI application pop up screens, Closing and opening screens etc.

    Disclosure: I am a real user, and this review is based on my own experience and opinions.
    PeerSpot user
    it_user242190 - PeerSpot reviewer
    it_user242190Test Manager with 501-1,000 employees
    Vendor

    what is the best way to convert UFT scripts to TC compatible scripts.

    PeerSpot user
    Systems Test Manager at a transportation company with 1,001-5,000 employees
    Real User
    The latest version can be used for mobile testing, but more effective object mapping for Citrix systems is needed.

    What is most valuable?

    Keywords-Driven frameworks and Data-Driven Testing (DDT) methodologies used for building effective frameworks.

    How has it helped my organization?

    The fact that the latest version can be used for mobile testing.

    What needs improvement?

    A more effective way in Objects Mapping for applications built on domains/platforms such as Sliverlight, and Citrix is needed.

    For how long have I used the solution?

    I've used it for three years.

    How are customer service and technical support?

    Technical support can be improved by being more efficient in responding to questions.

    Which solution did I use previously and why did I switch?

    Used some other products, but this was found to be the most suited for business requirements.

    How was the initial setup?

    The initial step was straightforward which was made by carrying out a POC and the decision was to go for this solution.

    What about the implementation team?

    Following initial training by the vendor, the implementations are done in-house.

    What's my experience with pricing, setup cost, and licensing?

    It is value for money, and different licensing options are available.

    What other advice do I have?

    Fig 1: Building of Automation Testing framework.

    Fig 2: Automation Testing Results

    Disclosure: I am a real user, and this review is based on my own experience and opinions.
    PeerSpot user
    it_user409368 - PeerSpot reviewer
    it_user409368Director of Product Management at a tech vendor with 201-500 employees
    Vendor

    Hello, Thank you very much for your feedback! We will use it to improve TestComplete.
    Could you please elaborate on what specific aspects of the application you would like us to improve as demonstrated in figure Figure 1 and Figure 2?
    Regards,
    Masha

    PeerSpot user
    Project Lead at Aspire Systems
    Real User
    A test automation project and all its value additions and work arounds are hosted on a Test Automation framework. It is tedious to monitor the progress since the logs are available only in the tool.

    Authored by Sabhari Murugan and Ujwal Unni from Aspire Systems Testing Community.

    Overview

    Software Quality is dependent on Test Automation for its ability to provide vast coverage with least possible resource. Having said that, automation has its own overhead and maintenance which requires additional effort. To reduce such additional effort, besides ensuring stability, robustness and increase the ROI - there are certain considerations to be focussed at different stages of the project. Here is an insight to such areas of considerations and how it can be dealt with ‘TestComplete’ – a potential test automation tool.

    What is TestComplete?

    Started as a tool that is preferred for Test Automation of Desktop applications, today TestComplete is one of the top competitors and a noticeable performer in the commercial market. The recent versions of the tool support Desktop, Web and mobile applications to a great extent. The tool gained its popularity due to its

    • Robust and stable execution engine
    • Rich in-built libraries
    • Support for multiple Scripting Language
    • Minimal knowledge required for beginners
    • Access to System level operations like system restart, Process management and monitoring
    • Cost effective, when compared with similar tools in the market
    • Events (Decide what to do at - Beginning of Test, End of Test, Error encountered, etc...)

    Doing it right:

    Test Automation is not an area with constant or limited needs and expectations. New technologies, frequent changes, emerging software systems and new feature additions create constant challenges for tools and testers. There are certain considerations in each phase of Test Automation solution to build a suite that could provide high ROI, with least effort and maintenance. Here is a brief about some of these challenges / requirements that are more likely in Test Automation solutions that use TestComplete.

    1. Frequent Changes

    “Change is good” – in many ways, for instance it makes way for new features and enhances user experience. Besides, who likes to use software that has no updates? For any product to be successful, it should have the ability to change and adapt to the cutting edge technology.

    Implementation stage: Framework design.

    Solution:

    1. Name Mapping – an object repository used to store the instance of controls. By doing so any change in the Properties of controls such as size, colour or caption can be isolated leaving the automation scripts unaffected.

    2. Modularity Framework- a Test Automation framework in which all the common functions such as entering text in text box, selecting items in a combo box, clicking buttons are developed as reusable functions and used in Test Case Script

    Doing so, the Test Case Script will not have any code that directly interacts with the product, the controls are referred through Name Mapping and actions are performed through the re-usable functions. Hence Test Case Script is isolated and no change will be required unless there is a change in the workflow of the product.

    2. Testing Multi-Language Products

    When testing applications that are released in different language versions it is crucial to test all the language versions to ensure quality.

    Implementation stage: Framework design.

    Solution:

    1. Using an XML or ini file. The file contains a common ID for each caption and their equivalent value in the respective language. For example, below is a representation of the xml file content for the caption of Ok button in three languages.

    <caption>
                    <id=’YesButtonCaption’>                
                    <English=’Yes’>
                    <German=’Ja’>
                    <Finnish=’Kylla’>
    </caption>

    In the Test Case Script or in the reusable functions instead of directly using the caption, use a re-usable function that fetches the appropriate language text from the XML when the id and the language are passed as parameters. For example: getCaption(“YesButtonCaption”, “German”)

    3. Different Levels of Execution

    Test Suites may vary from less than hundred cases to more than 20,000 cases. It is not always the case that the entire test suite has to be executed; hence cases should be grouped based on features or workflows. For example automation suite of a banking application can be grouped based on features like money transfer, account creation, payments and so on to enable execution at three levels – Case level, Selected Group level and suite level.

    Implementation stage: Framework design.

    Solution:

    1. Driver Script – Instead of matching each test Item to a test case, group them based on features and create a driver script for each feature. A driver script is just a function that calls all the test cases under the respective feature one after the other.

    2. Further enhancement can be done by using XLS file to have list of Cases and a column with Yes or No values that determine if the particular case has to be executed or not.

    4. Windows event log monitoring

    There are scenarios where the application may seem to work fine, even when there are some issues while updating data in the database or storing data to a file. Monitoring the Windows Event log gives a better insight into such situations and helps to identify them earlier.

    Implementation stage: At the time of requirement.

    Solution:

    1. TestComplete provides full read access to Windows event log, leveraging this feature at suite level can capture all the events logged related to the application under test during execution.

    2. TestComplete also provides access to real-time resource usage of any process such as CPU usage and Memory usage, logging these details will help identifying memory leakages.

    5. Memory issues due to large suites

    Scalability is one essential aspect to be considered in Test Automation. TestComplete is designed to handle any number of Test Cases which means almost any Lines of Code (LOC). But having too many script units will have a huge memory foot print which accumulates over execution, which eventually leads to tool crash or other performance issues.

    Implementation stage: At the time of requirement.

    Solution:

    1. If there are way too many Test Scripts in an Automation suite, the suite can be split into multiple smaller suites. All the smaller suites are independent of each other. In this case, when a whole suite execution is required Batch Files can be used to continue the execution from one suite to another.

    6. Continuous Long-time Execution

    Having around a few thousand Test Cases in a Test Suite means executions can go on continuously for days. This makes the system vulnerable for performance and memory issues.

    Implementation stage: At the time of requirement.

    Solution:

    1. The application under test can be closed and restarted in certain intervals to free memory and have better performance

    2. It is not just the application but also the tool that is in running state for too long, hence it makes a difference if the tool is restarted in some required intervals approximately once a day. This can be made possible by two ways- TestComplete’s in-built function “RebootandContinue”,which restarts the system and continues further execution or using batch file to close TestComplete after sometime into execution, let the system be idle for a while, then start TestComplete and continue the execution.

    7. Automatic Report generation and mail notifications

    When execution continues for days together it is tedious to monitor the progress since the logs are available only in the tool, this means either the execution has to be paused or we will have to wait till it completes to view the report or log

    Implementation stage: At the time of requirement.

    Solution:

    1. The in-built function to Save Logs can be used at regular intervals to automatically export logs to the specified location as and when required.

    2. TestComplete also supports automatic mail notifications with the logs as attachments. Once configured results will be mailed automatically as per the configuration

    8. Scheduling Tests in Test Execute

    SmartBear has an execution tool that is relatively less expensive than TestComplete that can be used only for placing executions. Unfortunately, TestExecute does not enable us to select the Tests that has to be executed. So one way or another TestComplete is required to modify the Tests to be executed.

    Implementation stage: At the time of requirement.

    Solution:

    1. The .PJS and .MDS files in a Test Suite are the two XML files that stores the data about the project and Test items to be executed. Developing a simple tool to read and edit these XML files will completely eradicate the need of TestComplete in placing Executions.

    Other Best Practices:

    • Always ensure each test script works independently.
    • Avoid test data dependencies between test scripts.
    • Test data is always externalized.
    • Write the test scripts from scratch instead of using record and playback.
    • Always ensured test automation scripts sequence are configurable for execution.
    • Frequent catch-up with Dev & Design team to understand the impact on automation scripts due to upcoming releases / changes.
    • Tool & Environment upgrades should be taken up as early as possible to avoid surprises at neck of the moment.

    A test automation project and all its value additions and work arounds are hosted on a Test Automation framework. Automation Framework is the basic building block of a robust automation suite and framework design is the most important phase of an automation project. This is the phase where all the above considerations should be predicted in order to design a framework that meets all the requirements of the project and accommodate further improvements and new requirements that may arise over the course of the project. Such Anticipation, preparation and worked arounds are the way to ’Doing it right, with TestComplete’.

    Disclosure: I am a real user, and this review is based on my own experience and opinions.
    PeerSpot user
    it_user347607 - PeerSpot reviewer
    Technical Leader at a tech services company with 10,001+ employees
    Consultant
    I was part of a systems architecture team and we were looking into various kinds of automation tools. We found this solution to be the most complete tool.

    Valuable Features

    It allows me to invoking an application and then to browse the application. I've done some good research with TestComplete along with Google Earth.

    Improvements to My Organization

    I was part of a systems architecture team and we were looking into various kinds of automation tools. We found this solution to be the most complete tool.

    Room for Improvement

    More APIs could be added.

    Use of Solution

    I used it for six months.

    Stability Issues

    I didn't encounter any stability issues in my short tenure, although I remember invoking the application twice although there is was an instance already open.

    Customer Service and Technical Support

    Customer Service:

    I haven't had to use customer service.

    Technical Support:

    I haven't had to use technical support.

    Initial Setup

    Initial setup was easy, but implementation was a little tricky initially. I could not find any sample projects that I could use as a guide.

    Pricing, Setup Cost and Licensing

    It's open source, so it's free.

    Other Solutions Considered

    We looked at other solutions, but this was the most complete.

    Other Advice

    It's a great tool that covers many features and is straightforward with logs.

    Disclosure: I am a real user, and this review is based on my own experience and opinions.
    PeerSpot user
    PeerSpot user
    Mobile QA Tester at a comms service provider with 1,001-5,000 employees
    Vendor
    It's enabled us to automate a lot of our tests for Android devices.

    Valuable Features

    I found the Keyword Test feature very valuable. Even with my limited programming knowledge, I was able to create automation tests using the Keyword Test feature. Also, the ability to create image based tests was extremely helpful, especially when automating tests for mobile devices.

    Improvements to My Organization

    TestComplete has enabled us to automate a lot of our tests for Android devices.

    Use of Solution

    I've used it for three months.

    Customer Service and Technical Support

    I would rate it 10/10. Every time I had a question or issue I couldn’t figure out, they were right there to help.

    Initial Setup

    Set-up was pretty straightforward.

    Implementation Team

    We implemented it through an in-house team.

    Disclosure: I am a real user, and this review is based on my own experience and opinions.
    PeerSpot user
    it_user340986 - PeerSpot reviewer
    Senior QA Analyst at a financial services firm with 501-1,000 employees
    Vendor
    It helps to organize test runs using the Test Items page of the project editor.

    Valuable Features

    • Data driven testing using Microsoft Excel
    • Viewing properties of objects using Object Spy
    • Organizing test runs using the Test Items page of the project editor

    Use of Solution

    For versions 8.X to 9.20.2460.7, I've been using TestComplete for about four and a half years.

    Stability Issues

    It seems to crash once every two to three months.

    Customer Service and Technical Support

    Customer Service:

    7/10

    Technical Support:

    7/10

    Initial Setup

    For a node-locked license, it's pretty straightforward. For floating licenses, it's a little more complex because you have to set-up the license manager and do a little bit of configuration on the computers that use the floating licenses.

    Disclosure: I am a real user, and this review is based on my own experience and opinions.
    PeerSpot user
    it_user340989 - PeerSpot reviewer
    Quality Assurance Specialist at a tech vendor with 201-500 employees
    Vendor
    It has saved time by eliminating the need for manual testing. It's also led to the discovery of bugs that wouldn't have been found otherwise.

    What is most valuable?

    With TestComplete, I can discover bugs faster and easier than testing manually. It eliminates the need for regression testing sessions as TestComplete can be set up to test automatically overnight against each day's build from programmers.

    How has it helped my organization?

    It has saved time by eliminating the need for manual testing. It's also led to the discovery of bugs that wouldn't have been found otherwise.

    What needs improvement?

    I'm completely satisfied with TestComplete 10, but looking forward to seeing what's in store for version 11!

    For how long have I used the solution?

    I've used it for about two years.

    What was my experience with deployment of the solution?

    The initial deployment of TestComplete was difficult, but with the assistance of SmartBear technical support, I was able to get it up and running.

    How are customer service and technical support?

    SmartBear's technical support is excellent. However, due to time zone differences, support can be a little slow.

    Which solution did I use previously and why did I switch?

    Previously, testing was performed manually. TestComplete was a great solution to automate the process.

    How was the initial setup?

    Installing TestComplete was easy, however "floating license" configuration was challenging with the use of VMs.

    What about the implementation team?

    I implemented TestComplete independently. Their "How-To" videos, forums, and support team are incredibly valuable!

    What's my experience with pricing, setup cost, and licensing?

    If you have any questions about licensing, Smart Bear's staff is very helpful!

    Disclosure: I am a real user, and this review is based on my own experience and opinions.
    PeerSpot user
    it_user340992 - PeerSpot reviewer
    Test Automation Specialist at a tech services company with 501-1,000 employees
    Consultant
    It gives us the option to work in the same suite without changing from one project to another.

    What is most valuable?

    • Object spy and object recognition are working very well, giving us a lot of possibilities.
    • Option to work in the same suite without changing from one project to another, which is better than HP QTP/UTF.
    • USEINIT feature is really nice (+/- like importing in Java)

    How has it helped my organization?

    It didn’t really change the way my organization works.

    What needs improvement?

    A test case design (schema) module would be nice.

    For how long have I used the solution?

    I've been using it for nine months, and my customer has been using it for three years.

    What was my experience with deployment of the solution?

    No issues encountered.

    What do I think about the stability of the solution?

    No issues encountered.

    What do I think about the scalability of the solution?

    No issues encountered.

    How are customer service and technical support?

    I just contacted them once. They responded very quickly, which is good, but the problem was on our side, so I cannot really evaluate.

    Which solution did I use previously and why did I switch?

    TestComplete was already in place when I arrived, but I know that they chose it because of the debugger and the fact that TestComplete recognizes more objects than QTP/UFT.

    What about the implementation team?

    It was done in-house.

    Disclosure: I am a real user, and this review is based on my own experience and opinions.
    PeerSpot user
    Haiping Gong - PeerSpot reviewer
    QA Automation Manager at a comms service provider with 501-1,000 employees
    Vendor
    We use it to do the automation test on our product’s web UI. In total, over 500 test cases are written and the automation coverage is over 80%.

    What is most valuable?

    The name mapping function to manage the objects across different web pages and the support for different web browsers are the most valuable features for us.

    How has it helped my organization?

    We use TestComplete to do the automation test on our product’s web UI. In total, over 500 test cases are written and the automation coverage is over 80%. This helps to reduce manual effort dramatically, and by using the same script, we can run the test on both IE and Google Chrome.

    What needs improvement?

    The scripts are saved as binary files on disk. This makes it troublesome to perform version control and merge it with the work that is done by other team members. I am not sure what the reason is that SmartBear chose to save the files as binary, but if they can change to plain text for the scripts in the future, that would be helpful.

    For how long have I used the solution?

    I've used it for one year.

    What do I think about the stability of the solution?

    Some of the objects/parent objects are dynamically created in the web page. During the test, TestComplete will have problem to identify these kind of objects. This will cause a lot of false rejection and block the following test cases. Thus it requires some rework when running all the test cases as a batch.

    How are customer service and technical support?

    I have not used any customer service/tech support yet. I cannot provide more comments on that.

    Which solution did I use previously and why did I switch?

    TestComplete is the first tool that I use for automation on UI and I have not used any other tools yet.

    How was the initial setup?

    It is pretty straightforward. Just by following the instructions given in the document and do the corresponding configuration on the web browsers that you want to test. However, we also encounter some problems such as the Chrome blocking users from proceeding because of an unsafe SSL connection, but we found a work around eventually.

    What about the implementation team?

    We implement the automation by ourselves, not through any other vendor team. My suggestion is that you need to decide how to divide the automation work into corresponding components carefully. Otherwise, it will be hard for you to share the common scripts with other team members. Also, the code architecture design is very important as well.

    What was our ROI?

    I am not responsible with the licensing quotation/renewal, but the benefit that TestComplete brings to us is, apparently, acknowledged.

    Disclosure: I am a real user, and this review is based on my own experience and opinions.
    PeerSpot user
    it_user341001 - PeerSpot reviewer
    Junior ASP/.Net Developer at a construction company with 501-1,000 employees
    Vendor
    Its environment exposes a significant portion of its functionality to the command line, where execution details can be left to .bat files or Windows Scripts.

    Valuable Features

    Object-based recording, and data-driven testing.

    Separation of data into Excel files made tests modifiable by QA personnel with limited development experience, and object-based recording kept maintenance to a minimum.

    Improvements to My Organization

    TestComplete's environment exposes a significant portion of its functionality to the command line, where execution details can be left to .bat files or Windows Scripts.

    This allowed us to schedule execution of lengthy tests for nighttime and non-core hours, and to synchronize tests with Jenkins build releases.

    This freed up QA assets to perform more specialized testing and reduced redundancy.

    Room for Improvement

    Native test result reporting does not provide overview reporting methods for tests that span multiple project suites. Features that allow for flagging a test as dependent on the result of another in multi-project hierarchies while maintaining name-mapping segregation would be much esteemed.

    Use of Solution

    I've used it for approximately eight months.

    Customer Service and Technical Support

    Customer Service:

    8/10 - SmartBear representatives are surprisingly responsive, and they go out of their way to offer assistance.

    Technical Support:

    8/10 - SmartBear representatives are surprisingly responsive, and they go out of their way to offer assistance.

    Initial Setup

    The initial set-up was exceedingly simple. There is a silent mode option available during set-up, which is very convenient when deploying to multiple machines or remotely.

    Implementation Team

    We implemented it through an in-house team.

    ROI

    We did not maximize our ROI until we put somebody full-time on our TestComplete endeavours. The tool does have a learning curve, and it wasn't until we had an in-house expert on it that we began to see the benefits of automated testing over traditional QA roles.

    Pricing, Setup Cost and Licensing

    The licensing options for TestComplete both running a licensing server. This prevents users from running more sessions simultaneously than purchased keys. This can prove problematic if you want multiple developers writing or running tests at once, and prevents you from using your key while a distributed test is running.

    For pricing, carefully consider how many machines you want running the software, rather than the number of developers.

    Other Solutions Considered

    We also evaluated another SmartBear product called SoapUI. The change to TestComplete occurred because we changed our target from web applications to desktop.

    Other Advice

    My advice in regards to implementation would be to choose carefully which tests to automate, specifically focusing on lengthy procedures, tasks that require looping, or places where you want to test against multiple data sets.

    Additionally, I found it beneficial to prefix my keyword tests with a character and number to provide logical ordering instead of alphabetic.

    I also found it beneficial to record "undo" steps with each keyword test; this allows each test to be more stand-alone and prevents your test from being dependent on the state the previous test left the application in.

    Finally, I would suggesting limiting the number of test applications per test suite to prevent bloated name-mapping schemes.

    Disclosure: I am a real user, and this review is based on my own experience and opinions.
    PeerSpot user
    it_user336978 - PeerSpot reviewer
    it_user336978Multimedia Solutions Verification Engineer at a comms service provider with 1,001-5,000 employees
    Vendor

    Test Complete provides detailed reports since every profit software should do, since people pay for its reporting functionality also. On the other hand, some big and expensive products stil has the reporting gaps, for example there are third party reporting tools for HP QC.

    See all 3 comments
    PeerSpot user
    Systems Engineer at a tech services company with 51-200 employees
    Consultant
    The automation module helped us save time in regression tests, but we needed to perform adjustments on our systems to get it to fit in.

    Valuable Features

    The automation has improved over time, allowing it to integrate with other external tools for defect tracking.

    Improvements to My Organization

    The automation module helped us save time in regression tests.

    Room for Improvement

    It works by recording and playing a script for some part of development. However, there's no way of knowing where it failed. The developers needed help to correct these errors to script them in order to work. I noticed many changes made from version to version.

    This tool is also very expensive compared to the similar tools in the market. Price needs to be affordable.

    Use of Solution

    I've used it since 2012.

    Deployment Issues

    We faced lots of code breaking, and needed to perform adjustments on our systems to get Test Complete to fit in.

    Stability Issues

    Sometimes it runs fast, and sometimes it runs slow.

    Scalability Issues

    No issues encountered.

    Customer Service and Technical Support

    Customer Service:

    They used to respond in less than 24 hours, and compared to SpiraTest, it's not that great.

    Technical Support:

    SpiraTest took a minute to respond to any technical support issues, but Test Complete takes longer.

    Initial Setup

    It's not complex.

    Implementation Team

    It was already in place when I joined the company.

    ROI

    6/10.

    Pricing, Setup Cost and Licensing

    It's priced high, and they should look into it to make it more competitive.

    Other Solutions Considered

    We also looked at Selenium IDE, and chose it because it's cheaper.

    Other Advice

    Check the price and compare to other available tools in the market and decide select the one best fits the needs.

    Disclosure: I am a real user, and this review is based on my own experience and opinions.
    PeerSpot user
    it_user69072 - PeerSpot reviewer
    QA Manager at Airpush India Pvt Ltd
    Vendor
    Automation testing using TestComplete

    For more than a year, I've been working on automation of functional testing, regression testing using TestComplete. It's a paid automation tool.

    It's an little tricky to learn and use but quite similar to Selenium. You can use xpath, record and play to script test cases.

    Pros:
    - Similar to Selenium IDE, you can record and play test scripts
    - Similar to Selenium webdriver, you can create test scripts in VBScript, JScript, DelphiScript, C++Script or C#Script. (personally, I liked VBScript)
    - Test results logs helps in finding precisely the issue where test failed
    - Tests can be recorded in keyword driven or screen capture modes
    - Tests can be created for regression, functional, DDT (data driven testing), ODT(object driven testing), and so on
    - Can easily do cross-browser testing

    Cons:
    - Automated tests can not be run on Mac OSX
    - No support for mobile testing automation (though there is a package 'seetest' which can be integrated with TC)
    - It can not read xpath values for browsers other than IE, Firefox, chrome, Safari.

    Also, you can learn about TestComplete by this book(written by Genndiy Alpaev and reviewed by me):

    https://www.packtpub.com/application-development/testcomplete-cookbook 

    Disclosure: I am a real user, and this review is based on my own experience and opinions.
    PeerSpot user
    it_user340998 - PeerSpot reviewer
    it_user340998Software Quality Analyst I at Bentley Systems Incorporated
    Consultant

    In our organization we are using TestComplete for desktop based application using Visual Basic. Not using record and play back because this technique is not stable and must be used for learning purposes. We have very comprehensive set of regression testing integration with Jenkins. Yes you are right it need alot of improvement in web and mobile automation. For web based application I would personally recommend to use Selenium Webdriver using Java.

    See all 3 comments
    it_user327474 - PeerSpot reviewer
    QA Automation Engineer at a computer software company with 501-1,000 employees
    Vendor
    We can test our applications manually for a full 48 hours continuously, but it lacks its own log viewer.

    What is most valuable?

    The opportunity to work with DevExpress and WPF objects.

    How has it helped my organization?

    • Approximately, we get new build once per week. So I need to provide smoke testing for it. In case of manual tests execution, it is needed 48 hours (six working days) to smoke it. With TestComplete, we need just eight hours (or even less - if we're using multiple machines)
    • Stability and Stress testing. It is impossible to test our applications manually for 48 hours (two full days) continuously. With TestComplete, it is possible

    What needs improvement?

    • Better stability, as sometimes, TestComplete crashes when attempting to delete over 10 logs
    • Support for the latest versions of DevExpress
    • More comfortable XML editor (like in Notepad++)
    • A better script editor. I will be happy, if the TestComplete editor would contain a design like the one in Visual Studio or Eclipse - errors and unusable variables being highlighte, and refactoring opporunities
    • Own log viewer in TestExecute. For now, it is possible to open it only within a browser, which is not very good, because if the log is 2GB or more, it will take 10-20 minuts to open with Internet Explorer

    For how long have I used the solution?

    I've used it for three years, including, v7, & v9 with TestExecute v9, and it's been in use on the project since 2010.

    What was my experience with deployment of the solution?

    When we migrated from v7 to v9, we encountered a problem with using recursive code; in DelphiScript recursion was completely broken.

    What do I think about the stability of the solution?

    Sometimes, TestComplete crashes when attempting to delete over 10 logs.

    How are customer service and technical support?

    Customer Service:

    It's acceptable.

    Technical Support:

    It's acceptable.

    Which solution did I use previously and why did I switch?

    I tried Visual Studio 2010 (Coded UI Tests feature). The tool was changed, because TC is cheaper and more acceptable for those apps testing. But for now, this project doesn't use QA automation and I am working in another project

    Disclosure: I am a real user, and this review is based on my own experience and opinions.
    PeerSpot user
    PeerSpot user
    Sr. Software Quality Consultant at a tech services company with 501-1,000 employees
    Consultant
    We're able to perform NPAPI test runs on Chrome, but the ExtJS extension and reading of JQuery standard modal screens is still not so good.

    Valuable Features

    Test Runs on Chrome. Test Runs are faster than the previosu version.

    Improvements to My Organization

    After NPAPI was unsupported by Chrome, our test runs were halted on Chrome for a previous version of TestComplete. Started running again with version 11.11.

    Room for Improvement

    ExtJS extension and reading of JQuery standard modal screens is still not so good. For Automating ExtJS based systems you have to do record and playback to extract methods and no substantial documentation has been provided.

    Deployment Issues

    None.

    Stability Issues

    The product is relatively stable.

    Customer Service and Technical Support

    Customer Service:

    It is quick.

    Technical Support:

    Not so helpful most of the times. They always fail to produce a work around.

    Initial Setup

    It was easy.

    Implementation Team

    In-House.

    Pricing, Setup Cost and Licensing

    Do not buy all the modules at once, if your primary system under test is for example Web based then only buy the web module.

    Disclosure: I am a real user, and this review is based on my own experience and opinions.
    PeerSpot user
    it_user323649 - PeerSpot reviewer
    QA Team Lead at a tech vendor with 10,001+ employees
    Vendor
    It supports .Net, but it needs to have templates of tests from scratch.

    What is most valuable?

    • Good IDE
    • Support of third party controls
    • Flexible in coding
    • Supports .Net assemblies
    • Good community

    How has it helped my organization?

    It's decreased the time needed for the regression testing cycle.

    What needs improvement?

    • Memory consumption
    • Speed of interaction with controls
    • Sets of prepared libraries like asserts
    • Templates of tests from scratch

    For how long have I used the solution?

    I've used TestComplete for three years.

    What do I think about the stability of the solution?

    It's stable.

    How are customer service and technical support?

    It's good, I like the support.

    Which solution did I use previously and why did I switch?

    Previously was QTP. TestComplete is much more flexible light,and easy to use.

    How was the initial setup?

    It's quite easy to start using it. There were no problems from the beginning.

    What about the implementation team?

    Use JScript for quick wins. Start creating tests as small as possible.

    What was our ROI?

    I would say that we have now automated about 100 tests, so it saves us three days every month..

    What's my experience with pricing, setup cost, and licensing?

    Disclosure: I am a real user, and this review is based on my own experience and opinions.
    PeerSpot user
    it_user324366 - PeerSpot reviewer
    QA Engineer at a computer software company with 51-200 employees
    Vendor
    It’s got flexible logging capabilities, but needs implementation of screen recording during testing.

    Valuable Features

    • Its Multi browser testing capabilities
    • Record-and playback
    • Network suite testing (remote testing) capabilities
    • Key word driven testing with ability to finetune tests using scripting languages
    • Its flexible logging capabilities
    • High level of support from Smartbear
    • Support of other users in large userbase

    Improvements to My Organization

    Using TestComplete, the application tests are executed at night unmanned, so that test engineers can focus on developing new tests instead of executing existing tests.

    Room for Improvement

    Implementation of screen recording during testing (video recording).

    Use of Solution

    I've used it for approximately four years.

    Stability Issues

    No issues encountered,

    Scalability Issues

    No issues encountered,

    Customer Service and Technical Support

    Customer Service:

    Very professional and adequate. I would rate this on a 9.5/10.

    Technical Support:

    Very professional and adequate. I would rate this on a 9.5/10.

    Initial Setup

    It's straightforward.

    Implementation Team

    I implemented it myself, using instructions from the vendor’s website.

    Pricing, Setup Cost and Licensing

    Start with evaluation version of the product, which can be downloaded from Smartbear's website, and after the initial evaluation, decide which licensing method you need.

    Other Solutions Considered

    I have evaluated other solutions, the cost model, the level of support of legacy software, the easy of use and easy implementation approach were main reasons to choose this product.

    Other Advice

    Focus on implementing a test automation framework. Start by downloading a free evaluation trial version of the product, and see if the product works well with the application(s) you want to design an automated test frame for.

    Disclosure: I am a real user, and this review is based on my own experience and opinions.
    PeerSpot user
    it_user323895 - PeerSpot reviewer
    Team Lead at a tech vendor with 501-1,000 employees
    Vendor
    It has a unified way of finding and manipulating controls from different vendors on different platforms, though it lacks support for different web UI frameworks and V8.

    What is most valuable?

    It supports JScript and it has unified way of finding and manipulating controls from different vendors on different platforms. As a key feature for beginners - It can record user actions and reproduce them.

    How has it helped my organization?

    First two companies began to rely on results of automated regression.

    What needs improvement?

    There is no qualified support for different web UI frameworks and no V8 support as well. The other thing that must be improved is their licensing policy with Virtual Machines.

    For how long have I used the solution?

    I've used it for six years.

    What was my experience with deployment of the solution?

    There were some issues.

    How are customer service and technical support?

    Customer Service:

    5/5.

    Technical Support:

    5/5.

    Which solution did I use previously and why did I switch?

    I used different solution afterwards and I can say there is no other product with such great flexibility.

    How was the initial setup?

    The initial setup is as simple as any product with a Windows installer.

    What was our ROI?

    ROI is simple. We've spent three months on development and automated one month of regular regression tests done manually for each release. Now we did it for each build.

    Disclosure: I am a real user, and this review is based on my own experience and opinions.
    PeerSpot user
    it_user323742 - PeerSpot reviewer
    Test Automation Engineer at a tech vendor with 1,001-5,000 employees
    MSP
    It has helped significantly improve and speed up regression testing, although it could be improved with cross-platform testing (Linux, MacOS).

    What is most valuable?

    Ability to test all types of application (desktop, mobile, and web).

    How has it helped my organization?

    In all projects where I participated it helped significantly improve and speed up regression testing, which is usually bottleneck before release.

    What needs improvement?

    Since v11 of TestComplete it has all the features I was dreaming about during these years. Probably the only one great thing could be cross-platform testing (Linux, MacOS).

    For how long have I used the solution?

    I've been using it since 2006.

    What was my experience with deployment of the solution?

    No, I have never had any difficulties with TestComplete. All possible issues are also covered in TestComplete's help system and available online.

    What do I think about the stability of the solution?

    No issues encountered.

    What do I think about the scalability of the solution?

    No issues encountered.

    How are customer service and technical support?

    It is impressive! What I like the most is the fact that support team helps solving problems not only for their registered customers, but everyone else as well.

    Which solution did I use previously and why did I switch?

    In last 12 years I also used SilkTest and Squish. They have some benefits which TestComplete doesn't have, but in general TestComplete overcomes them.

    How was the initial setup?

    Was it difficult to set up the application first time? No. Or was it difficult to start using it? No, it was easy thanks to its tutorials.

    What about the implementation team?

    I did everything by myself. There is nothing difficult in it, TestComplete is very user-friendly.

    Disclosure: My company has a business relationship with this vendor other than being a customer: My company doesn't, but I wrote a book on TestComplete and sometime run online courses for students in Russian.
    PeerSpot user
    it_user323700 - PeerSpot reviewer
    Assistant Lecturer at a university with 1,001-5,000 employees
    Vendor
    We're able to create test inputs with tens of thousands of fields, but it crashes or hangs at times.

    Valuable Features

    • Front-end click and record
    • Ability to read in from and output to text files

    Improvements to My Organization

    • Creating test input with tens of thousands of combinations of fields
    • Evolution of regression suite to sanity check new builds

    Room for Improvement

    • Product ungracefully crashes or hangs sometimes
    • Sometimes gets caught out by messages appearing which confuses what is in focus

    Use of Solution

    I used it for five years.

    Deployment Issues

    No issues encountered.

    Stability Issues

    No issues encountered.

    Scalability Issues

    No issues encountered.

    Customer Service and Technical Support

    Customer Service:

    It's good.

    Technical Support:

    It's good.

    Initial Setup

    It's a simple installation, plus a license key.

    Implementation Team

    • Some dev input required to develop app for playback
    • Products own script playback facility is unreliable and we needed something that could kill and relaunch TestComplete

    Pricing, Setup Cost and Licensing

    This was all handled by management.

    Other Solutions Considered

    An evaluation of Microsoft competition found it to not be suitable for front-end testing (it was more suitable to integration testing).

    Disclosure: I am a real user, and this review is based on my own experience and opinions.
    PeerSpot user
    it_user323592 - PeerSpot reviewer
    Software Engineer at a manufacturing company with 51-200 employees
    Vendor
    It has allowed us to make significant product changes, new server deployments, and hosting changes with high confidence.

    What is most valuable?

    The two most valuable features are the manual script library (projects) and Test Execute.

    How has it helped my organization?

    I set up a manual test for new functions and hand it off to a tester (who uses Test Execute). Once the new function test is completed, it is added to regression for all future versions. This has allowed us to make significant product changes, new server deployments, and hosting changes with high confidence.

    What needs improvement?

    From my understanding, the product has changed significantly since v5 was released. I don’t know that it has necessarily improved, for my needs. If I worked in a different shop, then I might need an entirely different feature set compared to what I use now.

    For how long have I used the solution?

    We've used it since May 2007.

    What was my experience with deployment of the solution?

    No issues encountered.

    What do I think about the stability of the solution?

    No issues encountered.

    What do I think about the scalability of the solution?

    No issues encountered.

    How are customer service and technical support?

    Customer Service:

    I can't recall the last time I needed support.

    Technical Support:

    I can't recall the last time I needed support.

    Which solution did I use previously and why did I switch?

    Typically, I used my own method for automation (scripting tool, test data stored in SQL, results stored in SQL, result comparison to baseline with custom script). At the time I purchased TestComplete, the script recording tool wasn’t that great and I didn’t have the skills to write my own scripts. Over time, I used a different automation program and picked up the syntax for its scripting language. The product is currently called WinAutomation, though it had a different name at the time, that I can’t recall.

    How was the initial setup?

    The initial setup was very straightforward. I installed the main app on my local workstation and test execute on five workstations available for testers. We use a shared network drive to store tests.

    What about the implementation team?

    I installed everything myself. It was very straightforward. The real key is knowing what you need for your environment and to adapt the available tools.

    What was our ROI?

    It’s hard to sell testing tools, especially in a company that is owned by someone who thinks of software as a mule to pull his cart. It was a very tough sell and it is hard to quantify in dollars the money saved. However, I can say, with confidence, that there have been over one hundred bugs caught through regression since the software was purchased, which is a great track record for a team this size.

    Disclosure: I am a real user, and this review is based on my own experience and opinions.
    PeerSpot user
    PeerSpot user
    Quality Assurance Practice Leader with 51-200 employees
    Vendor
    It reduced the regression timeline for a complex AUT, but there's an issue object mapping related to the need to modularize the test script code.

    What is most valuable?

    TestComplete is used for testing Windows desktop products; specifically the Embarcadero VCL interfaces created by Delphi/C++. All debug flags must be enabled during compilation that generates an output file required by TestComplete to identify and interact with the UI objects in the application.

    An important feature of TestComplete is the ability to modularize testing. A lot of effort has been put into breaking the test script into reusable functions/methods that can be called from any test. A number of function libraries were created. This enabled reuse of code and kept the projects and project suites small. This is important because the size of these test artifacts have an impact on project loading time.

    How has it helped my organization?

    Test automation with TestComplete significantly (estimated 80%) reduced the regression timeline for a complex AUT with a large number of test cases. Automation with TestComplete has significantly shortened the feedback loop and the timeline to get a release production ready. A secondary benefit is that manual testers have begun thinking more technically about writing tests cases.

    What needs improvement?

    There are two major areas for improvement:

    1. Version control integration embeds information in the project suite, and project files that include a direct reference to the location of the project or project suite in Team Foundation Server. When branching a set of scripts for the next version of the AUT, TFS gets confused about where the file should be mapped into source control. A workaround is to replace the hardcoded paths with relative paths.
    2. There was also an issue object mapping related to the need to modularize the test script code. With the implementation of other automation test suites such at HP UFT, it is possible to have one object map for each function library. TestComplete has a limitation of only one object map per project. In order to modularize the code it was necessary to have a single shared object map used across multiple projects and project suites.

    For how long have I used the solution?

    I have used TestComplete for 20 months. It was chosen as an incumbent toolset could not interact with the product to be tested.

    What was my experience with deployment of the solution?

    Integration with third-party products; specifically Microsoft Team Foundation Server and HP ALM could not be overcome. A custom integration to HP ALM was written using the HP ALM OTA API.

    What do I think about the stability of the solution?

    TestComplete has been a stable product.

    How are customer service and technical support?

    Customer Service can be slow to respond to electronic forms of communication and they do not have a way for a customer to speak directly to customer support. You create a ticket online, and request a phone call.

    The team seems to be very knowledgeable when communication is established.

    Which solution did I use previously and why did I switch?

    TestComplete was added to the toolset. HP UFT is used for automated tests for other products.

    How was the initial setup?

    Setup is straightforward unless third-party tool integration is required. Integration with Microsoft Team Foundation Server is a little complex for initial configuration. Once it is understood the process is repeatable.

    What about the implementation team?

    Implemented in-house. Implementation is not difficult to implement or write tests especially if you have experience with other test automation tools.

    What was our ROI?

    We have not done a ROI calculation. However, automated testing with TestComplete has cut regression test time by months.

    Which other solutions did I evaluate?

    HP UFT was tried but the object recognition did not work with the implementation technology.

    Disclosure: I am a real user, and this review is based on my own experience and opinions.
    PeerSpot user
    it_user303108 - PeerSpot reviewer
    Application Support Specialist at a logistics company with 1,001-5,000 employees
    Vendor
    It provides for automated testing, helps to install plugins and extensions, and we use it as a data-driven testing and debugging tool, but I've only used the free-trial period so far.

    What is most valuable?

    Possibility of the automated testing, installing plugins and extensions, data-driven testing & de-bugger.

    How has it helped my organization?

    I’ve used it only as a free 30-days trial and I'm not sure of how our company is using it.

    What needs improvement?

    I’m not a specialized tester and I’ve used it just for some private tests and have found in it what I was looking for.

    For how long have I used the solution?

    I've used it for 3 years.

    What was my experience with deployment of the solution?

    I've not had issues with deploying it.

    What do I think about the stability of the solution?

    I've had no issues with stability.

    What do I think about the scalability of the solution?

    I've had no issues with scalability.

    How are customer service and technical support?

    Customer Service:

    I didn't need customer service, so I have no experience with it.

    Technical Support:

    I didn't need technical support, so I have no experience with it.

    Which solution did I use previously and why did I switch?

    We did not use a different solution.

    How was the initial setup?

    It was neither straightforward nor complex.

    What about the implementation team?

    I'm not sure whether we did it in-house or through an outside team.

    What was our ROI?

    I'm not sure.

    Disclosure: I am a real user, and this review is based on my own experience and opinions.
    PeerSpot user
    it_user295749 - PeerSpot reviewer
    Quality Assurance Automation Development Engineer at a manufacturing company with 1,001-5,000 employees
    Vendor
    I would like to see improved language support, but it allows me to input parameters without having to write a quick VB script around it to try it.

    What is most valuable?

    The test tree makes for quick work of choosing which tests you want to run at any time. Whether it's all or nothing, I can re-run parts of a test suite without having to start over again.

    The Object Spy has surpassed any other "spy" tools I have ever used. It allows me to input parameters without having to write a quick Visual Basic script around it to try it. Diving deep into object trees can be made a breeze with the search function.

    What needs improvement?

    I would like to see improved language support, with Python being my

    first choice. Visual Basic is outdated, where C# has been its

    replacement for years.

    For how long have I used the solution?

    This solution has been in use for almost three years, since I started working at this company. It has been mostly stable with static test cases, and only minor maintenance. The updates typically are changing search criteria. We upgraded from v9 about one year ago.

    How are customer service and technical support?

    Technical support has been good. There was an issue where certain installations of TestComplete would give an ambiguous error on startup. After teleconferencing with one of their support specialists, we were able to solve it. Forum support on their community site has also been good.

    Which solution did I use previously and why did I switch?

    No previous solution I know of was implemented prior to my employment here. Attempting to implement the same solution in UFT would have been nearly impossible.

    How was the initial setup?

    The initial setup was straightforward. Licensing was simple, and getting the initial object mapping was painless. Only a high-level base set of object mapping was done.

    What about the implementation team?

    The implementation was done entirely in-house. The projects were made from scratch, where function libraries & test suites were made from existing manual test cases.

    Disclosure: I am a real user, and this review is based on my own experience and opinions.
    PeerSpot user
    it_user291057 - PeerSpot reviewer
    it_user291057QA Automation Engineer with 501-1,000 employees
    Vendor

    TestComplete released Python support with Version 11 earlier this month. They also did a complete revamp of the Version Control functionality. I'm not sure how good the new functionality is, but SmartBear is making an effort to update the product in meaningful ways.

    See all 2 comments
    PeerSpot user
    Assistant Manager - Quality Control at a tech services company with 1,001-5,000 employees
    Consultant
    I'm able to automate functional test cases which I previously did manually, but I've come across difficulties in recognizing dynamically-generated runtime objects.

    Valuable Features:

    • Dynamic Find methods
      • FindChild
      • FindAllChildren
      • FindAll
    • Keys method
    • Project variables
    • Time delay methods

    Improvements to My Organization:

    It helped to automate all the possible functional test cases which I previously used to do by manual testing.

    Room for Improvement:

    I have used it for Web Application automation and sometimes find difficulties while recognizing dynamically-generated runtime object.

    Disclosure: I am a real user, and this review is based on my own experience and opinions.
    PeerSpot user
    it_user294939 - PeerSpot reviewer
    Test Engineering Team Leader at a consultancy with 501-1,000 employees
    Consultant
    After installation, I was able to immediately begin coding regression test for available functionality.

    What is most valuable?

    • Initially support for flex/flash technologies, but since that project ended my primary use is Functional Automation against a web front-end built on AngularJS.
    • I use VBScript for a majority of my tests; it is one of many language choices available when writing scripts.
    • Out of the box support for desktop, mobile and web.

    How has it helped my organization?

    I am the only test engineer on the project, so time is limited in terms of automation development. Using TestComplete I was able to immediately (after installation) begin coding regression tests for available functionality.

    My current suite of tests numbers in the hundreds, and each test involves several UI elements. Depending on how your organization defines a test you may call this one test or many. An example to quantify this, is as follows - Automated Build Viability (smoke test) execution:

    • Number of tests = 75
    • Automated Time to execute = 15 minutes
    • Manual Time to execute = 360 minutes (approx.)
    • Automation time saved = 345 minutes (5+ hours)
    • 2300% improvement

    I have substantially reduced the amount of time it takes for regression and build viability activities.

    What needs improvement?

    I pay for support and maintenance; having used the “support” functions through online forums, I can say there is room for improvement. I would like to have access directly to TestComplete developers as opposed to “users” who have some level of expertise with the platform.

    For how long have I used the solution?

    I've used it for two years.

    What was my experience with deployment of the solution?

    No issues encountered.

    What do I think about the stability of the solution?

    No issues encountered.

    What do I think about the scalability of the solution?

    No issues encountered.

    How are customer service and technical support?

    5/10, there are knowledgeable users in the forums, but I would like to go straight to the TestComplete staff with solution questions. That said, I have had very little need for product or development specific support.

    Which solution did I use previously and why did I switch?

    Other tools were evaluated which supported Flex/Flash web apps, but we chose TestComplete for its rich feature set as well as Smartbear’s available suite of tools outside of Automated Testing.

    How was the initial setup?

    It was very straightforward! You just need to install and begin, no special plugins or out of the ordinary system requirements were necessary.

    What about the implementation team?

    We implemented it in-house. As with any automation platform, carefully planning the intended usage, understanding the technology under test and determining which features will be used are critical to success.

    What was our ROI?

    Given the cost was minimal, the best representation of ROI from my perspective is time saved using automated tests vs. manually execution.

    My current suite of tests numbers in the hundreds, and each test involves several UI elements. Depending on how your organization defines a test you may call this one test or many. An example to quantify this, is as follows - Automated Build Viability (smoke test) execution:

    • Number of tests = 75
    • Automated Time to execute = 15 minutes
    • Manual Time to execute = 360 minutes (approx.)
    • Automation time saved = 345 minutes (5+ hours)
    • 2300% improvement

    What's my experience with pricing, setup cost, and licensing?

    Our recent purchase was a single node locked license for one user which was very inexpensive. That fit our project needs and the automation strategy. The solution resides on inexpensive laptop for development and execution.

    Disclosure: I am a real user, and this review is based on my own experience and opinions.
    PeerSpot user
    PeerSpot user
    Mobile QA Developer at a tech vendor
    Vendor
    It provides direct access to object internal properties and methods, but scripting languages should be extended to support more popular programming languages.

    Valuable Features

    • High UI technologies coverage
    • Direct access to object internal properties and methods

    Improvements to My Organization

    Faster feedback on any product changes and uncovered problems before full regression starts. That can save a lot of time.

    Room for Improvement

    Scripting languages should be extended to support more popular programming languages like Java, C#, Ruby, Python. Many other vendors have such support or moved into such a direction. e.g. SilkTest has OpenAgent since 2008 with support of Java, C#. HP recently published LeanFT which integrates with popular development environments (Eclipse, Visual Studio). IBM solutions were initially designed for Java. Squish supports multiple languages. So, TestComplete should have support of such programming languages apart from VBScript and JScript modifications.

    Use of Solution

    About three years onwards.

    Stability Issues

    TestComplete had problems during long hours runs. In some cases it could crash without leaving any logs. Also, if there were some unhanded exception it could drop the error message waiting for user input. This is not really acceptable for nightly runs and required human assistance. But this problem is rather language-specific

    Customer Service and Technical Support

    Customer Service:

    I never had to speak to them.

    Technical Support:

    I never had to speak to them.

    Initial Setup

    The setup is pretty easy.

    Implementation Team

    We used a vendor team. My advice for implementation is only one, follow the best practices (some of them are available on the vendor site). They were formulated based on real experiences.

    ROI

    ROI was never calculated explicitly, but normally the ROI point was expected to be reached after three to five months of the project.

    Other Solutions Considered

    Mainly, when we did an evaluation and comparison, TestComplete had two big advantages -

    • Good technology support
    • Relatively low price in comparison to other vendor tools
    Disclosure: I am a real user, and this review is based on my own experience and opinions.
    PeerSpot user
    it_user3396 - PeerSpot reviewer
    it_user3396Team Lead at Tata Consultancy Services
    Top 5Real User

    Cool review

    it_user293901 - PeerSpot reviewer
    Principal Software Engineer - Automation at a tech services company with 10,001+ employees
    Consultant
    Our testing team can spend more time testing new features, but there often are issues with web test execution.

    What is most valuable?

    1. Object Browser is the most powerful feature. This has helped me tear apart every single component of the application under test. It's an awesome feature.
    2. Test Editor and Debugger: This has really made my life easy as I can trace memory and stack values very easily.
    3. Data driven testing has been made very easy. I am comfortable using any sort of backend to support data driven testing such as Excel/databases/etc.
    4. It supports using third party DLLs has given me an edge on several occasions.

    How has it helped my organization?

    Previously, we had to wait for the end-of-sprint to make a stable release. Now, rolling out a release is not a problem. As soon as a build is deployed, the scheduler quickly executes User Acceptance Tests, and verifies that the build is fit to be delivered to the clients.

    What needs improvement?

    1. Support for working with COM and ActiveX objects should be more powerful. Currently we have to drill down a lot to find solutions for certain tasks.
    2. Cross browser testing is a bit complex when it comes to scripting. Developers have to take care with it care, or else issues may arise at deployment time.

    For how long have I used the solution?

    Overall, I have used this for more than two years, including v1040. Currently I have been using it hands on for the last year.

    What do I think about the stability of the solution?

    There often are issues with web test execution. Web pages often do not load on the first attempt.

    How are customer service and technical support?

    Customer Service:

    Customer server is absolutely robust. The team respond within 12 hours.

    Technical Support:

    I never needed to use paid technical support as their forum is extremely helpful. The community members respond to issues within 24 hours. It’s really amazing that within the past twelve months, I have never had a technical question I asked remain unanswered.

    Which solution did I use previously and why did I switch?

    I am an expert on the following:

    • Selenium RC and Servers/Java - but this has multiple limitations
    • IBM Rational Functional Tester - it's expensive, and not as powerful as TestComplete
    • AppPerfect - is not in competition with any of the above, it’s pathetic

    How was the initial setup?

    It was straightforward. Both the node locked and floating licenses were easily deployed, and now the whole team is using it.

    At one point, we had to update the hardware of the servers where the licenses were deployed, and this was very easy. We just disabled the licenses from the account and deployed them onto the upgraded servers.

    What about the implementation team?

    I did it myself in-house. It’s very simple. No rocket science needed. Just read the manuals and you can do it.

    What was our ROI?

    We are saving a lot in terms of client satisfaction. We are not yet in a phase where we have savings in terms of a reduced QA team, but our clients are happy that they get bug free software. Moreover, our testing team spends more time testing new features, and we are confident enough at build roll out time that all previously developed modules are stable.

    Disclosure: I am a real user, and this review is based on my own experience and opinions.
    PeerSpot user
    it_user293901 - PeerSpot reviewer
    it_user293901Principal Software Engineer - Automation at a tech services company with 10,001+ employees
    Consultant

    Yes you are right. Our automation scripts are extensively deep and code coverage has been taken care of. BUT still, we don't blindly rely on the automation. The scripts only verify that build is fit for release but the scripts don't deliver the build. A human cycle for regression gets carried out in cases where needed.

    See all 2 comments
    PeerSpot user
    QA Automation Engineer with 501-1,000 employees
    Vendor
    A versatile tool for developing and executing automated test suites.

    What is most valuable?

    To me, the Object Browser has proven to be the most valuable feature of this product. This tab (and its associated tools) allows the test developer to locate objects within the application to be tested, and retrieve properties and methods that can be used in scripting. This saves a significant amount of time over referring to the application source code for such information. It also makes use of the recording feature periodically to rough out scripts for new tests. The recorded scripts are very fragile, but they can provide a good base for writing stable scripts.

    How has it helped my organization?

    While our test automation initiative is still in its early stages, TestComplete is allowing us to script and run complex business process tests against our desktop applications. Being able to create smaller modules that can be referenced by other scripts or keyword tests allows for a flexible and easy to maintain test suite.

    What needs improvement?

    A major part of the product that needs improvement is the licenser. This piece of the application is known to have issues installing that requires manual, command-line modifications to correct. While this doesn’t occur with all installations, it has been a frequent problem for me. The larger issue with the licenser is that it is incompatible with virtual machines. The applications themselves will run on physical or virtual machines, but they must call out to a physical server which controls the licenses for them.

    The latest version of TestComplete has introduced support for Git, but my initial impression is that it still needs some work before it can replace my VCS tool.

    For how long have I used the solution?

    The product has been in use at the company for about two years and I have been actively using it for about 8 months.

    What do I think about the stability of the solution?

    I haven’t experienced issues that I would attribute directly to the application. There have been some issues with timing and other common problems, but many of those have been corrected through modifications to the tests and the environment.

    How are customer service and technical support?

    My experience with their technical support has been less than satisfactory, but all of my issues were resolved. Of the technicians I have worked with, two of them were very knowledgeable and one of them did little more than email knowledge base articles that had been ruled out in the original issue report. The latter tech had the additional problem of being about 12 hours out of sync so a full day was lost with every email exchange. I received prompt responses and quick resolutions from both of the knowledgeable technicians I have dealt with although one of them was rather gruff and left me feeling like I was bothering her. Overall, SmartBear needs to work on their customer service and technical support.

    Which solution did I use previously and why did I switch?

    This application did not replace a previous automation effort.

    How was the initial setup?

    When deploying new installations of TestComplete, I ran into multiple issues with the licensing engine. The first time I experienced the problem, I contacted support and was directed to a knowledge base article (which I had been unable to find while searching for the error message). I have since bookmarked the article and reference each time I need to do a new install. Once the licenser has installed properly, the configuration and licensing goes smoothly.

    What about the implementation team?

    The implementation was performed in-house, and was quite easy despite the issues with the licenser. The best advice I can offer would be to plan out your testing environment and requirements before you begin. If you plan to use virtual machines for your test systems, make sure you also have a physical box to run the licenser that is dedicated to the lab. Do not attempt to use someone’s PC as it may not always be available when the test machine are trying to run.

    What was our ROI?

    We have not performed an ROI analysis.

    What's my experience with pricing, setup cost, and licensing?

    SmartBear’s product licenses are reasonably priced and generally in line with similar companies and products.

    Which other solutions did I evaluate?

    I reviewed several products before deciding to continue our automation efforts using TestComplete. Among the products reviewed were Telerik TestStudio, Testing Anywhere, and Ranorex.

    What other advice do I have?

    Disclosure: I am a real user, and this review is based on my own experience and opinions.
    PeerSpot user
    PeerSpot user
    Sr. Software Quality Consultant at a tech services company with 501-1,000 employees
    Consultant
    Whereas we did regressions and Smoke manually in the past, we now do automated testing. However, Intellisense demos do not work on methods in the classes.

    What is most valuable?

    • Dynamic Find methods - FindChild, FindAllChildren, and FindAll
    • Keys method
    • Project variables
    • Time delay methods
    • aqString methods
    • Intelisense

    How has it helped my organization?

    Regressions and Smoke was done manually in the past, which has been replaced by Automated Testing.

    What needs improvement?

    The web testing framework of TestComplete is not very helpful for an Automation Engineer. It requires the same effort as Selenium, and. in most cases, Selenium proves to be a better testing tool for web-based testing.

    Web Application Testing has the following issues:

    • Events do not work on Modal Dialog
    • Dynamic Object Capturing is too hard and takes too much time in finding the correct object
    • Intellisense demos do not work on methods in the classes

    Similarly, the mobile testing framework of TestComplete is not of much use either. I do not recommend anyone use this tool for web or mobile testing.

    For how long have I used the solution?

    I have been a user of this tool for five years, including previous versions, on desktop, mobile and the web. I have completed a few major projects with thsi tool and have enhanced numerous test suites created with this tool.

    What do I think about the stability of the solution?

    There have been quite a few issues. Delay methods are not so dynamic in TestComplete, so if a screen hasn't loaded, and the delay is a static delay, then the test moves forward in script, but the previous operation has not been done, resulting in failure of the whole test one after another.

    This also means that TestComplete and TestExecute work in a not so similar manner, causing TestExecute to be practically useless. This happens in the case of HTML5 based web application.

    How are customer service and technical support?

    Customer Service: Level of customer services is satisfactory. Replies are swift and prompt.
    Technical Support: Solutions are scarce. If there is no solution to a problem, the support people will tell you there is no way around it, and then you are stuck. The issues are not resolved in the next releases, mainly because these are not escalated on the backend.

    Which solution did I use previously and why did I switch?

    SilkTest was being used previously with my previous employer. It was abandoned because it was hard to script and advanced features were not being added.

    How was the initial setup?

    The previous solution I used was SilkTest, which was being used by my previous employer. It was abandoned because it was hard to script, and advanced features were not being added.

    What about the implementation team?

    It was done through a vendor. It's better to get it done through vendors, as they are then responsible in the future.

    What's my experience with pricing, setup cost, and licensing?

    It is not a good idea to choose this tool only if it is being used by others. In our case it is an expensive tool, and we should have exerted the same effort by using Selenium, and would have had better results. It appears that if you have a bif web application built in HTML5 and Jquery, and uses Divs and Modal dialogs, then avoid using TestComplete.

    Disclosure: I am a real user, and this review is based on my own experience and opinions.
    PeerSpot user
    it_user291057 - PeerSpot reviewer
    it_user291057QA Automation Engineer with 501-1,000 employees
    Vendor

    My experience with the web support in TestComplete matches yours. As such I have implemented all of our web-based automation using Selenium. I am currently using TestComplete for our desktop applications only.

    Buyer's Guide
    Download our free SmartBear TestComplete Report and get advice and tips from experienced pros sharing their opinions.
    Updated: November 2022
    Buyer's Guide
    Download our free SmartBear TestComplete Report and get advice and tips from experienced pros sharing their opinions.