What needs improvement with SmartBear TestComplete?

While using SmartBear TestComplete, we are fine with the current capabilities, however, it would be beneficial to improve some performance aspects, especially the image comparison feature. Occasionally, image comparison results in failures, possibly due to issues with resolution or font size on the server side, which can be challenging to identify.

The recording function, when using Python, could be improved, as it does not work well in recording testing.

The learning curve of the solution's user interface is a little high for new users.

In scenarios where two of our engineers work on the same task, merging codes is a bit difficult. The aforementioned area can be considered for improvement in the product. Based on what our company spoke with SmartBear TestComplete, I can say that SmartBear is working on resolving the issues related to the product.

Headless testing would be a big improvement. For example, when we run our regression suite, we need to start a virtual machine and run the tests interactively. There isn't a way to run the tests headlessly without logging into the virtual machine. That's the one feature I think would really take TestComplete to the next level. Another feature that would be a valuable addition is that instead of generating MHT files when you enter an incident in Azure DevOps, they should have a way to generate something like an HTML file. MHT files are an outdated technology that requires Internet Explorer to open. With current technologies and Microsoft's updates to Edge, you can end up with MHT files that you can't open. That's a feature they could improve upon. Each tool has its own way of generating reports. Maybe if you go open source with something like the Allure Report and add an Allure Report extension to Azure DevOps, that would provide clearer and nicer reports. I know TestComplete has its own way of implementing alerts, but the issue is that you don't see the screenshots in Allure report when you use it.

Name Mapping feature should be clearer. Whenever I use it, I do not really know what will work and what will not work.

At times, identifying or locating an element can be somewhat challenging. However, in a recent test update, they introduced the Optical Character Recognition (OCR) capability. This introduction has reduced the challenges to some extent, as we can now utilize OCR if the normal method doesn't work. Nevertheless, there is still significant potential for improvement in TestComplete's ability to identify various object elements. I don't have any specific concerns to mention. I have observed significant improvements in TestComplete over the past few years, especially in its support for highly dynamic object elements used in products like Salesforce Dynamics 365. In earlier versions, there were numerous challenges, but the current version is far superior to its predecessors.

The current price of SmartBear TestComplete is an area of concern. The price of SmartBear TestComplete needs to be lowered in the future since competition in the market is very high. Some companies also provide automation tools for free, so SmartBear TestComplete should consider improving its areas attached to costing or pricing while incorporating the latest technologies. I could not connect to BrowserStack with SmartBear TestComplete, whereas I could connect to BrowserStack with TestProject.

At the moment, our focus is automation and API contract validation. We are looking for tools that focus on delivering the aforementioned functionalities rather than more generic ones. In the future releases of SmartBear TestComplete, I think the integration towards Jira could be useful. Reporting and working with test managers is important to me without getting into too specific details. A solution needs to generate and shift through data results while improving reporting and automated reporting of errors to make it simpler. I want the solution to be more usable since someone with much to do can't afford to take very long on reporting. SmartBear products generally have a weak link when it comes to integration with other test management tools like Inflectra.

The integration tools could be better. It would be useful if we could use it with other test management tools. We'd like to see the solution add a few more features to the offering.

This solution could be improved by making it easier to visualize where there is a failure without having to look at it in detail. In comparison, Leapwork allows us that visibility and showed us exactly where it failed more easily.

The pricing is the constraint. They don't have any smart features for data-driven testing. The test data-driven framework should be more explanatory, and it should be more advanced. For example, if I want to evaluate some of the web applications, which have a lot of test data entries, and I want to put my data from Excel or that kind of integration. Maybe it has that feature but not a smart feature. In my previous experience, I have used some smart tools with very smooth features. If they could add a data-driven testing feature, it would be ideal.

Right now, the product only supports Windows. We'd like to see it work with other operating systems as well. They should definitely be working with iOS, for example. That should just be a given.

In SmartBear TestComplete the integration with Jenkins could be easier. Additionally, some of the controls could have better customization options. For example, if a grid is used and it contains multiple controls within it, it can be a checkbox, radio button, or any kind of control, the way the Object Spy is operating currently there is a lot of room for improvement. There are a lot of improvements that are required on the Object Spy, such as how we find the locators. That is a feature which I would love to see. It is currently very tightly coupled to the SmartBear ID. There should be a way that they could support the integration with external libraries and make it flexible to integrate with open-source frameworks. This would be a good option to add.

When we try to automate and keep the same script in a few cases, it fails due to control changes. You must have a session open when running in TestComplete, otherwise, your script will fail. When compared to the competitors, it is a little more complicated. During the distribution of our regression test cases, the control IDs are not always recognized correctly. Our frequency control IDs are getting changed. For example, if I develop a script in one mission and then try to deploy the same script in another mission, the script will not be executed properly. Selenium is more efficient than TestComplete at the point of execution, in my experience. In terms of execution, attempting to run a script from multiple missions will result in the script being deployed and failing to work in some of the missions. It would be beneficial if they provided a new open-source library that could be integrated with Selenium. Currently, we must create Java bridge libraries in order to integrate with Selenium. It would be preferable to have a simple APA for consuming the Selenium feature console.

We have something called Name Mapping in TestComplete it is the only Name Mapping that is available for TestComplete. If they allowed two Name Mappings, or run Name Mapping in run time, that would be ideal. TestComplete has its own XPath Identifiers. It auto-identifies the XPath of the application. If that engine could better identify more XPaths automatically and make the process more flexible, that would be better. Code analysis would be helpful. If they could give us a code analysis feature, similar to something called Ranorex Coach (which will give us auto-suggestions of how to use certain things), that would be ideal. In Tricentis, they have a feature whenever we identify an element or automate an element, they give a unique identification element that the tool notes when we map. Here, we don't have a feature that can mention the unique object identification kind of stuff. We need to highlight it on screen and then we need to check it.

I was going to say load and performance but I really didn't get a chance to get into the load and performance part of TestComplete. I can sense there's still a little bit more room for improvement in terms of making the whole testing process easier. Because it was desktop first, it kind of focused on COTS-based testing and API's because everyone's using it at the moment. To bring it up to a 10, I would be looking for the addition of some key functional API testing. What I mean by key is like the video streaming or a consistent flow of data being used and then how much can it handle and how much API testing can it do because that's where the load and performance comes in. The API testing is where I would see a lot more improvement.

The one issue we have is that the tool was installed in a local machine. Now, it's more popular when you use tools that work in online environments. It would be better if it was available on the cloud. We'd like to access it on remote desktops. I don't want to have to install it on every machine. It would be ideal if the solution could offer one subscription for time and one for life. Right now, when you buy the solution, you need to pay for one solution. You receive one set up and you install it and it's just in that one machine. It would be ideal if they could offer one subscription where you can connect to different machines with a group subscription. Or maybe they could offer different pricing. They need to be more flexible.

In the cross-browser domain, it has a few snags with Microsoft Edge and Chrome; although, these problems are not critical. Overall, The internal customer feedback is above average.

I didn't use it very heavily. One issue that I found was that there wasn't a quick way or a button to move Visual Basic scripts to TestComplete. We have a lot of such scripts in our organization, and it would be very useful to have some option to easily move these scripts. It is currently possible to convert these scripts to TestComplete, but it is not easy. I have to write some code, but everything is not available immediately.

They're working on many features. Of course, the roadmap is not news to me, but yes, of course, they are working on different features.

The licensing costs are a little bit high and should be reduced. I would like the ability to automatically distribute web applications to all of the concurrent users.

The test object repository needs to be improved. The hierarchy and the way we identify the objects in different applications, irrespective of technology, needs adjustments. The located and test objects are not as flexible compared to other commercial tools.

The solution needs to extend the possibilities so that we can test on other operating systems, platforms and publications for Android as well as iOS. Right now, Complete can test only on native Android and native iOS applications.

The solution needs more training manuals or some form of online forum for learning. It needs more documentation. Some sort of troubleshooting portal should be available for users. The solution could be faster. It would be good to increase the reusability of the code so that rerun time can be saved. It would be ideal to be able to run multiple scenarios at the same time. Right now, in terms of recording and scripting the solution is not user-friendly. They need to improve on this.

The solution needs Mac OS support. Right now, the solution has only been developed to accommodate Windows OS.

There is a problem with usability because the speed decreases, which could be an issue with scalability because of too many hits on the site. It depends on how distributed their systems are, and how well they can handle multiple connections. The artificial intelligence needs to be improved.

What is currently missing from this solution is better support for mobile testing. This is what we are currently negotiating with them for. Specifically, we would like to see Flutter support in the next release of this solution.

- Increased performance with less memory and cpu usage. - Supports various and different versions of browsers in web testing. - Support to Windows mobile application. (Native iOS and Android apps are supported currently.)