Founder and CEO at a computer software company with 1-10 employees
Real User
2021-10-15T17:37:17Z
Oct 15, 2021
It has got to be ROI (Return on Investment).
Most of the other answers have focused on features. Yes, it is important that it does the job you need, supports your applications, ease of use, widely used product, etc. Plus, these days it has built-in AI, to reduce script effort and maintenance.
However, if it doesn’t do those things, the ROI will not be proved. The reality is that tools are cheaper than people. The right tool for the job saves time, which saves money – ROI proved.
This article “7 Mistakes to Avoid When Buying Software to Improve Your Testing” could help you avoid making one or more mistakes.
Ensuring that the tool covers the technological characteristics of its infrastructure, its budget and above all ensuring that it serves to cover E2E throughout the life cycle of functional and non-functional tests, concluding with a PoC to show the capacity of the tool is essential because the client marries the solution shown in their infrastructure and validating that the tool is indeed useful.
Partner - Client Solutions at a tech services company with 51-200 employees
User
2020-02-18T09:52:32Z
Feb 18, 2020
Autonomous scripting based on test case leveraging NLP/AI and AI/ML for maintenance of Test Scripts.
For Complex environment E2E testing across multiple applications, enable use of modular test assets.
Quality Assurance Engineer at a computer software company with 201-500 employees
Real User
2020-02-03T16:11:28Z
Feb 3, 2020
Imho, there are such aspects:
1. Can a Functional Testing Tool (FTT) recognize controls of exactly your app on each of required platforms (MacOS, Andoid etc)?
2. Can a FTT work with apps' dialogs, system dialogs (like "Save file")?
3. Can a FFT recognize right-click menu items?
4. Does the developer of a FTT have good community? It helps with answers in the future.
Also, for commercial usage it is highly important that a FTT has 1) IDE, 2) test recording tool, 3) inspect tool.
If you plan to use a FFT on a large project be sure the FFT can work on few machines simultaneously.
It is important to be able to verify with the product owner that the correct thing is being built. The product owner should be able to execute the SUT against the specifications and have buy in early on what is being built. Other factors include continuous integration builds, code coverage, with automated data-driven tests.
1. Identify all the objects (including ShadowDOM) & support all web components (angular / polymer)
2. Supports all frameworks (Data driven testing, Keyword driven testing, Hybrid ...)
3. Supports mobile & desktop, web & application testing (Windows, Mac, ...)
4. Support for continuous integration (Bamboo, Jenkins, ...)
To me, the most important aspect to look for when evaluating functional testing tool is "It is fit for purpose and future-proof". Once this question is answered, then rest can be taken care of easily.
System Analyst at a tech vendor with 501-1,000 employees
Vendor
2017-03-11T02:45:54Z
Mar 11, 2017
The first thing to look for is its usability. Based on my experience, any user will not use your tool if it is not user friendly. Believe.me they will go back to the good old excel sheet if the GUI is hard to navigate. Also reliability, i find web based tool more reliable.
Programmer / Analyst at a religious institution with 10,001+ employees
Vendor
2017-02-01T17:52:42Z
Feb 1, 2017
Assuming that the tool is competent to test the application, how easy is it to set up, execute and manage test cases? If it requires a lot of support and time to prepare and execute tests, tests cannot be easily prepared and run when needed - thus the tool isn't helpful.
Senior Consultant, Professional Services at Tricentis
Vendor
2016-08-24T19:12:28Z
Aug 24, 2016
The stability of the tool is paramount. If the advertised features don't consistently work and cause frustration then the tool will never be adopted. You would want to make sure they have competent and accessible customer support as well.
Software QA Analyst at a healthcare company with 10,001+ employees
Real User
2016-03-23T03:56:45Z
Mar 23, 2016
To me, the most important aspect to look for when evaluating functional testing tools is how good the tools are in recognizing controls/objects from the Applications Under Test and how well they do in handling Custom Objects or objects not recognized.
The most important aspect to look for when evaluating functional testing tools is how much the tools are able go beyond GUI testing. In other words, to which extent they support non-GUI and API testing approaches.
It has got to be ROI (Return on Investment).
Most of the other answers have focused on features. Yes, it is important that it does the job you need, supports your applications, ease of use, widely used product, etc. Plus, these days it has built-in AI, to reduce script effort and maintenance.
However, if it doesn’t do those things, the ROI will not be proved. The reality is that tools are cheaper than people. The right tool for the job saves time, which saves money – ROI proved.
This article “7 Mistakes to Avoid When Buying Software to Improve Your Testing” could help you avoid making one or more mistakes.
https://www.calleosoftware.co....
Ensuring that the tool covers the technological characteristics of its infrastructure, its budget and above all ensuring that it serves to cover E2E throughout the life cycle of functional and non-functional tests, concluding with a PoC to show the capacity of the tool is essential because the client marries the solution shown in their infrastructure and validating that the tool is indeed useful.
Autonomous scripting based on test case leveraging NLP/AI and AI/ML for maintenance of Test Scripts.
For Complex environment E2E testing across multiple applications, enable use of modular test assets.
Low mainenece and make sure test doesn't break with slightest change.
Verity of feature and scalability of the platform for future use. I also prefer tools with test analytics.
Should support for all platforms and technologies. rich in features like hybrid framework, multiple scripting languages
Environment coverage - Standalone, Mobile, Web, API, Cloud, IOT etc.
Feature Rich - Scripting languages, Hybrid framework, Codeless
Future Ready - AI, ML support
Imho, there are such aspects:
1. Can a Functional Testing Tool (FTT) recognize controls of exactly your app on each of required platforms (MacOS, Andoid etc)?
2. Can a FTT work with apps' dialogs, system dialogs (like "Save file")?
3. Can a FFT recognize right-click menu items?
4. Does the developer of a FTT have good community? It helps with answers in the future.
Also, for commercial usage it is highly important that a FTT has 1) IDE, 2) test recording tool, 3) inspect tool.
If you plan to use a FFT on a large project be sure the FFT can work on few machines simultaneously.
- robust record and play
- maintainability
- customization (user code)
The ability for non-developers to quickly come up to speed and use.
To me, the most important aspect is regression testing and maintainability of the script. Customized Reporting is also one of the important factor.
Simulate user actions and verify the database state
For me, it's integration with other testing tools, such as TDM suites, performance measuring tools, etc.
It is important to be able to verify with the product owner that the correct thing is being built. The product owner should be able to execute the SUT against the specifications and have buy in early on what is being built. Other factors include continuous integration builds, code coverage, with automated data-driven tests.
1. Identify all the objects (including ShadowDOM) & support all web components (angular / polymer)
2. Supports all frameworks (Data driven testing, Keyword driven testing, Hybrid ...)
3. Supports mobile & desktop, web & application testing (Windows, Mac, ...)
4. Support for continuous integration (Bamboo, Jenkins, ...)
To me, the most important aspect to look for when evaluating functional testing tool is "It is fit for purpose and future-proof". Once this question is answered, then rest can be taken care of easily.
The first thing to look for is its usability. Based on my experience, any user will not use your tool if it is not user friendly. Believe.me they will go back to the good old excel sheet if the GUI is hard to navigate. Also reliability, i find web based tool more reliable.
Assuming that the tool is competent to test the application, how easy is it to set up, execute and manage test cases? If it requires a lot of support and time to prepare and execute tests, tests cannot be easily prepared and run when needed - thus the tool isn't helpful.
The stability of the tool is paramount. If the advertised features don't consistently work and cause frustration then the tool will never be adopted. You would want to make sure they have competent and accessible customer support as well.
Support for running across browsers/devices and robustness
To me, the most important aspect to look for when evaluating functional testing tools is how good the tools are in recognizing controls/objects from the Applications Under Test and how well they do in handling Custom Objects or objects not recognized.
The most important aspect to look for when evaluating functional testing tools is how much the tools are able go beyond GUI testing. In other words, to which extent they support non-GUI and API testing approaches.