When evaluating Load Testing Tools, consider these key features: ease of setup, high scalability, detailed analytics, diverse protocol support, real-time monitoring, reporting capabilities, cost-effectiveness, and reliable customer support.
Ease of setup
High scalability
Detailed analytics
Diverse protocol support
Real-time monitoring
Reporting capabilities
Cost-effectiveness
Reliable customer support
Ease of setup ensures quick deployment and execution of tests, while high scalability allows simulations of extensive loads, meeting different testing needs. Detailed analytics provide valuable insights, helping identify performance bottlenecks early.
Diverse protocol support ensures the tool can mimic varied user interactions. Real-time monitoring offers on-the-spot performance data, useful for immediate troubleshooting. Reporting capabilities facilitate data sharing among teams, making it easier to inform decisions. Assess cost-effectiveness to balance functionality with budget. Reliable customer support is crucial for troubleshooting complex issues swiftly.
Search for a product comparison in Performance Testing Tools
QA Lead at a tech consulting company with 1,001-5,000 employees
Real User
Oct 13, 2015
My Criteria on Tool Selection:
1. Easy to use
2. Support object-oriented/generic programming techniques.
3. Professional Support Services (like having Forums/mails/blogs up to date; providing webinars/Training)
4. License Cost - Both initial and maintenance costs
5. Support multiple Operating Systems. (like windows, Mac)
6. Support cross browsers
7. Fix the issues in the previous version and release the patches ASAP.
8. Easy to execute for everyone.
9. Any testing tool should fulfill all the automation needs (to setup the environment)
It really depends on what you test. In mobile app/game testing, it really is the coverage, capability to automate across devices and get parallel test runs done on those, easy adoption of test automation frameworks, cross-platformity between different OS platforms, and speed.
Addressing the need. I'll initially define what I expect a tool to do and, perhaps, how I think it should work in my project. The research process will be based on finding tools that meet this need (or set of needs).
Underlying this, during my research, I may reject options base on usability for its purpose. Even if a tool technically meets the needs, if a product looks difficult usability, I will reject it. I'd rather have a tool that people will adopt quickly and promote, than something that sits on the shelf, or worse, gets in the way.
Other key factors include:
- implementation costs: time, training, price/maintenance, resources required for operation
- integration with the rest of the tool chain
I'm primarily concerned with the tool's ability to work right "out of the box" with my software under test... with a realistic expectation that we may use several different tools at once to accomplish our testing goals.
QA Manager at a tech vendor with 501-1,000 employees
Vendor
Top 20
Oct 13, 2015
Simplistic and elegant - ease of use. The tools need to fit what we do and not force us to change and fit the tool. ROI is a big factor as far as time investment (price is secondary, unless extreme) .
My evaluation many time consists of "direct use" In other words install it and try to use it. If I have to refer to the docs for simple "surface" usage - I might drop it there and then, because that indicates problems as you try to do more complex actions
Head BI Solution Design & Development at a insurance company with 1,001-5,000 employees
Vendor
Top 20
Oct 13, 2015
There no unique set of requirments/Criteria,
those depent by Size of the Company, current Testing Maturity level, available FTE who can be assign.
than i do agree with Sailaja Kamineni, the good crateria to assess the tool after having correctly and honestly agree with CIO and CFO on the above mentions dependencies are:
1. Easy to use
2. Support object-oriented/generic programming techniques.
3. Professional Support Services (like having Forums/mails/blogs up to date; providing webinars/Training)
4. License Cost - Both initial and maintenance costs
5. Support multiple Operating Systems. (like windows, Mac)
6. Support cross browsers
7. Fix the issues in the previous version and release the patches ASAP.
8. Easy to execute for everyone.
9. Any testing tool should fulfill all the automation needs (to setup the environment)
Software Consultant at a manufacturing company with 501-1,000 employees
Vendor
Oct 8, 2015
My general criteria are:
Longevity/history to the tools vendor,
Test environment compatibility (interface, power source, intended performance requirements),
Cost (initial costs as well as maintenance costs),
Support/Training,
Ease of Use,
Reliability of the test platform,
Ability to trace results to requirements and design.
Sr. Director, Enterprise Apps at a tech company with 1,001-5,000 employees
Real User
Oct 5, 2015
UX and ease of use (really easy of maintenance of test-related content). The testing tool needs to reduce burden on the testers and not make their job more difficult than not having a tool.
IBM Rational Consultant at a tech services company with 501-1,000 employees
MSP
Sep 10, 2015
I will answer and positioning myself as a consultant and a tester,
As my experience, for consultant viewpoint, these point is the main concern:
1. Easy to understand for the tester, even with the less experience testers.
2. Easy to integrating with existing client systems, included but not limited to their Operating Systems, hardware configurations, etc.
3. Easy to be agile, can be adopt any changes in client environments.
4. Can attract the top level management attention, especially with the capability from tools itself.
5. Suitable when aligning with client IT strategy
There, for tester viewpoints, below its the concern:
1. Easily to be understand
2. 24/7 Support
3. Less coding,
4. Auto/manual testing capabilities
QA Automation Manager at a comms service provider with 501-1,000 employees
Vendor
Sep 9, 2015
Firstly, the tool MUST meet the requirement of the project. Secondly, easy to use and reliable. Thirdly, provide full support for a wide range of platforms including OS, web browsers, etc.
IT Supervisor at a university with 10,001+ employees
Real User
Aug 24, 2015
Ease of use. I want something I can delegate to Computer Science undergrads. Flexible and able to handle changes to the interface without breaking huge numbers of tests.
Assistant Manager - Quality Control at a tech services company with 1,001-5,000 employees
Consultant
Aug 21, 2015
Before we reach to a decision of accepting any tool, we need to first see the Application under test, type of testing involved(manually), technologies, no. of functionalities that needs to be automated and development approach. Then we should go for POC, define the POC steps, comparison criteria, select tools for comparison, execute your test, compare your test result and draw your conclusion.
End to End testing should be possible truth every single technology in UI and nonUI.
Non developers should have the possibility to build up and maintenance test cases
Easy to use
Should have a test data management possibility
Should support manually testing also
Should support the test team to find with data combinatorik the right test cases
Technology Program Manager at a non-tech company with 51-200 employees
Vendor
Jul 7, 2015
Ease of use, flexibility - can I expand its basic usage to fit our customized needs, does it work with our current architecture, will it work with our future architecture, does it work across platforms, can I run multiple copies (allow developers to run local copies), does it incorporate good regression testing, does it have good code coverage/metrics that are easily understandable for my developers and business users, does it address secure coding practices.
Be able to see the thing.. testing as a service, without the hassle of installation, setup, configurations and so on is the main key.. after that the features and how they are designed for use..
Software Test/QA Consultant at a tech services company with 51-200 employees
Consultant
Top 20
May 20, 2015
When speaking about test execution and management tools one major criteria would be how well it fits with the life cycle tools and processes that are used within the company or teams to deliver software or services, because as testing does not exist on its own (in most cases) it is necessary to go beyond the usability, easiness of maintenance, learning curve, price and overall availability of testing funcionalities in order to select a facilitator tool that fits the whole of the company and not only the testing needs.
Assoc Quality Analyst at OptumServe Technology Services
Real User
Top 20
Dec 26, 2014
Testing aforesaid is super important either automate or manual. There are lots of factors influences as all the experts’ comments on this topic. It is important to know how efficient for the project needs to accomplish. The tool that should have innumerable dependencies which demonstrates the tester not to rely on one environment such as understanding the coding and scripting invoke and support different languages and services. The security integrated functional testing tool advances the tester to learn the knowledge of how important is the security and vulnerability as a normal tester.
The key is to make sure you have defined WHY you need a tool first. A tool is never the solution. A tool is the way you have to support your solution. Make sure that you have determined the root of your problem to solve and then how the tool will support your response to it.
Quick Take-off for first time automation for any application
Less maintenace due to changes in application and quickly ready to use for testing
Esay GUI for end users for creation, maintenace and reporting
Suppport the wider range of technologies adapted by end user orgnisation
Test Analyst at a tech services company with 51-200 employees
Consultant
Nov 10, 2014
Need to know what my project needs to accomplish,
- Does the tool have relevant features?
- Installation Requirements (memory storage space, os etc)
- Can the tool be integrated with other project apps/ tools?
- Licensing cost?
Ease of use throughout - building the script, execution, analyzing the results, help features, support from the tool administration and lastly cost factor.
Test Consultant / Independant at a tech company with 51-200 employees
Real User
Aug 10, 2014
Depending on your needs of the tool when looking for or selecting a tool, you can usually look at the benefits of the tool;
1. Performing a needs finding exercise to determine your environment, this depends on the context of the environment either functional or automation based and or the methodology being used.
2. After defining your needs does the chosen tool or tools being researched apply you can methodically go through thousands of tools to get to the end result of a pool of 2-5 tools for example for a POC (Proof of Concept).
3. Understanding your staff what other tools have your staff used in other environments this helps natrow the list further, as when selecting a tool you have to think about the end user to enhance the capability.
4. Budget, you can narrow down from thousands of tools to a top 10 list based on you budget needs as a tool could cost x to implement and y to be continually used.
5. Product how the tool will interact with the product as you may be told a certain performance or automation tool may not work, however you can leverage the needs finding exercise to determing further research criterion.
6. Trial or play with a tool, when researching a tool or wanting to gain exposure down load a tool and trial it to see what makes it tick, as you can learn alot more by experience than by the sales information presented
Above all, if trying to research sometimes the classification of the tool helps in the relevant context, i.e. You can narrow the research field by understanding a tool better as some can be under a generalist category like automation or performance you may find a tool could be a specialist web automation tool and this will assis your research efforts even further.
Specifically for test automation tools:
1. That is can automate the software under test. This is very important since I have evaluated many tools and, as we all may agree, every test automation tool has its strengths and weaknesses but the most expensive tool is nothing compared to an open source if the former cannot do critical requirements.
2. How easy it is to find the answers. There are tools out there that are great and seems to be easy to use but when you need to use the help feature, it is so hard to find what you are looking for and once you find it, following it seems to have some ambiguity.
3. As Rusty and Aruna mentioned, ease of use.
4. Developers can also leverage the tool for their own tests. If the tools will be easy for testers but can frustrate the developers, it can be an issue because sometimes it is best to go to your developers first before sending a support ticket to the tool vendor.
5. Integration features - it should be able to integrate with Issue management tools such as JIRA or whatever is already used in the organization. The test result should also be able to be exported and displayed via Confluence or Sharepoint so that stakeholders do not need to go to the one who has the license to check the test results.
How is the usability?
How does it fit in our IT landscape?
Do I have interfaces or options to extend functions? (Rest API?)
Which technologies do I need to test? (one for all or specialized)
How many and what kind of personnel will I need to operate the tool?
Performance Testing Tools are essential for assessing the speed, scalability, and reliability of applications under different loads. These tools help in identifying bottlenecks and ensuring optimal performance.""Performance Testing Tools assist organizations in evaluating application behavior under anticipated workload conditions. They provide insights that aid in optimizing system performance and ensuring applications can handle expected user loads without failures. Employing these tools can...
When evaluating Load Testing Tools, consider these key features: ease of setup, high scalability, detailed analytics, diverse protocol support, real-time monitoring, reporting capabilities, cost-effectiveness, and reliable customer support.
Ease of setup ensures quick deployment and execution of tests, while high scalability allows simulations of extensive loads, meeting different testing needs. Detailed analytics provide valuable insights, helping identify performance bottlenecks early.
Diverse protocol support ensures the tool can mimic varied user interactions. Real-time monitoring offers on-the-spot performance data, useful for immediate troubleshooting. Reporting capabilities facilitate data sharing among teams, making it easier to inform decisions. Assess cost-effectiveness to balance functionality with budget. Reliable customer support is crucial for troubleshooting complex issues swiftly.
My Criteria on Tool Selection:
1. Easy to use
2. Support object-oriented/generic programming techniques.
3. Professional Support Services (like having Forums/mails/blogs up to date; providing webinars/Training)
4. License Cost - Both initial and maintenance costs
5. Support multiple Operating Systems. (like windows, Mac)
6. Support cross browsers
7. Fix the issues in the previous version and release the patches ASAP.
8. Easy to execute for everyone.
9. Any testing tool should fulfill all the automation needs (to setup the environment)
It really depends on what you test. In mobile app/game testing, it really is the coverage, capability to automate across devices and get parallel test runs done on those, easy adoption of test automation frameworks, cross-platformity between different OS platforms, and speed.
Addressing the need. I'll initially define what I expect a tool to do and, perhaps, how I think it should work in my project. The research process will be based on finding tools that meet this need (or set of needs).
Underlying this, during my research, I may reject options base on usability for its purpose. Even if a tool technically meets the needs, if a product looks difficult usability, I will reject it. I'd rather have a tool that people will adopt quickly and promote, than something that sits on the shelf, or worse, gets in the way.
Other key factors include:
- implementation costs: time, training, price/maintenance, resources required for operation
- integration with the rest of the tool chain
handles large loads of data
Test execution
Time
Easy to use
I'm primarily concerned with the tool's ability to work right "out of the box" with my software under test... with a realistic expectation that we may use several different tools at once to accomplish our testing goals.
Simplistic and elegant - ease of use. The tools need to fit what we do and not force us to change and fit the tool. ROI is a big factor as far as time investment (price is secondary, unless extreme) .
My evaluation many time consists of "direct use" In other words install it and try to use it. If I have to refer to the docs for simple "surface" usage - I might drop it there and then, because that indicates problems as you try to do more complex actions
There no unique set of requirments/Criteria,
those depent by Size of the Company, current Testing Maturity level, available FTE who can be assign.
than i do agree with Sailaja Kamineni, the good crateria to assess the tool after having correctly and honestly agree with CIO and CFO on the above mentions dependencies are:
1. Easy to use
2. Support object-oriented/generic programming techniques.
3. Professional Support Services (like having Forums/mails/blogs up to date; providing webinars/Training)
4. License Cost - Both initial and maintenance costs
5. Support multiple Operating Systems. (like windows, Mac)
6. Support cross browsers
7. Fix the issues in the previous version and release the patches ASAP.
8. Easy to execute for everyone.
9. Any testing tool should fulfill all the automation needs (to setup the environment)
My general criteria are:
Longevity/history to the tools vendor,
Test environment compatibility (interface, power source, intended performance requirements),
Cost (initial costs as well as maintenance costs),
Support/Training,
Ease of Use,
Reliability of the test platform,
Ability to trace results to requirements and design.
UX and ease of use (really easy of maintenance of test-related content). The testing tool needs to reduce burden on the testers and not make their job more difficult than not having a tool.
I will answer and positioning myself as a consultant and a tester,
As my experience, for consultant viewpoint, these point is the main concern:
1. Easy to understand for the tester, even with the less experience testers.
2. Easy to integrating with existing client systems, included but not limited to their Operating Systems, hardware configurations, etc.
3. Easy to be agile, can be adopt any changes in client environments.
4. Can attract the top level management attention, especially with the capability from tools itself.
5. Suitable when aligning with client IT strategy
There, for tester viewpoints, below its the concern:
1. Easily to be understand
2. 24/7 Support
3. Less coding,
4. Auto/manual testing capabilities
Firstly, the tool MUST meet the requirement of the project. Secondly, easy to use and reliable. Thirdly, provide full support for a wide range of platforms including OS, web browsers, etc.
Ease of use, covers all vendor products, flexible for all business solutions
Ease of use. I want something I can delegate to Computer Science undergrads. Flexible and able to handle changes to the interface without breaking huge numbers of tests.
Easy to use, Less customization and compatible with all the Operating systems ! It should give a comprehensive reports at the end of every iteration.
Before we reach to a decision of accepting any tool, we need to first see the Application under test, type of testing involved(manually), technologies, no. of functionalities that needs to be automated and development approach. Then we should go for POC, define the POC steps, comparison criteria, select tools for comparison, execute your test, compare your test result and draw your conclusion.
The test tool has to work well with the apps we are testing.
End to End testing should be possible truth every single technology in UI and nonUI.
Non developers should have the possibility to build up and maintenance test cases
Easy to use
Should have a test data management possibility
Should support manually testing also
Should support the test team to find with data combinatorik the right test cases
Ease of use, flexibility - can I expand its basic usage to fit our customized needs, does it work with our current architecture, will it work with our future architecture, does it work across platforms, can I run multiple copies (allow developers to run local copies), does it incorporate good regression testing, does it have good code coverage/metrics that are easily understandable for my developers and business users, does it address secure coding practices.
Nedd to be easy to use. Preferable in the cloud to avoid infrastructure high costs as well licensing.
Be able to see the thing.. testing as a service, without the hassle of installation, setup, configurations and so on is the main key.. after that the features and how they are designed for use..
When speaking about test execution and management tools one major criteria would be how well it fits with the life cycle tools and processes that are used within the company or teams to deliver software or services, because as testing does not exist on its own (in most cases) it is necessary to go beyond the usability, easiness of maintenance, learning curve, price and overall availability of testing funcionalities in order to select a facilitator tool that fits the whole of the company and not only the testing needs.
As less Coding as possible..Ability to create tests as fast as possible
Capacity of program and how many platforms is possible to automate in this tool!
Easy automation, less maintenance, exhaustive reporting, end to end tracking of test cases and defects.
Find one that will test native app on mobile devices.
Testing aforesaid is super important either automate or manual. There are lots of factors influences as all the experts’ comments on this topic. It is important to know how efficient for the project needs to accomplish. The tool that should have innumerable dependencies which demonstrates the tester not to rely on one environment such as understanding the coding and scripting invoke and support different languages and services. The security integrated functional testing tool advances the tester to learn the knowledge of how important is the security and vulnerability as a normal tester.
The key is to make sure you have defined WHY you need a tool first. A tool is never the solution. A tool is the way you have to support your solution. Make sure that you have determined the root of your problem to solve and then how the tool will support your response to it.
Licensing cost, Capabilities and use of easy
Quick Take-off for first time automation for any application
Less maintenace due to changes in application and quickly ready to use for testing
Esay GUI for end users for creation, maintenace and reporting
Suppport the wider range of technologies adapted by end user orgnisation
Need to know what my project needs to accomplish,
- Does the tool have relevant features?
- Installation Requirements (memory storage space, os etc)
- Can the tool be integrated with other project apps/ tools?
- Licensing cost?
Value for money. Effective usage across SDLC phases.
Ease of use throughout - building the script, execution, analyzing the results, help features, support from the tool administration and lastly cost factor.
Depending on your needs of the tool when looking for or selecting a tool, you can usually look at the benefits of the tool;
1. Performing a needs finding exercise to determine your environment, this depends on the context of the environment either functional or automation based and or the methodology being used.
2. After defining your needs does the chosen tool or tools being researched apply you can methodically go through thousands of tools to get to the end result of a pool of 2-5 tools for example for a POC (Proof of Concept).
3. Understanding your staff what other tools have your staff used in other environments this helps natrow the list further, as when selecting a tool you have to think about the end user to enhance the capability.
4. Budget, you can narrow down from thousands of tools to a top 10 list based on you budget needs as a tool could cost x to implement and y to be continually used.
5. Product how the tool will interact with the product as you may be told a certain performance or automation tool may not work, however you can leverage the needs finding exercise to determing further research criterion.
6. Trial or play with a tool, when researching a tool or wanting to gain exposure down load a tool and trial it to see what makes it tick, as you can learn alot more by experience than by the sales information presented
Above all, if trying to research sometimes the classification of the tool helps in the relevant context, i.e. You can narrow the research field by understanding a tool better as some can be under a generalist category like automation or performance you may find a tool could be a specialist web automation tool and this will assis your research efforts even further.
Does the tool meet my needs and does my team have the talent to use the tool.
Specifically for test automation tools:
1. That is can automate the software under test. This is very important since I have evaluated many tools and, as we all may agree, every test automation tool has its strengths and weaknesses but the most expensive tool is nothing compared to an open source if the former cannot do critical requirements.
2. How easy it is to find the answers. There are tools out there that are great and seems to be easy to use but when you need to use the help feature, it is so hard to find what you are looking for and once you find it, following it seems to have some ambiguity.
3. As Rusty and Aruna mentioned, ease of use.
4. Developers can also leverage the tool for their own tests. If the tools will be easy for testers but can frustrate the developers, it can be an issue because sometimes it is best to go to your developers first before sending a support ticket to the tool vendor.
5. Integration features - it should be able to integrate with Issue management tools such as JIRA or whatever is already used in the organization. The test result should also be able to be exported and displayed via Confluence or Sharepoint so that stakeholders do not need to go to the one who has the license to check the test results.
How is the usability?
How does it fit in our IT landscape?
Do I have interfaces or options to extend functions? (Rest API?)
Which technologies do I need to test? (one for all or specialized)
How many and what kind of personnel will I need to operate the tool?
Easy to use, less coding needed, easily costomizable and UI based, so that a less technical person could also be comfirtable to use the same.
Cheers!!