When evaluating Application Lifecycle Management suites, what aspects do you think are the most important to look for?

The most important aspects to consider when evaluating Application Lifecycle Management suites include their project management capabilities, version control, automated testing and deployment, collaboration features, and integration with other tools and systems.

When evaluating Application Lifecycle Management (ALM) suites, look for:

1. Integration and Compatibility


2. Collaboration


3. Customization


4. Traceability and Reporting


5. Security


6. Support and Training.

Ensure that the suite can integrate with your existing tools, offers collaboration features, can be customized to meet your needs, provides traceability and reporting, has robust security features, and offers training and support to help your team get up to speed quickly.

There are many considerations when choosing an ALM tool.  Application Lifecycle Management (ALM) is more than just requirements, development and testing, it encompasses the entire lifecycle from the idea conception, through to the
development, testing, deployment, support and ultimately retirement of
systems.

Here's some resources:

https://www.inflectra.com/Spir...

https://www.inflectra.com/Idea...

A fundamental aspect is the traceability of both functional and safety-related requirements. What is needed in these ALM-Suites is an overview that makes it possible to trace and see (with the possibility of an export function, e.g. in HTML format) the technical specifications related to the requirement in the question and the software or firmware modules that have been implemented to realize the requirement in question.

Adaptability, scalability, seamless collaboration with external tools along with near native interface for requirement, design, test and ccm activities within the collaborative suite. Process governed along with level of simplicity covers ALM as suite.

1. Easy of automation
2. Integration with functional and no functional solutions
3. Dashboards for governance

The ability to version control all database artifacts, and the ability to perform CI-triggered, automated database artifact deployment.

- Traceability
- Reliability
- Visibility
- Easy to trace the life cycle of defects
- Test Coverage
- Project Reports
- Plugins

The most important thing is to understand the life cycle model in which you operate your projects and look for ALM tools that support workflow in that model without too much custom configuration.
Secondly, the ability to report progress out of the tool enables project managers to ease out on manual efforts collating information from various team members.

Key points to note:
1. Visibility/Traceability of all artifacts traversing the application lifecycle
2. Ability to report against these artifacts; OOTB canned reports provide an excellent starting point
3. Integration is key to support automation of manual processes; the rest API should be available as well as ootb integration with popular solutions such as Jenkins and JIra.
4. Ease of use and configuration will go a long way to support adoption across the enterprise.

When evaluating ALM suites the following aspects are very important to
evaluate - broken down into 3 categories, Technical/Integrations/General
Product & Vendor. Evaluate the degree to which the ALM candidate satisfies
these criteria:

Technical

 Test case creation, organization & management

 Traceability between test assets (test cases, defects, automation)  and
requirements/use cases/user stories

 Customizable test case template

 Import of test cases

 Customization of test execution and test status workflows

 Test execution status capture (e.g., screen shots, automation status)

 Bi-modal support

 Multiple summary dashboards (e.g., requirements coverage, test case status, test
execution status)

 Multi-format printable, exportable reports (e.g., csv, XML, MS-Excel, MS-Word, etc.)

 Report filtering and Customizable reports

 Metrics

 Role-based security and permissions management

 Notifications/Alerts upon status change

 Ease of Use – tool installation, configuration and administration

Integrations

Supported Client OSes – e.g., Windows, MacOS, Linux/Unix, Mobile OSes – and associated
browsers

Requirements tool integration

Defect Management tool integration

DevOps tool chain integrations (e.g., build & code versioning)

Support for Development IDE integration

Test Automation tool integration (for planning, triggered execution and status)

Enterprise network/SaaS hosting

Seamless integration points (requiring no plug-ins]

Product / Vendor

Licensed/Freeware/Open Source

Licensing structure

Support costs incl. 3rd Party maintenance requirements

Considered Best of Breed?

Technology lifecycle considerations: major/minor releases, security patches

Active user community?

Vendor reputation: Customer satisfaction and service level standards

Ease of Support including documentation

Vendor continuity

I would recommend evaluating the following –

* User-friendliness and ease of use.
* What is the solution’s added-value over SAP pre-existing tools.
* The solution’s footprint on your systems.
* Ease of installation and configuration.
* Does the manufacturer also offers expertise in deciphering the results and how to best extract from them insights for SAP future use.

HI,

Simplicity. Visibility. Intuitive. Added value for the users, not only for the mangers.
Easy to modify and maintain. Good integration between all parts of the suite.

Experience on ALM:

v Have good experience in HP ALM Administrator for HP QC and HP PC
v Have good experience on HP ALM Octane installation
v Having created a number of the project using Site administrator.
v Having provided user access and reset the password using site administrator.
v Have good experience on implemented functional testing in ALM
v Have added the new fields, deleted the fields for system fields in the test plan, defect, requirement and test lab modules.
v Have set up the workflow script in all modules depends on the end user requirement.
v Have configured the new environment.
v Have migrated the projects on DB to another DB.
v Have good experience third-party software installation like Chrome, SAP.
v Have reset the -s password setup on all servers.
v Excellent problem solving, Analytical, communication and interpersonal skills.
v Good understanding of all phases of software test life cycle (STLC). Played Single point of contact for all end user for the test activities and issues.
v Ability to communicate clearly and work independently to understand the application.
v Having basic knowledge of SQL

SIMPLICITY. So many ALMs are bloated and horrible. Just make sure it has the features you NEED and make do without other "flashy" stuff.

#1. Polyvalence: Management of requirements, bug tracking, tasks, tests, code & code review, ci, delivery, integration, collaboration, etc.
#2. Traceability and synergy between components. Connection between all dev items (e.g. requirements linked to tasks, linked to code, linked to tests, linked to delivery, etc.)
#3. Independence from vendor. Long-term availability if vendor discontinues the product, Ability to refuse an indecent price augmentation. The solution is probably open-source products.

- Simplicity since the thing is to be used and run by people and they should easily learn how to use it.
- Configuration flexibility as each company may have special requirements or rules they have to adhere to.
- Interfaceability and still keeping the integrity complete since we are integrating more than one system.
- Quict setup reporting to give us the possibility to extract theright information for the stakeholders in different aspects of application life cycle.
- Process enforcement potential, since it is not just supposed to give overview as an application drawing - it will need to automate parts of the Application Lifecycle Management.

Governance about the life-cycle:
1. Continuous Integration
2. Continuous Testing
3. Continuous Security
4. Continuous Release
5. Continuous Deployment
6. Continuous Operations
7. Continuous Monitoring
Continuous means Automation

You should consider the suite supports both methodologies:
1. Waterfall
2. Agile

Don´t forget the #BizDevSecOps philosophy
Shift to the left means less costs (3x) than the right (30x)

We use MS TFS in BAU and qTest in my current project for our test management.

My key checks/parameters to select an ALM tool

How easy it is to configure and maintain, dont want a team of guys just to maintain the application.
How easy it is to Integrate with other tools in eco system. i have seen people extracting reports and doing manual reporting due to lack of integration in req, test and release tools.
How easy it is to learn and scale for enterprise customer.

One of the things I see when entering this discussion with many potential customers of mine is the focus on the tool itself, and the cost of it. The customer often states that a tool is often so expensive,, why can't I use open source tools instead. Those are for free... Well, to put it short, there is no such thing as a free lunch.

Other things my customers ask is; what can this tool do in terms of all the "traditional approach to test management"? Defect tracking? Check, test management? Check, etc. But what about other requirements, that necessarily don't come up in a requirements gathering, like integration with build systems, trigger automation, complete E2E visibility? Dependent on your organisation, I believe that your requirements for the traditional questions are answered by most vendors. The differentiator is how easy can you implement this into your organisation, and what will the cost be running the solution over time (including training, opeations, usage etc).

With these things in mind, I would state the follwing:
1) What is the cost of running the solution? Licenses, hardware, services, support, training etc.
2) What can you save by implementing the solution (do a proper ROI analysis...)
- Reduced number of testers by implementing more test automation
- Reduce hours for reporting
- Improve organisational visibility for project and product risk
- Increase speed and velocity for deliveries (DevOps)
3) Calculate the delta between 1 and 2.

If you come out negative, don't invest. If you come out positive, invest.

End to End Traceability :

* Communication to stay current with on-going changes
* Establish traceability through the use of integrated functionality
* Focus on the development, management and reuse of granular details, that can be shared in a collaborative environment
* The ability to manage and report on release activities, through the use of dashboard widgets and automated reports
* Transparency and communication across delivery teams, demonstrating how clients requirements were met and implemented.

· Understand the metrics the customer “lives and dies by”

· Identify the pain or “compelling event” to best understand the potential solution(s)

· How will the decision be made (ID “economic buyer”) and what are the processes to reach a “buy” decision

· Who has final say on the budget (is it Capex or Opex) and who is the “champion” we need to sell

· What is the procurement process

I would check the following:
1) It is appropriate for the methodologies you are using now, and plan to use in the future (agile, scrum, kanban, waterfall, etc.)
2) It integrates with the other tools you are using or plan on using (unit tests, SCM, IDEs, CD/CI, etc.)
3) It is usable with your current organization size, and will scale to meet your projected needs.

1) Fully integrated, easy to use with common touch and feel
2) Promotes real-time collaboration (not checking in / out documents)
3) Full traceability

Yes, agreed with most of the comments here. Ideally, I'd reach out to folks at OpsHub (www.opshub.com) and see if they already support a particular system. They support most of the common systems but they can also add a new connector in less than 2 months and that's really amazing.

built in integration with other tools, as seamless as possible.

As a Test Engineer i would consider customer Friendly,Defect Analysis,collaboration and Integration

an ALM system needs to facilitate 2 aspects:
1.team efficiency and collaboration
2.project data accessibility, reporting and communication within the organization (at all level)

the first point can be ensured via a tool that simplifies the SW dev lifecycle processes, ensures a proper communication within the team, represents a focal point (as a repository) along the entire project

the second point can be ensured via a customizable dashboard, a proper query system and with a good data analyzer tool.

How quickly a user can adopt to the tool, user friendly, integration with other tools for additional functionalities

Ease of use, user friendly, integration with other tools for additional functionalities

Integration with other tools

Scalability and Integration with other tools

Integration with client systems

Integration into Continuous Delivery / DevOps tool chain

collaboration and communication among team members, project management traceability, visibility and stability

Cloud readiness

Ease of the entire process and feasibility to integrate tests with automation tools

Your team should understand selected ALM process well, everyone should be on the same page.
Also for me the most important thing is predictability, you should know future for being successful.
I used customized Agile 2.0 process with TFS 2015 on my recent project, and explain to my team how it works. As a result we have done the work plans for 100%

I would say: scalability, integration to other tools like git/Gerrit, usability, easy collaboration, possible to customize

Integration with other tools, traceability, customization and implementation process

Integration with other tools, flexibility and adaptability to various conditions, maintainability and few other -ility. The first thing is to be aware about your needs and expectations - then check if evaluated tool meets the criteria.

Beyond covering the full product life cycle, usability is the most important. If team members do not use it then there is no collaboration and the only one using it is the project manager. If this happens spread sheets work great.

Integratable, lightweight, plaftform agnostic. The other usuals being ease of customized reporting, defect tracking, versionable, robust backup mechanism

High usability and customization, easy integration with external tools, catering to projects of different natures (iterative/waterfall)

Customization, Reporting, Traceability, Integration with other Tools, enforce Company Standards,.

I understand Collaboration, Integration, Dash – board &Status Reporting, Defect Analysis, Traceability, KPIs and Scalability are import – One important element is “Portfolio Management” which I believe Rally is on top of the list in comparison with

Atlassian JIRA
Microsoft TFS
CollabNet CollabNet TeamForge
IBM IBM Rational Team Concert (RTC)
IBM Rational ALM
HP HP ALM
HP Agile Manager
Parasoft Parasoft Concerto

After that being acquired by CA it’s even more strategic to integrate CA PPM to track financials and grid tools for Test data management to implement Agile and achieve end to end ALM solution

1) Integrity as we need to join multiple systems into one
2) Configuration flexibility as each project may have special requirements and system should meet those needs
3) Informativity as we need to get all necessary information on different aspects of application life cycle
4) Simplicity as entire solution is used by people and they should find it easy to use

There's a myriad of tools out there to use. I guess that's a good thing because every Organization is different. But before I go into my criteria for choosing one, I'd like to preface with saying:
The agility, maturity, and efficiency of any SDLC must start with the people first, not the tools. No tool can ultimately fix cultural issues. We can implement any tool we want and make it hum like we want, but if the culture of the developers, quality engineers, product owners, operations engineers, or any other group that is involved in making software development happen isn't of the same mind when it comes to the goals of delivering quality software, then those tools will ultimately fail.

That being said, here are a few that come to mind...

1) How well does it integrate with the rest of the tools being used to make the release pipeline as seamless as possible? Nobody wants to sign in to different systems with different interfaces and different workflows just to figure out the status of a bug fix in the pipeline. Making the flow from ticket creation to production as easy as possible is of utmost importance.

2) How does it add value to the Organization? Does it solve a problem we're facing or will face? All too often, I see investments made in new tools that don't really add value or solve an actual problem. Typically, the situation is someone saw a shiny new object and really wanted to use it. 3 months into the investment and users are having new problems, complaining that the new tool just does the same thing differently and they "liked how the old system worked better". Making sure it's actually bringing true value to the organization is important.

3) Does it provide good visibility to all stakeholders? IT engineers tend to forget that not everyone involved in software development knows how to decipher console logs. The tool must provide an easy way to translate to the business where features, bug fixes, and enhancements stand in the pipeline. Because at the end of the day, pushing new software is a business decision.

4) Does it have the ability to scale? Basically, if we go from 3 developers building 5 apps to 50 developers building and deploying 300 apps within a year, is the tool going to be able to handle the load and that type of scaling? On top of that, does it provide an easy way to manage and maintain the system?

I think that when you look for an ALM you should consider almost everything because it all matters since your complete work will be controlled by this ALM tool, but based on my humble experience maybe the following is the most
1. Easy to install and configure
if your ALM tool is a rocket science then your life is a mess since you will have to dig real deep to change one configuration key.

2. Provided by a strong vendor who can supports you whenever needed.
One day or another you will need support by the vendor so you will need to have a great customer support you can depend on.

3. Easy to integrate with other tools.
your next ALM tool will be controlling everything around you, which is good if it can completely apply your Application Life Cycle, so having an ALM that to be easy integrated with other tools like Development IDE, Reporting Tools, and other things like your mailing system is a must.

4. Easy to use by having a simple yet strong GUI by which you can reach your goal with minimal efforts.
if your ALM makes it easy for you to do the needed job with minimum clicks and efforts then your time is saved, and time is something we all love to save.

5. Improve collaboration between team members
One of the most needed features by ALM is enhance team members collaboration because simply they are your company's future, the more they do things together the most you benefit.

6. Has a strong reporting capabilities, or at least can easily be integrated with a reporting tool.
Your ALM should always help you see if your on Track or not, then consider reporting always.

According to me a complete dashboard which can provide comprehensive details about ROI, release managment, allow 3rd party tool integration, easy to undersnad and simple to use.

Accessibility, Integration with other tools, speed, ease of use.

Integration with different tools and end to end reporting

1. Accessibility: what browsers, Operating Systems this tool supports.
2. Integration with other tools.
3. Level of support to my application Life Cycle.
4. Level of Reporting and Reporting Customization.

1.Traceability: bug-requirement-code-release-test-bug ...
2.Flexability: can utilize different testing tools, source control tools
3.Must be simple and fast

Reporting features, Integration with tools in the market, Covering entire test life cycle

The most important criteria is simplicity. Yes it needs to be many other things, but if the application is complex and painful to use then getting users to accept it and actually use it is going to be a problem. If no one will use the tool, it does not matter how good the other functions are.

Collaboration, Analysis Reports, Requirement Elicitation tools, connections with 3rd party plugins.

Short time to root cause analysis is my primary requirement. Everything else is secondary, but supports that primary focus.

Right from the requirements gathering and source control to project management and monitoring

I consider the following criteria for ALM tools: Robust Requirements Management as well as Test Planning and Execution components all bound by seamless traceability mechanisms with a big footnote on ability to triage tests for automation. Reporting is an absolute must especially features to create "civilian" looking reports. Finally, defect tracking and setting up releases within the ALM tool are key. Take all of this and ensure the tool has connectivity to external tools especially TFS and Visual Studio (or Jama, Jira, Project management tools, etc), all make for acceptability criteria for ALM i look for.

Compatible with external tools, easy to integrate, creating reports in understandable format

I consider, reporting features and traceability. You can trace almost everything in a development process of an application (starting from requirement phase to production phase).

Entry criteria: full management support, organization maturity
Acceptance Criteria: THE Requirements validated

Collaboration, Integration, Synergies between components such as testing against requirements, ease of use, traceability and reporting.

KPIs, Integracion, Scalability

Traceability, Reporting, Complexity, Interfaceability, Process enforcement potential, Cost

Flexibility to interact with external tools, seamless integration within tools from the same vendor, comperhensive yet simple reporting and navigation.

Should have complete solutions in one pack and It should easily integrate with other tools. Should have good online support, tutorial and documentation.

Integration with other tools and detail reporting Each and every stage of application lifecycle

Collaboration , Status Reporting , Defect Analysis , Supported life cycle

I consider integration (both within the suite and with external tooling) and complete application chain coverage essential for ALM suites.

Collaboration