HP ALM 11.0
The ALM market is rapidly maturing as organizations recognize benefits of the holistic approach to managing requirement, builds, tests and releases, and vendors rush to satisfy the demand.
In addition to the numerous commercial offerings one might be able to assemble his own ALM stack with best-of-breed components. There are many tools that excel in specific are of ALM such as requirements management, or defect tracking (e.g. Atlassian Jira); then there are suites that integrate most of the capabilities (e.g. CollabNet, RallySoftware), and there are integrated stacks focused on specific technology (e.g. Microsoft TFS, even with a limited support for Java platform)…
But there are currently only two fully integrated enterprise class technology vendors - HP and IBM – that rule the ALM universe.
Most of the vendors offer the standard set of capabilities: requirements management, defect tracking, release management, IDE integration etc. The one component missing from the most ALM vendors is integrated Quality Assurance, and HP with its acquisition of Mercury Interactive in 2006 leapfrogged every other vendor (including IBM), and currently holds about 40% of the automated testing market with its Quality Center and Performance Center suites (QuickTest Pro, Sprinter, Service Test and LoadRunner) - all integrated into HP ALM 11.
HP ALM also takes top spots in Requirement Management and Integrated Software Quality Suites (Forrester Wave, Gartner's Magic Quadrant), with respectable showing in other areas. Ultimately, selection of the ALM suite would depend on number of criteria - strategic enterprise architecture, technology affinity, maturity of the enterprise, costs, to mention but a few.
In my case, after weighing a number of factors for my own organization, the decision was made in favor of HP ALM, strongly influenced by the suite's capabilities, QA integration, relatively low pricing (into seven figures), and ability to start right away with with preconfigured application (with SaaS deployments option).
What's included'
- Project Planning and tracking
- Release management
- Requirements management (definition and management - including versioning support)
- Quality Management (functional, performance, security)
- Development management (defect tracking, IDE and source code control integration)
HP offers both on premises and SaaS options for its ALM suite. The latter option provides convenience of a fast deployment but you do cede some control; some of the features (e.g. LDAP integration) might present challenges and incur additional costs.
The HP ALM application is a JEE app, with its server portion running on variety of platform - Solaris, HP-UX, AIX, Linux and Windows.
The user access is either through a browser or a desktop client, and, unfortunately, it currently imposes severe limitations - while administration can be performed through any browser (Firefox, Chrome, IE), the main user functionality is IE only (an attempt to access ALM from any other browser would give you a message "Only Internet Explorer 7 and Internet Explorer 8 are supported." - leaving IE 9 and 10 out of equation, as well. Hopefully, this deficiency will be addressed soon. Since the desktop client is Windows only as well, this makes all client activity "Windows only" (same goes for the rest of the tools - QTP, Sprinter etc.)
There are several licensing models available, and selection should be made based upon both your current and future needs, as well as anticipated mode of operation (e.g. concurrent license vs named license vs floating license); it also affects the pricing.
The next big decision is selecting topology of the projects; HP provides a best practices white paper to help you select the most appropriate one for your organization. While the application allows for mix-and-match approach, some capabilities such as sharing artifacts across domains and projects might be affected.
Security considerations are standard for any JEE application: firewalls, DMZ, reverse proxy, security certificates etc. Most of these would be taken care of with SaaS option; it is DIY for on-premises one. The application itself provides robust role-based security, with configurable attributes and customization.
SaaS option comes with a number of benefits as well as drawbacks, and deserves a consideration of its own. Keep in mind that you do lose some control with it: for instance, you won’t be able to have “Site Admin” privilege, the highest one will be TDAdmin - plenty to administer domains, projects and users - but for everything else you’d have to go through “official channels”, namely your HP SaaS representative (like, configuring third party integration, or enabling/disabling Performance/Quality Centers integration); the LDAP integration also becomes a separate integration project instead of a built-in feature and so on. Make sure that you weigh all the options before you choose the deployment model.
Once the application is up and running it is pretty straightforward to administer through a browser of your choice; keep in mind that your users will still have to use IE 7/8 or a desktop client. It is recommended to use the desktop client - especially on newer machines where older versions of IE are hard to come by. In order to provide rich client functionality, both IE and the desktop client will install sizeable .Net libraries, and will require VC++ redistributable to be installed first; for HP Service Test WSE 2.0 SP3 Runtime and MS Access Database Engine 2007 will be installed - a minor inconvenience but something to keep in mind when planning rollout.
The HP provides a number of tutorials (PDF, movie files, online resources) on how to use the application, as well as a number of classes to familiarize yourself with the product. While the tutorials are of high quality, and will get you through the initial steps, the complexity of the suite is such that budgeting some classes - online/on site - is highly recommended. You’ll learn about many best practices, ways to organize your projects for maximum reuse through libraries and templates, configuring KPI and dashboards; consider it as an upfront investment to jumpstart your efforts.
The product is geared towards traditional SDLC but can accommodates various project management styles (e.g. an Agile Accelerator plug-in is provided at additional cost), and does not impose many constraints - for instance, one could decide to start cranking out requirements, and then associate them with release cycles, and another might spend some time honing her release management strategies. The application provides built-in versioning control (an option to be enabled once project is created); use it - if something is not under version control, it does not exist. This goes for requirement, source code, attached documents - everything!
Once created, requirements can be converted into manual tests with a built-in wizard, accessible from pop-up menu. It creates an editable test suite which traces back to original requirements, and could be run - either in Sprinter or through manual runner - almost right away; this might prove to be a significant time saver. One of the most important features to maintain control over your project is traceability matrix - ability to link requirements to tests to test results to source code to defects - would allow you to keep tabs on your project, and drill down to the root cause quicker than otherwise possible.
The application supports variety of reposting options - from configurable graphs (trend, pie, bar etc), to reports in PDF, Microsoft Word and Excel formats. Reports could be assigned public or private folders, and be combined into dashboards of the same visibility.
The HP Quality Center and Performance Center are tightly integrated with HP ALM suite. These are based on LoadRunner (load and stress testing) and QuickTest Pro which, together with Sprint and Service Testing, is known as UFT - Unified Functional Testing. Both QC and PC are integrated into administrative console, and are available to the users based upon assigned role.
The HP ALM suite integrates into Software Configuration Management environments you might have assembled in your organization, including support for SCM suites (e.g. AccuRev, IBM ClearCase) the most popular products in each category: development environments (Eclipse, Microsoft Visual Studio, InteliJ IDEA - no out-of-box integration with Oracle JDeveloper/NetBeans), source code version control (Git, SVN, Perforce, TFS), continuous integration build servers (Jenking/Hudson, TeamCity, Microsoft TFS), source code quality tools (JUnit, NUnit, TestNG), code coverage analysis tools (NCover, Cobertura), static code analysis (Coverity, Fortify). Some of the integration capabilities are fee add-ons, and some require third party software (for example, IBM ClearCase is integrated through OpsHub, and IBM Rational Team Concert/Jira, Rally or Collabnet - with TaskTop connectors)
To sum it up:
Pros:
- Tight integration with quality assurance suite including functional testing, performance testing and security testing.
- Costs (compared to other integrated ALM vendors)
- Provides two deployment options - on premises, and SaaS, and for on-premises gives a choice of the platform - from Unix to Windows.
- Enterprise class application supporting variety of project management styles, fully integrated stack with full traceability between requirements to tests to defects to source code to release cycles.
- A number of integration options with developers’ tools, software configuration management suites, and third party tools (continuous integration, ALM and more).
- Decent support in resolving issues.
Cons:
- Complexity.
- Costs (well into seven figures, depending on configuration options)
- Steep learning curve.
- Limited choice for the desktop clients (Windows only), including both QA and HP ALM proper. Occasional bug requiring user to submit ticket to HP.
- Integration options are limited, some require third party software.
Disclosure: My company does not have a business relationship with this vendor other than being a customer.
Totally agree with you on the licencing costs and complexity. HPE do seem to use these as an easy cash cow but I always maintain you get what you pay for. I've yet to fully understand their way of doing things even after all these years.