OpenText ALM / Quality Center Reviews

The test generation ability, coverage reporting, and risk prioritization. From a test and defect management perspective, there are few that can compare for delivery in a traditional/waterfall project).

For our clients it provides a centralized means of accessing data, monitoring progress, and creating a singular view on the test status within the business. A number have used it to perform comparative analysis to determine the resultant cost saving and benefits achieved as a result of using the application, and determining process improvement.

The product could do with more native integration for agile projects, a greatly reduced cost model and closer integration with products that are non-HP.

On and off for various customers for 10 years (i.e. going back to when it was Mercury Quality Center).

Not normally.

No. This is an enterprise application and scales accordingly.

Seven out of 10.

We use multiple solutions depending on the needs of our client at the time.

Straightforward.

HPE is constantly updating its licensing to be competitive, but as a whole the pricing for ALM is very high for the local market and, as such, we see a larger adoption overseas.

Yes.

Perform the recommended due diligence when adopting any new tool and ensure that the tool adoption correctly addresses the problem being experienced. This is a good tool and if correctly implemented will provide a solution to a number of delivery focused issues within a business.

What I love about it is that it can be used for the entire lifecycle (from requirements to release) and the dashboard is great for reporting.

The organization I am currently with does not use ALM. I recently moved. At AMC, we began using it during my tenure. When I got there, it was shelfware and they were using spreadsheets. ALM improved our ability to share reports which were understandable to all levels of business. Also having all the bugs for all the projects in one place instead of in a spreadsheet allowed more visibility of issues to all teams. Having test cases in ALM also helped us with spending less time doing rework. Being able to reuse the tests instead of creating new tests saved time.

The product could be more user-friendly. It's pretty easy when you know what to do, but it's not intuitive learning that all the features are available. For example, it’s not obvious that you can update a set of test cases with multi-select cut/paste. This was not available in its previous versions.
A lot of people also think all you get is what you see out-of-the-box. A lot of inexperienced users don’t know almost everything is configurable.

16 years.

No.

No.

I rarely needed any as I am an expert administrator.

Yes, spreadsheets. We switched because we grew from three people to 10 and needed a tool. ALM was one most of the team had used at other companies.

Complex. There are many steps to follow to set everything up for an upgrade, and if you make a mistake, it's catastrophic.

It is definitely overpriced. Most of the cost is for support that you rarely need if you have an onsite admin.

No.

It's better suited as an enterprise tool that can support the licensing cost, instead of for a small shop.

I have used HPE ALM before and the feature which I loved the most is the ability to create Excel reports and inject macros to format your exported file/report.

Tests and test flows are organized properly.

Notifications to developers/testers is also a plus!

Probably more fields to customize.

Almost nine years, and I've been working it for 10 years now.

I have no experience setting up the server. I have only been an admin.

The product is pretty stable.

No.

Customer service is good. They respond on time to inquiries and requests.

The technical guys at HPE are good and knowledgeable on the solutions provided.

I have no experience using other test management softwares.

Initial setup is complex when you want to customize the product.

In-house.

We didn't look through the ROI of using this product. We look more into the ROI of automated tests.

Pricing is high compared to other solutions.

We're looking into using JIRA, but skills-wise we have more resources which are knowledgeable in using HPE ALM.

ALM: You cannot just say one feature is most important. You get the most value
using all modules from Management to Defects. When you use the tool end-to-
end, you can pull efficient project reports (especially scorecards) from the
Dashboard. So everything is integrated and only then you can evaluate the tool
fairly. ALM is very flexible and each module can be used independently, but
when you do that you are only using the tool as storage, not as a test
management tool.

UFT: It became much more stable tool in terms of object recognition over the
years. It is easy to use as long as the user has basic software development
knowledge and understands that the software automation process is not just a
record/playback.

ALM: We currently successfully manage all testing projects due to ALM’s invaluable capabilities, which are listed below:

• Built on best practices with a flexible structure, organization, and documentation for all phases of the application testing process.
• Serves as a central repository for all testing assets and provides a clear foundation for the entire testing process.
• Establishes seamless integration and smooth information flow from one stage of the testing process to the next.
• Supports the analysis of test data and coverage statistics, to provide a clear picture of an application’s accuracy and quality at each point in its life-cycle.
• Supports communication and collaboration among distributed testing teams.
• Reduces time needed to create test execution summary reports.
• Reduces the time needed to write and execute manual tests with HPE Sprinter tool.
• Users can capture their actions automatically as steps in a formal test.

UFT: We save time executing smoke and regression tests. We also use UFT to create test data.

I would like to see better Reporting functionality especially more sophisticated graphs, for example Actual vs. Planned or high level progress graphs using indicators like traffic lights. I would like to see more sophisticated or flexible Dashboard views, such as editing and resizing. I use scorecards and pull them into the Project Reports using customized templates. Scorecards can only be refreshed from the Site Admin, which then test leads has to depend on the ALM Admin to refresh the reports if needed after the scheduled auto run. There should be ability to refresh scorecards (execute KPIs) from the project itself or give more frequent auto refresh option like even every 5 min. This is a really burden on the team.

I would like to see Requirements mapped to test steps so we can combine multiple requirements validation in to one test case but map the verification steps to the associated requirements, so if the step fails only fails one requirement not all. When we are operating in an Agile world we do not have time to write test cases to capture one-to-one coverage. I know ALM allows many-to-many mapping but we cannot get true requirement pass/fail status if we use many-to-many option. Test configuration option kind of on the right path, but can only be use for data driven test cases, I cannot add design steps. If we can add design steps to a subset of a main test using Test Configuration option, I think we may be able to differentiate individual requirement that was failed without failing all the requirements mapped to the main test case.

We have used this solution for 17 years.

I did not encounter any issues with stability.

I did not encounter any issues with scalability.

In terms of technical support, I usually get solutions to my issues. I did not have any issues to call technical support for a long time.

If you follow the instruction, the setup is straightforward. It definitely requires an experience user to do the installation and setups, especially for upgrades.

I always used ALM and UFT. However, I had training and evaluated IBM JAZZ tools.

We identified an object that was supposed to have a width of 30 characters, but instead had 100,000. No manual tester would have found it, forcing developers to take a second look at all objects which uncovered similar size issues.

While my experience tells me successful automation projects are at 70% coverage of manual test cases, we have been able to hit well into the 90% range of .Net automation with this tool.

So the first impression that hits me about HP UFT 14.0 (formerly QTP) is that it seems to be a whole lot faster! But that could be subjective, as I'm running it on a high end gaming system.

And my second impression was "Oh man, why does it still do THAT?"

Let's review the good stuff:

VBscript language -Easy to learn, surprisingly powerful and extendable.

What I will call the "PDM.DLL feature" provides a list view of any object property and methods at run-time from the code as well as the two other windows.

Built in Excel Datatables for Data-driven design

Revamped beautiful HTML results report with screen and movie capture

Terminal Emulator automation.

Modular design (through functions, ALM components and Flows)

Launch through Jenkins brings CI to the test automation development team.

Can leverage Windows API calls as well as custom AutoItScript for enhanced features.

The wide range of supported current and legacy web technologies, desktop apps, and WebServices testing is by far the most valuable feature.

Even in the case where technology is only partially supported, being able to customize out-of-the-box object methods is another time saver.

For example, we recently started to investigate automation of an AngularJS application. The problem was record/playback (UFT 12.54) did not work on it. However, the Object Spy correctly adds objects to the Object Repository. In addition Descriptive Programming worked from our custom framework. We had a basic login/navigate/verify Proof of Concept test operational with AngularJS Buttons, Links and Images quickly. Minor custom coding was required to override .Set methods of WebEdit objects, and more will be needed to support it's Angular WebTable objects. Totally doable for an experienced level team or user.

There are always new features and more support for new and legacy technology architectures with each release. But the bad news is a growing list of long-standing issues with the product rarely gets addressed. While I have a larger list of issues that make day to day work harder than it needs to be, these are the Top Five that I do wish would capture someone's attention in upcoming releases. All hit the tool's ROI pretty hard.

#1) Jump To Source - The Silent Code Killer: In older QTP versions a double-click on any function in the Toolbox window would take the developer to the function's source code, while a drag from the Toolbox would add it to the code window. Since 12.0 a double-click on a function in UFT's Toolbox window now ADDS the function (same as drag) to the Code window - to whatever random location the cursor happens to be at - even if it is off screen, and it will replace sections of code if it is highlighted. We are not sure what the intention was, but our Best Practice is to avoid the Toolbox window entirely to avoid the real danger of losing days of work and needless bug hunts.

Now Jump to Source is not all bad. A right-click on any function called from a Script takes us to the code source, which is great! But it only half works: in a Library, only for functions declared within the same library. Our advance designs have well over twelve libs so a whole lot of extra time is spent searching the entire project for a function's source on a daily basis.

Lastly, while we can add custom methods to object, a Jump To Source from these methods is long overdue. So again our only option is to search the entire project.

#2) Object Spy: It needs to have multiple instances so that you can compare multiple object properties side-by-side. It lacks a Refresh button, so that automation engineers can quickly identify the property changes of visible and invisible objects.

Or HP could skip to option #3...

#3) Add RegEx integer support for .Height or .Width object properties when retrieving object collections. If this were possible, our framework could return collections that contain only visible objects that have a .height property greater that zero. (Side Note: the .Visible property has not returned a False value for us in nearly five years - a recent developer decision, not a product issue) Eliminating the need to separate the non-visible objects from visible ones would decrease execution time dramatically. (Another side note: Our experiments to RegEx integer-based .Height properties found that we could get a collection of just invisible objects. Exactly the opposite of what we needed.)

#4) The shortcut to a treasure trove of sample code in the latest release 14.0 has been inexplicably removed. This impeeds new users from having an easy time learning the tool's advanced capability. In fact the only users daring enough to go find it now will be you who is reading this review.

#5) Forced Return to Script Code. This again is a no-brainer design flaw. Let's say we run a script and throw an error somewhere deep in our function library. Hey it happens. In prior QTP versions when the Stop button would be clicked the tool would leave you right there at the point where the error occurred to fix. Now in recent releases, UFT always takes us back to the main Script, far from that code area that needed immediate attention.

The product is surprisingly stable. For the flaws that I mention, if stability was an issue we would not have been using it for the better part of two decades.

The only time that the product gets unstable is when you try to do wicked advanced coding. For example, when you are trying to execute dynamic code strings with the Execute command that might not have been generated correctly. My years of experience tells me that if something is going flaky, then it's the developer's fault (me) and not the tool's fault.

Scalability is quick and easy implemented through a framework. Let's say we write a custom function

VerifyValue (oObj, sExpectedValue)

that can compare the expected value, string or integer, to an object's actual Value property. By adding the function as a .VerifyValue method to all WebEdit class object, the functionality is available to all current and future edit field objects regardless if they are in the repository or programmatic descriptions. And it is done with just a single line of code called RegisterUserFunc()

Now Let's say we now want to extend the verification to include value falls within an expected range? Add the code to the VerifyValue() function and all fields support the new functionality.

Scaling to new pages with new objects is not an issue either. The tool allows advanced users to design frameworks that can identify objects on the fly from "Plain English" descriptions ("OK LINK") without using the Object Repository. This may remind you of a Gherkin/Cucumber approach and the tool is certainly flexible enough to design just such a framework.

Orasi's technical support is an A+.

HPE's direct technical support is okay.

Having an issue getting UFT to work with your technology stack? Some versions of QTP used to include an oft overlooked document called the Product Availability Matrix (aka the PAM) telling users exactly which Technology version worked with HP's UFT tool versions. Unfortunately, due to a inexplicable "horse-and-cart" decision, HP has chosen to remove this document from UFT 14.0 install and provide access to it only AFTER users have purchased the product. So I have to buy it to learn how to make it work with my technology stack? Wait, what?

The install/uninstall of the updates are fast and easy. Many of the no-brainer configuration settings are set up correctly out-of-the-box. To be fair, we do have a 20-page document that walks through all the settings to check.

Sure, HP UFT is not free. But consider what you get for that cost: A stable product that is easy to use; the kitchen sink of technology stack support; decades of code (which in many cases actually is free); a version that is a stepping stone to an easier Selenium design; and a support base that is more that just the kindness of strangers.

Want to take UFT to another level for free? Add AutoItScript calls.

If Selenium is your thing then make it easy on yourself with LeanFT, aka UFT Pro. This gets you the easy object recognition of UFT, in Java, plus execution of concurrent test cases in multiple browsers which is a trick standard UFT does not do.

I am always downloading and evaluating other products. SmartBear TestComplete is the next closest option.

Take tool training by someone with years of experience.

HP, Orasi and RTTS all offer the level of training that gets you to the advanced state quickly.

And if you might be longing for the IDE toolset that Microsoft offers in Visual Studio, then look at Test Design Studio from Patterson Consulting to enhance your UFT toolbelt.

Defect Management: This feature allows us to track the defect status for our project, send the notification to the user via email and all the details about the defects can be maintained for the future reference as knowledge center.

Graphs and Dashboard: This is one of the top features, by which we can track the status of the project with ease to keep track of project management and executive reporting.

The live graphs can be exported via public URL's and can be integrated with SharePoint and others as required.

More collaborative, ease of work, and better documentation of all project activities.

Certainly on the UI part, it has to be improved to make it user-friendly and more presentable.

More than three years.

No issues with stability.

No issues with scalability.

Technical Support is great and always responsive to solve the issues.

Nope, the architecture is simple to implement and scale.

The price is a bit high.

Go ahead with this tool. It is for the project management and test execution.

Do consult a few of the other folks using this tool to understand the tricks.

Features that support documentation and test results from Requirements through Test Cases to Scripts to Execution results (pass or fail), then to Defects.

Life was made easy by shifting the MS Office documentation to the product. We deal with our test cases and execution, and the mapping is taken care by the product. You pass/fail a test case and product tells you what requirements/features of the product are good to go into production.

Auto-generation of automation scripts. Integration into the UTF (earlier QTP) has little more scope to improve.

I've used this solution for four years.

No.

No.

A nine out of 10.

No.

Straightforward.

Very expensive.

Some Open Source tools, but did not choose them as they lack support.

Have a skilled person to do the administration. You will love the ease of reporting that results in.

• Requirements Management
• Test Management
• Test Execution
• Defect Management
• Business Model
• Traceability
• Test configuration
• Test Parameters
• Integration with HPE UFT and HPE LoadRunner

All the options are designed so if you are using this tool for Test Management, then you must have them all enabled for efficient use.

It has basic and advanced facilities that allow you to arrange all of your test data in very well arranged, systematic manner.

Graphs can be further improved to manage more requirements at a time.

We have you used this solution for seven years.

We never encountered any issues with stability.

I did not encounter any issues with scalability. It depends on the number of licenses. If you have them all active at the time, it still responds the same way, which is fast.

I would give technical support a rating of 10/10.

We used Excel, but it was so difficult to manage the various Excel files and sheets just like maintaining a register book for your records. ALM allows you to handle them all with just one click.

The initial setup was straightforward as it it required little knowledge of the server to do a simple server-side installation.

If we have huge data and more number of users managing the data, then ALM is the best option for our organization. It also applies if your team is distributed, out-sourced or in-sourced.

We did not evaluate other options. I was aware of their earlier version which had a very good market and feedback, I didn't have to waste my time doing a PoC on any other product.

You must look to implement the minimum system requirements, Configuration, for the server for optimum and efficient use.