What is most valuable?
I find that the offline package feature is the most valuable to be able to port suites of test cases to other servers to run.
How has it helped my organization?
It helped us to make decisions on whether to release our software for GA. Helped during testing period to perform regression tests. Valuable testing tool if implemented in your team correctly.
What needs improvement?
Complete Mac Version of Qualitia.
Ability to run scripts on both Mac and Windows without having to change paths and test cases to make things work.
If you create an offline package and copy it to another server to run, you need to make sure that all of the drivers are installed on that server. If the package could include all of the drivers and point to that folder it would be easier to maintain other servers.
Emailing of report functionality would be useful
Recording feature
For how long have I used the solution?
What was my experience with deployment of the solution?
When using team city to run offline packages, Press Keys do not work, thus giving errors. If this was fixed it would help out customers who need to use these. Sometimes there are inconsistent results. Seems that sometimes being able to find an element fails, where as if I did it with selenium did not fail.
What do I think about the stability of the solution?
As far as the software goes, its never crashed on me, as far as test cases, like I said previously, sometimes I get false positives. (Does not happen often, but the reality is that it should not happen at all)
What do I think about the scalability of the solution?
You are able to grow your suites/ projects to whatever size you please.
How are customer service and technical support?
Customer Service:
Great Customer Service
Technical Support:
The Qualitia technical support group is the best part of Qualitia. From the moment we demoed the product, until today, they are available for help basically 24/7. Never once have I had a time where they did not help me solve a problem that they could help out with. Any feature request, they kindly brought up to management.
Which solution did I use previously and why did I switch?
We used a product called Rapise at first (Part of Inflectra product line). This product conveyed inconsistent results when we ran suites. We could not count on it. We had demoed selenium before we demoed and purchased Qualitia, and decided that with the lack of programming/ development skills in our QA department that selenium would not be the best automation option. We decided to look at solutions that were easier for non technical QA members to use.
How was the initial setup?
The setup was actually a little more advanced than I thought it would be. With the help of the Qualitia Technical staff, the process went smoothly. For a user to attempt the install themselves, they might have some issues. I attempted the installation on other environments and was able to install successfully due to the fact that I am a technical person, but a non technical person might experience some issues/ unclarity while installing.
What was our ROI?
ROI has yet to be determined. We have not calculated a real number yet.
What's my experience with pricing, setup cost, and licensing?
I believe the perpetual license cost approximately $6,000 USD. The yearly maintenance fee is approximately $1200 USD. Currently we have 3 licenses. So $12k upfront and about $3800 USD Yearly. Close to $15 USD a day to use Qualitia.
Which other solutions did I evaluate?
We evaluated selenium (which is the best known test automaton tool on the market)
Qualitia does use selenium in the backend, which put us at ease while making a decision, knowing that selenium is being used.
What other advice do I have?
I think Qualitia's scriptless test automation tool is best used on teams that have Manual QA Testers who are looking to start to automate regression tests but are not very technical in nature. I think recognizing the overall strategy of implementing a test automation solution is extremely important when choosing a tool. If the tests are just going to be run from a local machine, then maybe a full solution like this is not what you need. But if you want to run tests on different servers and have them included in your build process, then a solution like Qualitia would help your team out. Just remember that Qualitia's technical support team is really something that will set their product apart from other similar solutions. They really do care about their customers success.
