What is most valuable?
The fact that it works on all the different browsers, it easily integrates into all the other tools, and that it looks like it will work with our pipeline 2.0 with a kind of DevOps in mind.
How has it helped my organization?
It helps us go to the true DevOps model, which means we can do shorter cycle times. Go from releasing every month, to every day. It's got a nice clean interface that people don't mind using. It integrates into the developers IDEs, like IntelliJ, which means that everybody gets to work in the tool they want to work in. Then it easily integrates across, so everybody can see the information in any place they want to see it.
What needs improvement?
It's the idea of, how do you share testing microservices across different projects? Today things are separated into different projects. I want to understand what the vision is of how you're supposed to test those across, because everything's interrelated now. You're not just testing for one project or for one application. Many of the applications have shared services. How are they gonna do that?
The next thing is, how do you test deployment objects? So after you're done testing your stories, your features, you want to build a deployment object. I want to take those same tests in automation, and then rerun them, and understand, now I've tested them, but now it's a deployment object, and I've included all the code, and it could include scripts and database changes.
What do I think about the stability of the solution?
The stability is good. Granted it is a new product; we use the SaaS version of it. But it's been relatively stable, and we haven't had too many issues. They are releasing new versions of it almost every six weeks, and we really haven't noticed problems with that.
What do I think about the scalability of the solution?
I imagine it should scale very well. We're using it kind of in a limited basis now, but going forward we're going to use it for a very large project with up to 100 testers. Because we're using the SaaS solution, and because we used the ALM HPE QC product before, and SaaS, I don't see any reason why it can't be scaled the same way. So, not too worried about the scaling.
How are customer service and support?
Yes we use the tech support. We use the ticketing system through the tool. We also talk to our customer success person, and generally speaking, the support's been pretty good. The development team also sometimes reaches out to us and asks us, are there any features we'd like to improve, or if there are any issues with the products. It's pretty interesting. I never talked to the actual developers of a product before, and been asked what we're looking for, so that's pretty cool.
Which solution did I use previously and why did I switch?
I'd gone to the Discover Conference in the summer, and saw them talk about Octane. And they presented it right at the moment that we were really looking for something. Half of our group that do the testing are on Macs, and were having to go through the Citrix clients to get into HPE QC. Everything they said just hit right on what we were trying to do with DevOps, and the fact that they developed it using DevOps principles, and they "drank the Kool-Aid" that they're trying to sell the product to be used for, was very compelling to us.
How was the initial setup?
We did the SaaS version, so if anybody has loaded up Octane SaaS, you just put in your email and request a version, that's basically the setup. So it's just as easy as implementing any kind of open source tool, maybe even easier, because you have built-in support right there. It's extremely easy to do.
Which other solutions did I evaluate?
We looked at JIRA with Zephyr. We built our own internal one in ServiceNow. We also looked at something called TestRail. We went with Octane, just because of the reputation of HPE. It looked like everything they were doing was going the right way. We did an evaluation between Zephyr and Octane, and we really liked the interface in Octane. We just weren't really happy with JIRA, we were already using our own storyboard that we built in ServiceNow, so it really didn't make sense, and Octane just seemed like a much better choice.
What other advice do I have?
When selecting a vendor, I think what's important is being able to scale to enterprise. Somebody that's going to be a partner, and somebody that's flexible and willing to help us. Some of the open source tools are great, but when you're dealing with an enterprise of $10 billion, you really want that real, dedicated support that you get from a dedicated corporation.
Regarding Octane, there are some features that it still needs, but apparently they are in the roadmap. I've given it to our most "open source" kind of DevOps developers, and they said, "Well, it's actually a pretty good tool." And these are the guys it was impossible to get into a test tool before. So just based on the adoption, it just seems like it's going to be much easier. So I'd rate it much higher in that sense.
I would say, you can go out and get a free trial of it, demo it. The integrations are extremely easy. Just try it out in parallel with production, and see how you like it. I think it's something worth looking at, and then understand the roadmap. And if you're truly going through a DevOps transformation, then this is a tool that's gonna align with what you're trying to do better than a lot of the tools out there.
*Disclosure: I am a real user, and this review is based on my own experience and opinions.