What is our primary use case?
We tested it, but in the end, we didn't purchase it.
We were using it to orchestrate our releases. We wanted to use it for deployment, builds, backfilling, etc.
We were having a hard time determining who would be the practitioners of the product. In the end, one of the things that drove the cost up was the need for so many people to be able to use the tool. There was probably a little learning curve with some of them, but predominantly, our technical leads and perhaps some senior developers were going to use this solution — also, on the infrastructure side, the DevOps team, too. Someone's got to build the release, so the infrastructure DevOps has to set the environment, and then the leads and the senior developers can build the release and supply all the build information to go with it.
We're evolving quite substantially at the moment. Right now, we're probably doing some releases a couple of times a week and we're also moving into some microservices. That probably would have increased our usage with this solution to some degree.
What is most valuable?
The orchestration from our perspective was nice because right now, we're having to manually launch deployment jobs out of Jenkins. The orchestration, building the release, and then just executing it and managing that pipeline — the orchestration capabilities are great for that.
It had the built-in metrics that we did use in selling to the executives. The metrics were good. It retained build-information so we didn't have to go back through it and dig out some old records to figure out what we've done — that was a really nice feature.
From my perspective, I don't know about the entire team, just automating the backfill off of the production deployment was one benefit.
What needs improvement?
The backfill could be improved, we could automate that. Right now it's subjective — it's up to the lead developer's memory to remember to backfill.
Overall, the price is just too high; especially considering we're in the middle of a pandemic.
There's definitely some overkill in these products. To me, they seem to be just a little over-complicated — I think they could skinny it down. It's just based on how many users so I don't know if the different roles are priced differently. The DevOps people might be in it a lot more than a lead. I don't know if pricing by role can be different.
If I've got a manager that just wants to go in and audit it, does that person have to have a full license, or is it just a read-only and it's a nickel or something?
For how long have I used the solution?
I tested this solution for roughly 30 days.
What do I think about the stability of the solution?
It's a pretty solid tool. It did everything we wanted it to. We could definitely rely on it to be consistent.
What do I think about the scalability of the solution?
We're not a complicated shop — I think we were only testing one or two deployment jobs. We really didn't get into a major release or maintenance update; however, I would imagine it scales easily. As you're going through building a release, it's just based on the number of objects you attach in that pipeline. Overall, I think it's capable of scaling both small and large deployments.
How are customer service and technical support?
I think they were members of the team. I am not sure what all their roles were, but if we had an issue with either installing or configuring the tool, they got the right resources on the phone — somebody always had an answer. We always had a satisfactory interaction with them.
How was the initial setup?
The initial setup was pretty straightforward. There were a lot of terminology differences that we had to overcome. It seems like each vendor and tool, they've got their own set of grammar that they use. Once we understood the terminology, it made it a lot easier. Even if it was a little complicated at times, that's to be expected. Some of that stuff's going to be a little convoluted because you have to define the technical environment.
The tool's got to make some kind of intelligent decision. The learning curve related to the terminology; once you understand the linkage between the configuration and the setup, it's not so bad — I anticipate that with every tool.
What about the implementation team?
XebiaLabs provided all of the resources needed and supplied us with an integrator. They did a fantastic job. They were always readily accessible — they make a good team.
What was our ROI?
On the operation side, we definitely saw a reduction. We went from mainly launching one to two releases at a time to just monitoring a pipeline that's pushing it all through.
No huge gains, it's not like we could reduce staff by 10% or something like that, or eliminate another tool. There weren't any direct gains, it was all on paper. The way I see it, we were just shifting — I am either working in the change ticket, doing what needs to be done, or I'm out there building the release.
What's my experience with pricing, setup cost, and licensing?
It's all subscription-based, based on the number of users. I don't know of any additional costs. I don't think it had any additional costs. We may have had to spit up a server or something, slicing the pie there somewhere, but we were not going to add staff and I don't think we needed any other tools underneath it. The cost was definitely in the six figures for an annual subscription. Just given the current economic position of the stock market at that time, we tend to be a little conservative, so it didn't get approved.
Which other solutions did I evaluate?
I think we looked at a couple of other solutions, like Octopus. We had some product demonstrations from a few other vendors, maybe four or five of them.
It was just a team decision and I think from a conceptual point of view, XebiaLabs XL Release seemed to match our philosophy. I don't know if our philosophy is right or wrong, but that's what we were sticking with. Overall, complexity was an issue, this solution seemed to be less complex, more logical and it fit our philosophy. It wasn't any pricing decisions that directed who we selected with POC.
What other advice do I have?
Even though we're not going to use the tool, I think we captured some ideas and we've moved forward with that. We've automated some of our backfilling, so it was influential and it did cause change within us. Overall, it's been a good experience.
Overall, on a scale from one to ten, I would give XebiaLabs XL Release a rating of eight — nothing is perfect, there is always room for improvement.
Which deployment model are you using for this solution?
On-premises
Disclosure: I am a real user, and this review is based on my own experience and opinions.
