OpenText Service Virtualization Reviews

Most importantly for us, it integrates well with other products and we're able to get great support from partners and from HP.

Everyone needs test environments, dead environments, environments for performance testing, a stage environment, etc. We can build them, but they're expensive. With Service Virtualization, it's much less expensive and we can turn them off and on when we need to.

The latest release of Service Virtualization had a great improvement to the management interface. However, there are still a few things missing.

For example, it's missing a feature to view multiple pages of the different projects that have virtualized services. It still requires us to go only one page at a time, and sometimes we virtualize thousands of services. So there should be a feature to allow a choice to view 20 or 50 projects at a time.

Another missing feature is the ability to select all or turn off all listed projects simultaneously. Right now, if we want to see just one project, we have to uncheck the boxes for 19 of them. Being able to "turn off all" or "turn on 1" would be a nice feature.

Finally, we initially only had the need to be able to do SOAP and REST services as well as some JBBC databases. However, we now need support for some Oracle products, which the solution doesn't have right now. HP added support for SAP recently, and we'd like to see the same for Oracle.

It deploys well without issues.

The stability seems to be very solid. Our boxes are idle, oversized, and don't have memory leaks. When they're in use, they're very stable without any performance of memory spikes.

Early on though, we saw some stability issues with the design that would occasionally crash. If we used it for six straight hours with many projects open, it had stability issues. But the design has been improved and the current version seems very stable, both on the server and design side.

We threw tens of thousands of requests per hour at it, totaling hundreds of thousands of requests, but the box seems as if it were just idle as it handled the load. It scales very well.

We plan on virtualizing significantly more things in the next three to six months. We don't feel the need to have to change hardware, buy additional licenses, or add more servers. We feel we're ready to go for quite a while.

Service Virtualization support is great, and I'd rate each tech support specialist highly. It's better than support for, say, LoadRunner, ALM or UFT. Service Virtualization tech support seems to have good logging so they can track what I've done in response to issues. They can see patterns and issues.

This is an important product in both the production and test environments.

With this solution, you don't have to use your production services for implementing other software uses. It saves us time, effort, manpower, and cost, in particular.

Other solutions have more functionalities.

It could needs to be a lot more scalable.

The main thing for us is that it does what we need it to do. There are many other product out there in the marketplace that will claim huge feature sets, but at the end of the day, you're actually looking for a product that just does what you need it to do.

Specifically, it's easy to set up and to get all your instructions from the HP website. You don't have to go through these horrible pre-sales and marketing pitches to get access.

I think a big thing for us is a tighter market and a reduction of wastage. We have a number of end points in the environment that are not terribly reliable. If they failed, then we could end up with tens of hundreds of people suddenly not being able to test or validate.

With Service Virtualization, we can replace them with something that is much more stable. We also get view commonality, so rather than developers and test groups creating their own mocks of environments, they'll have one project service with a tool set that does the same thing.

Also, we've got a high level of compliance now with everyone using the same tool. The downside is, if it's wrong, it's wrong for everybody. The upside is that everybody's testing against the same thing so we have a degree of commonality. When all these developers are doing the unit tests, once we get to the multiple test environment, we're unlikely to see a different problems caused by different issues.

One of the key components of the tool for us is the ability to write scripted rules in JavaScript or C#. As it stands now, the debugging capacity within the tool is fairly limited. I think some formal integration with an external ID that measures videotaping, like eClerx, would be nice in a future release.

There have been no issues with deployment.

In the four months we've used it, I've had no stability issues with very good load on the platform.

We're early in the journey, but so far no issues with scalability.

We've had a relationship now with the R&D team and that's been instrumental for us in making the decision to buy the product, and their involvement in addressing issues. The reality is that we have a legacy IT infrastructure and it's not going to change. The virtualization tool hadn't quite met our needs and they have been very responsive in fixes and patches.

I think the beauty of being able to order the 30 day trial from the HP website means that you can actually get in there and look, unfettered by the sales and marketing people, and actually understand what the product does for you and your organization.

Some of the valuable features that I found in the tool is: we’re a big TIBCO shop so there’s a lot around TIBCO that has been very valuable for us, TIBCO EMS Virtualization. We have a little MQ, MSMQ so that was enticing, haven’t really dabbled in that much yet. The ability of bringing down net services, have it stand up all the virtual endpoints pretty quick and put it into a learning mode to where I can turn it on, let the functional testers test against real services all day, take that snapshot, turn it over to my performance test team, minor tweaks, and now I’m running a full performance test with faked out third parties with minimal development effort on my side, that’s awesome. Another feature that, it’s still in beta but it’s coming up that’s, I’m already starting to leverage is the scripted role that they put out there. It’s Javascript and C#, we’re a heavy C# shop. So I have my choice of developers. I go around and say, hey help me develop something that’s a little smarter, a smarter, server-virtualized service and, you know it’s easy ‘cause I just show it to ‘em, they get it and stand it up, and it’s good.

So some other benefits I actually think I made the tool do something it wasn’t really meant to do but it worked perfect for my environment. So we had a big project to tokenize credit cards. We wanted to get out of the business of hanging on to credit card numbers because of the inherit risk. So we had a massive project to come in tokenize those credit cards. One problem is the vendor much like any other third party comes back and says, well we don’t have a performance environment for this so… And if you performance test against our environment we’re gonna cut your access off because that’s gonna impact other people. So naturally the team turned to me and said, hey what do we have, can we virtualize this? I reviewed what they were doing and it was not a service call, it was actually a website form post. So I turned the call into rest call, slid it into the config file, told it to point to my service virtualization, and it worked just like an HTML post call and it worked perfectly. We were up and running probably within a day with this virtualized uh, endpoint. I couldn’t believe it ‘cause that’s not what the tool was made for, but it worked perfect. So that was one of those added benefits, undocumented features, right?

I had to think about that a bit about what additional features I’d like in service virtualization quite a lot while here, especially when the solution architects are saying hey, we know you. Improvements, more like enhancements, with the rest API would be one. I think another thing I would rather see them implement is better use for their data driven model. Right now they’re using excel spreadsheets. I would like to see them implement a database instead of excel spreadsheets ‘cause sometimes you open excel spreadsheet that the data driven model built and it’s like 200 sheets big, it’s unmanageable. But the SQL server implementation that they provide now is through the-through the scripted roles and C#, and that’s not gonna be great for non-C# developers or JavaScript or you know, something at that point. They need to make it as part of the core tool.

The stability of the product, it’s still a work in progress. There are sometimes that, just honestly I do have to go onto the system and, just the other day we had one service that had been running for months with no problem, and then for some reason it just stopped working. I could’ve reached out to support to get an answer. They’re usually pretty responsive and get it solved for me pretty quickly, but for the sake of expediency because we were blocking people I had to go in and remove and rebuild service virtual-endpoint. The good thing is it took me 30 seconds to do it, the bad thing is I had all my data that I had saved gone. I could get it back, it’s not a big deal but-so stability wise it’s not bulletproof but I haven’t had a major outage until the other day of that one service. That hasn’t caused any issues, there’s been no ripple effects because of that.

I’ve had no problems with scale build. I’ve run, we’re doing a service gateway stress test right now in our production-like environment to understand if this gateway that we purchase is gonna be scalable for our big retail time period, fourth quarter time period. The service gateway said that they virtualize services which their definition of virtual services is different than a testing industry’s description of it. So I had to create my own endpoint, expose it through the gateway, execute stress test against it, and my bottleneck ended up being on the service gateway not on the virtual endpoint, and we were hitting well over 1,600 transactions per second. And my memory footprint on virtual machines. So that’s a virtual machine running service virtualization to was my CPU was probably right around 40 percent or so. So I still had way more scalability to go off this one instance.

So on technical support, now this is where it gets a little tricky because I purchased this through a reseller. So my front line support does come from my reseller. I’ve had to have a few things escalated up to HP Support outside of that tier one level. They’re pretty responsive, they get back pretty quick, they’re very interested. I actually, I was kind of shocked when I was walking around HP Discover and I had a few of their solution architects go oh, we know you! You have success stories and you’ve had issues and I’ve looked into your issues. I’m like I don’t even know you, how do you know me? So it’s nice to know that they you know, that they have this, you know they definitely have a pulse on it They want to improve the product and be on top of things. They’re not looking to just kind of you know, fix it as it goes. You know, it’s just they want, they want to continuously improve it, so.

Going into the purchase of HP Service Virtualization, prior to the purchase we actually ran into quite a few gotchas with third parties, with integrations, internal integrations where teams weren’t quite ready with their services. You know the consuming clients at a point to where they were held up, delayed in their delivery. So we watched their delivery cycles actually go from being in sprint, in lock step, in delivering you know with a, let’s say a three or four week cadence, three or four sprint cadence, it would bunch out. They’d wait ‘til the services team completed all their work, all the functional testing before they would start consuming the services and I looked at that and saw the inefficiencies of it. I kind of pulled the ROI together of getting a tool, started playing with all the various tools, open source tools, other big vendors, and it just turned out that this one worked best in our environment. The biggest savings we’ve had from it is by mocking out our third party calls. We don’t have those you know, the one cent, two cent per call hits anymore, bills coming back to us. We don’t have to worry about performance testing so we can mock it out with the service virtualization. And the biggest gain for me, that was cool but the biggest gain for me was actually having our internal customers be able to dev in sprint with the services team. We save so much time doing that.

I didn’t really have any hiccups with the installation, it’s pretty straightforward. There are a few things, if you want to use some of the new features that are in that beta mode that you have to go in and you have to be very familiar with how to modify specific config files. I’m just naturally inquisitive so I just started digging through the config files on my own. I don’t recommend that unless you’re just that kind of person but other than that just out of the box set-up is so easy to stand up. Maybe the most challenging thing that we had was, overcoming an issue with SQL server implementation because I didn’t realize that each user that accesses service virtualization has to have their own development database instance inside SQL server. So we all shared the same instance and kept overriding everyone’s work. I found it in the documentation, it was there. I’m one of those guys that kind of implements first and reads documents later, and once I found it my temper level went down with the vendor. Even though, I think there’s something that, I think that’s something that can be improved upon. You should be able to share databases, but outside of that everything else went pretty smooth.

We did look at the IBM Solution, I looked at ITKO LISA (CA LISA). We also looked into Ready API by SmartBear which the SmartBear one was gonna be a little too complicated for some of the team members I had. It wasn’t bad, it just was a little too complicated. IBM was coming in kind of at the same price but I guess I kind of had a little HP bias. LISA was just outrageous for our budget. We could not afford LISA, and I even worked with some people that worked at LISA in the Dallas area and said you know, you can’t afford it. It’s not gonna work so when we brought it and did the proof of concept I was told by quite a few people in the industry that said here’s the deal, if you get it in and you can implement the first service that you execute and it fires right up and it works for you, it’s gonna work for everything in your group. I brought it, implement it, it took me probably no less than 30 seconds to stand up a virtual service and start consuming it in my performance environment, I was sold from that.

Some of the criteria I looked at while looking to purchase one of these service virtualization tools was ease of use because I’m gonna have to give it to my org, the QA team owns it, not the developers. The developers, the whole shift left idea for development works partially but they are so slammed to meet tight deadlines that I can’t throw another tool on them. On the other note I don’t have on my team, I have QA analysts so they’re not all completely technical, they dabble a little on it. So I wanted something that they could pretty much do a drag and drop into it. I wanted something that would be easy to discuss with my ops teams to let them understand what I needed deployed to be successful in the environment. I was blown away with how fast the implementation was. I took the demo off the website, deployed it, ran with engineering team, probably spent maybe half a day getting everything I needed stood up, and then shortly thereafter we just all right we’re gonna buy this ‘cause it’s working fantastic. Bought the license, dumped it right on top of it and it still kept going, I mean it was no problem. So ease of use you know the ability to quickly stand something up, you know. When I mean quickly I mean someone’s calling me on the phone saying I’m blocked, I need this. The ability to stand up and test immediately is huge.

Rating - I’d probably put it closer to six or seven because it’s still maturing to be honest, just to be out there about it. I know that they went through a major change between I think the 2.0 release into this 3.0 release. I didn’t buy it prior to that but I’ve been reading up on it. Stability’s been all right, the implementation needs a little assistance like I talked about. I would probably say ease of use really puts my number up. When I was grading it between the other ones that was top notch for me was the ease of use. I’d like to see maybe some more documentation around API’s that are available to it so I don’t have to use other HP tools to integrate with it because I don’t have a whole suite of HP tools at my company. So like your automation for example is not UFT, it’s ran through homegrown automation solutions. So I would-I know the rest API available but it’s not documented and I know was that I can extract that information but I don’t want to do it that way ‘cause that’s a lot of work on my part. So I’ve talked to-talked to their solution architects and they said that yeah, this will become available just hold on, we’re baking it out. So I’m a little impatient I guess. I want-you know, moving into like a dev ops world you kind of want to get away from a lot of manual clicks and make it fully automated as much as possible, and that’s one of the components for me to make it automated.

So when evaluating these tools make sure that it works for the infrastructure that you have. I talked to a former colleague that was researching tools out there. He had alreadypurchased Greenhead, IBM Greenhead, only to find out that he inherited it and he found out that he couldn’t do anything with MQ series. Which that was a pretty big blow for him because he really needed MQ series out. So make sure that your technologies are definitely supported by the tool. Do a very thorough proof of concept on it. I said it was really easy to set up and it was. It was fantastic, but we ran it through different scenarios. I brought out some technical specialist not from HP but my reseller to sit there and show us how it works with TIBCO EMS, how it works with you know WCS services. How is it-how does it work with rest? Do a full soup du jour on it in our environment. That’s probably the biggest thing I would say is, don’t do it on the canned services or websites that always work in demo mode, do it on your stuff. If it works on your infrastructure then it’s probably you’re gonna be in good shape.