Broadcom Service Virtualization Reviews

This helps my team in virtualizing responses so we are not dependent on the systems when they are down. It also helps in mocking up a new functionality and virtualizing it which may not be yet dev-delivered. We also tried to automate services using LISA, but faced numerous challenges.

We don’t see enough value in virtualizing the services just because it requires a huge amount of maintenance and there is not much support available online. I’d like to see a more simple approach to virtualizing the services.

We've used it for two years.

There were no issues with the deployment.

There have been no issues with the stability.

We've had no issues scaling it.

In my experience, I'd say that technical support is not so good.

We have not used other solutions other than using UFT for automation.

The initial setup was a little complex.

CA LISA was initially brought onsite for training, but after that all development was done onsite.

If you are using this for automation of web services, we found it hard to have a customized solution.

The most valuable feature for us is that it provides configuration settings to make connections to the DB and web services. Because it has pre-written code exposed through an UI, it makes it much easier to configure tests.

It's helped with our automation testing.

• UI testing could be improved.
• Also, it should have predefined plug-ins to integrate with other automation tools to help in tool-chain enablement.

We've used it for four years.

There were no issues with deployment.

Although the initial products were highly unstable, there is still room for improvement in versions 7.5/8/10.

It's scalable in terms of running a performance test because it supports limited virtual users.

5/10 - The worst thing is the training provided by iTKO/CA as it's very costly and not comprehensive.

7/10

We previously used SoapUI and this is much easier to configure.

The initial setup was straightforward.

ROI typically comes over a longer duration (both Service Virtualization and Release Automation), although its significant in terms of overall project cost when taking into account hardware, testing delays, and third-party dependencies.

It’s very expensive and to really come out with an ROI, it takes years.

We compared it to our existing solution at the time, SoapUI.

It’s a good product, but comes with a hefty cost. If there is a better licensing/cost negotiation, then it can work, otherwise it’s not a viable solution. Some of the in-house products can beat this with limited cost investment.

The web services protocol for SOAP requests and responses is the most valuable feature. It is easy to set up the test for them without much coding, thus it is easy to train new testers to work on it in a graphical development environment where one can see the flow of information visually like the TIBCO product. It is easier than SoapUI to follow.

This product has introduced a simple-to-use middleware tool, which helped our team conduct web-services testing.

The most important thing is to have integration with Excel. It has a way to read the Excel sheet but not to write onto it. If I need to generate customized reports in LISA in Excel sheets for management purposes (which can show me number of test cases pass/fail/not started, etc.), I can't do this since you can't ask Service Virtualization to put specific values in particular cells with a particular color, etc. We were able to achieve it only using Java, which meant a lot of scripting, error debugging and rework, and this defeats the purpose of it offering a codeless environment to testers. A simple groovy scripting in SoapUI can achieve all of the above.

The Service Virtualization section needs to be more intelligent in offering a good solution to customers. For example, the virtual service created via the request response pairs needs spoon feeding of responses before the service starts performing. And I am not sure if it serves the purpose of virtual environment till the actual system becomes available. I have yet to see a proper implementation and successful usage of a virtual service through it.

CA should provide a free trial version of the tool (was not available till version 7.0.10) so that clients can try it out before making a purchase.

I've used it for two years.

There were no issues with the deployment.

The product is yet to become stable. I have seen instances when the tests have not been repeatable in LISA when scripting is done in Java. After removing and adding a character to the code, the test starts working again. Sometimes restarting it gets it working. Also, there are caching issues in LISA and it picks up the old values of the variables/parameters/properties in the tests even though the values have been updated and saved in the test before commencing the run. Thus, people prefer using more stable solutions like JMeter and SoapUI.

There were no issues with the scalability.

The customer support for LISA is average with the associates not being reachable all the time. Also, there were no specific communication protocols set up to interact with LISA support team and we were mostly relying on the Confluence forum to have our queries resolved. Also, the search results on Google do not provide answers or hints to the questions LISA testers face, unlike SoapUI and Ready API which have a lot of online support available. One can only rely on the user guide of LISA which does not cover all the scenarios.

The initial setup was straightforward, but the settings procedure, like providing the correct domain name, server name, username and password, did not work the first time they were provided from LISA.

I got it implemented through a vendor team. There are a couple of settings you have to play around with before it starts working.

Choosing this product was a management decision and we would prefer other solutions because in order to achieve a customized solution one would have to do scripting. This tool needs to offer a lot more options before it can be marketed as a codeless environment that can really help an organization chive results with such a testing team that is not good at coding.

QTP can achieve almost everything to do with scripting testing. Other tools like SoapUI/Jmeter being more popular become the obvious choice.

• MQ virtualisation
• Web service virtualisation

Time-to-market has been a lot easier and quicker. QA can start automating from the beginning and developers can do unit testing faster. Third-party integration is not required so you save money by virtualizing third-party services.

Integration with Selenium and GUI testing needs improvement. Supporting more new technologies and existing technologies such as Flex, Salesforce, etc. would also be nice.

I've used it for five years.

There were problems where it doesn't support the technology. We have to write custom functions ourselves. This means our licensing cost is more higher and makes it harder to sell our products to small vendors.

There were no issues with the stability.

There have been no issues scaling the solution.

Customer service is good.

Technical support is good.

I have worked with SoapUI Pro, HP ServiceTest (now UFT), IBM RIT, etc. This product was taken by a customer and it was one of the best products for virtualization.

It takes some time, but after one or two tries it becomes easy.

We used a vendor team who performed nice work. If they supported new technologies it would be better as it's expensive.

Over about a year, we have saved over half a million for our customer.

It's a good product for virtualization, but the price is high although we won’t regret it for this product.

It provides for automated testing of applications, including the following features:

• Wide technology support (SOAP/REST web services, message queues, web UI using Selenium, database queries);
• Support to build modular test cases using reusable blocks (called sub-processes in DevTest);
• Extensibility using Java;
• Integration with Jenkins continuous integration tool; and

• Detailed test execution reports.

We've already created and switched over 10,000 test cases from manual to automated execution. Our QA team can now focus more on complex manual test cases.

I would like to see a completely web-oriented automated test case design (currently need to use desktop application called DevTest Workstation). I'd also like to see IntelliSense-like functionality for custom scripting, as well as improvements to reports for test cases using datasets to quickly point to/locate results for specific data iteration.

I've used it for one year.

We experienced some minor issues, but they've been fixed in the current release.

We experienced some minor issues, but they've been fixed in the current release.

We experienced some minor issues, but they've been fixed in the current release.

Very good, as support person is usually trying to do even more than expected. There’s also a web site by CA called Communities where product users can help each other.

I’ve used HP QTP/UFT tool for other clients, specifically for automated GUI-based testing of desktop applications (.NET/WPF, Stingray, Java Swing). I can’t directly answer question about choosing CA DevTest as I was not in the tool decision group for current clients, but it’s very suitable for SOAP, REST, JMS, DB testing. CA DevTest would not be suitable for GUI-based testing of desktop applications which our current client does not need.

The initial set-up was not complicated thanks to the detailed documentation available.

We used an in-house team with occasional help from the vendor support team.

I believe this product will meet your expectations if you’re looking for automated testing solution for any of the supported technologies. It also offers other major functionalities such as CAI for transaction recording and automated generation of test cases and virtual services which we’re planning to use.

The two most valuable features for us are:

• Service Virtualization – It provides us with the ability to replace real downstream systems with virtual systems.
• Test Automation – It enables us to trigger events and automate test cases.

• It reduces cost when we can replace expensive downstream systems.
• It improves speed of testing when systems are not always available.

The biggest area of improvement is in the licensing. The licensing costs are very high.

I've used it for over four years.

There were no issues with the deployment.

There were no issues with the stability.

There have been no issues scaling it.

Technical support is excellent. I rate is a 9 out of 10.

We originally used Green Hat, but CA Service Virtualization provides greater flexibility.

The product is relatively simple to install and use. The complexities generally relate to system connectivity restrictions within the organization.

We developed using our existing contracting partners with support from the vendor.

Think “outside the box” when you think about how this product can be used. Don’t just think it can emulate systems because it can do much, much more.

• Codeless testing: There are no test scripts to write and maintain in DevTest. It helps to easily design test scripts without any programming knowledge.
• Complete test coverage for heterogeneous, distributed architectures: in a single test environment, DevTest invokes and verifies functionality at every layer of the application - from dynamic web UIs, to web services, ESB/JMS messaging layers, EJB servers, Java objects, databases, file systems, legacy objects and much more.
• Service Virtualization: Virtualization reduces the dependency of third-party services/system and allows us shift-left testing.

Our application requires the integration of many services across many APIs both internal and external. Due to this increased complexity, it is important that the testing of complex systems also starts early as possible. Leaving the testing until the end of product delivery increases project risk and the cost of fixing any defects found during later development stage. Using the power of CA Service Virtualization, it is now possible to perform integration testing on every application build. For application components still under development, service virtualization allows integration testing to occur even earlier in the development cycle simulating the missing parts of the system. Service virtualization also allows driving of error scenarios from both internal and external systems that may be hard or even impossible to produce, even with the real systems.

They still need to improve the reporting functionality for better customization. Currently the report functionality does not allow us to go beyond 5000 records where the display show all the records also more filters needs to provide to generate/customize test reports.

I've been using DevTest 8.4 and Application Insight. 8.4 for the last five years.

There were no issues with the deployment.

There have been no issues with the stability.

There have been no issues scaling it.

The customer service is excellent.

The technical support is excellent.

For API test previously I used Selenium. To create/maintain a test script in Selenium Java coding is required. With DevTest, Java programming knowledge is not mandatory. We can do most of the things using in built DevTest assertions and filters without any scripting.

Setup/installation is straightforward. The product installation guide helps lot.

We did it in-house. We have our own product expert team for implementation.

The CA community will help us with any difficulties with the implementation. You can find it https://communities.ca.com/places.

The ability for us to be able to record the live transactions, and create a virtual services out of that, as well as be able to actually use the request and response to create new virtual services for the back-ends that we don't have access to. We have a bunch of use cases there from that perspective.

With the radio's hardware costs allow the development team to what they call shift left, and start to develop their code against the other back-end services that we may not have available in time. These are pretty much the benefits there that we see.

I would like to see more integration support with Docker for release automation, a few things on service monitoring that's pretty much provided in the prior version of the product. I would like to see that a little bit more usable in the future.

It's really good. We use it to support performance testing as well. That can scale up to tens of thousands of users concurrently.

It's great. Basically, it's designed so that we have a load balancing virtual service environment, so we can scale it up as much as we need to.

That's pretty much the key thing I like about CA compared to other vendors I work with. Their support is great. We have an account manager that's really helpful. He helps us kind of navigate through a different escalation path as well after we open the ticket. We never felt like we were left hanging when we run into any issues with the product.

I think it's a little more technically involved to get it started, but once you get the hang of it you can pretty much do even internal knowledge transfer to any other team. We rely on CA-provided training to a really small group of our team, and then after that we first do our internal training and knowledge transfer to other teams to scale it up.

We quantify what kind of benefits that we have against the cost that we have to pay on the solution. It turns out to be a really good exercise there. It forces us to do the ROI on the technology product that we buy against the benefits that we reap from the product there.

I would rate it an 8/10. From a capabilities perspective it's there. Like I mentioned, the learning curve can be a little high. Usability in the prior version that we used, version 8.2, usability is not great. That's why I give it an eight. It requires a Thick Client, so we need to install that on the machine. It's not the most intuitive UI ever, so we'd like to see it improved a little bit from that perspective. I know that they're coming out with version 9, which is supposed to improve that quite a lot, so I'll be looking forward to that.

What we start off with is that we had what we called a enter of excellence that comprised of a few very technical individual to try out the platform, to install the software, and be the product expert within the organization. Then what we did is that we pretty much scaled the development of the virtual services to other teams and that have more of a knowledge of how the service is supposed to behave, so that the COE doesn't need to be in all of these type of sessions and what not. We actually used the start out small and then use that team to empower other teams to actually be able to develop their virtual services that way.