Parasoft SOAtest Reviews

Repeatable automated regression tests in a SOA architecture IT landscape, for several SOAP/REST webservices including Web Gui end-to-end testing, all scenario-based for 10+ applications. Also, scheduled for test runs at night, using Jenkins.

It provides us with fast(er) test results, more predictable results and automatic (regression) test runs outside the employee workstations. This tooling made us more agile and more ready for continuous integration. We use the night hours to run different regression tests over systems, for which we only need to check the results in the morning.

• Fast and easy automated tests for UI
• Interfaces (SOAP, REST)
• Additional database queries
• Easy validations on responses (soap, rest, json, sql, Gui)

Most valuable for us is the combination of all these items in one solution for us. Broad user story scenario's from A to Z, though a chain of systems, including test data prerequisites.
The next most valuable benefit over the past year is the possibility to use the tooling for continuous testing purposes, using the server-side run option 'as a web service' for (nightly) batch runs.

In general, its user-friendliness needs a little improvement since there are a few bugs that still remain. It needs to be (possibly) more lightweight and have more validation options. The bulk changes in the test cases are not always easy. But all in all, these are minor details compared to the benefits that we have.

Very seldom, there were some stability issues but they do appear only from time to time. Since the 9.10 release we haven't seen this any more. Stable!

We only encountered scalability issues with large test scenarios, combining interfaces with a lot of browser GUI tests. Work-around is cutting up the test cases into smaller scenarios.

The technical support is fairly okay. Technical tool-wise they help a lot if needed, but when it comes to complex test scenarios or weird issues, they do not always have a solution.

An exchange of examples is difficult, since we use tooling for internal applications. So, it is not accessible from the outside world, which makes combined debugging impossible.

Initially, we were using some lightweight open-source (free) tools, but none met with our requirements fully; so we switched solutions.

The setup was fairly easy. They provide good instructions. The workstation install is a piece of cake and some additional (simple) config is needed, for Git/SVN etc. The license server install can be tricky, but it turned out to be fairly easy compared to the earlier versions.

We did it our self, with a little help from the seller.

Use floating licenses, so it’s not attached to one workstation. Furthermore, the pricing is something for the staff to handle, not for us. ;)

We mainly looked into the SoapUI solution, along with some Selenium software and besides that, other stand-alone database tooling options. We also evaluated manual testing.

Start off simple. Build your test setup from there. Do think about setting up some specific test cases as building blocks (reference material) for other test scenarios to use, so that the maintenance in the future is more easily done.

Also be sure to read the Help function, might help you on your way when you're facing some difficulties.

• Being able to load existing .EDN and .XML messages and translate them into segments. I can use them to automate the generation of new messages.
• Generating new messages, based on the existing .EDN and .XML messages, is a crucial part or the testing project that I’m currently in.

We don’t have to manually create .EDN and .XML messages. We can do this with just one click now. This drastically decreases the time that is required to create these messages.

Parasoft SOAtest has an internal refresh function where you can refresh the software to show the changes you’ve made in your projects. Unfortunately this function does not work properly, because it often does not show the changes after you’ve hit te refresh button a few times. I have to restart the software for it to see the changes I’ve made. I have been in contact with the Parasoft Support Division and they told me that it was a known issue and that it has something to do with Java.

I’ve been using SOAtest for about four months now.

I haven’t experienced any stability issues.

I haven’t experienced any scalability issues.

I would give technical support a rating of 9/10. When I encountered an issue, I could always ask for support from Parasoft and they replied with answers very quickly.

The project hasn’t used any other solution before this one.

The setup was very easy. Just a plain installation on Windows and the activation of the license key are required.

From what I understand, Parasoft SOAtest isn’t the cheapest option. But it has a lot to offer. We are only using it for generating test messages, but there is so much more that you can do with it. I suggest getting full support from Parasoft.

I joined my current project at a later stage, but I’m sure that other options were evaluated before choosing Parasoft SOAtest for this project. Unfortunately, I don’t know which other options were evaluated.

You need to check all the features that Parasoft SOAtest offers. If you just need a specific feature from the program, then there could be a cheaper alternative option that works just as well. (This is an assumption.) If you’re looking for a complete testing software/suite, Parasoft SOAtest is a serious option.

For integration tests, it’s hard to set up component tests due to dependencies. This means you cannot test without using the other components in the loop. The combination of SOATest + Virtualize gives us the opportunity to do this much easier and also to test very early in the development cycle (#Fail Fast, #Fail Early).

We can now test as early as possible, so we are achieving better control over our components.

I would like to see more concrete examples, easy to download studies of very simple Java projects, or TIBCO stuff. Now, I’m sometimes stuck and only finding whitepapers or other reading material, without having concrete examples.

Sometimes I’m stuck, and in that case, it’s useful to have simple examples. I see some shipped examples that are good, but I focus more on the Test automation as part of Continuous Delivery.

For example, yes, there is maven integration and the plugins are explained on a high abstract level. I would to like see simple examples just to showcase and give people a jump start. I found it out by experimenting, I will make this public on my blog in a couple of weeks, just to help the community.

Normally, I’m looking first for some git-repo’s supported with wiki’s, this is what I sometimes miss with Parasoft.

Another topic: SOATest is an integration testing tools, so I was expecting more examples around Tibco BW. Since it commonly just in combination.

Bottom line, modernize the way of documentation.

We are using the solution for four months already. We are using Parasoft SOATest 9.9 and Virtualize 9.9.

Till now, it seems to be stable with no issues.

We didn't encounter any issues with scalability.

We have good support of our solution architect of Parasoft, but sometimes they don’t know all the answers and then it’s harder to find more knowledge in technical support from US teams, which slows things down.

I used to use more open-source tooling. SOATest speeds up our test implementation in the integration context.

With open source, you have to build a lot of plumbing code to get it to work. With SOATest, you can directly focus on the test effort.

BUT, within these open-source frameworks, I did more BDD-oriented testing than seems to be possible with SOATest. So I need to spend more time to find the correct direction within the combined SOATest/Virtualize. Compare this with frameworks like Cucumber or FitNesse and there seems to be more control in organizing the test code.

I think it’s worth the price as long you can set up the test implementation & maintenance.

We evaluated IBM RTW. I think it has a little bit more control, but it’s more expensive.

My advice to others would depend on where they come from and what they want.

We use a lot of tools from Parasoft toolset, but with SOAtest specifically, we use it for creating and running integration tests.

They have a feature where they can record traffic and create tests on the report traffic. That is one of the most valuable features that we've found. So we can run our UI automation tests and directly create services tests from that, which is really, really helpful.

The product is very slow to start up, and that is a bit of a problem, actually. It is not as fast and it's a bit clunky, especially if you use it for the whole day or for a long time, it doesn't save the data, for example. So, it's a bit clunky and slow.

The scalability is good.

The stability is good. Currently, in our organization, we have about twelve to fifteen people using the solution and as the organization grows, and the "number of people" requirement is growing, we have plans to increase our usage. We also plan to move to other areas and try to get other people to use it as well. We do plan to increase the usage in the next one or two years.

The technical support is very good. Any time we have any issues and we raise a question, we get fast answers and we get fairly good answers, because being a licensed tool, we don't get a lot of online applications, but if you raise a request, support is pretty good.

The initial setup was fairly straightforward, easy to install, and easy to get the license keys and use them. It's very easy.

They do have a confusing licensing structure. Their licensing structure is really complex and being able to find out the right product for you, for an organization specifically, is a bit harder, but once you find the thing, their scalability and stability is pretty good. 

It's not a simple licensing model. It's not a licensing model that just, "do this and you get this", it's a bit more complicated and quite complex. For a person who is not used to this at all, it's a bit complex.

We did evaluate the other options in the market, Tosca and SoapUI, which we evaluated. Ultimately, we went with this tool.

It is a good solution, and if you're looking to migrate from an existing toolset as well, it's a pretty good solution. It is quite vast, and they have a lot of other toolsets in there. It has a lot of other toolsets like virtualize which is really helpful. The toolset is really good, I'm really impressed with it.

In the base licensing, we would like to have CLI support included because CLI is the most basic stuff that we need. Currently, it's a separate licensing that we need for that. If I'm getting an application and I need to use it. I would like to use the command line interface as well. I would expect that to be part of the base licensing.

I would rate this solution at an eight and a half out of ten.

XML assertions, XML data bank, various tools available internallly

Achieved testing middleware services and automating them with the features in SOATest. API testing is made easy. Automation and regression test suite was achieved without the need to write much code.

Performance of the tool

5 years

No

Performance issues if there are too many sessions open on your system.

Good

Good

No

Easy

Knowledgeable

Web services and API test automation was made simple without writing much code.

I have used SOATest 9.6 and 9.8 version in my project for automating the webmethods. It is widely recognized as the leading enterprise-grade solution for API testing and API integrity. Thoroughly test composite applications with robust support for REST and web services.

It provides easy handling of RESTful API's and figuring out exact API scalability, responses, and failed API's, returning code captured by the fault injections. We also used other utilities provided by Parasoft like, Datasource, Extension tools and DB tools which reduces most of our manual effort and increased the requirement coverage in automation.

The beauty of Parasoft SOA test tool is that it will support more scripting languages based on QA engineer comfort.

1. More descriptive reports are needed. Very high level reporting is not available.

2. Also the price is a little expensive in terms of users when compared to other similar tools available in the market.

3. WorkSpace is getting corrupted very often.

4. Integrating with other test management tools is really difficult.