Parasoft SOAtest Reviews

We use a lot of tools from Parasoft toolset, but with SOAtest specifically, we use it for creating and running integration tests.

They have a feature where they can record traffic and create tests on the report traffic. That is one of the most valuable features that we've found. So we can run our UI automation tests and directly create services tests from that, which is really, really helpful.

The product is very slow to start up, and that is a bit of a problem, actually. It is not as fast and it's a bit clunky, especially if you use it for the whole day or for a long time, it doesn't save the data, for example. So, it's a bit clunky and slow.

The scalability is good.

The stability is good. Currently, in our organization, we have about twelve to fifteen people using the solution and as the organization grows, and the "number of people" requirement is growing, we have plans to increase our usage. We also plan to move to other areas and try to get other people to use it as well. We do plan to increase the usage in the next one or two years.

The technical support is very good. Any time we have any issues and we raise a question, we get fast answers and we get fairly good answers, because being a licensed tool, we don't get a lot of online applications, but if you raise a request, support is pretty good.

The initial setup was fairly straightforward, easy to install, and easy to get the license keys and use them. It's very easy.

They do have a confusing licensing structure. Their licensing structure is really complex and being able to find out the right product for you, for an organization specifically, is a bit harder, but once you find the thing, their scalability and stability is pretty good. 

It's not a simple licensing model. It's not a licensing model that just, "do this and you get this", it's a bit more complicated and quite complex. For a person who is not used to this at all, it's a bit complex.

We did evaluate the other options in the market, Tosca and SoapUI, which we evaluated. Ultimately, we went with this tool.

It is a good solution, and if you're looking to migrate from an existing toolset as well, it's a pretty good solution. It is quite vast, and they have a lot of other toolsets in there. It has a lot of other toolsets like virtualize which is really helpful. The toolset is really good, I'm really impressed with it.

In the base licensing, we would like to have CLI support included because CLI is the most basic stuff that we need. Currently, it's a separate licensing that we need for that. If I'm getting an application and I need to use it. I would like to use the command line interface as well. I would expect that to be part of the base licensing.

I would rate this solution at an eight and a half out of ten.

The primary use case for adopting Parasoft SOAtest is to test web services/middleware testing (SOAP/REST/MQ/SAP RFC/EDI/FTP). 

Inbuilt support:

• REST: WADL and JSON.
• SOAP: WSDL.
• SAP RFC
• Message queuing supported: JMS, MQ, and NET WCF.
• Protocols supported: TCP/IP, FTP, EDI, WSDL, WADL, WS-*, UDDI, RMI, EJB, webMethods, TIBCO, SMTP, and custom.
• Languages Supported: Java, JavaScript, Groovy, Jython
• Virtualization
• Third-party tool integration (plug and play)
• Stored procedure testing and DB validation
• UI recording through Selenium WebDriver.
• I've used this solution for more than 6.7 years.

Parasoft SOAtest has improved the quality of our automated web services, which can be easily implemented through service chaining and service virtualization. 

The best feature to integrate with third party tools is the integrated plug and play without any rework, which makes it more valuable than any other tools. 

Parasoft SOAtest has more visibility in regards to testing support data and reporting mechanisms.

Through Parasoft SOATest, we can automate our scenarios in a data driven format, which shows there is no rework on scripts. We only need to update the test data and run for a number of scenarios.

Advantageous Points:

• Easy handling of RESTful APIs/SOAP calls.
• Getting the data on the API performance side, such as response time and segregation of failed API issues based on returning code captured by fault injections.
• Comparison of current responses with baseline response to figure out issues in case of failure with an exact location point.

• Diff control tool
• Reporting
• Different service types of support (SOAP, REST, SAP RFC, and UI recording)
• Third-party tool integration: GitHub and Jenkins.
• Plug and play
• Service virtualization
• Marketplace

Reports could be customized and more descriptive according to the user's or company's requirements.                                     

No issues.

No issues.

I have had an awesome experience while working with the customer service/technical support teams. They have resolved issues within a day or two.

If issues are reported to the Parasoft Support Team, they will resolve them by taking it to t-con or v-con. Also, they are very supportive and well-versed with the proper information.

We have been using this solution for more than 6.7 years. 

I have used other solutions as well, but I did not get proper support during our testing due to the constraints for testing in particular areas.

It is simple to do the setup starting from installation until usage. It does not require any specific skill (i.e., it is the same as any other tool installation).

They were well-versed with the proper information. 

We are completed satisfied with Parasoft SOAtest. The ROI is more than 95%.

It is a commercial tool. The license price is a little expensive, but it provides a better outcome in terms of the end-to-end automation process.

While evaluating tools for web service testing, I tried different solutions (freeware and highly-used), but they did not provide some mechanism/features that I could test my respective scenarios to FTP, EDI, MQ, and even SOAP calls. Because of this, I moved back to the existing solution.

I have been using this solution for more than six years. I have not found any issues while doing automation. 

Repeatable automated regression tests in a SOA architecture IT landscape, for several SOAP/REST webservices including Web Gui end-to-end testing, all scenario-based for 10+ applications. Also, scheduled for test runs at night, using Jenkins.

It provides us with fast(er) test results, more predictable results and automatic (regression) test runs outside the employee workstations. This tooling made us more agile and more ready for continuous integration. We use the night hours to run different regression tests over systems, for which we only need to check the results in the morning.

• Fast and easy automated tests for UI
• Interfaces (SOAP, REST)
• Additional database queries
• Easy validations on responses (soap, rest, json, sql, Gui)

Most valuable for us is the combination of all these items in one solution for us. Broad user story scenario's from A to Z, though a chain of systems, including test data prerequisites.
The next most valuable benefit over the past year is the possibility to use the tooling for continuous testing purposes, using the server-side run option 'as a web service' for (nightly) batch runs.

In general, its user-friendliness needs a little improvement since there are a few bugs that still remain. It needs to be (possibly) more lightweight and have more validation options. The bulk changes in the test cases are not always easy. But all in all, these are minor details compared to the benefits that we have.

Very seldom, there were some stability issues but they do appear only from time to time. Since the 9.10 release we haven't seen this any more. Stable!

We only encountered scalability issues with large test scenarios, combining interfaces with a lot of browser GUI tests. Work-around is cutting up the test cases into smaller scenarios.

The technical support is fairly okay. Technical tool-wise they help a lot if needed, but when it comes to complex test scenarios or weird issues, they do not always have a solution.

An exchange of examples is difficult, since we use tooling for internal applications. So, it is not accessible from the outside world, which makes combined debugging impossible.

Initially, we were using some lightweight open-source (free) tools, but none met with our requirements fully; so we switched solutions.

The setup was fairly easy. They provide good instructions. The workstation install is a piece of cake and some additional (simple) config is needed, for Git/SVN etc. The license server install can be tricky, but it turned out to be fairly easy compared to the earlier versions.

We did it our self, with a little help from the seller.

Use floating licenses, so it’s not attached to one workstation. Furthermore, the pricing is something for the staff to handle, not for us. ;)

We mainly looked into the SoapUI solution, along with some Selenium software and besides that, other stand-alone database tooling options. We also evaluated manual testing.

Start off simple. Build your test setup from there. Do think about setting up some specific test cases as building blocks (reference material) for other test scenarios to use, so that the maintenance in the future is more easily done.

Also be sure to read the Help function, might help you on your way when you're facing some difficulties.

Automated execution and chaining of test validators is the most valuable feature; it is easily extensible via the built-in custom scripting API.

It has allowed us to build a large library of repeatable automated tests that are able to be quickly executed on demand. This builds a solid foundation for future initiatives expanding into a more continuous integration and the agile methodology adoption.

Memory Management suffers when using solutions other than the data repository for managing the test cases.

The platform can become unstable as the memory usage increases. For example, we use Excel spreadsheets for managing test cases, so the size of these test cases must be tightly controlled.

Scaling tests beyond a certain point can trigger issues with Memory Management, but good design practices usually indicate a need to break up a test before it would reach that point.

Technical support from Parasoft has been excellent.

The setup was straightforward. It was very easy to set up a test scenario that consumed assets in our web service layer and to run validation against the results. The tool itself allows for some more complex chaining of components within a test scenario, so familiarizing yourself with the tool enough to get more creative in your test setups can be more complex.

If you plan on using a floating-license model, plan on a buffer of at least 10 user tokens between the maximum allocated by your license and the expected usage for the tool.

I looked at SoapUI NG Pro.

Keep your test scenarios small and focused rather than trying to cover an entire web service with one scenario.

The facilities for testing API’s (services, REST API’s, messaging clients, etc..). Especially the features for parametrizing requests and checking responses are very powerful. But the product really shines in end-to-end testing, not only because of all the built-in tools supporting this, but also because of the great extensibility of the tool.

If you want something that’s not provided out of the box, then you can write it yourself and integrate it with SOAtest.

As an expert user, you can accomplish a lot with this tool.

Using this tool, we were able to set up a nightly, still growing, company-wide regression test -- dramatically decreasing time being spent on (manual) regression testing.

• Web-testing facilities. This was not very good when we were using version 9.2, so we built our own Selenium framework and integrated this with SOAtest (this being possible at all is a very positive point in my opinion ;-).
• The user-interface is not very lightweight or friendly, sometimes buggy and cumbersome. The product has a steep learning curve, but very well worth the effort.
• There is virtually no (online) user community, except for an outdated forum. When having issues you have to find everything out yourself (or call in help from Parasoft).
• Reporting facilities can be better.
• Last but not least: using the tool in current continuous integration/delivery practices is not very easy or straightforward.

Five years.

Yes, the product can be buggy or crash sometimes. But not to the extent that it hinders working with it.

We had some issues with stability and performance -- with a growing testbase and/or ‘big’ tests. But these also were minor issues.

8 out of 10.

Technical staff living nearby are always prepared to come by and help with issues. Communicating with developers abroad is also satisfactory.

Not really, QTP was used a little before but was not the right tool for our purposes.

It was complex, but that had more to do with our own complex infrastructure than with the product itself. But it took some time to get the setup of the product right.

It is an expensive product, so think carefully about whether it fits your purposes and is the right tool for you. Support pack is helpful. Due to high license costs, the tool is not easy to use by many people in your organization (like with open-source tools) and so to make test automation something for the whole team.

Yes, but before I was involved, so I don’t know more than that.

Take your time to get to know SOAtest very well, don’t get put off by the steep learning curve, it’s very well worth the effort.

Automatic testing is the most valuable feature.

Compatibility with HTTP 1.1 and TLS 1.2 needs to be improved.

I have used SOAtest for one year.

We get a lot of time-outs and after we start the run again it goes well.

We have had scalability issues.

I rate technical support 8/10.

It was our customer’s choice to use SOAtest.

The tool is complex.

We followed our customer’s choice.

Think twice before you start with it.

The most valuable features are the Browser Playback tool for recording and playing web scenarios (Web UI testing), and the messaging client for sending SOAP calls and for validating responses.

The testing time is shortened because we generate test data automatically with SOAtest.

During the process of working with SOAtest and building test cases, the .TST files will grow. A negative side effect is that saving your changes takes more time. The bigger the .TST file, the more time it costs. Sometimes a few seconds! An option is to split up the file into separate files, but often that’s no realistic option because you want to keep the test suites together.

TST-file is the container file in which all testsuites and teststeps are stored. It often happens that you have to change a teststep and want that change to be saved. Saving your teststep in an almost empty container (with only one testsuite and a few teststeps) will go very fast (within 1 second). Saving your step in a container with multiple testsuites and teststeps takes a lot more time. In my case sometimes five seconds.

Sometimes SOAtest hangs, but that is very sporadic. It could be caused by the stability of the environment of the customer in general.

We haven’t had any scalability issues.

The technical support in the Netherlands is very good. Questions about the tool are almost answered within a day. When needed, a Parasoft consultant joins the project for one day or more.

We used SoapUI for testing web services. In our case, the advantage of using Parasoft SOAtest is the use of the datasheet; with one message, you can loop through your datasheet and check all your data.

The setup was done by another team of the company and was out of our scope.

The products of Parasoft have the name; they’re expensive. I think it would be a great step to decrease the price of the licenses. A possible consequence could be getting more users and more familiarity in the Netherlands and around the world.

I have experience with a lot of other products, but when I started with this job (Dutch Government) three years ago, the product, SOAtest, had already been bought by the company.

Invest a lot of research time into what your exact needs are. After that, select some test tools to compare all the options. In my opinion, Parasoft has so many possibilities that it must have a place on your shortlist!

SOAtest is, at its base, a collection of tools that can be combined to generate test scenarios.

Input data can be gathered from any possible source from web GUIs, databases, Excel, and files and is used in SOAP/REST tools and web browser tools. The output of those tests can be saved in files and databases.

Every imaginable source in the entire world of information technology can be accessed and used.

The following features are particularly valuable to us:

• Generating SOAP/REST requests
• Validating responses against WSDL
• Scripting parts of an XML request
• Accessing DBs with JDBC
• Jenkins Continuous Integration
• Regression test automation
• Modular scenario setup

Our tests are run nightly on every environment using Jenkins CI.

We moved from the Waterfall/V-model to Agile a couple of years back. With SOAtest, we are able to test features in every sprint, quickly and efficiently.

We use SOAtest to test the interfaces of maybe a dozen back-end services, simulating multiple, different points of sale. For example, a customer web shop, a sales agent application, sales machines, or mobile apps.

Service interfaces are continually integrated and updated within SOAtest. We test most front-end features first with manual tests, and then integrate them with validations into SOAtest scenarios.

Before using SOAtest, we had a huge delay in acceptance testing due to the amount of manual regression tests that we had to do.

Sadly, almost everything needs improving. Don’t get me wrong, the tool is very powerful and immensely useful for day-to-day usage. But certain things are missing or are set up in a way that makes it unnecessarily complicated. Here are some examples:

• Replacing part of an XML request by script requires a variable in the scenario folder. This variable needs to be prepared before the SOAP request tool with the partial XML to be replaced in the request. This variable can then be set to replace a subtree of the XML request. It would be easier if we could directly set a script in the SOAP request tool as a replacement.
• Enabling/disabling an optional element of an XML request is only possible if a data source (e.g., Excel sheet) is connected to the test. Otherwise, the option is not available at all in the drop-down menu.
• True parallel testing is currently impossible.
• There is a separate product ‘Load Test’ that has parallelism built in. But this is only to multiply a scenario with all test cases by a number of users executing all test cases. In a future release, there should be the option of running all test cases of a single test run in parallel to speed up testing.
• To build a modular scenario, you can reference test suite files and execute the test tools contained therein. But every test contained in the referenced test suite file will be executed, as it is set up in the test flow logic of this scenario.
• Sadly, there is no way to setup single operations of a web service and then combine those operations into a scenario. This is how SoapUI does it.
• SOAtest is a plug-in for Eclipse. But actually, it comes bundled with its own Eclipse and several other Eclipse plug-ins like PyDev. Although this ensures compatibility of all contained tools, it also means that every part is a little out of date. For example, PyDev is available as v4.5.5, which means you get Python v2.5.2.
• You can manually upgrade to a newer version, but this means every installation using your scenarios, and especially scripts, must be updated as well.

• We have been using v9.10 for about six months.
• We have used SOAtest itself for around five years (since v9.3).

We had a stability issue, but Parasoft support was always very helpful in finding a solution or a workaround.

I would give the technical support a rating of 10/10, or even 20/10!

No matter the problem, from simple user errors to complicated Eclipse issues, the support team is always quick to respond and is very good at what they do.

If you actually encounter some issue in which the first or second level support really can’t fix in a couple of minutes, then the third level support will dig deep and come up with amazing ways to help you.

There is a user driven support forum where you can get help and examples from other users.

We did not use another solution previously. We do use SOAtest and SoapUI in parallel:

• SoapUI is used as freeware by developers for single/simple SOAP requests.
• SOAtest is used by the testing team to build extremely complex scenarios.

Installation is easy and straightforward, as is the initial scenario building. SOAtest gives you the ability to choose how complex you want to build your scenarios.

The simplest level of usage is a full graphical view of requests and responses, storing values in data bank tools for use in later SOAP/REST request and SQL tools.

The more intricate levels consist of scripting in several available languages and really revealing the beauty of SOAtest.

The documentation guides you through several examples and helps enormously with the initial steps of building your test scenarios.

We evaluated SmartBear SoapUI.

• Learn XML as well as XPath.
• Learn Jython, a variation of Python running in the JRE. It gives you the option to import Java classes into your scripts and use the latest Java improvements, but without the type safe limitations and language complexity of Java.
• Implement your scenarios from the beginning with continuous integration in mind. For relative paths everywhere, abstraction in SOAtest environments etc., use a version control system to save and version your scenarios and scripts.
• Take a short course (less than one day) from the Parasoft support team to familiarize your test team members with the product very quickly.