What is our primary use case?
We have a product, a financial application, that uses a lot of services to connect to our backend systems, so we use ReadyAPI to test those services. Primarily, we use it to build out or mimic the exact services that the development team uses. Second, we use it to build out scripts in Groovy to automate certain scenarios and verify without any manual intervention.
What is most valuable?
The feature that allows you to import an API collection or a project is valuable. It reduces a lot of manual effort to give the endpoint and the parameters, etc. If the development team gives me the collection, I can directly import it into ReadyAPI and it maps the entire project so that I can start hitting the APIs directly.
What needs improvement?
There is a lot of room for improvement, mainly from the point of view of integrating ReadyAPI into the CI pipelines, and also the scripting aspect into Bitbucket. ReadyAPI is an extremely good and efficient standalone tool, but if you're collaborating on a project, it kind of becomes like a bottleneck. For use as a tool for group collaboration, there is some room for improvement.
I would like to see two new features. Number one is version control with Bitbucket or Git, where we can actually see the scripts and have meaningful pull requests. Right now, everything sits inside an XML file, so it's very hard to review the code that anybody writes. If I have a ReadyAPI project, I want to not only have it on my local machine, but also in a version control system like Git or Bitbucket. At this time I can put the project there, but it's very hard to decipher the files that ReadyAPI exposes. It would be nice for them to have it more like a .gui file where the formatting of the file and the readability of the code is easier for other individuals.
Second, is integration into a continuous integration pipeline, like a CI pipeline, where we can easily trigger, via Bamboo or Jenkins, and execute the scripts and maybe get the response of the results via an email or notification. Maybe it also needs integration into tools like Microsoft Teams or Slack, to get immediate feedback about the results of our execution. Those would be the new improvements I would like to see in their upcoming versions.
For how long have I used the solution?
I have been using ReadyAPI for the past four and a half years.
What do I think about the stability of the solution?
I think ReadyAPI is very stable as a standalone tool. I haven't had much experience with using it with cloud or CI, but overall, my impressions are that it is a very solid and stable tool. There's no flakiness in it. As long as you have the right licenses and internet, it's pretty solid. It doesn't hang or crash. The time it takes to hit the API responses and process data is pretty much seamless.
What do I think about the scalability of the solution?
I think there is a lot of improvement required from the scalability point of view.
Anywhere between 20-50 people use the solution at the moment, but that can go up to around 100 users.
How are customer service and support?
One time I got help from the technical support team, and they were good. They were able to understand my issue and resolve it within a couple of days.
How was the initial setup?
As long as you have access to the right licenses, it's a very straightforward setup.
What about the implementation team?
I deployed the solution in-house and it was up and ready within a few hours. ReadyAPI already has documentation, so I made use of that to set up the project on my machine. Installation was pretty easy and straightforward since I had the right licenses and followed along with the documentation.
We have a team who handles the maintenance. We have at least three people on that team who work in DevOps, but their time is not required every single day. I would say minimal time and effort is needed for maintenance. It's mainly to check to see if a new version of ReadyAPI is out there, and see if it works with our internal company systems. Are there any restrictions? Are there any firewall rules that need to be opened or any IT support required? Maintenance is needed just to check those things. I've never heard anybody complaining or having any issues with it.
What was our ROI?
I don't have a quantification metric, but I would say there is a lot of return on investment. That's the reason we keep on renewing the licenses even though it's maybe slightly on the expensive side. It allows us to identify a lot of issues by just using the tool, and as long as we use the tool appropriately, there is a lot of value in it for us.
What's my experience with pricing, setup cost, and licensing?
I'm not familiar with the pricing specifics, but I would say it's somewhere in the middle. I don't think it's too expensive. I would give the pricing a three out of five.
The thing with ReadyAPI is that you have to buy different licenses for different purposes. The license that we have is mainly for functional testing, but if you want to run security tests or scans, you need to purchase a separate license. If you want to run performance tests, you need a separate license. I am not aware of one single license available for all the features of ReadyAPI, but it would be nice for them to have one license for everything rather than having individual separate licenses.
Which other solutions did I evaluate?
We didn't switch from another tool, but the other solution or alternative that we use is called Postman. Postman doesn't entirely replace ReadyAPI because ReadyAPI has many other features that make it stand out on its own merit. It has options to perform load testing and security testing, not just functional API services testing.
At the moment, we are mainly focused on functional API services testing, so there are some teams who use Postman, but I think it's mainly due to convenience and the level of comfort that each engineer has with ReadyAPI. Many prefer to use ReadyAPI as compared to Postman.
What other advice do I have?
I would say that it's a very strong tool by itself. I haven't seen any other tool that matches the capabilities from a standalone point of view. If your goal is, "Hey, I have a bunch of services I want to test and play around with and even automate some testing around on a standalone machine," ReadyAPI is very powerful and capable and has a lot of benefits.
It saves a lot of time for future testing, so if I want to repeat a test over and over again every single day, it saves time. It has good source connections, such as a database or an Excel or adjacent file. It can take the source input in various formats, and it can give the output in various formats, like Excel, or it can also write it into a database. It has that integration available with other technologies.
I think it is trying to improve itself by having integrations into Bamboo, Jenkins and Jira, but it's bit late in the game there. I think there's still a lot of work to do on the CI.
I would rate this solution as a seven out of ten.
Which deployment model are you using for this solution?
On-premises
Disclosure: My company does not have a business relationship with this vendor other than being a customer.