Ranorex Studio alternatives and competitors

The primary reason that we got Eggplant Digital Automation Intelligence is that we have a couple of systems that are run through virtual machines. Everybody who uses it in our company needs to go through a virtual machine. This solution was the only testing system that was able to run that virtual machine and test within it. It was the best that we found.

What we're mostly testing are web pages and Windows apps; Oracle transportation systems. They could have done local installs, but they put them in VMware and this was the best way to test them.

In a lot of the systems that I test, I don't have a real way to do full end-to-end testing. I don't have a way to set up data nicely, watch it flow through the system, give expected results and get to a final answer, and then clean everything up. It's very piecemeal. That's not necessarily why we got Eggplant. But what I have been able to do with it are a lot of smoke tests, which we didn't have, and several others. There are also some look-up screens where we set up look-up values for other areas, and we have been able to do full testing on those. We have actually found some errors in one application, errors that have gone into the backlog, and we hope to get those fixed so that we can keep building out more tests. It has enabled true end-to-end testing. So far, we've been able to get Eggplant to test anything that we've wanted to test.

Even with the mobile app, which consumes data from a different system, we'll be able to do some full testing within that, which is great.

Our use cases are somewhat unique, because we're not a software development shop. We are testing the pieces that we can and, for that purpose, Eggplant has helped. It has been great.

The model-based test automation has helped to reduce the test maintenance process. Now, I don't have to do a lot of the simple smoke tests and some other testing.

And while I wouldn't say that the solution has helped to uncover critical bugs that normal testing would have completely missed, it certainly has helped us find bugs and then verify that they've been fixed. In one particular instance, it found out that the continuous integration pipeline was broken, or that somebody forgot to push out all of the correct files, because pages weren't working. It pointed out that there was an issue with the deploy model.

I have really come to like Eggplant. Although I'm working in QA now, I worked as a dev for a very long time and I like the solution's scripting language, SenseTalk. It's pretty easy to use.

I also like the IDE and the helpers that it gives when you search something and double-click it. It gives you the outline that you can then fill out in your code, which is very similar to many others. I appreciate that they're doing that.

What I'm also really happy about—and perhaps this isn't technically about functional testing, but it's related to it—is that DAI's newest release allows us to test via scripts rather than models, because we have done 95 percent of our development in functional, not through modeling. I am really happy that then we can use the controller to run scripts rather than having to translate things to models. There are lots of options.

We have mobile apps that we are testing through it with Android Gateway, and that seems to be working great. And, of course, any web apps and Windows applications have been no problem. We are running several scrum lines, and we've just implemented with QA and, therefore, QA is important. What we're trying to do is get the ongoing suite of tests that look at things currently, and then it will be a built-on regression suite as we keep going and keep adding more and more tests. We want to improve quality before handing it off to our customers. If we have mainframe stuff, it's nothing that I've had to test or deal with. 

It's been very fast and very easy to use. Because there's that code behind it, we're able to write modularized code, so that when a new page is created, we don't have to rewrite everything, just the things that are specific to those pages.

The IDE could be even more full-featured. Because I was a developer, I was very spoiled by either Visual Studio Code or Visual Studio for shortcuts. For example, I was able to say "ctor" and hit Tab and it would create a template of a constructor for me. Or I was able to quickly type out a class mod with properties and methods using prop and hitting Tab. It would set up the template for me. It would be great, when I want to create a new function, if there were shortcut commands like those that help create all of the functions, or if there were shortcut features to do any of the complex plans. 

I would also like to see some of the syntax updated. They have the equivalent of a switch, but it's a very weird IF statement syntax. That could definitely be improved.

Another area that I would like them to improve is their database connectivity and ability within a database. Still, we've been able to use it with what they have and get it working.

I have been using Eggplant Digital Automation Intelligence for about 14 months.

I can't think of any problems. It's been stable for me.

In terms of scalability, since I'm writing functions, it's as fast as I can write them.

About eight months ago, we were going to have a fourth person work on it. They didn't want to get an extra license so we were having to work around the fact that we only had three licenses, but potentially four people working on it. However, given our setup, I could see that it would be easy to add another license if that's what we truly needed and get somebody up and running fairly quickly.

The roles of our users are two QA people and one intern. What they're doing is very similar to what I'm doing. The intern is working closely with me, doing a lot of the tests that I just haven't had time for. On our website we have four sub-domains. I would start the intern on the basics and then she would be able to completely write all of the smoke tests at least, and the basic code tests for that, no problem.

The other QA is the person who is using it through the VM because of the need for the Oracle system. He's also been using that for full integration testing, setting up data, verifying that it goes through, and that it gets the expected result at the end. He's also the one who is working on performance. He wants to start having the equivalent of hundreds of users hitting it to see how well our systems do.

In terms of maintenance of the solution, I handle all the DAI upgrades on the server as needed, but there isn't a lot of maintenance involved. I believe there are three updates a year. It's just a matter of letting everybody know how to redownload and install it. I've updated DAI once and it went very smoothly and worked well. I have a weekly meeting with one of their technical people and for the DAI update I said to them, "Hey, here's my plan," and made sure that it looked good to them, but I was the one who enacted it.

In terms of general runtime and being able to do whatever we need to do, anything that we have wanted to do for our tests, we've been able to get Eggplant to do it for us. We have mainly gotten some smoke and other automated tests going. We want to ramp up with more of those, but the area that we really need to expand into is reporting. Management is going to want some better reports than we've been giving them but we just haven't had time to look at that aspect, because we have wanted to keep going with the tests, first and foremost. We give them a little "okay," or we give them numbers, rather than providing actual reports. We do want to get to that eventually because I know the solution has that capability.

We have emailed their tech support and they have been absolutely fine. They have usually responded within a day. If needed, I can jump onto a Microsoft Teams call and work directly with their tech support to get problems solved.

Before Eggplant, what we had been doing was C# programming in Visual Studio using Selenium, Gherkin, and Cucumber against Chrome, which is our preferred browser. The reason we went with Eggplant was that we had to use a VM to get to a lot of the products that we use internally, to help test them. The ability to test in virtual machines was huge.

The initial setup was straightforward. It was just a matter of installing the solution locally. It took longer on our side because we had to get a dedicated server, but once we had that server, setting up the license manager on it was very straightforward. Putting in new licenses for it when they expired, until we got our permanent ones, was quite easy. I had no problems with any part of the setup or in running it.

For the third person on the team who's been using it, the biggest issue has been permissions. She's not a developer like the other two are, so she doesn't have local admin permissions. We've just had to go through help desk at times, but otherwise it's been very easy.

Our implementation strategy is that we want to get it up and running and to get scheduled tests and their results. That's what we're working towards. We're hoping to generate some sort of report, whether it's a CSV or something along those lines, to see the daily scheduled test-run results.

We're still in the process of setting up ROI metrics. The last number that I saw is that we probably have 200 scripts running. We're trying to get them on a daily schedule, and that is about three hours of testing that we don't have to do, and that's per day. Every day it does the regression testing and we know if we have any problems.

In terms of deployment of the solution, we have an internal server in a private cloud running the licensing machine and we have three dev licenses for the product as well as two run-time licenses.

The development license allows you to run Eggplant Digital Automation Intelligence and allows you to test it and run your script. The runtime license is what is used when you set up a schedule. All it needs is a run-time license to run the executables and run the tests. But since you don't need the full Eggplant Digital Automation Intelligence Functional IDE, you just need the run-time license.

The other QA in our organization was looking at a testing product called Ranorex. After they fully understood what it was he was trying to do, they're the ones who said that Eggplant is probably the best solution for that.

Jump in and try it. Don't be afraid of it. Before COVID, Eggplant would train a client for two weeks onsite and then everything would be handled through tech support. Not that the worldwide pandemic was good, but because of COVID we started getting a weekly meeting with one of their tech people, and that has been awesome. He's been a great resource. We go through scripts with him, we can send him something and he can respond back via email. But we also have face time where we can say, "Hey, I tried this, it didn't work," and we can sit there and work together through the issue and figure it out.

Utilize all the resources and play around with it. It's not a simple system, but it's worth learning the details of it because you're going to get so much more out of it.

If someone were to say to me, "We are not comfortable with automating 70 percent of our linear path," that would be fine with me, because I'm not trying to sell anything. But I would point out to them that, by having a robust regression suite that they can run and that they can rely on, they are going to free up their testers to be able to work on the edge cases or the strange business issues. They will be available for all of the manual testing that is either too complex or consists of one or two things that just change often enough that they're difficult to automate. It lets their people then focus on those things rather than just the basics of whether or not the code was released properly or files were forgotten.

The team that I'm on mainly works with web pages. I don't actually have to worry about the stuff that we got Eggplant for. It just so happens that I can also use it to test our web pages. We are not a software development company, we're a transportation company. With a lot of the different tools that we have, what our in-house developers have often done is translate data or move data in file form between databases. Sometimes, the front-end web pages, for example, might end up being a report, or they might verify that customer data has gone in, or they verify fuel prices for comparison to what we paid and to see if we can do better.

Eggplant is not worked into our development cycle. It's a tool that I use to automate tests, but it's not something that we've used in a way that would help accelerate the release of a major enterprise-wide upgrade.

We don't use the solution's AI-driven automated exploratory testing. If we let it loose on our website, it would find a lot of dead ends because if data pages are not updated, and they can't upload that, they can't really go any further. Even if they could upload a couple of files, that requires going off to other systems in the background for testing, and some of it doesn't even go back to a main interface directly. That's not to say that maybe someday we wouldn't work in the model area, but at the moment I personally have been more comfortable and happy to work within the functional model.

Being a developer, the solution has reinforced a lot of my development skills. In the context of the programming acronym SOLID, it is still possible to use programming skills and to make sure that you are writing small snippets of reusable code. It has also shown that QA isn't just manual testing anymore. There's a very big automated component and you really do need people who either want to be developers or are able to develop, because that is what is required now. It's what allows you to make better, more robust tests, and things that can either recover or give you good data on how your systems are performing.

We used it for data-driven automated tests that have numeric calculations with high precision requirements. We probably are using the version from two years ago.

It was implemented to solve a very large and specific test scenario with 24,000 test cases. It did that, and the company was quite happy with this solution, but it did not easily scale, and we could not find good and short term third-party help. So, we moved to Ranorex. We've now canceled the maintenance for Silk.

That one test scenario has been very valuable. We still use the data results from that. We used it to validate Ranorex. It has helped keep the company on the automated test path.

Scripting is the most valuable. We are able to record and then go in and modify the script that it creates. It has a lot of generative scripts.

We moved to Ranorex because the solution did not easily scale, and we could not find good and short term third-party help. We needed to have a bigger pool of third-party contractors that we could draw on for specific implementations. Silk didn't have that, and we found what we needed for Ranorex here in the Houston area. It would be good if there is more community support. I don't know if Silk runs a user conference once a year and how they set up partners. We need to be able to talk to somebody more than just on the phone. It really comes right down to that.

The generated automated script was highly dependent upon screen position and other keys that were not as robust as we wanted. We found the automated script generated by Ranorex and the other key information about a specific data point to be more robust. It handled the transition better when we moved from computer to computer and from one size of the application to the other size. When we restarted Silk, we typically had to recalibrate screen elements within the script. Ranorex also has some of these same issues, but when we restart, it typically is faster, which is important.

I've been with the company for a little over two years. They were using it when I got here. They have used it for several years.

It was stable. It didn't crash and ran as expected.

We had one specific large scale job on which we needed to have automated tests. We had 24,000 test cases, which were too much to do in a timely way by hand. We got Silk Test set up, and it ran. We wanted to run other 24,000 general test cases, but we didn't find cloning to be as effective as we would have wanted. It was easier with Ranorex. That might have been because we were able to hire a third-party consultant to come in for three weeks and get that kicked off for us, where we couldn't find that help with Silk. 

On the phone, they were fine, but we needed a full-time consultant for three weeks. We could not find that through Silk or their contractor base. 

I believe they used something called TestPartner.

It was done before my time.

It was all done in-house. We had a limited number of licenses for people. We took it off for maintenance a couple of times. That's probably the same challenge with any tool.

Everyone engaged with it worked in proper quality assurance, with the exception of one developer whose job was to set up the DLL link between Silk and our products. His role was limited. He got it set up, and he was done. On an ongoing basis, it was all on our SQA testers.

We paid annually. There is a purchase cost, and then there is an ongoing maintenance fee.

We now use Ranorex, and we had looked at Ranorex, TestComplete, and LEAPWORK. One of the deciding factors for Ranorex was a recommendation from a respected colleague in a different company.

Generally speaking, Silk Test was fine and better than Ranorex in some ways. The biggest thing was that we were able to get some short-term and very specifically-focused help when needed with Ranorex, but we couldn't get that with Silk. Otherwise, the tool has many comparable features.

It is a fine product. It is just like any other tool. It is a powerful tool, and it needs commitment. Our way to get that on top of our workload was to find a short term contractor. If you've got the manpower to commit to being there to get it started, it will be just fine. There is no real big objection to Silk Test. We just needed some other help with the designs.

I would rate Silk Test a seven out of ten.

We are primarily using this product as an automation tool, for regression, and as a revision tool. 

Using the automation tool, I can transfer test cases that are being done manually to ones that are automated. After about 30 such instances, I use the regression system from the suite to create all possible combinations of test cases to try with the new version. This gives us an overview of how the software is performing. Also, when major updates take place, we re-run all of the regression tests to make sure that it still conforms to the requirements.

The automation testing functionality is very powerful.

This tool is very easy to use and I think that anyone can come in, having no experience with it, and within four to six months be comfortable with it.

The graphical interface means that you don't have to write any code.

The interface is very organized. You can find folders and settings for everything that you need, and you can easily connect things together.  

I would like to see better integration with other testing tools. There is full integration with Selenium and having this with other tools would be good.

Integration with other programming languages would also be helpful.

I have been using Tricentis Tosca for the past three years.

I have not really found many problems or bugs when using this product, so stability has not been an issue.

Tricentis Tosca is very scalable. You can easily move from a small test suite to a larger test suit, which I think is pretty good. We about 100 users in the organization.

I have not been in touch with technical support from the vendor.

I also have experience with Selenium HQ, Katalon Studio, and Ranorex.

Our organization is broken into separate sections and there is a team for installation and configuration that handled the deployment. As I am not part of that team, I am unable to judge the complexity of the setup.

My advice for anybody who is implementing this product is to make use of the extensive library that they have. It is a good collection of training videos and documentation, and I would advise everybody to go through these materials. They are really helpful and it will allow them to learn much faster, bringing them to the point where they can use it to its full potential.

In summary, I am really comfortable with this tool, I haven't had many problems, and it's sufficient for our work. In the three years that I have been working with it, whenever I've needed something, I've been able to find it.

I would rate this solution a nine out of ten.