Apache Airflow alternatives and competitors

In terms of our use case, we are building a pipeline to manage data. It's pretty simple. We validate the data and then write it in the database.

It has enabled my organization to create workflows with a lot of different AWS services. It's a general solution that you can adapt to your own needs and is simple to use.

When it comes to valuable features, we like that it includes everything that’s connected to AWS which means that it can be integrated with everything in the AWS suite. We also like that the creation of the pipeline can be done using the graphical user interface. It's pretty easy because you don't have to write the code to create the tasks.

We are building our software and we just released our NDP internally, so I've been using it for almost three months.

It's stable and the responsibility is very good.

We are a four-person team, and for the moment I am the only one with access to Amazon Step Functions. We are an early-stage startup, so we are scaling and building our platform. We are looking for companies to test our software.

We met one of them just last week, so they are available. They are helpful.

Positive

When I used Apache Airflow, the main problems were that the debugging phase was not so easy and Apache Airflow only accepted Python code. It was only Python-based. On the other hand, Amazon Step Functions allows you to use different kinds of codes.

It was not so complex. It's pretty easy because as I mentioned, you can use the GUI to create the pipelines and the workflows.

It took about one day, because we had to set a couple of things. We had to understand some errors, but now everything is working good.

We did the implementation in-house.

I recommend speaking to a consultant if you are not familiar with workflow managers. Otherwise, it's very easy to understand how a workflow can be composed.

I rate it a 9 out of 10.