We changed our name from IT Central Station: Here's why

Apache Airflow OverviewUNIXBusinessApplication

Apache Airflow is #8 ranked solution in BPM Software. PeerSpot users give Apache Airflow an average rating of 8 out of 10. Apache Airflow is most commonly compared to Camunda Platform: Apache Airflow vs Camunda Platform. Apache Airflow is popular among the midsize enterprise segment, accounting for 58% of users researching this solution on PeerSpot. The top industry researching this solution are professionals from a computer software company, accounting for 24% of all views.
What is Apache Airflow?

Airflow is a platform to programmatically author, schedule and monitor workflows.

Use airflow to author workflows as directed acyclic graphs (DAGs) of tasks. The airflow scheduler executes your tasks on an array of workers while following the specified dependencies. Rich command line utilities make performing complex surgeries on DAGs a snap. The rich user interface makes it easy to visualize pipelines running in production, monitor progress, and troubleshoot issues when needed.

When workflows are defined as code, they become more maintainable, versionable, testable, and collaborative.

Apache Airflow was previously known as Airflow.

Apache Airflow Buyer's Guide

Download the Apache Airflow Buyer's Guide including reviews and more. Updated: January 2022

Apache Airflow Customers

Agari, WePay, Astronomer

Apache Airflow Video

Archived Apache Airflow Reviews (more than two years old)

Filter by:
Filter Reviews
Filter Unavailable
Company Size
Filter Unavailable
Job Level
Filter Unavailable
Filter Unavailable
Filter Unavailable
Order by:
  • Date
  • Highest Rating
  • Lowest Rating
  • Review Length
Showingreviews based on the current filters. Reset all filters
Solution Architect at EPAM Systems
Simple to automate using Python, but code does not cover all data warehousing tasks
Pros and Cons
  • "This is a simple tool to automate using Python."
  • "We need to develop our workflow description and notations because out of the box, Apache Airflow does not provide some features that are needed."

What is our primary use case?

The primary use case for this solution is to automate ETL process for datawarehouse.

What is most valuable?

The most valuable feature is the UI, for automation.One can monitor all ETL processes in single screen. Complex workflows are shown as DAGs SVG images.

This is a simple tool to automate using Python.

What needs improvement?

There are some drawbacks to this solution. The code does not cover all tasks in the data warehouse automation process.  Currently , in production, we have a large installation with a complex workflow that includes hundreds of tasks. Most of them are dispatched by existing engine, but not all.
For example, sometimes we need to create cycles in our workflow but we are not able to, because Airflow supports only Direct Acyclic Graphs ( DAGs )

We need to develop our workflow description and notations because out of the box, Apache Airflow does not provide some features that are needed. It is our understanding that it is limited by design.

We will wait for the latest 2.0 version, as it is awaited to be much more mature than the 1.8-1.10 version. We believe that it will be better.

There should be some improvement made to the Doc Management features from within the UI. They should think about Outlook integration, which should be out of the box, and the object model should be expanded to support cyclic graphs inside the workflow.

For how long have I used the solution?

We have been using this solution for eighteen months.

What do I think about the stability of the solution?

This solution is not very stable. There are a number of configurations issues.

What do I think about the scalability of the solution?

This solution is scalable. We use this solution in a single node, but it is possible to have a  cluster of workers.

It can be used for one or two thousand related tasks and should be done in a cluster configuration. 

We don't use a cluster, rather we only use single nodes. It is sufficient for our tasks. Tasks are long and the parallelism is limited by the database engine, and not by the workflow engine. 

We would like to evaluate clusters in the future.

We are using the Cron Task scheduling feature for Apache Airflow. Users can configure the Apache Airflow themselves. There are up to ten users that can configure Apache Airflow.

This is a part of the wage solution, and it is the initial point of the wage slot process. The wage solution has hundreds of users.

How are customer service and technical support?

We don't use any paid technical support, as it is an open-source solution. We have used Stack Overflow and other open information sources, but we know that some companies provide technical support. 

As we have studied their solutions that are available on the internet, it is my understanding, that, we are on a pretty high level and could provide commercial support ourselves. 

We don't use any support from commercial companies, but some very useful recent solutions we could extract from Apache Airflow GitHub, as an example.  

Which solution did I use previously and why did I switch?

Previously, we used Control-M for a short period. It was a solution used by our customers, and we needed to understand their difficulties and the results. 

For low to middle scaled tasks, Apache Airflow could be a substitute for Control-M

How was the initial setup?

The deployment model we used was through a private cloud. It was a private installation on Google Cloud.

What about the implementation team?

In-house team.

What was our ROI?

It 's measured jut now. Precise data is awaited in 3..4 months. First conclusion - positive ROI

What's my experience with pricing, setup cost, and licensing?

There are no costs associated with this solution. Apache Airflow is a free solution that can be downloaded and ready for use at any moment.

Which other solutions did I evaluate?

Our tasks can be automated by simple Jenkins, but our customer wanted to implement it on Apache Airflow. This was a solution used by our customer.

Apache Airflow is mainstream and everyone wants to use it. Google provides Apache Airflow as part of the Cloud services.

What other advice do I have?

My advice would be to use this solution for simple tasks. 

They should have a Python expert for features that are not available out of the box, as it is not enough. 

It could be a good solution for enterprise workflow automation and solutions like Control-M within the next two to three years.

We are happy and satisfied with this solution, but not fully satisfied, as this solution has some positive and negative aspects.

I would rate this solution a seven out of ten.

Disclosure: I am a real user, and this review is based on my own experience and opinions.
Buyer's Guide
Download our free Apache Airflow Report and get advice and tips from experienced pros sharing their opinions.