IT Central Station is now PeerSpot: Here's why
2019-09-02T05:33:00Z

What is your primary use case for Apache Airflow?


Hello peers, 

How do you or your organization use this solution?

Please share with us so that your peers can learn from your experiences.

Thank you!

PeerSpot user
Guest
99 Answers

NitinKumar - PeerSpot reviewer
Top 5LeaderboardReal User

The primary use case is the orchestration and automation of ELT/ETL data pipelines. 


Apache Airflow is great in this respect and there are scheduling options to make it fully automated based on the used case.

2021-10-04T07:20:31Z
reviewer1539081 - PeerSpot reviewer
Top 10Real User

I've used it at past companies to build a data warehouse for analytics (populating redshift/snowflake).



My current company is using it for similar purposes but more to pull data from data sources across the company, join them into a central data repository (we're currently using postgres), and build datasets with this data. Having this central data repository will help serve other use cases for us in the future.


2021-10-05T14:49:32Z
reviewer1539081 - PeerSpot reviewer
Top 10Real User

I'm a data engineer. In the past, I used Airflow for building data pipelines and to populate data warehouses. With my current company, it's a data product or datasets that we sell to biopharma companies. We are using those pipelines to generate those datasets.

2021-03-26T23:33:18Z
reviewer1502001 - PeerSpot reviewer
Top 10Real User

There are a few use cases we have for Apache Airflow, one being government projects where we perform data operations on a monthly basis. For example, we'll collect data from various agencies, harmonize the data, and then produce a dashboard. In general, it's a BI use case, but focusing on social economy. We concentrate mainly on BI, and because my team members have strong technical backgrounds we often fall back to using open source tools like Airflow and our own coded solutions. For a single project, we will typically have three of us working on Airflow at a time. This includes two data engineers and a system administrator. Our infrastructure model is hybrid, based both in the cloud and on-premises.

2021-02-11T12:31:46Z
reviewer1468407 - PeerSpot reviewer
Top 10Real User

We mainly used the solution in banking, finance, and insurance. We are looking for some opportunities in production companies, but this is only at the very early stages.

2021-01-15T22:07:12Z
Ashok Juyal - PeerSpot reviewer
Top 10Real User

Our primary use case is to integrate with SLAs.

2020-12-23T23:10:23Z
reviewer1447245 - PeerSpot reviewer
Top 5Real User

We normally use the solution for creating a specific flow for data transformation. We have several pipelines that we use and due to the fact that they're pretty well-defined, we use it in conjunction with other tools that do the mediation portion. With Airflow, we do the processing of such data.

2020-12-22T20:05:08Z
Sudhir Ganti - PeerSpot reviewer
Real User

We are a technology, media, and entertainment-technology company. We are using Apache Airflow for architecting our media workflows. We are using it for two major workflows. We have had it set up for some time on our own cloud. Recently, we migrated the setup to AWS.

2020-04-13T06:27:00Z
Alexey Nadenenko - PeerSpot reviewer
Vendor

The primary use case for this solution is to automate ETL process for datawarehouse.

2019-09-02T05:33:00Z
Buyer's Guide
Apache Airflow
May 2022
Learn what your peers think about Apache Airflow. Get advice and tips from experienced pros sharing their opinions. Updated: May 2022.
596,362 professionals have used our research since 2012.