Astro by Astronomer Reviews

We manage the deployment, governance, and everything for the different teams that want to orchestrate their workflows. All of their ingestion jobs, their analytics jobs, and their whole ETL or ELT jobs are built by the teams themselves, and we manage all of the deployment and governance for those jobs.

One of the jobs we handle involves consuming data from different sources like Google Analytics and Apple Analytics. We use dltHub to have a standardization layer across the different ingestion jobs. We source the data from there, pass it through a schema check, and then sync it to our Databricks table. We also run quality checks on them and then publish the final reports.

In the data platform, we use two different workspaces. One is for the admin deployments, which we, the central data team, use to orchestrate jobs such as rotating the credentials on schedule or running freshness checks and data quality checks. We then have a general workspace which different teams deploy their DAGs into. We use two branches for that: staging and main. We do not want anyone to deploy broken DAGs, so we have CI gates. Astro by Astronomer workspace cloud is connected to our main Astro by Astronomer repo. We have dispatch workflow jobs in GitHub which get triggered whenever there is a deployment in the child repos where the stakeholders deploy their DAGs. The dispatch workflow syncs the changes from the child repo to the main Astro by Astronomer repo. Based upon the change, if it is a simple DAG change, then we do not do a full deploy but just a DAG deploy, which is much faster. If there is a change in the infrastructure, then we do a full image deploy of Astro by Astronomer to that workspace or to that deployment.

There are a lot of new features that we have started using in Astro by Astronomer. Some of them I can list is their new AI agent, Auto. It is quite useful for investigating why a DAG has failed and getting more context around it. The other feature is Cosmos, where since most of our jobs are running dbt models, Cosmos provides us a way to run dbt natively. We can have granular-level logs and retries and lineage and everything. That is a useful feature.

Then there are Datasets where we can link different DAGs and to have a comprehensive framework and have a freshness check and quality checks on the entire stack. The Blueprint feature is quite useful since many of our ingestion jobs that are being created by stakeholders may not be quite familiar with Airflow, and with Blueprint, we provide them a way to use a no-code solution to build their DAGs. Astro by Astronomer Observe has a lot of features that we use: the observability and alerting that we set up.

Apart from that, the whole having the entire deployments and the DAGs and everything in one place is quite useful. In MWAA, everything is decentralized, and that creates a lot of friction and problems. In Astro by Astronomer, I find it really useful that everything can be in one place. Also, one of the good features in Auto, we can do the migration from, let's say, Airflow 2.x to 3.x using Auto seamlessly.

Instead of giving generic messages in, for example, a Slack channel when an Airflow DAG fails, you can use Auto to have a comprehensive root cause analysis of what has failed and based upon the context of the DAG and its historical runs, it gives a much more comprehensive analysis of what has failed, and that speeds up the fixing the bug and fixing the DAG cycle a lot more. Instead of going through the entire debugging process, having the root cause analysis provided helps tremendously. Sometimes we have to dig in a bit deeper, but it does help a lot.

Previously when we were using MWAA, since our DAGs were living in S3 buckets, the workers running in EC2 and having logging in CloudWatch, everything was scattered around the place and doing a deployment or debugging was quite a pain. Astro by Astronomer solves that seamlessly, where in one place, we have everything. Granting access control is quite easy. We have created different teams in Astro by Astronomer and integrating those teams with Okta, we can have seamless integration of which user can access which workspace. Also, whenever a user leaves the organization, their Okta is disabled and automatically they lose access. In that regard, Astro by Astronomer has helped tremendously.

When we have to do an upgrade, for example, upgrading the Airflow version, that was quite problematic in MWAA, and there used to be a downtime of approximately 40 to 45 minutes while the Airflow web server had to spin up and do its thing. That seems quite seamless in Astro by Astronomer.

Also having alerts, whenever a DAG fails, we have a direct PagerDuty integration for the important DAGs. If that DAG fails, we immediately get notified on our PagerDuty schedule. That is also one of the features that helped us tremendously. Having monitoring on top of that, Astro by Astronomer, integrating it directly with DataDog has been quite useful. That was quite challenging when using MWAA. Additionally, Astro by Astronomer seems to be quite cheaper than MWAA overall.

I would say it would be much more helpful if Astro by Astronomer can provide an MCP to use and integrate with Astro by Astronomer and their AI agent Auto, so that we can use our tools like Claude, Code, or Cursor to have a central location where we integrate that MCP in Databricks via the AI gateway, and then we can have a place where we can connect to Astro by Astronomer, Databricks, other tools such as Jira and DataDog, and then have a much more comprehensive analysis and overview and monitoring of the entire data ecosystem.

Having an MCP integration would tremendously help us to have a more comprehensive data platform that is more AI-ready. That is one of the features that I can think of. Another thought would be having an integration with Kubernetes instead of just scheduling everything on Astro by Astronomer workers. That would be helpful.

One thing I can think of is that the Astro by Astronomer local developer environment can be improved further, but perhaps that is a very niche use case that I have in mind, and it may not be useful for others.

There is room for improvement around having an improved UI and also integration with other tools.

On governance, I would say we can have a more granular level of governance even inside workspaces. Having much more granular control over people belonging to the same team in Astro by Astronomer could have different access to different DAGs, depending upon our tagging strategy. That is one of the areas I can suggest in terms of improvement.

The AI capabilities can be improved further. I would compare the current AI capabilities to newer models provided by Claude such as Opus or Codex.

I have been working as a Data Engineer for more than eight years. I started at Babbel last year, so I am almost going to complete one year at Babbel.

It was not 100% seamless, since there were a few complexities that we had to overcome. Some of our DAGs were tightly coupled with how MWAA works, particularly the whole IAM roles and everything. Additionally, Secrets Manager was an AWS native product. In MWAA, there was a seamless integration with that, but when we migrated to Astro by Astronomer, those were some of the challenges that we had to overcome. It was quite manageable. I would give that experience an 8 out of 10.

Scalability-wise, it is very good. We do not have to worry about selecting the right worker size, and it is quite scalable. We did hit the limits sometimes, but that was due to us not estimating the workload properly. However, with the monitors, the DataDog monitors and the Astro by Astronomer alerts that we had in place, we were notified quite in advance, and we could increase the worker capacity in time. That was good. We also use the Astro by Astronomer executor, which combines the features of both Celery and Kubernetes executor, so the overall experience was quite seamless.

Customer support is great. We have support engineers helping us if we face any issues. Additionally, we have a bi-weekly call with the Astro by Astronomer product team where they guide us to the new features and help us if we can use some of their new features or if we are following the best practices in our current setup. It is quite good.

We were using MWAA for Airflow before that. We switched because of the reasons I have already mentioned.

Astro by Astronomer acts as our orchestration layer. All of our jobs, including our ingestion jobs, our transformation jobs, analytics, machine learning, and AI workloads, are everything orchestrated using Astro by Astronomer now.

We had a call with the Astro by Astronomer product team, and then they pitched us the idea, and then we got onboarded with them.

Our MWAA workers and everything, including deployments and everything, used to cost us around three thousand dollars previously to run the entire stack. Now, currently, after Astro by Astronomer, we could manage it for probably one thousand five hundred to one thousand eight hundred dollars per month. That is quite a significant improvement.

Whenever there is a deployment, Astro by Astronomer seems to have zero downtime, which used to be around 40 minutes when doing an MWAA deployment.

If you want a one-stop solution for your orchestration, I think Astro by Astronomer is the best place to look. The overall product experience is quite good. The customer support is good. You should definitely check out Astro by Astronomer. I would rate this solution an 8 out of 10.

My main use case for Astro by Astronomer is managing Airflow as an Astronomer. I am building ETL workflows that get scheduled by Airflow DAGs. I use Astronomer as an instance or as a service of Airflow.

A specific example of how I am using Astro by Astronomer in my ETL workflow is that I have almost 500 to 600 different data pipelines where data gets ingested from different sources to the warehouse. The sources are hybrid by nature, and the target is also hybrid, but most of the time I am using Snowflake and Databricks to ingest data. Between the sources and targets, there are many steps to get data ingested, with different dependencies between the tasks and between the jobs. I have completed several API integrations using Airflow DAGs, which get executed based on events.

The ETL jobs and data pipelines are my main use cases. I also have some online, on-demand jobs that get executed by event-driven workflows. These are the two main workflows that I am working on currently.

Previously, I used an AWS-managed service as Airflow. After moving to Astro by Astronomer, my team spends considerably less time on Airflow infrastructure maintenance and more time delivering new data pipelines. Deployment has become more standardized, and production stability has improved. Troubleshooting has become faster due to the centralized monitoring system. My team spends less time on Airflow infrastructure maintenance while deliverables, quality, and the development lifecycle have improved significantly.

The best features Astro by Astronomer offers include that it is a managed Airflow platform, so it eliminates the complexity of maintaining the Kubernetes cluster or the scheduler metadata database. I also appreciate the Astro CLI, which makes local development and testing very easy. Developers can validate the DAGs locally. I value the scalability because my tasks have more than 500 jobs running, and automatic scaling handles increased workloads without requiring manual infrastructure changes. I appreciate the CI/CD integrations because it works well with the Git-based deployment workflow, and that helps me manage release and version management. The observability is also excellent with centralized logs and execution histories, which helps with troubleshooting.

Regarding how centralized logging and observability have helped my team with troubleshooting, my data pipeline is very complex with many interdependent elements. Often, an intermediate task fails, and dependent tasks need data to flow at a specific time. If the data has not arrived, I need to investigate and find the root cause. Finding the root cause is very difficult when many complex, deeply nested tasks are interdependent. The logger and observability help me find these issues very easily.

Astro by Astronomer has impacted my organization positively because the business impact is quite good. Previously, I used an AWS-managed service as Airflow, but after moving to Astro by Astronomer, my team spends considerably less time on Airflow infrastructure maintenance and more time delivering new data pipelines. Deployment has become more standardized. Production stability has improved significantly, as I found many issues when using AWS-managed services. Troubleshooting has become faster due to the centralized monitoring system.

Since switching to Astro by Astronomer, I can share specific outcomes and metrics, such as time saved. I had many difficulties when using Airflow through other service providers. When more tasks suddenly needed to work in parallel, I experienced resource constraints. This triggered many failures of downstream jobs. Development was more time-consuming. The CLI helps me perform local development much faster. Previously, finding the root cause of any issues took almost half a day. This observability is helping me find the exact problem within an hour, sometimes in minutes. The development lifecycle is shorter and much less error-prone. Previously, developing DAGs took days and sometimes a sprint. Now, I can deliver within a couple of hours.

I believe there are several areas where Astro by Astronomer can improve. The pricing has become a little expensive because I have very large and always-running workloads. The enterprise configuration sometimes requires a learning curve that could be easier. Advanced customizations could be improved, and the documentation of complex topics needs enhancement. In enterprise scenarios, better documentation would help me more.

I believe support is strong. Whenever I reach out to find issues or root causes, I get a very timely response from the support team. Integration is definitely an area that could be much better because of advanced enterprise requirements that need more configurable learning curves. If integration were easier, it would be simpler to deploy to different teams.

I have been using Astro by Astronomer for almost two years.

At this moment, I have seen a couple of hiccups with Astro by Astronomer, but most of the time, I can say it is very stable.

The scalability of Astro by Astronomer is good and quite easy, and it is maintained as a service by the Astronomer team. At the beginning, I saw some hiccups. I was not getting the number of instances I was looking for, and jobs got stuck, taking more time. However, day by day, it has become more stable.

Customer support for Astro by Astronomer is good. I get a very quick response. I have some SLA agreements, and more than 95 percent of the time, I have seen those SLAs get met.

I used to use MWAA of AWS. Different teams still work on that, but I found difficulties with that in terms of maintainability. I believe that is one of the reasons I switched many DAGs to Astronomer.

I believe I have started seeing ROI now since I have been using Astro by Astronomer for more than one year. It is hard for me to say that money has been saved, but definitely time has been saved. I am still working with the same number of employees, but the deliverables, quality, and development lifecycle have become shorter.

I believe my experience with pricing, setup, cost, and licensing could be better. It is a little expensive for my organization when running many large and always-runnable workloads. The pricing is on the higher side. In terms of licensing, the cost could also be better. I think it would be beneficial if pricing were more flexible based on organizational workloads and offered dynamic pricing options.

Before choosing Astro by Astronomer, I tried to build my own Airflow instance to run in my own data center. I found that in terms of the complexity of the job and the task, it is difficult to maintain ourselves and would need a very dedicated resource. That is why I stopped that approach and moved to Astro by Astronomer.

I believe there are definitely many commendable features that work very well, but there are many areas of improvement as mentioned previously. I believe day by day, as those improvements come and are available, this platform will continue to get better and better.

Regarding Astro by Astronomer's AI capabilities, I think governance and security are important because I am not using much of the AI capability at this moment due to security and governance concerns. I am very careful about how to use that specific service, and I am trying to learn more. I have many open questions for the Astronomer team to understand that specific area.

I believe the accuracy of Astro by Astronomer's AI capabilities is pretty good. In terms of accuracy, depending on the required prompt, a better prompt gives a better result. That helps me get closer to the development target. As accuracy is definitely not 100 percent, I can see developers saying they are getting 70 to 75 percent accuracy, and that is definitely helping with the reliability of the output.

My advice to others looking into using Astro by Astronomer is that I would definitely recommend it. I would recommend Astro by Astronomer to organizations that rely heavily on Apache Airflow workflows and want a production-ready managed platform. It is very valuable for medium to large enterprises or data engineering teams that prioritize delivery.

Astro by Astronomer is doing a great job. I believe my relationships with this platform will continuously grow as my services grow. I am looking forward to integrating and leveraging the AI service and capability with Astronomer. I would rate this platform a 7 out of 10.

I have been using Astro by Astronomer for around six years from when I first used Astronomer Airflow. At that time, we were just starting our data engineering journey, exploring big data tools like the Spark framework, and that is when we came across Astronomer Airflow and it started.

The main use case for Astro by Astronomer is that we had a lot of Spark workloads which we required to run on a data proc cluster, and we wanted to have an orchestrator that would facilitate the scheduling of pipelines, have resilience, have reliability, and have near to zero downtime. We started exploring Astro by Astronomer for this purpose. To date, we have our workloads scheduled on Astronomer, we have our own Astronomer instance for both a staging environment and production environment on which we created multiple pipelines and multiple DAGs, and no matter if it is a normal Spark job, a GPU task, or some sort of model training that needs to happen maybe daily or weekly or monthly or once in a quarter, we have the DAGs scheduled on Astronomer Airflow, and that works very well for us.

One specific example of a workflow or pipeline that we run with Astro by Astronomer can be a very basic ETL, which is part of any pipeline that we write irrespective of whether it is a model training or just reading the data and transforming. Let's say to start with a very simple use case that we almost encounter every second day where we have to read the data from some source, which could be a GCS location, another BQ table, a hive table, or it can be a Postgres DB. There are multiple operators on Astronomer Airflow which facilitate direct connections without needing to create them from scratch. After reading, to transform the data, we submit our Spark job onto the data proc cluster, and the operators present make it easy to just pass the required parameters to initiate our transformation. When it is time to load the results into a Cassandra DB, Postgres, or to write them to a different GCS location or create a BQ relative view on top of our data, we have multiple operators that help us directly integrate and seamlessly work for all use cases. This is a very basic and the most useful thing that we are doing every second day—the ETL pipelines in which we are extensively using this. Additionally, there are heavy use cases where we have to provision a GPU cluster, and Astro by Astronomer provides operators that help us directly create data proc clusters by specifying how many cores, how many nodes, and which type of machine to use, facilitating resource procurement without worrying about errors.

Astro by Astronomer has positively impacted my organization by significantly reducing manual efforts needed for setting up Airflow, which previously consumed more than fifty percent of engineers' time. We have moved away from cron jobs, managing state, retrying tasks, and scheduling them ourselves, allowing us to focus on daily work. This transition has dramatically increased our productivity. The introduction of DAG-level SLA and task-level SLA has proven invaluable, enforcing task completion within specific timeframes and enhancing our alerting systems. For example, the GCP file sensor operator has made it easier to trigger systems based on file existence, streamlining our operations. In summary, we now spend less time managing infrastructure and instead focus on delivering data products, achieving faster recovery from failures and meeting SLAs more consistently. Our reporting and observability improvements help us quickly detect issues, and overall operational costs have declined significantly.

The best features of Astro by Astronomer include many aspects, such as the DAG bag with an impressive refresh rate that allows me to create a new DAG and automatically scan it within fifteen to thirty seconds. The easier pipeline management also stands out because regardless of how many DAGs we have, we can easily label and name them without restrictions on naming conventions. Task dependencies and groupings are absolutely amazing for workflow orchestration, enabling smooth transitions from one task to the next based on defined success criteria. The easy UI provides clear observability into what is happening inside the DAG, along with reporting functionalities, XCOM variables, and integration with GCP cloud. The upgrades have all been backward compatible, making transitions between versions seamless and without significant changes necessary. I can manage how many tasks can run in parallel, controlling performance while utilizing resources effectively. The easy setup for local development allows us to push changes efficiently to our branch. Additionally, the CI/CD capabilities and the many available integrations, such as the Python operator and Data Proc operator, are invaluable. The security mechanisms, like role-based access control, and integration with secret management enhance overall functionality and reliability, making Astro by Astronomer a preferred choice for our data engineering team.

One area for improvement is the recent change in the UI with task groups introduced in version two point three and above. Previously, naming tasks underwent a more straightforward process; now, they appear in a dropdown that sometimes complicates visibility for tracking the names. Additionally, to enhance industry adaptability, cost discounts could lead to even wider use. As newer features roll out, the UI has become slightly more complex and less self-explanatory compared to the previous version, which was simpler and cleaner.

I have been using Astro by Astronomer for around six years from when I first used Astronomer Airflow.

I have experienced no stability issues or downtime with Astro by Astronomer.

Astro by Astronomer's scalability has been impressive, handling increased workloads without issues. My experiences have been positive; the platform's reliability and stable production setup have ensured no significant downtime has hindered business operations, consistently delivering dependably executed workflows.

Customer support for Astro by Astronomer has been excellent. We have not needed much support, but there was one situation when a high volume of scheduled tasks overwhelmed us. The Astronomer team provided instant support, helping us to increase the number of workers and quickly resolve the issue.

Earlier we used cron jobs that required manual management for backfills, retries, and infrastructure handling. The data platform's growth led us to find an alternative, as cron management became complex and lacked central monitoring and dependency support. We switched to Astro by Astronomer for its built-in central workflow management, dependency handling, automated retries, and robust monitoring capabilities.

In quantifying improvements, adoption of Astro by Astronomer reduced engineers' time by fifty percent on infrastructure management. The metrics show a drastic infrastructure cost reduction of forty-five percent since adopting it. The return on investment has been remarkable, with savings exceeding seventy-five percent compared to previous manual cron job management. Downtime has decreased by seventy percent, and operations have become nearly one hundred percent faster than prior systems. Developer productivity has surged over seventy-five percent as we do not spend time on infrastructure tasks, and we have seen increasingly positive customer feedback and overall operational efficiency up by ninety-two percent.

Before choosing Astro by Astronomer, I evaluated other options such as AWS MWAA and GCP Data Flow, but they turned out to be more expensive operationally and involved considerable setup efforts. Furthermore, legal constraints against AWS in our organization discouraged deeper analysis, making Astro by Astronomer the clear choice.

The advice I would give to others considering Astro by Astronomer is to recognize its immense value for teams wanting to focus on building reliable data pipelines instead of managing infrastructure. Its managed environment, scalability, and robust operational features make it an excellent choice for organizations modernizing their data orchestration. I would rate this product nine out of ten.

My main use case for Astro by Astronomer depends on the project use cases. For something like big data projects on ETL pipeline and ELT pipelines, it truly depends on projects to projects. I would say first you can go with something modern data warehouse and lake house orchestrations. Why do we choose Astro by Astronomer? Because it has built-in data lineage which tracks the data flow down to the table and column level, making it very easy for a data owner to know how the data is flowing. Secondly, it provides multi-tenant and multi-cloud pipeline integrations, allowing integration with any cloud vendors whether it is AWS, Azure, or GCP. Third, I would say that it has wonderful production machine learning and AI pipelines, where MLOps teams can use Astro by Astronomer to orchestrate end-to-end machine learning workflows with distributed frameworks such as Ray, Spark, or SageMaker to evaluate metrics. It creates complex DAGs for you. Lastly, on CI/CD driven analytics engineering, engineering teams moving towards data operations use Astro by Astronomer to automate testing and deployments in my current organization. Developers spin up the local Airflow environment using the Astro CLI, write some test code, and push it to GitHub or GitLab which automatically deploys to production via CI/CD. Astro by Astronomer, a commercialized version of Airflow, works on local dev and production, making it easy for people to test in the dev environment.

A quick specific example of one of these use cases is that in our recent team, we run Snowflake, Databricks, and BigQuery as well as DBT. Astro by Astronomer serves as the central orchestration pipeline or orchestration plane, triggering the DBT transformations and syncing the ingest parts, such as from Kafka or Fivetran, while running quality checks on the data and notifying downstream tools, such as BI tools including Looker or Tableau. We chose Astro by Astronomer for its built-in lineage, which helps track the data flow from the table and column level. This is a use case of Astro by Astronomer that we use in multiple teams within Zalando, where we rely on different cloud software or platform as a service, with Astro by Astronomer forming the central orchestration layer.

Regarding my main use cases, a small example of our e-commerce daily revenue and customer reporting occurs at 2:00 AM. The daily team pulls raw data, aggregates financial metrics, computes churn risk, and refreshes executive Looker dashboards in BI dashboards. The workflow begins when S3 ingestions come in, which go to Snowflake. When the data arrives in Snowflake, we have triggers written on Astro by Astronomer. From there, the DBT pipeline takes over, writing it into staging for clean and deduplicated data, with quality tests validating success or failure, including any malformed data issues. This way, we utilize Astro by Astronomer precisely, rather than writing messy bash operators to execute DBT runs as a single opaque block. Astro by Astronomer, developed by Astronomer Cosmos, alleviates this by allowing us to manage individual DBT models and test them in discrete observable Airflow tasks. It makes life easier by adhering to Astro CLI standards, which are module-based and consistent. It greatly aids us with granular model observability that is not found in a single DBT test and offers zero downtime secret and environment management, ensuring the environment remains consistently up. Moreover, it provides frictionless local development by allowing data engineers to run Astro dev start, spinning up a Docker context that mimics the production Airflow one-to-one, allowing for testing DBT models against a local development schema, pushing branches into GitHub, and letting CI/CD automatically deploy updated DAGs to Astro by Astronomer cloud.

Astro by Astronomer offers multiple features that depend on projects. In our case, I would say the best features include developer tooling and automation. Astro by Astronomer provides the Astro CLI, creating local parity where developers can work locally in a Docker container mimicking a production environment, which is fantastic. Additionally, Astronomer Cosmos is an open-source framework natively integrated into Astro by Astronomer that automatically converts DBT projects into fully observable native Airflow DAGs, allowing every DBT model to be tested and become an individual node in the Airflow UI without needing manual wrappers. This feature saves a lot of development time.

Furthermore, it has Auto AI, which assists with DAGs, troubleshooting task failures, and generating automated migrations across different Airflow version upgrades while enabling integration with various AI and MCP servers. It also provides no-coder blueprint authoring, functioning as a drag-and-drop workflow. On observability and data lineage, it presents an excellent pictorial landscape of data flows and lineage management. Cost optimization is also vital, with dynamic auto-scaling based on workload without needing to worry about scaling nodes under the hood. It offers zero downtime, and it includes a fantastic Terraform provider from a DevOps perspective.

The feature that has made the biggest difference for our team among all these offerings depends on the project at hand. I would say it is the cloud-native hosting that treats Airflow as a virtual machine. When using open-source Airflow or basic cloud providers such as AWS-managed Airflow MWAA or GCP Cloud Composer, the vendor simply spins up EC2 or Kubernetes instances and installs Airflow on top, leaving the rest to you. You still have to debug slow DAG parses, manage complex CI/CD containers, and build custom lineage connections. Astro by Astronomer operates at the pipeline and code level, which is vital. This allows Astro by Astronomer, designed by Astronomer, to streamline how data engineers write, test, and observe data workflows, enhancing scaling, cost efficiency, and maintaining zero downtime with automated upgrades.

Astro by Astronomer has positively impacted my organization by significantly improving developer productivity, reducing downtimes, and alleviating overhead associated with upgrades. The acceleration in developer velocity and time to market is evident, as our cost optimization efforts have led to a nearly 45% reduction in cloud spending, translating to a high return on investment.

Astro by Astronomer is doing great, and while improvement opportunities exist, particularly around the cost model, which can occasionally feel steep, I generally find it quite favorable. For smaller teams, the cost can be quite high; self-hosted Airflow or AWS MWAA options are cheaper. For larger companies, it seems acceptable. Aside from that, I do not have any other concerns, but perhaps adding advanced governance such as streamlining RBAC directly into the UI could be beneficial, which may not currently be available.

I have been using Astro by Astronomer for approximately five years.

Astro by Astronomer is definitely stable, which is one of the key reasons we chose it.

The scalability of Astro by Astronomer is very good.

Customer support for Astro by Astronomer is very good and top-notch.

I definitely see a return on investment, as the time to market has notably decreased. Fewer employees needed is not a point for discussion since we are already a lean team; the current number of employees is optimal for our work with Astro by Astronomer. When I mention a 45% reduction in cloud spend, it was measured against a previous solution we used, which was managed Airflow web services. Now, with Astro by Astronomer, our costs are significantly lower. However, it is not a definitive metric, as various factors come into play. It is essential to note that not the entire team has fully adopted Astro by Astronomer yet, which is vital for context.

My experience with pricing, setup cost, and licensing has been very good.

The accuracy of Astro by Astronomer's AI capabilities is very good, and its reliability is also very good. The advice I would give to others looking into using Astro by Astronomer is that it is a good product with great support and excellent AI integrations, making it suitable for industry scale. I would rate this product as an eight out of ten.

Our main use case for Astro by Astronomer in our organization is to schedule jobs, including PySpark pipelines and Managed Flink. The DAG for Flink is Cron-based, while the DAGs for PySpark jobs are asset-driven and event-driven.

An example of a workflow I schedule with Astro by Astronomer is a Cron-based job that runs every day at 6:00 PM in the evening. This DAG triggers an MSF application to run, which is pointed to Kafka. We ingest data from Kafka using Flink and store it in a Lakehouse, which is Iceberg.

Our scheduling process is based on Astro by Astronomer. Currently, we have approximately 150 DAGs and several PySpark jobs.

The best feature Astro by Astronomer offers for our use case is asset-driven event-driven scheduling. We don't need to wait or guess when our ingestion process from Flink will be done. As long as the ingestion process is complete, it automatically triggers. There are tasks that emit assets, and we can start our downstream jobs by receiving events from the assets.

This has changed my workflow compared to before. It's faster and more efficient, and we no longer need to wait or guess at what time the first ingestion process will finish. As long as it's finished, it will trigger the event and materialize an asset, which automatically starts the downstream processes.

Astro by Astronomer has positively impacted my organization because it continuously follows the latest version of Airflow as an open-source project. As long as there is a new release of Airflow, they try to catch up and release another version of Astro by Astronomer. This is a significant advantage since we mainly use assets everywhere. The development of assets is still in progress and not fully mature yet, but we chose to use it in order to progress with it and follow the improvements every time. Astronomer is trying to catch up with the latest features and releases.

There are some downsides to the event-driven scheduling features since it is new. We have faced several problems that we have managed to tackle. For example, Airflow may batch many events in one execution run, which can be problematic. Ideally, there should be one event and one execution run so that you don't miss any events. I think this will be fixed later on.

The feature can be better in other ways. We faced an issue with paused DAGs since they are asset-driven and asset-triggered. Due to regulatory requirements, we had to pause some DAGs. The problem is that the events on the asset accumulate, so when we unpause or activate the DAG, those events are still there. The next incoming event will batch and group all the events together, which causes us to miss events because there is no one-to-one relationship between events and executions.

It would be beneficial if we could handle these assets and events more selectively. For example, I would want to run only the last event or perhaps only one event among all the accumulated events. This functionality is not currently possible. I don't know if there is an open ticket to solve this, but it would be great if they could implement this capability.

Astronomer provides support from AWS. We recently had a problem and received assistance from their support team.

My advice to others looking into using Astro by Astronomer is that you must first understand how Airflow works. For example, if you want to add libraries or endpoints you need, you should know that Astro by Astronomer is essentially a managed version of Airflow. You don't need to create a scheduler, web server, or database, which is a good point as it eases your deployment process. However, you must understand how Airflow works and its components. When you have an issue or problem, you need to know how to monitor it. In my case, the AWS support team helped me check what was happening when I faced any issues. At minimum, you need to understand what's happening and what components are running behind the scenes.

Astro by Astronomer is a great tool for scheduling data pipelines. The deployment process means you don't need to worry about how it will be managed. For example, if you use Kubernetes to deploy your Airflow, it will be challenging at the beginning. Astro by Astronomer addresses this problem by managing infrastructure for you. I would recommend using it. I have given this review a rating of seven out of ten.

Astro by Astronomer allows me to view a complete pipeline, check the logs, and rerun jobs. I have also been using it for environment management and automatic scaling. I have been building data pipelines with Astro by Astronomer so that I can check tasks, see which task has failed and why it failed, review logs, and examine errors.

I can retry tasks multiple times if there is a failure. In one project, we used Astro by Astronomer to solve a problem where we had a pipeline ingesting 50 to 60 GB of data from cloud storage into our bronze, silver, and gold layers using Databricks. We use Astro by Astronomer for orchestrating the entire workflow. Each step—data ingestion, validation, transformation, and loading—was a separate Airflow task with dependencies. If one task failed, Astro by Astronomer would automatically retry it, send an alert, and allow us to rerun only the failed task instead of running the complete pipeline. This reduced our manual intervention, improved reliability, and made jobs easier to monitor and troubleshoot.

Astro by Astronomer is more reliable. In addition to orchestrating these pipelines, we use Astro by Astronomer for CI/CD development. We package our DAGs and deploy them consistently across development, QA, and production. We use the Astro UI to monitor pipeline health and troubleshoot failures. We also integrated it well with Databricks, cloud storage, and databases, which makes it easier to build reliable and scalable pipelines and to monitor these jobs. Overall, it has reduced our operational overhead and improved the reliability of our data pipelines.

Astro by Astronomer standard Airflow deployment time was reduced by more than 40 to 50 minutes overall, where manual effort was five to ten minutes per automated deployment. We also saw production pipeline failures decrease by roughly 20 to 30 percent because of better dependency management and consistent deployments. When failures did occur, troubleshooting time was reduced by around 40 to 50 percent since the logs and monitoring made it much easier to identify the root cause. Overall, the team saved several hours each week that were previously spent on manual deployments and operational support. We have observed noticeably fewer deployment issues, faster recovery from failures, and a significant reduction in manual effort due to automated deployments and better monitoring.

One of the best features Astro by Astronomer offers is Airflow deployment, built-in monitoring, and seamless CI/CD integration. I also appreciate the ability to manage different environments like development, QA, and production consistently. The task retry mechanism, alerting, and detailed logs make troubleshooting much easier for me. Since we integrate Astro by Astronomer with Databricks and cloud storage, it also helps us orchestrate complex data pipelines reliably without spending much time managing Airflow infrastructure.

CI/CD has made our workflow much more efficient by automating deployments with Astro by Astronomer. Whenever we make changes to our DAG, the code is pushed to a Git repository, and the CI/CD pipeline automatically runs validation and tests before deploying to the Astro by Astronomer environment. This reduces our manual deployment errors and ensures consistency in releases across development, QA, and production, which allows the team to deliver changes faster. It also lets us roll back to the previous version if an issue is found. With the alert mechanism, if there is any failure in a job, all users who have been associated with that particular job get notified immediately. As for the retry mechanism, if I have a pipeline with more than 50 tasks and a particular task is failing, we can run just that particular task instead of running the complete pipeline.

One of the biggest improvements since we started using Astro by Astronomer is that it has reduced our manual workflow. Since we have adopted Astro by Astronomer, we have seen pipeline deployments become much faster and easier in all specific environments. We do not need to change the code for a particular environment; Astro by Astronomer picks the configuration based on the environment. If it is deployed in development, it picks the configuration from development and runs the task. If it is deployed in QA, it picks the configuration for QA and runs the task, and the same applies for production. This reduces our manual workload, and continuous deployment is much easier. Pipeline failures also decreased due to built-in retries and better monitoring. When issues did happen, we could identify and resolve them much more quickly using the detailed logs. In the detailed logs, we can find at which stage the code failed, what the issue was, and what time it failed. It has reduced our manual effort and improved the reliability of ETL workflows. It has also improved downtime, so downtime has been minimized, which helps us ensure data is delivered on time for downstream reporting and analytics. From a business perspective, we have achieved faster deployments, reduced manual effort, improved reliability, and faster troubleshooting.

Overall, my experience with Astro by Astronomer was very positive, but there are a few areas where it could improve. For large environments with hundreds of DAGs, navigation and monitoring could be more intuitive, making it easier to identify critical issues quickly. More built-in analytics for pipeline performance and resource utilization would be valuable.

Additionally, while debugging is already good, having more detailed error insights and root cause analysis out of the box would help reduce troubleshooting time. For example, if an error is happening repeatedly in a particular log, could Astro by Astronomer itself identify the error in the log and rerun the job based on that particular error? These are areas where it can improve.

I have been using Astro by Astronomer for the last four years.

The documentation for Astro by Astronomer is fine. I have always preferred the documentation. You can get detailed information about tasks, how to write them, the syntax, and all features. I have also gone through the certifications of Astro by Astronomer, so that also helped me analyze what features are coming into the latest release, and that has been helpful for me.

Regarding governance and security, I think Astro by Astronomer is sufficient. No additional improvement is needed. I have not faced any challenges with governance and security. I was not very involved in it, so I cannot identify many issues.

The AI capabilities of Astro by Astronomer are generally reliable for routine tasks, such as generating DAG templates, providing code suggestions, and assisting with troubleshooting. For a production pipeline, I always review and validate the logic before deploying. Occasionally, the suggestions need more refinement for complex workflows or organization-specific requirements, but overall, I have found the output to be accurate and useful.

My advice to others looking into using Astro by Astronomer is to start with a well-designed Airflow architecture and follow the best practices for DAG design. Use Git and CI/CD from the beginning. Keep tasks modular and implement proper monitoring and alerting. Astro by Astronomer takes away much of the operational overhead of managing Airflow, so teams can focus on building reliable data pipelines instead of maintaining the infrastructure. For organizations running multiple ETL workflows, especially on Azure and with Databricks, Astro by Astronomer is a strong choice because it improves deployment consistency, scalability, and operational efficiency.

Astro by Astronomer is beneficial. I have used it in my project, and it is very beneficial for us. It has improved performance and reduced our workflow. I would always prefer Astro by Astronomer instead of any other technologies. I would rate my overall experience with this product as a 9 out of 10.

I use Astro by Astronomer in the main case of data pipelines orchestration and data quality.

It is useful because more than one person in the team can work on the data pipeline, seeing if the stages and the tasks were run successfully. If not, they can fix to run properly and to manage, build, or maintain a data pipeline that exists, not only one that was built for our team but was given from a business team, for example in the same company or from another team that was merged with our current team at our company.

I think that the best features from Astro by Astronomer are the data observability from the whole pipeline and the ability for more than one person from the same team to work in the same data pipeline or manage many data pipelines as we want to have. With orchestration, observability, data quality, reduced costs, and an integrated framework with other tools such as Databricks, Azure, AWS, or GCP and data tools, we can cover all the stages from the data pipeline from the beginning, from using APIs, from ingestion, from treating the data, and from the last mile that we can call as dashboards or machine learning studies or data scientist models that we can apply to our data from the last mile in the data pipeline. Astro by Astronomer can help us with that.

The biggest impact for our team is that we can integrate the tools and reduce costs and facilitate the maintainability of our process. We can observe the data, correct and fix problems when an issue appears, and report to our leader and the business teams the current status of the whole ingestion process of the data, guaranteeing the data quality and the reduced costs of our process.

I think that Astro by Astronomer is a good example of how data orchestration is important to the company when we think about big data and ingestion from many external companies or external institutions that bring to us data that is important to provide final products that can provide to the company the best decision-making in the future for the company regarding revenue, costs, and guidelines for the company that want to grow at a very fast pace.

Astro by Astronomer could impact the company in a way that the data teams and the business teams can learn in a facilitated and easy way how to manage data pipelines, how to manage tools from cloud to observability and to data visualization, and can manage the whole process and act in an easy and fast way when it is needed by our members in our team and in our company. It is important to have a tool that can show to us in an easy way the whole process and where we should act when an issue or some problem is occurring and how we can handle that and provide the perfect solution to reduce the impact in the company.

Especially in data quality, we have some kind of tool that we managed to have before Astro by Astronomer that we got the status of our pipelines from many different tools that was difficult to manage, and it was difficult to have the observability for the main pipelines that we had at that time. With Astronomer, we can synthesize in the same platform all the data pipelines and provide the best rules and the best technical cases to apply data quality to our huge amount of data that we ingest daily and have reduced costs and improve our time spending in fixing problems in the data pipelines and the data infrastructure that we have in our company.

Astro by Astronomer can be improved always by integrating and being in attention with the capabilities that are emerging in the technology as well. For example, AI models that we are building here in the company and how we can provide the observability for these processes that are emerging, talking about machine learning products, model products, GenAI products, chatbots, and agent AIs and all the processes that we are building and that we can and want to have observability and maintainability of this process in a very synthesized and very fast pace way of solving problems and acting when it is needed by our team.

It is so important to have the documentation updated and have some kind of tutorials in text and examples or some kind of AI product that can help a new member of the team or the actual members of the team to be updated with the improvements that Astronomer is doing to the product. It is important to keep the team updated to the products and how we can suggest some improvements thinking about new technologies that are emerging in the tech world nowadays such as AI and all the products that emerge from this theme, including agent AIs, GenAIs, chatbots, and machine learning models.

I think that there is always a margin of improvements thinking about how we can provide updated documentation, updated tutorials, use AI models to help our team to keep updated with the new technologies, improve the interface, improve how we can access the interface such as in a web app, mobile app, or desktop, and find new ways to keep updated with our data products such as pipelines and data quality in these pipelines. We always have some kind of improvements to do in the product and certainly in the future Astro by Astronomer will keep updated with all these new capabilities that the product can have to integrate not only the data pipelines and orchestration, but the cloud architectures, principal institutions such as Oracle, AWS, Azure and GCP, and the last mile of the data ingestion such as dashboards, data modeling, and machine learning models or agent AIs and AI products as well.

It is always important to have a good knowledge of data governance nowadays because not only is it good to have data products in the last mile as I mentioned before, but it is important to have the data governance in our team to know what we are producing, how we are storing this data, how we can keep our data safe in our environment to be attending the requirements of the LGPD and the laws that are in the countries that we provide our products and to know how we can manage this amount of data with observability, maintainability, and how to manage the data to specific teams in the company that need to access the data but actually do not have the high level of technical information that a data engineer can have in the daily work as a data professional in the company.

For now it is good enough, but it is always important to keep updated with the latest models and the improvements that the AI tech is providing in the tech world. It is important to know that AI is improving and Astro by Astronomer can keep updated with this. For now, to answer the question about accuracy and reliability of the outputs, I think it is good enough and provides feeds and correct answers to address the questions that the data teams are interested in knowing by the tool that we are using.

I have been using Astro by Astronomer for three years.

It is stable.

It is good enough. We can build and have many kinds of data orchestrations and provide observability and maintainability in a simplified way to our team members.

It is good enough for us. They provide to us important support when needed to build new products or to manage products that were already built.

I have used open source solutions and I use some kind of native solutions by other tools such as cloud services that have orchestration, Databricks as well, and DBT have documentation in SQL. Astro by Astronomer can simplify and unify all these features in a low cost way. I switched because I knew that Astro by Astronomer was a relevant and proven or tested solution to data orchestration in many companies. The benchmark was made and the tool was chosen.

I evaluated other options with integrated data orchestration in external tools such as GitHub Actions or some kind of open source products to orchestrate data. Astro by Astronomer was chosen because our benchmark with other companies, especially big ones, proved to be reliable to our needs talking about data quality and data orchestration in a huge amount of data ingested daily.

It is important to understand how the product works and how you can understand all the features and get the maximum value that the product can bring to you talking about a huge amount of data, big data pipelines, and using CLI and APIs to connect the cloud environments and to have proper observability, maintainability, and connect with security and safety environments to your data. Finally, you can have data governance with low costs compared to other strategies in the data architecture and have observability in the whole process. You can handle situations, issues, and problems that can emerge in a daily work of data ingestion. I would rate this product an 8 out of 10.

In our project, the main use case for Astro by Astronomer is to manage the DAGs, utilizing DBT workflows as well as PySpark running on EKS for triggers and scheduling the DAGs, ensuring our delta loads are scheduled in Astro by Astronomer.

A specific example of how I use Astro by Astronomer in my workflow is that daily there is a file dumped in our landing zone S3. We have the PySpark code to load from the S3 landing zone to our Snowflake landing zone, and then from landing to raw and raw to curated, featuring DAGs such as the EKS DAG which runs the PySpark. We trigger that DAG, and once that job is completed, we trigger the raw pipeline and then trigger the curated pipeline, allowing this delta pipeline to run daily and automate my delta loads.

The best features Astro by Astronomer offers are the scheduling, the monitoring, and especially the APIs, which provide flexibility. I can create custom dashboards through the API as well as implement scheduling, and it offers a great UI on top of Airflow, which is an open-source system. The self-hosting aspect is beneficial because I do not have to manage the Astronomer or Airflow instances myself and can spin them up very easily without any overhead, making it a user-friendly SaaS platform.

Day-to-day, I find myself relying most on the scheduling and the SaaS feature, as managing an Airflow instance is quite difficult. If the server crashes or there are machine problems, I would normally have to deal with the infrastructure myself, but with Astro by Astronomer, I do not have to worry about that and can spin up any deployment quickly. Since scheduling is a default feature of Airflow, if you consider Airflow, scheduling is essential, but for Astro by Astronomer, I emphasize the SaaS part, enabling me to create deployments on the go without any hassle.

Astro by Astronomer has positively impacted my organization because it is reliable, allowing me to depend on it for managing the Airflow instances along with the infrastructure and error handling.

I can share specific outcomes indicating that we have saved significant time, as hosting an Airflow instance on a server is a time-consuming task that we can avoid entirely. In just five minutes, I can spin up an instance in Astro by Astronomer, and the error handling is time-efficient since errors are notified instantly. Without Astro by Astronomer, someone would need to manually check for errors, which requires going through a separate integration, while in Astro by Astronomer, it works out of the box.

I believe there is a need for improvement in Astro by Astronomer, especially regarding the strict limitation of 10 DBT deployments. Increasing that limit would be very helpful since it currently acts as a bottleneck, and I cannot increase it even if I want to.

The main reasons I have not rated Astro by Astronomer higher or lower include the hard limit of 10 DBT deployments, which I find limiting, and the fact that the UI could be better, particularly the UX part. Improvements can be made in how easy it is to interact with it. Overall, I rate it a seven.

I have been using Astro by Astronomer for around two and a half years.

Regarding the UI, integrations, and how it handles notifications or error tracking, I appreciate that a separate smart flow is needed for that. The error messages are useful since any encountered error is automatically sent to me via email, indicating that the integration is quite good.

My advice to others considering using Astro by Astronomer is that it is great, flexible, and very useful. Because it operates as a SaaS, we do not have to deal with deployment, so I highly recommend it.

In terms of Astro by Astronomer's AI capabilities, I think the governance and security are commendable since it is integrated with good governance practices and does not allow data to flow out of the clusters, indicating strong governance and security.

Regarding the accuracy and reliability of output from Astro by Astronomer's AI capabilities, I find it accurate and reliable, although sometimes the logs disappear, so I would suggest improving that aspect. I rate Astro by Astronomer a seven overall.

My main use case for Astro by Astronomer is to create DAGs to run queries in AWS Athena or to instance jobs in EMR, Elastic MapReduce. Specifically, we use an EC2 server where we need to create jobs and take Parquet files to consolidate small files, so we use this to reduce costs with the API get object in S3.

Astro by Astronomer has positively impacted my organization because it is open source code, so we can use it in AWS, Azure, GCP, or on servers that we have on-premise, ensuring we can use it in every situation. Additionally, since it is open source, we do not have to pay any license, which I think is the best feature, along with the ability to control DAGs and create situations using CronJobs.

After adopting Astro by Astronomer, we reduced a lot of time and improved security using DAGs to work queries and run queries in Athena. Users can create the query and run the DAG to execute in Athena, improving security because not everyone has access to AWS Athena and reducing costs.

The best feature Astro by Astronomer offers is the user interface, which is very good, and the logs that we can access in the user interface are very helpful to debug problems.

What I appreciate about the user interface is that it is very friendly, very easy to navigate, and very easy to use.

I think Astro by Astronomer can be improved for newcomers, especially regarding the installation, making it more friendly for Windows users, as we use WSL update. I think this is an opportunity to develop something better for Windows.

I really do not work with governance and security in Astro by Astronomer, but we do not have any problems with it. We use a server, and we have never had any problems, so I think it is acceptable, even though I am not a specialist in security.

I have been using Astro by Astronomer for the last three years.

Astro by Astronomer is stable.

I think Astro by Astronomer's scalability works well. We can run a lot of concurrent jobs or concurrent queries without having any problems, so I think it is acceptable.

The customer support for Astro by Astronomer is very good because sometimes I try to reach even the developer on GitHub, and they respond. I do not have any problems with support. I think it works well and is very friendly.

We do not use a different solution. Sometimes we use Step Functions in Amazon Web Services, but I think Astro by Astronomer is the best solution.

I did not choose Astro by Astronomer; my company chose it, and I do not know if there were other options at that time because I was not working at the company then.

We did not purchase Astro by Astronomer through the AWS Marketplace. We use an EC2 server in a virtual machine and installed it on the EC2 server.

I think we see a return on investment in time, as we saved a lot of time running queries, initiating EMR jobs, and deploying jobs in EMR. Afterward, we can disable the job and terminate the EMR cluster. I think this saves a lot of time in the process to deploy and terminate the instance.

I do not work with pricing, but I think the cost is very cheap. The pricing is not something I know about, as I do not handle pricing, license, and security in my company. I am just a tech person.

My advice for others looking into using Astro by Astronomer is to see the opportunities and try to test new solutions because there are many opportunities to work with this tool. I would rate this product a 9 out of 10.

A specific example of how Astro by Astronomer has made one of these processes easier in practice is that I developed a flow for the lifecycle of Machine Learning models for the search engine, in which the process automatically validates the latest trained models, executes integration tests, and if those are successful, takes care of moving the models to the production area and performing their automatic deployment, covering each step of the full Machine Learning lifecycle flow while maintaining risk mitigation and traceability of each step.

In addition, many of the ETLs are connected to Slack, which allows us to have traceability of when a task fails and to maintain good monitoring.

Astro by Astronomer has positively impacted my organization in the sense that when we have problems with tasks, Kubernetes pods, and other infrastructure components, we already have quite a bit of clarity about the reasons or failures that may be occurring.

I have seen improvements in response times and error reduction since using Astro by Astronomer. Previously, the costs we had for our model training, ETL processes, and batch transformations were totally invisible and did not give us the visibility to know how many resources and how much money we were spending on them. Now, with the visibility that the Astronomer team gives us, we have a way to track excessive resource usage to optimize it and reduce costs.

To add to the platform's features and about the support we receive, since we switched to Astro by Astronomer, we have not had serious problems regarding the management of our instances.

My experience with customer support has been quite pleasant, as we always get a response.