What needs improvement with Astro by Astronomer?

The current version of Astro by Astronomer is good enough for our needs. The documentation part of Astro by Astronomer can be made easier and more generic, not specific to the integration with each of the services. There are a few areas where improvement would be helpful. While it integrates well with Airflow and cloud services, configuring some third-party agents can require additional setup and troubleshooting.

Astro by Astronomer's CLI might be a little challenging, and having the CLI commands along with the documentation more readily available would be great.

Last year, as I shared with the product manager, I thought that it would be great to have a slower response time with Astro by Astronomer. I used to wait a few minutes to get a response and to see that the DAG was created completely, which I think is annoying when you work and need to wait until the suggestions were made after a few minutes. The speed, I think, was the part to improve, but the quality, for instance, was amazing. I think that one year ago, Astro by Astronomer needed to improve in reliability because it was a bit slow to develop and sometimes it crashed. As I said, it was before the launching date, so if they give me the opportunity to try it again, I will do that and test again. But the pain points were there.

I would say it would be much more helpful if Astro by Astronomer can provide an MCP to use and integrate with Astro by Astronomer and their AI agent Auto, so that we can use our tools like Claude, Code, or Cursor to have a central location where we integrate that MCP in Databricks via the AI gateway, and then we can have a place where we can connect to Astro by Astronomer, Databricks, other tools such as Jira and DataDog, and then have a much more comprehensive analysis and overview and monitoring of the entire data ecosystem. Having an MCP integration would tremendously help us to have a more comprehensive data platform that is more AI-ready. That is one of the features that I can think of. Another thought would be having an integration with Kubernetes instead of just scheduling everything on Astro by Astronomer workers. That would be helpful. One thing I can think of is that the Astro by Astronomer local developer environment can be improved further, but perhaps that is a very niche use case that I have in mind, and it may not be useful for others. There is room for improvement around having an improved UI and also integration with other tools. On governance, I would say we can have a more granular level of governance even inside workspaces. Having much more granular control over people belonging to the same team in Astro by Astronomer could have different access to different DAGs, depending upon our tagging strategy. That is one of the areas I can suggest in terms of improvement. The AI capabilities can be improved further. I would compare the current AI capabilities to newer models provided by Claude such as Opus or Codex.

I think any further refinements as far as anything major is going to be on the user's side. Everybody's got a different need, so trying to build an overfit solution is only going to hurt other people. The reason I choose eight for Astro by Astronomer is that it is useful for learning and for companies, small to large. There is still a barrier for people who are not familiar with some of the prerequisite knowledge, but I do also think that is not something that needs to be fixed. It is just the nature of how this works. As I said before, if you try to tailor it too much where people were getting almost a no-code experience with it, I think due to the nature of the services it provides, it would cause more trouble than it would fix. An eight means you're doing great.

We want to explore how to deal with multi-tenancy while using Astro by Astronomer, as it is something really important for us because it enhances user experience. Additionally, we aim to resolve some bugs that Airflow has in managed AWS, such as isolated environments, and in general, we want to optimize DevOps around local instances, which is really something cool that Astro by Astronomer provides. I think Astro by Astronomer could improve its pricing, as it is really expensive, especially if you want to have large use in a large-scale enterprise edition, which requires you to create a VPC hosted instance that significantly increases expenses. I believe they should work on this to provide more affordable solutions. I chose 8 out of 10 because there are deployment issues, at least initially, regarding VPC hosting and many other networking issues. Second is pricing, which is really expensive even just to start experimenting, as they also provide you a developer package for this. The total plan needs improvement, but other than that, it is acceptable.

Astro by Astronomer can be improved by adding managed infrastructure with zero setup deliverable developer tools. It can include a high-performance executor, which eliminates manual Airflow maintenance through cloud-hosted or remote execution deployment models. Key improvements for Astro by Astronomer include integrating with Terraform provider and multiple other tools. Regarding Astro by Astronomer's AI capabilities, I think its governance and security need to improve, but currently it can be used quite well, though there are some improvements needed for governance and security related to AI.

I believe there are several areas where Astro by Astronomer can improve. The pricing has become a little expensive because I have very large and always-running workloads. The enterprise configuration sometimes requires a learning curve that could be easier. Advanced customizations could be improved, and the documentation of complex topics needs enhancement. In enterprise scenarios, better documentation would help me more. I believe support is strong. Whenever I reach out to find issues or root causes, I get a very timely response from the support team. Integration is definitely an area that could be much better because of advanced enterprise requirements that need more configurable learning curves. If integration were easier, it would be simpler to deploy to different teams.