AWS Glue Reviews

In my role as the global lead for AWS solutions and offerings, we work with various clients, including large-scale clients, to adopt and implement AWS cloud offerings. 

Our primary focus revolves around cloud lift-and-shift migration, modernization, re-platforming, rehosting, data architecture, design strategy, and implementing generative AI-specific solutions across different industries such as banking, capital insurance, energy utilities, manufacturing, automotive, semiconductor, and aerospace and defense. 

For example, we have implemented AWS Glue at several client locations, utilizing its serverless data integration capabilities during the data discovery process, enterprise transformation, cleansing, transforming, and centralizing data.

AWS Glue has significantly improved our data quality, enhancing the data by removing duplicates and providing timely and efficient insights. 

It also aids in real-time data processing, reducing effort and cost due to its serverless architecture. These features ensure we maintain the highest level of scalability, reliability, and security compliance.

AWS Glue is fully managed, providing an easy-to-use integration environment to create, run, and monitor ETL jobs. It's broadly compatible and seamlessly integrates with other AWS services like Amazon S3, Redshift, and Athena. It's flexible with data integration, manages various data formats (JSON, ORC, CSV, etc.), and is serverless, eliminating the need for infrastructure management.

Its ease of use, cost-effectiveness, and highly secure architecture are some of the most valuable features.

There could be an enhanced way of managing pure metadata management or data cataloging. 

Additionally, while it covers a wide range of integrations with AWS services, integrating with certain additional or legacy products is not seamless and can be complex. 

Increasing support for more programming languages and improving advanced analytics capabilities could also be beneficial.

We have been working with AWS Glue for almost three-plus years now.

We haven't faced any stability issues with AWS Glue. It is a scalable solution, provided that the right design principles and workload management are implemented.

AWS Glue is a scalable solution due to its serverless architecture and efficient design.

My team handles interactions with AWS for technical support, ensuring our design architectures are scalable, flexible, and well-integrated. We often reach out to the AWS team to double-check our implementation mechanisms and guidelines.

Positive

The initial setup of AWS Glue is straightforward due to its serverless architecture and fully managed nature. Specific prerequisites need to be followed, such as setting up data sources, configuring IAM permissions, creating crawlers, and running ETL jobs.

My team escalates technical questions to AWS support, ensuring our design architectures are optimal. We have a partnership with AWS, and the technical team frequently reaches out to AWS for guidance on scalability, flexibility, and integration mechanisms.

We have seen an efficient process with AWS Glue, providing the right return on investment at the right time. It ensures efficiency for our clients, giving them the desired ROI within their expected timelines.

Follow the right design principles and involve AWS at the right time to leverage the most current features and offerings from AWS Glue. Ensuring the right architecture will mitigate any issues. I'd rate the solution eight out of ten.

I use AWS Glue primarily for ETL jobs. In my organization, it's just me using it as we are a small company. The IT team consists of four people, and I am the data engineering specialist.

The best thing about AWS Glue is its scalability and how easy it is to process a large amount of data. It integrates well with Redshift, S3, and AWS Glue catalog. 

For processing extensive data, having a managed Spark service fulfills that role. If you're already working on AWS and you need to process a lot of data that can't be handled on a single node or server, AWS Glue will serve you well. While it's quite expensive, it's valuable for large data processing needs.

Setting up pipelines is challenging, especially with version control and testing requirements. While the initial setup is easy, it doesn't accommodate more complex development needs. You might feel hesitant about changing pipelines that are already running and processing business-critical data due to limited versioning and testing capabilities.

I've been using AWS Glue since 2022, so for two years.

The stability of AWS Glue is fine. I haven't had any problems with it.

The scalability of AWS Glue is commendable.

Previously, in different jobs, I have worked with Databricks for ETL processes. I've also utilized Lambda functions for handling smaller data. I didn’t switch to AWS Glue, but used it in a different context.

The initial setup of AWS Glue is easy, yet not adequate for more complex requirements. If you need to do something robust, like creating a notebook, it is straightforward. 

However, when dealing with complex pipelines handling critical business data, it's hard to set up versioning and testing.

AWS Glue receives a hesitant five out of ten from me. I recommend it if you're already on AWS and need to process large data sets. However, for smaller data volumes, I would suggest Airflow because AWS Glue can be quite expensive.

I use the solution in my company for building datalake and for a variety of data sources like Oracle, MongoDB, and other multiple data sources, like SQL server, and AWS S3 buckets as a datalake storage tool, and then further we use AWS Glue to process it and move to AWS' search engine which will be like a lakehouse solution.

AWS Glue handles a huge volume of data, and it is a serverless tool. We don't need to put extra servers as long as the job runs or goes on with AWS Glue. Maintaining other options like AWS EMR can be very costly when handling the company's functions. AWS Glue creates a data catalog.

The drawbacks associated with the product stem from the fact that, based on the data volume, it can become very costly. There is a huge cost if the source system is not properly designed. If the changes are frequent and not valid, then, initially, you will use huge amounts of data in the ETL. The biggest challenges are associated with AWS Glue's costs, and it takes one-third of my entire pipeline cost.

I have been using AWS Glue for two years. I am a customer of the tool.

It is a very scalable solution. The problem with the tool is that it has a huge set of options, so there are a lot of hidden costs involved. Suppose you have to get into data quality, the billing needs to be done for the data quality part. As per what I learned, there is a need to scrape off AWS Glue and look for other solutions.

We use Amazon's services to provide technical support for the product. If you want to have support, Oracle and others offer a single support, and other tools have a direct support window. For Amazon, we need to pay 10 percent of my billing amount for the tool to get support services. Whether to raise a support ticket or not is an issue since ten percent is a huge amount. My company ends up using all the options without help from support. It is very difficult for any common man to understand why there is a need to pay ten percent for support. If I find an issue in the product, and I need to get support from AWS to fix it, then I need to pay ten percent of the tool's bill amount to Amazon. AWS is a very tricky tool because everything is evolving nowadays. AWS engineers are getting hired from other places, and even after that, if I am not getting any technical support, then things will be very nasty. There are some good engineers who help users outside the normal support cycle, but it doesn't meet their needs.

I rate the technical support a four out of ten.

Neutral

The product's installation phase is easy, especially since it is serverless.

The solution can be deployed in a day or two since it is serverless. AWS Glue alone doesn't work perfectly. If we have the right data model, then we can use AWS Glue. Inside AWS Glue, we use PySpark.

My company is in the mode of scraping off AWS Glue. My company is not approving the budget for us to use AWS Glue. I am trying to see some solutions that are not costly, like the ClickHouse database, which is an open-source tool.

The costs of the tool are huge, especially when moving from the source to the datalake.

I rate the tool an eight on a scale of one to ten, where one is expensive, and ten is expensive. I cannot have any predictability factor regarding the costs associated with the tool.

The main piece of AWS Glue is the ETL part. AWS Glue is for ETL to deal with S3 data sources to Redshift. We use AWS Glue for the CDC.

As the product is serverless, the tool runs fine. Most of the maintenance and monitoring are among the biggest challenges of the tool.

In terms of the product's ability to handle data volumes during scaling needs, I would say that though it offers the area of data volumes, the challenges are associated with costs.

The latency would be there if the source had a huge amount of data coming in, and so based on it, it would read the source system sequentially because of the way the CDC works. If I need to capture the change in the source change in the order, it can happen, and if you have a better network, you can also scale up by bringing the source to S3 or AWS Glue. When you can scale up, it is not really relevant for the group. The latency is not because of AWS Glue but because when it comes to ETL or CDC, I need to process it the same way I do it with AWS Glue. I cannot do parallel processing, and I need to do it sequentially.

I don't see any AI capabilities in the product, and it is more of an ETL solution.

As the product has many problems, people are moving to Bare Metal and other cloud services. Our company has spent a lot of time investigating what AWS Glue does, including the time required to use it to maintain the servers.

I need to spend on the product's maintenance along with the other activities for which I need to make payments to use the solution. Once you are able to predict the data volumes and other factors that are there over the cloud, it is possible to predict what my server will cost for the next five years and then get the servers at a very low price instead of depending on AWS.

Though it is a good solution, it is not cost-sensitive.

I rate the tool a six out of ten.

We use the solution to build tables on CSV data. We get data from some different sources, pull it in S3, and then create tables using Glue to get some metrics out of that data.

The entire use case was very easily handled or solved using AWS Glue. We had to get the files available in S3. The workflow was seamlessly integrated with the data, landing in S3, and then it detected changes made in the data. Configuring it was really easy. It gave us what we were looking for without going through a lot of hassle and that too within our budget.

The AWS Glue crawlers are valuable features. It is very versatile. It can detect the nature of the underlying data. It is quite smart, and it takes a lot of offloads rather than having to worry about configuring it or managing it.

There are output limitations and configuration of its three parts. There was a lot of trial and error that we had to go through. It is not clear how the partition discovery would have been affected by more data coming in. We've made some expensive mistakes, which, if there were any tutorials available or if there was easy documentation available with FAQs, could have been avoided. There is documentation, but it doesn't cover all.

There are three specific partition changes, and AWS Glue is tightly tied to Athena. We don't have much flexibility in managing the Athena.

AWS Glue could integrate with an AI model or a more advanced version that processes chat-based inputs rather than configuration. This would align it more closely with the functionalities of chat-based interfaces, making it easier to adopt.

I have been using AWS Glue for four to five years.

The product is stable. There are no issues, errors, or downtime. It is managed by AWS.

I rate the solution’s stability a nine out of ten.

The solution’s scalability is quite high. It is flexible and scalable. We haven't seen any challenges.

I rate the solution’s scalability an eight or nine out of ten.

The customer support is nice and very helpful without AWS premium support. There are community support, medium articles, and AWS knowledge-based articles, which provide AWS support.

Positive

I have used Spark before. We switched to AWS Glue since it is a managed solution. We were writing lines of code, managing it separately on our own, and sending the jobs on a cluster. It is taken care of by the managed service itself. We have to configure it, and then it takes care of all.

AWS Glue catalog was complex to understand. Deployment was very quick. We did some POCs. We were able to take it to production in about six to eight weeks.

We were using the console directly. There are no automated CI/CD. You can manage, create, and set up via the console using the AWS Glue service.

I rate the initial setup an eight out of ten, where one is easy, and ten is difficult.

Deployment was done in-house with the help of two data analysts.

The solution is expensive. It has a pay-as-you-go model. Whatever you are using, you are paying for that.

I rate the product’s pricing a six out of ten, where one is cheap and ten is expensive.

We are looking for Databricks. It is comparable to Spark but it was excessive for a use case. We didn't have the workload. It has a lot of additional features which we don't need. The cost is not justifiable.

Two of us are sufficient for the solution’s maintenance.

The solution is easy to set up and starts with a lot of standard data analytical use cases where we extract data. If you want something customizable, then look at other solutions because cost might be a factor for more advanced solutions.

Overall, I rate the solution a seven out of ten.

We use AWS Glue for building ETL pipelines.

The most valuable feature of AWS Glue is that it provides a GUI format with a drag-and-drop feature. The solution provides a codeless feature or no code feature, where you can write a pipeline without adding code in AWS Glue.

If you are working on AWS services for your pipeline, AWS Glue better interacts with the AWS services than other third-party tools. The solution provides inbuilt data quality and cataloging features. AWS Glue provides a complete package, and you can do all things in one place.

AWS Glue is more costly compared to other tools like Airflow. It would be better if the solution's pricing could be reduced. The default scheduling that AWS Glue provides is not as good as Airflow. The scheduler of AWS Glue could be improved because you cannot customize it.

I have been using AWS Glue for more than three years.

AWS Glue is a stable product because it's an AWS-managed service. We can directly contact AWS for any issues we face. If there are any glitches in the version, AWS solves the issue by creating patches or version upgrades to the solution.

Around 100 to 200 people are using the solution in our organization.

As AWS Glue is a SaaS product, you don't have to set up anything. You can just create a new job, write your script, and run your job. You have to select the particular cluster or nodes you want to run and write the code. Not much admin part is required for the solution's setup.

AWS Glue is a paid service that doesn't come under the free trial of AWS. You have to pay a charge for using the solution.

If you are doing a job once or twice a day, AWS Glue will not cost you much. If you run jobs for four to five times a day or hourly jobs, it will be costlier compared to other tools. If you are required to run hourly jobs five to six times a day, then using other tools would be a better option. You can choose AWS Glue if you are running jobs only one or two times a day.

Our company decided to go with AWS Glue because the tools we were using in the pipeline were AWS services only. AWS Glue easily interacts with AWS services. The jobs we were running were also not frequent.

You can use AWS Glue for learning purposes. AWS Glue is a paid service that doesn't come under the free trial of AWS. You have to pay a charge for using the solution. You can learn the code by directly testing the basic spark code in any local system. Once you are comfortable that your code is working fine, then you can run your code in AWS Glue jobs. You should test the code in the local system first and then run it in AWS Glue. Testing on AWS Glue will be costly.

If a person is familiar with Spark jobs or Python jobs, they can easily learn AWS Glue. A new user will take the same amount of time to learn AWS Glue as he takes to get comfortable with Spark. Since it provides the GUI and no code thing, users can directly start using AWS Glue without having to learn any code. It's much easier to learn to use the solution.

Overall, I rate the solution a seven out of ten.

In my company, we use AWS Glue to build data engineering pipelines, so we ingest data from either S3 or other sources and put it back into Redshift, where we have a data lake or data warehouse.

With AWS Glue, there have been a lot of updates over the past couple of years. The feature which I like the most is that AWS Glue has now integrated Jupyter Notebook into it, causing AWS Glue to become interactive and develop new use cases. The fact that we don't have to worry about the resources required for any particular job since it gets taken care of by AWS Glue adds to the valuable feature set of the product.

AWS Glue Studio has undergone a lot of enhancements in the last couple of months. An improvement that can help the solution is if the user interface can become more user-friendly and allow for features like drag and drop, allowing it to build transformations. There can be a good improvement if the product itself supports different kinds of transformations so that the pipeline, which we want to create, can be done easily since right now, we have to write a code to do so in our company. Only people who can code, either in Java or Python, can use the product freely. Those who don't know Java or Python might find using AWS Glue difficult.

AWS has pricing for spot instances that reduces the cost substantially, but that is not available for AWS Glue AWS pricing for spot instances comes for products like EC2, and if the same gets introduced for AWS Glue, then the pricing can substantially reduce.

I have been using AWS Glue for five years. I am an end user of the product.

Stability-wise, I rate the solution an eight out of ten.

It is a very scalable solution. In our company, we don't have to bother about the latest features implemented by the solution since the solution itself allocates resources, causing the product to scale up or down as the job progresses.

The solution's technical support is good. Whenever we raise a use case where we face an issue in our company, we get a response from the solution's technical team.

AWS Glue is a platform as a service from AWS, so there is no setup required. It's very simple since you can just go into the UI and start using it.

The solution is deployed on a private cloud.

I am a part of the development team, which doesn't have access to all the different sets of features and has a different backup. There is a team that does the initial setup, user access, and all those things. Once the product is ready, my team takes it over, so the setup is done by someone else.

I rate the product's pricing a five on a scale of one to ten, where one is a high price, and ten is a low price.

AWS Glue can be compared to the database provided by Azure which has a lot of extra features or capabilities compared to AWS Glue which is only useful for processing.

I rate the overall product an eight out of ten.

I have been working as a data engineer, where dealing with the ETL process is essential. We are using AWS Glue as a primary ETL tool to serve our organization's needs. I have implemented several Glue jobs still in production.

AWS Glue has enabled us to perform ETL processes efficiently, with ease of use for AWS cloud users, providing a serverless service that eliminates the need for infrastructure maintenance.

The two features I find most valuable in AWS Glue are its user interface and ease of use. The user interface is intuitive, and navigating through the Glue console is seamless. 

Additionally, its ability to integrate with other AWS services is excellent, providing flawless coordination with services such as SNS, S3, and Lambda.

I see scope for improvement in the drag-and-drop feature of AWS Glue. Beginners need additional support as it currently lacks some features required for complex transformations, often necessitating custom Python coding.

I have been using Glue for more than five years now.

Overall, the stability of AWS Glue is excellent. I would rate it a nine out of ten. Some network-related issues may arise. That said, they are rare and do not affect its functionality significantly.

Regarding scalability, AWS Glue is nearly perfect. I would rate it a nine out of ten, although there is always room for improvement.

AWS customer service is great, but there is room for improvement. The issue I face is the inconsistency in dealing with different customer service representatives for the same issue, which disrupts personal touch.

Neutral

On an organizational level, the pricing of AWS Glue does not pose a concern. It is in line with other ETL tools in the market. However, AWS Glue's cost to free-tier users is an issue because it is not entirely free, even for trial purposes.

I advise potential users to adopt AWS Glue primarily due to its user-friendly interface, extensive documentation, and seamless integration with other AWS services, making it ideal for data engineers.

I'd rate the solution nine out of ten.

We use AWS Glue for data analytics.

AWS Glue is a stable and easy-to-use solution.

The solution’s stability could be improved.

I have been using AWS Glue for the last three years.

I rate AWS Glue a seven out of ten for stability.

AWS Glue is a very scalable solution, and you can connect multiple databases.

AWS Glue's technical support is very good.

AWS Glue is not a licensed solution. AWS Glue follows a pay-as-you-go model, wherein the cost of the data you use will be counted as a monthly bill.

Currently, there are many ETL tools in the marketplace. Compared to other ETL tools, AWS Glue is a low-cost and serverless solution.

Overall, I rate AWS Glue a nine out of ten.