AWS Data Pipeline [EOL] vs AWS Glue comparison

Cancel
You must select at least 2 products to compare!
Amazon Web Services (AWS) Logo
views| comparisons
100% willing to recommend
Amazon Web Services (AWS) Logo
12,012 views|8,420 comparisons
92% willing to recommend
Comparison Buyer's Guide
Executive Summary

We performed a comparison between AWS Data Pipeline [EOL] and AWS Glue based on real PeerSpot user reviews.

Find out what your peers are saying about Amazon Web Services (AWS), MuleSoft, Matillion and others in Cloud Data Integration.
To learn more, read our detailed Cloud Data Integration Report (Updated: March 2024).
768,578 professionals have used our research since 2012.
Featured Review
Quotes From Members
We asked business professionals to review the solutions they use.
Here are some excerpts of what they said:
Pros
"The most valuable feature of the solution is that orchestration and development capabilities are easier with the tool.""It is a stable solution...It is a scalable solution."

More AWS Data Pipeline [EOL] Pros →

"Its user interface is quite good. You just need to choose some options to create a job in AWS Glue. The code-generation feature is also useful. If you don't want to customize it and simply want to read a file and store the data in the database, it can generate the code for you.""Our entire use case was very easily handled or solved using this solution.""We have found it beneficial when moving data from one source to another.""It is AWS-integrated. There is end-to-end integration with the other AWS services. It is also user-friendly.""One of the best features of the solution is its ability to easily integrate with other AWS services.""The most valuable feature for me is the visual interface of AWS Glue.""The solution is highly user-friendly, and its features are easy to use. The new addition of AWS Glue Data Catalog is also very beneficial, making the tool even more helpful for its users.""AWS Glue is quite better than other tools, but you have to learn it properly before you start using it."

More AWS Glue Pros →

Cons
"The user-defined functions have shortcomings in AWS Data Pipeline.""It's almost semi-automatic because you must review and approve code push, which works well. Still, we had many problems getting there during the deployment process, but we got there."

More AWS Data Pipeline [EOL] Cons →

"The monitoring is not that good.""The solution’s stability could be improved.""There should be more connectors for different databases.""The solution's visual ETL tool is of no use for actual implementation.""I have encountered challenges with multi-region support.""The technical support for this solution could be improved. In future, we would like to connect more services like Athena or Kinesis to help control more loads of data.""Currently, it supports only two languages in the background: Python and Scala. From our customization point of view, it would be helpful if it can also support Java in the background.""On occasion, the solution's dashboard reports that a project failed due to runtime but it actually succeeded."

More AWS Glue Cons →

Pricing and Cost Advice
  • "The way we use it, I think it is fair as we're getting a good value for money compared to having a server or some other data pipeline."
  • "I rate the pricing between six to eight on a scale from one to ten, where one is low price, and ten is high price."
  • More AWS Data Pipeline [EOL] Pricing and Cost Advice →

  • "The pricing is a bit higher than other solutions like Athena and EC2. If the pricing becomes more scaled or flexible, it will be good because you have to pay 44 cents just for one DPU for an hour. If you increase DPUs to 5 or 10, the pricing gets multiplied. There are also some time limits like 0 to 10 minutes or 10 to 20 minutes. If the pricing is according to the minutes, it would be better because you have to limit your job to 10 minutes or 20 minutes."
  • "It is not expensive. AWS Glue works on the serverless architecture. We get charged for the time the server is up. For our use case, we have to use it once in a day, and it is not expensive for us."
  • "Its price is good. We pay as we go or based on the usage, which is a good thing for us because it is simple to forecast for the tool. It is good in terms of the financial planning of the company, and it is a good way to estimate the cost. It is also simple for our clients. In my opinion, it is one of the best tools in the market for ETL processes because of the fact that you pay as you use, which separates it from other big tools such as PowerCenter, Pentaho Data Integration, and Talend."
  • "Technical support is a paid service, and which subscription you have is dependent on that. You must pay one of them, and it ranges from $15,000 to $25,000 per year."
  • "This solution is affordable and there is an option to pay for the solution based on your usage."
  • "AWS Glue is quite costly, especially for small organizations."
  • "AWS Glue uses a pay-as-you-go approach which is helpful. The price of the overall solution is low and is a great advantage."
  • "The overall cost of AWS Glue could be better. It cost approximately $1,000 a month. There is paid support available from AWS Glue."
  • More AWS Glue Pricing and Cost Advice →

    report
    Use our free recommendation engine to learn which Cloud Data Integration solutions are best for your needs.
    768,578 professionals have used our research since 2012.
    Questions from the Community
    Top Answer:The most valuable feature of the solution is that orchestration and development capabilities are easier with the tool.
    Top Answer:I rate the pricing between six to eight on a scale from one to ten, where one is low price, and ten is high price.
    Top Answer:The user-defined functions have shortcomings in AWS Data Pipeline. The user-defined functions could be one of the areas where I can write a custom function and embed it as a part of AWS Data Pipeline… more »
    Top Answer:AWS Glue and Azure Data factory for ELT best performance cloud services.
    Top Answer:We reviewed AWS Glue before choosing Talend Open Studio. AWS Glue is the managed ETL (extract, transform, and load) from Amazon Web Services. AWS Glue enables AWS users to create and manage jobs in… more »
    Top Answer:AWS Glue's main use case is for allowing users to discover, prepare, move, and integrate data from multiple sources. The product lets you use this data for analytics, application development, or… more »
    Ranking
    Unranked
    In Cloud Data Integration
    1st
    Views
    12,012
    Comparisons
    8,420
    Reviews
    32
    Average Words per Review
    419
    Rating
    7.8
    Comparisons
    Learn More
    Overview

    AWS Data Pipeline is a web service that helps you reliably process and move data between different AWS compute and storage services, as well as on-premises data sources, at specified intervals. ... AWS Data Pipeline also allows you to move and process data that was previously locked up in on-premises data silos.

    AWS Glue is a serverless cloud data integration tool that facilitates the discovery, preparation, movement, and integration of data from multiple sources for machine learning (ML), analytics, and application development. The solution includes additional productivity and data ops tooling for running jobs, implementing business workflows, and authoring.

    AWS Glue allows users to connect to more than 70 diverse data sources and manage data in a centralized data catalog. The solution facilitates visual creation, running, and monitoring of extract, transform, and load (ETL) pipelines to load data into users' data lakes. This Amazon product seamlessly integrates with other native applications of the brand and allows users to search and query cataloged data using Amazon EMR, Amazon Athena, and Amazon Redshift Spectrum.

    The solution also utilizes application programming interface (API) operations to transform users' data, create runtime logs, store job logic, and create notifications for monitoring job runs. The console of AWS Glue connects all of these services into a managed application, facilitating the monitoring and operational processes. The solution also performs provisioning and management of the resources required to run users' workloads in order to minimize manual work time for organizations.

    AWS Glue Features

    AWS Glue groups its features into four categories - discover, prepare, integrate, and transform. Within those groups are the following features:

    • Automatic schema discovery: AWS Glue crawlers connect to the organization's source or target data source through a prioritized list of classifiers to determine the schema for users' data. This feature creates metadata in companies' AWS Glue Data Catalog.

    • Schemas for data stream management: The AWS Glue Schema Registry enables users to validate and control the evolution of streaming data through registered Apache Avro schemas for no additional charge.

    • Automatic scaling based on workload: This feature dynamically scales resources up and down based on workload. The feature controls job resources, removing them depending on how much the workload can be split up.

    • FindMatches: This feature is for machine learning-based data deduplication and cleansing, and works by finding records that are imperfect matches of each other to remove useless data copies.

    • Edit, debug, and test ETL code: This feature helps users who have chosen to interactively develop their ETL code by providing development endpoints for editing, debugging, and testing the code it generates for them.

    • AWS Glue DataBrew: An interactive, point-and-click visual interface for specialists to clean and normalize data without the need to write any code.

    • AWS Glue Interactive Sessions: This feature simplifies the development of data integration jobs by enabling data engineers to interactively prepare and explore data.

    • AWS Glue Studio Job Notebooks: This AWS Glue feature provides serverless notebooks with minimal setup, allowing developers to start working in a timely manner.

    • Complex ETL pipeline building: This feature allows the product to be invoked on a schedule, on demand, or based on an event, allowing users to start multiple jobs in parallel or specify dependencies to build complex ETL pipelines.

    • AWS Glue Studio: This AWS Glue feature allows users to visually transform data through a drag-and-drop interface. The product automatically generates the code for ETL processes for users' data.

    AWS Glue Benefits

    AWS Glue offers a wide range of benefits for its users. These benefits include:

    • Users of other AWS products can easily onboard with AWS Glue, as it is integrated across a wide range of the company's services.

    • The solution is serverless, which allows for a lower total cost of ownership.

    • AWS Glue offers more power for users, as it automates much of the effort in building, maintaining, and running ETL jobs.

    • The product allows customers to easily discover and search across all their AWS datasets through AWS Glue Data Catalog.

    • AWS Glue does not require additional payment for managing and enforcing schemas for data streams.

    • The solution facilitates the authority of scalable ETL jobs for beginners and non-coding experts through a drag-and-drop interface.

    Reviews from Real Users

    Mustapha A., a cloud data engineer at Jems Groupe, likes AWS Glue because it is a product that is great for serverless data transformations.

    Liana I., CEO at Quark Technologies SRL, describes AWS Glue as a highly scalable, reliable, and beneficial pay-as-you-go pricing model.

    Sample Customers
    bp, Cerner, Expedia, Finra, HESS, intuit, Kellog's, Philips, TIME, workday
    bp, Cerner, Expedia, Finra, HESS, intuit, Kellog's, Philips, TIME, workday
    Top Industries
    VISITORS READING REVIEWS
    Computer Software Company20%
    Financial Services Firm18%
    Government9%
    Manufacturing Company7%
    REVIEWERS
    Computer Software Company47%
    Financial Services Firm18%
    Pharma/Biotech Company12%
    Consumer Goods Company6%
    VISITORS READING REVIEWS
    Financial Services Firm19%
    Computer Software Company14%
    Manufacturing Company7%
    Insurance Company7%
    Company Size
    VISITORS READING REVIEWS
    Small Business19%
    Midsize Enterprise15%
    Large Enterprise65%
    REVIEWERS
    Small Business29%
    Midsize Enterprise13%
    Large Enterprise58%
    VISITORS READING REVIEWS
    Small Business15%
    Midsize Enterprise12%
    Large Enterprise73%
    Buyer's Guide
    Cloud Data Integration
    March 2024
    Find out what your peers are saying about Amazon Web Services (AWS), MuleSoft, Matillion and others in Cloud Data Integration. Updated: March 2024.
    768,578 professionals have used our research since 2012.

    AWS Data Pipeline [EOL] doesn't meet the minimum requirements to be ranked in Cloud Data Integration with 2 reviews while AWS Glue is ranked 1st in Cloud Data Integration with 37 reviews. AWS Data Pipeline [EOL] is rated 8.0, while AWS Glue is rated 7.8. The top reviewer of AWS Data Pipeline [EOL] writes "A tool with great orchestration and development capabilities but needs to improve its user-defined functions". On the other hand, the top reviewer of AWS Glue writes "Provides serverless mechanism, easy data transformation and automated infrastructure management". AWS Data Pipeline [EOL] is most compared with AWS Database Migration Service, Oracle Data Integrator (ODI), FME, Perspectium DataSync and IBM InfoSphere DataStage, whereas AWS Glue is most compared with AWS Database Migration Service, Informatica PowerCenter, SSIS, Informatica Cloud Data Integration and Talend Open Studio.

    See our list of best Cloud Data Integration vendors.

    We monitor all Cloud Data Integration reviews to prevent fraudulent reviews and keep review quality high. We do not post reviews by company employees or direct competitors. We validate each review for authenticity via cross-reference with LinkedIn, and personal follow-up with the reviewer when necessary.