AWS Glue vs StreamSets comparison

Cancel
You must select at least 2 products to compare!
Amazon Logo
22,376 views|17,391 comparisons
StreamSets Logo
5,913 views|3,583 comparisons
Comparison Buyer's Guide
Executive Summary

We performed a comparison between AWS Glue and StreamSets based on real PeerSpot user reviews.

Find out in this report how the two Cloud Data Integration solutions compare in terms of features, pricing, service and support, easy of deployment, and ROI.
To learn more, read our detailed AWS Glue vs. StreamSets Report (Updated: November 2022).
655,994 professionals have used our research since 2012.
Featured Review
Quotes From Members
We asked business professionals to review the solutions they use.
Here are some excerpts of what they said:
Pros
"Transformations are valuable because you can modify or override complex data logic from an open source or Spark to solve issues.""The facility to integrate with S3 and the possibility to use Jupyter Notebook inside the pipeline are the most valuable features.""The most valuable feature of AWS Glue is its ease of use and good documentation. Additionally, we can do all the transformations that we need.""One of the best features of the solution is its ability to easily integrate with other AWS services.""I like that it's flexible, powerful, and allows you to write your own queries and scripts to get the needed transformations.""It is AWS-integrated. There is end-to-end integration with the other AWS services. It is also user-friendly.""AWS Glue is a good solution for developers, they have the ability to write code in different languages and other software.""AWS Glue's most valuable features are the data catalog, including crawlers and tables, and Glue Studio, which means you don't have to use custom code."

More AWS Glue Pros →

"I have used Data Collector, Transformer, and Control Hub products from StreamSets. What I really like about these products is that they're very user-friendly. People who are not from a technological or core development background find it easy to get started and build data pipelines and connect to the databases. They would be comfortable like any technical person within a couple of weeks.""It is a very powerful, modern data analytics solution, in which you can integrate a large volume of data from different sources. It integrates all of the data and you can design, create, and monitor pipelines according to your requirements. It is an all-in-one day data ops solution.""In StreamSets, everything is in one place.""StreamSets’ data drift resilience has reduced the time it takes us to fix data drift breakages. For example, in our previous Hadoop scenario, when we were creating the Sqoop-based processes to move data from source to destinations, we were getting the job done. That took approximately an hour to an hour and a half when we did it with Hadoop. However, with the StreamSets, since it works on a data collector-based mechanism, it completes the same process in 15 minutes of time. Therefore, it has saved us around 45 minutes per data pipeline or table that we migrate. Thus, it reduced the data transfer, including the drift part, by 45 minutes.""StreamSets data drift feature gives us an alert upfront so we know that the data can be ingested. Whatever the schema or data type changes, it lands automatically into the data lake without any intervention from us, but then that information is crucial to fix for downstream pipelines, which process the data into models, like Tableau and Power BI models. This is actually very useful for us. We are already seeing benefits. Our pipelines used to break when there were data drift changes, then we needed to spend about a week fixing it. Right now, we are saving one to two weeks. Though, it depends on the complexity of the pipeline, we are definitely seeing a lot of time being saved."

More StreamSets Pros →

Cons
"The technical support for this solution could be improved. In future, we would like to connect more services like Athena or Kinesis to help control more loads of data.""On occasion, the solution's dashboard reports that a project failed due to runtime but it actually succeeded.""Overall, I consider the technical support to be fine, although the response time could be faster in certain cases.""The interface for AWS Glue could improve, they do not put a lot of details. You can write the code, in PySpark or in Scala, which is a big advantage, it is only easy to use for a developer. It will be difficult for new users to enter the cloud environment.""The solution should offer features for streaming data in addition to batching data.""There should be more connectors for different databases.""It would be better if it were more user-friendly. The interesting thing we found is that it was a little strange at the beginning. The way Glue works is not very straightforward. After trying different things, for example, we used just the console to create jobs. Then we realized that things were not working as expected. After researching and learning more, we realized that even though the console creates the script for the ETL processes, you need to modify or write your own script in Spark to do everything you want it to do. For example, we are pulling data from our source database and our application database, which is in Aurora. From there, we are doing the ETL to transform the data and write the results into Redshift. But what was surprising is that it's almost like whatever you want to do, you can do it with Glue because you have the option to put together your own script. Even though there are many functionalities and many connections, you have the opportunity to write your own queries to do whatever transformations you need to do. It's a little deceiving that some options are supposed to work in a certain way when you set them up in the console, but then they are not exactly working the right way or not as expected. It would be better if they provided more examples and more documentation on options.""If there's a cluster-related configuration, we have to make worker notes, which is quite a headache when processing a large amount of data."

More AWS Glue Cons →

"Currently, we can only use the query to read data from SAP HANA. What we would like to see, as soon as possible, is the ability to read from multiple tables from SAP HANA. That would be a really good thing that we could use immediately. For example, if you have 100 tables in SQL Server or Oracle, then you could just point it to the schema or the 100 tables and ingestion information. However, you can't do that in SAP HANA since StreamSets currently is lacking in this. They do not have a multi-table feature for SAP HANA. Therefore, a multi-table origin for SAP HANA would be helpful.""Sometimes, when we have large amounts of data that is very efficiently stored in Hadoop or Kafka, it is not very efficient to run it through StreamSets, due to the lack of efficiency or the resources that StreamSets is using.""We create pipelines or jobs in StreamSets Control Hub. It is a great feature, but if there is a way to have a folder structure or organize the pipelines and jobs in Control Hub, it would be great. I submitted a ticket for this some time back.""If you use JDBC Lookup, for example, it generally takes a long time to process data.""The logging mechanism could be improved. If I am working on a pipeline, then create a job out of it and it is running, it will generate constant logs. So, the logging mechanism could be simplified. Now, it is a bit difficult to understand and filter the logs. It takes some time."

More StreamSets Cons →

Pricing and Cost Advice
  • "Its price is good. We pay as we go or based on the usage, which is a good thing for us because it is simple to forecast for the tool. It is good in terms of the financial planning of the company, and it is a good way to estimate the cost. It is also simple for our clients. In my opinion, it is one of the best tools in the market for ETL processes because of the fact that you pay as you use, which separates it from other big tools such as PowerCenter, Pentaho Data Integration, and Talend."
  • "Technical support is a paid service, and which subscription you have is dependent on that. You must pay one of them, and it ranges from $15,000 to $25,000 per year."
  • "This solution is affordable and there is an option to pay for the solution based on your usage."
  • "AWS Glue is quite costly, especially for small organizations."
  • "AWS Glue uses a pay-as-you-go approach which is helpful. The price of the overall solution is low and is a great advantage."
  • "The overall cost of AWS Glue could be better. It cost approximately $1,000 a month. There is paid support available from AWS Glue."
  • "The solution's pricing is based on DPUs so it is a good idea to optimize use or it can get expensive."
  • "The current cost is around forty to fifty thousand a month."
  • More AWS Glue Pricing and Cost Advice →

  • "StreamSets Data Collector is open source. One can utilize the StreamSets Data Collector, but the Control Hub is the main repository where all the jobs are present. Everything happens in Control Hub."
  • "It has a CPU core-based licensing, which works for us and is quite good."
  • "There are different versions of the product. One is the corporate license version, and the other one is the open-source or free version. I have been using the corporate license version, but they have recently launched a new open-source version so that anybody can create an account and use it. The licensing cost varies from customer to customer. I don't have a lot of input on that. It is taken care of by PMO, and they seem fine with its pricing model. It is being used enterprise-wide. They seem to have got a good deal for StreamSets."
  • "The pricing is good, but not the best. They have some customized plans you can opt for."
  • More StreamSets Pricing and Cost Advice →

    report
    Use our free recommendation engine to learn which Cloud Data Integration solutions are best for your needs.
    655,994 professionals have used our research since 2012.
    Questions from the Community
    Top Answer:AWS Glue and Azure Data factory for ELT best performance cloud services.
    Top Answer:We reviewed AWS Glue before choosing Talend Open Studio. AWS Glue is the managed ETL (extract, transform, and load) from Amazon Web Services. AWS Glue enables AWS users to create and manage jobs in… more »
    Top Answer:The facility to integrate with S3 and the possibility to use Jupyter Notebook inside the pipeline are the most valuable features.
    Top Answer:It is really easy to set up and the interface is easy to use.
    Top Answer:We've seen a couple of cases where it appears to have a memory leak or a similar problem. It grows for a bit and then we'd have to restart the container, maybe once a month when it gets high.
    Top Answer:We typically use it to transport our Oracle raw datasets up to Microsoft Azure, and then into SQL databases there.
    Ranking
    2nd
    Views
    22,376
    Comparisons
    17,391
    Reviews
    14
    Average Words per Review
    442
    Rating
    7.9
    11th
    Views
    5,913
    Comparisons
    3,583
    Reviews
    5
    Average Words per Review
    2,142
    Rating
    8.4
    Comparisons
    Learn More
    StreamSets
    Video Not Available
    Overview

    AWS Glue is a serverless cloud data integration tool that facilitates the discovery, preparation, movement, and integration of data from multiple sources for machine learning (ML), analytics, and application development. The solution includes additional productivity and data ops tooling for running jobs, implementing business workflows, and authoring.

    AWS Glue allows users to connect to more than 70 diverse data sources and manage data in a centralized data catalog. The solution facilitates visual creation, running, and monitoring of extract, transform, and load (ETL) pipelines to load data into users' data lakes. This Amazon product seamlessly integrates with other native applications of the brand and allows users to search and query cataloged data using Amazon EMR, Amazon Athena, and Amazon Redshift Spectrum.

    The solution also utilizes application programming interface (API) operations to transform users' data, create runtime logs, store job logic, and create notifications for monitoring job runs. The console of AWS Glue connects all of these services into a managed application, facilitating the monitoring and operational processes. The solution also performs provisioning and management of the resources required to run users' workloads in order to minimize manual work time for organizations.

    AWS Glue Features

    AWS Glue groups its features into four categories - discover, prepare, integrate, and transform. Within those groups are the following features:

    • Automatic schema discovery: AWS Glue crawlers connect to the organization's source or target data source through a prioritized list of classifiers to determine the schema for users' data. This feature creates metadata in companies' AWS Glue Data Catalog.

    • Schemas for data stream management: The AWS Glue Schema Registry enables users to validate and control the evolution of streaming data through registered Apache Avro schemas for no additional charge.

    • Automatic scaling based on workload: This feature dynamically scales resources up and down based on workload. The feature controls job resources, removing them depending on how much the workload can be split up.

    • FindMatches: This feature is for machine learning-based data deduplication and cleansing, and works by finding records that are imperfect matches of each other to remove useless data copies.

    • Edit, debug, and test ETL code: This feature helps users who have chosen to interactively develop their ETL code by providing development endpoints for editing, debugging, and testing the code it generates for them.

    • AWS Glue DataBrew: An interactive, point-and-click visual interface for specialists to clean and normalize data without the need to write any code.

    • AWS Glue Interactive Sessions: This feature simplifies the development of data integration jobs by enabling data engineers to interactively prepare and explore data.

    • AWS Glue Studio Job Notebooks: This AWS Glue feature provides serverless notebooks with minimal setup, allowing developers to start working in a timely manner.

    • Complex ETL pipeline building: This feature allows the product to be invoked on a schedule, on demand, or based on an event, allowing users to start multiple jobs in parallel or specify dependencies to build complex ETL pipelines.

    • AWS Glue Studio: This AWS Glue feature allows users to visually transform data through a drag-and-drop interface. The product automatically generates the code for ETL processes for users' data.

    AWS Glue Benefits

    AWS Glue offers a wide range of benefits for its users. These benefits include:

    • Users of other AWS products can easily onboard with AWS Glue, as it is integrated across a wide range of the company's services.

    • The solution is serverless, which allows for a lower total cost of ownership.

    • AWS Glue offers more power for users, as it automates much of the effort in building, maintaining, and running ETL jobs.

    • The product allows customers to easily discover and search across all their AWS datasets through AWS Glue Data Catalog.

    • AWS Glue does not require additional payment for managing and enforcing schemas for data streams.

    • The solution facilitates the authority of scalable ETL jobs for beginners and non-coding experts through a drag-and-drop interface.

    Reviews from Real Users

    Mustapha A., a cloud data engineer at Jems Groupe, likes AWS Glue because it is a product that is great for serverless data transformations.

    Liana I., CEO at Quark Technologies SRL, describes AWS Glue as a highly scalable, reliable, and beneficial pay-as-you-go pricing model.

    StreamSets offers an end-to-end data integration platform to build, run, monitor and manage smart data pipelines that deliver continuous data for DataOps, and power the modern data ecosystem and hybrid integration.

    Only StreamSets provides a single design experience for all design patterns for 10x greater developer productivity; smart data pipelines that are resilient to change for 80% less breakages; and a single pane of glass for managing and monitoring all pipelines across hybrid and cloud architectures to eliminate blind spots and control gaps.

    With StreamSets, you can deliver the continuous data that drives the connected enterprise.

    Offer
    Learn more about AWS Glue
    Learn more about StreamSets
    Sample Customers
    bp, Cerner, Expedia, Finra, HESS, intuit, Kellog's, Philips, TIME, workday
    Availity, BT Group, Humana, Deluxe, GSK, RingCentral, IBM, Shell, SamTrans, State of Ohio, TalentFulfilled, TechBridge
    Top Industries
    REVIEWERS
    Computer Software Company50%
    Pharma/Biotech Company25%
    Consumer Goods Company13%
    Financial Services Firm13%
    VISITORS READING REVIEWS
    Computer Software Company17%
    Financial Services Firm14%
    Insurance Company8%
    Media Company7%
    VISITORS READING REVIEWS
    Financial Services Firm17%
    Computer Software Company14%
    Manufacturing Company7%
    Insurance Company7%
    Company Size
    REVIEWERS
    Small Business22%
    Midsize Enterprise22%
    Large Enterprise56%
    VISITORS READING REVIEWS
    Small Business14%
    Midsize Enterprise11%
    Large Enterprise75%
    REVIEWERS
    Small Business22%
    Midsize Enterprise33%
    Large Enterprise44%
    VISITORS READING REVIEWS
    Small Business15%
    Midsize Enterprise11%
    Large Enterprise74%
    Buyer's Guide
    AWS Glue vs. StreamSets
    November 2022
    Find out what your peers are saying about AWS Glue vs. StreamSets and other solutions. Updated: November 2022.
    655,994 professionals have used our research since 2012.

    AWS Glue is ranked 2nd in Cloud Data Integration with 14 reviews while StreamSets is ranked 11th in Data Integration Tools with 5 reviews. AWS Glue is rated 7.8, while StreamSets is rated 8.4. The top reviewer of AWS Glue writes "Easy to perform ETL on multiple data sources, and easy to use after you learn it". On the other hand, the top reviewer of StreamSets writes "Integrates with different enterprise systems and enables us to easily build data pipelines without knowing how to code". AWS Glue is most compared with AWS Database Migration Service, Informatica PowerCenter, Talend Open Studio, Informatica Cloud Data Integration and SSIS, whereas StreamSets is most compared with Informatica PowerCenter, SSIS, Oracle GoldenGate, Spring Cloud Data Flow and Matillion ETL. See our AWS Glue vs. StreamSets report.

    We monitor all Cloud Data Integration reviews to prevent fraudulent reviews and keep review quality high. We do not post reviews by company employees or direct competitors. We validate each review for authenticity via cross-reference with LinkedIn, and personal follow-up with the reviewer when necessary.