AWS Glue vs StreamSets comparison

Amazon Web Services (AWS) and IBM are both solutions in the Cloud Data Integration category. Additionally, 90% of Amazon Web Services (AWS) users are willing to recommend the solution, compared to 100% of IBM users who would recommend it.

AWS Glue

Read 41 AWS Glue reviews

10,583 Views
7,453 Comparison Views

90% willing to recommend

StreamSets

Read 24 StreamSets reviews

3,261 Views
2,029 Comparison Views

100% willing to recommend

AWS Glue

StreamSets

Comparison Buyer's Guide

Download the report

Executive Summary

We performed a comparison between AWS Glue and StreamSets based on real PeerSpot user reviews.

Find out in this report how the two Cloud Data Integration solutions compare in terms of features, pricing, service and support, easy of deployment, and ROI.

To learn more, read our detailed AWS Glue vs. StreamSets Report (Updated: October 2024).

Buyer's Guide

AWS Glue vs. StreamSets

October 2024

Download the complete report

Helped 813,418 peers since 2012

Categories and Ranking

AWS Glue

Average Rating

7.8

Number of Reviews

Ranking in other categories

Cloud Data Integration (1st)

StreamSets

Average Rating

8.4

Number of Reviews

Ranking in other categories

Data Integration (9th)

Featured Reviews

Ajaykumar Myana

Senior Software Developer at a computer software company with 10,001+ employees

Jul 31, 2023

Provides serverless mechanism, easy data transformation and automated infrastructure management

I had the source data, which was unstructured and non-fixable, and my responsibility was to convert it into structured data. For this task, I used PySpark as the programming language. With Python, I implemented the creation of a data frame using Glue jobs. Since Glue jobs are a serverless…

Read full review

Reyansh Kumar

Technical Specialist at Accenture

Mar 10, 2023

We no longer need to hire highly skilled data engineers to create and monitor data pipelines

The things I like about StreamSets are its * overall user interface * efficiency * product features, which are all good. Also, the scheduling within the data engineering pipeline is very much appreciated, and it has a wide range of connectors for connecting to any data sources like SQL Server, AWS, Azure, etc. We have used it with Kafka, Hadoop, and Azure Data Factory Datasets. Connecting to these systems with StreamSets is very easy. You just need to configure the data sources, the paths and their configurations, and you are ready to go. It is very efficient and very easy to use for ETL pipelines. It is a GUI-based interface in which you can easily create or design your own data pipelines with just a few clicks. As for moving data into modern analytics systems, we are using it with Microsoft Power BI, AWS, and some on-premises solutions, and it is very easy to get data from StreamSets into them. No hardcore coding or special technical expertise is required. It is also a no-code platform in which you can configure your data sources and data output for easy configuration of your data pipeline. This is a very important aspect because if a tool requires code development, we need to hire software developers to get the task done. By using StreamSets, it can be done with a few clicks.

Read full review

Quotes from Members

We asked business professionals to review the solutions they use. Here are some excerpts of what they said:

Pros

"It's fairly straightforward as a product; it's not very complicated."

"Our entire use case was very easily handled or solved using this solution."

"AWS Glue is quite better than other tools, but you have to learn it properly before you start using it."

"It's very good to manage."

"The most valuable feature of AWS Glue is that it provides a GUI format with a drag-and-drop feature."

"Its ease of use, cost-effectiveness, and highly secure architecture are some of the most valuable features."

"The solution is highly user-friendly, and its features are easy to use. The new addition of AWS Glue Data Catalog is also very beneficial, making the tool even more helpful for its users."

"What I like best about AWS Glue is its real-time data backup feature. Last week, there was a production push, and what used to take almost ten days to send out around fifty-six thousand emails now takes only two hours."

More AWS Glue pros

"The most valuable features are the option of integration with a variety of protocols, languages, and origins."

"The best thing about StreamSets is its plugins, which are very useful and work well with almost every data source. It's also easy to use, especially if you're comfortable with SQL. You can customize it to do what you need. Many other tools have started to use features similar to those introduced by StreamSets, like automated workflows that are easy to set up."

"The ETL capabilities are very useful for us. We extract and transform data from multiple data sources, into a single, consistent data store, and then we put it in our systems. We typically use it to connect our Apache Kafka with data lakes. That process is smooth and saves us a lot of time in our production systems."

"The most valuable feature is the pipelines because they enable us to pull in and push out data from different sources and to manipulate and clean things up within them."

"Important features include that it comprises lots of functionality to connect data from various sources through connector availability, scheduling pipelines at any time, and integration with third-party and security solutions for encryption."

"What I love the most is that StreamSets is very light. It's a containerized application. It's easy to use with Docker. If you are a large organization, it's very easy to use Kubernetes."

"It is really easy to set up and the interface is easy to use."

"The entire user interface is very simple and the simplicity of creating pipelines is something that I like very much about it. The design experience is very smooth."

More StreamSets pros

Cons

"AWS Glue is more costly compared to other tools like Airflow."

"Cost-wise, AWS Glue is expensive, so that's an area for improvement. The process for setting up the solution was also complex, which is another area for improvement."

"The start-up time is really high right now. For instance, when you start up a new job, you have to wait for five or eight minutes before it starts. If the start-up time is reduced to one or two minutes, it will be great. It will be better to have a direct linkage to Redshift in AWS. If we can use data catalogs from Redshift, it will be so easy to create some data catalogs. Currently, we can only use data catalogs from S3."

"AWS Glue's error handling is difficult."

"If there's a cluster-related configuration, we have to make worker notes, which is quite a headache when processing a large amount of data."

"I have encountered challenges with multi-region support."

"The product has only a few built-in transformations."

"There is a learning curve to this tool."

More AWS Glue cons

"Sometimes, it is not clear at first how to set up nodes. A site with an explanation of how each node works would be very helpful."

"The logging mechanism could be improved. If I am working on a pipeline, then create a job out of it and it is running, it will generate constant logs. So, the logging mechanism could be simplified. Now, it is a bit difficult to understand and filter the logs. It takes some time."

"The monitoring visualization is not that user-friendly. It should include other features to visualize things, like how many records were streamed from a source to a destination on a particular date."

"The documentation is inadequate and has room for improvement because the technical support does not regularly update their documentation or the knowledge base."

"The software is very good overall. Areas for improvement are the error logging and the version history. I would like to see better, more detailed error logging information."

"One area for improvement could be the cloud storage server speed, as we have faced some latency issues here and there."

"We've seen a couple of cases where it appears to have a memory leak or a similar problem."

"We create pipelines or jobs in StreamSets Control Hub. It is a great feature, but if there is a way to have a folder structure or organize the pipelines and jobs in Control Hub, it would be great. I submitted a ticket for this some time back."

More StreamSets cons

Pricing and Cost Advice

"I rate the product's pricing a five on a scale of one to ten, where one is a high price, and ten is a low price."

"This solution is affordable and there is an option to pay for the solution based on your usage."

"AWS Glue is a high-priced solution that bills the client $150,000 to $250,000 annually."

"The pricing is a bit higher than other solutions like Athena and EC2. If the pricing becomes more scaled or flexible, it will be good because you have to pay 44 cents just for one DPU for an hour. If you increase DPUs to 5 or 10, the pricing gets multiplied. There are also some time limits like 0 to 10 minutes or 10 to 20 minutes. If the pricing is according to the minutes, it would be better because you have to limit your job to 10 minutes or 20 minutes."

"AWS Glue uses a pay-as-you-go approach which is helpful. The price of the overall solution is low and is a great advantage."

"It is not expensive. AWS Glue works on the serverless architecture. We get charged for the time the server is up. For our use case, we have to use it once in a day, and it is not expensive for us."

"The solution's pricing is based on DPUs so it is a good idea to optimize use or it can get expensive."

"If you are using the solution for an enterprise business, it will be expensive."

More AWS Glue pricing and cost advice

"We are running the community version right now, which can be used free of charge."

"Its pricing is pretty much up to the mark. For smaller enterprises, it could be a big price to pay at the initial stage of operations, but the moment you have the Seed B or Seed C funding and you want to scale up your operations and aren't much worried about the funds, at that point in time, you would need a solution that could be scaled."

"StreamSets is an expensive solution."

"The overall cost is very flexible so it is not a burden for our organization... However, the cost should be improved. For small and mid-size organizations it might be a challenge."

"The overall cost for small and mid-size organizations needs to be better."

"The pricing is too fixed. It should be based on how much data you need to process. Some businesses are not so big that they process a lot of data."

"It has a CPU core-based licensing, which works for us and is quite good."

"I believe the pricing is not equitable."

More StreamSets pricing and cost advice

See which vendors are best for you

Use our free recommendation engine to learn which Cloud Data Integration solutions are best for your needs.

See recommendations

813,418 professionals have used our research since 2012.

Top Industries

By visitors reading reviews

Financial Services Firm

21%

Computer Software Company

14%

Manufacturing Company

Insurance Company

Financial Services Firm

18%

Computer Software Company

13%

Manufacturing Company

10%

Government

Company Size

By reviewers

Large Enterprise

Midsize Enterprise

Small Business

Questions from the Community

How do you select the right cloud ETL tool?

AWS Glue and Azure Data factory for ELT best performance cloud services.

See all answers

How does Talend Open Studio compare with AWS Glue?

We reviewed AWS Glue before choosing Talend Open Studio. AWS Glue is the managed ETL (extract, transform, and load) from Amazon Web Services. AWS Glue enables AWS users to create and manage jobs in...

See all answers

What are the most common use cases for AWS Glue?

AWS Glue's main use case is for allowing users to discover, prepare, move, and integrate data from multiple sources. The product lets you use this data for analytics, application development, or ma...

See all answers

What do you like most about StreamSets?

The best thing about StreamSets is its plugins, which are very useful and work well with almost every data source. It's also easy to use, especially if you're comfortable with SQL. You can customiz...

See all answers

What needs improvement with StreamSets?

We often faced problems, especially with SAP ERP. We struggled because many columns weren't integers or primary keys, which StreamSets couldn't handle. We had to restructure our data tables, which ...

See all answers

What is your primary use case for StreamSets?

StreamSets is used for data transformation rather than ETL processes. It focuses on transforming data directly from sources without handling the extraction part of the process. The transformed data...

See all answers

Comparisons

AWS Database Migration Service vs AWS Glue

Compared 33% of the time

Informatica Intelligent Data Management Cloud (IDMC) vs AWS Glue

Compared 8% of the time

Informatica PowerCenter vs AWS Glue

Compared 7% of the time

SSIS vs AWS Glue

Compared 5% of the time

Talend Open Studio vs AWS Glue

Compared 5% of the time

More AWS Glue Competitors

Informatica PowerCenter vs StreamSets

Compared 12% of the time

IBM InfoSphere DataStage vs StreamSets

Compared 11% of the time

Fivetran vs StreamSets

Compared 10% of the time

Azure Data Factory vs StreamSets

Compared 9% of the time

Oracle GoldenGate vs StreamSets

Compared 3% of the time

More StreamSets Competitors

Product Reports

Buyer's Guide

AWS Glue

October 2024

Download AWS Glue product report

Buyer's Guide

StreamSets

September 2024

Download StreamSets product report

Learn More

Amazon Web Services (AWS)

Video not available

IBM

Overview

AWS Glue is a serverless cloud data integration tool that facilitates the discovery, preparation, movement, and integration of data from multiple sources for machine learning (ML), analytics, and application development. The solution includes additional productivity and data ops tooling for running jobs, implementing business workflows, and authoring.

AWS Glue allows users to connect to more than 70 diverse data sources and manage data in a centralized data catalog. The solution facilitates visual creation, running, and monitoring of extract, transform, and load (ETL) pipelines to load data into users' data lakes. This Amazon product seamlessly integrates with other native applications of the brand and allows users to search and query cataloged data using Amazon EMR, Amazon Athena, and Amazon Redshift Spectrum.

The solution also utilizes application programming interface (API) operations to transform users' data, create runtime logs, store job logic, and create notifications for monitoring job runs. The console of AWS Glue connects all of these services into a managed application, facilitating the monitoring and operational processes. The solution also performs provisioning and management of the resources required to run users' workloads in order to minimize manual work time for organizations.

AWS Glue Features

AWS Glue groups its features into four categories - discover, prepare, integrate, and transform. Within those groups are the following features:

Automatic schema discovery: AWS Glue crawlers connect to the organization's source or target data source through a prioritized list of classifiers to determine the schema for users' data. This feature creates metadata in companies' AWS Glue Data Catalog.
Schemas for data stream management: The AWS Glue Schema Registry enables users to validate and control the evolution of streaming data through registered Apache Avro schemas for no additional charge.
Automatic scaling based on workload: This feature dynamically scales resources up and down based on workload. The feature controls job resources, removing them depending on how much the workload can be split up.
FindMatches: This feature is for machine learning-based data deduplication and cleansing, and works by finding records that are imperfect matches of each other to remove useless data copies.
Edit, debug, and test ETL code: This feature helps users who have chosen to interactively develop their ETL code by providing development endpoints for editing, debugging, and testing the code it generates for them.
AWS Glue DataBrew: An interactive, point-and-click visual interface for specialists to clean and normalize data without the need to write any code.
AWS Glue Interactive Sessions: This feature simplifies the development of data integration jobs by enabling data engineers to interactively prepare and explore data.
AWS Glue Studio Job Notebooks: This AWS Glue feature provides serverless notebooks with minimal setup, allowing developers to start working in a timely manner.
Complex ETL pipeline building: This feature allows the product to be invoked on a schedule, on demand, or based on an event, allowing users to start multiple jobs in parallel or specify dependencies to build complex ETL pipelines.
AWS Glue Studio: This AWS Glue feature allows users to visually transform data through a drag-and-drop interface. The product automatically generates the code for ETL processes for users' data.

AWS Glue Benefits

AWS Glue offers a wide range of benefits for its users. These benefits include:

Users of other AWS products can easily onboard with AWS Glue, as it is integrated across a wide range of the company's services.
The solution is serverless, which allows for a lower total cost of ownership.
AWS Glue offers more power for users, as it automates much of the effort in building, maintaining, and running ETL jobs.
The product allows customers to easily discover and search across all their AWS datasets through AWS Glue Data Catalog.
AWS Glue does not require additional payment for managing and enforcing schemas for data streams.
The solution facilitates the authority of scalable ETL jobs for beginners and non-coding experts through a drag-and-drop interface.

Reviews from Real Users

Mustapha A., a cloud data engineer at Jems Groupe, likes AWS Glue because it is a product that is great for serverless data transformations.

Liana I., CEO at Quark Technologies SRL, describes AWS Glue as a highly scalable, reliable, and beneficial pay-as-you-go pricing model.

StreamSets is a data integration platform that enables organizations to efficiently move and process data across various systems. It offers a user-friendly interface for designing, deploying, and managing data pipelines, allowing users to easily connect to various data sources and destinations. StreamSets also provides real-time monitoring and alerting capabilities, ensuring that data is flowing smoothly and any issues are quickly addressed.

Sample Customers

bp, Cerner, Expedia, Finra, HESS, intuit, Kellog's, Philips, TIME, workday

Availity, BT Group, Humana, Deluxe, GSK, RingCentral, IBM, Shell, SamTrans, State of Ohio, TalentFulfilled, TechBridge

Buyer's Guide

AWS Glue vs. StreamSets

October 2024

Free Report: AWS Glue vs. StreamSets

Find out what your peers are saying about AWS Glue vs. StreamSets and other solutions. Updated: October 2024.

DOWNLOAD NOW

813,418 professionals have used our research since 2012.

See our AWS Glue vs. StreamSets report.

See our list of best Cloud Data Integration vendors.

We monitor all Cloud Data Integration reviews to prevent fraudulent reviews and keep review quality high. We do not post reviews by company employees or direct competitors. We validate each review for authenticity via cross-reference with LinkedIn, and personal follow-up with the reviewer when necessary.