IBM InfoSphere DataStage vs StreamSets comparison

Read 24 StreamSets reviews

3,911 Views
2,178 Comparison Views

100% willing to recommend

IBM InfoSphere DataStage

Comparison Buyer's Guide

Download the report

Executive Summary

We performed a comparison between IBM InfoSphere DataStage and StreamSets based on real PeerSpot user reviews.

Find out in this report how the two Data Integration solutions compare in terms of features, pricing, service and support, easy of deployment, and ROI.

To learn more, read our detailed IBM InfoSphere DataStage vs. StreamSets Report (Updated: July 2024).

IBM InfoSphere DataStage vs. StreamSets

July 2024

Download the complete report

Helped 793,295 peers since 2012

Categories and Ranking

IBM InfoSphere DataStage

Ranking in Data Integration

7th

Average Rating

7.8

Number of Reviews

Ranking in other categories

No ranking in other categories

StreamSets

Ranking in Data Integration

8th

Average Rating

8.4

Number of Reviews

Ranking in other categories

No ranking in other categories

Mindshare comparison

As of July 2024, in the Data Integration category, the mindshare of IBM InfoSphere DataStage is 6.5%, up from 5.6% compared to the previous year. The mindshare of StreamSets is 1.9%, up from 1.4% compared to the previous year. It is calculated based on PeerSpot user engagement data.

Data Integration

Unique Categories:

No other categories found

Featured Reviews

Yusuf Arslan

Senior Data Architect at Anadolu Sigorta

Apr 15, 2024

Provides governance, data management with improved connectors and Kafka connectivity

When deploying using IBM InfoSphere DataStage, the initial steps involve defining the source and target connections. The first step is establishing these connections, ensuring data can flow from the source to the target. Subsequent steps, such as implementing changes or updates, can be approached in two ways. If a change data capture mechanism like IBM InfoSphere or similar tools like Oracle GoldenGate is available, DataStage can leverage these tools to propagate changes. Alternatively, if such mechanisms are not in place, DataStage can handle updates using its ETL capabilities, which may require more effort. DataStage performs incremental updates when dealing with large and smaller reference tables.

Read full review

Josua Mwesigwa

Software Engineer at ZIDIYO

Mar 30, 2023

Enables us to create streams and pipelines that our analytics team can utilize to identify areas for improvement

We use StreamSets' ability to connect to enterprise data stores such as Kafka. It is easy and simple to connect enterprise data stores as long as we follow the documentation. We use StreamSets' ability to move data into the analytic platforms easily because we can use the template provided to extract data from the pipeline. Being able to use Transformer for Snowflake to design both simple and complex transformation logic is important because it helps us break out a live amount of data interfaces that can be understood by the analytics team and identify areas of improvement. As the Transformer for Snowflake operates as a serverless engine, we can reduce our costs as we no longer need to purchase servers. StreamSets enables us to create streams and pipelines that our analytics team can utilize to identify areas for improvement. Additionally, our marketing team can leverage the data generated from these reports to understand how we can integrate our products and services to benefit our brand. StreamSets' data drift resilience is effective and user-friendly. We can use templates or use them from scratch. Data drift resilience saves us around 35 percent of the time fixing duplicates. StreamSets has helped us break down data silos within our organization by providing a clear path forward and enhancing our productivity by breaking down a large amount of data that we can understand. StreamSets saved us around 40 percent of our time. We can use a small team using StreamSets to create data pipelines that would normally require an expert that costs around $500 per month. StreamSets helps us scale our operations because we understand the quality of the data we have and how we can integrate the data into our marketing needs.

Read full review

Quotes from Members

We asked business professionals to review the solutions they use. Here are some excerpts of what they said:

Pros

"We are mostly using transmission rules. It has a lot of functions and logic related to transmission. It is a user-friendly tool with in-built functions."

"The data lineage report can be filtered for reporting. The reports are user-friendly and take less time to find what you need."

"Highly customizable: Allowing you to handle multiple data latencies (scheduled batch, on-demand, and real-time) in the same job."

"In IBM DataStage, the Transformer is the most valuable feature for me. It enables me to apply complex transformations, generate the gateway key, and map source tables into the session table."

"The product is a stable and powerful data management solution that can run in parallel mode for enhanced speed."

"The product is easy to deploy."

"The concept of integration is a valuable feature of the product."

"The most valuable feature is the data integration for data warehousing."

More IBM InfoSphere DataStage pros

"The best feature that I really like is the integration."

"Also, the intuitive canvas for designing all the streams in the pipeline, along with the simplicity of the entire product are very big pluses for me. The software is very simple and straightforward. That is something that is needed right now."

"The Ease of configuration for pipes is amazing. It has a lot of connectors. Mainly, we can do everything with the data in the pipe. I really like the graphical interface too"

"The ability to have a good bifurcation rate and fewer mistakes is valuable."

"What I love the most is that StreamSets is very light. It's a containerized application. It's easy to use with Docker. If you are a large organization, it's very easy to use Kubernetes."

"StreamSets data drift feature gives us an alert upfront so we know that the data can be ingested. Whatever the schema or data type changes, it lands automatically into the data lake without any intervention from us, but then that information is crucial to fix for downstream pipelines, which process the data into models, like Tableau and Power BI models. This is actually very useful for us. We are already seeing benefits. Our pipelines used to break when there were data drift changes, then we needed to spend about a week fixing it. Right now, we are saving one to two weeks. Though, it depends on the complexity of the pipeline, we are definitely seeing a lot of time being saved."

"StreamSets Transformer is a good feature because it helps you when you are developing applications and when you don't want to write a lot of code. That is the best feature overall."

"The ETL capabilities are very useful for us. We extract and transform data from multiple data sources, into a single, consistent data store, and then we put it in our systems. We typically use it to connect our Apache Kafka with data lakes. That process is smooth and saves us a lot of time in our production systems."

More StreamSets pros

Cons

"Working with some of the big data components is good, but I can see improvements are needed."

"It doesn't have any big data connections. It would be good to have them because most of the systems are moving towards big data. There should also be a user-friendly way to interact with the cloud. Its loading process is very slow. It takes a lot of time for around 5 or 6 million records, and we are not able to provide real-time data to the vendors due to this delay. Its performance needs to be improved. It is also like a legacy system. It is not updated much. In higher versions, they only do small changes. We would like to have new features and new technologies."

"The deployment could be more straightforward."

"It would be useful to provide support for Python, AR, and Java."

"The error messaging needs to be improved."

"The solution should be more user-friendly."

"The setup is extremely difficult."

"We would be happy to see in next versions the ability to return several parameters from jobs. Now, jobs can return just one parameter. If they could return several parameters, that would be great."

More IBM InfoSphere DataStage cons

"The design experience is the bane of our existence because their documentation is not the best. Even when they update their software, they don't publish the best information on how to update and change your pipeline configuration to make it conform to current best practices. We don't pay for the added support. We use the "freeware version." The user community, as well as the documentation they provide for the standard user, are difficult, at best."

"I would like to see it integrate with other kinds of platforms, other than Java. We're going to have a lot of applications using .NET and other languages or frameworks. StreamSets is very helpful for the old Java platform but it's hard to integrate with the other platforms and frameworks."

"The documentation is inadequate and has room for improvement because the technical support does not regularly update their documentation or the knowledge base."

"We often faced problems, especially with SAP ERP. We struggled because many columns weren't integers or primary keys, which StreamSets couldn't handle. We had to restructure our data tables, which was painful. Also, pipeline failures were common, and data drifting wasn't addressed, which made things worse. Licensing was another issue we encountered."

"StreamSet works great for batch processing but we are looking for something that is more real-time. We need latency in numbers below milliseconds."

"Currently, we can only use the query to read data from SAP HANA. What we would like to see, as soon as possible, is the ability to read from multiple tables from SAP HANA. That would be a really good thing that we could use immediately. For example, if you have 100 tables in SQL Server or Oracle, then you could just point it to the schema or the 100 tables and ingestion information. However, you can't do that in SAP HANA since StreamSets currently is lacking in this. They do not have a multi-table feature for SAP HANA. Therefore, a multi-table origin for SAP HANA would be helpful."

"The monitoring visualization is not that user-friendly. It should include other features to visualize things, like how many records were streamed from a source to a destination on a particular date."

"The execution engine could be improved. When I was at their session, they were using some obscure platform to run. There is a controller, which controls what happens on that, but you should be able to easily do this at any of the cloud services, such as Google Cloud. You shouldn't have any issues in terms of how to run it with their online development platform or design platform, basically their execution engine. There are issues with that."

More StreamSets cons

Pricing and Cost Advice

"The solution is cheap."

"Our internal team takes care of group licensing and cost. We don't have individual licenses. We have group licensing at the company level. Usually, IBM doesn't charge anything separately on the licensing side. For storage and everything else, we are paying around $6,000 per month, which is not very high. It includes Linux data storage, execution, and licensing. They're charging $40 for one-hour execution. Based on that, we are spending around $2,000 on the production environment and $1,000 on the lower environment for testing and development-side executions. For the mainframe, we are using the Db2 mainframe database, and we are spending around $1,000 on the Db2 mainframe database as well. All this comes out to be around $6,000. We, however, would like to have some cost reduction."

"The pricing depends on the setup. However, we paid $100,000 as a one-time cost for an on-premises setup."

"I have no information on the exact pricing for IBM InfoSphere DataStage because the solution is usually procured by the clients my company works with, though the pricing is higher compared to other solutions, so many clients choose to go with a different solution rather than IBM InfoSphere DataStage."

"It is quite expensive."

"The cost is too high."

"The product is expensive."

"It's very expensive."

More IBM InfoSphere DataStage pricing and cost advice

"The pricing is affordable for any business."

"I believe the pricing is not equitable."

"StreamSets is an expensive solution."

"The licensing is expensive, and there are other costs involved too. I know from using the software that you have to buy new features whenever there are new updates, which I don't really like. But initially, it was very good."

"There are two editions, Professional and Enterprise, and there is a free trial. We're using the Professional edition and it is competitively priced."

"The overall cost is very flexible so it is not a burden for our organization... However, the cost should be improved. For small and mid-size organizations it might be a challenge."

"Its pricing is pretty much up to the mark. For smaller enterprises, it could be a big price to pay at the initial stage of operations, but the moment you have the Seed B or Seed C funding and you want to scale up your operations and aren't much worried about the funds, at that point in time, you would need a solution that could be scaled."

"It has a CPU core-based licensing, which works for us and is quite good."

More StreamSets pricing and cost advice

See which vendors are best for you

Use our free recommendation engine to learn which Data Integration solutions are best for your needs.

See recommendations

793,295 professionals have used our research since 2012.

Top Industries

By visitors reading reviews

Financial Services Firm

26%

Computer Software Company

11%

Manufacturing Company

11%

Insurance Company

Financial Services Firm

17%

Computer Software Company

13%

Manufacturing Company

Government

Company Size

By reviewers

Large Enterprise

Midsize Enterprise

Small Business

Questions from the Community

Would you upgrade to more premium versions of IBM InfoSphere DataStage?

My company currently uses the free version of the product, and we are definitely switching to a paid one. We needed a tool that can help us not only integrate our data but use it effectively. For ...

Is IBM InfoSphere DataStage more difficult to use compared to other tools in the field?

I think the tool may cause some difficulties if you have not used other data integration solutions before. I have worked at companies that used different tools for data integration, and they work ...

Do you rely on IBM Cloud Paks for your data? Have you utilized this product, or do you use IBM InfoSphere DataStage without it?

IBM Cloud Paks makes a big difference in your data integration. My company has been using it alongside IBM InfoSphere DataStage and while the main product is good on its own, this one truly expands...

What do you like most about StreamSets?

The best thing about StreamSets is its plugins, which are very useful and work well with almost every data source. It's also easy to use, especially if you're comfortable with SQL. You can customiz...

What needs improvement with StreamSets?

We often faced problems, especially with SAP ERP. We struggled because many columns weren't integers or primary keys, which StreamSets couldn't handle. We had to restructure our data tables, which ...

What is your primary use case for StreamSets?

StreamSets is used for data transformation rather than ETL processes. It focuses on transforming data directly from sources without handling the extraction part of the process. The transformed data...

SSIS vs IBM InfoSphere DataStage

Comparisons

Compared 12% of the time

IBM Cloud Pak for Data vs IBM InfoSphere DataStage

Compared 11% of the time

Azure Data Factory vs IBM InfoSphere DataStage

Compared 11% of the time

Talend Open Studio vs IBM InfoSphere DataStage

Compared 10% of the time

Alteryx Designer vs IBM InfoSphere DataStage

Compared 2% of the time

More IBM InfoSphere DataStage Competitors

Fivetran vs StreamSets

Compared 12% of the time

Informatica PowerCenter vs StreamSets

Compared 12% of the time

Azure Data Factory vs StreamSets

Compared 10% of the time

SSIS vs StreamSets

Compared 7% of the time

webMethods.io Integration vs StreamSets

Compared 6% of the time

More StreamSets Competitors

Product Reports

Download IBM InfoSphere DataStage product report

IBM InfoSphere DataStage

July 2024

Download StreamSets product report

June 2024

Learn More

IBM

Video not available

Overview

IBM InfoSphere DataStage is a high-quality data integration tool that aims to design, develop, and run jobs that move and transform data for organizations of different sizes. The product works by integrating data across multiple systems through a high-performance parallel framework. It supports extended metadata management, enterprise connectivity, and integration of all types of data.

The solution is the data integration component of IBM InfoSphere Information Server, providing a graphical framework for moving data from source systems to target systems. IBM InfoSphere DataStage can deliver data to data warehouses, data marts, operational data sources, and other enterprise applications. The tool works with various types of patterns - extract, transform and load (ETL), and extract, load, and transform (ELT). The scalability of the platform is achieved by using parallel processing and enterprise connectivity.

The solution has various versions, catering to different types of companies, which include the Server Edition, the Enterprise Edition, and the MVS Edition. Depending on which version a company has bought, different goals can be achieved. They include the following:

Designing data flows to extract information from multiple sources, transform the data, and deliver it to target databases or applications.
Delivery of relevant and accurate data through direct connections to enterprise applications.
Reduction of development time and improvement of consistency through prebuilt functions.
Utilization of InfoSphere Information Server tools for accelerating the project delivery cycle.

IBM InfoSphere DataStage can be deployed in various ways, including:

As a service: The tool can be accessed from a subscription model, where its capabilities are a part of IBM DataStage on IBM Cloud Park for Data as a Service. This option offers full management on IBM Cloud.
On premises or in any cloud: The two editions - IBM DataStage Enterprise and IBM DataStage Enterprise Plus - can run workloads on premises or in any cloud when added to IBM DataStage on IBM Cloud Pak for Data as a Service.
On premises: The basic jobs of the tool can be run on premises using IBM DataStage.

IBM InfoSphere DataStage Features

The tool has various features through which users can integrate and utilize their data effectively. The components of IBM InfoSphere DataStage include:

AI services: The tool offers services such as data science, event messaging, data warehousing, and data virtualization. It accelerates processes through artificial intelligence (AI) and offers a connection with IBM Cloud Paks - the cloud-native insight platform of the solution.
Parallel engine: Through this feature, ETL performance can be optimized to process data at scale. This is achieved through parallel engine and load balancing, which maximizes throughput.
Metadata support: This feature of the product uses the IBM Watson Knowledge Catalog to protect companies' sensitive data and monitor who can access it and at what levels.
Automated delivery pipelines: IBM InfoSphere DataStage reduces costs by automating continuous integration and delivery of pipelines.
Prebuilt connectors: The feature for prebuilt connectivity and stages allows users to move data between multiple cloud sources and data warehouses, including IBM native products.
IBM DataStage Flow Designer: This feature offers assistance through machine learning design. The product offers its clients a user-friendly interface which facilitates the work process.
IBM InfoSphere QualityStage: The tool provides a feature that automatically resolves data quality issues and increases the reliability of the delivered data.
Automated failure detection: Through this feature, companies can reduce infrastructure management efforts, relying on the automated detection that the tool offers.
Distributed data processing: Cloud runtimes can be executed remotely through this feature while maintaining its sovereignty and decreasing costs.

IBM InfoSphere DataStage Benefits

This solution offers many benefits for the companies that utilize it for data integration. Some of these benefits include:

Increased speed of workload execution due to better balancing and a parallel engine.
Reduction of data movement costs through integrations and seamless design of jobs.
Modernization of data integration by extending the capabilities of companies' data.
Delivery of reliable data through IBM Cloud Pak for Data.
Utilization of a drag-and-drop interface which assists in the delivery of data without the need for code.
Effective data manipulation allows data to be merged before being mapped and transformed.
Creating easier access of users to their data by providing visual maps of the process and the delivered data.

Reviews from Real Users

A data/solution architect at a computer software company says the product is robust, easy to use, has a simple error logging mechanism, and works very well for huge volumes of data.

Tirthankar Roy Chowdhury, team leader at Tata Consultancy Services, feels the tool is user-friendly with a lot of functionalities, and doesn't require much coding because of its drag-and-drop features.

StreamSets is a data integration platform that enables organizations to efficiently move and process data across various systems. It offers a user-friendly interface for designing, deploying, and managing data pipelines, allowing users to easily connect to various data sources and destinations. StreamSets also provides real-time monitoring and alerting capabilities, ensuring that data is flowing smoothly and any issues are quickly addressed.

Sample Customers

Dubai Statistics Center, Etisalat Egypt

Availity, BT Group, Humana, Deluxe, GSK, RingCentral, IBM, Shell, SamTrans, State of Ohio, TalentFulfilled, TechBridge