IBM Cloud Pak for Integration vs StreamSets comparison

"The most valuable aspect of the Cloud Pak, in general, is the flexibility that you have to use the product."

"The most preferable aspect would be the elimination of the command, which was a significant improvement. In the past, it was a challenge, but now we can proceed smoothly with the implementation of our policies and everything is managed through JCP. It's still among the positive aspects, and it's a valuable feature."

"It is a stable solution."

"Cloud Pak for Integration is definitely scalable. That is the most important criteria."

"Redirection is a key feature. It helps in managing multiple microservices by centralizing control and access."

"It is really easy to set up and the interface is easy to use."

"StreamSets’ data drift resilience has reduced the time it takes us to fix data drift breakages. For example, in our previous Hadoop scenario, when we were creating the Sqoop-based processes to move data from source to destinations, we were getting the job done. That took approximately an hour to an hour and a half when we did it with Hadoop. However, with the StreamSets, since it works on a data collector-based mechanism, it completes the same process in 15 minutes of time. Therefore, it has saved us around 45 minutes per data pipeline or table that we migrate. Thus, it reduced the data transfer, including the drift part, by 45 minutes."

"The scheduling within the data engineering pipeline is very much appreciated, and it has a wide range of connectors for connecting to any data sources like SQL Server, AWS, Azure, etc. We have used it with Kafka, Hadoop, and Azure Data Factory Datasets. Connecting to these systems with StreamSets is very easy."

"The entire user interface is very simple and the simplicity of creating pipelines is something that I like very much about it. The design experience is very smooth."

"StreamSets is the leader in the market."

"The ETL capabilities are very useful for us. We extract and transform data from multiple data sources, into a single, consistent data store, and then we put it in our systems. We typically use it to connect our Apache Kafka with data lakes. That process is smooth and saves us a lot of time in our production systems."

"StreamSets Transformer is a good feature because it helps you when you are developing applications and when you don't want to write a lot of code. That is the best feature overall."

"The best feature that I really like is the integration."

"The initial setup is not easy."

"Its queuing and messaging features need improvement."

"Enterprise bots are needed to balance products like Kafka and Confluent."

"The pricing can be improved."

"Setting up Cloud Pak for Integration is relatively complex. It's not as easy because it has not yet been fully integrated. You still have some products that are still not containerized, so you still have to run them on a dedicated VM."

"Currently, we can only use the query to read data from SAP HANA. What we would like to see, as soon as possible, is the ability to read from multiple tables from SAP HANA. That would be a really good thing that we could use immediately. For example, if you have 100 tables in SQL Server or Oracle, then you could just point it to the schema or the 100 tables and ingestion information. However, you can't do that in SAP HANA since StreamSets currently is lacking in this. They do not have a multi-table feature for SAP HANA. Therefore, a multi-table origin for SAP HANA would be helpful."

"Visualization and monitoring need to be improved and refined."

"The monitoring visualization is not that user-friendly. It should include other features to visualize things, like how many records were streamed from a source to a destination on a particular date."

"The documentation is inadequate and has room for improvement because the technical support does not regularly update their documentation or the knowledge base."

"One area for improvement could be the cloud storage server speed, as we have faced some latency issues here and there."

"The software is very good overall. Areas for improvement are the error logging and the version history. I would like to see better, more detailed error logging information."

"We often faced problems, especially with SAP ERP. We struggled because many columns weren't integers or primary keys, which StreamSets couldn't handle. We had to restructure our data tables, which was painful. Also, pipeline failures were common, and data drifting wasn't addressed, which made things worse. Licensing was another issue we encountered."

"Sometimes, it is not clear at first how to set up nodes. A site with an explanation of how each node works would be very helpful."

"The solution's pricing model is very flexible."

"It is an expensive solution."

"The overall cost is very flexible so it is not a burden for our organization... However, the cost should be improved. For small and mid-size organizations it might be a challenge."

"StreamSets is an expensive solution."

"It's not so favorable for small companies."

"We are running the community version right now, which can be used free of charge."

"The pricing is affordable for any business."

"It has a CPU core-based licensing, which works for us and is quite good."

"The pricing is too fixed. It should be based on how much data you need to process. Some businesses are not so big that they process a lot of data."

"Its pricing is pretty much up to the mark. For smaller enterprises, it could be a big price to pay at the initial stage of operations, but the moment you have the Seed B or Seed C funding and you want to scale up your operations and aren't much worried about the funds, at that point in time, you would need a solution that could be scaled."