IBM Cloud Pak for Integration vs StreamSets comparison

"It is a stable solution."

"The most preferable aspect would be the elimination of the command, which was a significant improvement. In the past, it was a challenge, but now we can proceed smoothly with the implementation of our policies and everything is managed through JCP. It's still among the positive aspects, and it's a valuable feature."

"Redirection is a key feature. It helps in managing multiple microservices by centralizing control and access."

"The most valuable aspect of the Cloud Pak, in general, is the flexibility that you have to use the product."

"Cloud Pak for Integration is definitely scalable. That is the most important criteria."

"I really appreciate the numerous ready connectors available on both the source and target sides, the support for various media file formats, and the ease of configuring and managing pipelines centrally."

"The most valuable feature is the pipelines because they enable us to pull in and push out data from different sources and to manipulate and clean things up within them."

"The ETL capabilities are very useful for us. We extract and transform data from multiple data sources, into a single, consistent data store, and then we put it in our systems. We typically use it to connect our Apache Kafka with data lakes. That process is smooth and saves us a lot of time in our production systems."

"What I love the most is that StreamSets is very light. It's a containerized application. It's easy to use with Docker. If you are a large organization, it's very easy to use Kubernetes."

"The Ease of configuration for pipes is amazing. It has a lot of connectors. Mainly, we can do everything with the data in the pipe. I really like the graphical interface too"

"The best feature that I really like is the integration."

"The most valuable features are the option of integration with a variety of protocols, languages, and origins."

"I have used Data Collector, Transformer, and Control Hub products from StreamSets. What I really like about these products is that they're very user-friendly. People who are not from a technological or core development background find it easy to get started and build data pipelines and connect to the databases. They would be comfortable like any technical person within a couple of weeks."

"The pricing can be improved."

"The initial setup is not easy."

"Enterprise bots are needed to balance products like Kafka and Confluent."

"Setting up Cloud Pak for Integration is relatively complex. It's not as easy because it has not yet been fully integrated. You still have some products that are still not containerized, so you still have to run them on a dedicated VM."

"Its queuing and messaging features need improvement."

"Currently, we can only use the query to read data from SAP HANA. What we would like to see, as soon as possible, is the ability to read from multiple tables from SAP HANA. That would be a really good thing that we could use immediately. For example, if you have 100 tables in SQL Server or Oracle, then you could just point it to the schema or the 100 tables and ingestion information. However, you can't do that in SAP HANA since StreamSets currently is lacking in this. They do not have a multi-table feature for SAP HANA. Therefore, a multi-table origin for SAP HANA would be helpful."

"The logging mechanism could be improved. If I am working on a pipeline, then create a job out of it and it is running, it will generate constant logs. So, the logging mechanism could be simplified. Now, it is a bit difficult to understand and filter the logs. It takes some time."

"We often faced problems, especially with SAP ERP. We struggled because many columns weren't integers or primary keys, which StreamSets couldn't handle. We had to restructure our data tables, which was painful. Also, pipeline failures were common, and data drifting wasn't addressed, which made things worse. Licensing was another issue we encountered."

"The monitoring visualization is not that user-friendly. It should include other features to visualize things, like how many records were streamed from a source to a destination on a particular date."

"One issue I observed with StreamSets is that the memory runs out quickly when processing large volumes of data. Because of this memory issue, we have to upgrade our EC2 boxes in the Amazon AWS infrastructure."

"The execution engine could be improved. When I was at their session, they were using some obscure platform to run. There is a controller, which controls what happens on that, but you should be able to easily do this at any of the cloud services, such as Google Cloud. You shouldn't have any issues in terms of how to run it with their online development platform or design platform, basically their execution engine. There are issues with that."

"Sometimes, it is not clear at first how to set up nodes. A site with an explanation of how each node works would be very helpful."

"In terms of the product, I don't think there is any room for improvement because it is very good. One small area of improvement that is very much needed is on the knowledge base side. Sometimes, it is not very clear how to set up a certain process or a certain node for a person who's using the platform for the first time."

"It is an expensive solution."

"The solution's pricing model is very flexible."

"StreamSets Data Collector is open source. One can utilize the StreamSets Data Collector, but the Control Hub is the main repository where all the jobs are present. Everything happens in Control Hub."

"We use the free version. It's great for a public, free release. Our stance is that the paid support model is too expensive to get into. They should honestly reevaluate that."

"The pricing is affordable for any business."

"There are two editions, Professional and Enterprise, and there is a free trial. We're using the Professional edition and it is competitively priced."

"The licensing is expensive, and there are other costs involved too. I know from using the software that you have to buy new features whenever there are new updates, which I don't really like. But initially, it was very good."

"It has a CPU core-based licensing, which works for us and is quite good."

"It's not so favorable for small companies."

"The overall cost is very flexible so it is not a burden for our organization... However, the cost should be improved. For small and mid-size organizations it might be a challenge."