We performed a comparison between Azure Data Factory and Spring Cloud Data Flow based on real PeerSpot user reviews.Find out what your peers are saying about Microsoft, Informatica, Oracle and others in Data Integration Tools.
"StreamSets’ data drift resilience has reduced the time it takes us to fix data drift breakages. For example, in our previous Hadoop scenario, when we were creating the Sqoop-based processes to move data from source to destinations, we were getting the job done. That took approximately an hour to an hour and a half when we did it with Hadoop. However, with the StreamSets, since it works on a data collector-based mechanism, it completes the same process in 15 minutes of time. Therefore, it has saved us around 45 minutes per data pipeline or table that we migrate. Thus, it reduced the data transfer, including the drift part, by 45 minutes."
"In StreamSets, everything is in one place."
"It is a very powerful, modern data analytics solution, in which you can integrate a large volume of data from different sources. It integrates all of the data and you can design, create, and monitor pipelines according to your requirements. It is an all-in-one day data ops solution."
"I have used Data Collector, Transformer, and Control Hub products from StreamSets. What I really like about these products is that they're very user-friendly. People who are not from a technological or core development background find it easy to get started and build data pipelines and connect to the databases. They would be comfortable like any technical person within a couple of weeks."
"StreamSets data drift feature gives us an alert upfront so we know that the data can be ingested. Whatever the schema or data type changes, it lands automatically into the data lake without any intervention from us, but then that information is crucial to fix for downstream pipelines, which process the data into models, like Tableau and Power BI models. This is actually very useful for us. We are already seeing benefits. Our pipelines used to break when there were data drift changes, then we needed to spend about a week fixing it. Right now, we are saving one to two weeks. Though, it depends on the complexity of the pipeline, we are definitely seeing a lot of time being saved."
"The solution includes a feature that increases the number of processors used which makes it very powerful and adds to the scalability."
"I think it makes it very easy to understand what data flow is and so on. You can leverage the user interface to do the different data flows, and it's great. I like it a lot."
"It is very modular. It works well. We've used Data Factory and then made calls to libraries outside of Data Factory to do things that it wasn't optimized to do, and it worked really well. It is obviously proprietary in regards to Microsoft created it, but it is pretty easy and direct to bring in outside capabilities into Data Factory."
"It's cloud-based, allowing multiple users to easily access the solution from the office or remote locations. I like that we can set up the security protocols for IP addresses, like allow lists. It's a pretty user-friendly product as well. The interface and build environment where you create pipelines are easy to use. It's straightforward to manage the digital transformation pipelines we build."
"Microsoft supported us when we planned to provision Azure Data Factory over a private link. As a result, we received excellent support from Microsoft."
"Data Factory's most valuable feature is Copy Activity."
"An excellent tool for pipeline orchestration."
"It is beneficial that the solution is written with Spark as the back end."
"The most valuable features of Spring Cloud Data Flow are the simple programming model, integration, dependency Injection, and ability to do any injection. Additionally, auto-configuration is another important feature because we don't have to configure the database and or set up the boilerplate in the database in every project. The composability is good, we can create small workloads and compose them in any way we like."
"Sometimes, when we have large amounts of data that is very efficiently stored in Hadoop or Kafka, it is not very efficient to run it through StreamSets, due to the lack of efficiency or the resources that StreamSets is using."
"If you use JDBC Lookup, for example, it generally takes a long time to process data."
"We create pipelines or jobs in StreamSets Control Hub. It is a great feature, but if there is a way to have a folder structure or organize the pipelines and jobs in Control Hub, it would be great. I submitted a ticket for this some time back."
"Currently, we can only use the query to read data from SAP HANA. What we would like to see, as soon as possible, is the ability to read from multiple tables from SAP HANA. That would be a really good thing that we could use immediately. For example, if you have 100 tables in SQL Server or Oracle, then you could just point it to the schema or the 100 tables and ingestion information. However, you can't do that in SAP HANA since StreamSets currently is lacking in this. They do not have a multi-table feature for SAP HANA. Therefore, a multi-table origin for SAP HANA would be helpful."
"The logging mechanism could be improved. If I am working on a pipeline, then create a job out of it and it is running, it will generate constant logs. So, the logging mechanism could be simplified. Now, it is a bit difficult to understand and filter the logs. It takes some time."
"Some of the optimization techniques are not scalable."
"For some of the data, there were some issues with data mapping. Some of the error messages were a little bit foggy. There could be more of a quick start guide or some inline examples. The documentation could be better."
"It does not appear to be as rich as other ETL tools. It has very limited capabilities."
"The solution can be improved by decreasing the warmup time which currently can take up to five minutes."
"A room for improvement in Azure Data Factory is its speed. Parallelization also needs improvement."
"The need to work more on developing out-of-the-box connectors for other products like Oracle, AWS, and others."
"The deployment should be easier."
"The performance could be better. It would be better if Azure Data Factory could handle a higher load. I have heard that it can get overloaded, and it can't handle it."
"Spring Cloud Data Flow could improve the user interface. We can drag and drop in the application for the configuration and settings, and deploy it right from the UI, without having to run a CI/CD pipeline. However, that does not work with Kubernetes, it only works when we are working with jars as the Spring Cloud Data Flow applications."
StreamSets offers an end-to-end data integration platform to build, run, monitor and manage smart data pipelines that deliver continuous data for DataOps, and power the modern data ecosystem and hybrid integration.
Only StreamSets provides a single design experience for all design patterns for 10x greater developer productivity; smart data pipelines that are resilient to change for 80% less breakages; and a single pane of glass for managing and monitoring all pipelines across hybrid and cloud architectures to eliminate blind spots and control gaps.
With StreamSets, you can deliver the continuous data that drives the connected enterprise.
Azure Data Factory is a managed cloud service built for extract-transform-load (ETL), extract-load-transform (ELT), and data integration projects. This is a digital integration tool as well as a cloud data warehouse that allows users to create, schedule, and manage data in the cloud or on premises. The use cases of the product include data engineering, operational data integration, analytics, ingesting data into data warehouses, and migrating on-premise SQL Server Integration Services (SSIS) packages to Azure.
The tool allows users to create data-driven workflows for initiating data movement and data transformation at scale. Data can be ingested from disparate data stores via pipelines. Companies can utilize this product to build complex ETL processes for transforming data visually with data flows. Azure Data Factory also offers services such as Azure HDInsight Hadoop, Azure Databricks, Azure Synapse Analytics, and Azure SQL Database. These services are created to facilitate data management and control for organizations, providing them with better visibility of their data for improved decision-making.
Azure Data Factory allows companies to create schedules for moving and transforming data into their pipelines. This can be done hourly, daily, weekly, or according to the specific needs of the organization. The steps through which the data-driven workflows work in Azure Data Factory are the following:
1. Connecting to required sources and collecting data. After connecting to the various sources where data is stored, the pipelines move the data to a centralized location for further processing.
2. Transforming and enriching the data. Once the data is moved to a centralized data store in the cloud, the pipelines transform it through services like HDInsight Hadoop, Azure Data Lake Analytics, Spark, and Machine Learning.
3. Delivering the transformed data to on-premise sources or keeping it in cloud storage sources for usage by different tools and applications.
Azure Data Factory Concepts
The solution consists of a series of interconnected systems that provide data integration and related services for users. The following concepts create the end product for users:
Azure Data Factory Benefits
Azure Data Factory offers clients many several benefits. Some of these include:
Reviews from Real Users
According to Dan M., a Chief Strategist & CTO at a consultancy, Azure Data Factory is secure and reasonably priced.
A Senior Manager at a tech services company evaluates the tool as reasonably priced, scales well, good performance.
Spring Cloud Data Flow is a toolkit for building data integration and real-time data processing pipelines.
Pipelines consist of Spring Boot apps, built using the Spring Cloud Stream or Spring Cloud Task microservice frameworks. This makes Spring Cloud Data Flow suitable for a range of data processing use cases, from import/export to event streaming and predictive analytics. Use Spring Cloud Data Flow to connect your Enterprise to the Internet of Anything—mobile devices, sensors, wearables, automobiles, and more.
Azure Data Factory is ranked 1st in Data Integration Tools with 40 reviews while Spring Cloud Data Flow is ranked 20th in Data Integration Tools with 1 review. Azure Data Factory is rated 7.8, while Spring Cloud Data Flow is rated 7.0. The top reviewer of Azure Data Factory writes "There's the good, the bad and the ugly....unfortunately lots of ugly". On the other hand, the top reviewer of Spring Cloud Data Flow writes "Simple programming model, low maintenance, but interface could improve". Azure Data Factory is most compared with Informatica PowerCenter, Microsoft Azure Synapse Analytics, Informatica Cloud Data Integration, Alteryx Designer and Talend Open Studio, whereas Spring Cloud Data Flow is most compared with Apache Flink, Apache Spark Streaming, Amazon Kinesis, Databricks and Google Cloud Dataflow.
See our list of best Data Integration Tools vendors.
We monitor all Data Integration Tools reviews to prevent fraudulent reviews and keep review quality high. We do not post reviews by company employees or direct competitors. We validate each review for authenticity via cross-reference with LinkedIn, and personal follow-up with the reviewer when necessary.