Solution Architect | R&D Lead at a computer software company with 51-200 employees
Real User
Top 20
Nov 6, 2024
We had a project for content management, which involved multiple applications each handling content ingestion, transformation, enrichment, and storage for different customers independently. We wanted to unify this process and reduce operational costs by centralizing a platform to integrate, enrich, and distribute data content. We used it to create a scalable, resilient, and efficient processing pipeline, exploiting the architecture of source, process, and sync, as well as the non-blocking data streams feature.
I am a developer using Spring Cloud Dataflow. We primarily use it to convert our applications from monolithic to microservices. The solution is used for scheduling tasks in a specific order and ensuring that microservices are executed in the correct sequence. We leverage it to manage tasks effectively in our microservice architecture.
I work with a leading company in the recruitment domain in India. When a job seeker applies for a job, we capture the intent. It has to flow by a certain pipeline to check all the scores and whether the candidate profile matches the job profile. We use our data science algorithm in between, where we take help from a lot of APIs to compute that score. Then, we dump in different sources and databases to use that data in our analytics. We have built a pipeline that processes every job seeker application on our website. Finally, we pass it to the recruiter via this pipeline with all the scores and everything in place.
I did not use the solution in my company, but we did explore it. The tool is used to build the data pipeline. My company uses the tool for one of our in-house projects.
Senior Software Engineer at QBE Regional Insurance
Real User
Top 10
Mar 26, 2024
I used the solution for a payment platform we integrated with our organization. Our company had to use it since we had to integrate it with different payment platforms.
Spring Cloud Data Flow is used for asynchronous workloads. We are working on streams. For example, a workload is generated at a particular point, and at the source, it gets passed down through a series of processors down to a sync and within that sync, it is persisted in a database.
Mostly the use cases are related to building a data pipeline. There are multiple microservices that are working in the Spring Cloud Data Flow infrastructure, and we are building a data pipeline, mostly a step-by-step process processing data using Kafka. Most of the processor sync and sources are being developed based on the customers' business requirements or use cases. In the example of the bank we work with, we are actually building a document analysis pipeline. There are some defined sources where we get the documents. Later on, we extract some information united from the summary and we export the data to multiple destinations. We may export it to the POGI Database, and/or to Kafka Topic. For CoreLogic, we were actually doing data import to elastic. We had a BigQuery data source. And from there we did some transformation of the data then imported it in the elastic clusters. That was the ETL solution.
Senior Platform Associate L2 at a tech services company with 10,001+ employees
Real User
Oct 19, 2020
In my last project, I worked on Spring Cloud Data Flow (SCDF). We created a stream using this product and we had a Spring Kafka Binder as well. The project included creating a data lake for our clients. The platform that we created maintained a data lake for an internet banking user and provided an out-of-the-box solution for integration with it. We used SCDF to gather the data, as well as our ETL (extract, transform, and load) pipelines.
The organization I’m currently consulting for is performing a lift-and-shift, moving its existing software from an on-prem platform and infrastructure to the cloud. They have chosen Azure as their cloud provider. As part of this process, they have orders to move away from expensive, monolithic, proprietary software platforms, and to replace them with open-source, publicly available software technologies. One area we’ll be replacing is their current middleware software which consists of IBM WebSphere Message Broker. While it is a fine tool for the most part, it’s also bulky and expensive to operate. The final solution we’re working towards will be much more cloud-native, will support scalability, be able to process messages much faster, and consist of several different technologies and vendors (not just a single vendor, as is the case with the current IBM solution). This new middleware platform will consist of: Apache Kafka for the delivery of messages, Spring Cloud Data Flow, and a handful of RESTful APIs.
Spring Cloud Data Flow is a toolkit for building data integration and real-time data processing pipelines.Pipelines consist of Spring Boot apps, built using the Spring Cloud Stream or Spring Cloud Task microservice frameworks. This makes Spring Cloud Data Flow suitable for a range of data processing use cases, from import/export to event streaming and predictive analytics. Use Spring Cloud Data Flow to connect your Enterprise to the Internet of Anything—mobile devices, sensors, wearables,...
We had a project for content management, which involved multiple applications each handling content ingestion, transformation, enrichment, and storage for different customers independently. We wanted to unify this process and reduce operational costs by centralizing a platform to integrate, enrich, and distribute data content. We used it to create a scalable, resilient, and efficient processing pipeline, exploiting the architecture of source, process, and sync, as well as the non-blocking data streams feature.
I am a developer using Spring Cloud Dataflow. We primarily use it to convert our applications from monolithic to microservices. The solution is used for scheduling tasks in a specific order and ensuring that microservices are executed in the correct sequence. We leverage it to manage tasks effectively in our microservice architecture.
I work with a leading company in the recruitment domain in India. When a job seeker applies for a job, we capture the intent. It has to flow by a certain pipeline to check all the scores and whether the candidate profile matches the job profile. We use our data science algorithm in between, where we take help from a lot of APIs to compute that score. Then, we dump in different sources and databases to use that data in our analytics. We have built a pipeline that processes every job seeker application on our website. Finally, we pass it to the recruiter via this pipeline with all the scores and everything in place.
I did not use the solution in my company, but we did explore it. The tool is used to build the data pipeline. My company uses the tool for one of our in-house projects.
I used the solution for a payment platform we integrated with our organization. Our company had to use it since we had to integrate it with different payment platforms.
Spring Cloud Data Flow is used for asynchronous workloads. We are working on streams. For example, a workload is generated at a particular point, and at the source, it gets passed down through a series of processors down to a sync and within that sync, it is persisted in a database.
Mostly the use cases are related to building a data pipeline. There are multiple microservices that are working in the Spring Cloud Data Flow infrastructure, and we are building a data pipeline, mostly a step-by-step process processing data using Kafka. Most of the processor sync and sources are being developed based on the customers' business requirements or use cases. In the example of the bank we work with, we are actually building a document analysis pipeline. There are some defined sources where we get the documents. Later on, we extract some information united from the summary and we export the data to multiple destinations. We may export it to the POGI Database, and/or to Kafka Topic. For CoreLogic, we were actually doing data import to elastic. We had a BigQuery data source. And from there we did some transformation of the data then imported it in the elastic clusters. That was the ETL solution.
In my last project, I worked on Spring Cloud Data Flow (SCDF). We created a stream using this product and we had a Spring Kafka Binder as well. The project included creating a data lake for our clients. The platform that we created maintained a data lake for an internet banking user and provided an out-of-the-box solution for integration with it. We used SCDF to gather the data, as well as our ETL (extract, transform, and load) pipelines.
The organization I’m currently consulting for is performing a lift-and-shift, moving its existing software from an on-prem platform and infrastructure to the cloud. They have chosen Azure as their cloud provider. As part of this process, they have orders to move away from expensive, monolithic, proprietary software platforms, and to replace them with open-source, publicly available software technologies. One area we’ll be replacing is their current middleware software which consists of IBM WebSphere Message Broker. While it is a fine tool for the most part, it’s also bulky and expensive to operate. The final solution we’re working towards will be much more cloud-native, will support scalability, be able to process messages much faster, and consist of several different technologies and vendors (not just a single vendor, as is the case with the current IBM solution). This new middleware platform will consist of: Apache Kafka for the delivery of messages, Spring Cloud Data Flow, and a handful of RESTful APIs.