Try our new research platform with insights from 80,000+ expert users

Apache Spark Streaming vs Google Cloud Dataflow comparison

 

Comparison Buyer's Guide

Executive SummaryUpdated on Dec 17, 2024

Review summaries and opinions

We asked business professionals to review the solutions they use. Here are some excerpts of what they said:
 

Categories and Ranking

Apache Spark Streaming
Ranking in Streaming Analytics
10th
Average Rating
8.0
Reviews Sentiment
7.4
Number of Reviews
11
Ranking in other categories
No ranking in other categories
Google Cloud Dataflow
Ranking in Streaming Analytics
7th
Average Rating
8.0
Reviews Sentiment
7.3
Number of Reviews
13
Ranking in other categories
No ranking in other categories
 

Mindshare comparison

As of May 2025, in the Streaming Analytics category, the mindshare of Apache Spark Streaming is 2.6%, down from 3.8% compared to the previous year. The mindshare of Google Cloud Dataflow is 7.1%, down from 7.2% compared to the previous year. It is calculated based on PeerSpot user engagement data.
Streaming Analytics
 

Featured Reviews

AbhishekGupta - PeerSpot reviewer
Easy integration, beneficial auto-scaling, and good open-sourced support community
The service structure of Apache Spark Streaming can improve. There are a lot of issues with memory management and latency. There is no real-time analytics. We recommend it for the use cases where there is a five-second latency, but not for a millisecond, an IOT-based, or the detection anomaly-based. Flink as a service is much better. Apache Spark Streaming does not have auto-tuning. A customer needs to invest a lot, in terms of management and maintenance.
Jana Polianskaja - PeerSpot reviewer
Build Scalable Data Pipelines with Apache Beam and Google Cloud Dataflow
As a data engineer, I find several features of Google Cloud Dataflow particularly valuable. The ability to test solutions locally using Direct Runner is crucial for development, allowing me to validate pipelines without incurring the costs of full Dataflow jobs. The unified programming model for both batch and streaming processing is exceptional - requiring only minor code adjustments to optimize for either mode. This flexibility extends to language support, with robust implementations in both Java and Python, allowing teams to leverage their existing expertise. The platform's comprehensive monitoring capabilities are another standout feature. The intuitive interface, Grafana integration, and extensive service connectivity make troubleshooting and performance tracking highly efficient. Furthermore, seamless integration with Google Cloud Composer (managed Airflow) enables sophisticated orchestration of data pipelines.

Quotes from Members

We asked business professionals to review the solutions they use. Here are some excerpts of what they said:
 

Pros

"The solution is very stable and reliable."
"Apache Spark Streaming has features like checkpointing and Streaming API that are useful."
"Apache Spark Streaming is versatile. You can use it for competitive intelligence, gathering data from competitors, or for internal tasks like monitoring workflows."
"It's the fastest solution on the market with low latency data on data transformations."
"Apache Spark's capabilities for machine learning are quite extensive and can be used in a low-code way."
"The platform’s most valuable feature for processing real-time data is its ability to handle continuous data streams."
"The solution is better than average and some of the valuable features include efficiency and stability."
"As an open-source solution, using it is basically free."
"The most valuable features of Google Cloud Dataflow are scalability and connectivity."
"I would rate the overall solution a ten out of ten."
"The support team is good and it's easy to use."
"The most valuable features of Google Cloud Dataflow are the integration, it's very simple if you have the complete stack, which we are using. It is overall very easy to use, user-friendly friendly, and cost-effective if you know how to use it. The solution is very flexible for programmers, if you know how to do scripts or program in Python or any other language, it's extremely easy to use."
"It allows me to test solutions locally using runners like Direct Runner without having to start a Dataflow job, which can be costly."
"It is a scalable solution."
"The product's installation process is easy...The tool's maintenance part is somewhat easy."
"The service is relatively cheap compared to other batch-processing engines."
 

Cons

"In terms of improvement, the UI could be better."
"We don't have enough experience to be judgmental about its flaws."
"The cost and load-related optimizations are areas where the tool lacks and needs improvement."
"Integrating event-level streaming capabilities could be beneficial."
"The initial setup is quite complex."
"The debugging aspect could use some improvement."
"We would like to have the ability to do arbitrary stateful functions in Python."
"The service structure of Apache Spark Streaming can improve. There are a lot of issues with memory management and latency. There is no real-time analytics. We recommend it for the use cases where there is a five-second latency, but not for a millisecond, an IOT-based, or the detection anomaly-based. Flink as a service is much better."
"The deployment time could also be reduced."
"Google Cloud Data Flow can improve by having full simple integration with Kafka topics. It's not that complicated, but it could improve a bit. The UI is easy to use but the experience could be better. There are other tools available that do a better job."
"Promoting the technology more broadly would help increase its adoption."
"The technical support has slight room for improvement."
"I would like Google Cloud Dataflow to be integrated with IT data flow and other related services to make it easier to use as it is a complex tool."
"The solution's setup process could be more accessible."
"When I deploy the product in local errors, a lot of errors pop up which are not always caught. The solution's error logging is bad. It can take a lot of time to debug the errors. It needs to have better logs."
"They should do a market survey and then make improvements."
 

Pricing and Cost Advice

"On a scale from one to ten, where one is expensive, or not cost-effective, and ten is cheap, I rate the price a seven."
"Spark is an affordable solution, especially considering its open-source nature."
"People pay for Apache Spark Streaming as a service."
"I was using the open-source community version, which was self-hosted."
"The tool is cheap."
"On a scale from one to ten, where one is cheap, and ten is expensive, I rate the solution's pricing a seven to eight out of ten."
"On a scale from one to ten, where one is cheap, and ten is expensive, I rate Google Cloud Dataflow's pricing a four out of ten."
"Google Cloud is slightly cheaper than AWS."
"The solution is cost-effective."
"The solution is not very expensive."
"The price of the solution depends on many factors, such as how they pay for tools in the company and its size."
"Google Cloud Dataflow is a cheap solution."
report
Use our free recommendation engine to learn which Streaming Analytics solutions are best for your needs.
850,028 professionals have used our research since 2012.
 

Top Industries

By visitors reading reviews
Financial Services Firm
28%
Computer Software Company
20%
Manufacturing Company
6%
Healthcare Company
4%
Financial Services Firm
17%
Manufacturing Company
12%
Retailer
11%
Computer Software Company
10%
 

Company Size

By reviewers
Large Enterprise
Midsize Enterprise
Small Business
 

Questions from the Community

What do you like most about Apache Spark Streaming?
Apache Spark Streaming is versatile. You can use it for competitive intelligence, gathering data from competitors, or for internal tasks like monitoring workflows.
What needs improvement with Apache Spark Streaming?
We don't have enough experience to be judgmental about its flaws, as we've only used stable features like batch micro-batch. Integration poses no problem; however, I don't use some features and can...
What is your primary use case for Apache Spark Streaming?
We use Spark Streaming in a micro-batch region. It's not a full real-time system, but it offers high performance and low latency.
What do you like most about Google Cloud Dataflow?
The product's installation process is easy...The tool's maintenance part is somewhat easy.
What is your experience regarding pricing and costs for Google Cloud Dataflow?
Pricing is normal. It is part of a package received from Google, and they are not charging us too high.
What needs improvement with Google Cloud Dataflow?
I am not sure, as we built only one job, and it is running on a daily basis. Everything else is managed using BigQuery schedulers and Talend. However, occasionally, dealing with a huge volume of da...
 

Also Known As

Spark Streaming
Google Dataflow
 

Overview

 

Sample Customers

UC Berkeley AMPLab, Amazon, Alibaba Taobao, Kenshoo, eBay Inc.
Absolutdata, Backflip Studios, Bluecore, Claritics, Crystalloids, Energyworx, GenieConnect, Leanplum, Nomanini, Redbus, Streak, TabTale
Find out what your peers are saying about Apache Spark Streaming vs. Google Cloud Dataflow and other solutions. Updated: April 2025.
850,028 professionals have used our research since 2012.