We performed a comparison between Apache Spark and Google Cloud Dataflow based on real PeerSpot user reviews.
Find out what your peers are saying about Apache, Cloudera, Amazon Web Services (AWS) and others in Hadoop."The product's deployment phase is easy."
"The deployment of the product is easy."
"I feel the streaming is its best feature."
"AI libraries are the most valuable. They provide extensibility and usability. Spark has a lot of connectors, which is a very important and useful feature for AI. You need to connect a lot of points for AI, and you have to get data from those systems. Connectors are very wide in Spark. With a Spark cluster, you can get fast results, especially for AI."
"DataFrame: Spark SQL gives the leverage to create applications more easily and with less coding effort."
"Its scalability and speed are very valuable. You can scale it a lot. It is a great technology for big data. It is definitely better than a lot of earlier warehouse or pipeline solutions, such as Informatica. Spark SQL is very compliant with normal SQL that we have been using over the years. This makes it easy to code in Spark. It is just like using normal SQL. You can use the APIs of Spark or you can directly write SQL code and run it. This is something that I feel is useful in Spark."
"The memory processing engine is the solution's most valuable aspect. It processes everything extremely fast, and it's in the cluster itself. It acts as a memory engine and is very effective in processing data correctly."
"I found the solution stable. We haven't had any problems with it."
"The service is relatively cheap compared to other batch-processing engines."
"The most valuable features of Google Cloud Dataflow are the integration, it's very simple if you have the complete stack, which we are using. It is overall very easy to use, user-friendly friendly, and cost-effective if you know how to use it. The solution is very flexible for programmers, if you know how to do scripts or program in Python or any other language, it's extremely easy to use."
"The solution allows us to program in any language we desire."
"The support team is good and it's easy to use."
"Google Cloud Dataflow is useful for streaming and data pipelines."
"The most valuable features of Google Cloud Dataflow are scalability and connectivity."
"I don't need a server running all the time while using the tool. It is also easy to setup. The product offers a pay-as-you-go service."
"It is a scalable solution."
"They could improve the issues related to programming language for the platform."
"In data analysis, you need to take real-time data from different data sources. You need to process this in a subsecond, do the transformation in a subsecond, and all that."
"It needs a new interface and a better way to get some data. In terms of writing our scripts, some processes could be faster."
"The graphical user interface (UI) could be a bit more clear. It's very hard to figure out the execution logs and understand how long it takes to send everything. If an execution is lost, it's not so easy to understand why or where it went. I have to manually drill down on the data processes which takes a lot of time. Maybe there could be like a metrics monitor, or maybe the whole log analysis could be improved to make it easier to understand and navigate."
"At the initial stage, the product provides no container logs to check the activity."
"The management tools could use improvement. Some of the debugging tools need some work as well. They need to be more descriptive."
"When you are working with large, complex tasks, the garbage collection process is slow and affects performance."
"When using Spark, users may need to write their own parallelization logic, which requires additional effort and expertise."
"The authentication part of the product is an area of concern where improvements are required."
"When I deploy the product in local errors, a lot of errors pop up which are not always caught. The solution's error logging is bad. It can take a lot of time to debug the errors. It needs to have better logs."
"The solution's setup process could be more accessible."
"The technical support has slight room for improvement."
"Google Cloud Dataflow should include a little cost optimization."
"Google Cloud Data Flow can improve by having full simple integration with Kafka topics. It's not that complicated, but it could improve a bit. The UI is easy to use but the experience could be better. There are other tools available that do a better job."
"I would like Google Cloud Dataflow to be integrated with IT data flow and other related services to make it easier to use as it is a complex tool."
"There are certain challenges regarding the Google Cloud Composer which can be improved."
Apache Spark is ranked 1st in Hadoop with 60 reviews while Google Cloud Dataflow is ranked 7th in Streaming Analytics with 10 reviews. Apache Spark is rated 8.4, while Google Cloud Dataflow is rated 7.8. The top reviewer of Apache Spark writes "Reliable, able to expand, and handle large amounts of data well". On the other hand, the top reviewer of Google Cloud Dataflow writes "Easy to use for programmers, user-friendly, and scalable". Apache Spark is most compared with Spring Boot, AWS Batch, Spark SQL, SAP HANA and Cloudera Distribution for Hadoop, whereas Google Cloud Dataflow is most compared with Databricks, Apache NiFi, Amazon MSK, Amazon Kinesis and Talend Data Streams.
We monitor all Hadoop reviews to prevent fraudulent reviews and keep review quality high. We do not post reviews by company employees or direct competitors. We validate each review for authenticity via cross-reference with LinkedIn, and personal follow-up with the reviewer when necessary.
