Apache Hadoop vs Apache Spark comparison

The compared Apache Hadoop and Apache Spark solutions aren't in the same category. Apache Hadoop is ranked #7 in Data Warehouse , with an average rating of 7.9, and holds a 5.1% mindshare in the category. Apache Spark is ranked #1 in H , with an average rating of 8.6, and holds a 17.8% mindshare. Additionally, 89% of Apache Hadoop users are willing to recommend the solution, compared to 90% of Apache Spark users who would recommend it.

Apache Hadoop

Read 40 Apache Hadoop reviews

1,323 Views
1,068 Comparison Views

89% willing to recommend

Apache Spark

Read 66 Apache Spark reviews

1,404 Views
1,090 Comparison Views

90% willing to recommend

Apache Hadoop

Apache Spark

Comparison Buyer's Guide

Download the report

Executive Summary

We performed a comparison between Apache Hadoop and Apache Spark based on real PeerSpot user reviews.

Find out what your peers are saying about Snowflake Computing, Oracle, Teradata and others in Data Warehouse.

To learn more, read our detailed Data Warehouse Report (Updated: April 2025).

Buyer's Guide

Data Warehouse

April 2025

Download the complete report

Helped 849,686 peers since 2012

Review summaries and opinions

We asked business professionals to review the solutions they use. Here are some excerpts of what they said:

Categories and Ranking

Apache Hadoop

Average Rating

7.8

Reviews Sentiment

6.7

Number of Reviews

Ranking in other categories

Data Warehouse (7th)

Apache Spark

Average Rating

8.4

Reviews Sentiment

7.7

Number of Reviews

Ranking in other categories

Hadoop (1st), Compute Service (4th), Java Frameworks (2nd)

Mindshare comparison

Apache Hadoop and Apache Spark aren’t in the same category and serve different purposes. Apache Hadoop is designed for Data Warehouse and holds a mindshare of 5.1%, down 5.6% compared to last year.
Apache Spark, on the other hand, focuses on Hadoop, holds 17.8% mindshare, down 21.4% since last year.

Data Warehouse

Hadoop

Q&A Highlights

it_user1272297

Special Adviser Strategy at a university with 501-1,000 employees

Apr 19, 2020

Which is the best RDMBS solution for big data?

I haven't used SQream personally. However, if you are only considering GPU based rdbms's please check the following https://hackernoon.com/which-gpu-database-is-right-for-me-6ceef6a17505

See all answers

Featured Reviews

Sushil Arya

Software developer at Fiserv

Provides ease of integration with the IT workflow of a business

When working with Kafka, I saw that the data came in an incremental order. The incremental data processing part is still not very effective in Apache Hadoop. If the data is already there, it can be processed very effectively, especially if the data is coming in every second. If you want to know the location of some data every second, then such data is not processed effectively in Apache Hadoop. I can say that one of the features where improvements are required revolves around the licensing cost of the tool. If the tool can build some licensing structures in a pay-per-use manner, organizations can get the look and feel of Apache Hadoop. Apache Hadoop can offer a licensing structure of the product that can be seen as similar to how AWS operates. Apache Hadoop can look into the capability of processing incremental data. The tool's setup process can be a scope of improvement. Also, it is not very simple because while doing the setup, we need to do all the server settings, including port listing and firewall configurations. If we look at other products on the market, then they can be made simpler. There are certain shortcomings when it comes to the product's technical support part, making it an area where improvements are required. The time frame for the resolution is an area that needs to be improved. The overall communication part of the technical support team also needs improvement.

Read full review

Ilya Afanasyev

Senior Software Development Engineer at Yahoo!

Reliable, able to expand, and handle large amounts of data well

We use batch processing. It works well with our formats and file versions. There's a lot of functionality. In our pipeline each hour, we make a copy of data from MongoDB, of the changes from MongoDB to some specific file. Each time pipeline copied all of the data, it would do it each time without changes to all of the tables. Tables have a lot of data, and in the last MongoDB version, there is a possibility to read only changed data. This reduced the cost and configuration of the cluster, and we saved about $150,000. The solution is scalable. It's a stable product.

Read full review

Quotes from Members

We asked business professionals to review the solutions they use. Here are some excerpts of what they said:

Pros

"Two valuable features are its scalability and parallel processing. There are jobs that cannot be done unless you have massively parallel processing."

"The most important feature is its ability to handle large volumes. Some of our customers have really large volumes, and it is capable of handling their data in terms of the core volume and daily incremental volume. So, its processing power and speed are most valuable."

"Its integration is Hadoop's best feature because that allows us to support different tools in a big data platform."

"They have integrated other tools as well, like Power BI and Oracle BI, both on Azure, for reporting. Oracle BI is difficult to integrate."

"It is a reliable product."

"Initially, with RDBMS alone, we had a lot of work and few servers running on-premise and on cloud for the PoC and incubation. With the use of Hadoop and ecosystem components and tools, and managing it in Amazon EC2, we have created a Big Data "lab" which helps us to centralize all our work and solutions into a single repository. This has cut down the time in terms of maintenance, development and, especially, data processing challenges."

"The most valuable feature is scalability and the possibility to work with major information and open source capability."

"One valuable feature is that we can download data."

More Apache Hadoop pros

"Apache Spark provides a very high-quality implementation of distributed data processing."

"I like Apache Spark's flexibility the most. Before, we had one server that would choke up. With the solution, we can easily add more nodes when needed. The machine learning models are also really helpful. We use them to predict energy theft and find infrastructure problems."

"It provides a scalable machine learning library."

"ETL and streaming capabilities."

"The most crucial feature for us is the streaming capability. It serves as a fundamental aspect that allows us to exert control over our operations."

"It is highly scalable, allowing you to efficiently work with extensive datasets that might be problematic to handle using traditional tools that are memory-constrained."

"There's a lot of functionality."

"The most significant advantage of Spark 3.0 is its support for DataFrame UDF Pandas UDF features."

More Apache Spark pros

Cons

"The problem with Apache Hadoop arose when the guys that originally set it up left the firm, and the group that later owned it didn't have enough technical resources to properly maintain it."

"From the Apache perspective or the open-source community, they need to add more capabilities to make life easier from a configuration and deployment perspective."

"The integration with Apache Hadoop with lots of different techniques within your business can be a challenge."

"The product's availability of comprehensive training materials could be improved for faster onboarding and skill development among team members."

"Hadoop lacks OLAP capabilities."

"Hadoop's security could be better."

"It needs better user interface (UI) functionalities."

"It requires a great deal of learning curve to understand. The overall Hadoop ecosystem has a large number of sub-products. There is ZooKeeper, and there are a whole lot of other things that are connected. In many cases, their functionalities are overlapping, and for a newcomer or our clients, it is very difficult to decide which of them to buy and which of them they don't really need. They require a consulting organization for it, which is good for organizations such as ours because that's what we do, but it is not easy for the end customers to gain so much knowledge and optimally use it."

More Apache Hadoop cons

"Stream processing needs to be developed more in Spark. I have used Flink previously. Flink is better than Spark at stream processing."

"There could be enhancements in optimization techniques, as there are some limitations in this area that could be addressed to further refine Spark's performance."

"From my perspective, the only thing that needs improvement is the interface, as it was not easily understandable."

"There were some problems related to the product's compatibility with a few Python libraries."

"Its UI can be better. Maintaining the history server is a little cumbersome, and it should be improved. I had issues while looking at the historical tags, which sometimes created problems. You have to separately create a history server and run it. Such things can be made easier. Instead of separately installing the history server, it can be made a part of the whole setup so that whenever you set it up, it becomes available."

"Apache Spark lacks geospatial data."

"When you want to extract data from your HDFS and other sources then it is kind of tricky because you have to connect with those sources."

"Apache Spark can improve the use case scenarios from the website. There is not any information on how you can use the solution across the relational databases toward multiple databases."

More Apache Spark cons

Pricing and Cost Advice

"There are no licensing costs involved, hence money is saved on the software infrastructure."

"The product is open-source, but some associated licensing fees depend on the subscription level."

"For any big enterprise the costs can be handled, and it is suitable for big enterprises because the scale of data is large. For medium and small enterprises, the tool is on the high-price side."

"If my company can use the cloud version of Apache Hadoop, particularly the cloud storage feature, it would be easier and would cost less because an on-premises deployment has a higher cost during storage, for example, though I don't know exactly how much Apache Hadoop costs."

"We just use the free version."

"It's reasonable, but there's room for improvement in cost-effectiveness."

"This is a low cost and powerful solution."

"Do take into consider that data storage and compute capacity scale differently and hence purchasing a "boxed" / 'all-in-one" solution (software and hardware) might not be the best idea."

More Apache Hadoop pricing and cost advice

"On the cloud model can be expensive as it requires substantial resources for implementation, covering on-premises hardware, memory, and licensing."

"They provide an open-source license for the on-premise version."

"Licensing costs can vary. For instance, when purchasing a virtual machine, you're asked if you want to take advantage of the hybrid benefit or if you prefer the license costs to be included upfront by the cloud service provider, such as Azure. If you choose the hybrid benefit, it indicates you already possess a license for the operating system and wish to avoid additional charges for that specific VM in Azure. This approach allows for a reduction in licensing costs, charging only for the service and associated resources."

"Spark is an open-source solution, so there are no licensing costs."

"I did not pay anything when using the tool on cloud services, but I had to pay on the compute side. The tool is not expensive compared with the benefits it offers. I rate the price as an eight out of ten."

"It is quite expensive. In fact, it accounts for almost 50% of the cost of our entire project."

"Apache Spark is an open-source solution, and there is no cost involved in deploying the solution on-premises."

"Apache Spark is an expensive solution."

More Apache Spark pricing and cost advice

See which vendors are best for you

Use our free recommendation engine to learn which Data Warehouse solutions are best for your needs.

See recommendations

849,686 professionals have used our research since 2012.

Answers from the Community

it_user1272297

Special Adviser Strategy at a university with 501-1,000 employees

Apr 19, 2020

Which is the best RDMBS solution for big data?

I haven't used SQream personally. However, if you are only considering GPU based rdbms's please check the following https://hackernoon.com/which-gpu-database-is-right-for-me-6ceef6a17505

2 out of 4 answers

Russell Rothstein

Founder and CEO at PeerSpot

Jan 27, 2020

Morten, the most popular comparisons of SQream can be found here: https://www.itcentralstation.com/products/sqream-db-alternatives-and-competitors The top ones include Cassandra, MemSQL, MongoDB, and Vertica.

Read full answer

reviewer1219965

Data Architect at a tech services company with 201-500 employees

Jan 27, 2020

I haven't used SQream personally. However, if you are only considering GPU based rdbms's please check the following https://hackernoon.com/which-gpu-database-is-right-for-me-6ceef6a17505

Read full answer

See all 4 answers

Top Industries

By visitors reading reviews

Financial Services Firm

33%

Computer Software Company

11%

University

Energy/Utilities Company

Financial Services Firm

27%

Computer Software Company

13%

Manufacturing Company

Comms Service Provider

Company Size

By reviewers

Large Enterprise

Midsize Enterprise

Small Business

Questions from the Community

What do you like most about Apache Hadoop?

It's primarily open source. You can handle huge data volumes and create your own views, workflows, and tables. I can also use it for real-time data streaming.

See all answers

What is your experience regarding pricing and costs for Apache Hadoop?

The product is open-source, but some associated licensing fees depend on the subscription level. While it might be free for students, organizations typically need to pay for their subscriptions. Th...

See all answers

What needs improvement with Apache Hadoop?

The problem with Apache Hadoop arose when the guys that originally set it up left the firm, and the group that later owned it didn't have enough technical resources to properly maintain it. This wa...

See all answers

What do you like most about Apache Spark?

We use Spark to process data from different data sources.

See all answers

What is your experience regarding pricing and costs for Apache Spark?

Compared to other solutions like Doc DB, Spark is more costly due to the need for extensive infrastructure. It requires significant investment in infrastructure, which can be expensive. While cloud...

See all answers

What needs improvement with Apache Spark?

The Spark solution could improve in scheduling tasks and managing dependencies. Spark alone cannot handle sequential tasks, requiring environments like Airflow scheduler or scripts. For instance, o...

See all answers

Comparisons

Oracle Exadata vs Apache Hadoop

Compared 21% of the time

Teradata vs Apache Hadoop

Compared 13% of the time

Azure Data Factory vs Apache Hadoop

Compared 13% of the time

BigQuery vs Apache Hadoop

Compared 11% of the time

Microsoft Azure Synapse Analytics vs Apache Hadoop

Compared 8% of the time

More Apache Hadoop Competitors

Spring Boot vs Apache Spark

Compared 26% of the time

AWS Batch vs Apache Spark

Compared 12% of the time

SAP HANA vs Apache Spark

Compared 12% of the time

Cloudera Distribution for Hadoop vs Apache Spark

Compared 7% of the time

Spark SQL vs Apache Spark

Compared 7% of the time

More Apache Spark Competitors

Product Reports

Buyer's Guide

Apache Hadoop

April 2025

Download Apache Hadoop product report

Buyer's Guide

Apache Spark

April 2025

Download Apache Spark product report

Overview

The Apache Hadoop project develops open-source software for reliable, scalable, distributed computing. The Apache Hadoop software library is a framework that allows for the distributed processing of large data sets across clusters of computers using simple programming models. It is designed to scale up from single servers to thousands of machines, each offering local computation and storage. Rather than rely on hardware to deliver high-availability, the library itself is designed to detect and handle failures at the application layer, so delivering a highly-available service on top of a cluster of computers, each of which may be prone to failures.

Apache

Spark provides programmers with an application programming interface centered on a data structure called the resilient distributed dataset (RDD), a read-only multiset of data items distributed over a cluster of machines, that is maintained in a fault-tolerant way. It was developed in response to limitations in the MapReduce cluster computing paradigm, which forces a particular linear dataflowstructure on distributed programs: MapReduce programs read input data from disk, map a function across the data, reduce the results of the map, and store reduction results on disk. Spark's RDDs function as a working set for distributed programs that offers a (deliberately) restricted form of distributed shared memory

Apache

Sample Customers

Amazon, Adobe, eBay, Facebook, Google, Hulu, IBM, LinkedIn, Microsoft, Spotify, AOL, Twitter, University of Maryland, Yahoo!, Cornell University Web Lab

NASA JPL, UC Berkeley AMPLab, Amazon, eBay, Yahoo!, UC Santa Cruz, TripAdvisor, Taboola, Agile Lab, Art.com, Baidu, Alibaba Taobao, EURECOM, Hitachi Solutions

Find out what your peers are saying about Snowflake Computing, Oracle, Teradata and others in Data Warehouse. Updated: April 2025.

DOWNLOAD NOW

849,686 professionals have used our research since 2012.

We monitor all Data Warehouse reviews to prevent fraudulent reviews and keep review quality high. We do not post reviews by company employees or direct competitors. We validate each review for authenticity via cross-reference with LinkedIn, and personal follow-up with the reviewer when necessary.