We performed a comparison between Apache Spark and Spark SQL based on real PeerSpot user reviews.
Find out in this report how the two Hadoop solutions compare in terms of features, pricing, service and support, easy of deployment, and ROI."The product’s most valuable features are lazy evaluation and workload distribution."
"It is useful for handling large amounts of data. It is very useful for scientific purposes."
"It provides a scalable machine learning library."
"I found the solution stable. We haven't had any problems with it."
"Features include machine learning, real time streaming, and data processing."
"The most crucial feature for us is the streaming capability. It serves as a fundamental aspect that allows us to exert control over our operations."
"The most valuable feature of Apache Spark is its memory processing because it processes data over RAM rather than disk, which is much more efficient and fast."
"The memory processing engine is the solution's most valuable aspect. It processes everything extremely fast, and it's in the cluster itself. It acts as a memory engine and is very effective in processing data correctly."
"The solution is easy to understand if you have basic knowledge of SQL commands."
"The speed of getting data."
"The stability was fine. It behaved as expected."
"This solution is useful to leverage within a distributed ecosystem."
"Data validation and ease of use are the most valuable features."
"Certain data sets that are very large are very difficult to process with Pandas and Python libraries. Spark SQL has helped us a lot with that."
"Overall the solution is excellent."
"The performance is one of the most important features. It has an API to process the data in a functional manner."
"Its UI can be better. Maintaining the history server is a little cumbersome, and it should be improved. I had issues while looking at the historical tags, which sometimes created problems. You have to separately create a history server and run it. Such things can be made easier. Instead of separately installing the history server, it can be made a part of the whole setup so that whenever you set it up, it becomes available."
"It needs a new interface and a better way to get some data. In terms of writing our scripts, some processes could be faster."
"Apache Spark can improve the use case scenarios from the website. There is not any information on how you can use the solution across the relational databases toward multiple databases."
"Apache Spark could potentially improve in terms of user-friendliness, particularly for individuals with a SQL background. While it's suitable for those with programming knowledge, making it more accessible to those without extensive programming skills could be beneficial."
"When you are working with large, complex tasks, the garbage collection process is slow and affects performance."
"I would like to see integration with data science platforms to optimize the processing capability for these tasks."
"This solution currently cannot support or distribute neural network related models, or deep learning related algorithms. We would like this functionality to be developed."
"It requires overcoming a significant learning curve due to its robust and feature-rich nature."
"It would be useful if Spark SQL integrated with some data visualization tools."
"It takes a bit of time to get used to using this solution versus Pandas as it has a steep learning curve."
"In terms of improvement, the only thing that could be enhanced is the stability aspect of Spark SQL."
"The solution needs to include graphing capabilities. Including financial charts would help improve everything overall."
"In the next update, we'd like to see better performance for small points of data. It is possible but there are better tools that are faster and cheaper."
"In the next release, maybe the visualization of some command-line features could be added."
"Being a new user, I am not able to find out how to partition it correctly. I probably need more information or knowledge. In other database solutions, you can easily optimize all partitions. I haven't found a quicker way to do that in Spark SQL. It would be good if you don't need a partition here, and the system automatically partitions in the best way. They can also provide more educational resources for new users."
"It would be beneficial for aggregate functions to include a code block or toolbox that explains its calculations or supported conditional statements."
Apache Spark is ranked 1st in Hadoop with 60 reviews while Spark SQL is ranked 4th in Hadoop with 14 reviews. Apache Spark is rated 8.4, while Spark SQL is rated 7.8. The top reviewer of Apache Spark writes "Reliable, able to expand, and handle large amounts of data well". On the other hand, the top reviewer of Spark SQL writes "Offers the flexibility to handle large-scale data processing". Apache Spark is most compared with Spring Boot, AWS Batch, SAP HANA, Cloudera Distribution for Hadoop and AWS Lambda, whereas Spark SQL is most compared with IBM Db2 Big SQL, HPE Ezmeral Data Fabric, SAP HANA and Netezza Analytics. See our Apache Spark vs. Spark SQL report.
See our list of best Hadoop vendors.
We monitor all Hadoop reviews to prevent fraudulent reviews and keep review quality high. We do not post reviews by company employees or direct competitors. We validate each review for authenticity via cross-reference with LinkedIn, and personal follow-up with the reviewer when necessary.
