2019-07-16T05:40:00Z

What advice do you have for others considering Spark SQL?

Julia Miller - PeerSpot reviewer
  • 0
  • 3
PeerSpot user
Get the report
Helped 765,234 peers since 2012
14

14 Answers

SurjitChoudhury - PeerSpot reviewer
Real User
Top 5
2023-11-23T15:19:35Z
Nov 23, 2023

Overall, I would rate Spark SQL as a seven out of ten.

Search for a product comparison
SB
Real User
Top 5
2023-08-18T08:37:21Z
Aug 18, 2023

I recommend Spark SQL, but I will need to see what the results will be of our evaluation of Dremio. I'm especially expecting good performance because of the reflection mechanisms, which are actually materials used. But the open question is issues with the refresh rate. I don't know how bad or good that is. I rate Spark SQL a ten out of ten with the correct implementation.

Aria Amini - PeerSpot reviewer
Real User
Top 5Leaderboard
2023-07-26T11:55:00Z
Jul 26, 2023

If the user data has a big volume of data, I think they should use PySpark, but for scenarios where they use a medium amount of data, they should not use PySpark because they have some overheads. I rate Spark SQL a nine out of ten.

Sahil Taneja - PeerSpot reviewer
Real User
Top 5Leaderboard
2023-05-05T08:54:14Z
May 5, 2023

It's pretty good to use in the initial phases. Overall, I would rate the solution an eight out of ten.

Lucas Dreyer - PeerSpot reviewer
Real User
Top 10Leaderboard
2023-01-04T13:37:06Z
Jan 4, 2023

Training is quite important to get users up to scratch with Sparks SQL and Spark. Planning is needed in terms of training and skillsets. In terms of the typical DevOps MLOps deployment with pipelines, this training is particularly important. Otherwise you may end up with lots of functionality and queries that are difficult to change, deploy or maintain. I would rate this solution an eight out of ten. In terms of scalability, it is very useful.

KM
Real User
Top 10Leaderboard
2022-11-22T13:27:47Z
Nov 22, 2022

The solution is very similar to the generic Spark and SQL language. I rate the solution an eight out of ten.

Learn what your peers think about Spark SQL. Get advice and tips from experienced pros sharing their opinions. Updated: March 2024.
765,234 professionals have used our research since 2012.
Mahdi Sharifmousavi - PeerSpot reviewer
Real User
Top 10
2022-08-10T11:49:13Z
Aug 10, 2022

I recommend this solution. Spark provides good, clear documentation that is well organized.

AG
Real User
2021-12-02T15:07:38Z
Dec 2, 2021

I rate this solution an eight out of ten and would recommend it to others.

KG
Real User
2021-05-29T10:04:10Z
May 29, 2021

I rate Spark SQL a ten out of ten.

QG
Real User
2020-09-27T04:10:00Z
Sep 27, 2020

Being a new user, I would rate Spark SQL a four out of ten.

PK
Real User
2020-04-26T06:32:00Z
Apr 26, 2020

I would rate Spark SQL a nine out of ten. My advice would be to read Databricks books about Spark. It's a good source of knowledge. In the next update, we'd like to see better performance for small points of data. It is possible but there are better tools that are faster and cheaper.

SS
Real User
2020-03-18T06:06:00Z
Mar 18, 2020

We will have a lot of big data, which is why we need it. Otherwise, the solution is not needed. The solution really depends on the size of your data, its complexity, and the analysis that you are doing. Spark is good, but it is not mandatory. Since I don't have experience in production with the solution, the best I can rate it now is a five (out of 10).

DM
Real User
Top 20
2020-02-09T08:17:05Z
Feb 9, 2020

We use both the on-premises and cloud deployment models. We have a relationship with Cloudera and use their distribution channels. We don't have a relationship with Apache. Spark SQL is a good product. However, users need to have the capability of implementing the correct tools and efficiencies. I'd rate the solution seven out of ten.

it_user986637 - PeerSpot reviewer
Real User
2019-07-16T05:40:00Z
Jul 16, 2019

We've just started using this solution. We were using it until recently on a research basis, just to measure the performance, the cost, and so on and so forth. Many things could be improved, but are okay up till now, I'm happy with. I would recommend the product. I would rate this solution eight out of ten.

Spark SQL is a Spark module for structured data processing. Unlike the basic Spark RDD API, the interfaces provided by Spark SQL provide Spark with more information about the structure of both the data and the computation being performed. There are several ways to interact with Spark SQL including SQL and the Dataset API. When computing a result the same execution engine is used, independent of which API/language you are using to express the computation. This unification means that developers...
Download Spark SQL ReportRead more