IT Central Station is now PeerSpot: Here's why

AWS Lambda vs Apache Spark comparison

Cancel
You must select at least 2 products to compare!
Apache Logo
10,192 views|8,105 comparisons
Amazon Logo
8,873 views|8,100 comparisons
Featured Review
Buyer's Guide
AWS Lambda vs. Apache Spark
May 2022
Find out what your peers are saying about AWS Lambda vs. Apache Spark and other solutions. Updated: May 2022.
610,190 professionals have used our research since 2012.
Quotes From Members
We asked business professionals to review the solutions they use.
Here are some excerpts of what they said:
Pros
"The memory processing engine is the solution's most valuable aspect. It processes everything extremely fast, and it's in the cluster itself. It acts as a memory engine and is very effective in processing data correctly.""One of the key features is that Apache Spark is a distributed computing framework. You can help multiple slaves and distribute the workload between them.""Spark helps us reduce startup time for our customers and gives a very high ROI in the medium term.""The solution has been very stable.""Apache Spark can do large volume interactive data analysis.""AI libraries are the most valuable. They provide extensibility and usability. Spark has a lot of connectors, which is a very important and useful feature for AI. You need to connect a lot of points for AI, and you have to get data from those systems. Connectors are very wide in Spark. With a Spark cluster, you can get fast results, especially for AI.""Its scalability and speed are very valuable. You can scale it a lot. It is a great technology for big data. It is definitely better than a lot of earlier warehouse or pipeline solutions, such as Informatica. Spark SQL is very compliant with normal SQL that we have been using over the years. This makes it easy to code in Spark. It is just like using normal SQL. You can use the APIs of Spark or you can directly write SQL code and run it. This is something that I feel is useful in Spark.""Spark can handle small to huge data and is suitable for any size of company."

More Apache Spark Pros →

"The main features of this solution are the ability to integrate multiple AWS applications or external applications very quickly and organize all of them. Additionally, it is easy to use and you can run various programming languages, such as Python, Go, and Java.""Amazon takes care of the scalability. That's the right way. It's automatic and it's fully managed. That's one benefit of Lambda.""I have found all of the features valuable. It's an easy and cheap solution.""The installation and configuration of the solution is straightforward.""It is my preferred product, as it provides me with source code within the solution.""It's a fairly easy solution to learn.""We have no issues with the technical support.""Because AWS Lambda is serverless, server configuration is not required, and we can run it directly anywhere."

More AWS Lambda Pros →

Cons
"Its UI can be better. Maintaining the history server is a little cumbersome, and it should be improved. I had issues while looking at the historical tags, which sometimes created problems. You have to separately create a history server and run it. Such things can be made easier. Instead of separately installing the history server, it can be made a part of the whole setup so that whenever you set it up, it becomes available.""Spark could be improved by adding support for other open-source storage layers than Delta Lake.""Apache Spark could improve the connectors that it supports. There are a lot of open-source databases in the market. For example, cloud databases, such as Redshift, Snowflake, and Synapse. Apache Spark should have connectors present to connect to these databases. There are a lot of workarounds required to connect to those databases, but it should have inbuilt connectors.""The logging for the observability platform could be better.""We are building our own queries on Spark, and it can be improved in terms of query handling.""Stream processing needs to be developed more in Spark. I have used Flink previously. Flink is better than Spark at stream processing.""The initial setup was not easy.""The graphical user interface (UI) could be a bit more clear. It's very hard to figure out the execution logs and understand how long it takes to send everything. If an execution is lost, it's not so easy to understand why or where it went. I have to manually drill down on the data processes which takes a lot of time. Maybe there could be like a metrics monitor, or maybe the whole log analysis could be improved to make it easier to understand and navigate."

More Apache Spark Cons →

"Its price should be improved. Its pricing is on the higher side. I am not sure if it currently supports the Go language. If it doesn't support the Go language, they can introduce it.""We'd love to see more integration potential in the future.""The user-friendliness of the solution could be improved.""I wish to see better execution time in the next release.""We need to better understand Lambda for different scenarios. We need some joint effort between Amazon and the users to have the users identify how they can really leverage Lambda. It's not about Lambda itself; it's about the practice, the guidance. There needs to be very good documentation. From the user perspective, what exists now is not always enough.""Amazon doesn't have enough local support based in our country.""Its performance can be improved. There should also be more dynamic security permissions.""AWS Lambda could improve by having no-code or low-code options because currently, you need to be able to write code well to use it."

More AWS Lambda Cons →

Pricing and Cost Advice
  • "Apache Spark is open-source. You have to pay only when you use any bundled product, such as Cloudera."
  • "Since we are using the Apache Spark version, not the data bricks version, it is an Apache license version, the support and resolution of the bug are actually late or delayed. The Apache license is free."
  • "Spark is an open-source solution, so there are no licensing costs."
  • More Apache Spark Pricing and Cost Advice →

  • "Its pricing is on the higher side."
  • "The price of the solution is reasonable and it is a pay-per-use model. It is very good for cost optimization."
  • "The cost is based on runtime."
  • "The fees are volume-based."
  • "AWS Lambda is inexpensive."
  • "Lambda is a good and cheap solution and I would recommend it to those without a huge payload."
  • "For licensing, we pay a yearly subscription."
  • "We don't need to pay for licensing to use Lambda."
  • More AWS Lambda Pricing and Cost Advice →

    report
    Use our free recommendation engine to learn which Compute Service solutions are best for your needs.
    610,190 professionals have used our research since 2012.
    Questions from the Community
    Top Answer:I don't think using Apache Spark without Hadoop has any major drawbacks or issues. I have used Apache Spark quite successfully with AWS S3 on many projects which are batch based. Yes for very high… more »
    Top Answer:One of Apache Spark's most valuable features is that it supports in-memory processing, the execution of jobs compared to traditional tools is very fast.
    Top Answer:Spark is an open-source solution, so there are no licensing costs.
    Top Answer:AWS Lambda is a serverless solution. It doesn’t require any infrastructure, which allows for cost savings. There is no setup process to deal with, as the entire solution is in the cloud. If you use… more »
    Top Answer:AWS Lambda is interlinked with CloudWatch. When we have any errors we can directly go there and check the CloudWatch logs. Additionally, we can run it very fast and we can increase the RAM size and… more »
    Top Answer:AWS Lambda is a very inexpensive solution. They charge for the number of times we run it. If you run AWS Lambda for one time, they charge around 50 cents or 25 cents for the use. I don't know the… more »
    Ranking
    2nd
    out of 14 in Compute Service
    Views
    10,192
    Comparisons
    8,105
    Reviews
    11
    Average Words per Review
    447
    Rating
    8.1
    1st
    out of 14 in Compute Service
    Views
    8,873
    Comparisons
    8,100
    Reviews
    24
    Average Words per Review
    350
    Rating
    8.3
    Comparisons
    Learn More
    Overview

    Spark provides programmers with an application programming interface centered on a data structure called the resilient distributed dataset (RDD), a read-only multiset of data items distributed over a cluster of machines, that is maintained in a fault-tolerant way. It was developed in response to limitations in the MapReduce cluster computing paradigm, which forces a particular linear dataflowstructure on distributed programs: MapReduce programs read input data from disk, map a function across the data, reduce the results of the map, and store reduction results on disk. Spark's RDDs function as a working set for distributed programs that offers a (deliberately) restricted form of distributed shared memory

    AWS Lambda is a compute service that lets you run code without provisioning or managing servers. AWS Lambda executes your code only when needed and scales automatically, from a few requests per day to thousands per second. You pay only for the compute time you consume - there is no charge when your code is not running. With AWS Lambda, you can run code for virtually any type of application or backend service - all with zero administration. AWS Lambda runs your code on a high-availability compute infrastructure and performs all of the administration of the compute resources, including server and operating system maintenance, capacity provisioning and automatic scaling, code monitoring and logging. All you need to do is supply your code in one of the languages that AWS Lambda supports (currently Node.js, Java, C# and Python).

    You can use AWS Lambda to run your code in response to events, such as changes to data in an Amazon S3 bucket or an Amazon DynamoDB table; to run your code in response to HTTP requests using Amazon API Gateway; or invoke your code using API calls made using AWS SDKs. With these capabilities, you can use Lambda to easily build data processing triggers for AWS services like Amazon S3 and Amazon DynamoDB process streaming data stored in Amazon Kinesis, or create your own back end that operates at AWS scale, performance, and security.

    Offer
    Learn more about Apache Spark
    Learn more about AWS Lambda
    Sample Customers
    NASA JPL, UC Berkeley AMPLab, Amazon, eBay, Yahoo!, UC Santa Cruz, TripAdvisor, Taboola, Agile Lab, Art.com, Baidu, Alibaba Taobao, EURECOM, Hitachi Solutions
    Netflix
    Top Industries
    REVIEWERS
    Financial Services Firm36%
    Computer Software Company27%
    Marketing Services Firm9%
    Non Profit9%
    VISITORS READING REVIEWS
    Computer Software Company21%
    Comms Service Provider19%
    Financial Services Firm13%
    Media Company7%
    REVIEWERS
    Computer Software Company32%
    Financial Services Firm18%
    Educational Organization9%
    Manufacturing Company5%
    VISITORS READING REVIEWS
    Computer Software Company20%
    Media Company16%
    Comms Service Provider16%
    Financial Services Firm9%
    Company Size
    REVIEWERS
    Small Business43%
    Midsize Enterprise20%
    Large Enterprise38%
    VISITORS READING REVIEWS
    Small Business15%
    Midsize Enterprise15%
    Large Enterprise71%
    REVIEWERS
    Small Business41%
    Midsize Enterprise9%
    Large Enterprise50%
    VISITORS READING REVIEWS
    Small Business14%
    Midsize Enterprise12%
    Large Enterprise74%
    Buyer's Guide
    AWS Lambda vs. Apache Spark
    May 2022
    Find out what your peers are saying about AWS Lambda vs. Apache Spark and other solutions. Updated: May 2022.
    610,190 professionals have used our research since 2012.

    Apache Spark is ranked 2nd in Compute Service with 11 reviews while AWS Lambda is ranked 1st in Compute Service with 25 reviews. Apache Spark is rated 8.0, while AWS Lambda is rated 8.4. The top reviewer of Apache Spark writes "Provides fast aggregations, AI libraries, and a lot of connectors". On the other hand, the top reviewer of AWS Lambda writes "Programming is getting much easier and does not need a lot of configuration ". Apache Spark is most compared with Spring Boot, Azure Stream Analytics, AWS Batch, SAP HANA and Apache NiFi, whereas AWS Lambda is most compared with AWS Batch, Apache NiFi, Amazon EC2 Auto Scaling, Azure Stream Analytics and Amazon EC2. See our AWS Lambda vs. Apache Spark report.

    See our list of best Compute Service vendors.

    We monitor all Compute Service reviews to prevent fraudulent reviews and keep review quality high. We do not post reviews by company employees or direct competitors. We validate each review for authenticity via cross-reference with LinkedIn, and personal follow-up with the reviewer when necessary.