Try our new research platform with insights from 80,000+ expert users

AWS Lambda vs Apache NiFi vs Google Cloud Dataflow comparison

 

Comparison Buyer's Guide

Executive Summary

Review summaries and opinions

We asked business professionals to review the solutions they use. Here are some excerpts of what they said:
 

Mindshare comparison

Compute Service
Compute Service
Streaming Analytics
 

Featured Reviews

Bharghava Raghavendra Beesa - PeerSpot reviewer
The tool enables effective data transformation and integration
There are some areas for improvement, particularly with record-level tasks that take a bit of time. The quality of JSON data processing could be improved, as JSON workloads require manual conversions without a specific process. Enhancing features related to alerting would be helpful, including mobile alerts for pipeline issues. Integration with mobile devices for error alerts would simplify information delivery.
Andrew-Wong - PeerSpot reviewer
Convenience in deployment process with room for code preview improvement
Having a better preview would be helpful. Sometimes, if my Lambda code is too big, it can be inconvenient as I'm unable to see my code when it exceeds a certain size. AWS has a limit, like a three-megabyte limit, beyond which I cannot view or edit the code easily.
Jana Polianskaja - PeerSpot reviewer
Build Scalable Data Pipelines with Apache Beam and Google Cloud Dataflow
As a data engineer, I find several features of Google Cloud Dataflow particularly valuable. The ability to test solutions locally using Direct Runner is crucial for development, allowing me to validate pipelines without incurring the costs of full Dataflow jobs. The unified programming model for both batch and streaming processing is exceptional - requiring only minor code adjustments to optimize for either mode. This flexibility extends to language support, with robust implementations in both Java and Python, allowing teams to leverage their existing expertise. The platform's comprehensive monitoring capabilities are another standout feature. The intuitive interface, Grafana integration, and extensive service connectivity make troubleshooting and performance tracking highly efficient. Furthermore, seamless integration with Google Cloud Composer (managed Airflow) enables sophisticated orchestration of data pipelines.

Quotes from Members

We asked business professionals to review the solutions they use. Here are some excerpts of what they said:
 

Pros

"The initial setup is very easy. I would rate my experience with the initial setup a ten out of ten, where one point is difficult, and ten points are easy."
"It is highly effective for handling real-time data by working with APIs for immediate and continuous data extraction."
"The initial setup is very easy."
"It's an automated flow, where you can build a flow from source to destination, then do the transformation in between."
"Visually, this is a good product."
"We can integrate the tool with other applications easily."
"The most valuable feature has been the range of clients and the range of connectors that we could use."
"The user interface is good and makes it easy to design very popular workflows."
"Lambda being serverless is a great feature that is appropriate for our use cases."
"Thanks to this solution, we do not need to worry about hardware or resource utilization. It saves us time."
"It's a fairly easy solution to learn."
"The solution is highly scalable."
"The most valuable feature is that there is no need to implement it in a server because it is a service."
"You can spin up anything instantly without any investment."
"Lambda allows you to focus on the code itself."
"It makes configurations more convenient as changes can be made through the environmental variables without altering the main code."
"Google's support team is good at resolving issues, especially with large data."
"The product's installation process is easy...The tool's maintenance part is somewhat easy."
"I would rate the overall solution a ten out of ten."
"It allows me to test solutions locally using runners like Direct Runner without having to start a Dataflow job, which can be costly."
"I don't need a server running all the time while using the tool. It is also easy to setup. The product offers a pay-as-you-go service."
"The solution allows us to program in any language we desire."
"Google Cloud Dataflow is useful for streaming and data pipelines."
"The integration within Google Cloud Platform is very good."
 

Cons

"The tool should incorporate more tutorials for advanced use cases. It has tutorials for simple use cases."
"There is room for improvement in integration with SSO. For example, NiFi does not have any integration with SSO. And if I want to give some kind of rollback access control across the organization. That is not possible."
"The overall stability of this solution could be improved. In a future release, we would like to have access to more features that could be used in a parallel way. This would provide more freedom with processing."
"There should be a better way to integrate a development environment with local tools."
"More features must be added to the product."
"I think the UI interface needs to be more user-friendly."
"The quality of JSON data processing could be improved, as JSON workloads require manual conversions without a specific process."
"There are some claims that NiFi is cloud-native but we have tested it, and it's not."
"We face some problems with the event-driven execution model."
"Lambda would benefit from a debugging feature as well."
"Regarding layers, you need to manually zip and install them. This step needs practice, and you might need to do it three to four times to get a hang of it."
"Lambda can only be used in one account; there's no possibility to utilize it in another account."
"Lambda could be improved in the sense that some of the things done with Lambda function take some time. So the performance could be better and faster."
"A very minor improvement would be to simplify the instructions on setting a trigger, as I had to read through them multiple times at the start."
"We don't have the inbuilt modules in AWS Lambda. If more modules were built into or integrated with AWS Lambda, that would help developers to code."
"It can be a bit difficult to switch between accounts when creating services for customers."
"I would like to see improvements in consistency and flexibility for schema design for NoSQL data stored in wide columns."
"I would like Google Cloud Dataflow to be integrated with IT data flow and other related services to make it easier to use as it is a complex tool."
"The deployment time could also be reduced."
"The solution's setup process could be more accessible."
"The technical support has slight room for improvement."
"The authentication part of the product is an area of concern where improvements are required."
"There are certain challenges regarding the Google Cloud Composer which can be improved."
"When I deploy the product in local errors, a lot of errors pop up which are not always caught. The solution's error logging is bad. It can take a lot of time to debug the errors. It needs to have better logs."
 

Pricing and Cost Advice

"It's an open-source solution."
"The solution is open-source."
"We use the free version of Apache NiFi."
"I used the tool's free version."
"AWS is slightly more expensive than Azure."
"AWS Lambda is cheap."
"Price-wise, AWS Lambda is very cheap. It's not free, but it's not that expensive."
"I think the price is okay. However, if they add more functionality, they can have better prices. In fact, they should have better and more flexible packages for clients who have greater consumption of Lambda."
"It computes by the cycle, and it's very cheap."
"AWS Lambda license is paid on a monthly basis."
"The solution is free of cost for the first year, and after that, it becomes expensive."
"Lambda is a good and cheap solution and I would recommend it to those without a huge payload."
"The price of the solution depends on many factors, such as how they pay for tools in the company and its size."
"The solution is cost-effective."
"On a scale from one to ten, where one is cheap, and ten is expensive, I rate Google Cloud Dataflow's pricing a four out of ten."
"The tool is cheap."
"The solution is not very expensive."
"Google Cloud is slightly cheaper than AWS."
"On a scale from one to ten, where one is cheap, and ten is expensive, I rate the solution's pricing a seven to eight out of ten."
"Google Cloud Dataflow is a cheap solution."
report
Use our free recommendation engine to learn which Compute Service solutions are best for your needs.
856,278 professionals have used our research since 2012.
 

Top Industries

By visitors reading reviews
Financial Services Firm
15%
Computer Software Company
13%
Manufacturing Company
11%
Retailer
7%
Educational Organization
59%
Financial Services Firm
9%
Computer Software Company
6%
Manufacturing Company
4%
Financial Services Firm
18%
Manufacturing Company
13%
Retailer
11%
Computer Software Company
10%
 

Company Size

By reviewers
Large Enterprise
Midsize Enterprise
Small Business
 

Questions from the Community

What is your experience regarding pricing and costs for Apache NiFi?
Apache NiFi is open-source and free. Its integration with systems like Cloudera can be expensive, but Apache NiFi its...
What needs improvement with Apache NiFi?
The logging system of Apache NiFi needs improvement. It is difficult to debug compared to Airflow ( /products/apache-...
Which is better, AWS Lambda or Batch?
AWS Lambda is a serverless solution. It doesn’t require any infrastructure, which allows for cost savings. There is n...
What do you like most about AWS Lambda?
The tool scales automatically based on the number of incoming requests.
What is your experience regarding pricing and costs for AWS Lambda?
The pricing of AWS Lambda is reasonable. It's beneficial and cost-effective for users regardless of the number of ins...
What do you like most about Google Cloud Dataflow?
The product's installation process is easy...The tool's maintenance part is somewhat easy.
What is your experience regarding pricing and costs for Google Cloud Dataflow?
Pricing is normal. It is part of a package received from Google, and they are not charging us too high.
What needs improvement with Google Cloud Dataflow?
I am not sure, as we built only one job, and it is running on a daily basis. Everything else is managed using BigQuer...
 

Also Known As

No data available
No data available
Google Dataflow
 

Overview

 

Sample Customers

Macquarie Telecom Group, Dovestech, Slovak Telekom, Looker, Hastings Group
Netflix
Absolutdata, Backflip Studios, Bluecore, Claritics, Crystalloids, Energyworx, GenieConnect, Leanplum, Nomanini, Redbus, Streak, TabTale
Find out what your peers are saying about Amazon Web Services (AWS), Apache, Zadara and others in Compute Service. Updated: June 2025.
856,278 professionals have used our research since 2012.