AWS Lake Formation vs BigQuery comparison

"The solution is quite good at handling analytics. It's done a good job at helping us centralize them."

"AWS Lake Formation works hand in hand with other products."

"The most important advantage in using AWS Lake Formation is its ability to connect the data lake to the other technologies in AWS. This is what I advise my clients."

"We use AWS Lake Formation typically for the data warehouse."

"AWS Lake Formation lets you see all your data and tables on one screen."

"The solution has many features that are applicable to events such as audits."

"I can easily move data from cold storage to regular storage."

"It is seamlessly integrated within the AWS ecosystem, making it straightforward to manage access patterns for AWS-native services."

"I like that we can synch and run a large query. I also like that we can work with a large amount of data. You don't need to work separately, as it's a ready-made solution. It also comes with a built-in machine-learning feature. Once we start inputting the data, it will suggest some things related to the data, and we can come up with nice dashboards and statistics from a vast amount of data."

"BigQuery allows for very fast access, and it is efficient in handling large datasets compared to other SQL databases."

"BigQuery excels at data analysis. It processes vast amounts of information using its advanced architecture and sophisticated querying capabilities, making it crucial for critical insights and safe for handling sensitive data."

"It's similar to a Hadoop cluster, except it's managed by Google."

"The main thing I like about BigQuery is storage. We did an on-premise BigQuery migration with trillions of records. Usually, we have to deal with insufficient storage on-premises, but in BigQuery, we don't get that because it's like cloud storage, and we can have any number of records. That is one advantage. The next major advantage is the column length. We have some limits on column length on-premises, like 10,000, and we have to design it based on that. However, with BigQuery, we don't need to design the column length at all. It will expand or shrink based on the records it's getting. I can give you a real-life example based on our migration from on-premises to GCP. There was a dimension table with a general number of records, and when we queried that on-premises, like in Apache Spark or Teradata, it took around half an hour to get those records. In BigQuery, it was instant. As it's very fast, you can get it in two or three minutes. That was very helpful for our engineers. Usually, we have to run a query on-premises and go for a break while waiting for that query to give us the results. It's not the case with BigQuery because it instantly provides results when we run it. So, that makes the work fast, it helps a lot, and it helps save a lot of time. It also has a reasonable performance rate and smart tuning. Suppose we need to perform some joins, BigQuery has a smart tuning option, and it'll tune itself and tell us the best way a query can be done in the backend. To be frank, the performance, reliability, and everything else have improved, even the downtime. Usually, on-premise servers have some downtime, but as BigQuery is multiregional, we have storage in three different locations. So, downtime is also not getting impacted. For example, if the Atlantic ocean location has some downtime, or the server is down, we can use data that is stored in Africa or somewhere else. We have three or four storage locations, and that's the main advantage."

"The integrated data storage features are good."

"The most valuable aspect of BigQuery is its ability to handle high data workloads without causing friction with our online systems."

"What I like most about BigQuery is that it's fast and flexible. Another advantage of BigQuery is that it's easy to learn."

"The solution could make improvements around orchestration and doing some automation stuff on AWS front automation. It would be useful if we could use automation to build images and use hardened images which are CIS compliant."

"It falls short when it comes to more granular access control, such as cell-level or row-level entitlements which is a significant drawback for organizations that require precise control over who can access specific rows of data."

"Lake Formation could enhance its capabilities in audit logs, real-time monitoring, and advanced data governance."

"If I could improve AWS Lake Formation, I would add more integrations with SageMaker."

"In our experience what could be improved are not the support, performance or monitoring, but at a managerial level, the very expensive professional services of AWS. This could be an area of improvement for them. It's too expensive to acquire their support."

"You need to have data experience to use the product."

"AWS Lake Formation's pricing could be cheaper."

"For the end-users, it's not as user-friendly as it could be."

"Sometimes, support specialists might not have enough experience or business understanding, which can be an issue."

"Some of the queries are complex and difficult to understand."

"There is a limitation when copying data directly from BigQuery; it only supports up to ten MB when copying data to the clipboard."

"The initial setup could be improved making it easier to deploy."

"The main challenges are in the areas of performance and cost optimizations."

"The price could be better. Compared to competing solutions, BigQuery is expensive. It's only suitable for enterprise customers, not small and medium-sized businesses, as they cannot afford this kind of solution. In the next release, it would be better if they improved their AI bot. Although machine learning and artificial intelligence are doing wonders, there is still a lot of room to enhance them."

"We'd like to see more local data residency."

"There are some limitations in the query latency compared to what it was three years ago."

"AWS Lake Formation is a bit expensive."

"1 TB is free of cost monthly. If you use more than 1 TB a month, then you need to pay 5 dollars extra for each TB."

"The platform is inexpensive."

"One terabyte of data costs $20 to $22 per month for storage on BigQuery and $25 on Snowflake. Snowflake is costlier for one terabyte, but BigQuery charges based on how much data is inserted into the tables. BigQuery charges you based on the amount of data that you handle and not the time in which you handle it. This is why the pricing models are different and it becomes a key consideration in the decision of which platform to use."

"The solution's pricing is cheaper compared to other solutions."

"BigQuery pricing can increase quickly. It's a high-priced solution."

"The solution is pretty affordable and quite cheap in comparison to PDP or Cloudera."

"The price is a bit high but the technology is worth it."

"We are above the free threshold, so we are paying around 40 euros per month for BigQuery."