BigQuery Reviews

This is a solution from Google that is 100% cloud-based, based on GCP. BigQuery is similar to Snowflake in the way it manages data analytics. It completely decouples storage from Compute. It has a proprietary way of storing and accessing data in its own data store and is 100% managed without you needing to install or deploy anything. There is no need to arrange for any infrastructure in order to use this solution. Go to BigQuery.com, create an account and you will get a console on your browser where you can start creating the end to end data platform - databases, data warehouses, roles, users, ETL / ELT pipelines and write transformations - all via the workspace.

Although BigQuery in completely managed on cloud, one has to use many services of BigQuery and GCP in order to create the end-to-end data setup. BigQuery acts as the core Data Warehouse mechanism, but it needs additional services like - Google Cloud Dataflow, Cloud pub/sub, Cloud BigTable, Cloud DataPrep, Cloud DataProc, Cloud SQL. Being different from the traditional way of setting up end-to-end data engineering platform, the learning curve for BigQuery is a bit steeper. Google BigQuery ecosystem can surely make the ecosystem a bit more leaner.  

I have been using this solution for 3 years. 

A very stable solution. All native abilities of Google solutions are inbuilt in BigQuery. I would predict that Snowflake and BigQuery will occupy a much larger share of the cloud data analytics space in the coming years than Azure Synapse in the future. 

This is a very scalable solution. BigQuery's pricing is more suitable for large operations that plan to scale. For smaller businesses, this may be an expensive solution. Creating a BigQuery account is free, but as soon as you start using computations and data capabilities, charges start adding up.

There is no installation involved while using BigQuery. It is as simple as opening a Gmail account and creating your own end-to-end setup. You can start creating a database schema, data bases, create pipelines with step-by-step activities ranging from ingestion to transformation to updating the data marts. Its completely managed and one does not need to worry about licenses of installations.

At e-Zest, in our projects for our enterprise customers, typically between 2 to 8 people were needed for end-to-end data platform development. This included one or two admins, 2-3 ETL developers and 2-3 data warehouse members with strong SQL and database skills.

One terabyte of data costs $20 to $22 per month for storage on BigQuery and $25 on Snowflake. Snowflake is costlier for one terabyte but only marginally. Both charge differently for compute. BigQuery charges based on how much data is inserted into the tables. Reading values from tables has no cost.

BigQuery charges you based on the amount of data that you handle and not the time in which you handle it. This is why the pricing models of Snowflake and BigQuery are different and this becomes a key consideration in the decision of which platform to use. 

We evaluated Snowflake, Azure Synapse and Amazon Redshift along with BigQuery. Snowflake and BigQuery are very similar in the way they operate. However, I would rate Snowflake slightly higher than BigQuery. I would rate Azure Synapse third and AWS Redshift fourth. The way Snowflake operates, and allows integration with other systems makes it a better alternative to BigQuery. Also Snowflake's and BigQuery's underlying architectures are quite different, although for the end user they may be appearing similar for use.

BigQuery takes a different approach to design and this is important to consider. BigQuery on its own is not enough and you need other tools also offered by Google to transform data (some of which I have mentioned in an earlier section).

The BigQuery ecosystem is a little more complex than the Snowflake ecosystem. Those who have traditionally worked on on-premise data warehouses, find Snowflake much easier to set up. Those who are trying to establish warehouses for the first time, find Google easier. 

I would rate this solution a 7 out of 10. 

We use BigQuery at our organization to access daily transactional data from our POS solutions, which are used to sell products to our clients. We gather the most essential information for our clients and upload it to our data lake using BigQuery.

We gather the most essential information for our clients and upload it to our data lake using BigQuery. At the end of the month, we have sufficient information in our data lake to generate legal reports, balances, and reconciliations with partners.

The most valuable aspect of BigQuery is its ability to handle high data workloads without causing friction with our online systems. We can obtain significant amounts of data, which is critical, even if it's not in real-time. 

Additionally, we can solve small issues while working with the platform, and it's rare that we need external support.

Sometimes, support specialists might not have enough experience or business understanding, which can be an issue. They might have basic knowledge but lack specific insights related to the specific configuration or context required by the client.

Google's customer service is good but not the best. They receive a score of eight out of ten.

Positive

Setting up BigQuery is not difficult. Although I do not directly handle this aspect, my team appears comfortable with it and does not encounter major issues requiring outside assistance.

I rate BigQuery nine out of ten. I recommend it to others and have used it in various situations over the years.

We implement for customers. We work as a global company and we have 350,000 employees, we serve clients across all industries. There are many use cases. There is no use case that we would only apply in the context of BigQuery and not with Snowflake, or not with Synapse, et cetera. It is use case agnostic.

It can be for fraud, it can be for marketing analytics, customer 360, or any kind of real-time analytics. You can use it for all sorts of stuff.

It's a stable, reliable solution. It has a good reputation for that. 

The product can scale.

As a cloud solution, it's easy to set up. 

To be very specific, here in the Middle East, I'm based out of the UAE, and Google has a very narrow footprint, a very limited footprint here in the region. There is a lack or absence of local data residency compliance. They don't have a local data center here. Therefore, most of the big organizations like banks, and companies in the highly regulated public sector, are not using BigQuery products as it means that the data will have to move out of the country. We'd like to see more local data residency.

We've been implementing this solution since the inception of these products. We are Platinum Elite partners with most vendors.

The solution has a reputation for being stable. It's not a problem. 

The solution is  scalable up to a certain extent. According to the benchmarks, they would be stronger on the one hand, however, depending on the criteria that you're using, what kind of volumes, the velocity, et cetera, it can scale.

I've never dealt directly with technical support. I can't speak to how helpful or responsive they are. 

I did not handle the initial setup. That said, solutions like BigQuery, as opposed to non-cloud, on-prem versions equivalents are generally more straightforward to set up.

How long it takes to set up depends on the requirements. Typically, it takes six months to one year for end-to-end implementation. 

We have data engineers that can handle deployments. How many are needed depends on the scope of the project. 

I don't deal with licensing aspects of the product. The licenses are always purchased by our clients. 

I'd rate the solution seven out of ten.

We use BigQuery to store data in a table and query it. Data storage can be either an internal native table or an external table where the external source will point to Google Cloud Storage or Google Drive. 

Wherever we can have external storage, we can have a table built pointing to that external storage and query the tables. In BigQuery, we can query the table or even do DML operations, like insert, delete, etc.

The main thing I like about BigQuery is storage. We did an on-premise BigQuery migration with trillions of records. Usually, we have to deal with insufficient storage on-premises, but in BigQuery, we don't get that because it's like cloud storage, and we can have any number of records. That is one advantage. 

The next major advantage is the column length. We have some limits on column length on-premises, like 10,000, and we have to design it based on that. However, with BigQuery, we don't need to design the column length at all. It will expand or shrink based on the records it's getting.

I can give you a real-life example based on our migration from on-premises to GCP. There was a dimension table with a general number of records, and when we queried that on-premises, like in Apache Spark or Teradata, it took around half an hour to get those records. In BigQuery, it was instant. As it's very fast, you can get it in two or three minutes. That was very helpful for our engineers.

Usually, we have to run a query on-premises and go for a break while waiting for that query to give us the results. It's not the case with BigQuery because it instantly provides results when we run it. So, that makes the work fast, it helps a lot, and it helps save a lot of time. 

It also has a reasonable performance rate and smart tuning. Suppose we need to perform some joins, BigQuery has a smart tuning option, and it'll tune itself and tell us the best way a query can be done in the backend. 

To be frank, the performance, reliability, and everything else have improved, even the downtime. Usually, on-premise servers have some downtime, but as BigQuery is multiregional, we have storage in three different locations. So, downtime is also not getting impacted.

For example, if the Atlantic ocean location has some downtime, or the server is down, we can use data that is stored in Africa or somewhere else. We have three or four storage locations, and that's the main advantage.

It would be better if BigQuery didn't have huge restrictions. For example, when we migrate from on-premises to on-premise, the data which handles all ebook characters can be handled on-premise. But in BigQuery, we have huge restrictions. If we have some symbols, like a hash or other special characters, it won't accept them. Not in all cases, but it won't accept a few special characters, and when we migrate, we get errors. 

We need to use Regexp or something similar to replace that with another character. This isn't expected from a high-range technology like BigQuery. It has to adapt all products. For instance, if we have a TV Showroom, the TV symbol will be there in the shop name. Teradata and Apache Spark accept this, but BigQuery won't. This is the primary concern that we had.

In the next release, it would be better if the query on the external table also had cache. Right now, we are using a GCS bucket, and in the native table, we have cache. For example, if we query the same table, it won't cost because it will try to fetch the records from the cached result. But when we run queries on the external table a number of times, it won't be cached. That's a major drawback of BigQuery. Only the native table has the cache option, and the external table doesn't. If there is an option to have an external table for cache purposes, it'll be a significant advantage for our organization.

I have been using BigQuery for more than three years.

BigQuery is a stable solution.

BigQuery is highly scalable. We can have unlimited storage if we do 20 records, and It's very fast. Even if we scale it to 20 trillion, it will still be fast. 

In my organization, about two in five use BigQuery. When I joined the company a year back, usage was relatively moderate. However, now usage increased because of the on-premise to GCP migration. Because of many successful projects, several people are using BigQuery now.

We have dedicated support people who help us with the framework. If there is a technical issue in BigQuery, we just get help from the technical team. But if there are any engineering issues or some data issues, our team will handle them.

I use Teradata and then Apache Spark on-premises.

The initial setup is relatively straightforward. There are some restrictions, like the project's name. It has to be unique, but once that project is created, we can simply go to an option, query, and the query control will open, and we can start creating a table, loading data, querying, and everything. So that's quite simple and straightforward.

When I joined PayPal, the setup was done in-house. When I worked at another organization, Cognizant, we had Google's help. So a Google specialist helped us set up and everything.

I have tried my own setup using my Gmail ID, and I think it had a $300 limit for free for a new user. That's what Google is offering, and we can register and create a project.

On a scale from one to ten, I would give BigQuery an eight.

This is a cloud-based data warehouse. 

The product is updated automatically without people having to worry about doing anything. It is managed completely by Google. 

The performance is good. It's very user-friendly for people not coming from the technical area. 

It has a very friendly user interface and a console for command line. 

It has a well-structured suite of complimentary tools for data integration and so forth.

When it comes to queries or the code being executed in the data warehouse, the management of this code, like integration with the GitHub repository or the GitLab repository, is kind of complicated, and it's not so direct. When people are working on long queries, and so forth, they have to save them. It is a little bit clunky. The interface for saving them and version control is not really doable. We have to support the queries manually.

I've used the solution across different companies. I've used it for about six or seven years. 

In my previous company, we were not spending that much. You give more money away to the other tools from GCP. We paid maybe €200 or something like that and no more than that. This year, we pay €170 a month.

We are an end-user.

The product is a software as a service, and therefore, we are always on the latest version. They do everything for us. 

I'd rate the product eight out of ten as it's a very good data warehouse, and it's very easy to learn how to use it. It's very user-friendly. I can have my team handle it, even if they are non-technical and they can be doing a lot of coding there without problems. 

I like that we can synch and run a large query. I also like that we can work with a large amount of data. You don't need to work separately, as it's a ready-made solution. It also comes with a built-in machine-learning feature. Once we start inputting the data, it will suggest some things related to the data, and we can come up with nice dashboards and statistics from a vast amount of data.

The price could be better. Compared to competing solutions, BigQuery is expensive. It's only suitable for enterprise customers, not small and medium-sized businesses, as they cannot afford this kind of solution. 

In the next release, it would be better if they improved their AI bot. Although machine learning and artificial intelligence are doing wonders, there is still a lot of room to enhance them.

I have been working with BigQuery for two and a half years.

BigQuery is a stable solution.

BigQuery is a scalable solution. At present, we have about five different users using this solution. But BigQuery is handling the data of 3,000,000 customers.

We subscribed to technical support from Google. Whenever my team finds an issue, they contact support. I did not get a chance to contact the support team because we never had any difficulties or glitches while configuring it.

The initial setup is relatively straightforward. It's not simple, and it's not very complex. We are doing maintenance of our regular cloud services and working with some assistants and microservice architecture. I don't think we have ever set up in less than one day.

We implemented this solution.

The price could be better. Usually, you need to buy the license for a year. Whenever you want more, you can subscribe to it, and you can use it. Otherwise, you can terminate the license. You can use it daily or monthly, and we use it based on a project's requirements.

On a scale from one to ten, I would give BigQuery a nine.

Our company uses the solution as a data warehouse for implementing machine learning use cases and queries. 

The query tool is scalable and allows for petabytes of data. 

The NoSQL model and feeds for machine learning are based on the support of competent technologies. 

The solution includes plenty of additional features. 

The solution hinges on Google patterns so continued improvement is important. 

I have been using the solution for two years. 

The solution is stable. 

The solution is scalable and we have 200 users with no issues. 

Google has one technical support channel for all products and services. If you place a support ticket, they will respond to you in order of priority. 

There is no setup because the solution resides in the cloud. Once you enable the APIs in the Google Cloud ecosystem, you can start consuming right away. 

The price is a bit high but the technology is worth it. If you do not use the solution in the right way, it will be expensive.

There is not an equivalent competitor product because the solution is Google's proprietary technology. 

If you are interested in a NoSQL option, definitely try the solution. 

I rate the solution a ten out of ten. 

BigQuery is a PaaS solution. There's only one version available on Google Cloud. Because it's deployed on cloud, it will update automatically.

If I'm collaborating with Google Data Cloud, I can use the cache, and I don't have to pay again and again. There are some performance features like partitioning, which you can do based on an integer, and it improves the performance a lot. There's also the Array function. You can also enable Spark on BigQuery, which is actually faster than any other Spark. If you use Dataproc, Spark on BigQuery is much faster.

Spark will actually eliminate the usage of a lot of Adobe legacy things. It will act as a Spark SQL.

It is not that cost-friendly, but it is very performance-friendly. There are also machine learning features.

For example, if I have a query, and I have done everything to improve it, the query will still take 15 minutes. With other columnar databases like Snowflake, you can actually increase your VM size or increase your machine size, and you can buy more memory and it will start working faster, but that's not available in BigQuery. You have to actually open a ticket and then follow it up with Google support.

I have been using this solution for two and a half years.

BigQuery is very stable. It is getting used a lot.

It is definitely scalable. You do not have to do any configurations. It will be able to handle petabytes of data.

Technical support is excellent. It is Google, and they always provide the best. We haven't needed to contact Google for BigQuery specifically, but I have contacted Google support for other things and they were pretty responsive.

I have experience with Snowflake.

I was working on a project where we were building systems and loading the data manually. Once we moved to BigQuery, we saw ROI in terms of cost savings. We saw 27% performance improvement in most of our queries. Our total costs were reduced by about 17%. In terms of cost and time, we were able to save effort.

There was some learning and training involved, which lasted six months, so we saw the real ROI after a year.

I would rate this solution 8 out of 10.

My advice is to first identify your use case. If you have Google Cloud then you have two databases to compare, BigQuery and Snowflake. BigQuery is typically used to analyze petabytes of data. If you're looking for transitional query, then you should have a different system. BigQuery cannot handle unstructured data, so that is one thing you have to think about. 

In terms of latency, if you want single-digit millisecond latency then BigQuery is not good. It is very fast, but if you want single-digit millisecond latency, then you probably have to go to a no-SQL database solution.

My suggestion is to analyze your use case and then map it with the BigQuery features.