BigQuery Reviews

We implement for customers. We work as a global company and we have 350,000 employees, we serve clients across all industries. There are many use cases. There is no use case that we would only apply in the context of BigQuery and not with Snowflake, or not with Synapse, et cetera. It is use case agnostic.

It can be for fraud, it can be for marketing analytics, customer 360, or any kind of real-time analytics. You can use it for all sorts of stuff.

It's a stable, reliable solution. It has a good reputation for that. 

The product can scale.

As a cloud solution, it's easy to set up. 

To be very specific, here in the Middle East, I'm based out of the UAE, and Google has a very narrow footprint, a very limited footprint here in the region. There is a lack or absence of local data residency compliance. They don't have a local data center here. Therefore, most of the big organizations like banks, and companies in the highly regulated public sector, are not using BigQuery products as it means that the data will have to move out of the country. We'd like to see more local data residency.

We've been implementing this solution since the inception of these products. We are Platinum Elite partners with most vendors.

The solution has a reputation for being stable. It's not a problem. 

The solution is  scalable up to a certain extent. According to the benchmarks, they would be stronger on the one hand, however, depending on the criteria that you're using, what kind of volumes, the velocity, et cetera, it can scale.

I've never dealt directly with technical support. I can't speak to how helpful or responsive they are. 

I did not handle the initial setup. That said, solutions like BigQuery, as opposed to non-cloud, on-prem versions equivalents are generally more straightforward to set up.

How long it takes to set up depends on the requirements. Typically, it takes six months to one year for end-to-end implementation. 

We have data engineers that can handle deployments. How many are needed depends on the scope of the project. 

I don't deal with licensing aspects of the product. The licenses are always purchased by our clients. 

I'd rate the solution seven out of ten.

We use BigQuery to store data in a table and query it. Data storage can be either an internal native table or an external table where the external source will point to Google Cloud Storage or Google Drive. 

Wherever we can have external storage, we can have a table built pointing to that external storage and query the tables. In BigQuery, we can query the table or even do DML operations, like insert, delete, etc.

The main thing I like about BigQuery is storage. We did an on-premise BigQuery migration with trillions of records. Usually, we have to deal with insufficient storage on-premises, but in BigQuery, we don't get that because it's like cloud storage, and we can have any number of records. That is one advantage. 

The next major advantage is the column length. We have some limits on column length on-premises, like 10,000, and we have to design it based on that. However, with BigQuery, we don't need to design the column length at all. It will expand or shrink based on the records it's getting.

I can give you a real-life example based on our migration from on-premises to GCP. There was a dimension table with a general number of records, and when we queried that on-premises, like in Apache Spark or Teradata, it took around half an hour to get those records. In BigQuery, it was instant. As it's very fast, you can get it in two or three minutes. That was very helpful for our engineers.

Usually, we have to run a query on-premises and go for a break while waiting for that query to give us the results. It's not the case with BigQuery because it instantly provides results when we run it. So, that makes the work fast, it helps a lot, and it helps save a lot of time. 

It also has a reasonable performance rate and smart tuning. Suppose we need to perform some joins, BigQuery has a smart tuning option, and it'll tune itself and tell us the best way a query can be done in the backend. 

To be frank, the performance, reliability, and everything else have improved, even the downtime. Usually, on-premise servers have some downtime, but as BigQuery is multiregional, we have storage in three different locations. So, downtime is also not getting impacted.

For example, if the Atlantic ocean location has some downtime, or the server is down, we can use data that is stored in Africa or somewhere else. We have three or four storage locations, and that's the main advantage.

It would be better if BigQuery didn't have huge restrictions. For example, when we migrate from on-premises to on-premise, the data which handles all ebook characters can be handled on-premise. But in BigQuery, we have huge restrictions. If we have some symbols, like a hash or other special characters, it won't accept them. Not in all cases, but it won't accept a few special characters, and when we migrate, we get errors. 

We need to use Regexp or something similar to replace that with another character. This isn't expected from a high-range technology like BigQuery. It has to adapt all products. For instance, if we have a TV Showroom, the TV symbol will be there in the shop name. Teradata and Apache Spark accept this, but BigQuery won't. This is the primary concern that we had.

In the next release, it would be better if the query on the external table also had cache. Right now, we are using a GCS bucket, and in the native table, we have cache. For example, if we query the same table, it won't cost because it will try to fetch the records from the cached result. But when we run queries on the external table a number of times, it won't be cached. That's a major drawback of BigQuery. Only the native table has the cache option, and the external table doesn't. If there is an option to have an external table for cache purposes, it'll be a significant advantage for our organization.

I have been using BigQuery for more than three years.

BigQuery is a stable solution.

BigQuery is highly scalable. We can have unlimited storage if we do 20 records, and It's very fast. Even if we scale it to 20 trillion, it will still be fast. 

In my organization, about two in five use BigQuery. When I joined the company a year back, usage was relatively moderate. However, now usage increased because of the on-premise to GCP migration. Because of many successful projects, several people are using BigQuery now.

We have dedicated support people who help us with the framework. If there is a technical issue in BigQuery, we just get help from the technical team. But if there are any engineering issues or some data issues, our team will handle them.

I use Teradata and then Apache Spark on-premises.

The initial setup is relatively straightforward. There are some restrictions, like the project's name. It has to be unique, but once that project is created, we can simply go to an option, query, and the query control will open, and we can start creating a table, loading data, querying, and everything. So that's quite simple and straightforward.

When I joined PayPal, the setup was done in-house. When I worked at another organization, Cognizant, we had Google's help. So a Google specialist helped us set up and everything.

I have tried my own setup using my Gmail ID, and I think it had a $300 limit for free for a new user. That's what Google is offering, and we can register and create a project.

On a scale from one to ten, I would give BigQuery an eight.

BigQuery is a powerful tool for managing and analyzing large datasets. The versatility of BigQuery extends to its compatibility with external data visualization tools like Power BI and Tableau. This means you not only get query results but can also seamlessly integrate and visualize your data for better insights.

The product's most valuable feature is its ability to connect to visualization tools. 

Some of the queries are complex and difficult to understand. 

I have been using the product for more than a year. 

My company has 100 users for BigQuery. 

The tool's support is fast to respond. 

Positive

The tool's deployment is easy if you follow Google's documentation. 

If you have a big database and lots of analytics, BigQuery is a really good tool. It helps save and manage your queries and gives you results you can show clients and others. I rate it a nine out of ten. 

In our workflow, we initiate the process by fetching data, followed by a preprocessing step to refine the data. We establish pipelines for seamless data flow. The ultimate objective is to transfer this processed data into BigQuery tables, enabling other teams, such as analytics or machine learning, to easily interpret and utilize the information for various purposes, whether it's gaining insights or developing models.

The primary advantages include its speed, especially when dealing with large datasets or big data. It proves exceptionally useful in handling substantial amounts of data efficiently. A notable benefit is the ability to preview data without executing full queries, saving time and allowing for quick insights. This feature eliminates the need to run extensive queries solely for data preview purposes, streamlining the overall workflow.

The interface is what I find particularly valuable. When crafting queries, it offers estimations on data usage, providing a helpful indication of resource consumption. This predictive capability adds an extra layer of convenience, making the querying process more insightful and efficient.

It would be beneficial to integrate additional tools, particularly from a business intelligence perspective. For instance, incorporating machine learning capabilities could enable users to automatically generate SQL queries.

I have been working with it for over a year now.

I find it to be generally high and satisfactory. However, there is a notable issue we've encountered regarding query limitations at the organization level.

It is scalable up to a certain point. There seems to be a restriction on the number of queries one can run, for example, being limited to processing ten terabytes of queries. Exceeding this limit results in an inability to run additional queries, posing a potential challenge. Resolving this limitation could contribute to a smoother user experience. Currently, the user base exceeds two hundred individuals.

We used Google Cloud Storage, IAM, AWS (specifically VPC), and instances from both AWS and Google Cloud Platform. Regarding comparison with other solutions, particularly AWS, there are notable observations. AWS, being introduced earlier, appears to have more extensive features compared to Google Cloud Platform (GCP). AWS enjoys the advantage of having a more established history, resulting in robust support from their team. It offers a more comprehensive platform with a broader range of features, and its pricing structure appears to be more favorable.

The challenging part lies in the initial setup of the project, especially when integrating with project management tools. When establishing a project on the Google Cloud Platform, you need to navigate through various resources.

Setting up the account, whether at an individual or organizational level, involves providing necessary information, including credit card details for billing purposes. Once the account is set up, accessing resources like Cloud Storage or BigQuery becomes straightforward within the Google Cloud Platform.

For those venturing into cloud platforms, especially at an individual level, I would recommend considering AWS. Given its longer establishment in the industry, many companies utilize AWS. Additionally, both AWS and GCP offer free tiers for new users, but AWS extends this benefit to one year, while GCP limits it to three months. At the organizational level, AWS tends to provide more extensive features compared to GCP, making it a preferable choice. Overall, I would rate it eight out of ten.

This is a cloud-based data warehouse. 

The product is updated automatically without people having to worry about doing anything. It is managed completely by Google. 

The performance is good. It's very user-friendly for people not coming from the technical area. 

It has a very friendly user interface and a console for command line. 

It has a well-structured suite of complimentary tools for data integration and so forth.

When it comes to queries or the code being executed in the data warehouse, the management of this code, like integration with the GitHub repository or the GitLab repository, is kind of complicated, and it's not so direct. When people are working on long queries, and so forth, they have to save them. It is a little bit clunky. The interface for saving them and version control is not really doable. We have to support the queries manually.

I've used the solution across different companies. I've used it for about six or seven years. 

In my previous company, we were not spending that much. You give more money away to the other tools from GCP. We paid maybe €200 or something like that and no more than that. This year, we pay €170 a month.

We are an end-user.

The product is a software as a service, and therefore, we are always on the latest version. They do everything for us. 

I'd rate the product eight out of ten as it's a very good data warehouse, and it's very easy to learn how to use it. It's very user-friendly. I can have my team handle it, even if they are non-technical and they can be doing a lot of coding there without problems. 

BigQuery is a PaaS solution. There's only one version available on Google Cloud. Because it's deployed on cloud, it will update automatically.

If I'm collaborating with Google Data Cloud, I can use the cache, and I don't have to pay again and again. There are some performance features like partitioning, which you can do based on an integer, and it improves the performance a lot. There's also the Array function. You can also enable Spark on BigQuery, which is actually faster than any other Spark. If you use Dataproc, Spark on BigQuery is much faster.

Spark will actually eliminate the usage of a lot of Adobe legacy things. It will act as a Spark SQL.

It is not that cost-friendly, but it is very performance-friendly. There are also machine learning features.

For example, if I have a query, and I have done everything to improve it, the query will still take 15 minutes. With other columnar databases like Snowflake, you can actually increase your VM size or increase your machine size, and you can buy more memory and it will start working faster, but that's not available in BigQuery. You have to actually open a ticket and then follow it up with Google support.

I have been using this solution for two and a half years.

BigQuery is very stable. It is getting used a lot.

It is definitely scalable. You do not have to do any configurations. It will be able to handle petabytes of data.

Technical support is excellent. It is Google, and they always provide the best. We haven't needed to contact Google for BigQuery specifically, but I have contacted Google support for other things and they were pretty responsive.

I have experience with Snowflake.

I was working on a project where we were building systems and loading the data manually. Once we moved to BigQuery, we saw ROI in terms of cost savings. We saw 27% performance improvement in most of our queries. Our total costs were reduced by about 17%. In terms of cost and time, we were able to save effort.

There was some learning and training involved, which lasted six months, so we saw the real ROI after a year.

I would rate this solution 8 out of 10.

My advice is to first identify your use case. If you have Google Cloud then you have two databases to compare, BigQuery and Snowflake. BigQuery is typically used to analyze petabytes of data. If you're looking for transitional query, then you should have a different system. BigQuery cannot handle unstructured data, so that is one thing you have to think about. 

In terms of latency, if you want single-digit millisecond latency then BigQuery is not good. It is very fast, but if you want single-digit millisecond latency, then you probably have to go to a no-SQL database solution.

My suggestion is to analyze your use case and then map it with the BigQuery features.

We use BigQuery at our organization to access daily transactional data from our POS solutions, which are used to sell products to our clients. We gather the most essential information for our clients and upload it to our data lake using BigQuery.

We gather the most essential information for our clients and upload it to our data lake using BigQuery. At the end of the month, we have sufficient information in our data lake to generate legal reports, balances, and reconciliations with partners.

The most valuable aspect of BigQuery is its ability to handle high data workloads without causing friction with our online systems. We can obtain significant amounts of data, which is critical, even if it's not in real-time. 

Additionally, we can solve small issues while working with the platform, and it's rare that we need external support.

Sometimes, support specialists might not have enough experience or business understanding, which can be an issue. They might have basic knowledge but lack specific insights related to the specific configuration or context required by the client.

Google's customer service is good but not the best. They receive a score of eight out of ten.

Positive

Setting up BigQuery is not difficult. Although I do not directly handle this aspect, my team appears comfortable with it and does not encounter major issues requiring outside assistance.

I rate BigQuery nine out of ten. I recommend it to others and have used it in various situations over the years.

We are into conversational commerce platforms. All the conversations and the chat history are captured when we chat with a chatbot. Our application is built on NoSQL. We put the data into BigQuery as a data warehouse, where we refine the data. We analyze the chat history and give analytic reports to our merchants using our SaaS platform. It is to understand the chat conversation, how many people had a conversation, and what key buttons they clicked.

We also provide analytics on how many orders were completed. We are building a commerce and conversational dashboard for our enterprise customers and offering them on Looker. Looker was earlier known as Google Data Studio. For applications, we segment customers and use the customer segments to broadcast messages across social channels. All these things are being queried over BigQuery to do segmentations.

On the front end, we give them the option of segmenting based on different data attributes. Then, it goes to BigQuery to filter out the data and find the number of customers who meet the defined conditions. Based on that, we send the messages to the segmented customers. We are doing multiple things related to conversation commerce using BigQuery.

It is a cloud platform. We just need to query and get the output. Anyone can use the product. Even non-coders can review the data in BigQuery.

There should be an easier way to migrate from NoSQL to SQL. The process of migrating from Datastore to BigQuery should be improved. We use Datastore and BigQuery. If both products can be synced well, it will improve employee productivity. 

We had to write a lot of pipelines and logic for real-time streaming from Datastore, which is a NoSQL, to BigQuery, which is more of a structured database. However, because both products are internal to the Google Cloud Platform, they should have some provision to create and keep syncing it automatically. It will be an advantage for the customers. Currently, we build replicas. It would be easier if some simple connection replicates the changes in BigQuery.

My company has been using the solution for five years. I have been using it for a year.

I rate the product’s stability a nine out of ten.

The solution is more scalable because it is in the cloud. It is an advantage. I rate the scalability of the tool an eight out of ten. If we are integrating it with two different platforms, then it becomes a little difficult for us. If there is a data pipeline error, we cannot scale immediately. If we have to integrate NoSQL with BigQuery, it sometimes becomes a challenge for real-time streaming.

Five developers within my team are building all the logic on BigQuery. We have around 100 to 200 customers with five to six employees each using our platform. When they use our platform and query using different features, these queries hit BigQuery, and we render the data. We are the designers designing using BigQuery, and the end users use the UI.

I would rate technical support a little less. We have always struggled to get quicker support.

The initial setup is very simple. The solution is cloud-based.

The tool has competitive pricing. I rate the pricing an eight out of ten.

I have a technical team that works deeply into it and gives me the output. I don't extensively use BigQuery as a developer to develop things. Overall, I rate the solution an eight out of ten.