The main challenges are in the areas of performance and cost optimizations. Achieving optimal results demands a certain level of familiarity with the platform's internals. The key point for improvement lies in the performance optimization.
The primary hurdle in this migration lies in the initial phase of moving substantial volumes of data to cloud-based platforms. This becomes even more pronounced when dealing with terabytes of data. Uploading data to cloud services requires careful consideration and optimization to ensure a smooth and efficient migration, especially when dealing with large datasets.
Data Engineer at a recreational facilities/services company with 10,001+ employees
Real User
Top 5
2023-11-02T07:54:57Z
Nov 2, 2023
In Teradata, it's very fast compared to BigQuery. The processing capability and inbuilt MPP architecture support processing millions or billions of records in a few seconds. BigQuery faces challenges in processing and retrieving the same data. So, the processing capability can be an area of improvement. Another area of improvement is in terms of the storage area, as BigQuery does support some limited types of data storage file format. In order to see the data, we need to store the data in a relational database. So, in the future, they should be capable of querying the data from the data lake. Before storing it in the RDBMS. At the moment, they don't have this feature for how my raw data looks unless you store the data in tables. Never know what sort of data. That's one thing, like, definitely they need to improve because before we model the data to explore what kind of data I'm getting in the raw stage then it's easy to, like model and process the data.
Senior Managing Consultant at Abacus Cambridge Partners
Real User
Top 10
2023-09-26T12:06:51Z
Sep 26, 2023
SQL queries remain a preferred choice for many IT database administrators, and BigQuery's ability to handle SQL queries efficiently enhances its appeal. However, there's a challenge when it comes to integrating BigQuery with homegrown database solutions, which some medium and small-sized clients rely on. While it's possible to test database integration with it using a sandbox environment, achieving seamless integration can be complex, especially for open data solutions. For greater flexibility and ease of use, it would be beneficial if BigQuery offered more third-party add-ons and connectors, particularly for databases that don't have built-in integration options.
Senior Cyber Security Architect Global ICT at a construction company with 10,001+ employees
Real User
Top 20
2023-08-16T04:14:09Z
Aug 16, 2023
As a product, BigQuery still requires a lot of maturity to accommodate other use cases and to be widely acceptable across other organizations. It's not as old as other applications like Tableau or Power BI, but as long as it's supported by Google, I think it will continue to progress.
Vice President - Data Engineering and Analytics at a financial services firm with 10,001+ employees
Real User
Top 5
2023-02-21T13:42:00Z
Feb 21, 2023
There are some limitations in the query latency compared to what it was three years ago. Despite this, BigQuery still provides the necessary functionality as compared to the other platforms. An additional feature I would like is the one available in AWS, where you have a framework to onboard past services and start building analytical models and data design. The framework makes it easier for any new organization to adopt cloud computing quickly.
An area for improvement in BigQuery is its UI because it's not working very well. Pricing for the solution is also very high. In general, though, I like the solution very much.
Team Lead Data & Analytics at a hospitality company with 501-1,000 employees
Real User
Top 10
2022-11-24T08:13:11Z
Nov 24, 2022
When it comes to queries or the code being executed in the data warehouse, the management of this code, like integration with the GitHub repository or the GitLab repository, is kind of complicated, and it's not so direct. When people are working on long queries, and so forth, they have to save them. It is a little bit clunky. The interface for saving them and version control is not really doable. We have to support the queries manually.
Program Manager at a tech services company with 201-500 employees
MSP
Top 5
2022-11-01T13:04:01Z
Nov 1, 2022
The price could be better. Compared to competing solutions, BigQuery is expensive. It's only suitable for enterprise customers, not small and medium-sized businesses, as they cannot afford this kind of solution. In the next release, it would be better if they improved their AI bot. Although machine learning and artificial intelligence are doing wonders, there is still a lot of room to enhance them.
Machine Learning Enginee at a retailer with 201-500 employees
Real User
Top 20
2022-08-22T20:41:36Z
Aug 22, 2022
Machine learning could be improved. There are some machine learning models in BigQuery; however, maybe more libraries can be provided. We'd like it extended into the Spark ML library. I noticed recently it's more expensive now. I didn't compare them to others, however, and in our team, we don't consider the price of it much.
There are many tools that you have to use with BigQuery that are different services also provided for by Google. They need to all be integrated into BigQuery to make the solution easier to use.
Data Engineer at a financial services firm with 10,001+ employees
Real User
Top 20
2022-05-09T17:53:28Z
May 9, 2022
It would be better if BigQuery didn't have huge restrictions. For example, when we migrate from on-premises to on-premise, the data which handles all ebook characters can be handled on-premise. But in BigQuery, we have huge restrictions. If we have some symbols, like a hash or other special characters, it won't accept them. Not in all cases, but it won't accept a few special characters, and when we migrate, we get errors. We need to use Regexp or something similar to replace that with another character. This isn't expected from a high-range technology like BigQuery. It has to adapt all products. For instance, if we have a TV Showroom, the TV symbol will be there in the shop name. Teradata and Apache Spark accept this, but BigQuery won't. This is the primary concern that we had. In the next release, it would be better if the query on the external table also had cache. Right now, we are using a GCS bucket, and in the native table, we have cache. For example, if we query the same table, it won't cost because it will try to fetch the records from the cached result. But when we run queries on the external table a number of times, it won't be cached. That's a major drawback of BigQuery. Only the native table has the cache option, and the external table doesn't. If there is an option to have an external table for cache purposes, it'll be a significant advantage for our organization.
BigQuery is an enterprise data warehouse that solves this problem by enabling super-fast SQL queries using the processing power of Google's infrastructure. ... You can control access to both the project and your data based on your business needs, such as giving others the ability to view or query your data.
The main challenges are in the areas of performance and cost optimizations. Achieving optimal results demands a certain level of familiarity with the platform's internals. The key point for improvement lies in the performance optimization.
Some of the queries are complex and difficult to understand.
The primary hurdle in this migration lies in the initial phase of moving substantial volumes of data to cloud-based platforms. This becomes even more pronounced when dealing with terabytes of data. Uploading data to cloud services requires careful consideration and optimization to ensure a smooth and efficient migration, especially when dealing with large datasets.
In Teradata, it's very fast compared to BigQuery. The processing capability and inbuilt MPP architecture support processing millions or billions of records in a few seconds. BigQuery faces challenges in processing and retrieving the same data. So, the processing capability can be an area of improvement. Another area of improvement is in terms of the storage area, as BigQuery does support some limited types of data storage file format. In order to see the data, we need to store the data in a relational database. So, in the future, they should be capable of querying the data from the data lake. Before storing it in the RDBMS. At the moment, they don't have this feature for how my raw data looks unless you store the data in tables. Never know what sort of data. That's one thing, like, definitely they need to improve because before we model the data to explore what kind of data I'm getting in the raw stage then it's easy to, like model and process the data.
SQL queries remain a preferred choice for many IT database administrators, and BigQuery's ability to handle SQL queries efficiently enhances its appeal. However, there's a challenge when it comes to integrating BigQuery with homegrown database solutions, which some medium and small-sized clients rely on. While it's possible to test database integration with it using a sandbox environment, achieving seamless integration can be complex, especially for open data solutions. For greater flexibility and ease of use, it would be beneficial if BigQuery offered more third-party add-ons and connectors, particularly for databases that don't have built-in integration options.
As a product, BigQuery still requires a lot of maturity to accommodate other use cases and to be widely acceptable across other organizations. It's not as old as other applications like Tableau or Power BI, but as long as it's supported by Google, I think it will continue to progress.
The solution should reduce its pricing.
There are some limitations in the query latency compared to what it was three years ago. Despite this, BigQuery still provides the necessary functionality as compared to the other platforms. An additional feature I would like is the one available in AWS, where you have a framework to onboard past services and start building analytical models and data design. The framework makes it easier for any new organization to adopt cloud computing quickly.
An area for improvement in BigQuery is its UI because it's not working very well. Pricing for the solution is also very high. In general, though, I like the solution very much.
When it comes to queries or the code being executed in the data warehouse, the management of this code, like integration with the GitHub repository or the GitLab repository, is kind of complicated, and it's not so direct. When people are working on long queries, and so forth, they have to save them. It is a little bit clunky. The interface for saving them and version control is not really doable. We have to support the queries manually.
The solution hinges on Google patterns so continued improvement is important.
The price could be better. Compared to competing solutions, BigQuery is expensive. It's only suitable for enterprise customers, not small and medium-sized businesses, as they cannot afford this kind of solution. In the next release, it would be better if they improved their AI bot. Although machine learning and artificial intelligence are doing wonders, there is still a lot of room to enhance them.
Machine learning could be improved. There are some machine learning models in BigQuery; however, maybe more libraries can be provided. We'd like it extended into the Spark ML library. I noticed recently it's more expensive now. I didn't compare them to others, however, and in our team, we don't consider the price of it much.
There are many tools that you have to use with BigQuery that are different services also provided for by Google. They need to all be integrated into BigQuery to make the solution easier to use.
It would be better if BigQuery didn't have huge restrictions. For example, when we migrate from on-premises to on-premise, the data which handles all ebook characters can be handled on-premise. But in BigQuery, we have huge restrictions. If we have some symbols, like a hash or other special characters, it won't accept them. Not in all cases, but it won't accept a few special characters, and when we migrate, we get errors. We need to use Regexp or something similar to replace that with another character. This isn't expected from a high-range technology like BigQuery. It has to adapt all products. For instance, if we have a TV Showroom, the TV symbol will be there in the shop name. Teradata and Apache Spark accept this, but BigQuery won't. This is the primary concern that we had. In the next release, it would be better if the query on the external table also had cache. Right now, we are using a GCS bucket, and in the native table, we have cache. For example, if we query the same table, it won't cost because it will try to fetch the records from the cached result. But when we run queries on the external table a number of times, it won't be cached. That's a major drawback of BigQuery. Only the native table has the cache option, and the external table doesn't. If there is an option to have an external table for cache purposes, it'll be a significant advantage for our organization.