Sometimes when I'm trying to refresh the data, my different application or tool has to connect to its backend database through the connection I create. Sometimes, I face some issues like not having permissions. These aren't as crucial, but there are common errors sometimes where the database is down, or a table is nullified and a new table is added and you are not given access to that. With those errors, you don't have permissions.
The solution could use a little bit more UI. Something was in process, however, it's not yet deployed or in the version that I'm using it is not deployed. If we have to use any table or any schema in other DB features, the prompt comes over after a dot. Whenever we are using it it would really help if it looked better. If you're running a procedure, it just gives a standard error instead of the exact error captured. If you have to look at it, you need to go into the history to look into which query failed, and we need to figure it out. Instead, if the error, whatever it is, is instead displayed in the history, showing the point of failure, it would be more visible. It would save some time for all the people who are using it. There are some reporting analytics that we can use. I'm aware of those. However, if there was more reporting, we'd appreciate that. We'd like for it to become a complete one-stop-shop solution. We'd like to have some automation around small tasks, especially around scheduling.
There are some gray areas. For example, there is no clarity on where the data sits exactly. That is their proprietary information, and they are not sharing those details. Its price should be improved. On the cost-side, it is more expensive than others. If they could bring in some tools for data integration, it would be really great.
I would like to see more transparency in data processing, ATLs, and compute areas - which should give more comfort to the end-users. Improvement to the end-to-end process of loading data into Snowflake could be made as well.
The documentation could improve. They should provide architecture information. There could be better integration with tools other than the common databases used to receive data. There are other tools that have ETL tools within, such as Tableau. You are able to work with information prior to sending it to Tableau. This feature would be nice to have in a tool from Snowflake. In a future release, they should make it easier to do reporting. A drag and drop type feature would be good. If not a drag and drop feature, there should be some other easier way to do it than it is now.
One area for improvement would be the stored procedures. Currently, their stored procedures can only be executed at a transactional level versus being able to run and do updates and run things in a sequence. An additional feature I'd like to see is called materialized views, which can speed up some run times. I'd like it to be able to be used where you can have multiple tables inside them; materialized view. That would be nice. As well as being able to run cursors, to be able to do some bulk updates and some more advanced querying, table building on the fly.
There could be better ELT tools that are appropriate for Snowflake. We decided on Matillion and it seemed to be the only one. There need to be better choices, it would be great if Snowflake provided an ELT solution that people could use. Additionally, if there was a pure cloud-based ELT tool it would be useful.
There are a lot of features that they need to come up with. A lot of functions are missing in Snowflake, so we have to find a workaround for those. For example, OUTER APPLY is a basic function in SQL Server, but it is not there in Snowflake. So, you have to write complex code for it.
The UI could improve because sometimes in the security query the UI freezes. We then have to close the window and restart. There should be an IDE concept similar to the Java IDE or Eclipse feature. I should be able to see all of the functions available on a particular object. Every time we need to go to the Snowflake documentation and look if there are any methods we need. It is hard to remember everything, go and search, and use that that eventually found method. If it was possible to list out all the methods and functions available in an object that would help the developer's a lot. In an upcoming release, we should be able to send or receive data from external systems but this is not able to be done. There should be built-in logging and monitoring features, we should not need to be dependant on third-party solutions, such as Splunk. There should be more DevOps features to reduce the usage of third-party tools. If these features were part of Snowflake it would be a fully functional complete solution.
They need to improve its ETL functionality so that Snowflake becomes an ETL product. Snowpipe can do some pipelines and data ingestion, but as compare to Talend, these functionalities are limited. The ETL feature is not good enough. Therefore, Snowflake can only be used as a database. You can't use it as an ETL tool, which is a limitation. We have spoken to the vendor, and they said they are working on it, but I'm not sure when they will bring it to production. They should also make its initial setup easier. Its customization and configuration are not very straightforward. Their technical support is awesome, but it is just too expensive.
It would be better if they had a data profile tool that tells me where the gaps are in my time series data. We are anxiously waiting for them to release their data catalog and analytics capabilities, which is going to happen in June or July. If that works the way we think it might, then that would just extend our firm's capabilities into a space that we have never been interested in building ourselves. It could be a really good thing for us.
There are three things that came to my notice. I am not very sure whether they have already done it. The first one is very specific to the virtual data warehouse. Snowflake might want to offer industry-specific models for the data warehouse. Snowflake is a very strong product with credit. For a typical retail industry, such as the pharma industry, if it can get into the functional space as well, it will be a big shot in their arm. The second thing is related to the migration from other data warehouses to Snowflake. They can make the migration a little bit more seamless and easy. It should be compatible, well-structured, and well-governed. Many enterprises have huge impetus and urgency to move to Snowflake from their existing data warehouse, so, naturally, this is an area that is critical. The third thing is related to the capability of dealing with relational and dimensional structures. It is not that friendly with relational structures. Snowflake is more friendly with the dimensional structure or the data masks, which is characteristic of a Kimball model. It is very difficult to be savvy and friendly with both structures because these structures are different and address different kinds of needs. One is manipulation-heavy, and the other one is read-heavy or analysis-heavy. One is for heavy or frequent changes and amendments, and the other one is for frequent reads. One is flat, and the other one is distributed. There are fundamental differences between these two structures. If I were to consider Snowflake as a silver bullet, it should be equally savvy on both ends, which I don't think is the case. Maybe the product has grown and scaled up from where it was.
Several areas need to be improved, as follows: * Cost reduction: This is always good to have, and nobody would say no to it. * Implement multiple result windows as tabs to compare results. If we can have a feature where the results can be moved to different tabs, so that I can compare the results with earlier queries before applying the changes, it would be great. Also, it should be optional, where it can be enabled or disabled as needed. * I would like to be able to copy the results onto the clipboard and paste them directly into Excel.
I haven't found that the solution is lacking any features. It's quite complete. I would like to see a client version of the GUI. Right now, it is a web GUI, which has stored the SQL, the worksheets. We are asking for that from Snowflake. We'd like to understand how can we save these worksheets on our local desktop. That is not there at this point.
The pricing of the solution should be much easier to calculate or find by yourself. Clients want to get a rough idea as to how much it's going to cost for year one, year two, year three, et cetera. There is a calculator that Snowflake gives you, however, more often than not, you have to go to Snowflake to get those numbers. You can't do it yourself. To simplify things, it could be easier if they made it maybe easier to understand the cost and what the outgoing cost for a customer would be. In the future, if they added AI capabilities, that would be ideal.
There are always a few operation updates here and there that can be made. However, overall, there aren't any features or glaring shortcomings. It's pretty good. We can't complain. While the solution is quite inexpensive, there is always a push from clients that want it to be cheaper in the future.
Their strategy is just to leverage what you've got and put Snowflake in the middle. It does work well with other tools. You have to buy a separate reporting tool and a separate data loading tool, whereas, in some platforms, these tools are baked in. In the long-term, they'll need to add more direct partnerships to the ecosystem so that it's not like adding on tools around Snowflake to make it work. They can also consider including Snowflake native reporting tools versus partnering with other reporting tools. It would kind of change where they sit in the market.
I think that the area of improvement with Snowflake is to improve the administration. It would benefit from an administration that allows you to be aware of your credit consumption once you have the service so that you may be sure how many credits you are consuming when you use the platform and to make sure that you are making the most efficient use of these resources. In other words, to improve their interface so that you may monitor the consumption of your credits on Cloud. I also heard from a company we work for that it could be more user-friendly because it provides some tools but they are not user-friendly. Additionally, it would be very helpful if Snowflake integrated machine learning and some other advanced analytics features within their language or product capabilities. Right now, they do it through some other company where you have to buy these capabilities from other vendors. There are some customers that don't have complex needs for machine learning or advanced analytics so they don't have to buy it from another vendor but can use it from the product itself if they have it.
For the Snowflake database, there should be some third-party features for the ETL. It would also be good to be able to use some kind of controls to get the data either from another database or a flat file. Its price should be improved. It should be cheaper than Microsoft.
We've spoken with Snowflake about the fact that there are a few bare minimum requirements now these days for any data cloud, data lake, or platform. They've lacked a bit here, however, they're adopting some new measures that will be available in the next release, so that is sorted. Snowflake is partners with only AWS as a cloud platform. However, in India, Microsoft has got a big subscription. The product needs to be able to adapt to Azure a bit more in order to meet the local market demands. It's difficult to know how to size everything correctly. They should incorporate at least a basic ETL framework. It's early days, however, I would put the solution at a seven out of ten. It needs a bit more time to mature. If I were to look at it strictly from a warehousing perspective, I'd rate it at an eight out of ten.
Every product has room for improvement, although in this case, it needs some broadening of the functionality. They could become more complete. Including more AI functionality would be nice. Microsoft does have more in this regard, although that also has to do with the size of the company.
Whilst flexibility is one of its key strengths, it would be useful if there could be some better governance built into the product. In addition, functionality out of the box to deal with common data warehousing scenarios (e.g. slowly-changing dimensions etc.) would be great.
One thing which I believe Snowflake needs to improve is related to product activation queries. It's not possible to change the warehouse size while executing. It means, for example, that the query is sometimes activated on a larger surface even if I've switched to extra small, the grading in the background is running on extra large. The result is that I have to pay for extra large until the execution is complete. The cost implications can be quite large. There should be a system where they provide a prompt so you can know the current warehouse size. It would also be helpful if they would simplify the process of using Snowpipe. The way it works now is quite complicated. There are certain steps that you have to follow which is fine but if they could simplify that process, it would be helpful.
Snowflake has support for stored procedures, but it is not that powerful. They have a lot of limitations. For example, it is really basic and there are limitations on subqueries. The functions are not very good. Improving this would help to make sure data manipulation much easier. Right now, the inbuilt stored procedures and functions are all Java-based.
We've come to realize that for many customers, pricing of this solution is an issue. Maintaining Snowflake clusters is challenging and cost intensive. Reporting could also be improved. Any data that moves out of Snowflake is being cached. If I have 400 to 500 end users, with 100 or 200 reports on a daily basis, all the reports will be cached. It's a matter of ensuring that costs can be optimized. The combination of Red Warescape plus Snowflake is a combination from the design and development perspective. But the combination from the reporting perspective to micro strategy on top of Snowflake could be a better feature, so there's a combination that has to be considered.
Right now, Snowflake doesn't have any analytical functions, especially in comparison to Oracle and other databases. The analytical performance needs to improve. It would be ideal if Snowflake was able to use the analytical functions, and what we have in the relational database. That would be really helpful. They don't have any SLAs in place. It would be better if they did.
The user interface continues to be an issue, especially when we need to get data out of Snowflake. It's very easy to get data in, but it's not too easy to get it out or extract it. It would be nice to have some built-in solutions that would solve. for example, how to delete data from a customer when they request it. There needs to be stronger data protection.
It is difficult in some cases to perform ETL and this is something that should be included. As it is now, I use Informatica PowerCenter to load data from on-premises to the Snowflake cloud-based data warehouse. If this could be done by Snowflake directly, without an external integration tool, then it would become a full package. It would be awesome. They do have a native connector to connect with integration tools for loading data, but it would be much better to have the functionality built-in. We would like to be able to just write an SQL query and do our work.
The recovery process is very simple and very user-friendly as well. The constraint files could be improved. But the most important things are there. Also, the interface is good to work with and user-friendly. If I had to say what needs improvement, in my experience is that the availability is a problem. Availability should be high and has to be enhanced. I still have not explored all of the existing features. Because I have not been deeply involved with using the program it is hard to say what else is missing or what I would need.
The company is constantly working to improve the product. Now they have a focus on data sharing, which is really great. We already share data with others who do not have Snowflake. That alone is already great. But if the other counterparts also have Snowflake, then it is extremely easy to share data. You can control access at low levels and even on the cell level. It is very secure. With the improvements they continue to make, there is nothing now that I would say I miss or features that need to be added. Maybe there could be some more connectors to other systems, but this is what they are constantly developing anyway.
Right now, we need to write code, but if they could create a version of Snowflake that was more drag and drop for those managers that don't know how to code, it would be great for our business. The solution could improve the user interface and add functionality to the system.
I would like to have a tool where you can easily see the price because they need clarity of pricing. Support needs improvement, as it can take several days before you get some initial support. Integration could be the key to provide an optimal solution. In the near future, I would like to see a built-in basic analytics solution that can be embedded for testing purposes, so you can see data not only in tables but also in a graphical way in order to better prepare data for analytics. Embedded analytics would be nice.
It would be helpful if implementation could be handled more on the user-side. We need to train the users on best practices and how to use the solution properly. It's a cost issue. If they don't run it properly then it'll end up costing more money. There are some stored procedures that we've had trouble with. The solution also needs to fine-tune the connectors to be able to connect into the system source.
I work at an Insurance company with 5000+ employees and I'm interested in the collective view here.
Does anyone have a view on how Matillion ETL functionality compares to ADF for Snowflake integration, when ADF and Snowflake are hosted on Azure?
Note: Azure hosting is important, as ... Read More »
Download our free Snowflake Report and get advice and tips from experienced pros
sharing their opinions.