Hi,
It seems that Snowflake is becoming very popular nowadays. I have done some training and live sessions by Snowflake.
I would like to hear your opinion about the main key parameters we should look for when choosing Snowflake over other cloud-based data lakes, such as costs, ease of use, maintenance, etc?
And what would be success criteria?
I wrote a white paper on this that you can download here:https://bit.ly/3bRTCqp
It comes down to five factors, in my opinion:
1. Options for Deployment - Can you deploy on any cloud, or on-premises?
2. External tables and analyze in-place features - Can you leverage data outside the DB?
3. Optimizations - What are the options for slow-running queries?
4. Depth of analytics - can you do ML, time-series, geospatial, SQL, Python, etc.
5. Other data governance features - encryption and other governance features
Of course, speed and scalability is a huge factor, but most people are on top of that.
The white paper was commissioned by Vertica, a player you didn't mention above, but I attempted to keep it vendor-neutral.
@Steve Sarsfield Same for Teradata, plus In-Database processing in R & Python, SAS, Tableau and many others.
We did a PoC with Snowflake. scale up and down works as advertised. Be careful with large ETL streams. We would have to rewrite at least 20% of our ETL to make Snowflake meet or beat our current SLA's.
Snowflake is a columnar stored database. It is not a “data lake.”
The main success criteria would be "re-use" of data by SQL-driven users.
I can't speak for ALL cloud providers but on AWS the storage cost is comparable to S3. The advantage is once the data is housed in the Snowflake environment it can be re-used/re-purposed without the need of another SQL engine that has to be managed (Dremio, PrestoDB, etc.).
handling of semi-structured data like json. It has great support for json and we can write sql on json which is amazing. the performance on semi-structured data is little poor as compared to structured data but it is still great.
Hello @Anoop Anoop SLK , @NitinKumar , @Subhrajit Mitra and @Yash Mittal. Can you please share your experience?