What is our primary use case?
We are managing numerous data sources, including Excel, CSV, and ZIP files, with some of them being located on the public Internet. Our current task involves transferring this data into a data warehouse using Microsoft Azure Block Storage.
What is most valuable?
The major benefit of using Microsoft Azure Block Storage is that it supports various types of data, including structured, semi-structured, and unstructured data, without requiring different tools to read each type. This comprehensive approach simplifies the data handling process for data engineers and allows for seamless data transfer to downstream applications. In essence, this is a data paradigm shift that enables easy ingestion of diverse data types.
What needs improvement?
I encountered a challenge where certain metadata was located on a secured network that could only be accessed through a VPN. As there was no direct connector available to reach the private networks that require VPN access, we had to rely on manual effort to access the data. This could potentially be addressed as an area of improvement, especially with regard to accessibility issues where data is only accessible through a VPN connection.
It appears that the solution we are working with is already in a beta version, and we have been managing a large volume of files by pulling them in at fixed schedules to determine whether a file is new or has been modified, we need to obtain the last modified date and compare it with the previous value. However, there is a new concept of a rolling window that can automatically detect newly arrived files and push them into downstream processes. From the logs I have seen, it appears that Microsoft is actively working on simplifying the file management process and reducing the need for engineers to manually pull files using specific logic. This includes developing a feature that automatically identifies new files and distinguishes them from old ones based on their metadata. It would be helpful to have this feature implemented.
For how long have I used the solution?
I have been using Microsoft Azure Block Storage for approximately two years.
What do I think about the stability of the solution?
I did not face any glitches or stability issues.
I rate the stability of Microsoft Azure Block Storage a nine out of ten.
What do I think about the scalability of the solution?
We have more than 20 developers and 50 to 100 business users using the solution in my company.
The scalability of Microsoft Azure Block Storage has different options.
When opting for a pay-as-you-go service, several options exist, such as enabling the Autoscale feature or fixing the scalability ratio. It is essential to keep in mind that choosing the scale-out option comes with additional costs. Therefore, it is crucial to select whether to go with auto-scaling or manual scaling carefully. The system provides the flexibility to choose either option based on specific needs and budget constraints.
I rate the scalability of Microsoft Azure Block Storage an eight out of ten.
How are customer service and support?
The level of support provided by Microsoft Azure Block Storage is impressive, as they offer three different levels of assistance based on the priority level of the issue. For technical issues, they can typically provide support within one day for critical issues, two days for high-priority issues, and three or four days for normal-priority issues.
I rate the support of Microsoft Azure Block Storage a nine out of ten.
How would you rate customer service and support?
How was the initial setup?
The initial setup of Microsoft Azure Block Storage does not take long to deploy. Each service can take 10 to 15 seconds to set up.
What's my experience with pricing, setup cost, and licensing?
The price of Microsoft Azure Block Storage has a fair price. However, the price could be lower.
I rate the price of Microsoft Azure Block Storage a seven out of ten.
Which other solutions did I evaluate?
We have evaluated Amazon AWS. They provide a good service. However, Microsoft solutions have more user-friendly interfaces.
What other advice do I have?
I would advise others that this is a good solution and it is reliable. I strongly recommend this solution.
I rate Microsoft Azure Block Storage a nine out of ten.
The solution is compatible with every format, and different types of data. This is useful for data engineers. This is the best solution to have because everything is in a single point.
Which deployment model are you using for this solution?
Public Cloud
Disclosure: My company does not have a business relationship with this vendor other than being a customer.