Learn what your peers think about Microsoft Azure Machine Learning Studio. Get advice and tips from experienced pros sharing their opinions. Updated: May 2023.
Hi experts,
I work at a large financial organization.
I have been wondering what types of problems (=use cases) can be solved/optimized using the Edge AI technology in industries such as Banking and Finance, Power and Land and Agriculture?
I appreciate your thoughts and suggestions.
AI has already been used for decades in banking, finance and agriculture.
Credit and loan analysis use a lot of logistic regression to know who to loan money to. Finance uses AI for regression analysis of forecasts and optimum stock portfolios, agriculture to predict production and supplies needs based on the amount of rain forecasted.
What we have is the evolution of tools, methods and techniques. We no longer need cubes. Although Hadoop is meant for safe storage at low cost and not for AI purposes, tools have evolved and new tools arose providing the speed required for AI analysis.
We have evolved in terms of data governance with the use of DMBoK (Data Management Body of Knowledge), visual planning frameworks like the Machine Learning Canvas, Self Service BI approaches stimulating collaboration amongst users. We now have tools, practices, frameworks and enough manpower that makes the job more challenging.
We have been using AI in those sectors for quite a while, but now we have the momentum to reach a different level of data thinking organizations.
Regional Manager/ Service Delivery Manager at a tech services company with 201-500 employees
Nov 26, 2021
Hi @Evgeny Belenky ,
HERE ARE THE STORAGE REQUIREMENTS FOR DEEP LEARNING
Deep learning workloads are a special kind of beast: all DL data is considered hot data, which raises the dilemma of not being able to employ any sort of tiered storage management solution. This is because normal SSDs usually used for hot data under conventional conditions simply won’t move the data required for millions, billions, or even trillions of metadata transfers for an ML training model to classify an unknown something out of only a limited amount of examples.
Below are a few examples of a few storage requirements needed to avoid the dreaded curse of dimensionality.
COST EFFICIENCY
Enormous AI data sets become an even bigger burden if they don’t fall within the budget set aside for storage. Anyone who has been in charge of managing enterprise data for any amount of time knows well that highly-scalable systems have always been more high-priced on a capacity versus cost basis. The ultimate deep learning storage system must be both affordable and scalable to make sense.
PARALLEL ARCHITECTURE
In order to avoid those dreaded choke points that stunt a deep learning machine’s ability to learn, it’s essential for data sets t to have parallel-access architecture.
DATA LOCALITY
While it might be possible that many organizations may opt to keep some of their data on the cloud, most of it should remain on-site in a data center. There are at least three reasons for this: regulatory compliance, cost efficiency, and performance. For this reason, on-site storage must rival the cost of keeping it on the cloud.
HYBRID ARCHITECTURE
As touched on above, different types of data have unique performance requirements. Thus, storage solutions should offer the perfect mixture of storage technologies instead of an asymmetrical strategy that will eventually fail. It’s all about simultaneously meeting ML storage performance and scalability.
SOFTWARE-DEFINED STORAGE
Not all huge data sets are the same—especially in terms of DL and ML. While some of them can get by with the simplicity of pre-configured machines, others need hyper-scale data centers featuring purpose-built servers architectures that are previously set in place. This is what makes software-defined storage solutions the best option.
Our X-AI Accelerated is an any–scale DL and ML solution that offers unmatched versatility for any organization’s needs. X-AI Accelerated was engineered from the ground up and optimized for “ingest, training, data transformations, replication, metadata, and small data transfers.” Not only that but RAID Inc. offers all the aforementioned requirements such as all-flash NVMe X2-AI/X4-AI or the X5-AI, which are hybrid flash and hard drive storage platforms.
Both the NVMe X2-AI/X4-AI and the X5-AI support parallel access to flash and deeply expandable HDD storage as well. Furthermore, the X-AI Accelerated storage platform permits one to scale out from only a few TBs to tens of PBs.
Hi dear community members,
This is our latest community digest. It helps you catch up on recent contributions by community members. Comment below with your feedback and suggestions!
Trending
What are the Top 5 cybersecurity trends in 2022?
What are the main benefits of modern IT Asset Discovery tools?
Tip
Post an educational article from your Home feed and receive 20 point...
Hi community members,
Spotlight #2 is our fresh bi-weekly community digest for you. It covers cybersecurity, IT and DevOps topics. Check it out and comment below with your feedback!
Trending
What are the pros and cons of internal SOC vs SOC-as-a-Service?
Join The Moderator Team at IT Central Station (soon to be PeerSpot)!
Questions
Share your experience with other peers by ans...
Download our free Microsoft Azure Machine Learning Studio Report and get advice and tips from experienced pros
sharing their opinions. Updated: May 2023.
Hi,
Believe the below are the top emerging trends looking at the present market outlook.