What is our primary use case?
Azure OpenAI's main use case for me involves defining solutions for incident remediation where AI provides intelligence to solve problems, perform root cause analysis, or triage incidents or changes. I connect the knowledge base to Azure OpenAI, which performs intelligent analysis. I have defined several use cases along these lines.
In terms of Azure OpenAI generating human-like text content for my tasks, I have defined one use case for a doctor's app where patient voice recordings get translated to English, then transcribed, and the system generates an overall summary for doctors, which is saved in the database. It performs well, and I have tried multiple LLMs and found it effective.
What is most valuable?
The functionality in Azure OpenAI that I found most valuable is the simplicity of selecting any model and its superior intelligence compared to local LLMs.
Regarding language translation in Azure OpenAI, I am satisfied with this function because it performs well.
Concerning sentiment analysis in Azure OpenAI, I have defined using Azure OpenAI for sentiment analysis for airports, including defining one point of view and a demo for it, and it performs well.
What needs improvement?
The customization option in Azure OpenAI is quite challenging because any customization must be done through the knowledge base since Azure OpenAI models cannot be trained. I must build a knowledge base and feed it so that it will learn from that knowledge base. This differs from other local LLMs that I can train directly.
For integration of Azure OpenAI with other Azure services, I would rate it five out of ten because it is an open Azure product and integrations work well with Azure services. However, when it comes to services outside of Azure, integration is quite difficult and requires more exploration. It is not as convenient.
The point for improvement is integration with third-party services, which has a gap that needs addressing.
Regarding other points for improvement for Azure OpenAI, Azure OpenAI is performing well overall, but I believe their models should offer local dedicated models for customers. All data sent to the current models goes to public models. Azure OpenAI should provide solutions to deliver local dedicated models for customers and should enable model training based on customer data. Customers are mostly concerned about their data, so this option is not feasible as currently structured. Even if it were dedicated for the customer and not used by others, it still does not align with compliance requirements because it remains an open model.
What do I think about the stability of the solution?
I would rate the stability of Azure OpenAI at eight out of ten.
What do I think about the scalability of the solution?
In terms of scalability, I would rate it nine for technical ability to expand. However, from a cost perspective, I would rate it five because it is too costly.
How was the initial setup?
The initial setup for Azure OpenAI is simple.
Which other solutions did I evaluate?
In my opinion, the main competitors for Azure OpenAI are AWS and Google. AWS performs adequately, but configuration is somewhat complex when it comes to Lambda and using the LLM. Google is performing well in terms of competition with Gemini and Google ADK, which functions effectively. I can generate agents and use the models quite well. I think Google is a key competitor, more so than AWS.
What other advice do I have?
Regarding pricing, I would rate it five because it is too costly for any customer. The main intent for customers is to reduce costs by adding intelligence, where tasks are reduced and efforts are minimized. However, I am introducing more cost for this, so my cost reduction is not materializing as expected.
Regarding the purchase process, I usually prefer to have the costing directed to customers only and not processed on the client side.
I would recommend Azure OpenAI to other users, provided they are comfortable with the cost. If customers are not ready for the cost, I have solutions using open source alternatives like Mistral AI, and there are many other models available. On openrouter.com, I can find many models to use. However, I do need to perform some training with those models compared to Azure OpenAI.
My overall rating for Azure OpenAI is eight out of ten.
Disclosure: My company does not have a business relationship with this vendor other than being a customer.