What is our primary use case?
I deal mostly with tools such as Meta Llama for embeddings, and I was primarily using it for search, but now I use OpenSearch instead of Azure search, bringing some small language models and offline language models rather than cloud-based solutions.
I am working with Meta Llama and have found the features especially valuable in the embeddings, as I use it along with OpenSearch for embeddings and building the RAG. We focus on RAG agents rather than AI agents, which require more offline RAG capabilities, and that's where I build them.
I assess the benefits of Meta Llama's automation processes as substantial. First, I will make my customers happy without tokens because today, the cloud itself is unnecessarily expensive, and on top of that, tokens and security for public data add additional concerns. I ensure that we don't compromise on these three factors, especially from the compliance perspective.
The metrics I use to measure the precision of insights from Meta Llama's capabilities are based on dropouts. A majority of my measurement involves assessing how much customers understand when we give some insights, and whether they directly click to create a quote, directly click to order a product, or directly move towards a solution rather than saying they would like to talk to a human. Especially in the CRM world, the more they interact with the CRM, the more I fail in AI.
I have been a user, implementer, designer, and CTO and CIO for Meta Llama.
What is most valuable?
I do not make Meta Llama's data integration features public. I use Meta Llama for the RAG and embeddings purposefully while supporting the RAG and embeddings separately; otherwise, integration comes in separately, and I do not use it. By the way, I am bringing it offline and it is not cloud-based.
Meta Llama's customizable dashboards have helped tailor the application. We can use dashboards that are typically more of a conversational dashboard, and that's where we are entering it. I need to use further platforms that I am in the assessment of, and they are going to be integrated along with Meta Llama. I am using each one for a purposeful derivation rather than giving everything to one option because when you change the RAG, the RAG will keep on changing. Meta Llama can become something else tomorrow; it can become Qwen, which is another one that I am evaluating for my small language model. I might even drop Meta Llama because it is very heavy on memory. I might have to go for a lean memory and edge AI approach; that's where I am focusing on purposefully.
What needs improvement?
I would like to see additional features in the future where Meta Llama allows us to have the datasets by domain, rather than just covering everything.
There is nothing else I would add or improve at this moment; everything is still in progress. The world is moving in different directions, and I need to make a use case out of it properly for AI, especially in the supply chain and financial services. I am trying to see how we can bring AI on top of CRM or AI on top of ERP. That is the way things are moving. You cannot have AI as the main product; AI is just a subsidiary product or add-on. It is not mainstream and can never be mainstream. I would like to bring most implementers in business to understand this because in business, we go technology, but we need to go from the business line towards AI. The question is how do you deploy AI on top of your existing infrastructure? We need accessibility and affordability of AI for small and medium businesses; that is where the game changer is happening, and that is where I am investing my time and money.
For how long have I used the solution?
I have been dealing with Meta Llama for almost five months now.
Which solution did I use previously and why did I switch?
There are a few products I evaluated before choosing Meta Llama. There is Qdrant, which is a beautiful option. I moved on to Meta Llama, but I might even jump to Qwen, which is beautiful today and requires a smaller footprint. I am working with Qdrant with OpenSearch, with adaptive RAG, and trying to bring Redis for the memory model.
Which other solutions did I evaluate?
I finally chose Meta Llama because it is the only minimal option available to the world today. Now I am seeing Qwen coming in very nicely, especially if you look at the Vector DB. You cannot compromise Qdrant; it is so powerful. The question of how the retriever and Qdrant fit together along with LLMs or SLMs is what matters.
What other advice do I have?
I am working with a combination of products now, mostly as I work with air-gapped systems and have moved to all open sources.
Prediction is up to us, which is why I said it is on top of your ERP or your stored procedure. I am bringing embeddings of intelligence through existing legacy systems so that you do not build anything new. It just learns the RAG based on your existing intelligence so that it can predict through the business without hallucination; otherwise, it will create things on its own. It does not stop; we need to bring the guardrail, not only for data security but guardrails for speaking specific language, not beyond that.
