What is our primary use case?
We are using Pinecone because we have a good amount of documents which we use on a daily basis from our vendors. Based on those documents, we need to provide information to the end customer for that particular company. We have a UI where customers can ask any questions related to anything in the financial domain. We need to provide the latest information, so we are dynamically doing the chunking with the help of an OpenAI LLM model and then inserting into Pinecone. In Pinecone, we are using a very high dimension vector space, almost more than 3K dimension size. We then perform similarity search and provide the final response to the UI. For our RAG system implementation, we are using Pinecone.
What is most valuable?
There are multiple factors that impressed me while using Pinecone. I had an option to use Milvus as well, but I preferred Pinecone. The first is the UI. Pinecone's UI is really strong. If I need to do some debugging on the backend side, I simply log into the UI and can perform operations based on my demand. This is a valuable UI feature.
Second is the scalability option. I can either define my own workers or use the auto-scaling feature. From an enterprise application and scalability perspective, this is very useful. We had an incident during a Black Friday sale and other occasional events that directly impacted our product traffic. Because we selected the auto-scaling feature in Pinecone, it automatically handled all the traffic spikes and we did not face any performance issues.
Ease of troubleshooting is another valuable feature. If any transaction fails and we need to check and debug each transaction, we can perform a text search on the UI. Based on the text search, we get all the related vectors on the UI. The UI definitely helps us from a troubleshooting perspective. Selecting the infrastructure is also an important option. I can create multiple indexes based on demand so that it will not become messy for our enterprise application. Pinecone is the backbone of the entire system, helping us with cost and time savings.
What needs improvement?
I have a suggestion to expand the vector size in Pinecone. Right now, whatever the limit Pinecone is suggesting, I would recommend increasing that option. Currently, I believe it is around 3K, but if I want to go to 4K, 5K, or something higher, that would be beneficial. Embedding models are coming into the market and they are providing good amounts of vector sizes. Try to encapsulate all these features into Pinecone.
I have two main suggestions from my side. One is to increase the vector size. Currently, it is supporting only around 3K vector size, and I would recommend increasing that. The second suggestion involves creating a feature similar to LangSmith, which is a monitoring tool. In LangSmith, end-to-end API calls can be analyzed, showing what request came from the customer, what vector search was performed, what prompt was created, what call was given to the LLM, and what response was received from the LLM to the UI. The whole journey can be captured. I would appreciate if Pinecone could provide this capability from their side. I understand that Pinecone cannot capture the LLM call and everything. However, if it is possible, I could use the API key of Pinecone in my code where I can enable these feature logs and see all these things on the Pinecone dashboard.
The major improvement I am expecting from Pinecone is increased vector size. The second improvement would be to provide end-to-end debugging or the whole end-to-end call journey as a GenAI product, showing how the end-to-end journey works for a single request. If I am able to see the whole process on the Pinecone dashboard, it would be really valuable.
For how long have I used the solution?
I am using Pinecone as an enterprise application in my organization for almost three years.
What was our ROI?
I do not have specific metrics, but I can give some high-level approximations. The task that was happening before developing this product was taking around one hour, but now it is done in hardly one or two minutes. So from 60 minutes to one or two minutes, you can assume how much cost savings we are achieving. Additionally, we are engaging customers much better through the UI. Previously, customers used to wait for 60 minutes. Now they get results within one or two minutes. We are definitely increasing our customer database.
Which other solutions did I evaluate?
Milvus is another contender that we considered while deciding on a vector database. I suggested Pinecone because of its good amount of quality compared to Milvus. However, Milvus has the capability to handle any vector size, which is missing in Pinecone.