Manage too many namespaces for RAG

Support
,
1

I am new to RAG development. I have a RAG chatbot linked with Pinecone and has around 250 namespaces. I am having a tough time managing them. I have the namespaces in the following order:

  • product1_v1_namespace
  • product1_v2_namespace
  • product1_v3_namespace…
    and so on
    Whenever the user asks a question, I don’t know which namespace to search in since the documents are similar and differ only by version number. I do not want my RAG application to spew out information from the wrong version of the documents.

Any tips and suggestions would be appreciated.

Best,
Mustafa

2

Hi @mustafak and welcome to the Pinecone community forums!

Thank you for your question.

As you’ve determined, namespaces are one way of segregating your product information by version.

However, you could also look into Metadata Filtering.

With this approach, you would attach a property to the metadata when upserting your vectors that specifies the exact version of the product:

{
    "product": 1, 
    "version": v3
}

This way, you could use operators like equals: $eq to filter on the exact product and version your user is asking about.

Hope this helps!

Best,
Zack