Fetching Options

Hello Pinecone Folks!

I want to learn about options for generic vector retrieval, and have seen this page, on the top of which says there may be workarounds.

If there are workarounds, please let me know them. Thank you!

Specific How-to’s:

  • Let’s say I want to retrieve all the vectors in a namespace, or a subset of namespaces. How do I do that?
  • Let’s say I want to retrieve all the vectors within a namespace and filter by a specific metadata filter criteria. How do I do that? I imagine that will be useful for certain troubleshooting purposes.
  • How do I retrieve all the vectors in an index?
  • Optionally, or a special case of the above: how do I simply retrieve all vector IDs in an index?

Hey @startakovsky, We currently don’t have any workarounds for the fetch use cases that you mentioned, just the fetch API. We are working on features that would let you dump/retrieve data more easily.

Regarding your questions:

  • As of now, we don’t allow cross-namespace operations. The best method I would suggest would be to keep a copy of the ids separate from Pinecone for ease.
  • We don’t support filtering on fetches yet, just on queries. However you can send dummy queries with filters to troubleshoot vectors in a namespace. Just set include_data=True,include_metadata=True in the query request.
  • We unfortunately don’t have any direct features to give you back all the vectors/ids in the index at the moment. But those are WIP and hopefully we can release these features soon.

@rajat Thank you for this reply, I’ll stand by for updates.