Filter support for Delete

Hi!
I’m storing vectors from multiple sources (A Wiki and a Ticketing system) in Pinecone.

I frequently re-index the sources independent from one another. So after processing the Wiki, I want to delete all old Wiki vectors and add all new ones. Upserting won’t be enough because some pages of the Wiki might have been deleted in the meantime and they should not be in the index anymore.

  • I want them to appear in the same query, and queries can only use one namespace, so splitting the vectors into one namespaces per source will not work.

  • So I’m left with metadata filters, e.g.: one metadata source field. But from the docs it looks like the delete function does not support filters.

  • That means I’d have to keep a state of all current ids (per source) locally and delete them by their ids, and hope that this never gets “out of sync”, which seems quite fragile.

Is there anything I’m missing or is deleting by metadata filter on the horizon?
Best, Mathis

Hi @mathis,

You have two options to delete vectors: one, by ID, as you said. And also by metadata filter. This option uses a POST method to the API, rather than a DELETE method, and is mutually exclusive with providing IDs.

For more details see Delete.

Cory

1 Like

Ah perfect, it’s not mentioned in the Guides about deletion, thanks for pointing me to the API docs!