Serverless indexes do not support delete by metadata. Is the correct approach to this to do a query with the relevant metadata, set the topK to infinity, and delete based on IDs there? The documentation suggests using list, but list doesn’t have a filter param either?
1 Like
+1 - would love this feature for deleting by metadata in a serverless index! It’s quite difficult right now to delete appropriately.
@winston1 yes, that’s the recommended approach. Also, FWIW, delete-by-metadata can cause issues on pod-based indexes, and we recommend not to do it on those, either, and instead use a metadata query and batch the deletes. We have this technique outlined in a support article.
Right now there is no way to query all the vectors according to only a metadata value, instead we have to include either an ID or a vector value. This constraint is forcing me into building so much unnecessary complexity in my app just so I can be sure that when a user deletes a record all the appropriate vectors for it are also deleted.
I don’t mind querying and then batching deletes, but being forced to generated a dummy vector just so I can do the metadata query that is dynamic according to the dimensions of the index just feels so the opposite of elegant.
Excited to announce that Pinecone now supports delete-by-metadata on all indexes including Serverless!
Hi folks, one minor clarification on @perry’s comment above:
We’re still migrating some serverless indexes in the AWS us-east-1 and us-west-2 environments. We paused migrations to focus on stability work and general performance improvements, though we have resumed these efforts.
All serverless indexes will soon be migrated to our new architecture that supports delete-by-metadata!