I’ve just created an index, it would be great if I could retrieve all the IDs in it, in order to only insert new vectors. Not obvious how to do this from the API docs? https://www.pinecone.io/docs/api/operation/fetch/

Hi Greg, it’s quite a basic basic feature for an advanced commercial tool to not have. The sort of thing that makes using Pinecone feel like a much less polished experience.

We appreciate the feedback, George. We have a lot of great features in the pipeline, some of which we certainly wish we had sooner. If this is a blocker to you taking Pinecone to production send me an email ( greg@pinecone.io ) and we can work with you on this.

Is this possible yet? Any workaround for it?

It’s over a year later and this is apparently still not possible(?) I have a scenario where I want to re-process the metadata in the index but don’t have a cheap way to re-analyse the source data to determine the list of keys that would’ve been generated. I’m frankly amazed I can’t fetch all items…

+1 I’m facing a similar challenge.

this is a must have feature, i cannot believe it is not supported. Considering you had a bad network, and the re-retries of network operation may insert duplicate items in the database, and it seems you donot have a way to delete those duplicate items…This also means the similiarity search would ret…

import pinecone my_index = pinecone.Index("YOUR_INDEX_NAME") number_of_vectors = int(my_index.describe_index_stats()['namespaces']['YOUR_NAMESPACE_NAME']['vector_count']) def generate_search_vector( idx, vector_dimention=100, min_val=0, max_val = 1 ): if idx == 0: return [min…

Facing the same issue. I totally agree with @GeorgePearse , this is a basic feature for an advanced commercial tool to not have. I took some time to believe that this was not possible. :face_with_diagonal_mouth:

It is still pretty ridiculous that we have to do this, but here is a workaround: import numpy as np def get_ids_from_query(index,input_vector): print("searching pinecone...") results = index.query(vector=input_vector, top_k=10000,include_values=False) ids = set() print(type(results)) for …

index = pinecone.Index('jj') index.query( top_k=367, vector= [0] * 1536, # embedding dimension namespace='', include_values=True, filter={'docid': {'$eq': 'uuid here'}} # Your filters here ) We can use this to fetch list of ids in a namespace

Pinecone Community

How to retrieve list of IDs in an index

Support

vector-database

show post in topic

Home
Categories
Guidelines
Terms of Service
Privacy Policy

Powered by Discourse, best viewed with JavaScript enabled