Fetching vector ids (1M+ vectors)

[nsartor]

How to retrieve list of IDs in an index

It is still pretty ridiculous that we have to do this, but here is a workaround: import numpy as np def get_ids_from_query(index,input_vector): print("searching pinecone...") results = index.query(vector=input_vector, top_k=10000,include_values=False) ids = set() print(type(results)) for result in results['matches']: ids.add(result['id']) return ids def get_all_ids_from_index(index, num_dimensions, namespace=""): num_vectors = index.describe_index_stats()["namespaces"][namesp…

Something still hacky, but would take definitely less time to go through all the is records is something like this.

I suggested a structured approach, but the random outperforms.