Getting Index of an Embedding

Search, No Filter
, ,
1

I am trying to get the information from a pinecone index

db = PineconeVectorStore.from_documents(documents=splits, embedding=OpenAIEmbeddings(model = "text-embedding-ada-002"), index_name=index_name)

With this I get a list of indexes.

index = pc.Index(index_name)
for ids in index.list():
    print(ids)

How can I get the information of a specific index? For example, I want to know the index of my last emedding in the splits and then get the information like page content, metadata, …

I couldn’t find anything in the documentation

2

Hi @bmartinc80 and welcome to the Pinecone forums! Thank you for your question.

Could you please provide more details on what your end goal is? What are you trying to accomplish by getting additional Pinecone index information?

Here’s a couple of thoughts based on what I understand from your question:

  1. Initialization and Index Creation: When you use VectorStoreIndexWrapper.from_documents, it already initializes and manages the vector index for you, as demonstrated in the LangChain documentation for Pinecone’s vector store here - and the index will be named whatever you passed in as index_name to the constructor . You don’t need to separately create or manage indices since the wrapper handles these aspects.
  2. Simplification and Abstraction: The abstraction provided by VectorStoreIndexWrapper is intended to make it easier for developers by encapsulating lower-level details of vector storage and retrieval. This means you can focus more on what you want to achieve with your data rather than on how to manage the index.
  3. Expected methods: If you’re expecting the return value of the PineconeVectorStore.from_documents call to have additional methods for getting more details about the index itself, that may be the source of the confusion. The from_documents method returns a VectorStoreIndexWrapper as documented here, so you can only call those listed methods on it.
  4. Using Pinecone’s index description method. However, separate from the langchain libary and within Pinecone’s own Python SDK, you can use the Describe an index method documented here.

Here’s what that would look like in Python:

# pip install pinecone-client
from pinecone import Pinecone

pc = Pinecone(api_key="YOUR_API_KEY")

pc.describe_index("serverless-index")

Hope that helps, and let me know if you have any follow-up questions!

3

Hi Zack

The question is how to get information like the ID in the form of
“9a4c38b2-aec6-4652-9c5d-889a63ad4863” once I have added the documents in the index.

When I add the documents using “from_documents” I see everything in my index on the web. I can filter there by ID or Vector. But how do I know the index of a specific document or how can I find something based on metadata that are available in my documents like name, content, title,…?

I have found the option of upsert, but this requires defining the metadata in a function. And then is easy to find because I define the ID there, but using “from_documents” creates this ID automatically.

Is there any function available in Pinecone for this?

Cheers!
Benito

4

Hi @bmartinc80 ,

Thanks for the clarification!

I think you’ll find our LangChain guide useful in this case because it shows exactly how to use the same methods you’re referencing here.

Once you’ve gotten your vectorstore populated, you’ll be using the query methods to pull back the results you’re looking for.

If you scroll down on that page to the query examples, look for the commented out output that shows how the retrieved vectors will contain the content plus metadata you’re looking for:

query = "who was Benito Mussolini?"  
vectorstore.similarity_search(  
    query,  # our search query  
    k=3  # return 3 most relevant docs  
)  

# Response:
# [Document(page_content='Benito Amilcare Andrea Mussolini KSMOM GCTE (29 July 1883 – 28 April 1945) was an Italian politician and journalist...', metadata={'chunk': 0.0, 'source': 'https://simple.wikipedia.org/wiki/Benito%20Mussolini', 'title': 'Benito Mussolini', 'wiki-id': '6754'}),  
# Document(page_content='Fascism as practiced by Mussolini\nMussolini\'s form of Fascism, "Italian Fascism"- unlike Nazism, the racist ideology...', metadata={'chunk': 1.0, 'source': 'https://simple.wikipedia.org/wiki/Benito%20Mussolini', 'title': 'Benito Mussolini', 'wiki-id': '6754'}),  
# Document(page_content='Veneto was made part of Italy in 1866 after a war with Austria. Italian soldiers won Latium in 1870. That was when...', metadata={'chunk': 5.0, 'source': 'https://simple.wikipedia.org/wiki/Italy', 'title': 'Italy', 'wiki-id': '363'})]
5

In addition, be sure to check out our metadata filtering functionality for more control on the vectors you retrieve in a query.

Hope that helps!

Let me know if this unblocks you or not.

Best,
Zack

6

Hi there,

Looking for the same here. Is it possible to get the ID of a new vector following the embedding without doing any queries?
I’m using Llamaindex with index = VectorStoreIndex.from_documents(documents, storage_context=storage_context,show_progress=True) to embed a document and save it in Pinecone index.

7

Hi @shayan.zahedi, and thanks for your question.

I’m assuming you meant “following the upserting” your vectors, but no, the upsert call does not return an ID, and it’s up to you to provide the ID of your vectors when upserting.

What specifically are you trying to accomplish?

Best,
Zack