AttributeError: module 'pinecone' has no attribute 'list_indexes'

Support
1

I am creating a llm application with flask. And I am testing the API call using curl.

I have tried:-

  1. Using pc.list_indexes() instead of Pinecone.list_indexes()

but still the error is same

following is the code

import pinecone
from pinecone import Pinecone, ServerlessSpec

pc = Pinecone() #it's in latest


from app.services.openai_service import get_embedding
import os

PINECONE_API_KEY = os.environ.get('PINECONE_API_KEY')
# pinecone.init(api_key=PINECONE_API_KEY, environment='gcp-starter')  #it's outdated
EMBEDDING_DIMENSION = 1536

def embed_chunks_and_upload_to_pinecone(chunks, index_name):
    if index_name in pc.list_indexes():
        print("\nIndex already exists. Deleting index ...")
        pc.delete_index(name=index_name)
    
    print("\nCreating a new index: ", index_name)
    pc.create_index(name="index_name",
                          dimension=EMBEDDING_DIMENSION, metric='cosine')

    index = pc.Index(index_name)

    # Embedding each chunk and preparing for upload
    print("\nEmbedding chunks using OpenAI ...")
    embeddings_with_ids = []
    for i, chunk in enumerate(chunks):
        embedding = get_embedding(chunk)
        embeddings_with_ids.append((str(i), embedding, chunk))

    print("\nUploading chunks to Pinecone ...")
    upserts = [(id, vec, {"chunk_text": text}) for id, vec, text in embeddings_with_ids]
    index.upsert(vectors=upserts)

    print(f"\nUploaded {len(chunks)} chunks to Pinecone index\n'{index_name}'.")


def get_most_similar_chunks_for_query(query, index_name):
    print("\nEmbedding query using OpenAI ...")
    question_embedding = get_embedding(query)

    print("\nQuerying Pinecone index ...")
    index = pc.Index(index_name)
    query_results = index.query(question_embedding, top_k=3, include_metadata=True)
    context_chunks = [x['metadata']['chunk_text'] for x in query_results['matches']]

    return context_chunks   


def delete_index(index_name):
  if index_name in pc.list_indexes():
    print("\nDeleting index ...")
    pc.delete_index(name=index_name)
    print(f"Index {index_name} deleted successfully")
  else:
     print("\nNo index to delete!")

here is curl command which i am running

Invoke-RestMethod -Uri http://localhost:5000/embed-and-store -Method POST -Headers @{"Content-Type"="application/json"} -Body '{"url":"https://medium.com/@deepak.holla/what-are-data-connections-data-sources-and-data-models-in-tableau-a08860d54c1e"}'
2

hi @silas can you help me with this?

3

Hi @vinayakgavariya123. With the new Python client, you have to iterate over the list_indexes result a bit differently. Can you try changing if index_name in pc.list_indexes(): to if index_name in pc.list_indexes().names():?

You can find a complete example in the Python client migration guide.

4

thanks @jesse i will try that.

i guess i don’t have access to notion page so can you please help me with right link for complete example.

5

So sorry about that. Here’s the right link.

6

thanks @jesse

got all the updated things at one place.

but still the same error is there.

import pinecone
from pinecone import Pinecone, PodSpec

pc = Pinecone(api_key=os.environ.get('PINECONE_API_KEY')) #it's in latest


from app.services.openai_service import get_embedding
import os

# PINECONE_API_KEY = os.environ.get('PINECONE_API_KEY')
# pinecone.init(api_key=PINECONE_API_KEY, environment='gcp-starter')  #it's outdated
EMBEDDING_DIMENSION = 1536

def embed_chunks_and_upload_to_pinecone(chunks, index_name):
    if index_name in pc.list_indexes().names():
        print("\nIndex already exists. Deleting index ...")
        pc.delete_index(name=index_name)
    
    print("\nCreating a new index: ", index_name)
    pc.create_index(name=index_name,
                          dimension=EMBEDDING_DIMENSION, metric='cosine')

    index = pc.Index(index_name)

    # Embedding each chunk and preparing for upload
    print("\nEmbedding chunks using OpenAI ...")
    embeddings_with_ids = []
    for i, chunk in enumerate(chunks):
        embedding = get_embedding(chunk)
        embeddings_with_ids.append((str(i), embedding, chunk))

    print("\nUploading chunks to Pinecone ...")
    upserts = [(id, vec, {"chunk_text": text}) for id, vec, text in embeddings_with_ids]
    index.upsert(vectors=upserts)

    print(f"\nUploaded {len(chunks)} chunks to Pinecone index\n'{index_name}'.")


def get_most_similar_chunks_for_query(query, index_name):
    print("\nEmbedding query using OpenAI ...")
    question_embedding = get_embedding(query)

    print("\nQuerying Pinecone index ...")
    index = pc.Index(index_name)
    query_results = index.query(question_embedding, top_k=3, include_metadata=True)
    context_chunks = [x['metadata']['chunk_text'] for x in query_results['matches']]

    return context_chunks   


def delete_index(index_name):
  if index_name in pc.list_indexes().names():
    print("\nDeleting index ...")
    pc.delete_index(name=index_name)
    print(f"Index {index_name} deleted successfully")
  else:
     print("\nNo index to delete!")

here is the code

7

Hi @vinayakgavariya123. Have you upgraded your Python client? I’d try that. You can remove import pinecone and leave from pinecone import Pinecone, PodSpec.

Also, you need to include a spec parameter when creating an index. In your case, if you’re trying to create a pod-based index, you’d change

pc.create_index(name=index_name,
    dimension=EMBEDDING_DIMENSION, metric='cosine')

to

pc.create_index(
    name=index_name,
    dimension=EMBEDDING_DIMENSION, 
    metric='cosine',
    spec=PodSpec(
        environment="us-west1-gcp",
        pod_type="p1.x1",
        pods=1
    )
)

But be sure to set environment, pod_type, and pods the way you want.

If you’re trying to create a serverless index, replace PodSpec with ServerlessSpec in the import statement, and create the index like this:

pc.create_index(
    name=index_name,
    dimension=EMBEDDING_DIMENSION, 
    metric='cosine',
    spec=ServerlessSpec(
        cloud='aws', 
        region='us-west-2'
    ) 
)

Let me know if any of that helps.

8

hey @jesse i really appreciate your help here.

it resolved the error.

but now i am getting 429 error ( PineConeAPIException: too many requests)

so just wanted to ask how i can remove this limitation.

i am calling the api from Invoke-RestMethod -Uri "http://localhost:5000/embed-and-store" -Method Post -Headers @{"Content-Type"="application/json"} -Body '{"url":"https://dev.to/bobur/how-to-build-a-custom-gpt-enabled-full-stack-app-for-real-time-data-38k8"}' this command.

10

Sorry for the delay, @vinayakgavariya123. I think you’re probably running into the maximum number of serverless indexes allowed in a single project. Here are some docs on that, with suggested work-around: Troubleshooting.