pinecone.core.client.exceptions.ForbiddenException: (403)

Support
1

Hi there, I’ve been getting a pinecone.core.client.exceptions.ForbiddenException: (403) error whenever I try to use upsert. I’m able to upsert in the browser so I’m not sure what’s wrong. Any ideas? What information would be helpful to debug?

2

Hi @od123,

Welcome to the Pinecone community!

Check that the API key you’re using is the right one for the index you’re using. You can get your API key in the console; they’re listed under API Keys on the left side menu. API Keys are specific for projects, so if you have multiple projects, it’s possible you’re using an API key from a different one than the index you’re using is in.

Cory

3

Thank you, Cory. Any other ideas? Have verified the api key is correct.

4

In that case, can you share the code you’re using? We’ll need the index name, the project name or ID, any namespaces you’re using, and a sample vector. Please don’t share the API key; we won’t need that.

Cory

5

Thanks, Cory. If helpful, I also seem to get “urllib3.exceptions.ProtocolError: (‘Connection aborted.’, ConnectionResetError(54, ‘Connection reset by peer’))” issues a lot as well.Index name: company-search-description-10k-ada2
project name: Default Project
Code:

with open(env.embedding_id_file, ‘rb’) as handle:

embeddings_with_ids = [(str(i), embedding) for i, embedding in pickle.load(handle)]

print(len(embeddings_with_ids))

upload to pinecone

assumes [(id, [embedding])] format

create_pinecone_index(env.PINECONE_INDEX, env.OAI_EMBEDDINGS_DIMENSION)
index = pinecone.Index(env.PINECONE_INDEX)

i = 0
while i < len(embeddings_with_ids):
start = i
end = i+env.MAX_PINECONE_DB_BATCH_SIZE
batch = embeddings_with_ids[start:end]
if len(batch) > 0:
index.upsert(vectors=batch)
i = end

6 
    with open(env.embedding_id_file, 'rb') as handle:
            embeddings_with_ids = [(str(i), embedding) for i, embedding in pickle.load(handle)]
    # upload to pinecone
    # assumes [(id, [embedding])] format
    create_pinecone_index(env.PINECONE_INDEX, env.OAI_EMBEDDINGS_DIMENSION)
    index = pinecone.Index(env.PINECONE_INDEX)
    i = 0
    while i < len(embeddings_with_ids):
        start = i
        end = i+env.MAX_PINECONE_DB_BATCH_SIZE
        batch = embeddings_with_ids[start:end]
        if len(batch) > 0:
            index.upsert(vectors=batch)
        i = end
7

unfortunately the vector is too long for the space provided. I can send via email if possible?

8

Can you share the source of the create_pinecone_index() function? And why not just use the create_index() method on the client itself?

pinecone.create_index(env.PINECONE_INDEX, dimension= env.OAI_EMBEDDINGS_DIMENSION)

I suspect there’s a race condition at play, where you’ve submitted the index to be created, but it’s not actually ready yet by the time you start your upserts. Depending on what the create_pinecone_index() function is doing, that could be part of it; or it could be related to a bug we’ve discovered recently, where index creation can be erroneously reported as complete a few seconds before it actually is. We have a fix that will be deployed next week to correct this; in the meantime, adding time.sleep(5) between creation and first upsert should give enough time for the process to complete and avoid the 403 errors.

1 Like
9

Thank you, Cory, will try that.

def create_pinecone_index(index_name, dimensions):
    if index_name in pinecone.list_indexes():
        #return
        pinecone.delete_index(index_name)
    pinecone.create_index(name=index_name, dimension=dimensions)
1 Like
10

I got past the connection reset error but now are getting the “Reason: Forbidden
HTTP response headers: HTTPHeaderDict({‘date’: ‘Fri, 13 Jan 2023 18:28:35 GMT’, ‘server’: ‘envoy’, ‘connection’: ‘close’, ‘content-length’: ‘0’})” error again

11

More than likely, the index isn’t available when you’re calling to upsert against it. The create_pinecone_index() function looks fine since it’s just wrapping create_index() and is using the defaults, including a wait until the index is ready. So it’s probably the bug I mentioned before. Try adding time.sleep(5) after the index is created to give more time to ensure it’s available before starting your upserts.

12

Ahhh changed it to time.sleep(30) and it works now, thank you!

1 Like