Getting an Upsert Error even for small batch sizes

Support
1

I don’t know why this is happening. It was working fine for the last 2 weeks. Suddenly it broke the code and I am facing issues here.

Here is a code snippet.

from tqdm.auto import tqdm
import http

MODEL = "text-embedding-ada-002"

count = 0  # we'll use the count to create unique IDs
batch_size = 25  # process everything in batches of 32
for i in tqdm(range(0, len(trec['content']), batch_size)):
    # set end position of batch
    i_end = min(i+batch_size, len(trec['content']))
    # get batch of lines and IDs
    lines_batch = trec['content'][i: i+i_end]
    #print(lines_batch)
    #ids_batch = [str(n) for n in range(i, i_end)]
    ids_batch = generate_ids(i_end)
    
    # create embeddings
    
    res = openai.Embedding.create(input=list(lines_batch), engine=MODEL)
    embeds = [record['embedding'] for record in res['data']]
    #print(res)
    # prep metadata and upsert batch
    meta = [{'Content': line} for line in lines_batch]
    to_upsert = zip(ids_batch, embeds, meta)
    # upsert to Pinecone
    index.upsert(vectors=list(to_upsert))

Here is the error:

ApiException: (429)
Reason: Too Many Requests
HTTP response headers: HTTPHeaderDict({'content-type': 'application/json', 'date': 'Tue, 14 Mar 2023 15:44:45 GMT', 'x-envoy-upstream-service-time': '16', 'content-length': '135', 'server': 'envoy'})
HTTP response body: {"code":8,"message":"Your request is larger than the maximum supported size - 2MB. Please try to reduce your batch size.","details":[]}

Iterations completed before error : 11/39 [00:19<00:55, 1.97s/it]

And it barely has a lot of text. Can anyone please assist me with this?

2

Hi areeb,
The batch of 25 records being upserted is exceeding the overall data limit of 2MB per request. (as noted) This is a limitation of the data transfer process, not a limitation on being able to store that data in Pinecone.
In order to fix this, you would have to further lower your batch size, or reduce the amount of metadata that is contained in each vector.

3

Hi thanks for the reply. Here;s the working code.

from tqdm.auto import tqdm
import http
import openai

MODEL = "text-embedding-ada-002"

batch_size = 100 # process everything in batches of 32
for i in tqdm(range(0, len(trec['content']), batch_size)):
    # set end position of batch
    i_end = min(i + batch_size, len(trec['content']))
    # get batch of lines and IDs
    lines_batch = trec['content'][i: i_end]
    ids_batch = generate_ids(i_end)
    # create embeddings
    res = openai.Embedding.create(input=list(lines_batch), engine=MODEL)
    embeds = [record['embedding'] for record in res['data']]
    # prep metadata and upsert batch
    meta = [{'Content': line} for line in lines_batch]
    to_upsert = zip(ids_batch, embeds, meta)
    # upsert to Pinecone
    index.upsert(vectors=list(to_upsert))

It even let me upsert 300 records without an issue.

4

Thanks for the additional info. Your code looks ok to me. It still comes down to how much total data is being sent for each batch request. You would likely have to print out one of the batches to understand what data is being generated to determine why it is exceeding the 2MB request limit.