Title:Issues with Data Import from S3 and Namespace Handling

Support
,
1

Hello Pinecone Community,

I am facing a couple of issues while working with Pinecone:

  1. Slow Import from S3:
    Importing data from S3 to my Pinecone index is taking a considerable amount of time, even though I am only importing 12 vectors. Has anyone experienced similar delays? Any insights or solutions would be appreciated.
  2. Namespace Handling:
    When importing data, if a namespace already exists, the process doesn’t continue seamlessly for the other namespaces. It would be great if there were a warning for existing namespaces, but the import process continued for the remaining namespaces.

Are there any workarounds or solutions for these issues?

Thanks in advance for your help!

2

Hi @nikhil! Thanks for posting.

  1. The 10-minutes delay is to be expected with import (see here). The technical explanation is that import skips the freshness layer and waits for index building to kick in.
    If immediate freshness is a priority, I’d recommend using the regular upsert - but let me know if there are other considerations in play here.
  2. Thanks for the feedback. Can you provide more details on your use case? For example, what makes namespaces overriding likely?
    Either way, we are looking into adding more granular error handling.