Hi,I’ve been following the Pinecone docs for llamaindex to create a RAG app.Basically,in this doc we load, embedd and store the data in a single go.
I was wondering on how to insert a data without affecting the existing data. At first, i thought of using namespaces for different kinds of data but what if i just wanted to add a data to a pre-existing namespace without overriding the existing data. Since ill be using embedding model to first convert the data into vectors it’ll be expensive to do it from scratch. The doc i gave above uses ingestion pipeline to upsert the embedded documents hence i don’t have any manual control on the vectors upserted and the its meta-data unlike the ones in the documentation.
Hi @fawaz.dev and welcome to the Pinecone community forums!
Thank you for your question.
Please see this related post and its answer, as I believe they’re both relevant to your question.
If you review our docs on upserting data, you’ll see that a record will only be overwritten if you supply the exact same ID when upserting new data.
You therefore have the option of generating a unique ID on each upsert, or using the update method as detailed in the linked post above to only modify part of your record.
All that said, it’s difficult to provide you with a good response without seeing your relevant code.
If you’re able to share the relevant code snippets showing what you’re attempting to do, being careful not to post any secrets such as your Pinecone API key, we could provide you with better guidance.
Hope this helps!
Best,
Zack