# Incremental Upsert

**URL:** <https://community.pinecone.io/t/incremental-upsert/5969>\
**Category:** Support\
**Created:** [June 24, 2024, 9:45am UTC](https://community.pinecone.io/t/incremental-upsert/5969 "2024-06-24T09:45:35Z")\
**Posts on this page:** 1\
**Showing post:** 3

<div class="post-metadata">

**Author:** ![ZacharyProser](https://sea2.discourse-cdn.com/flex020/user_avatar/community.pinecone.io/zacharyproser/32/1146_2.png) [@ZacharyProser](https://community.pinecone.io/u/ZacharyProser)\
**Post date:** [June 25, 2024, 2:09pm UTC](https://community.pinecone.io/t/incremental-upsert/5969/3 "2024-06-25T14:09:18Z")

</div>

Hi @taimoorqureshi80, and thanks for your question!

I understand your concern about upserting new documents without overwriting existing ones in the same namespace.

1. Upsert Behavior: Pinecone’s [upsert operation](https://docs.pinecone.io/guides/data/upsert-data) only overwrites records with the same ID. If you’re upserting new documents with unique IDs, they will be added to the namespace without affecting existing records.
2. Generating Unique IDs: Ensure each new document you’re upserting has a unique ID. There are several ways you could achieve this:
  - Using a UUID or a similar unique identifier generator
  - Combining a timestamp with a document identifier
  - Incrementing a counter for each new document

3. Partial Updates: If you need to update only part of an existing record, consider using the update operation instead of upsert. This allows you to modify specific fields without overwriting the entire record.
4. Batching: You _don’t need to include all previous documents in the upsert call when upserting new documents_. Instead, you can upsert only the new documents in batches. We recommend batches of 100 or fewer records:

python

Copy

```auto
index.upsert(
  vectors=[
    {"id": "new_doc_1", "values": [...], "metadata": {...}},
    {"id": "new_doc_2", "values": [...], "metadata": {...}},
    # ... more new documents ...
  ],
  namespace="your_namespace"
)

```

1. Checking Existing Records: If you’re unsure whether a record already exists, you can use [the fetch operation](https://docs.pinecone.io/guides/data/fetch-data) to check before upserting. This allows you to decide whether to update an existing record or insert a new one.
2. Namespace Management: If your use case allows, consider using different namespaces for different sets of documents. This can help organize your data and make it easier to manage updates.

I hope this helps!

Best,  
Zack

---

_[View the full topic](https://community.pinecone.io/t/incremental-upsert/5969)._