Multiple embeddings per ID

Support
1

Does Pinecone support multiple embeddings per ID? It would appear that given the upsert commands of:

index.upsert([
    ("A", [0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1]),
    ("A", [0.2, 0.2, 0.2, 0.2, 0.2, 0.2, 0.2, 0.2]),
])

The second embedding would overwrite the first one.

The task I’m trying here is embedding multiple paragraph vectors for a single document (“A” being the document, in the example). If multiple embeddings per ID isn’t possible, what’s the recommended way to approach this? I can thing of encode the paragraph in the id, e.g. “A-0, A-1, A-2” and possibly in the metadata, but this doesn’t seem ideal, like if I want to delete or change all paragraphs for a document I wouldn’t have a consistent index of the IDs associated with a given doc.

2

Hi @etherealmachine,

You’re right, Pinecone doesn’t support lists of embeddings in a single vector. The recommended way to have separate embeddings for each paragraph in a document, and then associate them all together, would be to use metadata to store a unique key for the document itself. You can then filter by metadata to return or delete all of the vectors at once.

See this page on metadata filtering for more details.