Howdy, in the docs it states that: “Pinecone Assistant supports 40KB of metadata per file.”
However, when adding metadata, I’m getting back: {"code":"INVALID_ARGUMENT","message":"Metadata failed validation: Metadata size is 1050 bytes, which exceeds the limit of 1024 bytes per vector"},"status":400}
Can you clarify what the correct limit is?
Hi @ankur.kumar - Thank you for posting on the Pinecone forum!
I did confirm that the current limit is 1KB and the docs are incorrect. We plan to increase the limit to something bigger, but we don’t have a timeline for this yet.
If you can share more about your use case for this metadata, that would help us.
We will get the docs updated in the meantime to reflect the 1KB limit.
Hi Jenna, that’s a shame. I’m having trouble fitting what I need into 1KB, and it will probably disqualify Pinecone Assistant from our evaluation.
We are planning to store recorded call data (mainly transcripts + summaries) into Assistant and then “chat” with certain type of calls. Metadata will be a robust set of filters like: call type, customer type, date info, opportunity stage, tags, and more. This would fit well within 4-5KB for sure.
Hi Ankur -
Thank you for the feedback on your use case. I’ve passed it onto the engineering team and they’ll be increasing the limit beyond 5k. I’ll report back here when that change is made.
Wow, that’s awesome. Thank you!