Design data schema

Support
1

Hi,

Thank you for your product. I have a couple of questions:

  1. Is the data to upsert limited to the format [id, value, metadata], or can I design the data schema myself?
  2. It appears that the metadata value must be a string, number, boolean, or list of strings. Is there a way to use a nested dictionary (JSON) for the metadata?
2

Metadata must be flat. I’ve used a few (less than ideal solutions) in the past:

  1. If using Python, store any nested dicts as a string, and use ast when querying to expand them.
  2. If your metadata comes from an external DB, just add the index reference to the record in that DB to the metadata, and query the DB once once you retrieve a vector.

Neither of these are ideal since they impact metadata filtering. You could also try to flatten the dict before upserting, but if you a dict with a list with many records this is also not ideal.

1 Like