I need to implement an AI agent where it search across a text file and answer questions according to the text file.
My original idea was to convert each paragraph to vector, and upsert each paragraph along with corresponding vector, but seems like it’s not allowed or recommended to do that.
What is the best practice to do this kind of implementation?
You can upload the paragraph within the metadata field of your record — but yes it can slow down search times when you’re putting in many millions of vectors — ideally you should probably be storing text data in blob storage and just storing the url to your blob storage in the vector metadata (or naming the file in blob storage after your record ID), you’d then retrieve text from blob storage after retrieving your vectors