My usecase is we are parsing and storing potentially hundreds of documents in pincone/vector db. User asks a question, the question is embedded and returning lets say 20 topk results. Those results are fed into prompt engineering and LLM answers the question, pretty basic long term memory implementation.
Challenge were facing is the mixed results from so many similar documents are clouding the results. Ideally in the initial search result Id like to be able to grab the top result, find the source document (thought of storing that as metadata in each record) then get nearest neighbors above and below and feed that into the prompt.
Then I stumbled on SPADE technique and see that the sparse vector can represent the document itself with the dense search being semantic like normal.
Question is the pros and cons of each approach - sparse-dense vs nearest neighbors via metadata. Appreciate any insight or experiences anyone has with either approach!