Hybrid Search in Public Preview after 2 years?

Search, No Filter
1

I am considering using Pinecone as our Vector db for our production RAG implementation. One of the items I found interesting was the first class support for Hybrid Search. After developing most of the implementation for this, I realized the documentation states that its in Public Preview since October 2022

Documentation also states that Public Preview is not recommended for production usage

  • Public preview: Unlikely to change between public preview and general availability. Not recommended for production usage. Available to all users.

Is the documentation outdated and safe to use in active production software? Is the feature planning on being released to general availability in the near future? Or is it planning on being scrapped? I’m not really sure how to move forward with pinecone with a preview feature from over 2 years ago

2

@qhenkart Thanks for the question.

The short answer is: we believe there are better approaches to sparse-dense search than our current hybrid search implementation. You’ll see on our Considerations for serverless indexes page that the initial search is just based on the dense vectors. Cascading retrieval has been found to be much more performant for sparse-dense queries. Feel free to request Sparse Early Access (link on that blog post).

That said, we’re working on hybrid index improvements under-the-hood to improve recall. The documentation here is up-to-date, and we’re not scrapping this feature, but we do plan to release a better, more performant way to interact with sparse-dense vectors. More on this to come soon.