We’re currently engaged in a project akin to the functionality described in the smart trackers article by Gong. Our aim is to pinpoint a concept that goes beyond mere similarity. Despite employing embeddings and top-k searches, we’ve encountered irrelevant outcomes. Our consideration now leans towards reranking within a single query using the database. Can this be achieved? We’re interested in refining a reranking model and have noticed OpenSearch supports such capabilities. Is there a comparable feature available within this community?also we need to find the exact number of documents that are relevant to that topic, that represent the same concept, like a count. I understand this is not doable right? because we need to always define the top_k so that means knowing in advance the number of results.
Thanks for the questions! I just shot you an email looping in one of our solutions engineers. We have a few follow up questions to ensure we provide the proper guidance that would be easier to coordinate over a call, if that’d work for you.
Edit: Will provide an update here in thread afterwards.
Avery