We use Open AI (text-embedding-ada-002) and Pinecone to create and store user-character conversations so the character has a “memory” of past conversations with the user.
Everything is fine for a conversation like this:
User: I like pasta.
Bot: I’m so glad you like pasta.
User: What is my favorite food?
Bot: You like pasta.
But if the user has previously asked the bot that question more than 5 times something like “What is my favorite food?” Then the top 5 most relevant results will be the user’s question (“What is my favorite food?”) and the bot will not know about pasta because the words in the question are so similar.
Here is what ChatGPT says:
- Diversify the Results: Instead of just taking the top result, you could take a set of top results and then choose the most diverse one. This could be done by calculating the similarity between the results and choosing the one that is most different from the others. This would help to ensure that you’re not just getting the same result over and over again.
- Prioritize Recent Conversations: You could weight the results based on recency, so that more recent conversations are given more importance. This would help to ensure that the bot is taking into account the most recent information about the user’s preferences.
- Filter Out Repeated Questions: Another approach could be to filter out repeated questions from the results. This could be done by keeping track of the questions that the user has asked and then excluding those from the results. This would help to ensure that the bot is not just repeating the same information over and over again.
- Use a Hybrid Approach: You could also use a hybrid approach that combines several of these strategies. For example, you could diversify the results, prioritize recent conversations, and filter out repeated questions. This would help to ensure that the bot is providing the most relevant and diverse responses.
- Improve the Embedding Model: The quality of the embeddings used in the vector search engine can also have a big impact on the results. You could experiment with different embedding models to see if they produce better results. For example, you could use a model that is specifically trained on conversational data, or you could fine-tune a pre-trained model on your own data.
- Use Contextual Embeddings: Instead of using static embeddings for each conversation, you could use contextual embeddings that take into account the context of the conversation. This would help to ensure that the bot is understanding the conversation in context and not just based on the individual words.
Any ideas on how to solve this?