I build a chatbot to chat with pdf, so the pdf’s text (divided to chunks) is stored in vector database and whenever input new prompt it embedded and semantic searched and retrieve the closest texts to feed them to gpt to generate it’s answer.
So if the texts are about AI for example, and the user input prompt is something like “What is AI?”, the the texts that relate to this question will be retrieved and fed to gpt to make the answer, but what will be retrieved if the user’s next input prompt was something like “explain more”? I mean how to be sure that the retrieved chunks of texts are related to the context when the prompt is general like just “explain more”?
You would probably want to use the LLM to rephrase the user’s question to a knowledge-base query. That involves calling the LLM twice - once feeding it the entire history, and telling it to query the knowledge base, then again with Pinecone’s response and a different prompt "answer the user’s question using this context.