I’ve built a “talk with your books” chatbot using this chunking approach and Pinecone. A fork of GitHub - mayooear/gpt4-pdf-chatbot-langchain: GPT4 & LangChain Chatbot for large PDF docs
A big problem here is that the bot cannot summarize a book or any large document because it only sees 1-2 chunks at a time. It can answer questions as long as the answer is fully contained inside a chunk, but not more than that. Are there any approaches to chunking that will allow a model to be able to “digest” a large document?