I’m currently working on an AI project, but I’m relatively new to the field and need some guidance. My goal is to utilize AI models, specifically OpenAI models, to generate HTML templates based on some data I have.
I explored using Langchain and Pinecone I inserted my data into Pinecone as separate documents. Then, to generate templates based on a specific query, I performed a similarity search in the vector database. The retrieved documents were used as context for an OpenAI model to generate templates accordingly. However, I encountered a max token error when passing lengthy code as context to the AI model.
The context passed to the AI model contains extensive code, and I’m unsure how to handle this error. Is there a way to increase the token length or overcome this limitation?