I would like to use the Custom Text Splitter to add own vector to Pinecone for each of the text sections on my txt document. As understood, the default separator on the tools is \n\n. I’m using Windows note app which gives \r\n\r\n when I separate the text section from another section with empty row. I have tried to add \r\n\r\n and “\r\n\r\n” to the Custom Separator field, but it doesn’t split the text. Instead the text is added to Pinecone as one vector and I can see the “\r\n\r\n” separators in the text. Have I totally misunderstood how this tool should work?
Hi @tenordma, please share a reference to the text splitter you are using. If you are referring to a class from the langchain_text_splitters package, you should contact LangChain or post on their community forums for support.