Hello, I have a project where I’m searching through blogs using OpenAI and PineCone. Currently search works decent but I want the users to be able to ask questions related to date. I have the blog date stored in a Metadata field. I came across this post Query by group of sentences - #5 by axgalache where it mentions to insert the data you want to filter on by similarity, that data should be stored in the vector.
I tried that by inserting the date into the blog vector like so. Date: 4/13/2023 \n blog text. But I’m still unable to search a blog by date. Any ideas would be greatly appreciated.
I like how you went about it, but I would maybe add some more… “juice” to the date part of your text. Something that would give a bit more meaning to the whole search by date part.
I assume you are trying to answer questions like: What did someone wrote on 13.4.2023 (or Thursday 13th of April this year) on their blog post?
What I would try is writing this into your blog post chunk instead just “Date: …”. So the text you could be storing (via vector) into Pinecone would be like: User1234 wrote the following blog on Thursday 13th of April (3/14/2023) \n blog text.
Ok, its working better. What I did was insert 2 sentences into each chunck with different wording around dates like so:
April 13, 2023
4/13/2023
string dateVariation1 = $"The author wrote the following blog on {originalBlog.Date}.\n";
string dateVariation2 = $"The date of this blog article was {Helpers.DoDateConversion(originalBlog.Date!)}.\n";
string datesConcat = string.Concat(dateVariation1, dateVariation2);
int textSize = Encoding.UTF8.GetByteCount(datesConcat + originalBlog.Text!);
Code written in C#
Thank you for your help once again. I plan to do a YouTube video on the whole project this week-end, I’ll post a link.
