I have an index in the free-tier which recently ran full. I wanted to modify the index but had to upgrade to the Standard plan first.
After upgrading, I’m stuck with my free-tier and can’t modify any of the settings.
Am I missing something?
the project your index is in is still in “Free tier”. I think you will have to create a new Project and in that project an index that will not be free tier and you will be able to modify it.
Hope this helps
Hmm, that is annoying. I was expecting an upgrade path that would allow me to continue using the index.
Do you know of any source that talks about data migration?
I was searching in the docs but didn’t find much.
Hmm I don’t think the data migration over projects is supported in any way. There are hacky ways where you fetch all the vectors and then upsert them into a new index, but its messy imo.
What I would hope that would be one day possible is project independent collections where collections created would not be linked to a specific project. At the time I don’t think this is so.
If you have the Free index full (thats around 100k vectors?) the easiest and fastest way (not the cheapest) is to just run the process again and fill the new index.
Long story short. No there is sadly no clean or easy way to migrate your data from one index to another over different projects.
A comment regarding a similar topic (not the same issue):
Best of luck
Someone posted this very same question and I created an open-source tool that could help migrate everything from free to enterprise index in a few clicks.
Previous response: How to migrate data from my free account to enterprise account? - #2 by Jasper
Fantastic! Make sure you create a separate post about it so forum users know this exists! (if you have not already, if you have, ignore this )
Edit: I quickly checked how you did it and… oh my god… genious!! So simple yet just the thing one needs to do this!
I dont want to come off as spammy but I sincerely just want others to know about it cause Pinecone core is focused on building the hard part - a distributed vector database!!!
Figure the most I could do is build tools for those making applications on top of the infra! Ill make an original post soon!
This is a fine answer for an open source community product but not acceptable for a paid “managed” solution. Your UX leads people to believe they’re building something that can grow as they do, and that you’ll take care of the “management”. Scaling indexes is a core part of search, going back as long as the internet. Having to stop and build a bunch of scaffolding just to work around these limitations means we’re doing the “management”. Pod-sizing and index scaling needs to be addressed ASAP for this to be usable in a true production environment…