Retrieving Index from Collection between app sessions

Search, No Filter
1

Scenario:
I create MyIndex and ingest all the embeddings (along with metadata) that I created with OpenAI API.
Lets assume that everything is fine and I managed to query what I need.
Now, I don’t want to let Index service running because it costs money, so I backup the index into collection:

pinecone.create_collection("MyCollection", "MyIndex");

Then I delete the index:

pinecone.delete_index("MyIndex")

Then I exit the app. At this point the index is not running and I am not charged.

Next day, I need to query the embeddings that were ingested into pinecone.
So, how do I do it?
The documentations says:

To create an index from a collection, use the create_index operation and provide a source_collection parameter containing the name of the collection from which you wish to create an index. The new index is queryable and writable.

pinecone.create_index(“MyIndex”, dimension=128, source_collection=“MyCollection”)

However, when I look at the API reference at:
https://docs.pinecone.io/reference/create_index/

I do not see any parameter like source_collection…
Can anyone can clarify? Also about whether this is valid approach for saving and restoring the index between app sessions.

2

Hello!
Hmmm, I may not be understanding your problem correctly.
In cURL, you can generate an Index from a collection with the following code.

create_index
create_index2608×1376 292 KB

reference_create_index
reference_create_index3119×2595 1.09 MB

The method of disabling the Index to save money certainly seems effective, though,
I don’t think it is intended to be used in a way that creates an Index every day. (Such as very large data sets)
It might be a good way to save money for development purposes.

3

Basically, you are saying that once you deploy your application it runs all the time while consuming funds, even if you are not using it… otherwise, you have to recreate the index and need to wait between 10 min to few hours when index recreated every time you need it. So, it is not pay for you what you consume model…

4

Yes, Pinecone’s pricing model is based on payment by the hour, not by usage. This is an important consideration, especially when planning to use commercial applications.
(I have been on the Standard plan since March.)

If you are trying things out for personal development purposes, the Free plan may be sufficient.
Are you using it for commercial purposes?

I personally feel that the risk of the service not working well is higher than the benefit of frequently cold storing Index to reduce the billing amount.
I think it is not unusual for commercial use to charge several tens of thousands of yen per month.
(starting at $70/month)

There are several possible ways to reduce the billing amount outside of cold storage.