Updating indices vs. needing a new one every run

Pinecone supports live index updates, you do not need to create a new index every time. We also have a new gRPC flavor of the Pinecone Python client that allows us to upload ~5000 vectors / second. A few million vectors can therefore be uploaded in a few minutes. The non-gRPC version of the client can upload ~500 vectors / second.
If you have billing activated, there are other options for uploading data and you can get in touch with support for that.