How are people syncing large amounts of data into Pinecone currently? Is it purely with custom scripts?
I see one of the YC startups this year (Sidekick) start doing a “Fivetran for LLMs” which takes data from SaaS apis like Notion, Zendesk, Slack, etc and syncs it to Pinecone/Weaviate.
However, it seems like most companies already keep a full representation of their data in a data warehouse (BigQuery/Snowflake) already using tools like Fivetran/Airbyte/Stitch/etc.
Is there a tool that helps sync from data warehouses into Pinecone?