404 error: new API + serveless index

[younes.souilmi]

Hi guys,

upon migrating to the new Pinecone Node.js client (2.0.1) I am getting the following error:
call to https://api.pinecone.io/indexes/my-index-name returned HTTP status 404
the old fashioned index url works fine. my-index-name is serverless
Any idea what’s the issue?

Thanks in advance

[jesse]

Hi @younes.souilmi. Can you share a bit more about what you’re trying to do and the relevant code?

In the meantime, this docs page has examples that might help. Be sure to select the “Javascript” tab in the code blocks.

[younes.souilmi]

Hi Jesse and thanks for the quick response.
I am trying to build a new Pinecone index (serverless) from an old Pinecone index (starter).
Here are the Pinecone-related code snippets:

const { Pinecone } = require('@pinecone-database/pinecone'); // "@pinecone-database/pinecone": "^2.0.0"

// Initialize Pinecone clients for old and new indexes
const pineconeClientOld = new Pinecone({
  apiKey: 'oldindex-apikey'
});

const pineconeClientNew = new Pinecone({
  apiKey: 'newindex-apikey'
});

const indexOld = pineconeClientOld.index('oldindex-name');
const indexNew = pineconeClientNew.index('newindex-name');




async function upsertNewEntry(entry, newEmbedding) {
    // Format the data for upsert according to the new Pinecone API
    const record = [{
        id: entry.id,
        values: newEmbedding,
        metadata: entry.metadata
    }];

    // Upsert the entry with new embedding and all metadata into the new namespace
    await indexNew.namespace('my-namespace').upsert(record);
}



exports.rebuildPineconeDatabase = functions.runWith({ timeoutSeconds: 300 }).https.onRequest(async (req, res) => {
    try {
        // Fetch all entries from the existing index
        console.log("Fetching entries...");
        const entries = await fetchAllEntries();
        console.log(`Fetched ${entries.length} entries`);

        // Process each entry
        for (const entry of entries) {
            console.log(`Processing entry: ${entry.url}`);
            const newEmbedding = await generateEmbedding(entry);
            await upsertNewEntry(entry, newEmbedding);
        }

        res.send('Database rebuild process completed successfully.');
    } catch (error) {
        console.error('An error occurred:', error);
        res.status(500).send('Error during database rebuild.');
    }
});

On execution the upsertNewEntry function triggers a 404 error

call to https://api.pinecone.io/indexes/startupideas-prod returned HTTP status 404

accessing the index via the old format url https://controller.{environment}.pinecone.io works.

Let me know if any complementary details are needed.

[jesse]

That error suggests that the new serverless index doesn’t already exist, and I don’t see you creating that new index in the code above. Are you sure the index already exists? You can log into the Pinecone console and check the Indexes tab of your project, or you can list your indexes like this:

import { Pinecone } from '@pinecone-database/pinecone'

const pc = new Pinecone({ apiKey: 'YOUR_API_KEY' })

await pc.listIndexes();

[younes.souilmi]

Thanks Jesse. To your questions:

• List item the index (serverless) I am hitting exists already; created via the Pinecone console (UI). It appears in the console as active

• List item the indexes API endpoint (https://api.pinecone.io/indexes) returns the above index as well

It seems it exists!
I now suspect the issue is due to the fact that I am initializing/using 2 different clients within the above function. Are there any specific considerations in such a setup?

[jesse]

Another suggestion, @younes.souilmi: Can you double-check that the API keys you’re using are valid for the indexes you’re working with? For example, pineconeClientOld needs to use an API key from the project containing oldindex-name, and pineconeClientNew needs to use an API key from the project containing newindex-name.

[younes.souilmi]

Thanks for the suggestion. API keys are correct, the problem is most likely due to the fact may function used 2 pinecone clients: the problem disappear if I keep a single client.
I refactored my code to avoid having to use 2 different pinecone clients in a same function.

[jesse]

I’m glad you got this resolved, @younes.souilmi, but I’m also curious why I didn’t run into this in my own testing. I’ll ask our SDK team for more insight.