Assistant list files filter not working

Support
1

Im using pinecone assistant and I have more than 1400 files uploaded to it, when I try to run assistant.listFiles with a filter inside it returns 1400 files instead of the ones I’ve filtered, no type of filtering works, I’ve tried multiple

const pc = new Pinecone({apiKey: pineconeApiKey.value()});
const assistant = pc.Assistant(pineconeAssistantName.value());
const listed = await assistant.listFiles({
filter: {
metadata: {
sessionId: sessionId,
source: “extraContent”,
},
},
});

const listed = await assistant.listFiles({
        filter: {
            "$and": [
                {"sessionId": {"$eq": sessionId}},
                {"source": {"$eq": "extraContent"}},
            ],
        },
    });

const listed = await assistant.listFiles({
        filter: {metadata: {$contains: {sessionId}}},
    });

const listed = await assistant.listFiles({
        filter: {source: "laksjdf"},
    });

const listed = await assistant.listFiles({
        filter: {source: {$in: ["asdfasdfasdf"]}},
    });

const listed = await assistant.listFiles({
        filter: {source: {$eq: "asldkjf"}},
    });


const listed = await assistant.listFiles({
        filter: {sessionId: sessionId},
    });

None of these work and multiple others I’ve tried

The only thing the docs shows is this, but on the listFiles filter must be an object:
PINECONE_API_KEY="YOUR_API_KEY"ASSISTANT_NAME="example-assistant"ENCODED_METADATA="%7B%22document_type%22%3A%20%22manuscript%22%7D" # URL encoded metadata - See ``w3schools.com/tags/ref_urlencode.ASPcurl`` -X GET "https://prod-1-data.ke.pinecone.io/assistant/files/$ASSISTANT_NAME?filter=$ENCODED_METADATA" \ -H "Api-Key: $PINECONE_API_KEY"

Also, the metadata of the files is correct I believe:
“Metadata over here {\“hash\”:\“016cfe2bd9d034e1c47141be2982641b8d72a8cca515299165e48fe6c4d15e75\”,\“sessionId\”:\“3a45b0d2acff706155650bfd288a-proposal\”,\“source\”:\“extraContent\”,\“type\”:\“simulationResult\”}” }

2

Hi @ikaroisdm - Thanks for sharing this!

I’m also able to reproduce this with the node/js SDK but not with Python. I’ll check with the team to see if it’s documented incorrectly or if there’s a bug somewhere and get back to you.

1 Like
3

I did confirm with the team that this is a bug with the javascript SDK. We have a fix in process and I’ll reply back here when ready.

4

This is now fixed in the latest version of the javascript SDK, 6.1.3. I have verified this change myself, however you’ll need to make a minor change to the format of the metadata filter, removing the metadata key, which is not currently reflected in our documentation (work in progress).

Old, broken:

const filteredFiles = await assistant.listFiles({
    filter: { metadata: { external_file_name: '2023.md' } }
});

New, working:

const filteredFiles = await assistant.listFiles({
    filter: { external_file_name: '2023.md' },
});

or:

const filteredFiles = await assistant.listFiles({
    filter: { external_file_name: { '$ne' : '2023.md' } },
});

Give it a try and let us know how it goes.

5

Looking good, thanks, appreciate it.

1 Like