Filter query not working

Search, No Filter
,
1

Problem: Query does not return result when filter is applied. Returns results fine when the filter is not mentioned in the query. Metadata has filters and no apparent semantic issues.

This does not work even when i use the pinecone console client to upsert and query data. Shows the same behavior. This is the case even with the sample data shown in the documentation. i’ve run out of ideas. Any help is appreciated.
Upsert JSON

{
  "vectors": [
    {
      "id": "example-vector-1",
      "values":[....]
      "metadata": {
        "genre": "documentary",
        "year": 2019
      }
    }
  ]
}

Query JSON

{
  "vector": [...
  ],
  "topK": 5,
  "includeMetadata": true,
  "includeValues": false,
  "namespace": "",
  "filter":{
    "genre":{"$eq":["documentary"]}
  }
}

Tried other query combinations for filters like $in and without the operator . The query returns results if i remove the filter section.
Any help is much appreciated.

2

Hi @cvenkatesh

Your query structure appears to be correct, but it seems that your filter section is expecting a direct comparison, while an array (list) is provided instead.

In your filter, you have:

"filter": {
  "genre": {
    "$eq": ["documentary"]
  }
}

Here, the “$eq” operator is used for direct comparisons and typically does not expect an array (in this case, “documentary” is within an array). Try changing your query filter to:

"filter": {
  "genre": {
    "$eq": "documentary"
  }
}

This is likely to yield the expected results, as it’s directly comparing the genre to the string “documentary”.

If the filter notation is incorrect, you might expect to see an error message like the following: "message": "the $eq operator must be followed by a string, boolean or a number, got [\"documentary\"] instead",

Please check this part and try again. I hope this helps! Let me know if you need any further assistance.

3

@dra thanks for responding. My bad. I made a mistake in copy pasting and editing the payload. It does not work even with the proper $in or $eq syntax. I tried it a friend’s database. same result. Its not working for him as well. Attaching screenshots. Again, thanks for your help.

{
  "vector": [...  ],
  "topK": 5,
  "includeMetadata": true,
  "includeValues": false,
  "namespace": "",
  "filter":{
    "genre":{"$eq":"documentary"}
  }
}

Screenshot 2023-07-05 at 7.01.42 AM
Screenshot 2023-07-05 at 7.01.42 AM2154×796 64.2 KB

4

Same thing with a filter

Screenshot 2023-07-05 at 7.02.37 AM
Screenshot 2023-07-05 at 7.02.37 AM1828×704 45.1 KB

5

@cvenkatesh
This is just a guess, but are you specifying “metadata_config” when creating the Index?
If “metadata_config” is not appropriately set, the filter will cease to function.

Try recreating the Index without setting the “metadata_config”.

I think it will look something like this:

{
    "metric": "cosine",
    "pods": 1,
    "replicas": 1,
    "pod_type": "p1.x1",
    "name": "test-index",
    "dimension": 8
}
6

@dra thanks very much. You were spot on and it fixed the problem… I had copy pasted the index creation code and forgot to change the metadata_config. It did not strike me to check the index creation parameters when debguging this issue.
You helped resolve this. Thanks much.