Similarity score treshold not working - langchain

Support
1 
    # Set up the conversational retrieval chain
    chain = ConversationalRetrievalChain.from_llm(
        ChatOpenAI(model_name="gpt-4o", temperature=0, streaming=True),
        chain_type="stuff",
        retriever=docsearch.as_retriever(
        search_type="similarity_score_threshold",
        search_kwargs={'score_threshold': 0.8}
        ),
        memory=memory,
        return_source_documents=True,
    )

I am trying to return only documents with similarity score > 0.8 but for some reason this is not working, it’s always returning 5 results (i guess default is k:5). Does anyone have idea how to make this to work?

2

Hi @Artic,

Welcome to the Pinecone forums and thanks for your question!

Could you please share the code you’re using to index the documents?

On your Pinecone dashboard, how many vectors do you see and what happens if you query results directly from your console - do you see expected results?

Best,
Zack

3

Hi Zack,

Thanks for your response! Here is the code I am using to index the documents in Pinecone:

# Initialize the database (create tables if they don't exist)
init_db()

# Assuming the namespace and embeddings are the same as in the original app
index_name= os.getenv('INDEX_NAME')
namespace = os.getenv('NAMESPACE')

embeddings = OpenAIEmbeddings()

# Connect to the Pinecone index
docsearch = Pinecone.from_existing_index(
    index_name=index_name, embedding=embeddings, namespace=namespace
)

In my Pinecone dashboard, I can see the expected number of vectors. However, when querying the results, it seems to always return 5 results regardless of the similarity score threshold I set. Any insights on this would be appreciated.

4

For additional context, complete code is somewhat similar to this example:

5

We do not currently offer a minimum score filter. Pinecone queries will always return the top_k most similar records in the query result.

For more details, please see: Filtering by score

1 Like