I’m currently working on a project to identify most similar sentences in a document to a specified query sentence. I’m using a pre-trained SBERT bi-encoder model for this but have just started looking at using a cross-encoder model to refine a shortlist of results. Would it actually be better for me to fine tune the bi-encoder model rather than use cross-encoder?
1 Like
Hi Laura, it depends on your use-case.
Cross-encoder models don’t produce sentence embeddings and so performing search or clustering can be much slower as you must perform a full cross-encoder inference for comparing every single pair. However, they are more accurate and require less data to fine-tune.
Bi-encoder models do produce sentence embeddings, so we perform a single inference for each vector and then store the vectors to be compared later, so then all we need to calculate at comparison time is a metric like cosine similarity.