Are embeddings only for documents? What about passages, sentences, words?
Do they work for documents of arbitrary length?
1 Like
Depending on the embedding model they can work for words, sentences, passages, and any text up to ~2 pages of text (in the case of OpenAIs embedding model, most open source models can handle ~1 paragraph). If you want to go beyond that, and embed a document of 10 pages, you can average the embeddings of smaller sections (paragraphs/pages). Averaging in high dimensional spaces makes sense although different models handle it better than others