I am following the very interesting tutorlal: Semantic Search - Typescript. I have a problem inserting into Pinecone the embeddings of the Test File. The Index create to store the embeddings of the @xenova transformer has dimension of 384. However when I run the hereunder code I find that each ‘question’ is translated in an embedding of different size so I get vectors of different dimensions that does not match the index dimension (384). How can I ask the Xenova transformer pipeline to respect the index size?
Thanks
Luca
async embed(text: string): Promise {
const result = this.pipe && (await this.pipe(text));
return {
id: uuidv4(),
metadata: {
text,
},
values: Array.from(result.data),