Vector dimension does not match the dimension of the index

Support
1

Running into error upserting data to index. I followed steps from OpenAI

  1. I created index swith dimension 8 (max)
  2. Ran steps OpenAI

Getting the following error:
Reason: Bad Request

HTTP response headers: HTTPHeaderDict({‘content-type’: ‘application/json’, ‘date’: ‘Fri, 31 Mar 2023 21:56:18 GMT’, ‘x-envoy-upstream-service-time’: ‘2’, ‘content-length’: ‘101’, ‘server’: ‘envoy’})

HTTP response body: {“code”:3,“message”:“Vector dimension 1536 does not match the dimension of the index 8”,“details”:}

1 Like
2

Hi @sjai,

Indexes must be created with a dimension that matches the model’s embedding dimension. The dimension of an index cannot be changed after creation. You will need to create a new index with 1536 dimensions to match OpenAI’s text-embedding-ada-002 model.

Thanks!

1 Like
3

I have less dimensions than my index of 1536 and get the following messages {“code”:3,“message”:“Vector dimension 1409 does not match the dimension of the index 1536”,“details”:}

OpenAI text embedding didn’t gave me more than 1409. Should it be possible to add 1409 dementions to an 1536 index database? How?

4

Hi @rensehuijg

if your index dimensions are set to 1536 then you can not store vectors of any other dimesion. OpenAi embeddings (if you use the ada model) will always give embeddings with dimension 1536.

Hope this helps

5

Thanks Jasper, I found it. I didn’t save the duplicate vector values in my system that where returned form ada model.

6

Hey, Can you pls help on how to determine the dimension so that we can create relevant index
@rensehuijg @Jasper

7

Hi @vr14

the dimension of your index are determined by which embeddings you will be using (OpenAi, BERT, etc.)

When you know that, you can set the dimensions of the index and start with upserting :slight_smile:

Hope this helps

1 Like
8

for embeding openai it you can find it here: OpenAI Platform

9

Yes, Thanks a lot :slight_smile:

10

Thanks a lot for the links :slight_smile:

11

Reason: Bad Request
HTTP response headers: HTTPHeaderDict({‘content-type’: ‘application/json’, ‘date’: ‘Sat, 19 Aug 2023 11:10:03 GMT’, ‘x-envoy-upstream-service-time’: ‘1’, ‘content-length’: ‘104’, ‘server’: ‘envoy’})
HTTP response body: {“code”:3,“message”:“Vector dimension 1024 does not match the dimension of the index 1019”,“details”:}

I tried changing dimensions of the index but still I get the same error. Any idea on how to fix this?

12

This topic was automatically closed 24 hours after the last reply. New replies are no longer allowed.