Pinecone ERROR Vector dimension 768 does not match the dimension of the index 384

lisboua · June 4, 2024, 1:44pm

I’m building a chatbot RAG using HuggingFace, Mistral, LangChain and Pinecone.

I have a Python Script to watch changes in my MongoDB collection and send the data to Pinecone as a vector.

import os
from pymongo import MongoClient
from pinecone import Pinecone, ServerlessSpec
from pymongo.errors import OperationFailure
from sentence_transformers import SentenceTransformer, util
from certifi import where  # Import certifi library

# mongodb stuff
client = MongoClient(
    "my-mongodb-uri",
    tls=True,  # Enable TLS encryption
    tlsAllowInvalidCertificates=False,  # Don't allow invalid certificates
    tlsCAFile=where()  # Use certifi library for CA bundle
)
db = client['test']
collection = db['reflections']

# Pinecone initialization
pc = Pinecone(api_key='my-pinecone-key')
index = pc.Index("langchain-demo")

# transformer stuff
model = SentenceTransformer('sentence-transformers/all-MiniLM-L6-v2')
test_vector = model.encode("This is a test string")
print("Dimension of test vector:", test_vector.shape)

# Watch for changes
try:
  cursor = collection.watch()
  for change in cursor:
    print("Change detected:", change)
    if change['operationType'] == 'insert':
      document = change['fullDocument']
      vector = model.encode(document['content']).tolist()
      print("Extracted Vector:", vector)

      # Extract document ID from ObjectId
      document_id = str(document['_id'])
      user_id = str(document['user'])
      created_at = document['createdAt']

      # Wrap upsert call with empty vector check
      if vector:  # Check if vector is not empty
        metadata = {'user_id': user_id, 'created_at': str(created_at)}
        index.upsert([(document_id, vector, metadata)])

    elif change['operationType'] == 'update':
      document_id = str(change['documentKey']['_id'])
      updated_fields = change['updateDescription']['updatedFields']
      if 'content' in updated_fields:
            vector = model.encode(updated_fields['content']).tolist()
            index.upsert([(document_id, vector, metadata)])

    elif change['operationType'] == 'delete':
      document_id = str(change['documentKey']['_id'])
      index.delete(ids=[document_id])

except OperationFailure as e:
  print("Error watching collection:", e)
except Exception as e:
  print("An error occurred:", e)

And this is working fine, here a entry from my Pinecone Index:

2

SCORE
0.0179

ID
665882f313b3c25b81fe46f7

VALUES
0.0780179054, 0.0113870166, -0.118430212, 0.0226640049, 0.00780098559, -0.0503292121, -0.0238182824, 0.0469664969, -0.0270168502, 0.0369489267, 0.0325907134, -0.0476229116, -0.0786545724, 0.024753511, 0.027189346, -0.0125029637, 0.00541313691, -0.0757853836, 0.0212387145, -0.0110286493, 0.0117090391, -0.0193786472, 0.0253583211, 0.033223819, -0.047959242, -0.0769486949, -0.0650763139, 0.0521994196, -0.0230765603, -0.0348826386, 0.101876184, 0.0702511, -0.00723150745, -0.000215011096, 0.0366995223, -0.0220702309, 0.0689907297, -0.00627420563, 0.00119058753, -0.0264558308, 0.0431874134, -0.010541168, -0.0586702935, -0.0786846578, 0.0561939925, -0.0782630071, -0.0292404443, 0.0599261038, 0.106215701, 0.00162592565, 0.00396974338, -0.084600091, -0.108621657, 0.0980778411, 0.0299655739, 0.128577992, -0.0200762413, -0.0653520674, 0.0102492804, 0.0322388411, 0.0313118957, 0.0121116294, -0.0409101136, 0.0448457114, -0.0308046602, -0.0109536722, -0.0264064465, -0.0215834882, 0.0305376686, -0.0735122, 0.0205106437, 0.0107715921, -0.0321323, -0.0110485516, 0.0149518345, 0.154741883, -0.0143986866, -0.0952939615, 0.0817126408, 0.0272761323, -0.118013233, -0.0656163543, 0.0275144801, -0.0715595558, 0.00360014034, -0.0505540147, 0.0993294567, -0.0181424897, -0.0340541303, -0.0191219747, 0.0576984696, -0.016041344, -0.0117336037, 0.0854231268, 0.000578647654, 0.054585509, 0.0607333779, 0.0251228772, 0.0586750954, 0.0542409495, -0.0147840325, 0.0171087608, 0.0243709181, 0.0218990557, -0.0393281505, 0.0587186627, 0.0294246785, -0.0300889853, -0.104504265, 0.01126011, 0.0200448446, 0.0346874595, -0.0424731597, -0.0501565561, -0.0336206853, 0.0125492243, -0.0602271, -0.0255396832, -0.0807785168, 0.0296369921, 0.0213591736, 0.0517560728, -0.0236208886, -0.00668002106, 0.0482311361, 0.00484784134, -0.00671314076, 2.48858068e-33, 0.13539581, -0.0779394507, 0.0531481355, -0.0325026, 0.0539878421, 0.0422413386, 0.00721841305, 0.0636715069, -0.0271209199, 0.0237933304, -0.012324837, 0.0318006687, -0.00415049167, -0.0791502222, 0.00476903608, 0.0398657173, -0.0222511888, 0.00637130346, -0.026622, 0.0662136674, 0.0718791559, -0.0128180161, -0.00474599516, -0.0390690714, 0.0125142019, 0.050622154, -0.0718687624, -0.014537137, -0.0000649832582, 0.0659983233, -0.0874705464, -0.0598177947, -0.0425723866, 0.116614014, 0.00950913411, 0.00521008205, 0.00299354899, -0.0490386747, -0.101944029, -0.00193977414, 0.0737567469, 0.0489828475, 0.0479842573, -0.0121582411, -0.0683459863, 0.0465017371, 0.0244375356, -0.0415155031, 0.0733279511, 0.0411045067, -0.0990792513, -0.0168382581, -0.128719345, 0.0162787139, 0.00180200755, 0.0361118391, 0.00714902952, 0.0225948617, 0.0598230809, -0.027883, -0.0634493604, 0.01690959, -0.00616647629, 0.102524318, -0.00962299388, -0.00800243765, -0.0349691175, -0.067561388, 0.00328797475, 0.038413655, -0.0277767256, 0.0493801422, -0.0626680329, 0.0100066448, 0.0245917551, -0.0696575642, 0.0435989201, -0.0336911827, -0.11178454, 0.0231342558, 0.0650513396, -0.0489158109, -0.118925206, 0.0456594117, -0.000165144767, 0.0263350084, -0.0126295025, -0.00819321815, 0.02433425, -0.130400553, 0.0154193891, 0.104591407, 0.0529140271, -0.0677612349, 0.006422868, -3.15768235e-33, 0.0266072527, -0.0650565699, 0.0751486719, 0.0352391638, 0.0913304538, -0.0166126471, 0.0618819185, 0.0104150325, -0.0678095892, -0.0207110438, 0.0114265652, -0.0825057179, -0.0197282247, -0.0340087563, -0.0661914572, -0.0287549552, -0.00654957304, -0.114685677, -0.00108200416, 0.032330893, -0.0842154548, 0.0676763728, 0.0241372753, 0.0217358116, 0.00449138787, -0.110978812, -0.00896465, -0.113323905, -0.0244096816, 0.0437684767, 0.0635139272, 0.0932535827, -0.0722304285, 0.0176913701, -0.0259954911, 0.0359196104, 0.0873655, -0.0387332886, 0.0215564184, 0.087338239, 0.0205500964, 0.0888643935, -0.0694097877, -0.0502204783, 0.0232924949, 0.0244588424, -0.0220373627, 0.0513052084, 0.0780876428, -0.0470482521, -0.0335381292, -0.012535166, 0.0251501985, -0.0287306085, 0.0485889763, -0.0687309802, 0.0517236702, -0.0322456844, -0.00322602759, 0.0309814233, 0.0314734243, -0.00763608702, -0.0108202593, -0.0168912616, 0.0146153346, 0.0058498932, -0.0463481061, -0.0149578694, 0.00604433473, 0.0156758986, -0.00293513224, -0.0301138815, -0.0343872979, 0.0298203304, 0.0256143659, -0.0255653039, -0.0625488907, -0.0271613039, 0.0419909954, 0.0695206523, 0.062871322, -0.034802638, 0.0640893504, 0.0120009324, 0.0246663559, -0.0750950128, 0.00124487793, 0.0673942715, -0.133396164, 0.0429567732, -0.00271692313, -0.00318611646, -0.0404472388, 0.0490305126, 0.0105391946, -1.60218168e-8, -0.0698085204, 0.000768667669, -0.00646897545, 0.0643798038, -0.0233389419, 0.00319286506, 0.0344442315, 0.00670952629, 0.0701468587, 0.0244643614, 0.0203008968, 0.0470311418, -0.0328549594, 0.0701879337, -0.03013069, 0.0344369523, 0.0444342941, 0.0185571574, 0.0324901864, 0.028787395, -0.00257497164, 0.016713731, 0.00826486666, -0.108173572, 0.0234305598, 0.0513667837, 0.0751283243, 0.0887183845, 0.00413105683, -0.00771792512, -0.00882443134, 0.010960429, 0.0386422388, -0.0143630169, 0.0658110827, 0.0273497198, 0.0315157361, 0.0515595078, -0.0557875521, 0.0250054803, -0.00490543433, 0.0140227228, 0.0185526628, -0.063243337, -0.0253285952, -0.0361511707, 0.0092082452, 0.109378465, -0.0141137447, -0.0900633484, 0.0360222086, -0.0720594451, 0.0374338441, -0.0112864282, 0.0303963758, 0.0577182584, -0.0215295721, -0.0536189973, -0.0685039461, -0.0390378311, 0.0779438391, -0.070971, -0.015317292, -0.0375189707

METADATA
created_at: "2024-05-30 13:45:23.389000"
user_id: "65d8937f6408bf2c0ca8d264"

The problem is when I try to ask something to my chatbog. Here is my code:

from transformers import AutoTokenizer
from pinecone import Pinecone, ServerlessSpec
from dotenv import load_dotenv
import os
from langchain_community.embeddings import HuggingFaceEmbeddings
from langchain_community.vectorstores import Pinecone as PineconeStore
from langchain_huggingface import HuggingFaceEndpoint
from langchain.prompts import PromptTemplate
from langchain.schema.runnable import RunnablePassthrough
from langchain.schema.output_parser import StrOutputParser

class ChatBot():
    def __init__(self):
        load_dotenv()

        self.pc = Pinecone(api_key=os.getenv("PINECONE_API_KEY"))
        self.index_name = "langchain-demo"
        self.index = self.pc.Index(self.index_name)

        # Initialize tokenizer for the language model
        self.tokenizer = AutoTokenizer.from_pretrained('mistralai/Mixtral-8x7B-Instruct-v0.1')

        # Initialize embeddings and Pinecone store for document retrieval
        self.embeddings = HuggingFaceEmbeddings()
        self.docsearch = PineconeStore.from_existing_index(self.index_name, self.embeddings)

        # Setup LLM with tokenizer
        self.llm = HuggingFaceEndpoint(
            repo_id='mistralai/Mixtral-8x7B-Instruct-v0.1',
            temperature=0.8, top_p=0.8, top_k=50,
            huggingfacehub_api_token=os.getenv('HUGGINGFACE_API_KEY')
        )

        # Define the prompt template
        self.template = PromptTemplate(template="""
            You are a seer. These Humans will ask you questions about their life. 
            Use the following piece of context to answer the question. 
            If you don't know the answer, just say you don't know.
            You answer with short and concise answers, no longer than 2 sentences.

            Context: {context}
            Question: {question}
            Answer: 
        """, input_variables=["context", "question"])

        self.rag_chain = (
            {"context": self.docsearch.as_retriever(), "question": RunnablePassthrough()}
            | self.template
            | self.llm
            | StrOutputParser()
        )

    def get_user_data_from_pinecone(self, user_id):
        results = self.index.query(
            vector=[0]*384,
            filter={"user_id": user_id},
            top_k=10,
            include_metadata=True,
            include_values=True
        )
        return results
    

# Usage example outside the class
bot = ChatBot()
user_id = input("Enter your user ID: ")
user_data = bot.get_user_data_from_pinecone(user_id)

question = input("Ask me anything: ")
# Tokenize the question
inputs = bot.tokenizer(question, return_tensors="pt")
result = bot.rag_chain.invoke(question)
print("AI's response:", result)

Here is the ERROR I received:

HTTP response body: {"code":3,"message":"Vector dimension 768 does not match the dimension of the index 384","details":[]}

My Pinecone Configuration:

METRIC cosine DIMENSIONS 384 CLOUD aws AWS REGION us-east-1 TYPE Serverless

ZacharyProser · June 4, 2024, 1:50pm

Hi @lisboua, and welcome to the Pinecone community forums!

Thank you for your question.

This error message indicates that the length of the vectors array you’re attempting to upsert to Pinecone and the length of the vectors array your Pinecone index expects are not the same.

One useful sanity check can be to print(len(my_vectors)) and ensure the value is exactly 384. If it is not, you’ve found your issue.

If you changed your embedding model or index configuration while you were working, that would explain why you’re getting this error.

Here’s some more info:

Vector Dimension Mismatch: You’re trying to upsert vectors with a dimension of 768 into an index configured for vectors with a dimension of 384. This mismatch will cause the error.
Possible Causes:

Index Configuration: The index was created with a specific dimension (384 in this case). Any vectors upserted to this index must match this dimension.
Embedding Model: The embedding model used to generate the vectors may have changed. For example, switching from one model to another that outputs vectors of a different dimension.
Code Changes: There might have been changes in the code that generates or processes the vectors, leading to a different dimensionality.

Solutions:

Check Index Configuration: Verify the dimension of the index. This can be done by checking the index’s metadata in Pinecone.
Check Embedding Model: Ensure that the embedding model consistently generates vectors of the expected dimension. If the model changed, update the index or revert to the original model.
Update Vectors: If the new vector dimension is intentional, recreate the index with the new dimension (768) and upsert the vectors again.

In your specific context, ensure the dimensions of your vectors match the index configuration and take steps to either update the vectors or the index accordingly.

Hope that helps!

Best,
Zack