Hello Pinecone Community,
We’re facing a critical issue with our production system, and would appreciate your insight.
System Context:
-
Backend: FastAPI (Python 3.11)
-
Vector DB: Pinecone (serverless, free and considering upgrade to Standard)
-
ORM: SQLAlchemy with PostgreSQL (hosted on Railway)
-
Framework: LangChain
-
LLM/Embeddings: OpenAI GPT-4o-mini, text-embedding-ada-002
How it works:
-
Users upload PDF/TXT files via FastAPI endpoint.
-
Text is extracted (PyPDF2/pdfplumber), split into chunks (RecursiveCharacterTextSplitter).
-
Chunks are embedded and upserted to Pinecone under namespace format:
user_{user_id}_cond_{condo_id}. -
Each condominium and user has an isolated namespace; there is strict separation by design.
-
Queries are routed to Pinecone via LangChain with the respective namespace.
Symptoms:
-
Upload endpoint reports success (no errors shown).
-
Chunks are successfully processed and upserted to Pinecone (confirmed by logs).
-
When querying, LangChain searches the correct namespace (confirmed by logs).
-
Query always returns empty/None, or otherwise “No documents found,” even immediately after upload.
What we’ve checked:
-
Document model strictly matches DB schema (no extraneous or missing fields).
-
Upload process and query both use the same embedding model and chunking parameters.
-
Namespace string is identical between upsert and query.
-
Pinecone’s
describe_index_stats()shows the namespace being created, but sometimes the vector count is inconsistent or remains zero after upload. -
PostgreSQL database does not report any constraint violations.
What we’ve tried:
-
Re-uploaded files with minimal and default metadata.
-
Completely cleared and recreated Pinecone namespaces and indexes.
-
Stripped any special characters from namespace names and file names.
-
Debugged LangChain pipeline with smaller texts and synthetic documents: still no results.
Questions:
-
What are the most common causes for successfully upserted vectors being “invisible” to queries (especially in serverless/Starter tier)?
-
Are there edge cases in metadata, namespace, or embedding config that would cause vectors to not be returned or to be dropped silently?
-
Could there be issues with chunking length, batch size, or API rate limits where the upsert reports no error but nothing is persisted?
-
Is there a restricted feature set on Starter tier that could block retrieval in isolation scenarios?
-
Any recommended debugging steps with Pinecone CLI/API to confirm vectors, or to test “raw” queries bypassing LangChain?
We are ready to provide code samples/logs and can migrate to Standard plan if that helps unlock advanced diagnostics/support.
Thank you for any help,
Team AlexandraLex