Help Needed: Persistent SSL Handshake Failures with Pinecone Vector Storage
The Problem
I’m experiencing a persistent SSL handshake error when storing vectors in Pinecone from my Cloud Run artifact processor. The specific error is:
Copy
SSLError(SSLEOFError(8, '[SSL: UNEXPECTED_EOF_WHILE_READING] EOF occurred in violation of protocol (_ssl.c:1006)'))
The Strange Part
What makes this particularly puzzling is that my diagnostic endpoint succeeds in connecting to Pinecone and storing vectors, but the main service fails consistently. Both use the same code pattern to create Pinecone clients and store identical test vectors.
Please note: I have seen posts that this error is caused by malform vectors or improper upsert , a dummy vector is being used to eliminate this as an issue, my upsert follows the documents to the best of my knowledge.
Environment Details
- Cloud Run with 8GB memory allocation
- Pinecone index: 1024 dimensions, cosine similarity
- Both operations use identical test vectors:
[0.1] * 1024 - Both create fresh Pinecone clients before each operation
- The failing service call happens after image processing is complete
Troubleshooting Steps Taken
- Verified credentials are working (diagnostic call succeeds)
- Replaced actual vectors with fixed test vectors to eliminate vector-related issues
- Confirmed dimensions match between config and Pinecone index (1024)
- Examined the retry mechanism (fails consistently after 3 retries)
Logs
The error consistently appears in logs after processing completes:
Copy
ERROR:database.pinecone_service:Error storing vector in Pinecone: HTTPSConnectionPool(host='api.pinecone.io', port=443): Max retries exceeded with url: /indexes/infitwin-01-pinecone-db (Caused by SSLError(SSLEOFError(8, '[SSL: UNEXPECTED_EOF_WHILE_READING] EOF occurred in violation of protocol (_ssl.c:1006)')))"
Has anyone encountered similar G Cloud SSL handshake issues with Pinecone, particularly the UNEXPECTED_EOF_WHILE_READING error? Any insights on what might cause this to happen in one context but not another would be greatly appreciated.
Code Comparison
Here’s the diagnostic code that successfully connects to Pinecone:
From diagnostics.py - test_pinecone_vector_storage function
def test_pinecone_vector_storage() → Dict[str, Any]:
“”"
Test storing a minimal vector in Pinecone to verify full operation.
Returns:
Dict with test results
"""
results = {
"timestamp": datetime.now().isoformat(),
"success": False,
"vector_dimension_match": False,
"storage_time_ms": 0,
"errors": []
}
try:
# Get Pinecone configuration
try:
from database.database_config import VECTOR_DIMENSION, PINECONE_INDEX, PINECONE_API_KEY
except ImportError:
VECTOR_DIMENSION = int(os.getenv("VECTOR_DIMENSION", "1024"))
PINECONE_INDEX = os.getenv("PINECONE_INDEX", "")
PINECONE_API_KEY = os.getenv("PINECONE_API_KEY", "")
if not PINECONE_API_KEY:
results["errors"].append("Pinecone API key not configured")
return results
if not PINECONE_INDEX:
results["errors"].append("Pinecone index not configured")
return results
# Create test vector with correct dimensions
test_vector = [0.1] * VECTOR_DIMENSION
test_id = f"diagnostic_test_{int(time.time())}"
# Fall back to direct Pinecone client
try:
from pinecone import Pinecone
pc = Pinecone(api_key=PINECONE_API_KEY)
# Check index dimensions
index_info = pc.describe_index(PINECONE_INDEX)
actual_dim = getattr(index_info, "dimension", None)
if actual_dim is not None:
results["vector_dimension_match"] = (actual_dim == VECTOR_DIMENSION)
if not results["vector_dimension_match"]:
results["errors"].append(
f"Vector dimension mismatch: config={VECTOR_DIMENSION}, index={actual_dim}"
)
return results
# Get index
index = pc.Index(PINECONE_INDEX)
# Try storage
start_time = time.time()
result = index.upsert(
vectors=[{"id": test_id, "values": test_vector, "metadata": {"test": True}}],
namespace="diagnostic"
)
end_time = time.time()
results["success"] = True
results["storage_time_ms"] = (end_time - start_time) * 1000
return results
except Exception as e:
results["errors"].append(f"Direct Pinecone operation failed: {str(e)}")
return results
except Exception as e:
results["errors"].append(f"Unexpected error: {str(e)}")
return results
And here’s the service code that fails with SSL errors:
From pinecone_service.py - store_vector function
def store_vector(vector_id: str, embedding: List[float], metadata: Dict[str, Any], namespace: str = PINECONE_NAMESPACE) → Tuple[bool, Dict[str, Any]]:
“”"
Store a vector in Pinecone following the exact pattern in the documentation.
Args:
vector_id: Unique identifier for the vector
embedding: Vector embedding data
metadata: Metadata to store with the vector
namespace: Pinecone namespace
Returns:
Tuple of (success, result)
"""
embedding = [0.1] * 1024 # Use a fixed test vector of 1024 dimensions
try:
# Import Pinecone
from pinecone import Pinecone
# Initialize client
pc = Pinecone(api_key=PINECONE_API_KEY)
# Connect to index
index = pc.Index(PINECONE_INDEX)
# Store the vector following the docs
index.upsert(
vectors=[
{
"id": vector_id,
"values": embedding,
"metadata": metadata
}
],
namespace=namespace
)
logger.info(f"Successfully stored vector in Pinecone: {vector_id}")
return True, {"success": True}
except Exception as e:
logger.error(f"Error storing vector in Pinecone: {str(e)}")
return False, {"error": str(e)}