# Error using pinecone-client at lambda aws

**URL:** <https://community.pinecone.io/t/error-using-pinecone-client-at-lambda-aws/3478>\
**Category:** Support\
**Created:** [October 20, 2023, 5:47pm UTC](https://community.pinecone.io/t/error-using-pinecone-client-at-lambda-aws/3478 "2023-10-20T17:47:47Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![ruly.altamirano](https://sea2.discourse-cdn.com/flex020/user_avatar/community.pinecone.io/ruly.altamirano/32/1559_2.png) [@ruly.altamirano](https://community.pinecone.io/u/ruly.altamirano)\
**Post date:** [October 20, 2023, 5:47pm UTC](https://community.pinecone.io/t/error-using-pinecone-client-at-lambda-aws/3478/1 "2023-10-20T17:47:47Z")

</div>

i got an error thats seems is originates from the creation of a ThreadPool within the pinecone library. Specifically, the error occurs when you try add data to my pinecone index when i use from\_documents , seems a multiprocessing not supported by AWS Lambda that has restrictions on the ability to use certain multiprocessing features.

pinecone has any settings that allow you to turn off the use of ThreadPool or change the way it handles concurrency?

---

<div class="post-metadata">

**Author:** ![tim](https://sea2.discourse-cdn.com/flex020/user_avatar/community.pinecone.io/tim/32/1503_2.png) [@tim](https://community.pinecone.io/u/tim)\
**Post date:** [October 20, 2023, 5:59pm UTC](https://community.pinecone.io/t/error-using-pinecone-client-at-lambda-aws/3478/2 "2023-10-20T17:59:19Z")

</div>

Are you using the actual Pinecone python library or Langchains wrapper? The reason I ask is because I’m not familiar with a `from_documents` method in the Pinecone library, but I am with Langchain.

By default, upserts are synchronous [(source)](https://docs.pinecone.io/docs/insert-data#sending-upserts-in-parallel) and _can_ be made async using `pool_threads` kwarg.

If using Langchain the default kwarg for pool\_threads is 4 [(source code)](https://github.com/langchain-ai/langchain/blob/5fee61a207b6da18a30fbdd89f8ea9dc666abf23/libs/langchain/langchain/vectorstores/pinecone.py#L336)

So you may need to forcefully pass in `1` or even `None` here? Or you can just do the upsert yourself using the Pinecone library natively. If using Langchain, the reason above is why pool thread is set to a non-singular value.

Lastly, I think the issue on the AWS Lamdba side preventing you from multi-processing is probably about [how shared memory doesn’t exist](https://aws.amazon.com/blogs/compute/parallel-processing-in-python-with-aws-lambda/#:~:text=Python-,Parallel%20Execution,-The%20multiprocessing%20module) so instead of `multiprocessing.Queue` you need to use `Pipe`

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/flex020/uploads/pinecone/original/1X/d16664eba369eeee623700cf12feab5562b37eaf.jpeg) [@system](https://community.pinecone.io/u/system)\
**Post date:** [November 3, 2023, 5:59pm UTC](https://community.pinecone.io/t/error-using-pinecone-client-at-lambda-aws/3478/3 "2023-11-03T17:59:59Z")

</div>

Closed on November 3, 2023, 5:59pm UTC

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.
