Building a Multi-User Chatbot with Langchain and Pinecone in Next.JS

Search, No Filter
1

Building a chat bot has become a hot skill, and with the release of ChatGPT we see a huge number of chat applications being released. At the root of all of these applications live Large Language Models - the engine of the generative AI train. But this beast must be tamed - and that’s not always an easy task.

Since LLMs are now such an integral piece of the puzzle, there are several challenges we need to tackle in order to productionize our chat bots,

This is a companion discussion topic for the original entry at https://www.pinecone.io/learn/javascript-chatbot/
1 Like
2

Where’s the code? It would be helpful to be able to review it.

3

He adds a link to the code in the article.

4

I am facing error in ably . The api key provided in the .env file is correct.

Screenshot from 2023-04-03 23-13-59
Screenshot from 2023-04-03 23-13-591917×877 238 KB

1 Like
5

You need to give more permissions for the Ably API. I just used root API for testing.

With Ably I’m getting occasionally Error code: 42911 - Maximum account-wide instantaneous messages rate exceeded as free version is only capable to 15 messages per second and speed is around 20 for some prompts.

6

Any idea why the response prompt shows up only in the terminal?

  • No errors
  • No entries in the cockroachdb (table has been created)
  • Frontend shows “Pinecone is typing” during the response

Edit: Just noticed that there is an error message:

Error adding entry: Error: Positional replacement (?) 0 has no entry in the replacement map (replacements[0] is undefined).

Perhaps this is something to do with cockroachdb.

7

I’m getting an error - FetchError: The request failed and the interceptors did not return an alternative response

I tried searching but could not find any docs.

8

It would be nice to have a bit more documentation about the setup of the services that are used to this. It’s interesting, but a there is a lot of room for mistakes in the setup process!

(Edit: I got this working, I had used API key with wrong settings from Fingerprint. Public is the right one. Excited to start testing prompts and different documents :+1:)

1 Like
9

Glad to hear you got this working. For a complete novice who followed the tutorial and would like to test out the chatbot, can you please share some instructions for the services?

I am already using the public api key for Fingerprint and I created an Ably token with all access.

I am getting this error when I try to run the app

13:22:13.876 Ably: Rest(): invalid key parameter
error - [ErrorInfo: invalid key parameter] {
  code: 40400,
  statusCode: 404,
  cause: undefined,
  page: '/api/createTokenRequest'
}
10

The code stated in the article seems to be quite different from the code in the repo. It’s very difficult to follow because of a lot oof missing context. I think the tutorial experience of this could be really improved.

11

Thanks for the code! very useful. I modified quite a few things, removed fingerprint, abbly and cockroach, and used my own db. i noticed answers from the ai are not being stored in the conversation log in the database, is that itended ? or my modifications messed with that ? (i dont think so since your original code doesnt seem to be storing the answers either)

Thanks.

12

I’m getting the following error, any ideas?
Error querying embeddings: [Error: PineconeClient: Error calling query: TypeError: stream.getReader is not a function]

13

I Got the have same error. Any progress? @Oskar

14

Make sure you’re running node>=18.

15

Does anyone know how to add the context to the chatbot?

18

error - [Error: PineconeClient: Error calling upsert: Error: PineconeClient: Error calling upsertRaw: FetchError: The request failed and the interceptors did not return an alternative response] {
page: ‘/api/crawl’
}

19

I’m getting this same upset/intercepters error. Trying to run the sample LangChain Pinecone code they posted in the docs.

Code:

import { PineconeClient } from "@pinecone-database/pinecone";
import * as dotenv from "dotenv";
import { Document } from "langchain/document";
import { OpenAIEmbeddings } from "langchain/embeddings/openai";
import { PineconeStore } from "langchain/vectorstores/pinecone";

dotenv.config();

const client = new PineconeClient();
await client.init({
  apiKey: process.env.PINECONE_API_KEY,
  environment: process.env.PINECONE_ENVIRONMENT,
});
const pineconeIndex = client.Index(process.env.PINECONE_INDEX);

const docs = [
  new Document({
    metadata: { foo: "bar" },
    pageContent: "pinecone is a vector db",
  }),
  new Document({
    metadata: { foo: "bar" },
    pageContent: "the quick brown fox jumped over the lazy dog",
  }),
  new Document({
    metadata: { baz: "qux" },
    pageContent: "lorem ipsum dolor sit amet",
  }),
  new Document({
    metadata: { baz: "qux" },
    pageContent: "pinecones are the woody fruiting body and of a pine tree",
  }),
];

await PineconeStore.fromDocuments(docs, new OpenAIEmbeddings(), {
  pineconeIndex,
});

.env:

OPENAI_API_KEY=<key>
PINECONE_API_KEY=<key>
PINECONE_ENVIRONMENT=us-west1-gcp-free
PINECONE_INDEX_NAME=<short index name>

OUTPUT:

node:internal/process/promises:288
            triggerUncaughtException(err, true /* fromPromise */);
            ^

[PineconeError: PineconeClient: Error calling upsert: PineconeError: PineconeClient: Error calling upsertRaw: FetchError: The request failed and the interceptors did not return an alternative response]

Node.js v18.16.0

Any ideas? I’m using a Starter plan Pinecone account with a single index if that’s relevant.

20

I’m having the same issue. What was your resolution>

21

Getting Ably Errors and the Key (appears) to be set up properly in .env but still getting these errors.

02:16:46.419 Ably: Auth(): No authentication options provided; need one of: key, authUrl, or authCallback (or for testing only, token or tokenDetails)
error - ErrorInfo: No authentication options provided; need one of: key, authUrl, or authCallback (or for testing only, token or tokenDetails)

**I'm just trying to get it up and running to start, and it's crashing with the above error message. At the bottom of this person's blog post, it looks some others have had the same issue. **

**Any thoughts on where I could look to see what the issue might be?** 

**For more detailed error, I've pasted it below:** 

error - ErrorInfo: No authentication options provided; need one of: key, authUrl, or authCallback (or for testing only, token or tokenDetails)
    at new ErrorInfo (/workspaces/chatbot-demo/node_modules/ably/build/ably-node.js:1064:28)
    at new Auth (/workspaces/chatbot-demo/node_modules/ably/build/ably-node.js:3217:23)
    at Realtime.Rest (/workspaces/chatbot-demo/node_modules/ably/build/ably-node.js:6626:21)
    at new Realtime (/workspaces/chatbot-demo/node_modules/ably/build/ably-node.js:9112:28)
    at new RealtimePromise (/workspaces/chatbot-demo/node_modules/ably/promises.js:21:10)
    at handler (webpack-i

**On the front end I get this output, with Ably mentioned, for any additional context this may provide.** 
 1 of 1 unhandled error

Unhandled Runtime Error
Error: Connection to server unavailable

Call Stack
new ErrorInfo
node_modules/ably/build/ably-commonjs.js (1155:0)
ErrorInfo.fromValues
node_modules/ably/build/ably-commonjs.js (1184:0)
Object.eval
node_modules/ably/build/ably-commonjs.js (4846:0)
nested_webpack_require_723
node_modules/ably/build/ably-commonjs.js (38:0)
Object.eval
node_modules/ably/build/ably-commonjs.js (5314:46)
Object.eval
node_modules/ably/build/ably-commonjs.js (6943:28)
nested_webpack_require_723
node_modules/ably/build/ably-commonjs.js (38:0)
Object.eval
node_modules/ably/build/ably-commonjs.js (11764:50)
nested_webpack_require_723
node_modules/ably/build/ably-commonjs.js (38:0)
Object.eval
node_modules/ably/build/ably-commonjs.js (11601:43)
nested_webpack_require_723
node_modules/ably/build/ably-commonjs.js (38:0)
Object.eval
node_modules/ably/build/ably-commonjs.js (10975:41)
nested_webpack_require_723
node_modules/ably/build/ably-commonjs.js (38:0)
eval
node_modules/ably/build/ably-commonjs.js (102:17)
eval
node_modules/ably/build/ably-commonjs.js (105:0)
webpackUniversalModuleDefinition
node_modules/ably/build/ably-commonjs.js (11:0)
eval
node_modules/ably/build/ably-commonjs.js (18:1)
./node_modules/ably/build/ably-commonjs.js
file:///workspaces/chatbot-demo/.next/static/chunks/pages/_app.js (50:1)
options.factory
/_next/static/chunks/webpack.js (661:31)
webpack_require
file:///workspaces/chatbot-demo/.next/static/chunks/webpack.js (37:33)
fn
/_next/static/chunks/webpack.js (316:21)
eval
webpack-internal:///./node_modules/@ably-labs/react-hooks/dist/mjs/AblyReactHooks.js (8:62)
./node_modules/@ably-labs/react-hooks/dist/mjs/AblyReactHooks.js
22

I’d like to do something similar - create a chatbot using chatGPT with the datastore being Pinecone. Microsoft Azure has a great demo for using the Azure Cognitive Search: GitHub - Azure-Samples/azure-search-openai-demo: A sample app for the Retrieval-Augmented Generation pattern running in Azure, using Azure Cognitive Search for retrieval and Azure OpenAI large language models to power ChatGPT-style and Q&A experiences.

Ideally, I’d like to use Pinecone since Cognitive Search is $450/mo :scream:

I’ve been able to follow Pinecone’s insert/query examples using the all-MiniLM-L6-v2 model, however my query results are horrible. I’ve basically taken many of the larger PDFs, broken them into pages (per azure demo), and then broken them down further into 4 sentence chunks and sent it to Pinecone. I chose 4 sentence chunks since the Model states it ignores anything after 256 tokens (thus a full page PDF might lose a lot of data during the model embedding).

Any guidance would be appreciated!