With Cloudflare Worker

Before you start, make sure you have try LlamaIndex.TS in Node.js to make sure you understand the basics.

Also, you need have the basic understanding of Cloudflare Worker.

Adding environment variables

export default {
  async fetch(request: Request, env: Env): Promise<Response> {
    const { setEnvs } = await import("@llamaindex/env");
    setEnvs(env);
    const { OpenAIAgent } = await import("@llamaindex/openai");
    // Start your code here
    return new Response("Hello, world!");
  },
};

Then, you need create .dev.vars and add LLM api keys for the local development, such as OPENAI_API_KEY for OpenAI API key.

Integrating with Hono

import { Hono } from "hono";
 
type Bindings = {
  OPENAI_API_KEY: string;
};
 
const app = new Hono<{
  Bindings: Bindings;
}>();
 
app.post("/llm", async (c) => {
  const { setEnvs } = await import("@llamaindex/env");
  setEnvs(c.env);
 
  // ...
 
  return new Response('Hello, world!');
})
 
export default {
  fetch: app.fetch,
};

Difference between Node.js and Cloudflare Worker

In Cloudflare Worker and similar serverless JS environment, you need to be aware of the following differences:

• Some Node.js modules are not available in Cloudflare Worker, such as node:fs, node:child_process, node:cluster...
• You are recommend to design your code using network request, such as use fetch API to communicate with database, insteadof a long-running process in Node.js.
• Some of LlamaIndex.TS packages are not available in Cloudflare Worker, for example @llamaindex/readers and @llamaindex/huggingface.
• The main llamaindex is designed to work in all JavaScript environment, including Cloudflare Worker. If you find any issue, please report to us.
• @llamaindex/env is a JS environment binding module, which polyfill some Node.js/Modern Web API (for example, we have a memory based fs module, and Crypto API polyfill). It is designed to work in all JavaScript environment, including Cloudflare Worker.