Skip to main content

Class: HuggingFaceInferenceAPI

Wrapper on the Hugging Face's Inference API. API Docs: https://huggingface.co/docs/huggingface.js/inference/README List of tasks with models: huggingface.co/api/tasks

Note that Conversational API is not yet supported by the Inference API. They recommend using the text generation API instead. See: https://github.com/huggingface/huggingface.js/issues/586#issuecomment-2024059308

Extends​

  • BaseLLM

Constructors​

new HuggingFaceInferenceAPI()​

new HuggingFaceInferenceAPI(init): HuggingFaceInferenceAPI

Parameters​

• init: HFConfig

Returns​

HuggingFaceInferenceAPI

Overrides​

BaseLLM.constructor

Source​

packages/core/src/llm/huggingface.ts:53

Properties​

contextWindow​

contextWindow: number

Source​

packages/core/src/llm/huggingface.ts:50

hf​

hf: HfInference

Source​

packages/core/src/llm/huggingface.ts:51

maxTokens?​

optional maxTokens: number

Source​

packages/core/src/llm/huggingface.ts:49

model​

model: string

Source​

packages/core/src/llm/huggingface.ts:46

temperature​

temperature: number

Source​

packages/core/src/llm/huggingface.ts:47

topP​

topP: number

Source​

packages/core/src/llm/huggingface.ts:48

Accessors​

metadata​

get metadata(): LLMMetadata

Returns​

LLMMetadata

Source​

packages/core/src/llm/huggingface.ts:74

Methods​

chat()​

chat(params)​

chat(params): Promise<AsyncIterable <ChatResponseChunk>>

Parameters​

• params: LLMChatParamsStreaming<object, object>

Returns​

Promise<AsyncIterable <ChatResponseChunk>>

Overrides​

BaseLLM.chat

Source​

packages/core/src/llm/huggingface.ts:85

chat(params)​

chat(params): Promise <ChatResponse<object>>

Parameters​

• params: LLMChatParamsNonStreaming<object, object>

Returns​

Promise <ChatResponse<object>>

Overrides​

BaseLLM.chat

Source​

packages/core/src/llm/huggingface.ts:88

complete()​

complete(params)​

complete(params): Promise<AsyncIterable <CompletionResponse>>

Parameters​

• params: LLMCompletionParamsStreaming

Returns​

Promise<AsyncIterable <CompletionResponse>>

Inherited from​

BaseLLM.complete

Source​

packages/core/src/llm/base.ts:22

complete(params)​

complete(params): Promise <CompletionResponse>

Parameters​

• params: LLMCompletionParamsNonStreaming

Returns​

Promise <CompletionResponse>

Inherited from​

BaseLLM.complete

Source​

packages/core/src/llm/base.ts:25

messagesToPrompt()​

private messagesToPrompt(messages): string

Parameters​

• messages: ChatMessage <ToolCallLLMMessageOptions>[]

Returns​

string

Source​

packages/core/src/llm/huggingface.ts:97

nonStreamChat()​

protected nonStreamChat(params): Promise <ChatResponse<object>>

Parameters​

• params: LLMChatParamsNonStreaming<object, object>

Returns​

Promise <ChatResponse<object>>

Source​

packages/core/src/llm/huggingface.ts:117

streamChat()​

protected streamChat(params): AsyncIterable <ChatResponseChunk>

Parameters​

• params: LLMChatParamsStreaming<object, object>

Returns​

AsyncIterable <ChatResponseChunk>

Source​

packages/core/src/llm/huggingface.ts:134