API Reference - Broker

The LlmBroker is the main interface for interacting with LLMs.

LlmBroker Class

class LlmBroker {
  constructor(
    modelId: string,
    gateway: LlmGateway,
    config?: CompletionConfig
  )
}

Central orchestrator for LLM interactions.

Parameters:

• modelId: Model identifier (e.g., 'qwen3:32b', 'gpt-4')
• gateway: LLM gateway instance
• config: Optional default configuration

Example:

import { LlmBroker, OllamaGateway } from 'mojentic';

const gateway = new OllamaGateway('http://localhost:11434');
const broker = new LlmBroker('qwen3:32b', gateway, {
  temperature: 0.7,
  maxTokens: 2000
});

Methods

generate

async generate(
  messages: LlmMessage[],
  tools?: LlmTool[],
  config?: CompletionConfig
): Promise<Result<string, Error>>

Generate a text response from the LLM.

Parameters:

• messages: Conversation history
• tools: Optional tools the LLM can call
• config: Optional configuration overrides

Returns:

• Result<string, Error>: Ok with generated text or Err with error

Example:

const messages = [
  Message.system('You are a helpful assistant'),
  Message.user('What is the capital of France?')
];

const result = await broker.generate(messages);

if (isOk(result)) {
  console.log(result.value); // "The capital of France is Paris."
}

With Tools:

const tools = [new DateResolverTool()];

const messages = [
  Message.user('What day is next Friday?')
];

const result = await broker.generate(messages, tools);
// The broker will automatically:
// 1. Send request to LLM
// 2. Execute any tool calls
// 3. Send results back to LLM
// 4. Return final response

With Config Override:

const result = await broker.generate(messages, undefined, {
  temperature: 0.9,
  maxTokens: 500
});

generateObject

async generateObject(
  messages: LlmMessage[],
  schema: Record<string, any>,
  config?: CompletionConfig
): Promise<Result<Record<string, any>, Error>>

Generate structured output matching a JSON schema.

Parameters:

• messages: Conversation history
• schema: JSON Schema defining expected structure
• config: Optional configuration overrides

Returns:

• Result<Record<string, any>, Error>: Ok with parsed object or Err with error

Example:

const schema = {
  type: 'object',
  properties: {
    name: { type: 'string' },
    age: { type: 'number' },
    city: { type: 'string' }
  },
  required: ['name', 'age']
};

const messages = [
  Message.user('Extract: John Smith, 34, lives in Seattle')
];

const result = await broker.generateObject(messages, schema);

if (isOk(result)) {
  console.log(result.value);
  // { name: "John Smith", age: 34, city: "Seattle" }
}

Complex Schema:

const schema = {
  type: 'object',
  properties: {
    items: {
      type: 'array',
      items: {
        type: 'object',
        properties: {
          product: { type: 'string' },
          quantity: { type: 'number' },
          price: { type: 'number' }
        },
        required: ['product', 'quantity', 'price']
      }
    },
    total: { type: 'number' }
  },
  required: ['items', 'total']
};

const result = await broker.generateObject(messages, schema);

generateStream

async *generateStream(
  messages: LlmMessage[],
  config?: CompletionConfig,
  tools?: LlmTool[],
  correlationId?: string
): AsyncGenerator<Result<string, Error>>

Generate a streaming response. The broker runs the tool calls that the LLM requests, then streams the next response.

Parameters:

• messages: Conversation history
• config: Optional configuration overrides
• tools: Optional tools the LLM can call
• correlationId: Optional id for the tracer events. The default is a new UUID.

Yields:

• Result<string, Error>: Ok with the text of one chunk, or Err with an error

generateStream is an async generator. Do not await it. Use for await to read the items.

Example:

const messages = [
  Message.user('Write a short story')
];

for await (const result of broker.generateStream(messages)) {
  if (isOk(result)) {
    process.stdout.write(result.value);
  } else {
    console.error('Stream error:', result.error);
    break;
  }
}

With tools:

for await (const result of broker.generateStream(messages, { temperature: 0.7 }, [
  new DateResolverTool(),
])) {
  if (isOk(result)) {
    process.stdout.write(result.value);
  }
}

generateStreamEvents

generateStreamEvents(
  messages: LlmMessage[],
  config?: CompletionConfig,
  options?: { correlationId?: string; signal?: AbortSignal }
): AsyncGenerator<LlmStreamEvent>

Stream one turn as content events that end in exactly one terminal event: { type: 'completed', metadata: CompletionEvidence } or { type: 'error', error: StreamEventError }. No tools, no tool iterations, no retry, one HTTP request. Stopping the iteration or aborting signal cancels the request. See Streaming.

Message Management

Building Conversations

const messages: LlmMessage[] = [
  Message.system('You are a helpful coding assistant'),
  Message.user('How do I read a file in TypeScript?'),
  Message.assistant('You can use fs.readFile...'),
  Message.user('What about async/await?')
];

const result = await broker.generate(messages);

Appending Responses

let messages = [Message.user('Hello')];

const result1 = await broker.generate(messages);
if (isOk(result1)) {
  messages.push(Message.assistant(result1.value));
}

messages.push(Message.user('Tell me more'));

const result2 = await broker.generate(messages);

Managing Context

// Keep last N messages
function trimMessages(messages: LlmMessage[], keepLast: number): LlmMessage[] {
  const system = messages.filter(m => m.role === MessageRole.System);
  const others = messages.filter(m => m.role !== MessageRole.System);
  return [...system, ...others.slice(-keepLast)];
}

let messages = [...longConversation];
messages = trimMessages(messages, 10);

const result = await broker.generate(messages);

Configuration

Default Configuration

Set default config in constructor:

const broker = new LlmBroker('qwen3:32b', gateway, {
  temperature: 0.7,
  maxTokens: 2000,
  topP: 0.9
});

// Uses default config
await broker.generate(messages);

Per-Request Configuration

Override config per request:

// More creative
const creative = await broker.generate(messages, undefined, {
  temperature: 1.2
});

// More focused
const focused = await broker.generate(messages, undefined, {
  temperature: 0.3
});

// Shorter response
const brief = await broker.generate(messages, undefined, {
  maxTokens: 100
});

Common Settings

// Deterministic responses
const config = {
  temperature: 0.0,
  topP: 1.0
};

// Creative responses
const config = {
  temperature: 1.5,
  topP: 0.95
};

// Balanced
const config = {
  temperature: 0.7,
  maxTokens: 1000
};

Tool Execution

Automatic Execution

The broker automatically executes tools:

const tools = [
  new DateResolverTool(),
  new WeatherTool(),
  new CalculatorTool()
];

const messages = [
  Message.user('What will the weather be like next Monday in Tokyo?')
];

const result = await broker.generate(messages, tools);
// 1. LLM calls DateResolverTool('next Monday')
// 2. LLM calls WeatherTool('Tokyo', '2025-12-15')
// 3. LLM provides final answer

Tool Call Limit

The broker limits recursive tool calls to prevent infinite loops:

// Maximum 10 tool execution rounds by default
const result = await broker.generate(messages, tools);

If limit is exceeded, returns an error.

Tool Results

Tool results are automatically added to conversation:

// Initial messages
[
  Message.user('What day is tomorrow?')
]

// After tool execution
[
  Message.user('What day is tomorrow?'),
  Message.assistant('', [toolCall]),
  Message.tool('{"date": "2025-12-11", "day": "Thursday"}', toolCallId, 'resolve_date'),
  Message.assistant('Tomorrow is Thursday, December 11, 2025')
]

Error Handling

Result Type

All methods return Result<T, Error>:

const result = await broker.generate(messages);

if (isOk(result)) {
  // Success path
  const text = result.value;
} else {
  // Error path
  const error = result.error;
}

Error Types

const result = await broker.generate(messages);

if (isErr(result)) {
  const error = result.error;

  if (error instanceof GatewayError) {
    console.error('Gateway failed:', error.gateway, error.statusCode);
  } else if (error instanceof ToolError) {
    console.error('Tool failed:', error.toolName);
  } else if (error instanceof TimeoutError) {
    console.error('Timeout after', error.timeoutMs, 'ms');
  }
}

Structured Output Errors

const result = await broker.generateObject(messages, schema);

if (isErr(result)) {
  const error = result.error;

  if (error instanceof ParseError) {
    console.error('Failed to parse JSON');
  } else if (error instanceof ValidationError) {
    console.error('Response does not match schema');
  }
}

Best Practices

1. Reuse Broker Instances

// Good: Create once, reuse
const broker = new LlmBroker('qwen3:32b', gateway);

async function chat(message: string) {
  return await broker.generate([Message.user(message)]);
}

// Bad: Create for each request
async function chat(message: string) {
  const broker = new LlmBroker('qwen3:32b', gateway);
  return await broker.generate([Message.user(message)]);
}

2. Handle Errors

// Good: Check result
const result = await broker.generate(messages);
if (isOk(result)) {
  return result.value;
} else {
  console.error(result.error);
  return 'An error occurred';
}

// Bad: Assume success
const result = await broker.generate(messages);
return result.value; // Type error! Could be undefined

3. Use System Messages

// Good: Clear system context
const messages = [
  Message.system('You are a helpful coding assistant specializing in TypeScript'),
  Message.user('How do I use async/await?')
];

// Less effective: No system context
const messages = [
  Message.user('How do I use async/await?')
];

4. Manage Context Length

// Monitor message count
if (messages.length > 20) {
  messages = trimMessages(messages, 15);
}

const result = await broker.generate(messages);

Complete Example

import {
  LlmBroker,
  OllamaGateway,
  Message,
  DateResolverTool,
  isOk,
  isErr,
  GatewayError
} from 'mojentic';

// Setup
const gateway = new OllamaGateway();
const broker = new LlmBroker('qwen3:32b', gateway, {
  temperature: 0.7
});

// Tools
const tools = [new DateResolverTool()];

// Conversation
const messages = [
  Message.system('You are a helpful assistant'),
  Message.user('What day is next Monday?')
];

// Generate
const result = await broker.generate(messages, tools);

if (isOk(result)) {
  console.log('Response:', result.value);

  // Continue conversation
  messages.push(Message.assistant(result.value));
  messages.push(Message.user('And the Monday after that?'));

  const result2 = await broker.generate(messages, tools);
  if (isOk(result2)) {
    console.log('Response 2:', result2.value);
  }
} else {
  const error = result.error;

  if (error instanceof GatewayError) {
    console.error(`Gateway ${error.gateway} error:`, error.message);
  } else {
    console.error('Error:', error.message);
  }
}

See Also

• Getting Started
• Broker Guide
• Tool Usage
• Structured Output
• Core API
• Gateway API