Skip to main content

Module omlx

Module omlx 

Source
Expand description

Gateway for oMLX, an LLM server for Apple Silicon.

oMLX speaks the OpenAI chat completions protocol, but this gateway does not treat it as OpenAI. It sends every setting as configured, with no per-model parameter adaptation and no OpenAI model registry. It maps the reasoning trace oMLX reports (reasoning_content) to the response’s thinking, and it sends embeddings in one request with no client-side chunking.

It reuses the OpenAI message adapter and the OpenAI stream parsers.

Structs§

OmlxConfig
Configuration for connecting to an oMLX server.
OmlxGateway
Gateway for a local oMLX server.

Constants§

RESPONSE_FORMAT_WARNING
Response metadata key for the Warning header oMLX sends when it could not enforce a requested structured output format.