Expand description
Gateway for oMLX, an LLM server for Apple Silicon.
oMLX speaks the OpenAI chat completions protocol, but this gateway does not
treat it as OpenAI. It sends every setting as configured, with no per-model
parameter adaptation and no OpenAI model registry. It maps the reasoning
trace oMLX reports (reasoning_content) to the response’s thinking, and
it sends embeddings in one request with no client-side chunking.
It reuses the OpenAI message adapter and the OpenAI stream parsers.
Structs§
- Omlx
Config - Configuration for connecting to an oMLX server.
- Omlx
Gateway - Gateway for a local oMLX server.
Constants§
- RESPONSE_
FORMAT_ WARNING - Response metadata key for the
Warningheader oMLX sends when it could not enforce a requested structured output format.