Skip to main content

OllamaGateway

Struct OllamaGateway 

Source
pub struct OllamaGateway { /* private fields */ }
Expand description

Gateway for Ollama local LLM service

This gateway provides access to local LLM models through Ollama, supporting text generation, structured output, tool calling, and embeddings.

Implementations§

Source§

impl OllamaGateway

Source

pub fn with_stream_clock(self, clock: Arc<dyn StreamClock>) -> Self

Select a monotonic measurement clock. The default uses std::time::Instant.

Source

pub fn with_recovery( self, policy: RecoveryPolicy, ) -> Result<RecoveryGateway<Self>>

Opt into per-request recovery without changing legacy entrypoints.

Source

pub fn new() -> Self

Create a new Ollama gateway with default configuration

Source

pub fn with_config(config: OllamaConfig) -> Self

Create a new Ollama gateway with custom configuration

Source

pub fn with_host(host: impl Into<String>) -> Self

Create gateway with custom host

Source

pub fn effective_context_window( window: Option<&ModelContextWindow>, config: &CompletionConfig, ) -> Option<usize>

Resolve the request’s context capacity without sending a request. Explicit num_ctx takes precedence over the model parameter; both are clamped to the advertised maximum. With neither, this is an upper bound because the server’s OLLAMA_CONTEXT_LENGTH is unknown to the client.

Source

pub async fn pull_model(&self, model: &str) -> Result<()>

Pull a model from Ollama library

Trait Implementations§

Source§

impl Default for OllamaGateway

Source§

fn default() -> Self

Returns the “default value” for a type. Read more
Source§

impl LlmGateway for OllamaGateway

Source§

fn stream_clock(&self) -> Arc<dyn StreamClock> ⓘ

Monotonic clock used by maintained streaming adapters.
Source§

fn get_model_context_window<'life0, 'life1, 'async_trait>( &'life0 self, model: &'life1 str, ) -> Pin<Box<dyn Future<Output = Result<Option<ModelContextWindow>>> + Send + 'async_trait>>
where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Query advertised context capacity without changing subsequent requests. Read more
Source§

fn complete<'life0, 'life1, 'life2, 'life3, 'life4, 'async_trait>( &'life0 self, model: &'life1 str, messages: &'life2 [LlmMessage], tools: Option<&'life3 [Box<dyn LlmTool>]>, config: &'life4 CompletionConfig, ) -> Pin<Box<dyn Future<Output = Result<LlmGatewayResponse>> + Send + 'async_trait>>
where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait, 'life3: 'async_trait, 'life4: 'async_trait,

Complete an LLM request with text response
Source§

fn complete_json<'life0, 'life1, 'life2, 'life3, 'async_trait>( &'life0 self, model: &'life1 str, messages: &'life2 [LlmMessage], schema: Value, config: &'life3 CompletionConfig, ) -> Pin<Box<dyn Future<Output = Result<Value>> + Send + 'async_trait>>
where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait, 'life3: 'async_trait,

Complete an LLM request with structured JSON response
Source§

fn complete_json_response<'life0, 'life1, 'life2, 'life3, 'async_trait>( &'life0 self, model: &'life1 str, messages: &'life2 [LlmMessage], schema: Value, config: &'life3 CompletionConfig, ) -> Pin<Box<dyn Future<Output = Result<LlmGatewayResponse<Value>>> + Send + 'async_trait>>
where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait, 'life3: 'async_trait,

Complete a structured JSON request and keep the provider’s evidence. Read more
Source§

fn get_available_models<'life0, 'async_trait>( &'life0 self, ) -> Pin<Box<dyn Future<Output = Result<Vec<String>>> + Send + 'async_trait>>
where Self: 'async_trait, 'life0: 'async_trait,

Get list of available models
Source§

fn calculate_embeddings<'life0, 'life1, 'life2, 'async_trait>( &'life0 self, text: &'life1 str, model: Option<&'life2 str>, ) -> Pin<Box<dyn Future<Output = Result<Vec<f32>>> + Send + 'async_trait>>
where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait,

Calculate embeddings for text
Source§

fn complete_stream<'a>( &'a self, model: &'a str, messages: &'a [LlmMessage], tools: Option<&'a [Box<dyn LlmTool>]>, config: &'a CompletionConfig, ) -> Pin<Box<dyn Stream<Item = Result<StreamChunk>> + Send + 'a>>

Stream LLM responses chunk by chunk Read more
Source§

fn complete_stream_events<'a>( &'a self, model: &'a str, messages: &'a [LlmMessage], config: &'a CompletionConfig, ) -> Result<StreamEventStream<'a>, StreamEventError>

Stream one turn as crate::llm::StreamEvents with terminal completion evidence. Read more

Auto Trait Implementations§

Blanket Implementations§

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

§

impl<T> Instrument for T

§

fn instrument(self, span: Span) -> Instrumented<Self> ⓘ

Instruments this type with the provided [Span], returning an Instrumented wrapper. Read more
§

fn in_current_span(self) -> Instrumented<Self> ⓘ

Instruments this type with the current Span, returning an Instrumented wrapper. Read more
Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

§

impl<T> PolicyExt for T
where T: ?Sized,

§

fn and<P, B, E>(self, other: P) -> And<T, P>
where T: Sized + Policy<B, E>, P: Policy<B, E>,

Create a new Policy that returns [Action::Follow] only if self and other return Action::Follow. Read more
§

fn or<P, B, E>(self, other: P) -> Or<T, P>
where T: Sized + Policy<B, E>, P: Policy<B, E>,

Create a new Policy that returns [Action::Follow] if either self or other returns Action::Follow. Read more
Source§

impl<T> Same for T

Source§

type Output = T

Should always be Self
Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = Infallible

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, <T as TryFrom<U>>::Error>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.
§

impl<T> WithSubscriber for T

§

fn with_subscriber<S>(self, subscriber: S) -> WithDispatch<Self> ⓘ
where S: Into<Dispatch>,

Attaches the provided Subscriber to this type, returning a [WithDispatch] wrapper. Read more
§

fn with_current_subscriber(self) -> WithDispatch<Self> ⓘ

Attaches the current default Subscriber to this type, returning a [WithDispatch] wrapper. Read more