pub struct OllamaGateway { /* private fields */ }Expand description
Gateway for Ollama local LLM service
This gateway provides access to local LLM models through Ollama, supporting text generation, structured output, tool calling, and embeddings.
Implementations§
Source§impl OllamaGateway
impl OllamaGateway
Sourcepub fn with_stream_clock(self, clock: Arc<dyn StreamClock>) -> Self
pub fn with_stream_clock(self, clock: Arc<dyn StreamClock>) -> Self
Select a monotonic measurement clock. The default uses std::time::Instant.
Sourcepub fn with_recovery(
self,
policy: RecoveryPolicy,
) -> Result<RecoveryGateway<Self>>
pub fn with_recovery( self, policy: RecoveryPolicy, ) -> Result<RecoveryGateway<Self>>
Opt into per-request recovery without changing legacy entrypoints.
Sourcepub fn with_config(config: OllamaConfig) -> Self
pub fn with_config(config: OllamaConfig) -> Self
Create a new Ollama gateway with custom configuration
Sourcepub fn effective_context_window(
window: Option<&ModelContextWindow>,
config: &CompletionConfig,
) -> Option<usize>
pub fn effective_context_window( window: Option<&ModelContextWindow>, config: &CompletionConfig, ) -> Option<usize>
Resolve the request’s context capacity without sending a request.
Explicit num_ctx takes precedence over the model parameter; both are
clamped to the advertised maximum. With neither, this is an upper bound
because the server’s OLLAMA_CONTEXT_LENGTH is unknown to the client.
Sourcepub async fn pull_model(&self, model: &str) -> Result<()>
pub async fn pull_model(&self, model: &str) -> Result<()>
Pull a model from Ollama library
Trait Implementations§
Source§impl Default for OllamaGateway
impl Default for OllamaGateway
Source§impl LlmGateway for OllamaGateway
impl LlmGateway for OllamaGateway
Source§fn stream_clock(&self) -> Arc<dyn StreamClock> ⓘ
fn stream_clock(&self) -> Arc<dyn StreamClock> ⓘ
Monotonic clock used by maintained streaming adapters.
Source§fn get_model_context_window<'life0, 'life1, 'async_trait>(
&'life0 self,
model: &'life1 str,
) -> Pin<Box<dyn Future<Output = Result<Option<ModelContextWindow>>> + Send + 'async_trait>>where
Self: 'async_trait,
'life0: 'async_trait,
'life1: 'async_trait,
fn get_model_context_window<'life0, 'life1, 'async_trait>(
&'life0 self,
model: &'life1 str,
) -> Pin<Box<dyn Future<Output = Result<Option<ModelContextWindow>>> + Send + 'async_trait>>where
Self: 'async_trait,
'life0: 'async_trait,
'life1: 'async_trait,
Query advertised context capacity without changing subsequent requests. Read more
Source§fn complete<'life0, 'life1, 'life2, 'life3, 'life4, 'async_trait>(
&'life0 self,
model: &'life1 str,
messages: &'life2 [LlmMessage],
tools: Option<&'life3 [Box<dyn LlmTool>]>,
config: &'life4 CompletionConfig,
) -> Pin<Box<dyn Future<Output = Result<LlmGatewayResponse>> + Send + 'async_trait>>where
Self: 'async_trait,
'life0: 'async_trait,
'life1: 'async_trait,
'life2: 'async_trait,
'life3: 'async_trait,
'life4: 'async_trait,
fn complete<'life0, 'life1, 'life2, 'life3, 'life4, 'async_trait>(
&'life0 self,
model: &'life1 str,
messages: &'life2 [LlmMessage],
tools: Option<&'life3 [Box<dyn LlmTool>]>,
config: &'life4 CompletionConfig,
) -> Pin<Box<dyn Future<Output = Result<LlmGatewayResponse>> + Send + 'async_trait>>where
Self: 'async_trait,
'life0: 'async_trait,
'life1: 'async_trait,
'life2: 'async_trait,
'life3: 'async_trait,
'life4: 'async_trait,
Complete an LLM request with text response
Source§fn complete_json<'life0, 'life1, 'life2, 'life3, 'async_trait>(
&'life0 self,
model: &'life1 str,
messages: &'life2 [LlmMessage],
schema: Value,
config: &'life3 CompletionConfig,
) -> Pin<Box<dyn Future<Output = Result<Value>> + Send + 'async_trait>>where
Self: 'async_trait,
'life0: 'async_trait,
'life1: 'async_trait,
'life2: 'async_trait,
'life3: 'async_trait,
fn complete_json<'life0, 'life1, 'life2, 'life3, 'async_trait>(
&'life0 self,
model: &'life1 str,
messages: &'life2 [LlmMessage],
schema: Value,
config: &'life3 CompletionConfig,
) -> Pin<Box<dyn Future<Output = Result<Value>> + Send + 'async_trait>>where
Self: 'async_trait,
'life0: 'async_trait,
'life1: 'async_trait,
'life2: 'async_trait,
'life3: 'async_trait,
Complete an LLM request with structured JSON response
Source§fn complete_json_response<'life0, 'life1, 'life2, 'life3, 'async_trait>(
&'life0 self,
model: &'life1 str,
messages: &'life2 [LlmMessage],
schema: Value,
config: &'life3 CompletionConfig,
) -> Pin<Box<dyn Future<Output = Result<LlmGatewayResponse<Value>>> + Send + 'async_trait>>where
Self: 'async_trait,
'life0: 'async_trait,
'life1: 'async_trait,
'life2: 'async_trait,
'life3: 'async_trait,
fn complete_json_response<'life0, 'life1, 'life2, 'life3, 'async_trait>(
&'life0 self,
model: &'life1 str,
messages: &'life2 [LlmMessage],
schema: Value,
config: &'life3 CompletionConfig,
) -> Pin<Box<dyn Future<Output = Result<LlmGatewayResponse<Value>>> + Send + 'async_trait>>where
Self: 'async_trait,
'life0: 'async_trait,
'life1: 'async_trait,
'life2: 'async_trait,
'life3: 'async_trait,
Complete a structured JSON request and keep the provider’s evidence. Read more
Source§fn get_available_models<'life0, 'async_trait>(
&'life0 self,
) -> Pin<Box<dyn Future<Output = Result<Vec<String>>> + Send + 'async_trait>>where
Self: 'async_trait,
'life0: 'async_trait,
fn get_available_models<'life0, 'async_trait>(
&'life0 self,
) -> Pin<Box<dyn Future<Output = Result<Vec<String>>> + Send + 'async_trait>>where
Self: 'async_trait,
'life0: 'async_trait,
Get list of available models
Source§fn calculate_embeddings<'life0, 'life1, 'life2, 'async_trait>(
&'life0 self,
text: &'life1 str,
model: Option<&'life2 str>,
) -> Pin<Box<dyn Future<Output = Result<Vec<f32>>> + Send + 'async_trait>>where
Self: 'async_trait,
'life0: 'async_trait,
'life1: 'async_trait,
'life2: 'async_trait,
fn calculate_embeddings<'life0, 'life1, 'life2, 'async_trait>(
&'life0 self,
text: &'life1 str,
model: Option<&'life2 str>,
) -> Pin<Box<dyn Future<Output = Result<Vec<f32>>> + Send + 'async_trait>>where
Self: 'async_trait,
'life0: 'async_trait,
'life1: 'async_trait,
'life2: 'async_trait,
Calculate embeddings for text
Source§fn complete_stream<'a>(
&'a self,
model: &'a str,
messages: &'a [LlmMessage],
tools: Option<&'a [Box<dyn LlmTool>]>,
config: &'a CompletionConfig,
) -> Pin<Box<dyn Stream<Item = Result<StreamChunk>> + Send + 'a>>
fn complete_stream<'a>( &'a self, model: &'a str, messages: &'a [LlmMessage], tools: Option<&'a [Box<dyn LlmTool>]>, config: &'a CompletionConfig, ) -> Pin<Box<dyn Stream<Item = Result<StreamChunk>> + Send + 'a>>
Stream LLM responses chunk by chunk Read more
Source§fn complete_stream_events<'a>(
&'a self,
model: &'a str,
messages: &'a [LlmMessage],
config: &'a CompletionConfig,
) -> Result<StreamEventStream<'a>, StreamEventError>
fn complete_stream_events<'a>( &'a self, model: &'a str, messages: &'a [LlmMessage], config: &'a CompletionConfig, ) -> Result<StreamEventStream<'a>, StreamEventError>
Stream one turn as
crate::llm::StreamEvents with terminal completion evidence. Read moreAuto Trait Implementations§
impl !RefUnwindSafe for OllamaGateway
impl !UnwindSafe for OllamaGateway
impl Freeze for OllamaGateway
impl Send for OllamaGateway
impl Sync for OllamaGateway
impl Unpin for OllamaGateway
impl UnsafeUnpin for OllamaGateway
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more