Providers

In one line

Eight provider-* features, each owning one dispatch_* method and one ProviderEndpoint variant

You need

At least one provider-* feature enabled; the crate does not compile without one

Read this if

You need a base URL, an auth header, a route, or the exact behavior one adapter does not share with the others

Compare the adapters

• The default base URL applies when the client's base URL is empty. Set another with CucaClientBuilder::with_base_url.
• Each provider page lists the route, the request and stream translation, thinking, errors and limits under the same headings.
• Every HTTP feature enables reqwest with its rustls backend and tokio-stream. provider-anthropic also enables sha2, getrandom and base64.
• The feature matrix maps every feature to its dependencies.

Wire formats

The seven HTTP adapters speak four wire formats and share two translation modules.

• src/provider/openai_compat.rs holds the OpenAI-compatible translation, documented on the OpenAI page.
• src/provider/anthropic.rs holds the Anthropic translation, documented on the Anthropic page.
• Gemini and the llama.cpp native route each have their own translator.

The in-process adapter

provider-local runs a GGUF language model inside your process and exposes no HTTP endpoint. It has no base URL, auth header or route.

• ProviderEndpoint::Local selects it, and CucaClientBuilder::with_local_model attaches the model. build fails when the selected provider is Local and no model is attached.
• The CPU is the default device. provider-local-vulkan, provider-local-metal, provider-local-cuda and provider-local-rocm add a GPU engine that LocalModelConfig::device selects.
• It does not use tokio-stream. It uses reqwest only for the one-time model download.
• The engine is pure Rust. The candle dependency tree builds one C crate, onig_sys.

See Local (in-process) for configuration and behavior, and Local models and formats for the supported architectures and presets.

Dispatch

There is no provider trait. Each adapter module contributes an inherent impl CucaClient block with one method:

pub(crate) async fn dispatch_<provider>(
    &self,
    req: UnifiedRequest,
) -> Result<ProviderDispatch, CucaError>

CucaClient::generate_stream sets request.provider to the client's selected provider, runs the on_request hooks, then selects an adapter with a match on self.selected_provider. Each arm is gated on its feature. A mirrored #[cfg(not(feature = ...))] arm returns CucaError::ProviderNotEnabled with the missing flag name.

Custom endpoints

ProviderEndpoint has nine variants: the eight above and Custom(String). Custom has no feature and no adapter. Dispatching it returns CucaError::Config:

configuration error: custom endpoints require a registered adapter

• ProviderEndpoint::default() is OpenAi.
• UnifiedRequest::new sets the request's provider to Custom(String::new()), and generate_stream overwrites it with the client's provider before any hook runs.

Environment variables

None of the seven HTTP adapters reads an environment variable. Base URLs, API keys and bearer tokens reach them only through CucaClientBuilder. The CUCA_* variables that belong to the examples and the test harness are listed in Environment variables.

provider-local reads two variables as fallbacks:

• CUCA_CACHE_DIR, then the platform's cache variables, when LocalModelConfig::cache_dir is None.
• HF_TOKEN when LocalModelConfig::hf_token is None.

Setting the config fields keeps it from reading either.

Related pages

• Unified request explains the request every adapter translates.
• Use another OpenAI-compatible server shows how to point an adapter at a server not listed here.