CUCA

The unified request and stream

In one line
The unified types are the narrowest shape every backend can agree on, and every remaining difference lives in the provider adapter
You need
Nothing running
Read this if
You are wondering why the request has the fields it has, or where a provider quirk belongs

The seven HTTP backends use four wire shapes. OpenAI, vLLM, LM Studio, DeepSeek's native route and llama.cpp's chat route post /chat/completions. Anthropic and DeepSeek's bridge route post /messages. Gemini posts streamGenerateContent. llama.cpp's native route posts /completion with raw token frames. The eighth provider runs a model in process and has no wire.

Switching between two of them would otherwise mean rewriting the request body, the auth header, the frame reader and the tool-call round trip. UnifiedRequest and MessageContentBlock make that switch a feature flag and a base URL.

One turn

The six stages of one CUCA turn Six stages, left to right. A UnifiedRequest passes through the on_request hook of every registered plugin. The selected provider adapter builds the vendor body and posts it over HTTP. The reply bytes feed SseStreamParser, which emits protocol level SSE events. The same provider adapter translates those events into MessageContentBlock values. Each block then passes through every plugin's on_stream_chunk hook before reaching the caller. One turn through CucaClient::generate_stream Unified Request on_request hooks provider adapter SseStream Parser adapter translate on_stream _chunk stages 3 and 5 are the provider adapter; the wire format never leaves it

Stages 3 and 5 are the only places vendor JSON exists. Stage 4 is protocol work with no vendor knowledge, and stages 2 and 6 see normalized types only.

generate_stream overwrites request.provider with the client's selected provider before any hook runs. Setting the field on a UnifiedRequest therefore cannot mis-route it, and a plugin reading req.provider in on_request always sees the real endpoint. UnifiedRequest::new starts with ProviderEndpoint::Custom("") for that reason.

What the request carries

UnifiedRequest has nine fields: model, provider, messages, temperature, max_tokens, stream, thinking, tools and prompt_cache. Three are Option: temperature, max_tokens and thinking. An unset Option means the adapter omits the key instead of inventing a default, so an OpenAI-compatible body has no temperature or max_tokens unless the caller set them.

Anthropic is the exception because its API requires max_tokens. That adapter sends 1024 when the field is unset, since the alternative is refusing a request the caller considers complete.

The request has no field for a vendor-specific knob, and the type is built around that constraint. A HashMap<String, Value> of pass-through options would make UnifiedRequest easy to extend and would end its portability the first time anyone used it. The one vendor override is ThinkingParams, a closed enum with variants for Anthropic, Gemini and OpenAI-compatible reasoning, plus None.

Thinking: one level, several translations

ThinkingEffort has five levels. Each provider gets its closest native control, and where a provider has fewer gradations the levels collapse.

EffortOpenAI-compatible reasoning_effortAnthropic budget_tokensAnthropic adaptive effortGemini thinkingLevel
Minimal"minimal"1024"low""LOW"
Low"low"2048"low""LOW"
Medium"medium"8192"medium""MEDIUM"
High"high"16384"high""HIGH"
XHigh"high"16384"xhigh""HIGH"
  • DeepSeek's native route has no effort knob. It takes a mode object, {"type":"enabled"} or {"type":"disabled"}, and drops the effort level.
  • llama.cpp's chat route uses the OpenAI-compatible column. Its native /completion route has no reasoning control and ignores thinking.
  • ThinkingParams carries a native value for a caller who needs it, and a set override wins over the unified effort. Using it costs portability.

Five block types

MessageContentBlock has Text, ImageBase64, Thinking, ToolCall and ToolResult. Every adapter's translator emits only these, so code written against one backend behaves the same against the others.

The serialization is adjacently tagged with #[serde(tag = "type", content = "value")]. Internal tagging cannot represent a newtype variant, and Text(String) is one. Session logging serializes Text blocks, so every variant has to round-trip.

The Custom provider

ProviderEndpoint has nine variants. Eight have adapters: LlamaCpp, Vllm, LmStudio, OpenAi, Anthropic, GoogleGemini, DeepSeek and Local. The ninth is Custom(String), and dispatching it always fails.

configuration error: custom endpoints require a registered adapter

The crate does not assume an unknown gateway is OpenAI-compatible. A wrong guess sends a malformed request to a real endpoint and returns a confusing error from the far side. A caller who knows the gateway's protocol selects that protocol's feature and overrides the base URL, as in Point the client at another OpenAI-compatible server.

Where a provider quirk goes

Normalized types know nothing about vendors, and vendors are known only inside src/provider/. A quirk is therefore an adapter change, never a new field on UnifiedRequest.

  • Gemini's functionResponse names the function, not the call id, so a ToolResult is matched by UnifiedMessage::name when set, with the call id as a fallback.
  • DeepSeek's bridge route rewrites claude-opus to deepseek-v4-pro, claude-sonnet and claude-haiku to deepseek-v4-flash, and forces prompt caching off.
  • llama.cpp's native route builds one prompt string with ### User: and ### Assistant: markers, because that route has no message array.

None of this is visible in the unified types. Each provider's page under Providers states its own.

Next page: The SSE parser.

↑↓ to navigate ↵ to open esc to close