Local requests and responses

In one line: The request is rendered through the model's chat template; the stream carries Text, Thinking and ToolCall blocks with exact usage counts.
You need: The provider-local feature and a loaded LocalModel.
Read this if: You need to know how a request field reaches the template, or what the stream and the usage numbers mean.

Request translation

UnifiedRequest fields map to the model's chat template and the sampler:

messages: System and User messages carry their Text blocks joined with a newline as content. An Assistant message adds its Thinking blocks, concatenated, as reasoning_content, and its ToolCall blocks as tool_calls entries {id, type: "function", function: {name, arguments}}. A Tool message carries the tool_call_id (from the message, else from its first ToolResult), an optional name, and its ToolResult outputs joined with a newline as content.
Tool call arguments stay JSON objects. An object, null or a string holding an object is accepted; anything else fails the request.
tools reaches the template as {"type": "function", "function": {name, description, parameters}} with parameters from input_schema. The template sees the tools variable only when the request declares at least one tool.
temperature and max_tokens apply per request. top_k, top_p and seed come from SamplingConfig.
thinking sets the enable_thinking and thinking template variables; see Thinking.
A message with an ImageBase64 block is rejected: the engine is text only.
The prompt is rendered with add_generation_prompt true. Template variables and helpers are listed on Local models and formats.

The stream yields Result<MessageContentBlock, CucaError> items: Text, Thinking and ToolCall blocks. A ToolCall id is generated by the engine as call_ plus 16 hexadecimal digits.
How reasoning and tool calls are recognized depends on the model's template; see Reasoning and Tool calls.
A malformed tool call becomes a CucaError::Provider item with the text malformed tool call: <body>, and the stream continues.
Generation stops at an end-of-generation token, at max_tokens, or when the context is full.
Dropping the stream stops generation. The work already done stays in the K/V cache and is reusable.

Usage is exact, not estimated: prompt_tokens equals the prefix-reuse read plus write counts, and a max_tokens cap is met to the token.
prompt_cache_usage reports read_tokens (reused from the previous prompt) and write_tokens (prefilled). The engine keeps the longest common prefix of the cached ids and the new prompt, always leaving the last prompt token to prefill.
On the CPU the output of a request that reuses a prefix equals the output of a cold request, bit for bit.
reasoning_tokens is set only when the request enabled thinking. It counts the tokens inside reasoning blocks, markers included.

Read the usage per call with CucaClient::generate_stream_with_response.

Thinking is on or off. A request whose thinking is enabled sets the template variables enable_thinking and thinking to true; its effort level is ignored. A disabled or missing thinking sets both to false. preserve_thinking is always false.
Reasoning arrives as Thinking blocks. If the rendered prompt ends inside an open reasoning block, generation starts in thinking.
A model whose template reads neither variable, such as the DeepSeek-R1 distills, reasons on every request.
LocalModel::capabilities() reports what the model supports, derived from its own chat template and output conventions: reasoning (it writes a reasoning block), can_disable_thinking (the template reads enable_thinking) and tools (it has a tool syntax the output parser understands).