After the first run

In one line: Four changes that turn the demo into the start of your own client
You need: Stream a first reply working, with llama-server still up on port 1234
Read this if: The demo ran and you want to know what to change first

Each section below stands alone. Take the one that matches what you need next.

See every block type

llamacpp_gemma prints Text blocks and drops the rest. stream_all_blocks matches on all five MessageContentBlock variants, which is the shape real consuming code has:

cargo run --example stream_all_blocks --features provider-llamacpp

Text goes to stdout. Everything else goes to stderr, tagged:

[reasoning] ...
[tool call] <name> <arguments> (id: <id>)
[image] <media_type> (base64 data omitted)
[tool result] for tool call <tool_call_id>
[error] <CucaError Display text>

An [error] line stops the drain. Which tags appear depends on the server and the model: a plain prose reply is all Text, while a reasoning model buries it. One run against google/gemma-4-12b-qat wrote 1506 [reasoning] lines to stderr for two lines of answer on stdout:

1. In Romanian folklore, the **Cuca** is a legendary bogeyman used to frighten children into behaving.
2. It is traditionally depicted as a witch-like creature with a long nose and long hair.

Add a plugin

Plugins are the crate's only extension point, and a plugin is one impl block. custom_plugin counts blocks as they stream and reports the aggregated response when the stream ends:

cargo run --example custom_plugin --features provider-llamacpp

CUCA most commonly refers to either the Credit Union of Central Alabama or the Center for Urban Community Action, depending on the context.
[example-block-counter] model=google/gemma-4-12b-qat duration=81.89s completion_tokens=1991 blocks=1991

completion_tokens and prompt_tokens are the server's own counts: the chat route asks for stream_options.include_usage, and llama-server answers with a final usage frame. A streamed delta carries about one token, so the two numbers on the line roughly agree; both are this large because the example sets no max_tokens and this model reasons before it answers. A server that reports no usage leaves prompt_tokens at 0 and completion_tokens counting one per Text, Thinking and ToolCall block.

To write your own, see Write a custom plugin.

Depend on the crate from your own project

The crate is named cuca and is not on crates.io, so a dependant names the checkout by path and picks its features explicitly:

cargo add cuca --path /path/to/cuca --features provider-llamacpp

Swap provider-llamacpp for whichever adapter speaks to your backend. The eight choices (seven HTTP adapters and the in-process provider-local) and what each one talks to are in Providers. Add plugin-* features the same way, one per capability you want compiled in.

Turn thinking on

UnifiedRequest::thinking is None by default, and providers with no reasoning knob ignore it either way. One unified effort level maps onto each provider's native control:

use cuca::{ThinkingEffort, UnifiedRequest};

let request = UnifiedRequest::new(model)
    .add_user_message("Explain the trade-off.")
    .enable_thinking(Some(ThinkingEffort::Medium));

Medium becomes reasoning_effort: "medium" on OpenAI-compatible routes, budget_tokens: 8192 on Anthropic's budget mode, thinkingLevel: "MEDIUM" on Gemini. The full mapping table, including where two effort levels collapse onto one provider value, is in The unified request and stream.

Point somewhere other than llama.cpp

Any OpenAI-compatible server, including Ollama, vLLM and LM Studio, is a base URL change plus the right provider feature. That is its own guide: Point the client at another OpenAI-compatible server.

Run the tests

The unit tests need no server:

cargo test --no-default-features --features provider-openai

The integration suite is server-dependent and skips what it cannot reach. Its feature list, its three environment variables and how to make a skip fail loudly are in Run the live integration suite.

Next page: The unified request and stream, which explains why the request has the fields it has.