Skip to content

yaab.models

The model layer: LiteLLM provider, router, resilience, TestModel.

yaab.models

Model layer: provider protocol, LiteLLM, instrumentation, and test doubles.

ModelProvider

Bases: Protocol

The pluggable model interface.

complete async 

complete(messages: list[Message], *, tools: list[dict[str, Any]] | None = None, output_schema: dict[str, Any] | None = None, tool_choice: Any | None = None, **kwargs: Any) -> ModelResponse

Return a single completion for messages.

tool_choice follows the OpenAI convention: "auto", "required", "none", or a {"type": "function", "function": {"name": ...}} dict.

stream 

stream(messages: list[Message], *, tools: list[dict[str, Any]] | None = None, **kwargs: Any) -> AsyncIterator[StreamChunk]

Yield incremental chunks for messages.

ModelResponse

Bases: BaseModel

A normalized model response in OpenAI-ish shape.

StreamChunk

Bases: BaseModel

An incremental delta during streaming.

InstrumentedModel

Decorate a model with GenAI-convention tracing.

LiteLLMModel

Provider-agnostic model over LiteLLM's unified interface.

ModelRouter

A :class:ModelProvider that dispatches each request to one of routes.

Parameters

routes: Map of route key -> model. Values may be a :class:ModelProvider instance or a string spec resolved lazily via :func:yaab.models.resolve_model. classifier: A built-in name ("length"), a callable (messages, tools) -> key (sync or async), or None to use the default "length" classifier. default: Route key used when the classifier returns an unknown key. Defaults to the first route key, so routing always resolves to a real model. complexity_threshold: Char threshold for the built-in "length" classifier.

FunctionModel

A model whose response is computed by a user-supplied function.

TestModel

A scripted, deterministic model.

Parameters

custom_output: Text to return as the final assistant message. responses: A sequence of :class:ModelResponse (or strings) returned in order, one per complete call. Overrides custom_output when set. call_tools: Tool names to call (with empty args) on the first response, before producing a final text answer. Useful for exercising the tool loop. structured_output: A dict returned (JSON-encoded) when the agent requests structured output, so output validation can be tested without a real model.

resolve_model 

resolve_model(model: str | ModelProvider) -> ModelProvider

Coerce a model spec into a :class:ModelProvider.

Strings are treated as LiteLLM model identifiers (e.g. "openai/gpt-4o", "anthropic/claude-..."); provider instances pass through unchanged.