Guides

• LLM Calls

Use LLMService for direct completions, streaming, and embeddings without the full agentic loop.

• Tools

Decorate functions and classes with @tool() to expose them to agents. Covers schema generation, ToolContext, DI injection, and caching.

• Agents

Build agents with @agent(), @use_tools(), lifecycle hooks, delegation, and full DI integration via AgentRunner.

🗄 Memory

Four-tier memory architecture: short-term window, conversation history, per-user long-term facts, and vector store for RAG.

• User Memory

Persist facts about individual users across conversations with @remember(), UserMemoryStore, and MemoryFact.

• Multi-Agent Systems

Delegate between agents via tool-based handoff — every transition is visible in the tool-call log and composes with run_stream().

• Agent Teams

Compose specialist agents with @team() in coordinator or collaborate mode. Stream TeamEvent instances as workers produce results.

• Knowledge Base

Load documents, chunk them, embed them into InMemoryVectorStore, and inject relevant context into agents automatically.

• Prompt Templates

Build reusable, composable prompts with PromptTemplate, ChatPromptTemplate, and FewShotPromptTemplate.

• Output Parsers

Transform raw LLM text into typed Python objects with StrOutputParser, JSONOutputParser, and PydanticOutputParser.

• Structured Output

Guarantee every completion matches a Pydantic schema using StructuredLLM and llm_service.with_structured_output(Model).

• Multimodal Inputs

Send images, audio, and documents to the LLM using ImageContent, AudioContent, and DocumentContent alongside text messages.

• Semantic Router

Route natural-language queries to named handlers using embedding-based similarity with SemanticRouter and Route.

• Streaming

Stream tokens, tool results, and agent turns via run_stream() and CompletionChunk — with SSE controller integration.

• Guardrails

Block prompt injection, redact PII, enforce topic and length constraints, and build custom LLM-evaluated guardrails.

🗺 Workflows & Chains

Compose templates, LLM calls, and parsers into typed pipelines with Chain, RunnableLambda, and | operator chaining.

• Cost Tracking

Track token usage and USD cost per model and conversation with CostTracker, TokenBudget, and RateLimiter.

• Tracing & Observability

Record structured spans with @traced(), export to OpenTelemetry or a custom TraceStore, and inspect the full agent execution tree.

• Testing

Write deterministic, zero-network tests with MockTransport, AgentTestClient, and queued mock responses.

• Evaluation

Score agent outputs against expected answers using built-in evaluators and the EvalSuite runner.

• Extended Thinking

Enable chain-of-thought reasoning in supporting models by passing thinking=True to the transport or AgentConfig.