AI & Agents

The AI runtime: single-shot model calls, a tool-using ReAct agent with memory, a multi-agent orchestrator, LLM-powered branching, RAG retrieval, web search, and vision/multimodal analysis. See AI Agents & Tools for the conceptual overview, built-in tool list, and memory model — this page is the field-level configuration reference.

llm

Single call to a chat-completion model. Returns the raw text response plus token usage and an estimated USD cost.

{
  "id": "summarize",
  "type": "llm",
  "config": {
    "provider": "anthropic",
    "model": "claude-sonnet-4-6",
    "api_key": "{{ secret.ANTHROPIC_API_KEY }}",
    "system": "You are a concise summarizer.",
    "prompt": "Summarize: {{steps.fetch.output.body}}",
    "max_tokens": 1024,
    "temperature": 0.3
  }
}

Field	Type	Default	Description
provider	string	"anthropic"	anthropic, openai, gemini, ollama, openrouter, groq, deepseek, nvidia, or custom (any OpenAI-compatible endpoint).
model	string	provider-specific	E.g. claude-sonnet-4-6 (anthropic), gpt-4o (openai), gemini-2.5-flash (gemini), llama3.2 (ollama), llama-3.3-70b-versatile (groq), deepseek-chat (deepseek), meta/llama-3.1-70b-instruct (nvidia), anthropic/claude-sonnet-4-6 (openrouter).
prompt	string (expression)	—	Required. The user message.
system	string	"" (anthropic/gemini) or "You are a helpful assistant." (others)	System prompt.
api_key	string (expression)	—	Required for all providers except ollama. Usually {{ secret.* }}.
base_url	string	provider default	Required for custom; overrides the endpoint for ollama (default http://localhost:11434/v1) and other providers.
max_tokens	number	1024	Maximum output tokens.
temperature	number	provider default	Sampling temperature; omitted from the request if unset.
top_p	number	provider default	Nucleus sampling.
top_k	number	provider default	Anthropic/Gemini only.
frequency_penalty / presence_penalty	number	provider default	OpenAI-compatible providers only.
seed	number	—	OpenAI-compatible providers only (deterministic sampling where supported).
stop_sequences	string (comma- or \n-separated)	—	Stop sequences.
response_format	string	—	"json_object" or "json_schema" — OpenAI-compatible providers only.

Output:

Field	Type	Description
text	string	The model's response text.
model / provider	string	The resolved model and provider.
usage	object	Raw usage object as returned by the provider.
input_tokens / output_tokens	number	Prompt and completion token counts.
cost_usd	number	Estimated cost based on a built-in per-model pricing table (per-million-token input/output rates for Anthropic, OpenAI, Gemini, NVIDIA NIM, Groq, DeepSeek, and OpenRouter). Returns 0 for ollama (local) and for any model not in the pricing table.
resolved_prompt / system_prompt	string	The prompt and system prompt after expression resolution.

agent

Runs a full ReAct (reason + act) tool-calling loop as a single durable Temporal activity, powered by the Eino agent framework (EinoAgentNode, the registered implementation of type: "agent"). An older AgentNode in internal/nodes/ai/agent.go defines the same Type() but is not registered with nodes.GlobalRegistry — it is legacy/unused, retained only as a helper invoked internally by the orchestrator node's per-role workers.

See AI Agents & Tools for the full list of builtin_tools, connector tools, skills, and how memory works.

{
  "id": "research_agent",
  "type": "agent",
  "config": {
    "provider": "anthropic",
    "model": "claude-sonnet-4-6",
    "api_key": "{{ secret.ANTHROPIC_API_KEY }}",
    "system": "You are a research assistant. Always cite sources.",
    "prompt": "Research recent news about {{trigger.body.topic}} and write a 3-paragraph brief.",
    "max_turns": 15,
    "temperature": 0.3,
    "builtin_tools": ["web_search", "kb_search", "current_datetime"],
    "tools": [
      { "type": "connector", "connector_type": "tool-slack", "actions": ["send_message"] }
    ],
    "skills": ["skill_abc123"],
    "memory_mode": "session",
    "memory_session_id": "{{trigger.body.user_id}}",
    "memory_window": 20
  }
}

Field	Type	Default	Description
provider	string	"anthropic"	anthropic, openai, nvidia, or any other value combined with base_url (OpenAI-compatible).
model	string	claude-sonnet-4-6 (anthropic), gpt-4o (openai), meta/llama-3.1-70b-instruct (nvidia)	Model ID.
api_key	string (expression)	—	Required.
base_url	string	provider default	Required for providers other than anthropic/openai/nvidia.
system	string	"You are a highly capable AI agent. Think step by step. Use tools when needed."	Base system prompt. The platform appends a tool manifest and (if memory_mode is enabled) prior conversation context.
prompt	string (expression)	—	Required. The task/user message for this run.
temperature	number	provider default	-1 (unset) leaves it to the provider.
max_turns	number	15	Maximum reasoning/tool-call iterations. Internally mapped to an Eino graph step budget (max_turns*3 + 5); exceeding it returns an error suggesting you raise max_turns or simplify the prompt.
builtin_tools	array of strings	[]	Platform tool names — see Built-in tools.
tools	array of objects	[]	Connector-backed tool definitions, e.g. { "type": "connector", "connector_type": "tool-github", "actions": ["create_issue"] }.
skills	array of strings	[]	Skill IDs exposed via progressive disclosure (name + description until called).
max_skill_rounds	number	20	Max internal reasoning rounds when a skill is invoked.
searxng_url	string	"http://localhost:8888"	SearXNG instance used by the web_search/search built-in tool.
memory_mode	string	"none"	"none" or "session" — see Memory.
memory_session_id	string (expression)	derived	Conversation key for memory. Falls back to trigger.session_id, then the execution ID.
memory_window	number	20	Number of past turns loaded into the system prompt.

Output:

Field	Type	Description
text	string	The agent's final answer.
tool_calls	array	Full reasoning trace — each entry has name, input, output, error, started_at, duration_ms.
turns	number	Number of tool calls made (len(tool_calls)).
resolved_prompt / system_prompt	string	Prompt and system prompt after resolution (system prompt includes the appended tool manifest and memory context).
provider / model	string	The resolved provider and model.
warning	string	Present only if tools were configured but none were called.

note

Unlike llm, the agent node's output map does not currently include input_tokens, output_tokens, or cost_usd — token/cost accounting referenced in AI Agents & Tools applies to the llm node's output shape; per-step cost tracking for agent runs is derived from the underlying model calls rather than the top-level step output.

orchestrator

Runs a supervisor + specialist multi-agent team for complex tasks (e.g. software engineering), iterating between workers and a reviewer.

{
  "id": "build_feature",
  "type": "orchestrator",
  "config": {
    "task": "Build a REST API for user management in Go",
    "provider": "anthropic",
    "model": "claude-sonnet-4-6",
    "api_key": "{{ secret.ANTHROPIC_API_KEY }}",
    "team": ["architect", "developer", "reviewer", "tester"],
    "max_rounds": 3
  }
}

Field	Type	Default	Description
task	string (expression)	—	Required. The overall objective.
provider	string	"anthropic"	anthropic, openai, or nvidia (passed through to each role's underlying agent).
model	string	provider default	Model ID used by every role.
api_key	string (expression)	—	Required.
team	array of strings	all four roles	Subset/order of architect, developer, reviewer, tester. Unrecognized or empty input falls back to the full default team.
max_rounds	number	2	Number of architect → workers → reviewer iteration rounds. Stops early once the reviewer responds with APPROVED after round 1.

Each role runs with a fixed system prompt and a fixed built-in tool set (architect: search, browser; developer/tester: terminal, write_file, read_file, list_files, git; reviewer: terminal, read_file, list_files) via the legacy internal agent runner, with max_turns: 20 per role.

Output:

Field	Type	Description
report	string	Markdown report combining each role's output, in architect → developer → reviewer → tester order.
results	object	Map of role name → that role's raw text output (or "ERROR: ..." if the role failed).
task	string	The resolved task string.

condition_agent

Uses an LLM to pick exactly one of several named scenarios, exposing the choice as a branch for branch_deps — a semantic counterpart to the rule-based condition/switch nodes.

{
  "id": "classify_ticket",
  "type": "condition_agent",
  "config": {
    "provider": "anthropic",
    "api_key": "{{ secret.ANTHROPIC_API_KEY }}",
    "model": "claude-haiku-4-5-20251001",
    "instructions": "Classify this support ticket.",
    "input": "{{trigger.body.message}}",
    "scenarios": ["billing", "technical", "other"]
  }
}

Field	Type	Default	Description
provider / api_key / model / base_url	—	same as llm	Passed straight to an internal llm call.
instructions	string (expression)	""	What the router should decide, e.g. "Is the user asking about billing?".
input	string (expression)	—	Required. The text/value to classify.
scenarios	array, JSON-array string, or comma/newline-separated string	—	Required, minimum 2. Each entry is either a plain string or { "scenario": "..." }.
temperature	number	0	Deterministic by default.

Matching is exact → prefix → substring → last scenario as the catch-all "else" branch.

Output:

Field	Type	Description
branch	string	The selected scenario — used by the workflow engine's branch_deps to gate downstream steps.
scenario	string	Same value as branch.
index	number	Index of the matched scenario in scenarios.
scenarios	array	The full resolved scenario list.
llm_output	string	Raw text returned by the underlying model.

query-kb

Performs vector similarity search against a Knowledge Base and returns the top-matching chunks, both structured and as a ready-to-inject text block.

{
  "id": "search_docs",
  "type": "query-kb",
  "config": {
    "kb_id": "f47ac10b-58cc-4372-a567-0e02b2c3d479",
    "query": "{{trigger.body.question}}",
    "top_k": 5
  }
}

Field	Type	Default	Description
kb_id	string (UUID)	—	Required. The Knowledge Base to search.
query	string (expression)	—	Required. The search query; embedded using the KB's configured embedding model.
top_k	number	5	Clamped to the range 1–20.

Output:

Field	Type	Description
results	array	Each entry: chunk_id, doc_id, doc_name, content, score, metadata.
context	string	All results concatenated as [Source: <doc_name>, Score: <score>] followed by <content>, separated by ---, ready for prompt injection.
total	number	Number of results returned.
query	string	The resolved query.
kb_id / kb_name	string	The KB queried.

The same retrieval is also available to agents as the kb_search built-in tool (see AI Agents & Tools).

ai-search

AI-powered web search that returns a synthesized answer with citations — distinct from query-kb, which searches your own Knowledge Base documents rather than the live web.

{
  "id": "web_research",
  "type": "ai-search",
  "config": {
    "provider": "perplexity",
    "api_key": "{{ secret.PERPLEXITY_API_KEY }}",
    "query": "{{trigger.body.topic}} latest news",
    "model": "sonar",
    "max_results": 5
  }
}

Field	Type	Default	Description
provider	string	"perplexity"	perplexity, tavily, or brave.
api_key	string (expression)	—	Required.
query	string (expression)	—	Required.
max_results	number	5	Maximum results returned.
model	string	"sonar"	Perplexity only — sonar, sonar-pro, sonar-deep-research.
search_depth	string	"advanced"	Tavily only — basic or advanced.
include_answer	string ("true"/"false")	"true"	Tavily only — whether to include a synthesized AI answer.
country	string	"US"	Brave only — country code.
freshness	string	—	Brave only — pd (past day), pw, pm, or py.

Output:

Field	Type	Description
answer	string	Synthesized answer (Perplexity and Tavily only — empty for Brave, which returns raw results only).
results	array	{ title, url, snippet, score } (fields populated vary by provider; Perplexity only populates url per citation).
citations	array of strings	Source URLs.
query	string	The resolved query.
provider	string	The provider used.
model	string	Perplexity only — model used.
input_tokens / output_tokens	number	Perplexity only — prompt/completion token counts.

vision

Sends an image to a multimodal model for analysis — OCR, chart reading, image classification, general description.

{
  "id": "read_screenshot",
  "type": "vision",
  "config": {
    "provider": "openai",
    "api_key": "{{ secret.OPENAI_API_KEY }}",
    "image_url": "{{steps.upload.output.url}}",
    "prompt": "Extract all text from this image.",
    "model": "gpt-4o"
  }
}

Field	Type	Default	Description
provider	string	"openai"	openai or anthropic.
api_key	string (expression)	—	Required.
image_url	string (expression)	—	URL to the image. One of image_url or image_base64 is required. Anthropic accepts URL sources directly; OpenAI accepts a hosted URL or data URL.
image_base64	string (expression)	—	Base64-encoded image data, alternative to image_url. MIME type is auto-detected from the data's leading bytes (PNG, GIF, WEBP; defaults to JPEG).
prompt	string (expression)	"Describe this image in detail. Extract any visible text (OCR). List the main objects and their approximate positions."	What to ask about the image.
model	string	gpt-4o (openai), claude-haiku-4-5-20251001 (anthropic)	Vision-capable model. Other examples: gpt-4o-mini, claude-opus-4-7.
detail	string	"auto"	OpenAI only — auto, low, or high.
max_tokens	number	1024	Maximum output tokens.

To analyze a file produced inside a sandbox (e.g. a generated chart or a PDF page rendered to an image), first write it to durable storage or convert to base64 with a code step, then pass the result as image_url/image_base64 — vision itself does not read directly from the sandbox filesystem.

Output:

Field	Type	Description
description	string	The model's response text.
model / provider	string	The resolved model and provider.
input_tokens / output_tokens	number	Prompt/completion token counts.

Next

• HTTP, Files & Storage
• Documents & Media
• AI Agents & Tools (Core Concepts)