Agentic AI

A model that loops: it observes, decides, calls something outside itself, reads the result, and decides again. Autonomy is a dial, not a category — and every notch costs predictability.

1. Agent, workflow, chatbot: three different things

	Chatbot	Workflow	Agent
Control flow	Human writes every turn	Code decides the path	Model decides the next step
Number of model calls	One per message	Fixed, known in advance	Variable, bounded by a budget
Failure surface	Bad answer	Brittle branch	Loops, drift, side effects
Best for	Conversation, drafting	Repeatable business process	Open-ended goals over tools
Testability	Prompt evals	Deterministic unit tests	Trajectory evals plus budgets

Autonomy ladder

Level	Who decides	Example	Risk
L0 — suggested	Human	Draft a reply	Almost none
L1 — assisted	Human, model retrieves	Search and propose a fix	Bad evidence
L2 — approve each action	Model proposes, human confirms	Create a ticket after review	Confirmation fatigue
L3 — bounded autonomy	Model, inside limits	Refund up to a threshold	Silent accumulation of small errors
L4 — unsupervised	Model	Nightly reconciliation with escalation	Blast radius

2. The loop

The canonical formulation is ReAct: interleave a short reasoning trace with an action, feed the observation back, repeat. Modern implementations add an explicit verification step before the exit.

while budget.remaining() and steps < MAX_STEPS:
    thought  = model.reason(goal, history, tools)
    if thought.is_final: break
    action   = thought.tool_call            # validated against schema
    result   = registry.invoke(action, timeout=..., idempotency_key=...)
    history += [thought, action, truncate(result)]
verify(history)                             # own call, own criteria
return answer_or_escalation(history)

3. The four capabilities

Capability	Question it answers	Implemented by
Planning	What sequence of steps reaches the goal?	Decomposition prompt, plan-and-execute, tree search
Tool use	How do I affect the world or read live data?	Typed function calling, MCP servers, HTTP APIs
Memory	What must survive the next turn?	Scratchpad, summary, vector store, structured state
Reflection	Is the result good enough to ship?	Self-critique, verifier model, executable checks

4. Orchestration patterns

Pattern	Shape	Use when	Cost
Single agent + tools	One loop, many tools	Most tasks; start here	Low
Router	Classify, then dispatch to a specialist prompt	Distinct intents with different contracts	Low
Supervisor	Manager agent delegates to workers, merges results	Parallelisable sub-tasks	Medium
Pipeline / chain	Fixed sequence of agents	Stages with clean hand-offs	Low, deterministic
Hierarchical	Supervisors of supervisors	Large tool surfaces	High, hard to debug
Swarm / handoff	Peers pass control	Support triage with specialisms	Medium, unbounded

5. Designing tools the model can use

Rule	Why
One verb per tool; no boolean mode flags	The model picks tools by description, not by branching inside a call
Descriptions state when not to use it	Prevents the most common wrong selection
Schema validated before execution	Turns a malformed call into a clear error the model can fix
Errors are structured and actionable	{"error":"not_found","hint":"use search_orders first"}
Writes require an idempotency key	Retries are normal in agent loops
Hard timeout and page-size limits	A single slow call can end the whole task
Results truncated to what matters	Tool output is the main driver of context growth
Permissions scoped per tool, per tenant	The agent must not inherit the user's full authority

{
  "name": "get_order_status",
  "description": "Return status for one order. Use only when the user supplies an order id; use search_orders when they describe the order instead.",
  "inputSchema": { "type": "object", "properties": { "order_id": { "type": "string", "pattern": "^[A-Z]{2}-\\d{6}$" } }, "required": ["order_id"] },
  "outputSchema": { "type": "object", "properties": { "status": { "enum": ["placed","shipped","delivered","cancelled"] }, "eta_days": { "type": "integer" } } }
}

6. Memory and context

Layer	Contents	Lifetime	Failure if misused
Working context	Goal, recent steps, tool results	One task	Overflow, lost early instructions
Scratchpad	Plan, open questions, decisions	One task	Stale plan followed blindly
Episodic store	Past tasks, outcomes, corrections	Weeks	Retrieving irrelevant precedent
Semantic store	Domain knowledge, documents	Long	Outdated or poisoned content
Structured state	Entities, ids, statuses	Long	Divergence from the system of record

7. Why long tasks fail

At 99% per step, a 50-step task succeeds 60% of the time. At 95%, it succeeds 8% of the time. Autonomy multiplies error, which is why budgets and checkpoints matter more than model choice.

Failure mode	Symptom	Control
Loop	Same tool called with the same arguments	Repeat detection, max steps, forced replan
Goal drift	Answers a nearby question	Restate the goal every N steps
Context rot	Ignores early instructions	Compaction, pinned system block
Overreach	Deletes or sends more than asked	Least privilege, dry-run mode, approvals
False completion	Claims success without evidence	Verifier with executable checks
Cascading bad data	A wrong id propagates	Validate entities against the source before use

8. Guardrails and authority

Control	Implementation	Blocks
Least privilege	Per-tool, per-tenant credentials, no wildcard scopes	Full-account damage
Human in the loop	Approval gate for irreversible actions	Silent destructive writes
Sandbox	Container, no network by default, read-only mounts	Code execution escapes
Budgets	Tokens, steps, money, wall-clock per task	Runaway cost
Allow-list	Domains, tables, repositories the agent may touch	Scope creep
Dry run	Produce the diff or the payload, apply on approval	Irreversible mistakes
Audit log	Every call with arguments, result hash, actor	Unexplainable outcomes

9. Observability and evaluation

Signal	Why it matters
Trajectory (steps, tools, arguments, results)	The only way to explain a wrong outcome
Steps and tokens per task	Leading indicator of loops and cost drift
Tool error rate by tool	Finds bad schemas faster than prompt tweaks
Escalation and abstention rate	Distinguishes "confident" from "correct"
Outcome evals on frozen scenarios	Compares versions on end state, not wording
Replay of failed trajectories	Turns incidents into regression tests

Evaluate on three levels: final answer correctness, trajectory efficiency (did it take the short path?), and safety (did it stay inside its permissions?). A single end-to-end score hides all three.

10. Economics and latency

Pattern	Steps	Relative cost	When it pays
Single call with tools listed	1–2	1×	Simple lookup or classification
ReAct loop	3–10	3–8×	Multi-source research, remediation
Supervisor + 3 workers	6–15	5–12×	Independent sub-tasks in parallel
Plan + verify + repair	4–12	4–10×	Anything that must be right

11. Production checklist

• Each tool has a narrow verb, a validated schema and a hard timeout
• Writes are idempotent and reversible or gated by approval
• Budgets exist for steps, tokens, money and wall-clock, with a clean stop
• Credentials are scoped per tool and per tenant, never the operator's own
• Untrusted content is isolated and cannot trigger actions
• Every run produces a replayable trajectory with tool arguments and results
• Frozen scenario suite runs on every prompt, tool or model change
• Escalation to a human is a designed path, not an exception
• Failure produces a partial result and an explanation, never a silent retry loop
• Cost per successful task is tracked, not just cost per call

12. Sources

• Yao et al., ReAct: Synergizing Reasoning and Acting — the reason/act loop.
• Shinn et al., Reflexion — self-reflection with feedback.
• Park et al., Generative Agents — memory streams and retrieval.
• Anthropic, Building effective agents — workflow versus agent.
• NIST AI Risk Management Framework — governance controls.