LLM Workflows (Level IV)

Three tiers, in ascending order of power and descending order of predictability.

A single call handles classification, extraction, summarization, rewriting — anything where one input maps to one output. Most production LLM usage is this, and should be.

A workflow is multiple calls wired together by your code. You decide what runs next. Prompt chaining (each step's output feeds the next), routing (classify, then dispatch to a specialist prompt), parallelization (fan out, then aggregate), evaluator-optimizer (generate, critique, revise). The control flow is in a language with a debugger.

An agent is a model deciding what runs next, in a loop, with tools. You give it a goal and it plans its own path.

The rule: use the simplest tier that solves the problem. Agents are for tasks that are genuinely hard to specify in advance — "fix this failing test suite", where the steps depend on what you find. If you can draw the flowchart, build the flowchart. It will be cheaper, faster, and it will fail in ways you can reproduce.

Here is the same decision as a table. Match your task shape to the row, and read across.

Two columns there deserve more attention than they usually get.

Cost is not linear in calls. An agent's fifth call carries the transcript of the first four, so a twenty-step run is the sum of twenty growing prefixes, not twenty single calls. This is why the agent row says 10–100× rather than 20×.

The failure column is the real decision. Every tier fails; what differs is whether the failure is reproducible. A single call that gets the answer wrong gets it wrong the same way every time and you can fix the prompt. An agent that took a bad turn on step seven of a run that will never happen identically again is a different class of problem, and if your organisation cannot debug that class of problem, the agent is the wrong shape regardless of how well the demo went.

Ask all four, and take a "no" seriously.

Complexity — is the task multi-step and hard to fully specify up front? If you can enumerate the steps, an agent is overhead.

Value — does the outcome justify the cost and latency? An agent run can be twenty model calls. That's fine for a code migration and absurd for a support-ticket tag.

Viability — is the model actually good at this class of task today? Test on ten real examples before you architect around an assumption.

Cost of error — can mistakes be caught and undone? Tests, review, rollback, a staging environment. An agent acting irreversibly on production data with no gate is not a design, it's an incident with a launch date.

The pattern in the failures: teams answer yes to complexity and never ask the other three.

There is a fifth question nobody asks and everybody should: who reads the transcript when it goes wrong? An agent run produces a long, branching record of decisions. If the answer is "nobody, we'd just rerun it", you have built something you cannot operate. The teams that succeed with agents are the ones that treat the transcript as a first-class product artifact from day one — logged, searchable, and read by a human at least weekly. The rest discover after four months that they have a system whose behaviour nobody can explain.

Every agent is a loop around one API call. Frameworks decorate it; none of them change it. Here is the whole thing, runnable, with the details that separate a working loop from a broken one marked in the comments.

import anthropic

client = anthropic.Anthropic()

MODEL = "claude-opus-5"
MAX_ITERATIONS = 12

TOOLS = [
    {
        "name": "read_file",
        "description": (
            "Read a UTF-8 text file from the repository. Call this whenever you need "
            "the contents of a file — never answer from memory about what a file holds."
        ),
        "input_schema": {
            "type": "object",
            "properties": {
                "path": {"type": "string", "description": "Repo-relative path, e.g. src/parser.py"}
            },
            "required": ["path"],
            "additionalProperties": False,
        },
    },
    {
        "name": "run_tests",
        "description": (
            "Run the test suite and return the output. Call this to confirm a fix, and "
            "before claiming any test passes."
        ),
        "input_schema": {
            "type": "object",
            "properties": {
                "path": {"type": "string", "description": "Optional test file or directory"}
            },
            "required": [],
            "additionalProperties": False,
        },
    },
]


class Truncated(Exception):
    def __init__(self, message="hit max_tokens mid-answer", partial=""):
        super().__init__(message)
        self.partial = partial          # whatever text arrived before the cut

class Refused(Exception): pass
class ContextExhausted(Exception): pass # the input no longer fits, not the output
class Unknown(Exception): pass          # a stop_reason this code has never seen
class CapReached(Exception): pass


def execute(name, args):
    """Run one tool. Returns (text, is_error). Never raises — the model recovers, not you."""
    try:
        if name == "read_file":
            return read_file(args["path"]), False
        if name == "run_tests":
            return run_tests(args.get("path")), False
        return f"FAILED: no tool named {name!r}. Available: read_file, run_tests.", True
    except FileNotFoundError:
        return f"FAILED: {args.get('path')!r} does not exist. Check the path and retry.", True
    except Exception as exc:
        return f"FAILED: {name} raised {type(exc).__name__}: {exc}", True


def final_text(response):
    return "".join(b.text for b in response.content if b.type == "text")


def agent(task: str, system: str) -> str:
    messages = [{"role": "user", "content": task}]

    for turn in range(MAX_ITERATIONS):
        response = client.messages.create(
            model=MODEL,
            # Thinking is ON BY DEFAULT on Claude Opus 5 when `thinking` is omitted,
            # and max_tokens caps thinking PLUS response text together. Size this for
            # both or you get the silent truncation the next lesson is about.
            max_tokens=32_000,
            system=system,
            tools=TOOLS,
            messages=messages,
            output_config={"effort": "medium"},
        )

        # (1) Append the assistant turn VERBATIM — every block, in order.
        #     The tool_use blocks must survive or the results you send reference nothing.
        messages.append({"role": "assistant", "content": response.content})

        # (2) Branch on why it stopped. Exactly one of these means "keep going".
        if response.stop_reason == "end_turn":
            return final_text(response)
        if response.stop_reason == "max_tokens":
            raise Truncated(f"turn {turn}: hit the output cap mid-answer; result is partial",
                            partial=final_text(response))
        if response.stop_reason == "model_context_window_exceeded":
            raise ContextExhausted(f"turn {turn}: history no longer fits; compact and retry")
        if response.stop_reason == "refusal":
            raise Refused(getattr(response, "stop_details", None))
        if response.stop_reason == "pause_turn":
            continue          # server-side tool paused; resend history to resume
        if response.stop_reason == "stop_sequence":
            return final_text(response)
        if response.stop_reason != "tool_use":
            raise Unknown(response.stop_reason)   # new values get added; never assume success

        # (3) Execute every requested tool. One result per tool_use block, always —
        #     including failures, marked with is_error.
        results = []
        for block in response.content:
            if block.type != "tool_use":
                continue
            text, is_error = execute(block.name, block.input)
            results.append({
                "type": "tool_result",
                "tool_use_id": block.id,
                "content": text[:20_000],   # bound it; see the tool-results lesson
                "is_error": is_error,
            })

        # (4) All results go back in ONE user message, together.
        messages.append({"role": "user", "content": results})

    raise CapReached(f"no answer after {MAX_ITERATIONS} turns")

Four details, numbered to match the comments.

(1) Append the full assistant turn, not just the text. response.content is a list of blocks — text, thinking, tool_use — and the API expects it back exactly as it came. Extract .text and append a string and the tool-call records vanish; the results you send next reference tool_use ids that are no longer in the history, and the request is rejected. This is the single most common bug in hand-written loops.

(2) Branch on every stop reason. The next lesson is entirely about this. Note that the happy path is last in the list, not first — write it this way and it becomes impossible to accidentally treat a truncated response as an answer.

(3) Return every result, including failures. A tool that threw still gets a result, marked with is_error: true, carrying the message. Dropping it leaves the model waiting for an answer that never comes. Fabricating a success sends it down a path built on a result that does not exist. Note that execute never raises: an exception escaping into the loop kills a run that the model could have recovered from.

(4) Return parallel results together. If the model requested three tools in one turn, all three results go in one user message. Splitting them across messages teaches the model to stop requesting things in parallel, and you lose the concurrency for the rest of the run.

And the thing that is not numbered because it wraps everything: for turn in range(MAX_ITERATIONS). Not because the model is reckless, but because a tool that always errors and a model that always retries is an infinite loop that bills by the token. Twelve is a reasonable default for a scoped task; pick a number, and make hitting it a loud failure rather than a silent return.

One more thing about max_tokens, because getting it wrong causes the bug the next lesson is about. On Claude Opus 5, thinking is on by default — omitting the thinking parameter runs adaptive thinking, unlike Opus 4.8 and 4.7 where omitting it meant no thinking at all. And max_tokens is a single cap over thinking plus the response text, not two separate budgets. So a loop that sets max_tokens=8192 because "the answers are short" is sizing for the answer and forgetting the reasoning, and it will truncate mid-thought on exactly the hard turns where the reasoning was worth paying for. Size it for both — 32K is a sane floor for an agent turn — and if you genuinely need a tighter budget, turn the effort dial down rather than starving the cap.