AI and LLM Foundations (Level II)

One sentence

A language model reads what it has so far, guesses the next word, sticks it on the end, and does the whole thing again. That is the entire mechanism.

     "The invoice was"
            |
            v
     +-------------+
     |    model    |   ranks every word that could come next
     +-------------+
            |
            v
     approved (61%)   sent (12%)   paid (9%)   ...
            |
            v   pick one, add it to the end
     "The invoice was approved"
            |
            +------> go round again, until it picks "stop"

Two lazy conclusions, both wrong

• "It's just autocomplete." Ignores that guessing the next word in a contract, a proof or a bug report well enough to be useful means modelling the thing being described.
• "It understands." Ignores that this mechanism has no goals, no memory between calls, and no access to truth — only to the statistical shape of an enormous amount of text.

The honest version: it is a very good simulator of the kind of text a knowledgeable person would write. Genuinely useful, genuinely limited. Almost every surprise you will have with these tools comes from forgetting one half of that sentence.

Three consequences that explain almost everything else

• No memory. Each call starts from nothing but the text you send. Any appearance of continuity is the application resending the history.
• No source. It is not looking anything up unless you gave it a way to. Facts come from the shape of its training text — right often enough to be dangerous.
• No sense of when it is guessing. It cannot write a sentence and simultaneously judge that the sentence is unsupported, because writing plausible sentences is the mechanism. Confidence is a writing style, not a signal.

Hold those three and most of the industry's confusing behaviour becomes predictable.

A token is roughly a word

Close enough for every decision you will make: 750 words ≈ 1,000 tokens. The model chops text into these pieces, and they are the unit of the bill.

You pay for tokens going in and coming out, at different rates:

   what you send                        what you get back
   (prompt, history, documents)         (the answer)
            |                                  ^
            |    INPUT tokens                  |   OUTPUT tokens
            |    cheaper                       |   ~5x the price
            v                                  |
        +----------------------------------------+
        |                model                   |
        +----------------------------------------+

That is the whole pricing model. Everything in the next section is arithmetic on those two numbers.

Not all text costs the same

The same fact, written three ways:

Prose:  The invoice was approved on Tuesday.

JSON:   {"document": "invoice", "status": "approved", "approved_on": "Tuesday"}

Code:   if invoice.status == "approved" and invoice.approved_on == "Tuesday":
            notify(invoice.owner)

Seven tokens for a six-word sentence: the full stop is its own token. That is the "roughly" in "roughly a word."

Braces, quotes and colons all cost tokens and carry no meaning a human would call content. So "just have it return JSON" is never free — the same information can cost three times as much on the expensive side of the bill.

Two footnotes, then move on

• Counts vary slightly between models, so a cost estimate copied from last year's model is not an estimate. Every provider has a token-counting endpoint; use it before you commit to a budget.
• The model sees chunks, not letters — which is why "how many R's in strawberry" was ever hard, and why it is a useless test of anything.

What an agent actually is

A model in a loop with tools. It acts, sees the result, decides what to do next, and repeats until the task is done or it gives up. That is it — a real and useful pattern, and the single most over-claimed word in this industry.

   goal
    |
    v
  +-------+   "read the log file"     +--------+
  | model | ------------------------> |  tool  |
  |       | <------------------------ |        |
  +-------+   result, as text         +--------+
    |   ^
    |   |__ round again -- each pass re-sends the whole history so far
    v
  answer        stops when: model says done | step limit | you cancel

Four questions, in order

• What tools can it actually call? (Often: one.)
• What stops the loop? No iteration limit and no success criterion is a way to spend money.
• What happens when a step fails? Retry, escalate, or confidently proceed on bad data?
• Who approves the irreversible actions?

A scripted sequence of three model calls is a workflow. That is frequently the better design — more predictable, cheaper, easier to debug — but it is not an agent, and a team that blurs the two usually has not thought about failure modes.

The cost profile is the part that surprises people

A single call is one request. An agent run is however many requests the model decides it needs, each carrying the entire history of the run so far — the growing-prefix problem from the last section, but with the model in charge of how many turns there are.

A twenty-step run is not twenty times a single call. It is the sum of twenty growing prefixes, and it can be fifty to a hundred times the cost of the same task done as one well-specified request.

Fine when the task genuinely cannot be specified in advance — "fix this failing test suite". Absurd for something you could have drawn a flowchart for. So the question is not "is it agentic?" but "what makes this task impossible to script?" If nobody can answer that, you are looking at an expensive way to run a workflow.