The Pattern Ladder

The idea in one minute#

There are five ways to make a model useful, and they form a ladder. Each rung adds capability and adds cost, latency and things that can break.

1. A prompt — instructions and examples, one call.
2. Retrieval — fetch the relevant facts first, then one call.
3. A workflow — several calls in an order you fixed.
4. An agent — the model chooses its own steps and tools in a loop.
5. Fine-tuning — change the model itself with your examples.

The rule is: start on the lowest rung, measure, and climb only when the measurement says you must. Most failed AI projects started two rungs too high. An architect’s most valuable habit is asking “what is the simplest thing that could pass the evaluation?” before anything is drawn.

Fine-tuning is drawn as the top rung because it is the last thing to try, not because it is the most powerful. It solves a different problem from the other four, and it combines with any of them.

A picture#

flowchart TB
  R1[":i-message-square: <b>1. Prompt</b><br/><small>instructions + examples, one call</small>"]
  R2[":i-search: <b>2. Retrieval</b><br/><small>fetch facts, then one call</small>"]
  R3[":i-workflow: <b>3. Workflow</b><br/><small>several calls, your order</small>"]
  R4[":i-bot: <b>4. Agent</b><br/><small>model picks steps and tools</small>"]
  R5[":i-wrench: <b>5. Fine-tune</b><br/><small>change the model's behaviour</small>"]
  R1 -->|"it lacks our facts"| R2
  R2 -->|"one call cannot do the whole task"| R3
  R3 -->|"steps cannot be known in advance"| R4
  R1 -.->|"style, format or cost still wrong after all else"| R5
  E[":i-flask-conical: <b>Evaluation</b><br/><small>the only reason to climb</small>"]
  E -.-> R1
  E -.-> R2
  E -.-> R3
  E -.-> R4
  class R1,R2 neutral
  class R3 queue
  class R4 compute
  class R5 warn
  class E memory

How it really works#

The rungs side by side#

Rung 1 — a prompt, properly done#

Before deciding a prompt is not enough, make sure it was a good one. A serious prompt has:

• A role and a goal — who the model is acting as and what a good result is.
• Context — what it needs to know that it could not guess.
• Examples — three to five input-and-output pairs. Often worth more than a page of rules.
• An output contract — a schema or template. Use the provider’s structured output feature so the reply is valid JSON every time.
• Room to think — for hard tasks, allow reasoning before the final answer.

A surprising share of “we need an agent” requests are solved here. Classification, extraction from a document, summarising, drafting, translation and rewriting are all rung 1.

Rung 2 — retrieval#

Use it when the answer depends on facts the model does not have: your policies, your product catalogue, yesterday’s tickets. The model is not retrained; the facts are fetched and placed in the context window at question time. That is retrieval-augmented generation (RAG), and because it is the pattern you will design most often it has its own lesson.

Rung 3 — a workflow#

A workflow is several model calls arranged by ordinary code. You decide the order; the model fills in each step. Five shapes cover nearly everything:

Workflows are predictable: you know how many calls happen and can test each step alone. Prefer them whenever the steps can be written down.

Rung 4 — an agent#

An agent is a model in a loop that decides its own next action — which tool to call, when to stop. Use one when you genuinely cannot list the steps in advance: debugging code, open-ended research, handling a request that could go ten ways.

The price is real. Cost and duration become distributions rather than numbers; errors compound over many steps; and anything with tools can take an action you did not intend. Workflows and Agents gives the test for when that price is worth paying.

Rung 5 — fine-tuning#

Fine-tuning continues a model’s training on your examples. It changes how the model behaves, not what it knows today. That distinction settles most arguments:

Fine-tuning needs hundreds to thousands of good examples, an evaluation set to prove it helped, and a plan to redo it whenever the base model is retired. Reach for it last, and only with a measured gap that the other rungs could not close.

How to decide: four questions#

Ask these in order. The first “no” tells you the rung.

1. Can the model do it from the prompt alone? If the task needs no private or recent facts → prompt.
2. Is one call enough once the facts are supplied? → retrieval.
3. Can you write the steps down in advance? → workflow.
4. Is the task valuable enough to accept variable cost and to supervise actions? → agent. If not, narrow the task until the answer to question 3 is yes.

Then, separately: after the best of those, is there still a measured gap in style, format, latency or unit cost? Only then consider fine-tuning.

What climbing costs#

Reliability multiplies. If each step of a process is right 95% of the time, two steps in a row are right about 90% of the time and ten steps about 60%. Every rung adds steps. A design that looks more sophisticated can be measurably worse than the simple one it replaced — which is why the ladder is climbed with an evaluation in hand, not on instinct.

Code#

The four questions as a program. It takes a task description as flags an architect can answer in a discovery meeting and returns the lowest rung that fits, with the reason.

// ladder.go — pick the lowest rung that fits a task.
package main

import "fmt"

type task struct {
	name            string
	needsOurFacts   bool // depends on private or recent information
	oneCallEnough   bool // a single call can finish it once facts are supplied
	stepsKnown      bool // the steps can be written down in advance
	takesActions    bool // changes something in another system
	valuePerTaskUSD float64
	measuredGap     string // gap left after the best non-tuned design, "" if none
}

func rung(t task) (string, string) {
	switch {
	case !t.needsOurFacts && t.oneCallEnough:
		return "1 prompt", "the model can do it from instructions and examples"
	case t.oneCallEnough:
		return "2 retrieval", "it needs our facts, then one call is enough"
	case t.stepsKnown:
		return "3 workflow", "several steps, but we can write them down"
	case t.valuePerTaskUSD < 1 && t.takesActions:
		return "3 workflow (narrow the task)", "too little value to supervise an agent that acts"
	default:
		return "4 agent", "steps depend on what it finds along the way"
	}
}

func main() {
	tasks := []task{
		{"tag incoming emails by topic", false, true, true, false, 0.05, ""},
		{"answer HR policy questions", true, true, true, false, 2, ""},
		{"turn a claim form into a decision letter", true, false, true, false, 15, "tone drifts from house style"},
		{"investigate a failed payment and fix it", true, false, false, true, 40, ""},
		{"reorder stock when a customer complains", true, false, false, true, 0.4, ""},
	}
	for _, t := range tasks {
		r, why := rung(t)
		fmt.Printf("%-42s → %-28s %s\n", t.name, r, why)
		if t.measuredGap != "" {
			fmt.Printf("%-42s   then consider fine-tuning: %s\n", "", t.measuredGap)
		}
	}

	fmt.Println("\nwhy climbing costs: chance that every step is right")
	for _, steps := range []int{1, 2, 5, 10, 20} {
		p := 1.0
		for i := 0; i < steps; i++ {
			p *= 0.95
		}
		fmt.Printf("  %2d steps at 95%% each → %.0f%%\n", steps, p*100)
	}
}

The last table is the argument to bring to any meeting where someone proposes ten cooperating agents for a task a single prompt could do.

Remember this#

• Five patterns: prompt, retrieval, workflow, agent, fine-tune. Start at the bottom.
• Climb only when an evaluation shows the lower rung is not enough.
• Retrieval changes what the model knows; fine-tuning changes how it behaves.
• If you can write the steps down, it is a workflow, not an agent.
• Every extra step multiplies the chance of an error.

Try it#

1. Run ladder.go. Add three tasks from your own work and check whether you agree with the rung it picks. Where you disagree, which flag was too coarse?
2. Take an “AI agent” product announcement and decide which rung it really is.
3. Write the best prompt you can for a task at your work, with five examples and an output schema. Try it on twenty real inputs before deciding you need anything more.

Check yourself#

1. Why is fine-tuning the wrong tool for “the model should know our latest prices”?
2. What is the difference between a workflow and an agent?
3. A ten-step agent has steps that are each 95% reliable. Roughly how often is the whole run right, and what does that say about adding steps?
4. Name the five things a serious prompt contains.

Sources#

• Anthropic: Building effective agents (19 December 2024) — the workflow shapes and the “simplest solution” rule