The Architecture Interview

The idea in one minute#

An interview for an AI solution architect tests one thing in several costumes: can this person be put in front of a customer or an executive with a vague problem, and come back with a sound, defensible plan? Knowing the technology is assumed. What is examined is how you think when the problem is underspecified — whether you ask before you answer, whether you reach for the simple design first, whether you can put numbers on a trade-off, and whether you can say “I would not use AI for that”.

The centre of the process is a design or case exercise: forty-five minutes on a prompt such as “design a support assistant for a bank”. Candidates fail it in a predictable way — they start drawing in the first minute. The method in this lesson is the one this whole course has taught, compressed into the time available: clarify, numbers, simplest design, deepen where it matters, risks, cost, plan.

A picture#

flowchart LR
  A[":i-message-square: <b>Clarify</b><br/><small>5 min · who, what, how often, how wrong</small>"] --> B[":i-gauge: <b>Numbers</b><br/><small>3 min · volume, tokens, cost</small>"]
  B --> C[":i-layers: <b>Simplest design</b><br/><small>7 min · lowest rung that fits</small>"]
  C --> D[":i-search: <b>Deepen</b><br/><small>12 min · the two hard parts</small>"]
  D --> E[":i-shield-alert: <b>Risks and controls</b><br/><small>6 min</small>"]
  E --> F[":i-flask-conical: <b>Evaluation and rollout</b><br/><small>6 min</small>"]
  F --> G[":i-check: <b>Summarise and trade-offs</b><br/><small>3 min</small>"]
  class A,B queue
  class C,D compute
  class E warn
  class F memory
  class G io

How it really works#

The usual rounds#

Vendor pre-sales roles weight the role-play and presentation. In-house roles weight the design exercise and behavioural rounds. Consultancies add a business case.

The seven steps, in a 45-minute design round#

1. Clarify (5 minutes). Do not skip this to look fast; skipping it is the commonest reason for a fail. Ask, and write the answers where the interviewer can see them.

• Who uses it, and what are they trying to get done?
• What happens today, and what is wrong with it?
• How many users, how many requests a day?
• What data would it need, and where is it?
• What does a wrong answer cost? Is this a regulated activity?
• What must it integrate with? Any platform already chosen?
• What does success look like in six months?

If the interviewer says “you decide”, state an assumption out loud and move on. That is the right answer.

2. Numbers (3 minutes). Turn the answers into three figures: requests per day and at peak, tokens per request, and a rough monthly model cost. Rough is fine; absent is not.

200,000 customers · 5% contact per month · 3 questions each ≈ 30,000 questions/month
≈ 1,400 per working day, peak perhaps 6 per minute
~5,000 tokens in, ~300 out, mid-tier model ≈ $0.013 each ≈ $400 a month
→ model cost is negligible; the cost is people and integration; a small model may do

3. The simplest design (7 minutes). Name the lowest rung of the ladder that fits and say why nothing higher is needed yet. Draw five to seven boxes: user, application, gateway, model, knowledge source, data stores, observability. Trace one request through them.

4. Deepen where it matters (12 minutes). Pick the two parts that are actually hard for this problem and go deep. For a bank assistant: permission-aware retrieval, and what happens when it does not know. Say which parts you are deliberately not detailing.

5. Risks and controls (6 minutes). Name the three risks specific to this system. For each: the control, and whether it is deterministic or depends on the model. Mention the legal classification if people are affected.

6. Evaluation and rollout (6 minutes). How would you know it works? What is in the test set, what is the pass mark, what are the stages and gates? This is where strong candidates separate themselves — most never mention it.

7. Summarise and trade-offs (3 minutes). Restate the design in three sentences. Name the two biggest trade-offs you made and what would make you revisit them. Then ask what the interviewer would like explored further.

What interviewers score#

Practice prompts#

Work each one aloud in forty-five minutes, then compare with the lesson named.

Technical questions to be ready for#

You should be able to answer each of these in two minutes, plainly.

• When would you use retrieval, and when fine-tuning?
• What is the difference between a workflow and an agent, and how do you choose?
• How do you evaluate a system whose output is free text?
• Why do RAG systems give wrong answers, and how do you diagnose which stage failed?
• What is prompt injection, and why can it not simply be filtered out?
• How do you make sure an assistant only shows a user what they are allowed to see?
• How do you estimate the cost of an AI feature before it is built?
• What happens to your design when the model provider has an outage?
• How do you choose between two models?
• Hosted API or self-hosted model — how do you decide?
• What does the EU AI Act require of a system that helps make hiring decisions?
• How would you take a proof of concept to production?

The customer role-play#

A common format: the interviewer plays a customer executive. They will do three things on purpose.

Ask for a solution immediately. “We want an agent that handles all our support.” Respond with curiosity, not a design: “Happy to get there. Can I ask what is driving this now, and what a good result would look like for you?”

Raise an objection. “Our data cannot leave the building”, “the last AI project failed”, “your competitor says they can do it in a month”. Acknowledge it, ask what is behind it, and answer the real concern — never argue with the surface.

Push for a promise you cannot keep. “Can you guarantee 100% accuracy?” The right answer is honest and specific: “No system can, including the people doing it today. What we can do is measure it on your own cases before you rely on it, and design what happens when it is unsure.”

You pass this round by leaving the customer feeling understood and clearer about their own problem — not by closing.

Behavioural stories to prepare#

Have one true story for each, with a number in the result.

1. A time you talked a stakeholder out of the solution they asked for.
2. A design decision you got wrong, and what you changed.
3. A project that stalled between pilot and production, and why.
4. A disagreement with security, legal or finance, and how it was resolved.
5. A time you had to explain something technical to a non-technical decision-maker.
6. A trade-off you made between cost and quality.

Tell them as situation, what I did, what happened, what I would do differently. The last part is the one that shows seniority.

Questions to ask them#

Your questions are scored too.

• How many AI systems are in production here, and what stopped the ones that are not?
• Who owns a system after launch?
• How are model and platform choices made, and who can change them?
• How does security review work for AI projects — when does it start?
• What does a successful first six months in this role look like?

The answers also tell you whether the job is architecture or slideware.

Remember this#

• The interview tests whether you can turn a vague problem into a defensible plan.
• Never draw in the first five minutes. Clarify, then put numbers down.
• Propose the simplest design, then go deep on the two parts that are hard for this problem.
• Raise risks, evaluation and rollout without being asked. Most candidates do not.
• In a role-play, the goal is a customer who feels understood, not a closed deal.

Try it#

1. Work the first practice prompt aloud with a timer and a blank page. Record yourself. How long before you drew a box?
2. Write two-minute answers to five of the technical questions and read them to someone who is not an engineer.
3. Prepare the six behavioural stories. Check each has a number and a “what I would do differently”.

Check yourself#

1. What is the most common way to fail a design round?
2. What three figures should you produce in the numbers step?
3. A customer asks you to guarantee 100% accuracy. What do you say?
4. Which part of the seven-step method do most candidates leave out?