What Does an AI Solution Architect Do? The Role Explained

The idea in one minute#

An AI solution architect turns a business problem into a design that can be built, paid for and trusted. The job sits between three groups that do not speak the same language: business people, who know the problem and hold the budget; engineers, who will build it; and risk owners — security, legal, compliance — who can stop it.

The architect does not usually write the production code and does not train models. The architect decides what to build, with which parts, in what order, and whether to build it at all, and writes those decisions down so that all three groups can check them.

The output of the job is not a diagram. It is a decision that survives contact with reality: it gets funded, it gets built, it passes review, and it is still running a year later.

A picture#

flowchart TB
  subgraph BIZ["Business"]
    direction TB
    B1[":i-building-2: <b>Sponsor</b><br/><small>owns budget and outcome</small>"]
    B2[":i-users: <b>Users and process owners</b><br/><small>know how the work is done</small>"]
  end
  subgraph ENG["Engineering"]
    direction TB
    E1[":i-code: <b>AI and software engineers</b><br/><small>build and run it</small>"]
    E2[":i-database: <b>Data and platform teams</b><br/><small>own data, cloud, pipelines</small>"]
  end
  subgraph RISK["Risk owners"]
    direction TB
    R1[":i-shield-check: <b>Security</b>"]
    R2[":i-scale: <b>Legal, privacy, compliance</b>"]
    R3[":i-coins: <b>Finance and procurement</b>"]
  end
  SA[":i-layers: <b>AI solution architect</b><br/><small>one design, three audiences</small>"]
  BIZ -->|"problem, value, constraints"| SA
  SA -->|"design, decisions, plan"| ENG
  SA -->|"risks, controls, evidence"| RISK
  RISK -.->|"conditions"| SA
  ENG -.->|"what is feasible"| SA
  SA -->|"cost, benefit, options"| BIZ
  class B1,B2 neutral
  class E1,E2 compute
  class R1,R2,R3 warn
  class SA queue

How it really works#

The five things you are paid to do#

#	Responsibility	What it looks like on a Tuesday
1	Find the real problem	A workshop where “we need a chatbot” turns into “claims handlers spend 40% of their day reading documents”
2	Decide if AI fits	A one-page assessment that says yes, no, or “yes, but not with a model — a rule engine does this”
3	Choose the shape of the solution	Prompt, retrieval, workflow, agent, fine-tune; which model; which platform; what the organisation already owns
4	Make it safe to say yes to	Cost per task, three-year cost, expected benefit, risks with controls, a plan with gates
5	Stay until it works	Design reviews during the build, the evaluation gate before launch, the post-launch review

Most people new to the role over-invest in number 3 and under-invest in 1 and 4. Number 3 is the part that feels like architecture. Numbers 1 and 4 are where projects are won or lost.

Why this role exists at all#

Enterprise AI has a well-documented delivery problem, and it is not a model problem.

McKinsey’s 2025 global survey reported that 88% of organisations use AI in at least one business function, while roughly two-thirds had not begun scaling it and only 39% reported any effect on enterprise-level profit.
A July 2025 MIT Project NANDA study — preliminary, built on about 300 public deployments, 52 organisation interviews and 153 survey responses — reported that 95% of the organisations it studied saw no measurable profit-and-loss return from generative AI pilots. Its stated causes were brittle workflows, tools that did not learn from feedback, and poor fit with how the work was really done — not weak models.
Gartner predicted in June 2025 that more than 40% of agentic AI projects would be cancelled by the end of 2027, naming rising cost, unclear business value and inadequate risk controls.

Read those three causes again: cost, value, risk controls, workflow fit. None of them is fixed by a better prompt. All of them are architecture decisions made — or not made — before the build starts. That gap is the job.

note

These figures are widely quoted and often misquoted. The 95% number measures profit impact in one preliminary sample; it is not “95% of AI projects fail”. Use such numbers to explain why discipline matters, not as proof.

The roles next door#

Role	Main question	Main output	How you work with them
AI / ML engineer	How do I build and tune it?	Working code, prompts, evals	You set the boundaries; they find what works inside them
Data scientist	What does the data say, and can we predict it?	Models, analyses	They tell you whether the data supports the idea
Data engineer	How does data get there, clean and on time?	Pipelines, tables	Most AI designs fail on their part, so involve them first
Cloud / platform architect	How does it run in our cloud, safely?	Landing zones, networks, identity	Your design must fit inside theirs
Enterprise architect	How does it fit the whole company?	Standards, roadmaps	They tell you what already exists and what is forbidden
Product manager	What should it do for the user?	Requirements, priorities	You share ownership of “what” and split “why” and “how”
Forward-deployed / solutions engineer	How do I make it work for this customer?	Prototypes on site	Often the same person in a smaller company

Three places the role lives#

The title is the same; the day is not.

Where	Who you serve	You are measured on	Typical trap
Vendor (pre-sales) — a cloud or model provider	Many customers, briefly	Deals won, customers reaching production	Designing for the product you sell rather than the problem
Consultancy or integrator	A few customers, deeply	Projects delivered, follow-on work	Designs that need you forever
In-house — inside the enterprise	One company, for years	Systems in production, cost, incidents	Becoming the committee that says no

What you hand over#

An architect’s work is judged by documents other people can act on. You will write all of these in this course:

Artefact	Reader	Lesson
Discovery notes and a problem statement	Sponsor	Discovery
Use-case scorecard	Steering group	Qualifying and Prioritising Use Cases
Requirements with measurable targets	Engineers, testers	Requirements and Success Metrics
Cost model and business case	Finance	Cost, TCO and ROI
Solution design document and decision records	Everyone	The Solution Design Document
Evaluation plan and release gate	Engineers, risk owners	Evaluation as the Contract
Risk register and control map	Security, legal	Governance and Regulation

What good looks like#

You can explain the design in two minutes to the sponsor and in twenty to the engineers, and both versions are true.
Every choice has a written reason and a named alternative you rejected.
You know the cost of one task, and what happens to it at ten times the volume.
You said what would make you change your mind — before anyone asked.
You have a number that says whether it works, and a plan for the day it stops working.

Remember this#

The job is deciding what to build and making it safe to approve; building is someone else’s.
Three audiences: business, engineering, risk owners. One design must satisfy all three.
Projects fail on cost, value, risk controls and workflow fit — all decided before the build.
“No, and here is why” is a deliverable.

Try it#

Pick an AI feature in a product you use. Write one sentence each for: the business problem it solves, who paid for it, and who could have stopped it.
Find three job postings titled “AI Solutions Architect”. Sort every bullet into the five responsibilities above. Which responsibility has the fewest bullets? That is the one employers find hardest to describe and most valuable to find.
Write down which of the three places — vendor, consultancy, in-house — fits you, and why.

Check yourself#

Why is a diagram not the output of this job?
Name the three audiences and what each needs from the architect.
The MIT study blamed failed pilots on three things. What were they, and why are they architecture problems rather than model problems?

Sources#

McKinsey: The State of AI in 2025 — Agents, Innovation, and Transformation
MIT Project NANDA: The GenAI Divide — State of AI in Business 2025 (press coverage; the report itself is labelled preliminary)
Gartner: Over 40% of agentic AI projects will be canceled by end of 2027