The Cloud and Software Ground Floor

The idea in one minute#

An AI feature is a small new part inside an ordinary software system. The model call is one box. Around it sit the same boxes every system has had for twenty years: something the user touches, an application that does the work, places where data is kept, a way to prove who you are, a network that decides who may talk to whom, and a way to see what happened.

Most AI projects that stall do not stall on the model. They stall on these boxes: the data cannot be reached, security will not open the network, nobody knows whose identity the assistant acts under. An architect who can name the boxes and ask the right question about each one removes months of delay.

This lesson is the ground floor. If you already design cloud systems, read the last two sections and move on.

A picture#

flowchart LR
  U[":i-smartphone: <b>Client</b><br/><small>browser, app, chat tool</small>"] --> EDGE[":cloudflare: <b>Edge</b><br/><small>TLS, firewall, rate limits</small>"]
  EDGE --> APP[":i-server: <b>Application</b><br/><small>your business logic</small>"]
  IDP[":okta: <b>Identity provider</b><br/><small>who is this, what may they do</small>"] -.-> EDGE
  IDP -.-> APP
  APP --> DB[(":postgresql: <b>Database</b><br/><small>structured records</small>")]
  APP --> OBJ[(":s3: <b>Object storage</b><br/><small>files, documents</small>")]
  APP --> Q[":apachekafka: <b>Queue</b><br/><small>work to do later</small>"]
  APP --> AI[":i-brain: <b>Model API</b><br/><small>the new box</small>"]
  APP --> SAAS[":i-globe: <b>Other systems</b><br/><small>CRM, ERP, ticketing</small>"]
  APP -.-> OBS[":opentelemetry: <b>Logs, metrics, traces</b>"]
  class U neutral
  class EDGE,Q queue
  class APP,AI compute
  class IDP warn
  class DB,OBJ memory
  class SAAS io
  class OBS neutral

How it really works#

The boxes#

Cloud in five nouns#

A cloud provider — Amazon Web Services, Microsoft Azure, Google Cloud — rents computing by the minute. Whatever the brand, you are renting five things.

Three more words locate everything. A region is a geographic area where the provider has data centres — it decides latency and which country’s law applies to the data. An account, subscription or project is a billing and isolation boundary. A landing zone is the pre-built, pre-approved set of accounts, networks and policies an enterprise makes every team deploy into; your design must fit inside it.

How much you run yourself#

The same capability can be bought at four levels. Higher is faster to start and less flexible.

This ladder returns in Build, Buy or Partner. The rule of thumb now: start as high as the requirements allow and move down only when a written requirement forces you.

Identity, the part beginners skip#

Two verbs matter. Authentication answers “who is this?”. Authorisation answers “what may they do?”. In enterprises both are handled by a central identity provider with single sign-on, and people get permissions through roles and groups.

AI adds one hard question to this old subject. When an assistant looks something up for a user, it must see only what that user is allowed to see. A search index built from every document in the company, queried by an assistant that ignores permissions, is the classic enterprise AI incident: the intern asks about salaries and gets an answer. The fix is a design decision — carry the user’s identity through to the data source and filter at retrieval — and it has to be made at the start. See Security, Privacy and Residency.

Three ways to reach a model#

Enterprises overwhelmingly prefer the second route, for reasons that have little to do with technology: procurement, data-processing agreements and security approval already exist for the cloud provider. The Cloud AI Platforms maps what each offers.

Reading an architecture diagram#

When someone puts a diagram in front of you, ask these in order. They work on any system.

1. Where does a request start and end? Trace one with your finger.
2. Where does data rest? Every cylinder is a place data can leak from and must be backed up.
3. Where are the trust boundaries? Which lines cross from the internet into the company, or from one team’s system to another’s?
4. What is synchronous? Which arrows make the user wait?
5. What fails alone? If this box is down, what still works?
6. Who owns each box? An unowned box is tomorrow’s outage.

Five numbers worth memorising#

These are orders of magnitude, not promises. Measure before you commit a number to a requirement.

Remember this#

• An AI feature is one new box in an ordinary system. The old boxes cause most delays.
• Cloud is five nouns: compute, storage, network, identity, managed services.
• Start as high on the ladder as the requirements allow; move down only for a written reason.
• The assistant must see only what its user may see. Decide how on day one.
• Six questions read any diagram: start and end, data at rest, trust boundaries, what waits, what fails alone, who owns it.

Try it#

1. Draw the boxes for an online shop you use. Mark where an AI feature such as “summarise the reviews” would attach, and which existing boxes it needs to read from.
2. Open the free tier of one cloud provider and find, without creating anything: the list of regions, the identity service, and the AI model catalogue. Write down their names.
3. Take any architecture diagram from a cloud provider’s reference library and answer the six reading questions for it.

Check yourself#

1. What is the difference between authentication and authorisation?
2. Why do enterprises usually reach models through their cloud provider rather than directly?
3. An assistant answers a question using a document the user is not allowed to open. Which box was designed wrongly, and when should it have been decided?
4. Why does a 500-token answer take seconds no matter how fast the network is?