The AI Platform and Operating Model

The idea in one minute#

When an organisation has three AI projects, each team builds its own everything: its own way to reach models, its own retrieval, its own logging, its own argument with security. By the tenth project that is ten sets of keys, ten bills nobody can explain and ten security reviews of the same design.

An AI platform is the set of shared services that every team uses instead: model access, retrieval, tool connections, guardrails, evaluation, observability and cost reporting. An operating model is the human side: who runs the platform, who builds on it, who sets the rules and who pays.

The aim is a paved road — a route so much easier and faster than going alone that teams choose it, with security, cost control and governance built into the surface. The test of a platform is not how much it does. It is how long it takes a new team to get from an approved idea to a safely running system: days, not quarters.

A picture#

flowchart TB
  subgraph TEAMS["Product teams — own use cases and outcomes"]
    direction LR
    T1[":i-users: Claims"]
    T2[":i-users: Customer service"]
    T3[":i-users: Engineering"]
    T4[":i-users: Finance"]
  end
  subgraph PLAT["AI platform — owned by a platform team"]
    direction LR
    GW[":envoyproxy: <b>Model gateway</b><br/><small>keys, limits, routing, cost tags</small>"]
    KN[":i-search: <b>Knowledge services</b><br/><small>ingestion, permission-aware search</small>"]
    TL[":modelcontextprotocol: <b>Tool registry</b><br/><small>approved MCP servers</small>"]
    GR[":i-shield-check: <b>Guardrails</b>"]
    EV[":i-flask-conical: <b>Evaluation service</b>"]
    OB[":opentelemetry: <b>Tracing and cost</b>"]
    RT[":i-bot: <b>Agent runtime</b><br/><small>identity, sandboxes, approvals</small>"]
  end
  subgraph GOV["Governance — sets the rules"]
    direction LR
    G1[":i-scale: AI governance board"]
    G2[":i-list-checks: Inventory and risk tiers"]
    G3[":i-book-open: Standards and reference designs"]
  end
  TEAMS --> PLAT
  GOV -.->|"policies as defaults"| PLAT
  GOV -.->|"review by tier"| TEAMS
  PLAT --> M[":i-brain: <b>Models</b><br/><small>hosted, provisioned, self-hosted</small>"]
  PLAT --> D[(":i-database: <b>Enterprise data and systems</b>")]
  class T1,T2,T3,T4 neutral
  class GW,RT queue
  class KN,TL,EV,OB compute
  class GR warn
  class G1,G2,G3 warn
  class M io
  class D memory

How it really works#

What belongs in the platform#

Put a capability in the platform when every team needs it, when doing it badly creates risk, or when only a central view makes sense of it. Leave everything else to the teams.

What stays with the product team: the use case, the prompts, the evaluation cases, the user experience and the outcome. The platform provides the road; it does not drive.

Build the platform in the order it is needed#

A platform built ahead of demand is the most expensive kind of shelfware. Grow it from real projects.

The first two projects from Qualifying and Prioritising Use Cases should be chosen partly for what they force the platform to build.

Operating models#

Three structures, usually reached in this order.

The body at the hub is often called an AI centre of excellence. A good one does four things: runs the platform, publishes standards and reference designs, reviews high-risk systems, and spreads what works. A bad one writes policy documents and approves requests. Measure it by what its customers ship.

Who does what#

Ambiguity here is the most common cause of platform failure. Write it down.

Who pays#

Cost that nobody sees is cost nobody controls. Move through three stages.

Start showback as soon as there are two teams. It needs only the gateway to tag every call with team, application and use case — which is why attribution is the first thing the platform builds. Shared and fixed costs — provisioned capacity, the platform team — are allocated by a simple, published rule, because any allocation people understand beats a precise one they do not.

Standards worth setting#

A short list that removes most repeated arguments.

• Approved models and the path to add one. A catalogue with tiers and logical names.
• Data classification rules. Which data may go to which class of model deployment.
• A reference design per pattern — knowledge assistant, extraction, agent with tools — already reviewed by security.
• Minimum evaluation by risk tier.
• Logging and retention. What is traced, where, for how long.
• Tool standards. How a system is exposed to agents, and who approves it.
• The intake path. How an idea becomes a registered, classified, approved project.

Standards work when the compliant way is also the easiest way. If the reference design saves a team six weeks, nobody needs to be told to use it.

Measuring the platform#

Shadow AI#

When the official route is slow, people use personal accounts and unapproved tools on company data. Bans do not stop this; they hide it. What works is a combination: a sanctioned tool that is genuinely good, available quickly; a clear, short policy on what may never be pasted anywhere; and an intake path that answers in days. Treat shadow AI as a measurement of how well the platform serves its customers.

Code#

Showback: allocate a month’s AI spend to teams — direct usage as tagged by the gateway, and the shared costs by a published rule.

// showback.go — attribute a month's AI spend to teams.
package main

import (
	"fmt"
	"sort"
)

type usage struct {
	team              string
	inTok, outTok     float64 // millions of tokens, from gateway tags
	inPrice, outPrice float64
}

func main() {
	month := []usage{
		{"claims", 6200, 240, 0.45, 2.20}, // routed mostly to small models
		{"customer service", 3100, 410, 2.00, 10.00},
		{"engineering", 9800, 1300, 2.00, 10.00}, // coding agents: input-heavy
		{"finance", 400, 60, 2.00, 10.00},
	}
	// Shared costs for the month.
	provisioned := 38_000.0 // reserved model capacity
	platformRun := 95_000.0 // platform team, search, tracing, guardrails

	direct := map[string]float64{}
	tokens := map[string]float64{}
	var totalDirect, totalTokens float64
	for _, u := range month {
		c := u.inTok*u.inPrice + u.outTok*u.outPrice
		direct[u.team] = c
		tokens[u.team] = u.inTok + u.outTok
		totalDirect += c
		totalTokens += u.inTok + u.outTok
	}

	// Rule: provisioned capacity by share of tokens; platform cost half evenly, half by spend.
	teams := make([]string, 0, len(month))
	for _, u := range month {
		teams = append(teams, u.team)
	}
	total := map[string]float64{}
	for _, t := range teams {
		share := provisioned*tokens[t]/totalTokens +
			platformRun*0.5/float64(len(teams)) +
			platformRun*0.5*direct[t]/totalDirect
		total[t] = direct[t] + share
	}
	sort.Slice(teams, func(i, j int) bool { return total[teams[i]] > total[teams[j]] })

	fmt.Printf("%-18s %10s %10s %10s %8s\n", "team", "direct", "shared", "total", "share")
	grand := totalDirect + provisioned + platformRun
	for _, t := range teams {
		fmt.Printf("%-18s %10.0f %10.0f %10.0f %7.0f%%\n",
			t, direct[t], total[t]-direct[t], total[t], 100*total[t]/grand)
	}
	fmt.Printf("%-18s %10.0f %10.0f %10.0f\n", "all", totalDirect, provisioned+platformRun, grand)
	fmt.Printf("\nshared costs are %.0f%% of the bill — the part no provider invoice attributes\n",
		100*(provisioned+platformRun)/grand)
}

Two things to notice in the output. The claims team sends the second-largest volume of tokens and has the smallest model bill among the three large teams, because routing sent most of it to small models — a design decision visible in the finance report. And the smallest team’s bill is mostly shared cost, which is the argument you will have about the allocation rule; it is better to have it once, in the open.

Remember this#

• A platform is shared services plus a paved road: the compliant way is the easy way.
• Share what every team needs or what is risky done badly. Leave use cases to the teams.
• Grow the platform from real projects, in four stages. Never ahead of demand.
• Hub and spoke is the usual end state: the hub owns road and rules, the spokes own outcomes.
• Attribute every call. Showback from the second team; chargeback when attribution is trusted.

Try it#

1. Run showback.go. Change the allocation rule so platform cost is split entirely by spend. Who gains and who loses? Which rule would you defend to the finance team?
2. For an organisation you know, list which of the eight shared services exist today, and which project should force the next one into existence.
3. Fill in the who-does-what table for that organisation. Where are two columns claiming the same decision, or none?

Check yourself#

1. What is the test of whether a platform is working?
2. Why should a platform be built from projects rather than ahead of them?
3. What distinguishes showback from chargeback, and what does each need?
4. Why is shadow AI best treated as a measurement rather than a violation?