Adopting AI in Organizations: Use Cases, TCO and ROI

Most organisations do not fail at AI because the models are weak. They fail because they pick the wrong problems, underestimate the full cost, never measure the benefit, cannot get a pilot into daily work, or lack the data and governance to scale it. This lesson is the management toolkit for getting AI from slideware to measurable value: how to find and rank use cases, decide whether to build, buy or partner, calculate total cost of ownership and return on investment, run pilots that can graduate to production, and bring people, data and risk management along.

Intuition: AI adoption is a portfolio, not a moonshot

Think like an investor rather than an inventor. You are not looking for the single most impressive AI idea; you are building a portfolio of bets with different sizes, odds and payback times. A few quick, low-risk wins (drafting, summarising, search over internal documents) fund learning and credibility. A smaller number of larger bets (redesigning a core process around AI) carry more risk and more reward. And like any investor you need a way to value each bet, a stage-gate process to stop the ones that are not working, and the discipline to actually stop them.

What the evidence says, briefly

It is worth knowing a few careful studies, because the field is full of unsourced statistics.

• Productivity gains are real but uneven. In a study of more than 5,000 customer-support agents at one company, access to a generative-AI assistant raised issues resolved per hour by about 14% on average, and by about 34% for novice and lower-skilled workers, with little effect for the most experienced (Brynjolfsson, Li and Raymond, NBER working paper, 2023).
• The "jagged frontier". In a field experiment with 758 Boston Consulting Group consultants, those using GPT-4 on tasks inside the model's capabilities completed 12.2% more tasks, 25.1% faster, with results rated more than 40% higher in quality. On a task deliberately chosen to lie outside its capabilities, consultants using AI were 19 percentage points less likely to reach the correct answer than those without it (Dell'Acqua et al., Harvard Business School working paper, 2023). The lesson for adopters: map where the tool helps and where it misleads before rolling it out.
• Why projects fail. A RAND Corporation study based on interviews with 65 experienced data scientists and engineers (Ryseff, De Bruhl and Newberry, 2024) identified five root causes: leaders misunderstanding or miscommunicating the problem to solve; lacking the data needed; chasing the newest technology rather than users' real problems; underinvesting in data and deployment infrastructure; and applying AI to problems too hard for current methods. The report repeats an often-quoted estimate that more than 80% of AI projects fail, but attributes it to "some estimates", not to its own measurement; treat that number as indicative, not established.
• Pilots stall. In July 2024 Gartner predicted that at least 30% of generative-AI projects would be abandoned after proof of concept by the end of 2025, citing poor data quality, inadequate risk controls, escalating costs or unclear business value. This was a forecast, not a measured outcome.

Notice the pattern: almost none of the failure causes is "the model was not good enough".

Step 1: Find and rank use cases

Start from business problems, not from the technology. Good sources: processes with high volumes of repetitive judgement (triage, classification, extraction), work bottlenecked on reading or writing text, decisions made with poor information, and complaints from customers and staff. For each candidate, write a one-paragraph brief: the decision or task, who does it today, volume, current cost and quality, what "better" means, and what data exists.

Then score candidates on two axes, each from 1 to 5:

• Value: size of the benefit (cost saved, revenue gained, risk reduced), how many people or transactions it touches, and strategic importance.
• Feasibility: data availability and quality, technical maturity of the approach, integration effort, risk and regulatory exposure, and the tolerance for errors in that task.

Plot them on a 2×2 grid. High value and high feasibility: start now. High feasibility and modest value: quick wins that build skills. High value and low feasibility: strategic bets that need data work or risk mitigation before a pilot. Low on both: drop. Weight the criteria if your organisation cares more about some (a bank might weight regulatory risk heavily). Re-score as you learn; feasibility in particular changes as models improve.

A useful filter on feasibility is error tolerance: what happens when the AI is wrong? Drafts reviewed by a human tolerate errors well. Unreviewed actions with legal, financial or safety consequences do not, and need far more evidence and controls.

Step 2: Build, buy or partner

Most organisations mix them: buy for commodity productivity, build where the data and process are a moat, partner to start fast and transfer skills. "Build" today rarely means training a model from scratch; it usually means building an application, retrieval and evaluation around existing models (see Fine-tuning vs prompting and RAG vs long context vs fine-tuning).

Step 3: Total cost of ownership

Total cost of ownership (TCO) is every cost over the system's life, not just the licence or API bill. Categories:

• One-time: integration engineering, data preparation and labelling, evaluation set creation, security and legal review, initial training of staff.
• Recurring: licences or seats, model and API usage, hosting and infrastructure, monitoring and evaluation, maintenance engineering, vendor management, ongoing training, governance and audit.
• Hidden: human review time, error handling, re-work when models or vendors change, and the opportunity cost of the team.

Worked example: buy vs build for support reply drafting

A company has 60 support agents. Each handles 40 tickets a day for 250 working days a year, so the annual volume is $60 \times 40 \times 250 = 600{,}000$ tickets. Average handling time is 10 minutes. All figures below are illustrative assumptions for the exercise. Loaded labour cost is $40 per agent-hour; loaded engineer cost is $180,000 per year.

Buy (a vendor's AI drafting add-on):

• Licences: $60 \times \text{\textdollar}50 \times 12 = \text{\textdollar}36{,}000$ per year
• Integration with the ticketing system (one-time): $40,000
• Administration and vendor management, 0.2 of an IT person at $150,000: $30,000 per year
• Staff training (one-time): $15,000

Three-year TCO: one-time $\text{\textdollar}40{,}000 + \text{\textdollar}15{,}000 = \text{\textdollar}55{,}000$; recurring $3 \times (\text{\textdollar}36{,}000 + \text{\textdollar}30{,}000) = \text{\textdollar}198{,}000$; total $253,000.

Build (in-house app on a hosted model API):

• Build: 2 engineers for 6 months = 1 engineer-year = $180,000 (one-time)
• Evaluation set and labelling (one-time): $20,000; staff training (one-time): $15,000
• Maintenance, 0.5 engineer: $90,000 per year
• Model API usage: $600{,}000 \text{ tickets} \times \text{\textdollar}0.01 = \text{\textdollar}6{,}000$ per year
• Hosting, monitoring and observability: $12,000 per year

Three-year TCO: one-time $215,000; recurring $3 \times \text{\textdollar}108{,}000 = \text{\textdollar}324{,}000$; total $539,000. (For simplicity both options' recurring costs run for all three years.)

Note what dominates: people, not tokens. The model API bill ($18,000 over three years) is about 3% of the build option's three-year cost.

Step 4: Measuring ROI

$\text{ROI} = \frac{\text{Benefit} - \text{Cost}}{\text{Cost}}, \qquad \text{Payback period} = \frac{\text{Up-front investment}}{\text{Net benefit per period}}.$

A pilot with 10 agents measured average handling time falling from 10.0 to 8.5 minutes, with no drop in customer satisfaction. Gross time saved per year: $600{,}000 \times 1.5 = 900{,}000$ minutes $= 15{,}000$ hours, worth $15{,}000 \times \text{\textdollar}40 = \text{\textdollar}600{,}000$.

But saved minutes only become money if the organisation does something with them: absorbs growth without hiring, reduces overtime or contractors, or moves people to higher-value work. Assume a realisation rate of 50%, so the realised benefit is $300,000 per year, or $900,000 over three years.

• Buy: $\text{ROI} = (\text{\textdollar}900{,}000 - \text{\textdollar}253{,}000)/\text{\textdollar}253{,}000 = 2.56$, i.e. 256%. Monthly net benefit $= (\text{\textdollar}300{,}000 - \text{\textdollar}66{,}000)/12 = \text{\textdollar}19{,}500$, so payback $= \text{\textdollar}55{,}000 / \text{\textdollar}19{,}500 \approx 2.8$ months after go-live.
• Build: $\text{ROI} = (\text{\textdollar}900{,}000 - \text{\textdollar}539{,}000)/\text{\textdollar}539{,}000 = 0.67$, i.e. 67%.

Buying wins here, unless building creates something the vendor cannot offer (deep integration with proprietary knowledge, a differentiating customer experience) worth more than the $286,000 difference. Do a sensitivity check: at a realisation rate of 25% the benefit is $450,000 over three years, so buying still returns 78% but building loses money (−17%). Always show decision-makers the range, not a single number.

Measurement design matters as much as arithmetic. Use a baseline period, a comparison group (agents without the tool, or a staggered rollout), and pre-agreed metrics: a primary outcome (handling time), quality guardrails (customer satisfaction, error and escalation rates), and adoption (share of drafts used). Without a comparison group, seasonal changes and the novelty effect masquerade as AI impact.

def three_year_case(one_time, recurring, annual_benefit, years=3):
    cost = one_time + recurring * years
    benefit = annual_benefit * years
    roi = (benefit - cost) / cost
    monthly_net = (annual_benefit - recurring) / 12
    payback = one_time / monthly_net if monthly_net > 0 else float("inf")
    return round(cost), round(roi, 2), round(payback, 1)

for realisation in (0.25, 0.5, 0.75):
    benefit = 600_000 * realisation
    print(realisation, "buy", three_year_case(55_000, 66_000, benefit),
          "build", three_year_case(215_000, 108_000, benefit))

Step 5: From pilot to production

Pilots often succeed in a demo and die in production. Design them to graduate:

1. Define success criteria before starting: target metric, minimum improvement, quality guardrails, and a decision date.
2. Use real users, real data and the real workflow, not a sandbox with curated examples.
3. Build the evaluation set during the pilot: it becomes the regression suite for production.
4. Plan the production path early: security review, data-processing agreements, identity and access integration, monitoring and support ownership. These take weeks; start them in parallel.
5. Stage-gate: proof of concept (can it work?) → pilot (does it work for real users?) → limited production (does it hold at scale?) → full rollout. At each gate, continue, change, or stop.
6. Name an owner in the business, not only in IT, who is accountable for the outcome.

Step 6: People, change management and training

AI changes how work is done, so adoption is a change-management problem. What helps:

• Explain why, what changes for each role, and what does not. Be honest about effects on jobs; rumours fill silence.
• Involve the people who do the work in design and evaluation. They know the edge cases and become champions.
• Train by role: hands-on practice with real tasks, how to verify outputs, what not to paste into tools, and when to escalate. Refresh as tools change.
• Redesign the workflow, not just add a tool. Decide who reviews AI output, how errors are reported, and how incentives and targets change.
• Measure adoption (active use, share of outputs accepted) and talk to non-users: low adoption is a design signal.
• Remember the jagged frontier: train people on where the tool is unreliable, not only where it shines.

Step 7: Data readiness

Ask, for each use case: Does the needed data exist? Can we legally use it for this purpose (consent, contracts, privacy law)? Is it accessible (APIs, permissions), of sufficient quality (accuracy, completeness, timeliness), and representative of the cases the system will see? For generative AI over documents, readiness also means: are documents current, deduplicated, permissioned correctly (the assistant must not reveal what a user could not otherwise see), and in formats you can parse? Fixing data is often the longest part of the project; budget for it.

Step 8: Risk and governance

Integrate AI into existing risk processes rather than inventing a parallel bureaucracy. A proportionate approach:

• An AI inventory: every system in use, its owner, purpose, data, vendor and risk tier.
• Risk tiering: light-touch review for internal drafting tools; deeper assessment (impact assessment, bias testing, human oversight design, legal review) for systems that affect people's rights, money, health or safety.
• Policies: acceptable use, data handling, vendor due diligence, incident reporting.
• Frameworks: the NIST AI Risk Management Framework (Govern, Map, Measure, Manage) and ISO/IEC 42001 (an AI management-system standard) give structure; sector rules and laws such as the EU AI Act may impose specific duties. See the governance and law lessons for details.

Common pitfalls

• Starting from "we need an AI strategy" or a specific technology rather than a business problem.
• Counting gross time saved as profit, without a realisation plan.
• Comparing licence fees with API prices and ignoring people, integration and governance costs.
• Pilots with no success criteria, no comparison group and no decision date ("pilot purgatory").
• Ignoring data rights and permissions until legal review blocks launch.
• Rolling out tools without training on their failure modes.
• Treating governance as a final sign-off instead of a design input.

In practice

Run a short discovery to list 15–30 candidate use cases, score them on value and feasibility, and pick two or three for pilots with written success criteria. Build a three-year TCO and a sensitivity table for each. Start production paperwork in parallel with the pilot. Report results honestly, including stopped projects, and reinvest in what works. Keep an AI inventory from day one; it is far easier than reconstructing one later.