AI by field: where each kind of AI is used

The previous lesson gave you a map of AI types. This lesson walks the same territory by field: the domains where AI is actually deployed. Each field has settled on characteristic model types, data sources and, crucially, metrics. If you know which field a problem belongs to, you already know a good starting architecture and how its success will be measured, which is most of the way to a project plan.

Intuition: fields are recipes

Think of each field as a cuisine. Italian cooking has staple ingredients (data), standard techniques (model types) and an agreed idea of what "good" tastes like (metrics). A chef new to a cuisine starts from its staples, not from scratch. Likewise, an engineer starting a speech project reaches for audio spectrograms, an encoder-decoder transformer and word error rate; a fraud project reaches for transaction logs, gradient-boosted trees and precision/recall. Recipes evolve, and fusion happens (transformers now appear in almost every field), but the staples are a reliable starting point.

The fields at a glance

The rest of the lesson gives each field a short profile.

Computer vision

Tasks: classification (what is in the image), detection (where each object is, as boxes), segmentation (which pixels belong to which object), tracking across video frames, OCR. Models: convolutional neural networks and vision transformers, usually pretrained on large image collections and fine-tuned. Metrics: top-1 accuracy for classification; intersection over union (IoU), the overlap area of predicted and true boxes divided by their union area, for localisation; mean average precision (mAP), which averages precision over recall levels and classes, for detection. Landmark: in 2012, the AlexNet CNN (Krizhevsky, Sutskever and Hinton) won the ImageNet image-classification challenge by a wide margin, a turning point for deep learning. Today: manufacturing defect inspection, retail shelf monitoring, document scanning, medical imaging, the perception stacks of cars and robots.

Language (NLP)

Tasks: classification (sentiment, topic, intent), extraction (names, dates, amounts), translation, summarisation, question answering, open-ended generation. Models: transformers; encoder models such as BERT-style networks for classification and extraction, decoder LLMs for generation and increasingly for everything. Metrics: accuracy and F1 for classification and extraction; exact match for short answers; BLEU, chrF or COMET for translation; for open-ended output, human ratings or rubric-based grading by another model (with its own validation). Today: support-ticket routing, contract review, search, chat assistants, coding assistants.

Speech

Tasks: automatic speech recognition (ASR, speech to text), text-to-speech (TTS), speaker identification, spoken-language understanding. Models: audio is usually converted to a log-Mel spectrogram (a picture of energy at each frequency band over time) and fed to a transformer encoder-decoder or a model trained with CTC (connectionist temporal classification). OpenAI's Whisper (Radford et al., 2022) is an encoder-decoder transformer trained on 680,000 hours of weakly supervised multilingual audio. Metric: word error rate,

$\text{WER} = \frac{S + D + I}{N}$

where $S$, $D$ and $I$ count the substituted, deleted and inserted words needed to turn the system's transcript into the reference, and $N$ is the number of words in the reference. Lower is better, and WER can exceed 1 when there are many insertions.

Worked example. Reference: "turn on the kitchen lights" ($N = 5$). Hypothesis: "turn the kitten lights please". Aligning: "turn" matches; "on" is deleted ($D=1$); "the" matches; "kitchen" became "kitten" ($S=1$); "lights" matches; "please" is inserted ($I=1$). WER $= (1+1+1)/5 = 0.6$, or 60%.

def wer(reference: str, hypothesis: str) -> float:
    r, h = reference.split(), hypothesis.split()
    # d[i][j] = edit distance between first i ref words and first j hyp words
    d = [[0] * (len(h) + 1) for _ in range(len(r) + 1)]
    for i in range(len(r) + 1):
        d[i][0] = i                      # i deletions
    for j in range(len(h) + 1):
        d[0][j] = j                      # j insertions
    for i in range(1, len(r) + 1):
        for j in range(1, len(h) + 1):
            sub = 0 if r[i - 1] == h[j - 1] else 1
            d[i][j] = min(d[i - 1][j] + 1,        # deletion
                          d[i][j - 1] + 1,        # insertion
                          d[i - 1][j - 1] + sub)  # match or substitution
    return d[len(r)][len(h)] / len(r)

print(wer("turn on the kitchen lights", "turn the kitten lights please"))  # 0.6

This is the standard Levenshtein edit distance computed over words instead of characters. Production toolkits also normalise case and punctuation before scoring, which can change WER noticeably, so always state your normalisation.

Recommendation

Tasks: rank items (products, videos, songs, posts) for each user. Models: collaborative filtering and matrix factorisation (users and items as vectors whose dot product predicts interest); two-tower networks that embed users and items separately for fast retrieval; then a heavier ranking model (often gradient-boosted trees or a transformer over the user's history). Data: mostly implicit feedback: clicks, watch time, skips, purchases. Metrics: offline, recall@k (what fraction of items the user actually engaged with appear in the top $k$) and NDCG (normalised discounted cumulative gain, which rewards putting relevant items near the top); online, A/B tests on engagement, retention or revenue, which are what businesses ultimately trust. Landmark: the Netflix Prize (2006 to 2009) offered a 1-million-US-dollar prize for a 10% improvement in rating-prediction error and popularised matrix factorisation.

Forecasting

Tasks: predict future values of a time series: demand, energy load, staffing, prices, weather. Models: classical statistics (exponential smoothing, ARIMA) remain strong baselines; gradient-boosted trees with lag features are often the practical winner; deep sequence models and pretrained time-series foundation models help with many related series. Metrics: mean absolute error (MAE), root mean squared error (RMSE), mean absolute percentage error (MAPE, which breaks down when actual values are near zero) and quantile (pinball) loss for probabilistic forecasts that give ranges rather than single numbers. Landmark: in weather, machine-learning models such as Google DeepMind's GraphCast (Lam et al., Science, 2023), a graph neural network trained on decades of reanalysis data, have become competitive with traditional numerical weather prediction for medium-range forecasts at a tiny fraction of the compute per forecast.

Anomaly and fraud detection

Tasks: flag rare, suspicious events: fraudulent card payments, account takeovers, failing machines, network intrusions. Models: supervised gradient-boosted trees when labelled fraud exists; unsupervised methods (isolation forests, autoencoders that flag inputs they reconstruct poorly) when it does not; graph neural networks to spot fraud rings connected by shared devices or addresses. Data: extremely imbalanced; fraud may be well under 1% of events, and labels arrive late (a chargeback can take weeks). Metrics: accuracy is useless here, because predicting "never fraud" scores 99%+. Use

$\text{precision} = \frac{TP}{TP + FP}, \qquad \text{recall} = \frac{TP}{TP + FN}$

where $TP$ = frauds correctly flagged, $FP$ = legitimate transactions wrongly flagged, $FN$ = frauds missed.

Worked example. 100,000 transactions contain 200 frauds. The model flags 500 transactions, of which 150 are fraud. Then $TP = 150$, $FP = 350$, $FN = 50$. Precision $= 150/500 = 0.30$; recall $= 150/200 = 0.75$. Accuracy would be $(100{,}000 - 350 - 50)/100{,}000 = 99.6\%$, which tells you nothing useful. The business question is whether catching 75% of fraud is worth having analysts review 350 false alarms; teams decide with a cost-weighted threshold.

Robotics and autonomous systems

Tasks: perceive, localise, plan and act in the physical world: warehouse picking, drones, self-driving cars, surgical assistance. Models: a stack: deep networks for perception; probabilistic state estimation (Kalman and particle filters) for localisation; search-based or optimisation-based planners; increasingly, reinforcement learning and imitation learning from demonstrations. Data: multimodal sensor logs (cameras, lidar, radar, joint encoders), simulation, human demonstrations. Metrics: task success rate, time to complete, safety-critical events and human interventions per distance or hour. Landmark: Waymo opened fully driverless (no safety driver) ride-hailing to members of the public in the Phoenix area in 2020. The hard part of the field is the long tail: rare situations that are cheap to describe and expensive to handle safely.

Scientific discovery

Tasks: predict structures and properties, propose candidates (molecules, materials), speed up simulations, analyse instrument data. Models: transformers and attention-based networks, graph neural networks for molecules, diffusion models for generating structures, physics-informed networks. Landmark: DeepMind's AlphaFold2, built for the 14th Critical Assessment of protein Structure Prediction (CASP14) in 2020, predicted protein 3D structures from amino-acid sequences with accuracy in most cases close to experimental methods such as X-ray crystallography. The team went on to predict structures for nearly all of the roughly 200 million known proteins, and by October 2024 more than two million people in 190 countries had used AlphaFold2. Demis Hassabis and John Jumper shared half of the 2024 Nobel Prize in Chemistry "for protein structure prediction"; the other half went to David Baker "for computational protein design". Metrics: domain measures (for protein structure, scores such as GDT and lDDT that compare predicted and experimental atomic positions) and, ultimately, experimental confirmation in the lab.

Healthcare

Tasks: reading medical images, triaging patients, predicting deterioration, drafting clinical notes, analysing signals such as ECGs. Models: CNNs and vision transformers for imaging; boosted trees and sequence models on electronic health records (EHRs); LLMs for documentation. Metrics: sensitivity (recall of true cases), specificity (fraction of healthy cases correctly cleared), AUROC, and calibration (do predicted probabilities match real frequencies?), validated prospectively and across hospitals. Landmark: in 2018 the US FDA authorised IDx-DR, the first AI device permitted to make a screening decision for diabetic retinopathy without a clinician interpreting the image. Regulators maintain public lists of authorised AI-enabled devices, with radiology the largest category. The AI 410 lesson on healthcare and science covers regulation and evidence standards.

Finance

Tasks: credit scoring, fraud and anti-money-laundering, trading signals, risk modelling, document analysis (filings, contracts), customer service. Models: logistic regression and gradient-boosted trees dominate regulated credit decisions because they are accurate and explainable; sequence models for transaction streams; LLMs for reading and summarising documents. Metrics: AUROC (or the Gini coefficient, $2 \times \text{AUROC} - 1$), calibration, profit-and-loss backtests, and fairness checks. In the US, lenders must give applicants the principal reasons for an adverse credit decision, which strongly favours models whose decisions can be explained.

Creative work

Tasks: image, video, music and voice generation; writing assistance; design ideation; code generation. Models: diffusion models (latent diffusion, introduced by Rombach et al. in 2022, underlies Stable Diffusion) for images and video; LLMs for text and code; transformer and diffusion models for audio. Metrics: automatic metrics such as FID (Fréchet Inception Distance, comparing statistics of generated and real images) and CLIP score (text-image agreement) are rough guides; human preference studies are the real test. Open issues as of November 2026: copyright and training-data licensing, consent for voice and likeness, and content provenance (the AI 230 course covers deepfakes and provenance standards).

Common pitfalls

• Using the wrong metric for the field. Accuracy on imbalanced fraud data, or MAPE on series with zeros, will mislead you badly.
• Ignoring the deployment loop. Recommenders and fraud models change user and attacker behaviour, so offline metrics drift from online reality.
• Assuming lab results transfer. A medical model validated in one hospital can degrade on another hospital's scanners and population.
• Skipping the boring baseline. Exponential smoothing for forecasting or logistic regression for credit often gets within a few percent of complex models, with far lower cost and risk.
• Treating one landmark as the whole field. AlphaFold solved one central problem superbly; it did not make drug discovery automatic.

In practice

When you face a new problem, first ask which field it belongs to, then borrow that field's recipe: its standard data preparation, a strong baseline model and its accepted metrics. Define the business metric (fraud losses, forecast-driven stock-outs, clinician time saved) next to the technical metric, and agree on both before building. Expect cross-pollination: transformers and pretrained foundation models now appear in every field, but each field's data quirks (imbalance, time ordering, regulation, safety) still decide what works. The AI 220 Applied AI Fields course gives each of these fields a full lesson.