How We Got Here: A Short History of AI

Why this matters

AI did not appear overnight in 2022. It is the result of about seventy years of ideas, setbacks and breakthroughs. The history also carries a practical lesson: periods of excitement have been followed by disappointment before, so it pays to separate real progress from hype.

The beginning: naming the field (1955–1956)

The term "artificial intelligence" comes from a 1955 proposal by John McCarthy, Marvin Minsky, Nathaniel Rochester and Claude Shannon for a summer research workshop at Dartmouth College. That workshop, held in 1956, is usually treated as the founding of AI as a research field. Early optimism was high: many researchers expected human-level machine intelligence within a generation.

The perceptron (1958)

In July 1958, the US Office of Naval Research publicly demonstrated psychologist Frank Rosenblatt's perceptron. It was an early artificial neuron that learned from examples by adjusting its weights, the same basic idea behind today's neural networks. Rosenblatt then built a dedicated machine, the Mark I Perceptron, for recognising visual patterns.

In 1969, Marvin Minsky and Seymour Papert published the book Perceptrons, which showed that a single-layer perceptron could not learn certain simple patterns, the textbook case being "exclusive or" (XOR). Their analysis helped make neural network research unfashionable for years.

The AI winters

An AI winter is a period when funding and interest collapse after promises go unmet. There were two major ones:

• Roughly 1974 to 1980. In the UK, the 1973 Lighthill report, commissioned by the Science Research Council, judged that AI had failed to meet its grand goals, and UK funding was sharply cut. In the US, the defence research agency DARPA moved away from open-ended AI research.
• Roughly 1987 into the 1990s. In the 1980s, expert systems (programs built from many hand-written if-then rules supplied by human experts) were a commercial boom. They proved expensive to maintain and brittle with unusual inputs, and the market for specialised AI computers collapsed around 1987.

Backpropagation goes mainstream (1986)

In 1986, David Rumelhart, Geoffrey Hinton and Ronald Williams published a paper in Nature showing how backpropagation could train neural networks with hidden layers, the method that works out how to adjust every weight to reduce error. Earlier researchers, including Paul Werbos in the 1970s, had described related ideas; the 1986 paper made the technique widely known. Multi-layer networks could now learn patterns like XOR that had defeated the single perceptron. Even so, computers and datasets were too small for neural networks to dominate for another two decades.

Deep learning arrives: AlexNet (2012)

ImageNet was a large collection of labelled photos used for an annual image-recognition competition. In 2012, a deep neural network later called AlexNet, by Alex Krizhevsky, Ilya Sutskever and Geoffrey Hinton, won it decisively. Its top-5 error rate (the share of images where the correct label was not among its five best guesses) was 15.3%, compared with 26.2% for the next-best entry. It had about 60 million parameters and was trained on two consumer graphics cards (GPUs) over five to six days.

The lesson the field took away: large neural networks, lots of data and GPU computing together beat hand-engineered approaches. That combination is what we now call deep learning.

The transformer (2017)

In June 2017, researchers at Google published "Attention Is All You Need", introducing the transformer. It replaced older step-by-step text-processing designs with attention, which lets a model relate every word to every other word and trains efficiently in parallel. Nearly every major language model since is built on it.

Scale: GPT-3 (2020)

In May 2020, OpenAI described GPT-3, a language model with 175 billion parameters. Its paper showed that a big enough model could often perform a new task from just a few examples written into the prompt, without retraining. Growing model size, data and computing power became a central strategy across the industry.

ChatGPT (November 2022)

On 30 November 2022, OpenAI released ChatGPT, a chat interface to a GPT-3.5 model refined with reinforcement learning from human feedback. Its growth was extraordinary: it was estimated to have reached about 100 million monthly users by January 2023. Generative AI became a mainstream technology almost overnight.

Open-weight models (2023 onward)

An open-weight model is one whose trained parameters can be downloaded and run on your own hardware. In February 2023, Meta released LLaMA to researchers under a non-commercial licence; its weights leaked publicly about a week later. In July 2023, Llama 2 followed with a licence that allowed many commercial uses. Many other organisations released open-weight models after that. In January 2025, DeepSeek published DeepSeek-R1, a reasoning model whose weights were released under the permissive MIT licence, showing that openly available models could compete near the frontier.

Reasoning models and agents (2024–2026)

In September 2024, OpenAI introduced o1, a model trained with reinforcement learning to work through a hidden chain of reasoning before answering. Spending more computation "thinking" improved results on maths, science and coding. Other developers released their own reasoning models soon after.

At the same time, models increasingly acted rather than only talked. An AI agent is a system that uses a model to plan and carry out multi-step tasks with tools. In October 2024, Anthropic released "computer use" in public beta, letting its model operate a computer by looking at the screen, moving the cursor, clicking and typing. In November 2024, Anthropic introduced the Model Context Protocol (MCP), an open standard for connecting AI applications to tools and data. In December 2025, MCP was contributed to the newly formed Agentic AI Foundation under the Linux Foundation, alongside agent projects from OpenAI and Block.

Going deeper

History makes more sense when you can see the technical problem each breakthrough solved. This section works through the most important one: why a single perceptron hit a wall, and why hidden layers plus backpropagation got past it.

The perceptron as an equation

A perceptron takes inputs $x_1, \dots, x_n$, multiplies each by a weight $w_i$, adds a bias $b$, and outputs 1 if the total is positive:

$\hat{y} = \begin{cases} 1 & \text{if } \mathbf{w} \cdot \mathbf{x} + b > 0 \\ 0 & \text{otherwise} \end{cases}$

Rosenblatt's learning rule nudges the weights only when the output is wrong:

$\mathbf{w} \leftarrow \mathbf{w} + \eta\,(y - \hat{y})\,\mathbf{x}, \qquad b \leftarrow b + \eta\,(y - \hat{y})$

where $y$ is the correct answer and $\eta$ is the learning rate. If the answer is right, $y - \hat{y} = 0$ and nothing changes. If the perceptron said 0 but should have said 1, the inputs are added to the weights, making that example more likely to fire next time; the opposite mistake subtracts them.

Worked example: learning AND. Start with $\mathbf{w} = (0, 0)$, $b = 0$, $\eta = 1$. In the first pass the inputs $(0,0)$, $(0,1)$ and $(1,0)$ all score $0$, which is not greater than 0, so the output is 0 and correct. For $(1,1)$ the output is 0 but the target is 1, so $\mathbf{w}$ becomes $(1, 1)$ and $b$ becomes $1$. On the next pass $(0,0)$ now scores $1$ and wrongly fires, so $b$ drops back to $0$, and so on. The last correction happens on the fifth pass, and the sixth pass goes through without a single mistake, with $\mathbf{w} = (2, 1)$ and $b = -2$. Check: $(0,0) \to -2$, $(0,1) \to -1$, $(1,0) \to 0$, all output 0; $(1,1) \to 1$, output 1. Correct on all four. A classic result, the perceptron convergence theorem, guarantees this rule stops after a finite number of mistakes whenever some straight line (in general, a flat boundary) separates the two classes.

Why XOR is impossible for one perceptron

XOR outputs 1 when exactly one input is 1. Suppose some $w_1, w_2, b$ worked. The four cases would require:

1. $(0,0) \to 0$: $\; b \le 0$
2. $(1,0) \to 1$: $\; w_1 + b > 0$
3. $(0,1) \to 1$: $\; w_2 + b > 0$
4. $(1,1) \to 0$: $\; w_1 + w_2 + b \le 0$

Add inequalities 2 and 3: $w_1 + w_2 + 2b > 0$, so $w_1 + w_2 + b > -b$. By inequality 1, $-b \ge 0$, so $w_1 + w_2 + b > 0$. That contradicts inequality 4. No weights exist: the XOR classes are not linearly separable. No amount of training can fix a model that cannot represent the answer.

How a hidden layer fixes it

XOR equals "OR and not both", which is (OR) AND (NAND). Each of those is linearly separable, so a network with two hidden units and one output unit can compute it:

step = lambda z: int(z > 0)
def xor_net(x1, x2):
    h_or   = step(1*x1 + 1*x2 - 0.5)     # hidden unit 1: OR
    h_nand = step(-1*x1 - 1*x2 + 1.5)    # hidden unit 2: NAND
    return step(1*h_or + 1*h_nand - 1.5) # output: AND of the two
for a in (0, 1):
    for b in (0, 1):
        print(a, b, xor_net(a, b))

It prints the XOR truth table: 0, 1, 1, 0. Here the weights were set by hand. The catch in the 1960s was that nobody had a practical way to learn the weights of hidden units, because the perceptron rule needs a target for each unit and hidden units have none. The 1986 backpropagation paper supplied the missing piece: replace the hard step with a smooth function (such as the S-shaped sigmoid) so that the output changes gradually with every weight, then use the chain rule of calculus to pass the error backwards and compute how much each hidden weight should change. A hard step has zero slope almost everywhere, so it gives gradient descent nothing to follow.

You can watch the original limitation in code:

import numpy as np

X = np.array([[0, 0], [0, 1], [1, 0], [1, 1]])
targets = {"AND": [0, 0, 0, 1], "OR": [0, 1, 1, 1], "XOR": [0, 1, 1, 0]}

def train_perceptron(X, y, lr=1.0, epochs=20):
    w, b = np.zeros(2), 0.0
    for _ in range(epochs):
        errors = 0
        for xi, yi in zip(X, y):
            pred = int(w @ xi + b > 0)
            w += lr * (yi - pred) * xi      # Rosenblatt's update rule
            b += lr * (yi - pred)
            errors += int(pred != yi)
        if errors == 0:
            return w, b, True
    return w, b, False

for name, y in targets.items():
    w, b, ok = train_perceptron(X, np.array(y))
    print(f"{name}: converged={ok}  w={w}  b={b}")

The rule converges on AND and OR but never on XOR; after 20 passes it is still making mistakes.

Reading a benchmark jump

AlexNet's top-5 error of 15.3% against 26.2% for the next-best entry is an absolute drop of $26.2 - 15.3 = 10.9$ percentage points and a relative drop of $10.9 / 26.2 \approx 42\%$. Both framings appear in headlines, and they can make the same result sound very different. Later improvements on the same benchmark were often a point or two in absolute terms; what made 2012 a turning point was the size of the jump and the method behind it.

Common pitfalls

These are the common ways AI history gets misread.

• "One book killed neural networks." Perceptrons proved limits of single-layer networks; the long slump also reflected weak computers, small datasets and funding decisions. Single causes make tidy stories and poor history.
• Crediting one inventor. Backpropagation, like many ideas, had several precursors before the 1986 paper made it widely known. Check primary sources before attributing.
• Assuming smooth, steady progress. The record is bursts and stalls. Extrapolating a few good years in a straight line has repeatedly produced bad predictions.
• Mixing up announcement, research preview and general release. Dates differ, and claims are often made about the announcement.
• Comparing numbers across different benchmarks or test sets. An error rate only means something against the same data and the same metric.
• Ignoring hardware and data. Many "algorithmic" breakthroughs only became possible when compute and datasets caught up.

In practice

• When you read that AI will soon do everything, remember the 1950s and 1980s predictions; progress is real but uneven.
• The modern recipe (neural networks, large data, GPUs, transformers) explains why today's AI is good at pattern-heavy tasks and still weak at guaranteed accuracy.
• Open-weight models mean capable AI can run on your own hardware, a topic covered in the Running AI Locally track.