The History of AI and How Deep Learning Transformed It: A Timeline

Key takeaways

• 1950: Turing asks whether machines can think. 1956: the Dartmouth workshop names the field.
• Two "AI winters" followed over-promising, in the 1970s and late 1980s into the 1990s.
• Deep learning took off in 2012 when AlexNet won ImageNet using GPUs and large data.
• The 2017 transformer paper led to GPT, BERT and ChatGPT (2022).
• Three ingredients made it work: much more data, faster hardware and better training methods.

What is AI, and what is deep learning?

Artificial intelligence is the effort to make computers do tasks that normally need human intelligence, such as understanding language, recognising images or making decisions. Machine learning is the part of AI where systems learn patterns from data instead of following hand-written rules. Deep learning is a kind of machine learning that uses neural networks with many layers. Most headline AI progress since 2012 comes from deep learning.

How did AI begin (1950s to 1960s)?

• 1950: Alan Turing publishes "Computing Machinery and Intelligence" and proposes what became known as the Turing test.
• 1956: The Dartmouth workshop, organised by John McCarthy and colleagues, is where the name "artificial intelligence" was adopted.
• 1958: Frank Rosenblatt demonstrates the perceptron, an early learning neural network.

Early optimism was high. Programs could prove theorems and play simple games, and researchers predicted human-level AI soon.

What were the AI winters?

Progress was slower than promised. Limits of single-layer perceptrons, weak computers and little data led to cuts in funding in the 1970s. In the 1980s, expert systems, programs with hand-coded rules from specialists, were a commercial success, but they were costly to maintain and brittle. When that market collapsed in the late 1980s and early 1990s, funding fell again. These periods are called AI winters.

How did machine learning take over (1980s to 2000s)?

• 1986: Rumelhart, Hinton and Williams popularise backpropagation, a way to train multi-layer neural networks.
• 1989 to 1998: Yann LeCun and colleagues develop convolutional networks for reading handwritten digits.
• 1997: IBM's Deep Blue beats world chess champion Garry Kasparov, using search and custom hardware rather than learning.
• 1990s to 2000s: Statistical methods such as support vector machines and decision trees dominate practical machine learning.

What changed in 2012?

Three things came together. There was far more labelled data, such as the ImageNet dataset created by Fei-Fei Li and colleagues. GPUs, designed for graphics, turned out to be excellent for the matrix maths in neural networks. And training methods had improved. In 2012, AlexNet, a deep convolutional network by Alex Krizhevsky, Ilya Sutskever and Geoffrey Hinton, won the ImageNet image recognition competition by a large margin. Soon, deep learning led in speech recognition, translation and more.

What happened between 2013 and 2020?

• 2013 to 2014: Word embeddings (word2vec) and generative adversarial networks (GANs) appear.
• 2015: ResNet makes very deep networks trainable.
• 2016: DeepMind's AlphaGo defeats Go champion Lee Sedol.
• 2017: The paper "Attention Is All You Need" introduces the transformer architecture.
• 2018: BERT shows large pre-trained language models can be adapted to many tasks. Bengio, Hinton and LeCun receive the Turing Award for deep learning.
• 2020: OpenAI's GPT-3 shows strong few-shot abilities, and DeepMind's AlphaFold 2 makes a leap in protein structure prediction.

What is the foundation model and chatbot era?

Transformers scale well with data and compute, so researchers trained ever larger models on text, images and code. In November 2022, OpenAI released ChatGPT, bringing a conversational interface to millions. Many companies followed with their own models. In 2024, the Nobel Prize in Physics went to John Hopfield and Geoffrey Hinton for foundational work on artificial neural networks, as announced in the official press release, and the Chemistry prize recognised AlphaFold and protein design.

How does deep learning work, simply?

1. A network of layers of simple units takes input numbers, such as pixels.
2. Each layer transforms its input using adjustable weights.
3. The network's output is compared with the right answer to compute an error.
4. Backpropagation works out how to nudge each weight to reduce the error.
5. Repeating this over millions of examples lets the network learn useful features by itself.

What are the limits and concerns?

• Models need large data and energy, and can be costly to train.
• They can be biased, can fail in unexpected ways and are often hard to interpret.
• Language models can generate false statements confidently.
• Security issues such as data poisoning and prompt injection are active areas of research.

Next steps

Read what ChatGPT is and how it works for the current chapter. To learn the methods behind this history, see our machine learning course and AI engineering course.

Related reading

• How did OpenAI's ChatGPT bypass CAPTCHA without detection and what are the cybersecurity risks?
• What is the role of AI and Machine Learning in modern networking? The Detailed Guide