Three programs, written decades apart, have all been called artificial intelligence. The first is a checkers player of the late 1950s that improves its play with experience (Samuel, 1959). The second is a spam filter that counts how often words appear in mail people have marked as junk and in mail they have kept, and scores each new message with naive Bayes. The third is a chat assistant that writes a passable sonnet, a working Python function and a summary of a legal contract, all from the same trained network. The three share almost nothing in method. What they share is the aim: each does something that, when a person does it, is taken as a sign of intelligence.
That aim is the most stable thing about the field, and it is why the name has covered so many techniques. This article sets out what the name has meant to the people who coined and shaped it, the two broad strategies for pursuing it, how the field's history alternated between confidence and disappointment, and what can be said carefully about the systems of today. The rest of the topic covers machine learning, the two large application areas of vision and language, generative models and how they are prompted, and the risks that come with deploying all of them.
Can machines think?
Alan Turing's 1950 paper opens: "I propose to consider the question, 'Can machines think?'" (Turing, 1950, §1). He then declines to answer it as posed. Defining "machine" and "think" by how the words are commonly used would, he argues, reduce the question to an opinion poll, so he replaces it with a game. In the imitation game, an interrogator exchanges typed questions and answers with two hidden players and must decide which is which. Turing's question becomes whether the interrogator will "decide wrongly as often" when a machine takes one of the parts as when the game is played between a man and a woman (§1). He considered the original question "too meaningless to deserve discussion" (§6), and offered a forecast in its place: that in about fifty years computers with a storage capacity of about $10^9$ would play the game so well that an average interrogator would "not have more than 70 per cent chance of making the right identification after five minutes of questioning" (§6).
The game, now usually called the Turing test, moves attention from what a machine is to what it does, which is the stance most of the field has taken since. It also has well-known limits. John McCarthy called it "a one-sided test": a machine that passes should be considered intelligent, but a machine could be intelligent without knowing enough about people to imitate one, and "some people are easily led into believing that a rather dumb program is intelligent" (McCarthy, 2007, §1). The second point has become more pressing as programs that produce fluent text have become common; the risks article returns to it.
Most of Turing's paper is a reply to objections, and its final section, "Learning Machines", proposes a different route to intelligent behavior from programming it directly: "Instead of trying to produce a programme to simulate the adult mind, why not rather try to produce one which simulates the child's?" (§7). He compares the trial-and-error search for a good child machine with evolution, identifying the structure of the machine with hereditary material, its changes with mutation, and natural selection with the judgment of the experimenter (§7). That analogy is the starting point of the course wiki's History of Evolutionary Computation; the idea of a machine that is educated rather than programmed is the starting point of machine learning.
The name and the definitions
The name comes from a funding proposal. In August 1955 John McCarthy, Marvin Minsky, Nathaniel Rochester and Claude Shannon proposed "a 2 month, 10 man study of artificial intelligence" at Dartmouth College for the summer of 1956 (McCarthy, Minsky, Rochester and Shannon, 1955, p. 1). The proposal states its working assumption directly:
The study is to proceed on the basis of the conjecture that every aspect of learning or any other feature of intelligence can in principle be so precisely described that a machine can be made to simulate it.
It then lists the problems the group would attack: how to make machines "use language, form abstractions and concepts, solve kinds of problems now reserved for humans, and improve themselves", along with neuron nets, the size of a calculation, and the role of randomness in creative thinking (p. 2). Most of those headings are still research areas.
Half a century later McCarthy gave a short definition: artificial intelligence "is the science and engineering of making intelligent machines, especially intelligent computer programs", and intelligence "is the computational part of the ability to achieve goals in the world" (McCarthy, 2007, §1). He added two qualifications that are easy to lose. The field is related to understanding human intelligence, "but AI does not have to confine itself to methods that are biologically observable"; and intelligence is not a single thing, so the question "is this machine intelligent or not?" has no yes-or-no answer, since a program can have some of the mechanisms and lack others (§1).
Russell and Norvig's textbook sorts the many published definitions along two lines: whether they concern thinking or acting, and whether they measure success against human performance or against an ideal standard of rationality. The four combinations are thinking humanly (modeling human cognition), acting humanly (the Turing test approach), thinking rationally (the "laws of thought" tradition of logic), and acting rationally (building agents that do the best they can given what they know). The textbook itself adopts the last, the rational-agent view (Russell and Norvig, 2020, §1.1; cited by attribution). The four are useful less as rival definitions than as a reminder that "artificial intelligence" can name a scientific question about minds or an engineering goal about behavior, and that a system can succeed at one without the other.
A common short definition, that AI is the use of algorithms and models "to mimic human thought", fits the first of the four but not the others. McCarthy's qualification applies: a chess program that searches millions of positions per move, or a classifier that weighs ten thousand word counts, achieves goals by means no person uses.
The parts of the field
Russell and Norvig's discussion of the Turing test lists the capabilities a machine would need to pass it: natural language processing to communicate, knowledge representation to store what it knows or hears, automated reasoning to answer questions and draw conclusions, and machine learning to adapt and detect patterns; a version of the test that includes physical interaction adds computer vision and robotics (Russell and Norvig, 2020, §1.1.1; cited by attribution). Those six headings are still a fair map of the field's subdivisions:
| Area | The question it asks | Where this site covers it |
|---|---|---|
| Machine learning | How can a program improve its performance from data? | Machine Learning and most of this collection |
| Computer vision | How can a program extract information from images and video? | Computer Vision |
| Natural language processing | How can a program analyze and generate language? | Natural Language Processing, the course wiki's Text Processing and Indexing |
| Knowledge representation | How should facts and relationships be stored so that a program can use them? | the course wiki's Bayesian Inference Networks (a probabilistic example) |
| Automated reasoning and search | How can a program draw conclusions or find a solution among many possibilities? | the course wiki's Local Search and Evolutionary Computing |
| Robotics | How can a program sense and act in the physical world? | the course wiki's Evolutionary Robotics |
The boundaries blur. Robots rely on vision and planning, language systems are built by machine learning, and a modern language model is used for reasoning and, through tools, for acting. Deep learning cuts across all of them. It refers to neural networks with many layers of learned transformations, the subject of the collection's neural network articles; it is a family of machine learning methods, not a separate area, and since about 2012 it has been the dominant approach in vision, speech and language.
Two strategies: rules and learning
From the start there were two ways to pursue the Dartmouth conjecture. One is to write the knowledge and the reasoning down. The proposal's second heading speculates that "a large part of human thought consists of manipulating words according to rules of reasoning and rules of conjecture" (McCarthy et al., 1955, p. 1), and much early work took that path: programs that proved theorems in logic, searched game trees, planned sequences of actions, and later, in the expert systems of the 1970s and 1980s, encoded a specialist's knowledge as hundreds or thousands of if-then rules. This tradition is called symbolic AI, because its programs manipulate explicit symbols for objects, properties and relations. Its strengths are transparency and precision: every conclusion can be traced to the rules that produced it, and a rule can be corrected by editing it.
The other way is to let the program find the regularities itself, from examples. The same proposal's third heading asks how "a set of (hypothetical) neurons" can be arranged "so as to form concepts" (p. 2), and Turing's child machine is the same idea in different words. This tradition, which includes neural networks and statistical learning, is often called statistical or connectionist AI. Its strength is that it does not need the knowledge to be articulated: nobody can write down rules that distinguish every handwritten 4 from every 9, but a network can learn to do it from labeled examples.
For most of the field's history the two coexisted, and the balance shifted toward learning as data and computation grew. LeCun, Bottou, Bengio and Haffner put the case for the shift plainly in their 1998 paper on reading handwritten checks: "better pattern recognition systems can be built by relying more on automatic learning, and less on hand-designed heuristics" (LeCun et al., 1998, §I). The trade-off runs in both directions. A learned model generalizes from data it was given, inherits that data's gaps and biases, and is much harder to inspect than a rule set; the library's Anthropic Research collection holds dozens of reading notes on research into what large learned models actually compute, and why it is difficult to find out. Many working systems combine the two, for example a learned language model that calls a calculator, a search engine or a database query written in SQL.
A history of promises and corrections
The field's history reads well as a sequence of problems that looked close to solved, the discovery that they were not, and the new approaches those discoveries prompted.
Early optimism (1950s and 1960s). After the Second World War several people independently began work on intelligent machines; McCarthy (2007, §1) suggests Turing may have been the first, in a lecture of 1947, and notes that by the late 1950s most researchers worked by programming computers rather than building special machines. The first programs played games, proved theorems and manipulated symbols, and the early neural networks, Rosenblatt's perceptron among them, learned simple classifications. The introduction to neural networks tells the neural side of the story, including the perceptron's limits.
The first reckoning (1966 to the mid-1970s). Progress on hard problems was slower than forecast. Machine translation is the clearest case: a review by the US National Academy of Sciences in 1966 found that very large sums had been spent with little useful result (as summarized by Lighthill, 1973, §3). In Britain the Science Research Council commissioned the mathematician James Lighthill to survey the field. His report, dated July 1972 and published in 1973, concluded: "In no part of the field have the discoveries made so far produced the major impact that was then promised." It singled out one general cause: "failure to recognise the implications of the combinatorial explosion", the explosive growth in the number of possibilities a program must consider as a problem grows (Lighthill, 1973, §3). Reduced funding followed in Britain and elsewhere, and the period is now often called the first AI winter.
Expert systems and a second winter (1980s). Knowledge-based systems that encoded a narrow domain's rules found commercial use in the early 1980s, and a boom followed. The systems proved expensive to build and maintain and brittle outside their domains, and the collapse of the market in the late 1980s is often called a second winter (Russell and Norvig, 2020, §1.3; cited by attribution). The history of information retrieval on the course wiki shows a parallel course in a neighboring field, where hand-built indexing and exact-match Boolean search gave way to statistical ranking.
Statistical learning (1990s and 2000s). Learning from data became the main approach to many problems. LeCun et al. (1998) report a convolutional network that reads handwritten digits and was deployed commercially to read checks, and methods such as support vector machines, boosting and random forests, all covered in this collection, became standard tools. James, Witten, Hastie and Tibshirani (2021, Ch. 10) describe how in this period such methods often matched or beat neural networks with less tuning.
Deep learning (2010s). Larger labeled datasets and graphics processors made it practical to train much larger networks. In 2012 a convolutional network trained on the 1.2 million labeled images of the ImageNet challenge, entered in a variant form, won its classification task with a top-5 error rate of 15.3%, against 26.2% for the second-best entry (Krizhevsky, Sutskever and Hinton, 2012, abstract). The computer vision article explains what that network did and what the benchmark measured. Speech recognition and translation followed the same path.
Transformers and language models (2017 onward). The transformer architecture, introduced for machine translation, replaced recurrence with attention (Vaswani et al., 2017), and language models built from it and trained on very large text collections turned out to perform many tasks from instructions or a few examples written into their input (Brown et al., 2020). The language article and the generative AI article cover this period, and the library's Language Models from Scratch builds a small model of the same kind step by step.
The pattern across these episodes is not simply that optimism was wrong. Problems that resisted the methods of one period (reading handwriting, recognizing objects, translating) were later solved to a useful standard by different methods, usually ones that relied more on data and computation and less on hand-written knowledge.
Narrow and general
A system that does one task, such as reading postal codes or recommending films, is often called narrow AI; a hypothetical system that could learn and perform the full range of intellectual tasks a person can is called general AI, or artificial general intelligence. The distinction is useful but blurrier than the two words suggest. A single large language model now attempts translation, programming, summarization and question answering from the same weights, which is far broader than a postal-code reader, yet its competence across those tasks is uneven and difficult to predict.
McCarthy's comparison of program and human abilities, written in 2007, already described that unevenness: "Some abilities that children normally don't develop till they are teenagers may be in, and some abilities possessed by two year olds are still out" (McCarthy, 2007, §1). A 2022 Anthropic paper describes the same tension for large generative models, which combine predictable improvement in overall loss as they grow with specific capabilities that appear unpredictably; the library's note on Predictability and Surprise in Large Generative Models summarizes the argument and its evidence. For a reader of this collection, two cautious statements are safe. Performance on a benchmark is evidence about that benchmark, and the methods of honest evaluation apply to AI systems as much as to a regression model. And claims that a system "understands" or "reasons" are claims about mechanisms, which the library's Anthropic Research collection, especially its interpretability notes such as Tracing the thoughts of a large language model, treats as open research questions rather than settled facts.
Where data science fits
Almost every method in this collection is part of artificial intelligence in the broad sense. Logistic regression is the single neuron of a neural network; nearest neighbors, decision trees and ensembles are standard machine learning methods; clustering and dimension reduction are how a learning system finds structure without labels. The difference between a data science project and an AI product is often one of deployment: the model is placed inside software that acts on its predictions, at scale and often without a person checking each decision. That shift raises the stakes of everything the collection says about evaluation, and it is why this topic ends with an article on risks and ethics.
The articles in this topic
- Machine Learning: the three settings of supervised, unsupervised and reinforcement learning, what generalization means, and a reinforcement learning experiment on the 10-armed bandit.
- Computer Vision: images as arrays, convolution worked by hand on a handwritten digit, convolutional networks from LeNet to AlexNet, ImageNet, and an experiment that shifts test digits by one pixel.
- Natural Language Processing: tokens, bags of words, word vectors computed from the Brown corpus, attention and transformers, and how language systems are evaluated.
- Risks and Ethics in AI: bias and fairness worked on census income data, hallucination, privacy, misuse, environmental and labor costs, and regulation.
- Generative AI: generative and discriminative models, a small generative model of digits, autoencoders, adversarial networks, diffusion, autoregressive language models and scaling laws.
- Prompt Engineering: zero-shot and few-shot prompting, chain of thought, system prompts, structured outputs, sampling temperature, and how to evaluate a prompt.
References
- Brown, T. B., Mann, B., Ryder, N., et al. (2020). Language models are few-shot learners. In Advances in Neural Information Processing Systems 33. arXiv:2005.14165. record · 4 pages
- James, G., Witten, D., Hastie, T., and Tibshirani, R. (2021). An Introduction to Statistical Learning with Applications in R (2nd ed.). Springer. Ch. 10. record · 30 pages
- Krizhevsky, A., Sutskever, I., and Hinton, G. E. (2012). ImageNet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems 25 (pp. 1097–1105). record · 2 pages
- LeCun, Y., Bottou, L., Bengio, Y., and Haffner, P. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11), 2278–2324. record · 2 pages
- Lighthill, J. (1973). Artificial intelligence: A general survey. In Artificial Intelligence: A Paper Symposium. Science Research Council. Report dated July 1972; read in the online edition of the Chilton Computing archive. record · 2 pages
- McCarthy, J. (2007). What is artificial intelligence? Stanford University, revised November 12, 2007. http://jmc.stanford.edu/articles/whatisai/whatisai.pdf record
- McCarthy, J., Minsky, M. L., Rochester, N., and Shannon, C. E. (1955). A proposal for the Dartmouth summer research project on artificial intelligence. Dated August 31, 1955. http://jmc.stanford.edu/articles/dartmouth/dartmouth.pdf. Reprinted in AI Magazine, 27(4), 12–14 (2006). record
- Russell, S., and Norvig, P. (2020). Artificial Intelligence: A Modern Approach (4th ed.). Pearson. Cited by attribution; not read for this article. record · 5 pages
- Samuel, A. L. (1959). Some studies in machine learning using the game of checkers. IBM Journal of Research and Development, 3(3), 210–229. Cited by attribution; not read for this article. record
- Turing, A. M. (1950). Computing machinery and intelligence. Mind, 59(236), 433–460. record · 2 pages
- Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., and Polosukhin, I. (2017). Attention is all you need. In Advances in Neural Information Processing Systems 30. arXiv:1706.03762. record · 7 pages