A computer can follow instructions for a game of checkers. But could it play a better game tomorrow than it played today?
In 1959, IBM researcher Arthur Samuel published an account of a program built to investigate that question. Rather than writing down the correct move for every possible position, Samuel studied ways for a program to improve from experience. Checkers gave him a small, familiar world in which he could watch the idea work. 1
The story reaches well beyond one board game. Decades later, researchers used games such as Go to study much more powerful systems that learn through play. The machines and methods changed enormously, but the appeal of the game remained: it gives a computer a task, rules and an outcome that can be checked.
A game lets researchers test learning in public. It also reminds us that getting better inside a well-defined game is different from navigating the messier world outside it.
Remembering a board is one way to learn
Checkers has a manageable set of rules, but it still offers many possible positions. A player has to judge whether a move improves the board now and what might happen several moves later.
Samuel investigated a method he called rote learning. The program could retain information about positions it had already encountered and reuse it later. A remembered position gave the program a way to take advantage of an earlier calculation instead of beginning from scratch each time. 1
That helps when the same position returns. It is less helpful when the board is unfamiliar. Samuel also explored a second approach, learning by generalization: adjusting how the program evaluated features of a position so that experience could inform its judgment about other positions too. 1
The program learned through several forms of experience, including play against itself, games involving people and recorded games. Samuel reported that a program could improve beyond the level of play of its author. That was a striking research result, though it did not mean the program had solved every aspect of checkers or possessed a general understanding of the world. 1
Go gave a later generation a bigger board
Go is a board game with simple rules and an enormous number of possible positions. In 2016, researchers described AlphaGo, a system that combined learned neural networks—software trained to recognize patterns—with a search through possible moves. It learned from human games and from further play against itself. 2
A later system, AlphaGo Zero, took a different approach. Its researchers reported training it from the rules through self-play rather than starting with human game records. It was not simply Samuel's checkers program running faster. It used different methods and vastly different computing resources. 3
The comparison is still illuminating. Both projects used games as a place to ask how a system can improve its decisions from experience. The board supplies an agreed set of legal moves and a way to determine who won. Researchers can change the program, play many games and compare results.
The world does not always announce the winner
A game is a useful laboratory partly because so much has been settled in advance. The players share rules. The board tells them which moves are legal. Eventually, the result can be scored.
Many real problems are not so tidy. A medical system, for example, must deal with missing information, changing conditions and consequences that may take years to understand. A system can improve on a test while missing something important about the situation where people use it.
That does not make games trivial. It makes them carefully chosen experiments. Samuel's checkers program helped demonstrate that a machine could use experience to change how it played. Later systems pushed the idea in new directions. Each board offered researchers a controlled place to ask a much larger question: what does it take for a computer to get better at something?