When Go was announced in 2009, many programmers read “concurrent language” as “parallel language”, ran their programs on more cores, watched them get slower and decided Go was broken. Rob Pike gave this 31-minute talk at Heroku’s Waza conference in January 2012 to correct the mistake. He thought the problem was the programmers’ mental model, not the language. The talk is a classic because its central distinction still gets confused in code reviews, and because the explanation is a cartoon about gophers burning manuals that’s hard to forget.

Two different ideas

Slide: concurrency vs. parallelism; not the same, but related; one is about structure, one is about execution

Pike’s definitions are short:

“Concurrency is about dealing with a lot of things at once and parallelism is about doing a lot of things at once.”

Concurrency is the composition of independently executing processes. He means “process” in the most abstract sense, covering threads, coroutines and OS processes. Parallelism is the simultaneous execution of multiple things, which may or may not be related. One is a property of how the program is structured, the other of how it runs. His example of concurrency without parallelism is an OS kernel managing mouse, keyboard, display and network drivers as independent units on a single core: only one of them runs at a time, and the design is still concurrent. A vector dot product split across SIMD lanes is the reverse case, parallel with no interesting concurrent structure.

Independent pieces still have to be coordinated, so concurrency needs communication. Here Pike sends the audience to Tony Hoare’s 1978 paper Communicating Sequential Processes, “truly one of the greatest papers in computer science”. If they take only one thing from the talk, he says, it should be to go home and read it. (We covered it in our weekly papers series.)

Gophers, carts and an incinerator

Slide: more gophers and more carts, with the note that the gophers now need to synchronize

The problem: a pile of obsolete C++98 manuals has to be moved into an incinerator. One gopher with a cart is slow. A second gopher without a cart does nothing, because a process needs its resources. With two carts it’s faster, but the gophers now get in each other’s way at the pile and the incinerator, so they have to coordinate (Pike’s “little Tony Hoare messages”). Doubling everything (two piles, two incinerators, two gophers) doubles throughput. Pike’s point is that this is better described as the concurrent composition of two gopher procedures than as “parallelism”, because the design is just as correct if only one gopher is allowed to move at a time, the equivalent of a single core.

Slide: observation: we improved performance by adding a concurrent procedure to the existing design

Then he changes the design. One gopher loads the cart, one drives it, one loads the incinerator, and a fourth returns the empty carts. That’s more gophers and more total work, yet it can run faster, because each piece is simpler and in principle all four can be busy at once. The slide calls this “a deeper insight than mere parallelism”: adding things to a design can make it more efficient, and the parallelism comes from expressing the problem better.

Slide: another design, with two gopher procedures plus a staging pile

He keeps going. He adds a staging pile so that two different gopher procedures hand books off, then replicates that design, then combines it with the four-gopher pipeline, and ends with sixteen gophers. Each design can be scaled along a different axis, and none of them is harder to get right than the first. “If we got the concurrency right, the parallelism is actually a free variable.” Then he relabels the picture. Books become web content, gophers become CPUs, carts become the network or marshalling code, and the incinerator becomes the browser or proxy. What you have is a web-serving architecture with buffers, forwarding agents and replicas, which people rarely draw this way.

Go’s three primitives

Slide: goroutines are not threads; they’re much cheaper and multiplexed onto OS threads as required

The second half is just enough Go to make the ideas concrete.

  • A goroutine is started with go f(x, y), which Pike compares to & in the shell. Goroutines share an address space like threads but are much cheaper and are multiplexed onto OS threads by the runtime, so a goroutine blocked in a system call doesn’t hold up the others. He mentions debugging a production Go program with 1.3 million goroutines, about 10,000 of them active.
  • A channel is a typed pipe. One goroutine sleeps and sends time.Now() on a timer channel, and the receiver blocks until the value arrives.
  • select is a switch statement for communications. It runs whichever channel operation is ready, picks at random when several are, and runs the default clause (if there is one) when none are. He points to Dijkstra’s guarded commands as the precursor.

Add closures, and you can start a background daemon with go func() { for v := range input { output <- v } }().

A load balancer in a page of code

Slide: a simple load balancer, with a runner that starts NumWorkers worker goroutines reading one input channel and writing one output channel

The simple version starts NumWorkers goroutines reading from a shared input channel and writing to a shared output channel, plus a goroutine that generates work, then collects results as they finish. It’s correct on one processor or a thousand, and there are no locks or mutexes anywhere. NumWorkers “could be huge” and the program would still work.

Slide: the balancer function: a two-way select over incoming requests and worker completion notifications

The realistic version gives each request a closure to run and a reply channel. Channels are first-class values in Go, and Pike compares passing one to passing a file descriptor. That means the worker replies straight to the requester, and the balancer drops out of the conversation once they’re connected. The balancer is a loop around a two-way select. On a new request it pops the least-loaded worker from a heap, sends it the request, increments its pending count and pushes it back. On a completion it decrements the count and fixes the worker’s place in the heap. That’s about four lines of executable code per case. Pike says the runnable version keeps every worker at uniform load “modulo quantization”.

The first answer wins

Slide: query a replicated database: one goroutine per replica, all writing to a buffered channel, return the first result

The last example fits on one slide. To query replicated database shards and take the fastest reply, make a channel buffered to the number of replicas, start one goroutine per connection that sends c.DoQuery(query) into it, and return the first value received. A replica that’s down or slow doesn’t matter. The buffer lets the losing goroutines finish and exit instead of blocking forever. The one thing missing is a clean teardown that tells the slower servers to stop, which he says takes more code but “wouldn’t fit on the slide”.

Slide: conclusion: concurrency is powerful; concurrency is not parallelism; concurrency enables parallelism; concurrency makes parallelism (and scaling and everything else) easy

His reading list includes Bob Harper’s “Parallelism is not concurrency”, Doug McIlroy’s concurrent power-series paper, and Sawzall, a language Pike designed at Google that is highly parallel with no concurrency at all. By the end of the talk, he says, that combination should make sense.

Key takeaways

  1. Concurrency describes how a program is structured, as independently executing pieces that communicate. Parallelism describes how it runs, with multiple things executing at the same instant.
  2. A concurrent design is still correct on a single core. Parallelism becomes a scaling choice you make afterwards, not a requirement of correctness.
  3. Breaking a problem into more, simpler concurrent pieces can make it faster even though it adds total work, because more of the pieces can run at once.
  4. Communication is what makes independent pieces work together. Hoare’s 1978 CSP paper is the source, and Pike says it’s the one paper to read.
  5. Passing channels as values lets components connect directly to each other and then drop out of the path, as the balancer does once a worker and requester are connected.
  6. With goroutines, channels and select, a load balancer or a “first replica to answer wins” query needs no explicit locking.
  7. Using more cores won’t speed up a program that was designed sequentially. The speedup comes from restructuring the problem.

Source