In December 2011 Cory Doctorow gave the 28C3 keynote “The coming war on general computation”. His argument was that regulators who want “a computer that runs every program except the one that bothers me” can only get one thing, a computer with spyware built in. Eight months later he gave this sequel at Google, after trying it out at the Long Now Foundation and DEFCON. Its premise is that the freedom side wins the first war, and its question is what follows. The next fight, he says, is between the owners of computers and the people who use them.
It’s this week’s classic because the questions it asks in 2012 have since become product decisions: Secure Boot and user-enrolled keys, measured boot and remote attestation, MDM and TLS interception on work laptops, right-to-repair fights over tractor and phone firmware, implanted devices whose vendors go out of business. The examples are dated, but the framework still sorts current arguments well.
Recap: the war on general computation
Doctorow starts with a summary of the 28C3 talk. The world is made of computers now: “cars are computers we put our bodies into”, a 747 is “a badly secured Solaris box”, and hearing aids and pacemakers are computers inside us. So every future social problem will have a computer in the middle, and a regulator will ask for a computer that can’t do the one bad thing.
“We don’t have a theoretical model for a Turing-complete-minus-one.”
The closest thing to that is a computer that runs secret programs the user can’t see or stop, which is DRM. He says it fails twice. It doesn’t stop determined attackers, since one person breaks the scheme and publishes it. And it weakens security for everyone, because a facility that hides processes from the owner is a gift to malware. His example is the Sony BMG rootkit, which hid any file starting with $sys$, so malware authors renamed their files to match.
The PS3 root key leak illustrates the first failure. After DRM comes the next incentive, which is to make it illegal to explain how the DRM works or where it’s broken.
The TPM as a nub of certainty
The surprising part of the talk is Doctorow speaking warmly about Trusted Platform Modules. A TPM is a tamper-evident secure coprocessor. At boot it can check a signed hash of the bootloader, which checks the OS, which checks the programs above it, so you can know what’s running on your machine. He compares it to Descartes, who finds one small certainty and builds a whole worldview on it.
He then separates two ways to deploy one:
- In a locked TPM the vendor’s signing keys are fixed. That’s the iPhone model: you run what the vendor approves.
- In a certainty TPM the owner enrolls the keys at first boot (“only run OSes signed by Canonical, EFF, ACLU or WikiLeaks”), and the hardware then reports accurately on what’s running.
Android in 2012 gave you freedom without certainty: you could choose what to run, but you couldn’t verify that what you think is running is actually running. On a device with a microphone and a camera that you carry everywhere, he says that’s a real gap. Both sides want control and both sides need knowledge. They disagree about where control should sit.
In Q&A a Chromebook engineer says they’re about two months away from letting owners put their own keys on the device and boot their own signed images. That’s the certainty model shipping in a product.
When owners win: the property argument
Assume the freedom side wins, so owners know what’s running and decide who may sign their bootloader. Doctorow gives two arguments for that outcome. One is human rights: if your world is made of computers, devices built to override their owners are a human-rights problem. The other is property, “you bought it, you own it”, which he notes is especially effective in arguments with Silicon Valley engineers and in policy circles. The Blackstone slide is his picture of “property maximalism”: the owner sets every rule and anyone who disagrees can use a different computer.
He doesn’t think owners should have total control in every case. Self-driving cars are the wedge issue. You’d want the firmware open for inspection, but a state-signed locked bootloader won’t stop determined attackers, so the safety model has to assume other cars misbehave anyway (“conservative in their own conduct, liberal in the conduct they expect from others”). And once roads justify state-certified firmware, he asks, how do you argue that the information superhighway shouldn’t get the same treatment? Avionics and reactors are different because they already sit under no-notice inspection regimes.
Users of computers they don’t own
The talk’s main argument is that more and more of us will use computers we don’t own, and nobody has worked out rules for that. IT departments spend a lot of effort protecting owners from users. Doctorow, a former sysadmin and CIO, admits users are a real problem, and then gives two reasons users should still get rights.
The first is a business case. Owners need an override because programmers can’t foresee every situation, and employees need one for the same reason. He cites Hayek and Marx, who agree for opposite reasons that the people at the coalface know things the centre doesn’t. The PC got into the enterprise by the back door, against IT’s objections, and made companies a great deal of money. Taylorist scientific management, in his phrase, “is about as scientific as trepanation and Myers-Briggs tests.”
The second is human rights, and this part of the talk has aged best. Hugh Herr, head of biomechatronics at the MIT Media Lab, lost both legs below the knee and climbs on prostheses that cost as much as a house. Asked who could afford them, Herr says everyone, because people will take a 40-year mortgage on their legs. Doctorow’s point: “we are only temporarily able-bodied.” You should be able to change the firmware vendor on a cochlear implant when a competitor’s signal processing gets better. And users, as distinct from owners, need protection too: a child whose religious parents buy implants that filter out blasphemy, a poor person offered ad-supported implants that listen to conversations, a government that archives everything you hear.
Same tools as Syria and China
The existing market for lockware and spyware, he says, has four customers: repressive governments, large corporations, schools, and anxious parents. Their goals differ but they want the same technology. People were outraged when employers asked job candidates for Facebook passwords, yet barely objected to employers installing their own intermediate CA certificates on work devices to read TLS traffic, including your bank, your union forum, and your doctor’s portal. The work/personal boundary has largely disappeared, and he doesn’t think “it’s company property” covers the case of learning about a terminal diagnosis over Facebook on a work laptop.
He makes the case concrete with the 2004–05 Greek wiretapping scandal, where someone switched on the lawful-intercept features in Vodafone Greece’s switches and listened to the top of the government, and with Bavaria’s “state trojan”, which also let anyone else into the machines it infected. In Iran, customs could install a government root of trust on every imported computer. Tamper evidence then works against the owner, because the next policeman to look can see that it’s been changed.
A user override, purely hypothetical
Doctorow repeatedly calls this a thought experiment, not a proposal. Picture a bootloader that reports accurately on the OS, can suspend it to an encrypted disk image, and can boot a different OS from the network or a USB stick. In an internet café whose OS has a lawful-intercept back door, you’d press a key combination, the café’s OS would be parked and sealed, and yours would run. When you finished, your files would be erased and the bootloader would confirm it, so neither side could spy on the other. Owners are protected from users, users from owners, and users from each other.
Owner overrides have their own dangers. In 2006 the vendor of a 314-space robotic parking garage in Hoboken shut it down in a licence dispute, holding the cars hostage until the owners paid again. Doctorow extends that to implants. If your legs can walk themselves to the repo depot, then whoever buys your payday-loan debt can threaten to take your legs, and the leverage is “total and terrifying”. On the other side, there are good reasons to stop users overriding a smart meter, and banning owner override would mean some poor people don’t get credit they would otherwise get. He says plainly that he has no clean rule here.
“Agreeing to disagree on this one is not good enough. We need to start thinking now about the principles we’ll apply when the day comes.”
Q&A
The Q&A is worth watching too. On “if you’ve done nothing wrong”, he argues that private isn’t the same as secret and that people need privacy to make mistakes, which is how they learn; his example is a child who stops playing on noticing an adult watching. On why he publishes under Creative Commons, he gives three reasons, commercial, moral and artistic, the last being that making art you don’t expect to be copied isn’t making contemporary art. On illegal numbers, he says technology law only has to last for the half-life of the technology, not forever.
Key takeaways
- A general-purpose computer that can’t run one specific program doesn’t exist, and the nearest approximation is spyware that also weakens the machine’s security.
- Trusted hardware is neutral. The same TPM supports vendor lock-in when the vendor holds the keys and owner certainty when the owner enrolls them, so the policy argument is about who holds the keys.
- Being free to run any code without being able to verify what is running is a real security gap on devices that see, hear and travel with you.
- Winning owner control leads straight to a second conflict, because more and more people use computers they don’t own: work laptops, rentals, leased cars, implanted devices.
- Governments, corporations, schools and parents want the same surveillance and lockdown capabilities, so anything built for one is available to all four.
- Any user override has to leave the owner’s system intact and verifiable when control is handed back, protecting each party from the other.
- Remote-disable and repossession features become coercive when the device is a body part, and the 2006 Hoboken garage is an early real example.
- Doctorow ends without an answer and asks the audience to start working on one.
Source
- Talk: The Coming Civil War Over General-Purpose Computing (YouTube title: “Civil War General-Purpose Computing | Cory Doctorow | Talks at Google”)
- Speaker: Cory Doctorow
- Event: Talks at Google, Mountain View, summer 2012 (a sequel to his 28C3 keynote, December 2011)
- Length: 1 h 10 min (about 48 min talk plus Q&A)
- Archive: Talks at Google (uploaded 22 August 2012)
- Video: https://www.youtube.com/watch?v=gbYXBJOFgeI
- Prequel: 28C3, “The coming war on general computation”: https://www.youtube.com/watch?v=HUEvRyemKSg