When Hackers Teamed Up With the Pentagon to Defend The Internet

June 17, 2019

From “Cult of the Dead Cow: How the Original Hacking Supergroup Might Just Save the World” by Joseph Menn

Peiter Zatko, known to even close friends as Mudge, was not the most engaged executive at @stake, even though he was the lead creator of the pioneer hacker consulting group. The most famous member of hacking organization Cult of the Dead Cow was elsewhere much of the time, fighting his own demons and, after 9/11, America’s demons as well. What he saw made him very afraid. Mudge knew as much as anyone about the basic failings of tech security and about their root causes. The internet’s inventors built it on trust and it got loose in its test version, before Vint Cerf and his team could come up with reliable security. It still ran that way.

All software has bugs, some of which can be exploited. Layering software on software makes it less secure. The software vendors had all escaped legal liability for poor craftsmanship and had little incentive to devote significant resources to making their products safer. (This hard line on liability has only begun to fray in 2018 in extreme cases, as with deaths blamed on automated vehicles’ programming.) Regulation ranged from nonexistent in most commercial markets to negligible in industries such as financial services, health care, and power distribution. All of which meant everything was unsafe and would only get less safe as the economy grew more dependent on technology.

This was classic market failure, compounded by political failure. One could debate the largest causes of the political failure, but they included the capture of the regulators by industries that did not want to be regulated, the dominant pursuit of short-term business gains by short-term business executives, and the failure to distinguish when private companies should be responsible for their own defense and when the federal government needed to step in. That last was nontrivial, since the same techniques could be employed by criminal hackers, fending off whom would generally be considered a corporate responsibility, and nation-state spies, who would generally be considered a Homeland Security or FBI responsibility, with backup from the Department of Defense. Even if those lines were clear, what do you do about criminals who work for spies, or spies who moonlight as criminals? Congress’s inaction loomed large. But without blood on the streets, Mudge held little hope of that changing anytime soon.

In 2003, as largely Russian organized crime groups took the leading role in spreading computer viruses for spam and extortion, Mudge saw that the big picture was about to look a lot worse. He figured the best way to help was to go to the place that had the best understanding of the problem, the most power to deal with it, and the greatest responsibility: the federal intelligence agencies. Given his sketchy associations and general antiestablishment attitude, it would have been draining to apply directly at the CIA or NSA. But Mudge could at least start where he was a known quantity, and where he had geographical and employment buffers from the people wearing braids and stars on their uniforms.

A year after Mudge’s top government sponsor, Richard Clarke, resigned from the Bush White House, Mudge rejoined BBN Technologies. Starting in 2004, he worked at BBN on research and development for US intelligence agencies, and he trained people who would become the core of the NSA’s elite hacking unit, Tailored Access Operations. Over the next six years, he worked on a lot of things he can’t talk about. “I think domestic lives have been saved as my ideas went operational,” Mudge said. He told me that lives in the Middle East were also saved because his tools were used instead of bombs.

In 2010, the new head of the Defense Advanced Research Projects Agency asked Mudge to come in-house and lead the agency’s cybersecurity efforts. Mudge had thought about DARPA before, but he hadn’t been enthusiastic about the agency’s prior leadership. The new boss, Regina Dugan, he liked. And DARPA, founded in 1958 in response to Russia’s stunning Sputnik satellite, had the coolest mission in government: “the creation and suppression of strategic surprise.” Like many positions inside DARPA, the post was for a fixed three-year term, during which he would award grants for offensive and defensive breakthroughs in security. But the opportunity was incredible. This agency had steered the creation of the ARPANET, which became the modern internet. “I obviously wanted to make sure the things I depend on, that my family and friends depend on, are secure,” Mudge said. “I also owe a lot to my country. A lot of countries would not have allowed me to influence the intelligence community and the Department of Defense, hopefully in ways that have them make less stupid mistakes.”

Mudge’s personal slogan had long been “Make a dent in the universe.” Now he called in a dozen of the smartest hackers he knew to help figure out how. He told them to be ready to discuss where the security industry was failing, what they as researchers were angriest about, and what DARPA could do to help. They convened in a bland Arlington, Virginia, building that housed the massive intelligence contractor Booz Allen Hamilton, the company that would employ Edward Snowden. Mudge’s call brought out “a bunch of misfits,” said Dug Song, who was among them. The group included @stake veterans Dave Aitel, now running zero-day seller Immunity Inc., and Dino Dai Zovi, a former federal labs researcher and chief scientist at government zero-day supplier Endgame. Also there was sometime intelligence contractor H. D. Moore, who had created Metasploit, a penetration-testing tool that used vulnerabilities as soon as they were disclosed, often within a day. Ninja Strike Force stalwart and intelligence contractor Val Smith came too.

Mudge convened the meeting by telling them that his DARPA slot had given the entire hacking community, at long last, “a seat at the table.” Now, he said, “let’s not waste this opportunity.” As they brainstormed priorities, Song asked about something different: a change in process. DARPA funded the big guys—defense contractors, other major corporations, and some university departments. Those operations knew how to navigate the paperwork, come up with slick pitches, and leverage their previous work. This left out talented small teams and individuals who had great insights from being hands-on hackers and no idea where to go from there. The son of a liquor-store owner, Song had used a small-business grant to start Arbor Networks. He said DARPA should go small as well, and Smith agreed.

Mudge had spent enough time around government to realize they were right, and he convinced Dugan. “The process itself was an impediment,” Dugan said. Mudge announced the Cyber Fast Track not long after, the first program at DARPA aimed at giving small amounts to small teams, instead of large amounts to large ones. Mudge funded nearly two hundred proposals, all of which let the researchers keep their own intellectual property. Among the recipients was Moxie Marlinspike, whose invention Signal would come years later, and Charlie Miller, who studied flaws in near-field communication as those protocols were getting embedded in more smartphones.

At the hacker convention Def Con in 2011, Miller was presenting a near-field talk and bumped into Mudge, who was also speaking. Miller told Mudge some of the things he was interested in and asked if DARPA would buy him a car he could hack. “Submit and find out,” Mudge said, so Miller did. He got the car and hacked away. Building on that work later, Miller hacked a moving jeep being driven by a Wired reporter, prompting a mass recall and drawing global attention to the safety issues of computerized vehicles. The initial equipment and the money was one thing. But DARPA’s backing became even more important when a car company, upset at Miller’s revelations, threatened to sue. Mudge warned them that if they did, the Pentagon would join the suit on Miller’s side, with a significant number of well-trained lawyers.

“Those grants also provided a certain amount of legitimacy to the research that really helped when people were having objections,” Miller said. “There are lots of research projects you see around now that would have never existed without those CFT grants, including the car hacking we did.” Everyone at the Pentagon wanted to get the papers explaining the research. But before they could get the briefing books, they had to sit through a demonstration by the hackers themselves, so they really understood them. In the years that followed, other areas at the Pentagon began mimicking the fast track Mudge developed.

Mudge did much more than streamline the way the federal government acquired good ideas. He also tackled a fundamental problem with the way the government, and everyone else, evaluated security. For decades, no one had come up with a reasonable way to estimate the worth of security products, which draw attention mainly when they fail. Likewise, DARPA couldn’t figure out a logical basis for determining what to fund. “We are not going to approve a single new project until we do the deep strategic work,” Dugan said. She insisted that Mudge and his boss, long-serving DARPA software chief Dan Kaufman, find a new way of looking at the issue.

Mudge and Kaufman came up with what they called the Cyber Analytic Framework. The major concept: as predictable complexity increases, the defenders’ job gets harder more rapidly than the attacker’s job does. To illustrate the problem, Mudge used the common language of Washington, a slide deck. The most eye-popping chart showed that the average advanced defense software had bloated to contain 10 million lines of code over the past decade. The average number of lines in malicious software, meanwhile, had held steady at 125.

Since every thousand lines of code has one to five bugs in it, that meant big security products were making the situation worse. DARPA needed to seek simple and elegant approaches instead. “It was a clear articulation of trend lines,” Dugan said. Mudge began asking defensive grant applicants whether their approaches were tactical or strategic, how their project would increase or decrease the overall attack surface being defended, and how they would beat it themselves.

The Framework approach became the basis for DoD spending beyond DARPA, and it got DARPA some money that otherwise would have gone to Cyber Command, one of several things Mudge worked on that annoyed Cyber Command head and NSA director Keith Alexander. Mudge didn’t mind that at all. Alexander had presided over a massive expansion of global and US surveillance, as well as a culture that produced several whistle-blowers and leakers while allowing employees to be hacked.

Mudge loved betting on promising ideas, but he also considered it his duty to strangle bad ones in the crib. While still an outside contractor, he decried a product that automated some “active defense,” the industry term for measures that range from blocking suspicious connections to disabling the computers used by an attacker. Though hacking back tempts targets that feel powerless relying on the government, most intelligence professionals think it is a bad idea that would lead to chaos and perhaps an unintended war. Automating that “is a terrible idea, because then an outsider can make you do things,” Mudge said.

Mudge also expended considerable energy arguing against demands for backdoors in encryption. Intelligence and military officials said that back doors worked well in their offices—that access was logged and controlled and that abuse was rare. But those were closed systems, where the people in charge could completely govern the environment. Out in the regular world, configurations get looser and privileged access leaks.

Mudge didn’t stop telling the truth just because he was at the seat of great power. It probably helped that his position would end after just three years, so officials expected less sucking up. Mudge briefed the Joint Chiefs and the secretary of defense, helping them understand when one of the armed forces or a contractor was claiming an improbable capability in a turf or budget fight. “The Joint Chiefs and the Pentagon would call me in because I didn’t have a horse running, and I was able to explain to them ground truth,” he said.

Mudge remained iconoclastic. Amid widespread outcry over the constant breaches of American defense vendors by other nations, Mudge observed shortly after leaving DARPA that contractors had a perverse incentive to allow their weapons systems to be stolen. Once that happened, Mudge mused at the Black Hat conference, they could ask the Pentagon to pay for a new and improved version of their system that was not yet in enemy hands. “Game theory is a bitch,” he said.

Yet Mudge managed to play the inside game well. DARPA always sent off its creations to new homes within the Pentagon or intelligence establishment where they would best develop. With Alexander and others predisposed to dislike much of what Mudge had handled, he sometimes engaged in subterfuge, handing off to a midlevel operative who could remove evidence of a project’s heritage. At one briefing with the deputy secretary of defense, Alexander explained that he had five “silver bullets” that he could deploy in cyberoperations. “Three of those are mine,” Mudge thought with satisfaction.

Mudge got the Pentagon to stop seeing hackers as the natural enemy. In fact, Mudge showed that people who grew up knowing exactly where the line was were habitually more careful about not crossing it than people constantly protected by their uniforms, bureaucracy, and lawyers. During one discussion at a large agency that was witnessed by Kaufman, an employee asked Mudge if the agency could just hack into a system in order to get information Mudge was deducing. “Absolutely, you could do that,” Mudge told him. “But just suggesting that is illegal, and it’s wrong.” Even within DARPA, Mudge provided a moral compass.