14 February 2021

How the United States Lost to Hackers

By Nicole Perlroth

If ever there was a sign the United States was losing control of information warfare, of its own warriors, it was the moment one of its own, a young American contractor, saw first lady Michelle Obama’s emails pop up on his screen.

For months, David Evenden, a former National Security Agency analyst, questioned what he was doing in Abu Dhabi. He, like two dozen other N.S.A. analysts and contractors, had been lured to the United Arab Emirates by a boutique Beltway contractor with offers to double, even quadruple, their salaries and promises of a tax-free lifestyle in the Gulf’s luxury playground. The work would be the same as it had been at the agency, they were told, just on behalf of a close ally. It was all a natural extension of America’s War on Terror.

Mr. Evenden started tracking terror cells in the Gulf. This was 2014, ISIS had just laid siege to Mosul and Tikrit and Mr. Evenden tracked its members as they switched out burner phones and messaging apps. The images they traded back and forth could be brutal, but this was his calling, Mr. Evenden told himself. A theology major, he’d set out to be a chaplain. He was a long way from that, but what better way to prove your faith, he thought, than hunting those who sought to murder good Christians. Soon, though, he was assigned to a new project: proving the Emiratis’ neighbor, Qatar, was funding the Muslim Brotherhood. The only way to do that, Mr. Evenden told his bosses, would be to hack Qatar.

“Go for it,” they told him. No matter that Qatar was also an American ally or that, once inside its networks, his bosses showed no interest in ever getting out. Before long his team at the contractor, CyberPoint, was hacking Emirati enemies, real and perceived, all over the world: Soccer officials at FIFA, the monarchy’s Twitter critics, and especially Qatari royals. They wanted to know where they were flying, who they were meeting, what they were saying. This too was part of the mission, Mr. Evenden was told; it had all been cleared up high. In the War on Terror and the cyber arms market, you could rationalize just about anything.

All the rationalizations were stripped away the day emails from the first lady of the United States popped up on his screen. In late 2015, Michelle Obama’s team was putting the finishing touches on a trip to the Middle East. Qatar’s Sheikha Moza bint Nasser had invited Mrs. Obama to speak at her annual education summit in Doha, where the first lady would promote her “Let Girls Learn” initiative. Mrs. Obama and her team were in constant communication with Sheika Moza. And every last email between the first lady, her royal highness, and their staff — every personal reflection, reservation, itinerary change and security detail — was beaming back to former N.S.A. analysts’ computers in Abu Dhabi. “That was the moment I said, ‘We shouldn’t be doing this,’ he told me. “We should not be targeting these people.”

Mr. Evenden and his family were soon on a flight home. He and the few colleagues who joined him tipped off the F.B.I. (The agency does not comment on investigations, but interviews suggest its review of CyberPoint is ongoing.) To pre-empt any fallout, some employees came clean to Reuters. The hack of Sheika Moza’s emails with Mrs. Obama has never been reported.

It wasn’t long after Mr. Evenden settled back in the states that he started fielding calls and LinkedIn messages from his old buddies at the N.S.A., still in the service, who had gotten a “really cool job offer” from Abu Dhabi and wanted his advice. By 2020, the calls had become a drumbeat. “Don’t go,” he pleaded. “This is not the work you think you will be doing.”

You might think you’re a patriot now, he wanted to warn them, but one day soon you too could wake up and find you’re just another mercenary in a cyber arms race gone horribly wrong.

Three decades ago, the United States spawned, then cornered, the market for hackers, their tradecraft, and their tools. But over the past decade, its lead has been slipping, and those same hacks have come boomeranging back on us.

Yet no one in government has seriously paused to recalibrate the strategy. Not with Michelle Obama’s emails caught in an American contractor’s dragnet in 2015. And not today, with Russian hackers inside our government networks. We went from occasional wake-up calls to one continuous, blaring alarm — and got better and better at ignoring it all.

Months after Mr. Evenden returned home, in 2016, the N.S.A.’s own hacking tools were hacked, by a still unknown assailant. Those tools were picked up first by North Korea, then Russia, in the most destructive cyberattack in history.

Over the next three years, Iran emerged from a digital backwater into one of the most prolific cyber armies in the world. China, after a brief pause, is back to pillaging America’s intellectual property. And, we are now unwinding a Russian attack on our software supply chain that compromised the State Department, the Justice Department, the Treasury, the Centers for Disease Control, the Department of Energy and its nuclear labs and the Department of Homeland Security, the very agency charged with keeping Americans safe.

We know this not because of some heroic N.S.A. hack, or intelligence feat, but because the government was tipped off by a security company, FireEye, after it discovered the same Russian hackers in its own systems.

The hubris of American exceptionalism — a myth of global superiority laid bare in America’s pandemic death toll — is what got us here. We thought we could outsmart our enemies. More hacking, more offense, not better defense, was our answer to an increasingly virtual world order, even as we made ourselves more vulnerable, hooking up water treatment facilities, railways, thermostats and insulin pumps to the web, at a rate of 127 new devices per second.

At the N.S.A., whose dual mission is gathering intelligence around the world and defending American secrets, offense eclipsed defense long ago. For every hundred cyberwarriors working offense — searching and stockpiling holes in technology to exploit for espionage or battlefield preparations — there was often only one lonely analyst playing defense to close them shut.


America remains the world’s most advanced cyber superpower, but the hard truth, the one intelligence officials do not want to discuss, is that it is also its most targeted and vulnerable. Few things in the cybersecurity industry have a worse reputation than alarmism. There is even an acronym for it: “FUD,” short for “fear, uncertainty, and doubt.”

When Leon Panetta, then secretary of defense, warned of a coming “Cyber Pearl Harbor” in 2012, he was dismissed as stoking FUD. The Cyber Pearl Harbor analogy is, indeed, flawed: The U.S. government did not see the Japanese bombers coming, whereas it has seen the digital equivalent coming for decades.

And the potential for a calamitous attack — a deadly explosion at a chemical plant set in motion by vulnerable software, for example — is a distraction from the predicament we are already in. Everything worth taking has already been intercepted: Our personal data, intellectual property, voter rolls, medical records, even our own cyberweaponry.

At this very moment, we are getting hacked from so many sides that it has become virtually impossible to keep track, let alone inform the average American reader who is trying to grasp a largely invisible threat that lives in code, written in language that most of us will never fully understand.

This threat often feels too distant to combat, but the solutions have been there for decades: Individuals just decided that access and convenience, and in governments’ case, the opportunities for espionage, were worth leaving windows open, when we would have all been better off slamming them shut.

“The N.S.A.’s fatal flaw is that it came to believe it was smarter than everyone else,” Peter Neumann, a computer scientist and cybersecurity sage, told me. “In the race to exploit everything and anything we could, we painted ourselves into a dead end where there is no way out.”

There’s a reason we believed the fallacy that offense could keep us safe: The offense was a bloody masterpiece.

Starting in 2007, the United States, with Israel, pulled off an attack on Iran’s Natanz nuclear facility that destroyed roughly a fifth of Iran’s centrifuges. That attack, known as Stuxnet, spread using seven holes, known as “zero days,” in Microsoft and Siemens industrial software. (Only one had been previously disclosed, but never patched). Short term, Stuxnet was a resounding success. It set Iran’s nuclear ambitions back years and kept the Israelis from bombing Natanz and triggering World War III. In the long term, it showed allies and adversaries what they were missing and changed the digital world order.

In the decade that followed, an arms race was born.

N.S.A. analysts left the agency to start cyber arms factories, like Vulnerability Research Labs, in Virginia, which sold click-and-shoot tools to American agencies and our closest Five Eyes English-speaking allies. One contractor, Immunity Inc., founded by a former N.S.A. analyst, embarked on a slippier slope. First, employees say, Immunity trained consultants like Booz Allen, then defense contractor Raytheon, then the Dutch and the Norwegian governments. But soon the Turkish army came knocking.

Companies like CyberPoint took it further, stationing themselves overseas, sharing the tools and tradecraft the U.A.E. would eventually turn on its own people. In Europe, purveyors of the Pentagon’s spyware, like Hacking Team, started trading those same tools to Russia, then Sudan, which used them to ruthless effect.

As the market expanded outside the N.S.A.’s direct control, the agency’s focus stayed on offense. The N.S.A. knew the same vulnerabilities it was finding and exploiting elsewhere would, one day, blow back on Americans. Its answer to this dilemma was to boil American exceptionalism down to an acronym — NOBUS — which stands for “Nobody But Us.” If the agency found a vulnerability it believed only it could exploit, it hoarded it.

This strategy was part of what Gen. Paul Nakasone, the current N.S.A. director — and George Washington and the Chinese strategist Sun Tzu before him — call “active defense.”

In modern warfare, “active defense” amounts to hacking enemy networks. It’s mutually assured destruction for the digital age: We hacked into Russia’s troll networks and its grid as a show of force; Iran’s nuclear facilities, to take out its centrifuges; and Huawei’s source code, to penetrate its customers in Iran, Syria and North Korea, for espionage and to set up an early warning system for the N.S.A., in theory, to head off attacks before they hit.


When we discovered openings in the systems that govern the digital universe, we didn’t automatically turn them over to manufacturers for patching. We kept them vulnerable in the event the F.B.I. needed to access a terrorist’s iPhone or Cyber Command had reason to drop a cyberweapon on Iran’s grid one day.

There were big payoffs, to be sure, many the public will never know, but all one needs to do is look at the attacks of the past five years to see that “active defense” and NOBUS aren’t working that well.

In a leaked N.S.A. memo in 2012, an analyst warned as much, “Hacking routers has been good business for us and our Five Eyes partners for some time, but it is becoming apparent that other nation states are honing their skillz and joining the scene.”

Only when the N.S.A.’s tools were hacked in 2017, then used against us, could we see how broken the trade-off between offense and defense had become. The agency had held onto a critical vulnerability in Microsoft for more than five years, turning it over to Microsoft only after the N.S.A. was hacked.

By then it was too late. Businesses, schools and hospitals had yet to patch the hole when North Korea used it to attack one month later, or even two months later, when Russia baked it into a cyberattack that decimated vaccine supplies at Merck, cost FedEx $400 million and prevented doctors from accessing patient records. All in, that incident costs victims an estimated $10 billion in damages.

In the wake of those strikes, in 2017, Gen. Michael Hayden, the former director of the N.S.A., and one of its most vocal supporters, was unusually speechless. “I cannot defend an agency having powerful tools if it cannot protect the tools and keep them in its own hands,” he said.
The Typewriters Were Listening

To understand how we got here, facing one escalating attack after another, and how we might possibly claw our way out, it’s useful to look back at the Russian attack that put us on this offensive course.

That year, 1983, workers at the American embassy in Moscow came to believe that everything they said and did was being captured by the Soviets. They suspected a mole, and had it not been for a tip from the French, who discovered a bug in their teleprinters, they might have never discovered the mole was in their machines.

In 1984, President Ronald Reagan personally approved a classified project, code-named Gunman, to find and eradicate any Soviet bugs in embassy equipment. It took 100 days just to get every last piece of equipment back to Fort Meade and nearly 100 more days to uncover the most sophisticated exploit the agency had ever seen.

Sitting in the back of an embassy typewriter was a tiny magnetometer, a device that measured the slightest disturbance in the earth’s magnetic field. It had been recording the mechanical energy from every last typewritten stroke and transmitting the results via radio to a nearby Soviet listening unit, hidden in the embassy’s chimney. By the time Gunman was complete, and more implants were discovered, it was clear that the Soviets had been siphoning American secrets from our typewriters for eight years.

“That was our big wake up call,” James R. Gosler, the godfather of American cyberwar, told me. “Or we’d still be using those damn typewriters.”

If any single technologist can be credited with spurring the United States to scramble, catch up, and take the lead as the world’s most advanced digital superpower, it is Mr. Gosler. When I asked nearly every one of the men who guided the N.S.A. and C.I.A. through the turn of the century to name the father of American cyber offense. None hesitated: “Jim Gosler.”

In Mr. Gosler’s lexicon, there’s BG — Before Gunman — and AG. BG, “Americans were fundamentally clueless,” he told me. “We were in la-la land.”

AG, we were hacking into anything with a digital pulse.

Over his long career at Sandia national labs, the N.S.A., and later the C.I.A., Mr. Gosler made it his personal mission to draw the government’s attention to vulnerabilities in the microchips, code and software seeping into our lives.

He does not discuss any of the classified programs he was privy to, but under his tenure, he helped create a taxonomy of adversaries that could exploit these vulnerabilities and led teams of American analysts and spies to make sure the United States was on top.

But every calorie the United States expended on offense came at the cost of defense. And over the decades, this trade-off gnawed at Mr. Gosler. Finding Gunman in those typewriters had been a feat. Finding its equivalent in our fighter jets or even the average high-end car, which now has more than 100 million lines of code? Good luck.

This, essentially, is the predicament the United States now faces as it hunts down every last vector and backdoor used in the recent SolarWinds attack, so dubbed because Russians used SolarWinds, a Texas company that sells network software to government agencies, grid operators and more than 400 of the Fortune 500, as a conduit.

Occasionally we respond to attacks with indictments, sanctions or cyberattacks of our own. President Biden added $10 billion in cybersecurity funds to his Covid-19 recovery proposal and said Thursday that the United States was “launching an urgent initiative” on cybersecurity, to improve America’s “readiness and resilience in cyberspace.”

But finding every Russian back door could take months, years even. And climbing out of our current mess will entail a grueling choice to stop leaving ourselves vulnerable.

For individuals, this means making life less convenient. It’s not ignoring password prompts and software updates, turning on two-factor authentication, not clicking malicious links. For businesses, it requires testing code as engineers write it, not after it has made its way into consumer hands. It requires adding moats around the crown jewels: using hand-marked paper ballots, removing the controls that govern our nuclear plants, medical equipment and air traffic from anything else.

For the government, perhaps, an easy place to start is setting clear rules that prevent the N.S.A.’s own, like Mr. Evenden’s former employer, from doing the dirty work for other governments where the rules that govern our own spycraft do not apply. And it’s long past time to shut all the doors and windows that should never have been left open.

Jim Gosler worked for decades to keep Americans, and our secrets, safe, to make sure we never had to know just how close to a catastrophic cyberattack we could come. Now, as the country reckons with scenarios he long feared, he realizes the way forward is understanding just how unsafe we already are.

“Gunman didn’t impact the average American where they would feel it, but SolarWinds is getting pretty darn close,” Mr. Gosler told me recently. “It’s so pervasive. It’s one step from SolarWinds into the electrical grid. If the average American can’t feel that? What is it going to take?”

Nicole Perlroth, a cybersecurity reporter at The Times, is the author of the forthcoming book “This Is How They Tell Me the World Ends,” from which this article is adapted.

No comments: