“Our ability to collect stuff far outstrips our ability to understand what we collect.”
—Eric Haseltine, former director of research, National Security Agency

Once upon a time the National Security Agency (NSA) was so secret that just knowing about it was a breach of security. When President Truman created the NSA back in 1952—not long after he’d formed that dysfunctional phoenix the Central Intelligence Agency (CIA) out of the ashes of the old wartime Office of Strategic Services (OSS)—he put them in charge of the nation’s communications intelligence (COMINT). That meant, in essence, they would be the ones that tapped the phones, secretly read the mail, and generally stuck their eyes and ears into what people around the world were writing and talking to each other about. Given how critical codebreaking and surveillance had been to Allied success in World War II, it was no surprise that the very existence of the NSA, much less the particulars of their work, was a tightly-held secret for so many years. The joke went that their initials actually stood for No Such Agency.

Shrouded for so long in such secrecy, and lavished with a budget that would make other intelligence agencies weep with jealousy, it was inevitable that the NSA would develop a reputation as cold-blooded dataspooks of formidable intellect and cunning. In James Bamford’s The Shadow Factory: The Ultra-Secret NSA from 9/11 to the Eavesdropping of America, the author makes a few nods to this image by referencing the Tony Scott thriller Enemy of the State, in which the NSA’s sheen of Orwellian omniscience appears well-nigh invulnerable. As Bamford makes clear, not only is the NSA’s reputation wildly (and somewhat dangerously) overblown, but their old nickname strangely appropriate, given their seeming lack of utility and apparent immunity from accountability.

The most damning part of Bamford’s case is contained in the book’s first section, “Attack”. Here the former investigative reporter and now nearly full-time national security researcher (Bamford’s 1982 bombshell, The Puzzle Palace, is considered the definitive book on the NSA) turns his gimlet eye on some of the 9/11 hijackers in the days leading up to the attack. But instead of just methodically deconstructing the men’s day-to-day operations, Bamford threads the timeline of their actions a more surprising narrative: the men were based out of a Yemen safe house the NSA was monitoring and spent their final days just blocks from the NSA’s headquarters.

Of course, it wasn’t just the NSA that dropped the ball, practically every other national security entity did as poor a job. In early 2000, aspiring al-Qaeda soldiers Khalid al-Mihdhar and Nawaj al-Hazmi arrived in California. The NSA, aware of Osama bin Laden’s connection to the Yemen ops center that served as the men’s base, had started to follow the men’s communications when it appeared they were going to be traveling to an al-Qaeda meeting in Malaysia. But between the NSA never sending word to the State Department, and inexplicable bungling and bureaucracy at the CIA, the two slipped through the net completely and landed in California, ready to begin flight training and culture acclimation. Mihdhar and Hazmi even rented a room from a man who had been a longtime “asset” for the FBI. It was so easy for Mihdhar to get in and out of the country (even though he was a person of interest to the NSA, which had never gotten him added to any watch list) that he even went home to his wife and child in Yemen and returned in July 2001, just like any other visitor.

What happened was that the 9/11 hijackers pulled together their plan at a time when even the supposedly omniscient NSA was struggling to keep up with what it was following. The technological revolutions of the 1990s upended the world of COMINT, leaving the NSA floundering in a flood of communication. According to NSA chief General Michael Hayden, during the 1990s, international telephone traffic leaped from 38 billion minutes to over 100 billion. Bamford gravely comments that “Out of those 100 billion minutes, however, the NSA needed only a few dozen to discover and stop bin Laden’s planned air attack—and those few dozen were sitting nearby in a cavernous tape library collecting dust.”

Bamford describes how the art of modern American COMINT began in earnest right after World War II, when the government pressured companies handling overseas telegram traffic, like Western Union, to hand over copies for analysis. The NSA took over this program in 1952 and earnestly and routinely spied on Americans communicating with the outside world until the mid-1970s, when intelligence investigations nearly sent several NSA officials to jail and set up the FISA court to keep the nation’s spymasters from monitoring communications to or from US civilians without a warrant.

Fast forward to the post-9/11 era, and Hayden—who previously had barely dared to smudge the line of illegality in terms of domestic surveillance—decided to ignore the FISA court completely and went straight to the telecommunication companies for help in spying on their fellow citizens. For the most part, the industry was all too happy to play along. Perversely, Hayden’s post facto gusto for rule-breaking didn’t help to solve the main problem.

Bamford illustrates the point with a program proposed in the 1990s called Thinthread, which would have automatically encrypted all messages and phone calls leaving and entering the US, preserving the privacy of the content but leaving visible their exact destination and origin. Thinthread could then have allowed the NSA to establish patterns of communication—such as between the hijackers calmly residing in the FBI asset’s upstairs bedroom and their Yemeni ops center—and obtained a FISA warrant to look into the body of the calls themselves. Worried about even the appearance of legal impropriety, Hayden killed Thinthread. And so a few years later, the monstrous agency’s dazzling gadgetry continued to vacuum the sky clean of phone calls and data transmissions while its small armies of code miners struggled to make sense of it all, blithely unaware that the 9/11 hijackers were holed up in a motel room just down the street in the NSA’s hometown of Laurel, Maryland.

The Shadow Factory is jammed with infuriating anecdotes such as these, most of them related in Bamford’s dust-dry newspaperman’s prose. It’s occasionally an informational overload, particularly the further Bamford delves into the technical particulars of exactly how the NSA was burrowing into the world’s telecommunications infrastructure. His approach can at times resemble that of the NSA itself, sucking up all related information and imparting it in a not always connected fashion. There is a strong narrative here of national security neglect and the deep dangers that they leave the country in, but Bamford too often gets sidetracked into issues like the legality of surveillance, contractor corruption, and omnipresent ultra-surveillance. The problem is that each issue is practically its own book and Bamford at times seems to forget the core of what his book is.

The story of The Shadow Factory is simple: America has the world’s most advanced and well-funded surveillance network, it had several 9/11 hijackers in its sights, and it never did anything about them. It’s more Get Smart than Enemy of the State. In the end, what may be more frightening is not that our phone calls are being listened to and emails read—and be assured, they are—but that such a gross violation of our basic civil liberties won’t keep us any safer, anyway.