Long before Edward Snowden’s whistleblowing revelations hit the world and the Obama administration’s approval ratings like a ton of bricks, we ran a story in March 2012 which exposed the NSA’s unprecedented domestic espionage project, codenamed Stellar Wind, and specifically the $1.4+ billion data center spy facility located in Bluffdale, Utah, which spans more than one million square feet, uses 65 megawatts of energy (enough to power a city of more than 20,000), and can store exabytes or even zettabytes of data (a zettabyte is 100 million times larger than all the printed material in the Library of Congress), consisting of every single electronic communication in the world, whether captured with a warrant or not. Yet despite all signs to the contrary, Uber-general Keith Alexander and his spy army are only human, and as the WSJ reports, the NSA’s Bluffdale data center – whose interior may not be modeled for the bridge of the Starship Enterprise – has been hobbled by chronic electrical surges as a result of at least 10 electrical meltdowns in the past 13 months.
The facility above is where everyone’s back up phone records and emails are stored.
Such meltdowns have prevented the NSA from using computers at its new Utah data-storage center which then supposedly means that not every single US conversation using electronic media or airwaves in the past year has been saved for posterity and the amusement of the NSA’s superspooks.
This being the NSA, of course, not even a blown fuse is quite the same as it would be in the normal world: “One project official described the electrical troubles—so-called arc fault failures—as “a flash of lightning inside a 2-foot box.” These failures create fiery explosions, melt metal and cause circuits to fail, the official said. The causes remain under investigation, and there is disagreement whether proposed fixes will work, according to officials and project documents. One Utah project official said the NSA planned this week to turn on some of its computers there.”
More from the WSJ on this latest example of what even the most organized and efficient of government agencies ends up with when left to its non-private sector resources:
Without a reliable electrical system to run computers and keep them cool, the NSA’s global surveillance data systems can’t function. The NSA chose Bluffdale, Utah, to house the data center largely because of the abundance of cheap electricity. It continuously uses 65 megawatts, which could power a small city of at least 20,000, at a cost of more than $1 million a month, according to project officials and documents.
Utah is the largest of several new NSA data centers, including a nearly $900 million facility at its Fort Meade, Md., headquarters and a smaller one in San Antonio. The first of four data facilities at the Utah center was originally scheduled to open in October 2012, according to project documents. The data-center delays show that the NSA’s ability to use its powerful capabilities is undercut by logistical headaches. Documents and interviews paint a picture of a project that cut corners to speed building.
Backup generators have failed numerous tests, according to project documents, and officials disagree about whether the cause is understood. There are also disagreements among government officials and contractors over the adequacy of the electrical control systems, a project official said, and the cooling systems also remain untested.
The Army Corps of Engineers is overseeing the data center’s construction. Chief of Construction Operations, Norbert Suter said, “the cause of the electrical issues was identified by the team, and is currently being corrected by the contractor.” He said the Corps would ensure the center is “completely reliable” before handing it over to the NSA.
But another government assessment concluded the contractor’s proposed solutions fall short and the causes of eight of the failures haven’t been conclusively determined. “We did not find any indication that the proposed equipment modification measures will be effective in preventing future incidents,” said a report last week by special investigators from the Army Corps of Engineers known as a Tiger Team.
The architectural firm KlingStubbins designed the electrical system. The firm is a subcontractor to a joint venture of three companies: Balfour Beatty Construction, DPR Construction and Big-D Construction Corp. A KlingStubbins official referred questions to the Army Corps of Engineers.