Boiled down to its essence: The internet does not work the way the all-powerful National Security Agency needs it to work. The intelligence agency discovered at some point between 2008 and 2011 that the aperture at the end of one of its collection programs was too wide, and was vacuuming up detritus in the form of your emails and mine, and that there was no way to fix this without cutting off a source of valuable foreign intelligence. And this would be okay if the NSA made a good faith effort to identify and destroy the "dirty" communication after the fact.
But NSA did not. So focused were they on their foreign intelligence mission that they — and I'm talking about the leadership here — placed a lower premium on following the spirit of the law, much less its actual wording.
With these emails, they were not careful. They were cavalier. They were indifferent. And that indifference was not constitutional. The Constitution requires the government to be attentive to privacy when engaging in the daily business of government, a part of which is intelligence collection.
Here are the relevant phrases from Judge John Bates' September 2011 Foreign Intelligence Surveillance Court opinion. Caution: There's a lot of intel speak. I will translate below.
A footnote says that "[t]he court is troubled that the government's revelations...marked the third time that the NSA had disclosed a substantial misrepresentation regarding the scope of a major collection program."
The NSA engages in upstream collection when it diverts and copies internet traffic that flows through fiber-optic cables transiting the United States. Until this point, the NSA had assured the FISA court that its technological doohickeys could sort out foreign communications and discard domestic communications, and where this was not possible, that it had aggressive mechanisms in place to do this after the collection.
Turns out that internet content sticks together in a way that is too dense to be segregated by NSA's filters, meaning that a small but significant number of emails from Americans to foreign targets, or that were transmitted in packets that included emails of foreign targets, would pass through these detectors like neutrinos. And if NSA did not know about them, and it did not bother to look, they would sit in NSA repositories, unmarked, and unminimized. (I must emphasize here: Human analysts were not reading emails. Computers would read some of them, including those that referenced a foreign target.)
NSA compliance officials discussed the problem with the Department of Justice. Both Congress and the Foreign Intelligence Surveillance Court were notified. The court ruled that the collection itself was not problematical because it was unavoidable. What was unconstitutional was what NSA did with the data it incidentally collected. The agency did not do nearly enough to find these emails after the fact and destroy them, or minimize them. The NSA says that it worked with the court to rewrite its minimization rules so that the agency would be required to search for the dirty content, and then to reduce the amount of time that all of the content, whether dirty or clean, would be retained in NSA databases.
The "dirty" collection can happen in one of two ways.
NSA officials disclosed that the "selectors" used to pull in tranches of communications are names, telephone numbers, or email addresses — and not keywords, like "bomb." They said that if a U.S. citizen's email metadata happened to be transiting through the U.S. and included a valid selector, it would be pulled from the internet stream. At that point, NSA would not know whether the communication was from a U.S. person or not.
Step two: The NSA filters go to work. If the filter finds evidence from the metadata that the author of the email is a U.S. person, it instructs the NSA's computers to "minimize" the email, making its source invisible to the analyst. The analyst would use a variety of means to see if the U.S. person (whose identity has been made anonymous) is on a watch list, or is already subject to FISA collection, or is communicating with a target, or otherwise meets a threshold that warrants further scrutiny. At that point, the NSA would seek a FISA order or turn the identity of the person over to the FBI.
If NSA were legally monitoring the emails of a valid foreign target, like a nuclear proliferator, its filters would sweep in every communication that included the IP address, email address, or name of the bad guy. Every email the bad guy sent out and every email that the bad guy received would get collected. NSA discovered that ISPs bundle or "push" tranches of data to specific addresses when the addresses request the data. In practical terms, when you open up your Gmail, you'll see a number of new messages all at once. What NSA says is that it cannot technologically separate each individual email before it is collected.
If the proliferator had signed up for an email alert that I had created, my email alert would be collected by NSA in full. If the bad guy was a Russian spy, and I knew him only by his cover identity, my email to him about a concert we were going to attend would be collected by the NSA in full, content and metadata, everything. The Foreign Intelligence Surveillance Court had no problem with this. What the NSA did next was unconstitutional. They did not take steps to minimize my email after the fact, even after they began to examine all of the bad guy's emails. At that point, they could very easily have determined that I was a U.S. person, and that my communications had to be minimized.
That's the first problem.
The second is even more disturbing. A lot of emails that were not connected to the emails sought by the NSA but happened to be included in the same tranche for the sake of internet efficiency were also swept in. That is, a lot of random email traffic was collected on purpose, kind of like a kid using a scoop to get that Jolly Rancher out of a bowl of candy and grabbing all the candy around it, just because.
So, Judge Bates ordered the NSA to come up with a way to better identify U.S. persons' emails after the fact, to actively minimize and destroy what they had already illegally collected.
NSA did this to Bates' satisfaction. It retrained analysts and diverted resources to post-collection data segregation.
The NSA says that the whole affair is an example of how the compliance and oversight process is supposed to work. It is not pretty and it wasn't easy, but out of the view of the public, it recognized a problem, brought it to the attention of its overseers, and fixed it. I give NSA credit for that.
I'm not worried that the NSA read any of the emails it obtained. There'd be no reason to. In fact, the evidence suggests that the analysts did not look at the emails as much as they should have because they were too busy with other matters and did not prioritize their minimization procedures.
The lesson here: I'll save it for a future post. There are a lot of... call them cultural indicators... revealed in these opinions that ought to help Congress figure out better ways to perform its oversight functions.