pageok
pageok
pageok
Interesting Crime-Facilitating Speech Incident:

It was in the news several days ago, but I missed it, and thought other readers might have, too:

The federal government has asked the National Academy of Sciences not to publish a research paper that feds describe as a "road map for terrorists" on how to contaminate the nation's milk supply.

The research paper on biological terrorism, by Stanford University professor Lawrence M. Wein and graduate student Yifan Liu, provides details on how terrorists might attack the milk supply and offers suggestions on how to safeguard it.

The paper appeared briefly May 30 on a password-protected area of the National Academy of Science's Web site. . . . [T]he Department of Health and Human Services, which asked the academy to stop the article's publication. . . .

The paper gives "very detailed information on vulnerability nodes" in the milk supply chain and "includes . . . very precise information on the dosage of botulinum toxin needed to contaminate the milk supply to kill or injure large numbers of people," [HHS Assistant Secretary Stewart Simonson wrote in a letter to the science academy chief Dr. Bruce Alberts]. . . .

The NAS did indeed pull down the paper and delay its publication, and they're apparently reviewing it further. I'm naturally interested in this as an example of crime-facilitating speech — but also as the father of a boy who drinks lots of milk.

The NAS is a private organization, and as best I can tell, this was a request, not a command or even a threat, so there's no First Amendment problem here. But it's still an interesting question about public safety, scientific openness, and what mix (and timing) of openness and secrecy is the best way to deal with potential security problems.

Wein describes the problem in the course of arguing in favor of some potential solutions, such as "that the FDA guidelines for locking milk tanks should be made mandatory, and . . . the dairy industry should improve pasteurization to eliminate toxins." To get such potentially expensive procedures implemented, one may well need to explain precisely why they're necessary, and do so publicly, so that it's harder to sweep the objections under the rug. On the other hand, there are obvious costs to public disclosure, too. A hard and important question.

Comments are enabled -- please keep them on-topic, substantive, and polite.

Greg D (mail):
Well, I'm not a big fan of enabling people to cover up their screwups in the name of "security", but, OTOH, I don't see any reason why we need to know "very precise information on the dosage of botulinum toxin needed to contaminate the milk supply to kill or injure large numbers of people."

Is a little common sense from security researchers too much to ask?
6.15.2005 8:47pm
Bob Woolley (mail):
Well, you'd need to determine whether the amount of toxin needed could plausibly be obtained, for example, so you'd first have to calculate how much would be required.

A less dramatic example, I suppose, was a year or so ago when some bicycle enthusiasts posted on the web videos of how some Kryptonite locks could be easily circumvented with a Bic pen. It could be argued that because of the manufacturer's long-standing reputation for having the toughest locks available, users would tend not to believe that they could be busted with a piece of a cheap pen, absent one or more videos showing how it could be done. Furthermore, the company had known of the vulnerability for a couple of years, and hadn't done anything to alert its customers or alter its design, so the videos were a highly effective means of prompting action that the company probably should have taken long before on its own initiative.

Certainly at least some thieves first learned how to break into these locks because of the distribution of the video clips, but it may well be that the net effect was a reduction in bicycle thefts, both because of the manufacturer suddenly taking action and because lock owners were alerted to take additional precautions.

Bob Woolley
6.15.2005 9:45pm
bld (mail):
The problem here is the very fact that the document is known, means that someone with an interest in attacking the milk supply knows to look there. Interested parties, of course, will invest the time to attempt to reproduce the research. The rest of us are being limited in discussing the impact, which of course limits the interest of professional politicians.

The only way out of this bind is either to demand pre-vetting of research publication (which I doubt many reading this would favor - that's terribly totalitarian), or to act as an open society.

I fear that muddling through (with voluntary restrictions asked for by government agencies, takedown requests for "sensitive" information, etc.) will, over the long run, give us the worst of both.

In the long run, restricting information is a loser's bet. One of the few cases I can see a pragmatic benefit is when something is actively being done to secure X, and someone else has a particularly untimely paper. Delaying publication with a published reason, in that case, seems pragmatically sensible.

I'm putting aside other costs of this sort of thing, such as manipulation of security threats for political gain, etc.
6.15.2005 9:52pm
Patrick McKenzie (mail):
Speaking as someone who has done security research in a few contexts (most related to IT), like many academics we're a very insular bunch. Like many academics, we're quite inclined to disagree with assessments partly out of "Not invented here"-type reasoning and partly out of recognition that our profession has often had a tendency to swing towards the Chicken Little end of the spectrum at the slightest provocation. But for the description of what agent to use and what levels would be toxic, you can be sure the first comment on the paper would be "This proposed attack is unrealistic and since the adversary likely does not have access to extremely virulent substances mere contamination with a less troublesome substance, such as cultured E Coli, would be largely prevented through existing safeguards". Mentioning "botulism" and the correct dosage both reminds people that "There exist substances which are highly virulent, avoid some pasteurization procedures, and easily obtainable" and also allows you to use standard medical texts to verify that the claims of toxicity are indeed in the correct ballpark.

In the IT setting, you're expected to go farther than even a very descriptive account of the vulnerability -- an actual implementation of the attack ("proof of concept") is the gold standard. For example, if the claim is that I have discovered an attack which allows remote execution of programs on your computer by manipulating your browser's stack, I would be expected to craft a web page which could do this in a non-destructive manner, and then tell you "Go to www.exampleattack.com, you will notice a Notepad window pops up with text specified by me assuming your browser/OS combination is one of the vulnerable ones I have listed. Note that opening Notepad proves I have general execution privileges in on your machine, which means I can execute arbitrary code."

Minus the proof-of-concept, that vulnerability discussion is substantially less persuasive, because it boils down to the claim "I can fit a key I won't show you into a lock you can't see" (security researchers do not regularly have access to the source code of programs with security vulnerabilities -- a fact which is changing, slowly).
6.15.2005 9:53pm
Jim Douglas (mail):
The National Academy of Sciences is an organization of roughly 2,000 outstanding scientists from all disciplines (one-half from industry, 5% from government laboratories and 45% from acedemia) establihed by Abraham Lincoln to provide independent advice to the government on how scientific issues will impact society. Thus, it is not surprising that the NAS would respond to a government request to delay and/or modify publication.
6.15.2005 9:59pm
bld (mail):
Patrick -

I, too, come from an IT perspective, although I'm consulting on a wider range of topics now. The proof-of-concept exploit serves well in IT, because demonstration is easy. In the wider world of security, it isn't so. In fact, a proof of concept would make one a terrorist.

Additionally, per this specific example, botulinum is not at all hard to produce. It happens by accident many thousands of times per year, and a "lab" (your kitchen would work) to produce it in quantity is simple enough for someone with motivation. The interesting parts of this paper likely have to do with research into the transportation network, safety precautions taken on farms, knowledge of inspection regimes, faults in wholesaler purchase patterns, etc.

All of which is public knowledge, if not widely dispersed. Someone posing as, variously, a farm hand, cattle farmer, truck driver, government job applicant and commodities buyer could probably reassemble most of the interesting parts with a couple of months of effort.
6.15.2005 10:18pm
DaveJ (mail):
"Security through obscurity is no security at all."

Publish it. The various providers will look at their processes and assess their risks. The potential liability lawsuits from failing to respond at all are a sufficient spur to make any dairy and/or it's insurer take things seriously. And until the report is widely read, any alarm will be ignored. Once the local paper has an article on the weaknesses in general, perhaps with an evaluation of or interview with the local business, nobody will ignore it.
6.15.2005 10:36pm
Guest1 (mail):
This is off-topic free speech question, so I apologize, but it has been bugging me for a while, and this seems as good a place as any to ask it: Why is extortion permitted by the 1st Amendment in the case where the threatened activity is otherwise legal? (Thus, it seems, not implicating the policy concerns of Brandenburg.)

For example, a woman wishes to leave her husband because he is a philanderer, and she has video of his exploits. She threatens to file divorce and introduce the video into evidence, unless he pays her $x and allows her to leave. Given that both divorce and the introduction of evidence are legal, how can the speech which threatens this be criminalized?
6.15.2005 11:27pm
William Spieler (mail) (www):
DaveJ: Although security through obscurity isn't good, it's better than nothing. Of course, we should be at least somewhat peeved that the myriad of government agencies collude with the dairy industry to deliver a product that will likely turn out to be shielded from any significant liability. I'd hope that the concerns addressed in this paper are at least considered by the appropriate people.
6.15.2005 11:40pm
DaveK:
(No connection to DaveJ, but I couldn't resist the name since I agree with him and am a Dave K.)

William: Security through obscurity isn't necessarily better than nothing. Oftentimes--and particularly when one is talking about potentially dangerous information--it hides information from the public, who might act on or defend against it, but hides nothing from the potential evildoers, who more often than not have figured out how to do damage anyway.

I suspect that the simple idea "put botulinum in milk" is more than enough for a clever and diligent terrorist to replicate most of the findings of this article. Hiding the article from the public eye doesn't achieve much, and has high costs.
6.16.2005 12:17am
bld (mail):
I wrote:

All of which is public knowledge, if not widely dispersed. Someone posing as, variously, a farm hand, cattle farmer, truck driver, government job applicant and commodities buyer could probably reassemble most of the interesting parts with a couple of months of effort.



I'm actually tempted to do so. If I didn't have other, paying committments, it is possible that I would, and I might in the future. Great press for my firm.



What I meant to point out is that, while I think I'm clever and all, this isn't hard. Social engineering is not trivial, per se, but if you slip in to the proper mindset, it isn't hard. I suspect anyone reading this can do it, with a little practice. A theater background helps, but is not needed.



Bottom line: if you want to know something "sensitive" about a corp, government agency, or NGO: all you have to do is ask in the right way, at least most of the time. Secrets (or Top-) are harder. But I can vouch for a paraphrase of Kissinger here.

So, the question becomes if we deprive the citizens of a nation of the knowledge that X is vulnerable, and let them do what they think is right, or do we allow the suppression of the free exchange of knowledge, in their name, because it is good for them not to know the risks. The Bad Guys will have it, if they have motive and means. Should a free society be hobbled from making rational choices about risks?

6.16.2005 12:58am
heldmyw (mail):
I am reminded of a news report during one of the interminable brush fires that sweep California every year. The lackwit with the microphone not only described the arsonists technique for causing a butane lighter to remain lit when cast into some underbrush, he actually demonstrated it.

I am of the opinion that anyone might have figured this out for themselves, (given a few moments of boredom and a destructive bent), but to show this to the all-too-large population of mental midgets that wander loose about the landscape was inexcusable. Like giving the baby a ball peen hammer, the question becomes no longer 'if' something will be damaged or injured, it is 'when'.
6.16.2005 11:08am
cathyf:
Publish it. The various providers will look at their processes and assess their risks. The potential liability lawsuits from failing to respond at all are a sufficient spur to make any dairy and/or it's insurer take things seriously.

I think that's a bit naive... We have an analogous situation in meth production. The basic building blocks of meth are ammonia fertilizer and cold medicine, and rural areas have a huge problem with drug producers stealing fertilizer from the fields. The risks are very well understood and appreciated -- meth addicts are crazy and violent and have killed farmers or their kids in order to get to the fertilizer tanks. Potential slaughter of your kids is way more attention-focusing than lawsuits, but even that is inadequate to prevent all fertilizer thefts.

And your analysis suffers way too much from an "us/them" attitude, where dairy producers are the good guys and they protect themselves and us from the bad guys. The genius of terrorism is that they hide themselves among us and we can't tell who they are until it's too late. (And yes, it is a significant law-enforcement challenge to make sure that all of the fertilizer that a farmer orders ends up on his fields and that he doesn't have a meth lab side business back in his barn.) Liability lawsuits aren't much of a threat to the terrorists who buy and operate a dairy farm -- not to mention if AI says that Gitmo is a "gulag" and the ICRC says that indefinite detention is "tantamount to torture" there is no way that we'll get away with the horrific "abuse" of (*gasp*) filing lawsuits.

cathy :-)
6.16.2005 11:16am
Christopher (mail):
It seems to me that we are only talking about one group of baddies, terrorists, while ignoring the crazies. As has been noted by other posters, you don't need to give terrorists the complete plan for security exploitation; they are smart enough and well funded enough to take it from the title of the article to where they want to be, so the rational for omitting details won't be to stop the terrorists. While simple crazies on the other hand, if you give them a complete plan might do it, but without one probably won't have the focus to figure out even a trivial missing piece. For instance it might have caused a few incidents if in the movie "Fight Club" the ingreadiencts for home made napalm were not the laughable "gasoline and frozen orangejuice concentrate", but instead were the real anarchist's cookbook recipe.
6.16.2005 12:08pm
dafydd (mail):

Publish it. The various providers will look at their processes and assess their risks. The potential liability lawsuits from failing to respond at all are a sufficient spur to make any dairy and/or it's insurer take things seriously. And until the report is widely read, any alarm will be ignored. Once the local paper has an article on the weaknesses in general, perhaps with an evaluation of or interview with the local business, nobody will ignore it.


Publish it. Liability concerns aside, effective countermeasures can't be implemented if potential attacks can't be discussed in detail. Since these folks have gone to all this trouble to identify low-probability attacks, using those attacks to design defences against higher-probability attacks pays off.

On a more philosophical level, information is never bad. "Badness" only comes in how that information is used. Let's distinguish the tool from the purpose to which it is put.
6.16.2005 2:39pm
Robert Schwartz (mail):
Milk is not good for you. Teach them to drink beer:-)
6.16.2005 6:39pm