The Pebble and the Avalanche

Moshe Thumbnail
Current Revolutions in Business and Technology

by Dr. Moshe Yudkowsky,

author of The Pebble and The Avalanche: How Taking Things Apart Creates Revolutions


Tue, 2011-Jun-07, 06:09

Story Marker
The Irony: Twitter and Apple

Yesterday was precisely either the right day or the wrong day for Apple to announce that the next release of Apple's operating system for the iPhone will integrate Twitter.

As Apple announced their new Twitterized features, back in New York City Representative Anthony Weiner (D-New York) called a press conference to admit that he'd lied about his use of Twitter to send an embarrassing photograph of himself.

Apple may view Twitter integration as a win for their users, and that might even be the case for some of these users (and if Twitter proves reliable enough to provide service and remains profitable enough to continue as business). But I know that I don't want to accidentally send all my emails out in my Twitter feed; that I don't want all my photos uploaded for the world to see; that I don't care for anyone other then my spouse to know my location at all times. Apple's integration of Twitter should make everyone nervous — a single misplaced tap on the screen away from sharing corporate secrets, vacation plans, or your child's birthday party photos with the entire world.

I'd say that Weiner's timing was impeccable. Weiner inadvertently performed a public service with his demonstration of the pitfalls of improper use of Twitter. With Apple's integration of Twitter into every nook and cranny of the iPhone, expect to see an entire new level of inadvertent disclosures.

Mon, 2010-Nov-29, 09:15

Story Marker
The Fallacy of Profiling

I'd like to address a particularly silly notion about airport security: profiling. The idea is that security personnel should profile certain ethnic groups and select them for special scrutiny, and devote less time to other ethnic groups and religions.

The argument for profiling rests on several assumptions, and all of them are either flat-out wrong or are based on ideas that have not been fully thought out. These ideas include:

  • Israelis profile passengers to provide the world's best security.
  • Profiles will fix current Transportation Security Agency problems, such as three-year olds placed on special watch lists.
  • Profiles increase security by focusing on likely suspects.
  • Profiles are feasible, affordable, and accurate.
Let's tackle these one by one.

The Israelis do not profile passengers according to race or religion. The Israelis examine each and every passenger as he or she arrives at the airport and at multiple checkpoints, to look for behavior or history that requires further scrutiny. Security is not confined to "risky population groups," i.e., Moslems or people with dark skins; risk assessment extends to every passenger. In the most famous case, a pregnant Irish woman attempted to board an El Al flight; after questions indicated risk because of the woman's associates, the Israelis dismantled her luggage and found that, unknown to her, the unborn child's Arab father had concealed a bomb in the woman's luggage.

Ethnic and religious profiling will not solve the TSA's problems with passengers on the no-fly list or the watch list. Even during the darkest days of Nazi terror in Europe, you could walk into the local Gestapo outpost, argue with commander, and sometimes get a person released from a concentration camp. Here in the US, the TSA will place you on a watch list for reasons they won't disclose and apparently will never take you off. Tens of thousands of people undergoe special scrutiny because somewhere, someplace there's an alleged terrorist with a vaguely similar spelling or pronunciation. The TSA deliberately does nothing about absurd situations such as three-year olds on the watch list; no person at the agency is willing to take the risk to his own personal career of taking a person off the list in case that particular person really turns out to be a terrorist. This is an institutional problem that we can expect in any secretive agency, especially one that does not have to respect due process of law. Profiling will not fix that problem; profiling will likely make the TSA act even worse.

I should mention at this point that the TSA already does profile according to behavior; they just do it very badly. When my wife or daughter fly in the summer they are often selected for additional screening because (as religious Jews) they do not wear sleeveless shirts or shorts. This triggers the primitive "behavioral" screening procedures used by the TSA, regardless of the fact that hundreds of women pass through the airport in such clothing every day. Worse yet is the complete illogic of the the screening: why is long skirt a security risk in the summer but not in the winter? Being "different," particularly in a diverse society, is not the same as acting suspiciously.

Ethnic and religious profiles do not increase security — they degrade security. For the sake of discussion, let's accept the incorrect premise that only Moslems wish to destroy or interfere with US airplanes. Let's further assume that just like in other places around the world we will one day see female suicide bombers in the United States. The argument that "the TSA should not frisk nuns" means that female suicide bombers will dress up as nuns, nurses, business women, and the like. Security should focus on behavior and risk factors; if a nun triggers suspicions she should not be immune from a pat-down search.

Finally, religious and ethnic profiling won't work because we don't have information about religion and ethnicity. Unlike other countries, I don't have my religion stamped on my identity papers. Would any sane white or black citizen in the US confess to being a Moslem if that means hassling from "security" at every airport, train station, and drivers license facility? How will the government verify my religious affiliation? Will the government open a dossier on every person in the US and examine his religious background for suspicion of unreported Islamic sympathies? What about FBI investigations for the taint of "risky" ethnic background? Reliable ethnic and religion-based profiling requires a level of intrusive government scrutiny that would cost us vast amounts of money and above all our precious remaining shreds of privacy and dignity.

Or will the TSA personnel simply guess at religion, political beliefs, and ethnicity by looking at people's skin color?

If we do profile according to race and religion, we will introduce a profound change in US society: in public spaces and our every day life, the official policy of the US government will be to exercise its police powers differently for different people, based on ethnic and religious background. The most basic, fundamental principle of the US is that everyone — everyone — is equal before the law. And in this case as so many others, it's not only sound ideology, it's the policy that works best.

Thu, 2010-Jul-08, 05:08

Story Marker
US Cyber Command

I personally find any product or service to be less believable when the prefix "cyber" is applied; as someone who works in high technology, to me the term smacks of decades-old terminology and attitudes, used by ignorant marketing executives who are ignorant of technology but desparately want to sound modern and up to date. The US "Cyber Command" falls into the category of distrusted services.

I've just seen this thoroughly unbelievable assertion in the Wall Street Journal:

Intelligence officials have met with utilities' CEOs and those discussions convinced them of the gravity of the threat against U.S. infrastructure, an industry specialist said, but the CEOs concluded they needed better threat information and guidance on what to do in the event of a major cyber attack.
Frankly, I find it very hard indeed to believe — given the Obama Administration's blundering, dithering, and outright obstructionism in the Gulf oil spill — that any CEO would look to this administration, or any other government agency, for instructions on how to respond to a security attack. That is, of course, unless the authorities require them to do so...

Mon, 2010-May-10, 09:24

Story Marker
Facebook and Security

Security expert and author David Levine will delete his Facebook account because of Facebook's policies on information privacy.

The specific problem that pushed him over the edge was applications added to your account without your permission — something that Facebook claims was a bug, but as Levine points out, Facebook's implementation of web technology makes mistakes and outright scams possible. Then there's the controversy pointed out by TechCrunch. I believe I'll be leaving Facebook myself as well.

Fri, 2010-Apr-02, 12:03

Story Marker
Seize the Network

Two US Senators outline plans for "cyberwar" security. I admit that the term "cyberwar" is so hokey that it makes me cringe every time I type it.

If you think their plan sounds reasonable, think about just how much government the Senators' plan implies. Innovation on the Internet comes when people are free to improvise and extend Internet services at will. Now imagine a world when a new service, such as Twitter, must pass government review before it can operate — so that the service can be audited for security risks. Now imagine what blog and newspaper web sites would have said about the security risks of potential-competitor Twitter, or what AT&T would have said about Skype's voice services. The Senators' plan will bring innovation and competition on the Internet to a screeching halt. (This topic is covered in my book, where I discuss the triumph of the upstart Internet over its heavily-supported but licensed and regulated rival, the X.25 network.)

Reading between the lines of the Senators' essay I see plans to license and regulate every corporate (and perhaps even private) connection to the Internet in the name of security; after all, a single insecure connection can be an entry point for "cyberterrorists" who will wage "cyberattacks" in a "cyberwar" against our "cyberdefenses" (cringe). The plan will also inevitably lead to vast new regulatory powers over Internet Service Providers (ISPs). Expect to see government-mandated licensing requirements for network engineers — after all, who but a licensed engineer can touch a connection that will be "audited" by some other regulatory body? I see hints of definitive mandates to require ISPs to somehow stop file sharing ("intellectual property rights"). And, and surely as water flows downhill, I expect additional taxes on Internet connections to support all this bureaucracy.

I have to wonder just how much traction this plan has: every lawyer, lobbyist, regulator, auditor, and certification-granting agency has a vested interest in the bill. The innovators and start-ups, who are vulnerable, simply do not have as much money.

Thu, 2010-Apr-01, 08:29

Story Marker
Movie Plot Threat Contest Contents

Thu, 2010-Mar-25, 04:51

Story Marker
"Movie-Plot" Threats: The Danger of Dismissal

Bruce Schneier will soon start his annual "movie-plot threat" contest. I'm one of the people who refused to enter the contest; I've always worried that someone might implement my scenario.

No need for me to worry any longer: Someone just implemented my "movie-threat" plot and successfully carried out the attack. As far as authorities can tell, the perpetrators were drug dealers. They blocked roads all through the town of Monterrey, Mexico, which caused massive traffic jams. At the same time (and it's hard to tell at this point if this was part of the plan or just typical Mexican lawlessness) small groups of armed men attacked various vehicles that were trapped in traffic.

This isn't the first time that an apparent "movie-plot threat" has turned up in real life. If I had entered the contest, I would have proposed as my entry a more violent, higher-casualty version of the Monterrey attack, one with different strategic goals.

The ultimate "movie-plot threat" (MPT) was — of course — the September 11th attacks against the United States. Hijacking airliners? Flying them into two skyscrapers, and the Pentagon, and the White House? No scenario could be more outlandish.

I find the term MPT to be pejorative and dismissive. but it does have potential. In order for the idea of "movie-plot threats" to be useful, we have to find a way to distinguish between realistic and unrealistic attacks. Take the idea of an attack by terrorists on a toddler day care center. Is this an MPT or not? Here in the US I've heard it dismissed as just another MPT; in Israel, bitter experience with Palestinian terrorists forced parents to post armed guards at all day care centers. There is absolutely no question that such attacks are feasible here in the US and would serve the strategic goals of certain terrorist organizations. Does labeling toddler day care center attacks as just another MPT have real meaning? Or does it reflect someone's subjective judgment, or worse, wishful thinking?

Simply because a threat scenario is outlandish, imaginative, new to your locale, or resembles an actual movie is not a reason to dismiss the threat as a mere MPT. Only a proper assessment of the threat will tell you if it's likely to be successful or not, and then you must then judge if it's likely to be perpetrated or not.

From the attacker's perspective, possible attacks incorporate several factors: the risk to the attacker (high to low); the damage to the target (low to high, multiplied by the probability of damage); expected political/social/strategic payoffs; the logistical risk; the operational risk; and the tactical risk.

The September 11th attacks had a very high strategic payoff, and therefore the attackers went forward despite the moderate operational and tactical risks. The attackers correctly assessed their opposition and did not allow themselves to be blinded by the MPT nature of the proposed attack. I admit that do wonder if they misjudged the strategic costs of September 11th — if they failed to understand the difference that a new President (Mr. Bush replacing Mr. Clinton) would make to the US's response.

For any given attack, we can create a cost-benefit analysis to the attacker based on the factors I outlined above. Of course we run a risk: self-assessment remains a difficult task (read Schneier on the difficulties that cryptographers face when they assess their own work), and people easily overlook the weaknesses of systems they design. But is there an objective measurement that puts a threat scenario into the realm of "improbable" or "unrealistic?"

No such objective measure exists, which is my main objection to the idea of "movie-plot threats" (MPT). Take the attack on a day care center here in the US. Certainly such attacks happen in Israel, perpetrated by Moslem extremists; certainly Moslem extremists have attacked the US before; why wouldn't day care centers be a target here in the US? We certainly can't dismiss such a possibility just because such an attack would bring the fury of the US down on the attackers and therefore deter the attack; after all, the Japanese did not understand how the US would respond to the attack on Pearl Harbor.

The strategic payoff of an attack on a day care center would be extremely high; the operational and tactical risks would be very low; the logistical costs negligible. The question then becomes why we don't see such attacks, and I have to wonder if the strategic costs are simply too high — that the attackers believe that a day care center attack will bring certain, furious, and unrelenting retribution from the US. But dismissing such attacks as MPT just because "they have not happened here before" is simply a subjective judgment with a pejorative edge.

Perhaps movie-plot threats can be a useful security tool. From the defender's perspective, movie-plot threats might provide a narrative for brainstorming. For example, most bank managers have considered and defended against ordinary bank robberies; but if they consider a more outlandish MPT, such as a helicopter landing on the roof, the manager might discover a hidden vulnerability to an attacker scaling the roof with a ladder.

The problem of MPT lies in how to constrain the range of possible scenarios — as always, the starting point must be the strategic payoff to the attacker. The other problem — the problem that earns Schneier's ire and notoriety in his blog — occurs when defenders confuse the possible with the probable, and confuse the scenario with the response. If you decide that your bank is vulnerable to helicopter, don't put anti-aircraft guns on the roof; you'll remain vulnerable to an attack by someone with a grapnel gun and a rope. Defenders should implement defenses not be against a specific scenario but against the systemic weaknesses exposed by the scenario. And each weakness must be subject to cost/benefit/probability analysis. After all, any neighborhood bank can be overwhelmed by a platoon of well-armed soldiers, but that does not mean that neighborhood banks must become fortresses.

Dismissing a threat as a "movie-plot threat" because it is similar to a movie-plot or seems outlandish (September 11th) is wrong; instead, dismiss it when the threat poses a low risk. With any luck, we can educate managers to respond to MPTs with reasonable analysis and proper security enhancements; and MPTs might even become a beneficial method to brainstorm security risks.

As for myself, I doubt I will enter Schenier's movie-plot threat contest. After all, one of my scenarios just showed up in the real world, and if I had entered the scenario into a previous contest I probably would wonder (no matter that it would be foolish to do so) if I'd managed to contribute to the plans of the Mexican attackers. What the Mexican attack demonstrates is that attackers can be very clever indeed, vulnerabilities combine in unexpected ways, and we should encourage rather than suppress imagination.

Wed, 2009-May-13, 07:38

Story Marker
Abandoning Flickr

One of the more pointless efforts to "secure" online accounts are the "security questions," in which system administrators ask pointless questions such as "What is your favorite candy?" and store the answer. Of course, this all presupposes that two years later I remember the answer that I give today; not only do my favorite foods change over time, but I don't even have a favorite candy. All this is done in lieu of real security, but frankly your bank is unwilling to spend the money to actually secure your account.

Your mother's maiden name and your social security number is recorded in dozens if not hundreds of different databases. I know of security professionals that choose a different mother's maiden name for each entity that demands it. I do, too.

I have my own little ways to answer these other bogus security questions, but apparently Yahoo/Flickr has caught on to the methods that I and apparently many others use to get past the nonsense — and Yahoo/Flickr won't let me register for a new Yahoo/Flickr account unless I follow their strict rules for answering security questions. It's hard to imagine, but even though they offer a free account the cost in aggravation is too high, and I've decided to look for a photo-sharing site that doesn't require such obnoxious security measures. Not to mention, of course, that I have no intention of creating a database just to record my bogus answers to their bogus security quetsions.

Mon, 2009-Feb-09, 09:29

Story Marker
Smaller Files Mean Easier Security

One of the fun parts of writing a book is when someone "gets it" and learns to apply the ideas.

I've just received an interesting missive from Tracy Snell, who has just left his post as CTO of Interactions:

I've used MobileMe for years and one thing I loved was the OSX keychain syncing. It's worked pretty well, but to have my keychains synced across all the Macs I use I was welded to MobileMe.  Kept hearing good things about 1passwd. So I installed it recently and it is nice.
Former versions were just a nice front end on the OSX keychain, but now it uses its own keychain format. Suddenly I lost sync across machines! So I go digging and find out that their new format is not one monolithic file storing all the keys but instead each key gets its own file. [This] makes syncing and merging a trivial issue. Just store your 1passwd keychain on DropBox or iDisk and you're done. You don't risk a corrupted keychain if you have a failure in the middle of a sync. Dawned on my last night it was a great example of disaggregation.
In other words, the location of the password keychain is no longer tied to the Mac, and the keychain entries are more robust and easier to synchronize.

And I guess I congratulate the makers of 1passwd as well, at least in theory, but don't forget to read some reviews before you try this product.

Fri, 2008-Dec-19, 05:33

Story Marker
Thoughts and Actions: Fifteen Years in the Slammer for You Tube Video

An Egyptian student, studying in Florida, just received a fifteen year sentence for posting a video on You Tube. The video explained how to build an Improvised Explosive Device and also advocated terrorism.

This is a thought crime: the intention of the defendant was on trial, not just his actions. He hasn't done much more than collect information and publish it. Furthermore others have done the same. Reporters drop hints all the time about how these bombs are built; one of my favorite libraries has British manuals on how to destroy railroads (written in Dutch, for distribution during World War II); a recent book on the building of the first atomic bomb gives away a wealth of detail, including previously elusive detail on the shape of the crucial beryllium initiator. Are all these people to stay out of jail because they didn't also preach jihad?

What about my old copy of Rogers Manual Industrial Chemistry, which contains a section on "war gasses" — if I sell it on Amazon, am I guilty of something? Why aren't the staff of Amazon in jail for selling copies of The Anarchist Cookbook and the U.S. Army's Field Manual FM 5-31: Boobytraps?

While there certainly are limits to free speech in the U.S., such as shouting "Fire!" in a crowded theater, these limits restrict the very outer limits of behavior, and even so these limits are the subject of controversy.

Conspiracy to plant bombs is one thing, and as much as I find charges of "conspiracy" to be slippery I'd still be unsurprised to see the defendant and his co-conspirator in the slammer. But since when has it been illegal to disseminate knowledge? And why is this defendant in jail while the staff of The New York Times is not? While intent is important in many criminal and civil cases — e.g., "Did the defendant fire his gun because he believed his life was in danger?" — when it comes to First Amendment and other freedoms, the introduction of "intent" crimes is a very slippery slope.

Comments: 1, Trackbacks: 0