Orangeek, the original hand-drawn mascot

orangeek's blog

Is it worth it?

6 Comments

Via Bruce Schneier I found this paper (in a downloadable 26 pages pdf) from researchers at the New Castle University (Australia).
"ASSESSING THE RISKS, COSTS AND BENEFITS OF UNITED STATES AVIATION SECURITY MEASURES" by M.G. Stewart and J. Mueller

The goal of the research is to perform a cost-benefit analysis over the countermeasures taken in the last years by the TSA United States Transportation Security Administration.
Even if reading some of the assumptions (giving to a human life an economic value, for example) could be synical, this is how the homo economicus thinks.

Hint: I know what are you thinking.
"Finally someone is going to prove that all the humiliating procedures (or, at least, time consuming) we - passengers - have to live, are completely wrong and useless."
Sorry. It's not about that. And better: actually the researchers assign a 50% success rate (in avoiding 9/11-like attacks) to the pre-boarding security(such as baggage check, no-fly lists, laptop check, ...). [UPDATE: after the jump, and after dasnake's comment, I inserted also an extract with the reason why the researchers assigned such rate]

After the jump, the parts I found most interesting.

Who - and how - is ruling the business?

For many years now the United States Office of Management and Budget (OMB) has recommended the use of cost-benefit assessment for all proposed federal regulations
[...]
Yet despite the massive expenditures involved, a senior economist at the Department of Homeland Security recently acknowledged that “We really don't know a whole lot about the overall costs and benefits of homeland security.

Abstract

Following this approach [of a cost benefit analysis], Stewart and Mueller conducted an assessment of increased United States federal homeland security expenditure since the 9/11 attacks and of expected lives saved as a result of such expenditure. The cost-benefit analysis suggests that the annual cost per life saved ranges from $64 million to $600 million, greatly in excess of the regulatory safety goal of $1-$10 million per life saved. This means that $300 billion spent by the United States government to protect the American homeland from terrorism since 2001 fails a costbenefit analysis. These findings focus on the total homeland security budget.

Give a value to life saving countermeasures

Elizabeth Paté-Cornell suggests that a cost per life saved of $2 million or less is appropriate for current practice, and the United States Department of Transport adopts a figure of $3 million. For most activities a cost per life saved not exceeding $1-$10 million is typical. This value is consistent with many studies as well as values currently used by most United States federal agencies. In other words, if the annual cost per life saved exceeds $1-$10 million, such risk reduction expenditure is deemed to have failed a cost-benefit analysis and so is not cost-effective. In such cases it is more rational to divert the expenditure to reduce the risks for other hazards where the benefits (lives saved) will be higher.

Security weakness or people's weakness? Maybe the best effect of 9/11 attacks (this is my comment, not author's)

One reason for the extent of the losses of 9/11 was the reluctance of crew and passengers to confront and resist the hijackers. This is perfectly understandable as most previous hijackings ended peacefully or with minimal loss of life as the main response to a hijacking was to “get
the plane on the ground so negotiations can begin”. Indeed, only a few months earlier three terrorists, in this case Chechens, had commandeered a Russian airliner, demanding that it be flown to Saudi Arabia at which point they were overcome by local security forces with almost
no loss of life. The 9/11 suicide attacks on the World Trade Center and Pentagon changed this perception.
Hence, on hearing of these attacks, the crew and passengers on the fourth plane, United Airlines Flight 93, overpowered the hijackers before the aircraft could reach its intended target.
Passengers will now fight back if there is any indication that the terrorists’ intent is to enter the cockpit or that they are suicidal in any way.
As pilot Patrick Smith puts it forcefully:

“We first need to revisit the morning of September 11th, and grasp exactly what it was the 19 hijackers so easily took advantage of. Conventional wisdom says the terrorists exploited a weakness in airport security by smuggling aboard box-cutters. What they actually exploited was a weakness in our mindset - a set of presumptions based on the decades-long track record of hijackings. In years past, a takeover meant hostage negotiations and standoffs; crews were trained in the concept of "passive resistance." All of that changed forever the instant American Airlines Flight 11 collided with the north tower. What weapons the 19 men possessed mattered little; the success of their plan relied fundamentally on the
element of surprise. And in this respect, their scheme was all but guaranteed not to fail. For several reasons - particularly the awareness of passengers and crew - just the opposite is true today. Any hijacker would face a planeload of angry and frightened people ready to fight back. Say what you want of terrorists, they cannot afford to waste time and resources on schemes with a high probability of failure. And thus the September 11th template is all but useless to potential hijackers"

Could be better to...

Risk reduction measures that cost tens or hundreds of millions of dollars per life saved cannot be justified on rational life-safety grounds—or, essentially, on moral ones. If some of the additional federal government and private sector spending on aviation security were invested
in other hazard or risk reduction programs, many more lives would have been saved. For example, Ahrens estimates that 890 lives could be saved annually if all homes had a working smoke alarm and Tengs and Graham estimate that an investment of $200,000 per year in smoke alarms will save one life. From this it can be inferred that an expenditure of only $5 million per year will save approximately 25-30 lives per year. Similar examples can be found for other risk reduction measures or regulations. While these numbers are approximate, they do illustrate the enormous opportunity costs of the Federal Air Marshal Service. In a multi-hazard environment, such expenditure should be invested in a wide range of more cost-effective risk reduction programs such as flood protection; vaccination, screening and other health programs; vehicle and road safety; occupational health and safety.

UPDATE

After dasnake's comment who asks why the assign a success rate of 50% to pre-boarding measures, here is the reason, directly from the paper:

The extra and more vigilant intelligence, immigration and passport control, airport screening, and other pre-boarding security measures implemented since 9/11 as arrayed in the TSA’s 14 layers of security should result in an increased likelihood of detection and apprehension of terrorists. Increased public awareness is also of significant benefit to aviation security. Added to this are the much enhanced preventative policing and investigatory efforts that have caught potential terrorists including, in the U.K. in 2006, some planning to blow up airliners.

Combined, we suggest, these measure by themselves reduce the risk of a replication of 9/11 by at least 50%, and this is likely to be a lower bound value. There has been no successful hijacking anywhere in the world since 9/11 and very few attempts at blowing up airliners - and none of these in the United States. In consequence, we suspect, R(pre-boarding security)
is likely to be much greater than 50%. Nonetheless, for the present analysis assume R(preboarding security)=50%.

Written by orangeek

Monday 21 July 2008 at 7:45 PM

Posted in pop

airport, costbenefit, schneier, security, tsa

6 Comments

  1. 50% di successo? E di grazia, dato che non ci sono stati altri attentati come le torri gemelle come diavolo ha fatto a misurare un rate di successo diverso dal 100%?
    Mah, l'homo economicus ha dei problemi in matematica base, evidentemente.

    dasnake

    Monday 21 July 2008 at 10:51 PM

  2. hai letto il paper? :)
    in fondo al post, ho aggiunto un commento sulla tua domanda.

    orangeek

    Tuesday 22 July 2008 at 10:37 AM

  3. Appunto, l'homo economicus farebbe bene a farsi un corso di matematica. Quel discorso e` una puttanata.

    Partendo dal fondo, se tu hai un sistema di sicurezza con 50% di rate di successo faresti bene a prendere chi l'ha progettato e insegnargli come impugnare a modo una zappa. Pensaci bene: vuol dire che UN ATTENTATO SU DUE RIESCE. Il paradiso dei terroristi. Voglio sperare che il numero che i designer del sistema di sicurezza hanno venduto sia molto sopra il 99.9%.

    Andiamo al dato specifico, non si vede per quale motivo lui debba assumere il 50% di rate di successo. Quel rate si puo` misurare, la stima puntuale e` ovviamente il 100% e non vuol dire molto, mentre per vare una stima d'intervallo bisognerebbe conoscere il numero di terroristi che sono stati presi in flagrante. In assenza di questo dato ogni discorso e` fuffa. Se questo dato e` zero allora anche il lower bound e` 0%, se e` diverso da zero puoi con un conto banale (e un'ipotesi realistica sui terroristi nel mondo) calcolarti il tuo intervallo.

    E che cazzo, un po' di serieta`.

    Cmq, l'altro dato interessante per valutare lo scassamento di cazzo e` il rate di falsi positivi. Su quello si che ci sarebbe da fare qualche discorso.

    dasnake

    Tuesday 22 July 2008 at 9:43 PM

  4. Mmmh, non sono certo un matematico, e forse neanche un homo economicus tout court, ma: gli autori dividono il "merito nel success rate" tra tre aspetti: preboarding, porta rinforzata, air marshall.
    Sul fatto che non vi siano stati altri attentati (e che quindi farebbe pensare a un rate del 100%), lui divide il rate in questi tre fattori.
    Su quelli presi in flagrante, c'? il riferimento ai casi in UK (intelligence, quindi preboarding security) e un altro caso in australia in cui invece ha funzionato la reazione dei passeggeri.
    ma ripeto, hai letto il paper? :)
    Comunque, scherzi a parte, a me pare interessante, anche perch? sostanzialmente il succo di tutto il lavoro ? quantificare la spesa per "persona salvata" che nel caso degli aerei e della TSA sembra decisamente troppo elevato.

    orangeek

    Wednesday 23 July 2008 at 9:27 AM

  5. Oh mio dio, l'ho letto.

    La matematica e` folle, le stime sono ancora piu` folli. Ritiro quanto ho detto sulle zappe, dubito che persone del genere siano in grado di usarle.

    Una critica piu` approfondita quando ho tempo, ma ti dico solo questo: la loro equazione per il costo-per-vita-salvata e` inversamente proporzionale al numero di vite salvate prima delle nuove misure di sicurezza. Fanno un discorso ragionevole e questo numero viene zero. Ovviamente con zero il costo-per-vita-salvata diverge e l'agricoltura e` l'unico futuro per questi brillanti autori, quindi decidono di giocarsi il jollone: prendono l'UNICO EVENTO DI TERRORISMO AEREO DELLA STORIA, cioe` il fattaccio dell'undici settembre, prendono un periodo a caso (una stima sull'attivita` di al-quaeda, come se prima di al-quaeda non ci fossero aerei e/o non ci fossero nemici degli stati uniti) e dividono, ottenendo un comodo 300 che evita di far esplodere il tutto. Rabbrividiamo.

    dasnake

    Thursday 24 July 2008 at 3:58 PM

  6. mmmmh, a questo punto penso che dovrei rileggerlo... che dici? :)

    orangeek

    Thursday 24 July 2008 at 7:52 PM

Comments are closed.