The profiling myth

The drumbeat for racial profiling grows louder in New York City (thanks, DesiDancer):

Two elected New York City officials say Arabs should be targeted for searches on city subways. They claim the NYPD has been wasting time with random checks in its effort to prevent terrorism in the transit system… The New York Police Department said in a statement that racial profiling is illegal, of doubtful effectiveness and against department policy. [Link]

… they are most likely to be young Muslim men. Unfortunately, however, this demographic group won’t be profiled. Instead, the authorities will be stopping Girl Scouts and grannies… commuters need to be most aware of young men praying to Allah and smelling like flower water. [Link]

Even Tunku Varadarajan of the WSJ came out for profiling desis:

I find that I am–for the first time in my life–part of a “group” that is under broad but emphatic visual suspicion. In other words, I fit a visual “profile,” and the fit is most disconcerting… one must be satisfied either that profiling ought to be done or at least… that it isn’t something that “ought not to be done…” The practice cannot be rejected with the old moral clarity. The profiling process is not precisely racial but broadly physical according to “Muslim type…” [Link]

· Â· Â· Â· Â·

I’m pretty sure the 7/7 bombers did not leave the house all gulab attar-fabulous. It’s a practice more Arab than Pakistani, and the smell would have drawn too much attention. Racial profiling, the knee-jerk reaction to terrorist attacks on public transit, is a fool’s game. Instead of detecting inaccurate signatures (black, Arab, South Asian), the goal must be to detect behavior (carrying a bomb). The goal is accuracy. Otherwise you let deadly attacks succeed while wasting massive amounts of resources searching ordinary people.

The arms race between black hat and white hat has deep analogues in the military, the human immune system, antivirus tools, firewalls, spam filters and so on. In realm of computer security, behavior detection has utterly buried signature detection in terms of effectiveness. Signatures are trivial to spoof once you know what’s being looked for. Most viruses, worms and spam now mutate with every attack, it’s designed in from the beginning.

On 7/7, Al Qaeda switched from using Arabs to using Pakistanis and a Caribbean. Not two weeks later, they switched to using Africans. The pool of Muslim phenotypes is enormous; they can tap Chechens, Uzbeks, Filipinos, Indonesians, Chinese, Malays, white converts, black Americans, red-haired Kashmiris, blue-eyed Afghans. This is why the NYC mayor says the NYPD will use a true random sample instead of racial profiling. It’s not out of liberal fuzzy-mindedness, it’s because they’re being hard-nosed about saving lives. A race-based approach fails completely. It’s suicidal to rely on it.

Keep in mind that this discussion is about filtering out a tiny number of threats from a vast population. The NYPD, like any police force, should and does stop people with obvious indicators like wires, ticking or a chemical smell. That’s neither in dispute nor the point of this post.

· Â· Â· Â· Â·

From the standpoint of political conservatives, there are extremely good reasons to use the behavior approach rather than profiling. You can’t consistently be against affirmative action without also being against racial profiling. Just as race-based affirmative action is an inaccurate signature, sweeping up wealthy minorities as well as those who are genuinely poor, race-based threat detection is inaccurate as well. Above all, the burden is to be accurate — not to be politically correct, but to save lives.

Profiling also leads to big government abuses by expanding the number of law enforcement interactions with innocent citizens. Every discretionary interaction opens up the opportunity for abuse, harassment and bribery by humans with very human prejudices and flaws. Every interaction allows someone in a position of power to hassle you for arbitrary reasons. As with the computerization of government elsewhere, the point of automation is to eliminate human discretion and increase the accuracy of execution.

Exactly when countries like India and China are eliminating their red tape, profiling advocates would have NYC start up a new profiling raj. I already get pulled aside on flights all the time, which is not just a waste of my time, it’s a waste of law enforcement time. Furthermore, singling out an ethnicity for suspicion always increases the incidence of racist attacks in the world outside the subway. Right now we’ve got Sikh guys in turbans getting beaten up, when Muslim suicide bombers in the West don’t even wear turbans. It’s utter idiocy.

Those clamoring loudest for profiling are not those who will be affected by it. Their solution is not a system (‘Would this work for the entire population?’), it’s a hack, a band-aid. And those who are desi and ask to be profiled are like the soccer moms who step up to cops in the subway and demand to be searched because it make them feel better: they’re a public nuisance, and they get in the way of solving the problem. I say let them be taken aside for search — but only them

· Â· Â· Â· Â·

The heart of the terrorism problem is the ever-familiar technology cycle: high tech becomes low tech over time, specialized becomes commoditized. Making time bombs used to be a skill reserved for the criminal elite. Now a single bomb maker, along with religious brainwashers, can convert petty criminals and teens still in their peach fuzz days into effective suicide bombers. And then there are cleanskins not on any watch list: the 7/7 bombers, the Columbine teens, the Terry McVeighs. Humankind will always have random idiots. What’s changed is that the random idiots are now much more lethal. Bombs have become turnkey, McTerrorism has been franchised.

“We are seeing a terrorist threat that keeps changing,” said Pierre de Bousquet, the director of France’s domestic intelligence service, known as the D.S.T., in an interview in Paris. “Often the groups are not homogeneous, but a variety of blends… Hard-core Islamists are mixing with petty criminals,” he added. “People of different backgrounds and nationalities are working together. Some are European-born or have dual nationalities that make it easier for them to travel. The networks are much less structured than we used to believe. Maybe it’s the mosque that brings them together, maybe it’s prison, maybe it’s the neighborhood. And that makes it much more difficult to identify them and uproot them.” [Link]

One very helpful aspect of the problem is that bombers focus on enclosed spaces because of physics: the power of a blast decreases with the square of the distance from its source. In other words, any blast dissipates rapidly — physics are your friend. Bombers seek out enclosed spaces to amplify the blast and to generate sources of shrapnel. In addition, the spread between any shrapnel packed in the bomb increases rapidly with the distance from the source, reducing the the chance of anyone being struck.

To fix the problem, you must harness that same, powerful technology cycle to beat it. Investing in scanners is the only way up the learning curve: it’s bombs now, but later it’ll be chemical weapons and suitcase nukes. You need to start into that cycle to even have a prayer of keeping up. Right now there’s been an improvement in the price/performance ratio of weapons with no corresponding improvement in detectors. And that’s mainly due to our misplaced priorities.

· Â· Â· Â· Â·

The first lesson of technology design is to limit the problem domain. The reason your inkjet costs only fifty bucks is that it doesn’t have to print in free space, it only has to put ink in a space 8Å“ inches wide. Similarly, the subway is an enclosed space because it reduces costs: you only have to move people on tracks a few feet wide, you only have to air-condition a limited volume of air. Since the NYC subway system already has enclosed entrances to ensure payment, all you have to do is put explosive detectors at those points.

You also must automate the process just to scale. Cities will become ever-denser, subways will move ever-increasing numbers of people, and keeping up with that growth would require an army of expensive, inaccurate human screeners.

Explosive sniffers are already in wide use at airports. Millimeter wave body scanners let you see through clothing to detect bombs. These solutions are expensive right now, but like every other mass market product in the tech cycle, they’ll become cheap over time. You seed competition by buying from multiple manufacturers. The companies get the funding and the volume to manufacture them inexpensively, and pretty soon they’re as much of a commodity as the bombs against which they defend.

The point of technological solutions is accuracy and permanence. History is replete with problems which once dominated the yammering editorial pages, becoming non-issues due to an advance in the underlying technology. In the age of Kevlar, nobody lies awake at night worrying about arrows.

Tube passengers are to have their bodies scanned by machines that see through clothing in an attempt to prevent further terrorist attacks. The millimetre wave imagers will be used to carry out random checks as people enter stations after services resume today… The technology is already used to catch illegal immigrants who hide in lorries at Channel ports…

The scanners can spot the waistcoat bombs usually worn by suicide bombers and automatically send an alert to nearby officers. Unlike other scanners, they can cover crowded entrances without the need for people to be stopped for individual checks… the advantage of millimetre wave was its ability to scan large numbers of people simultaneously and produce an instant moving image…

The system works by measuring the solar radiation reflected by people’s bodies and measuring anything which interferes with the reflection. It can be linked to closed-circuit television cameras that will automatically pick out and follow a suspect until he can be stopped and questioned… the scanners could be fitted to all 270 Underground stations within 18 months. [Link]

Bag searches are ok for a couple of weeks, but without automated scanning they’re a strike against the heart of New York City’s greatness: not food, the arts or the opposite sex, but what underlies it all, speed. If you’ve spent any time in New York, you’ll notice that it’s friendly but highly efficient. People on the street will give you exactly the directions you need, no more and no less. The deli chefs making your sandwiches take pride in being as fast as industrial robots. You could lose a finger just watching them make a bagel. The subway train doors stay open only 7-8 seconds at peak hours. If you miss the train, you know another one’s coming just a couple of minutes later.

Preserve what makes New York City great. Instead of wasting millions of dollars every week on bag search overtime, NYC should invest heavily in automated scanners and push their creators for faster, cheaper and more accurate versions. Every costly week of bag checks is another few scanners foregone and a step away from solving the problem.

20 thoughts on “The profiling myth

  1. Its interesting that one of the 7-7 bombers and 4 of the 7-14 bombers were blacks from East Africa. No one is now clamoring for profiling blacks in the US. Between 7-7 and 7-14 I saw numerous talking heads pontificate on the ‘appropriateness’ of profiling Arabs plus South Asians. It used to be only Arabs but between 7-7 and 7-14 they added South Asians to the suspect category. Funny how after 7-14, none of the same talking heads will dare add black to Arabs plus South Asian category. ‘Mum’ is the word now about the ‘race’ of the 7-14 bombers and how their race should now be profiled.

  2. NY State Assemblyman Dov Hikind weighs in:

    Associated Press, 8/3/05

    “They all look a certain way,” said Hikind, a Democrat from Brooklyn. “It’s all very nice to be politically correct here, but we’re talking about terrorism.”

    What a guy. Hikind is an Orthodox Jew that is publically calling for the government and its agents to single out an entire group of people as criminals and non-desirables based solely on appearance. He should know better.

  3. Yeah that is a perfect analogy – the Holocaust – yeah – no lack of perspective or hysteria there at all.

    Interesting post Manish – personally I think it is only a matter of time before a nice respectable white boy from a middle class home who has converted to Islam carries out the next attack and then its eyes out for the white lads too – as well as blacks and Asians – so everyone has to be profiled.

    The circumstance where I would concede that it has a strong justification is in the instance of specific intelligence giving a time and place or whatever – but these terrorists are sly dogs and canny and if they know one group of people is under intense scrutiny they would switch to the next race of wannabe shaheedis from the ranks of their volunteers.

  4. The shifting face of Al-Queda was just about the easiest thing to predict for us. Most of us “brown” folk already knew we nor Arabs/Muslims aren’t phenotypically monolithic. Morever, Islamsists exist on every continent. You don’t look (fill in social group).

    Manish’s point is well taken. But I’d like to believe random bag checks and increased airport/train security are just one prong in a multi-faceted security approach. And it’s the security feature with most noticeably, scrutinized public face. The authorities were right in pushing for it just in case the London bombings were part of a coordinated wave. Even with no concrete evidence of a plot against NYC, I put forward that the mere introduction of the measures were a success: (1) heightened awareness of the public to stay vigilant (increased paranoia and harassment backlash notwithstanding (2) foiling of possible of copycat attacks.

    We don’t hear about cargo searches, forsenics measures, network interception, et al. because it’s not sexy enuff to make the evenings news or we don’t hear about till a major breakthrough is made.

    Doug Hikind is such a classic archetypcial NY pol, overly sensitive about his own ethnic group, but not so thoughtful when he overreaches: Here is on the Passion of the Christ:

    “This film is dangerous for Jews all over the world. I am concerned that it would lead to violence against Jews.”
  5. That was a very interesting idea! Being a techie myself, I appreciate the analogy with virus attacks, and the technological solution proposed.

  6. Keep in mind that there’s a big gulf between “NO profiling” and “ONLY profiling”

    I’ll readily agree that both “no” and “only” are intrinsically error-prone strategies – particularly if used exclusively. The false positives in the “only” case are well illustrated by Manish above. But that doesn’t automatically mean we swing over to “No”. And particularly it’s close political cousin – “not even things that look like profiling!”

    Even though AQ et. al. can be expected to take profiles into account when recruiting their next batch of Jihadists, it’s nevertheless worth noting that – relative to the general pop – muslim / arab males are still disproportionately involved here. NOT “always”, just “disproportionately”.

    The key, of course, is that to make “Some profiling” useful, all you need are little statistical advantages like these – instances of “disproportionality”. And we should all remember that prevention as opposed to prosecution – a key distinction Tunku Varadarajan notes – is ultimately a statistical game. Even with 99.99% accuracy, when you’re dealing with a system with millions of “tests” per day, that’s still a couple thousand False Positives / False Negatives.

    Like a Bayesian Spam Filter, in the end what you really end up doing is adding up all the little tools that “may” indicate some proclivity and hope you get an aggregate score that works. The key is to not use “profiling” exclusively.

    It’s worth mentioning that it’s (almost) equally scary for law enforcement being pushed to the “No profiling – not even its appearance!” extreme. The false positive in the “only” case is obviously bad (“detained just for being brown”) but the false negative in the “no” case is terrifying (“not detained just for being brown”). You’ve probably seen / heard of one of the few law enforcement success stories leading up to 9/11 – Jose Melendez-Perez

    Customs officer Jose Melendez-Perez stopped the 20th terrorist, who was supposed to be on Flight 93 that crashed in Pennsylvania. Probably because of the shorthanded muscle on that team, the passengers were able to overcome the terrorists. Melendez-Perez did this at great personal risk, because his colleagues and his supervisors told him, “You can’t do this. This guy is an Arab ethnic. You’re racially profiling. You’re going to get in real trouble, because it’s against Department of Transportation policy to racially profile.” He said, “I don’t care. This guy’s a bad guy. I can see it in his eyes.” As he sent this guy back out of the United States, the guy turned around to him and said, “I’ll be back.” You know, he is back. He’s in Guantanamo. We captured him in Afghanistan.

    Terrified of even being accused of profiling, “the system” nearly let a bad guy in against an individual’s gut instinct. Now we don’t know what exactly Melendez-Perez saw “in his eyes” and it’s doubtful that Melendez-Perez could ever elucidate it more than to simply say “gut instinct.” But it’s clear that under the standards of the DoT (much less Lefty “White Racism is Everywhere!” Activists) it could’ve constituted profiling. From the Ivory Tower, it’s hard to construct the tight logical case for why Melendez-Perez did the right thing. But in the messy, organic place where the rubber meets the road, he was almost certainly correct. And clearly no millimeter wave body scanner of any sort would’ve caught him.

    I fear that instead of “no profiling” and “only profiling” we’re gonna be stuck in a place for a long time that’s “things that look like profiling.” Or perhaps a “of course we don’t profile .” Life is rarely fair or perfect and, alas, the terrorists will make it less so. Bastards.

  7. The al-Kahtani / Melendez case sounds like garden-variety immigration enforcement, any agent would’ve turned him away:

    The officials were suspicious because al-Kahtani had little money, could speak no English, and used a one-way ticket. He frequently changed his story and could not adequately explain why he was visiting the United States. Thinking he was likely to become an illegal immigrant, al-Kahtani was sent back to Dubai. [Link]
  8. Profiling has its place, but it isn’t where the arguement heats up at.

    A profile is a part of a giant information mosaic that collectively will help you visualize a better picture.

    Profiling people where you believe a terrorist attack may happen is simply NOT effective. Using profiles to sort out terrorists when researching and attempting to circumvent a crime before it happens IS effective. Just because they are both called profling does not make them the same.

    The profile of a serial murdered is a white male of a certain age. A serial murder is suspected and you have no information on where to begin, well thats a start. Similarly, you have information that suggests terrorist activity may occur, but you are cold on leads, you start with a basic profile.

    Attempting to sort people out at train stations and other soft targets is simply impossible. Instead of blowing himself up inside the train station, he will do so at the checkpoint. Practically speaking, it is impossible to to defend soft targets.

    I agree with Manish on the emphasis of technology. One must take the advantage of the enemy away, which for terrorists is anonymity and concealment. Passive, accurate, and deployable technology is the key.

    Unfortunately both sides of the aisle get it wrong on this one. You’ve got the extreme left screeching how EVERYONE must be randomly checked, just so domestic terrorists can also be nabbed. On the extreme right you’ve got pseudo-nazis wanting to shake down every brown muslim looking dude possible. Neither works. One is massive waste of resources and energy that yields nothing. The other, even though focused, leaves all the room for terrorists to manuever.

    If a plan is in the execution phase for such activity, you’re already too late, particularly for soft targets. Currently, if one is facing such a situation, you really have to hope the guy makes a mistake or luck smiles luckily upon you.

  9. Al Mujahid:

    It used to be only Arabs but between 7-7 and 7-14 they added South Asians to the suspect category. Funny how after 7-14, none of the same talking heads will dare add black to Arabs plus South Asian category.

    Punjabi Boy:

    and then its eyes out for the white lads too – as well as blacks and Asians – so everyone has to be profiled.

    Yes,yes…gimme more. I’d like my profilers equal-opportuniy types. In other words: hum tau doobeiN haiN sanam, tujhe bhi saath le doobengey (come, drown with us)

  10. one thing about terrorists and ethnicities. as i noted before, these radical groups tend to work via networks affects, and so the networks often cluster by ethnicities. ergo, it seems plausible you will see shifts (though not necessarily top-directed) over time through various networks which cluster with various racial groups.

    btw, it’s great living a rural and white part of the country, isn’t it? all those new york liberals are so racist!

  11. The argument for profiling is simple. Let T = terrorist, M = Muslim.

    P(T|M) = .00001 (or less), but P(M|T) = .99 (according to State Department Report on Terrorism). That is, the probability that someone is a terrorist given that they’re Muslim is low, but the probability that someone is a Muslim given that they’re a terrorist in the US is very high (and if you say “Tim McVeigh”, you don’t know what the word “probability” means).

    Given background assumptions on P(M) in the population, you can work out the explicit Bayesian profiling decision.

    The pool of Muslim phenotypes is enormous; they can tap Chechens, Uzbeks, Filipinos, Indonesians, Chinese, Malays, white converts, black Americans, red-haired Kashmiris, blue-eyed Afghans.

    Doubtful that they can draw equally from each pool. This is the core fallacy: the pool of potential Swedish grandmother jihadists is far less than the pool of young brown males from predominantly Muslim countries.

  12. … the probability that someone is a Muslim given that they’re a terrorist in the US is very high…

    Do you have a link for P(M|T) = 0.99? It’s an incomplete metric regardless. There are plenty of bombers who are neither Muslim nor defined as terrorists by the State Department, e.g. Eric Rudolph. The Columbine teens built a bomb, it just didn’t work– that will change as bomb tech gets commoditized.

    Racial profiling can be useful in intelligence work. But once a carrier is at the subway station, there’s no better metric than being able to detect the bomb itself, regardless of what the carrier looks like.

    Doubtful that they can draw equally from each pool. This is the core fallacy…

    For attacks which only need four people to execute, there are plenty of potential recruits from each pool. These are not high-scale or high-knowledge operations. The teens were essentially mules. Last week, Brit intelligence was still arguing over whether the carriers knew it was a suicide op.

  13. Nice post Manish. It does confuse me a bit though. Are you saying it is ok to pick Muslims out? The larger issue seems to be profiling, racial or otherwise. As pointed out by others, non-biased algorithms for profiling might eventually gravitate towards Muslims, which is a mirror of the recent statistics. Humans need to inject political correctness , but not for the sake of political correctness alone. A lot of people don’t seem to understand that most of these malcontents have ended up in that situation precisely because of the treatment dished out to them or their community. You don’t want to exacerbate that. Your bayesian model could come to that conclusion too. What if the likelihood of an otherwise unhappy (of a benign kind) muslim, pushed over the threshold of fanaticism because he was asked to kneel in the gutter and subjected to a humiliating strip search, is diminishingly small. The price we pay for his transformation is far too overwhelming for us to take up simplistic profiling for the sake of convenience and costs. Simply put, a small number multiplied by an enormous number is BIG. We need to get that number down, and that means throwing some money at random searches.

  14. I’ve now gone through the subway / mass transportation system something like 4 times wearing a kurta, and I’ve yet to get profiled. More to the point, only once did I see a cop, so this whole debate seems pretty theoretical to me at this point from a security standpoint. It’s so easy to get into the NYC subway system or the Long Island Railroad. Profiling, random searches, whatever–I wonder if there’s truly a way to reduce the number of terrorist incidents at the last minute (like at a subway turnstile or on the tracks at Penn Station).

    Also, in practice, I’d guess that “random searches” are never totally random, so lets be real and say we’re talking about minimal, subconscious profiling (of some kind or another) vs. overt, systematized profiling.

    btw, if anyone wants to have the $hit scared out of them, watch this documentary on Indian Point. Maybe some of the cops at the turnstiles could be moved upstate a little bit to keep all of Manhattan from becoming uninhabitable? 😉

  15. Perhaps the NYC cops can follow the helpful guidelines laid out for their British counterparts. I’m sure the techniques will help so very much in making their terror sweeps speedy and sensitive all at the same time.

    Here is a sample:

    • Police should seek to avoid looking at unclad Muslim women and allow them an opportunity to dress and cover their heads. • For reasons of dignity officers should seek to avoid entering occupied bedrooms and bathrooms even before dawn. • Use of police dogs will be considered serious desecration of the premises and may necessitate extensive cleaning of the house and disposal of household items. • Advice should be sought before considering the use of cameras and camcorders due to the risk of capturing individuals, especially women, in inappropriate dress. • Muslim prisoners should be allowed to take additional clothing to the station. • If people are praying at home officers should stand aside and not disrupt the prayer. They should be allowed the opportunity to finish

    Maybe they should even call in their raids 24 hours or more in advance, so that the suspects can prepare themselves properly…

  16. btw, it’s great living a rural and white part of the country, isn’t it? all those new york liberals are so racist!

    If there was a New York style terrorist attack in a rural and white part of the country and the attack was carried out by swarthy men, the browns there might have more problems than worrying about the statements by two of the state senators.

  17. Here in the UK we think we are going to have racial profiling but it depends which copper you talk to from BBC News Ethnic groups not ‘search target’

    Police have stepped up their presence since the London attacks Police should not rule anyone out when using stop and search powers to prevent further bomb attacks, Home Office minister Hazel Blears has said. She spoke after British Transport Police suggested young ethnic minority men were more likely to be stopped.

    Ms Blears said she had “never, ever endorsed” the use of racial profiling.

    “You can equally have white people who could be the subject of intelligence”, she told the BBC. “I don’t accept it is right simply to target groups.”

    Ms Blears began a series of meetings with Muslim leaders on Tuesday in an attempt to improve community relations.

    Package of powers

    The BTP had said it did not intend to “single out” any particular community.

    But a spokesman said it would “target the people who we think are maybe involved… it may be disproportionate when it comes to ethnic groups”.

    Exercise this power on the basis of the intelligence available to you and you explain that to communities

    Hazel Blears

    Police warned on stop and search

    In the past Mrs Blears has said she believed innocent Muslims would accept that they may be more likely to be targeted in the search for Islamic extremists.

    She told BBC Radio 4’s Today programme on Tuesday: “I have said time and time again to police in the guidance I have issued that you exercise this power on the basis of the intelligence available to you and you explain that to communities.

    “That is the way you get their trust and confidence.”

    The minister said stop-and-search had to form part of a “package of powers”.

    “Tackling terrorism is absolutely dependent on the confidence of these communities to feel that they can come forward, give information and be part of the fight against this threat,” she said.

    ‘Extraordinary times’

    The BTP remarks prompted the legal adviser to the National Black Police Association’s to say people from ethnic minorities would understand being stopped by police after the London bomb attacks.

    Ch Supt Ali Dizaei said this would be true as long as people were treated courteously and given an explanation.

    “These are extraordinary times and people are committed from all communities to work together with the police in order to sort this problem out,” he said.

    But civil rights group Liberty said terrorists could just use bombers with a different profile to avoid targeted stops by police officers.

    And Muslim Council of Britain spokesman Inayat Bunglawala warned the strategy could be “counter-productive”.

    “While it is understandable that the police need to undertake every step to thwart would-be bombers it is crucial that they do not unnecessarily alienate and stigmatise an entire segment of society,” he said

  18. Just as a side small point – Timmothy McVey already did the white boy terrorist as did the American Taliban John Walker – So White american has had its opportunity to wake up – they just don’t want to.