intellectual self-defence psychology

Facebook’s persuasion architecture and human reason

Originally published 2017-11-12 at

Facebook is a specific, known, threat to democracy, not a general unknown threat to our capacity for rationality

@Zeynep Tufekci has a TED talk ‘We’re building a dystopia just to make people click on ads’. In it she talks about the power of Facebook as a ‘persuasion architecture’ and she make several true, useful, points about why we should be worried about the influence of social media platforms, platforms which have as their raison-d’être the segmentation of audiences so they can be sold ads.

But there’s one thing I want to push back on. Tufeki’s argument draws some of its rhetorical power from a false model of how persuasion works. This is a model in which persuasion by technology or advertising somehow subverts normal rational processes, intervening on our free choice in some sinister way ‘without our permission’. I’m not saying she would explicitly endorse this model, but it seems latent in the way she describes Facebook, so I thought it worth bringing into the light, pausing just for a moment to look at what we really mean when we warn about persuasion by advertising.

Here’s Tufeki’s most worrying example: targeted Facebook ads aimed at mobilising, or demobilising voters, which are effective enough in changing voter turn out to swing an election. She reports an experiment which tested a fairly standard ‘social proof’ intervention, in which some people (the control group) saw a “get out and vote” message on Facebook, and others (the intervention group) saw the same message but with extra information about which of their friends had voted. People who saw this second message were likely to vote (0.4% more likely). Through the multiplier effect of the social networks they were embedded in, the researchers estimate that 340,000 extra people voted that otherwise wouldn’t have.

Now 340,000 votes is a lot, enough to swing an election, but it would be a mistake to think that these people were coerced or tricked into acting out of character by the advert. These were people who might have voted anyway, and the advert was a nudge.

Think of it like this. Imagine you offer someone an apple and they say yes. Did you trick them into desiring fruit? In what sense did you make them want an apple? If you offer apples to millions of people you may convert hundreds of thousands into apple-eaters, but you haven’t weaved any special magic. At one end, the people who really like apples will have one already. At the other, people who hate apples won’t ever say yes. For people who are in between something about your offer may speak to them and they’ll accept. A choice doesn’t have to originate entirely from within a person, completely without reference to the options presented to them, to be a reasonable, free, choice.

No model of human rationality is harmed by the offer of these apples.

Our choices are always codetermined by ourselves and our environment. Advertising is part of the environment, but it isn’t a privileged part — it doesn’t override our beliefs, habits or values. It affects them, but it no more so and in no different way than everything else which affects us. This is easy to see when it is offers of apples, but something about advertising obscures the issue.

Take the limit case — some political candidate figures out the perfect target audience for their message and converts 100% of that audience from non-voters into voters with a Facebook advert. Would we care? What would that advert — and those voters — look like? They would be people who might vote for the candidate anyway, and who could be persuaded to vote for someone else by all the normal methods of persuasion that we already admit into the marketplace of ideas / clubhouse of democracy. They wouldn’t vote for a candidate they didn’t sincerely believe in, and the advert wouldn’t mean that their vote couldn’t be changed at some later point, whether by another advert, by new information, by arguing with friend or whatever.

There are still plenty of reasons to worry about Facebook:

  • Misinformation —how it can embed and lend velocity to lies.
  • Lack of transparency — both in who is targeting, who is targeted and why.
  • Lack of common knowledge —consensus politics is hard if we don’t all live in the same informational worlds.

Tufeki covers these factors. My position is that it hasn’t been shown that there is anything special about Facebook as a ‘persuasion architecture’ beyond these. Yes, we should worry something with the size and influence of Facebook, but we already have frameworks for thinking about ‘persuasional harm’— falsehoods are not a legitimate basis for persuasion, for example, so we are particularly concerned to hunt down fake news; or, it is worrying when one interest group controls a particular media form, such as newspapers. Yes Facebook persuades, but it doesn’t do so in a way that is itself pernicious. Condemning it in general terms would be both misplaced, a harm to any coherent model of citizens as reasonable agents, and a distraction from the specific and novel threats that Facebook and related technologies constitute to democracy.

intellectual self-defence

Too smart for facebook?

Originally published 2016-04-07

It interests me how the generations younger than me have changed their social media habits.

Whilst the professional-age world seems happy with twitter and facebook, younger people have moved their attentions elsewhere (you can google for the evidence of this, here I am relying on a completely unsystematic sampling from biased personal experience).

Is this because they want to avoid the social media that their parents are on? Because they are driven by a relentless and fickle need novelty and illicit thrills? Maybe in part, but there’s also a logic to the specific social media networks which are in vogue.

  1. WhatsApp offers end-to-end encryption. As well as one-to-one messaging you can use group chat for the sort of thing you might use twitter @ messaging or facebook chat, with better privacy.
  2. YikYak offers anonymous location-based chat. Obviously they’ll bust you the moment the law wants your details, but if you are just after hook-ups or advice about embarrassing personal problems this is good enough anonymity to protect your identity from your employer (or partner, or friends).
  3. Snapchat messages disappear after a number of seconds, offering the illusion of impermanence for web-messaging. So different from message boards or most IM, where casual banter from years ago is stored for eternity.

So in this way new social media offers some sort of address to three of the big threats of life online: lack of privacy, irrevocable binding of your online actions to your offline identity, and permanent records of even your tiniest online actions.

Maybe The Youth aren’t so fickle and short sighted in their use of social media after all.

intellectual self-defence

Epistemic crisis

In this article, With impeachment, America’s epistemic crisis has arrived, David Roberts, argues that impeachment proceedings against Donald Trump will be the stress test for a right-wing alternate-reality. This alternate reality has been fomented over decades, through the cultivation of a right wing media ecosystem – from talk radio, think tanks and talking points. Now, Roberts argues, the Fox News junkies have taken over the White House and are going to try and get away with carrying on as if the crimes Trump committed are not obvious and impeachable – throwing up enough doubt and uncertainty over the proceedings that they can get away with it.

Two elements of this Roberts highlights are:

Tribal epistemology: “… when tribalism comes to systematically subordinate epistemological principles.”

And strategical exploitation of consensus: “The right has hacked the cognitive biases of voters and reporters..[there is] a strong tendency, especially among low-information, relatively disengaged voters (and political reporters), to view consensus as a signal of legitimacy. It’s an easy and appealing heuristic: If something is a good idea, it would have at least a few people from both sides supporting it.” Robert’s argument is that by denying consensus they deny legitimacy to mainstream (i.e. Democrat) positions, and keep alive the legitimacy of the alternative (i.e. Republican) reality.

As a wail of despair, or prophesy of doom, I find this piece as appealing as the next liberal, but the lie in it is revealed by its impracticality. The piece doesn’t tell us what to do about hyperpartisan alternative realities, and it can’t, because at root it subscribes to the same model of human irrationality it decries.

The idea that epistemology can be subverted to tribalism implies that there are ways of knowing – or people who know – which aren’t influenced by their social group. The idea that the right-wing base have had their cognitive biases hacked implies that the rest of us have resisted this, or never had a vulnerability in those biases in the first place.

I’m not claiming that there aren’t biases, or tribalism. Reality exists, errors exist and sometimes tribalism does provoke the sacrifice of accuracy. But a useful view of these things will tell us what do when epistemological realities seem to diverge.

My problem with the “alternative reality” account is that is plays to a model where people who disagree with us are irrational. And once you have decided that people are irrational it blocks any possibility of rational engagement – why try to persuade these people (the model suggests), isn’t it hopeless?

intellectual self-defence

The organising functions of fake news

Lots has been written about the dangers of misinformation, including how to innoculate people against it, or debunk it once people have swallowed it. A common theme is the idea that the risk of fake news is that people will believe it.

But fake news has other functions besides getting people to believe something that isn’t true, and understanding these other functions may be as important as understanding its belief-altering properties.

Topic change. When a blatent untruth is put out, it becomes something to talk about, orienting attention. This can be deliberate – as in the dead cat strategy.

Energy sink. Untruths evoke refutations, further orienting attention and redirecting the energies of your opponents.

Loyalty test. Propaganda doesn’t exist to be believed, it exists as a flag around which people must organise, or display their opposition. The more blatant the untruth the stronger the test of loyalty.

Status display. What better display of power than asking people to humiliate themselves by affirming an untruth merely because you said it?

Misinformation is about power. As long as we try and deal with it as a class of information we’ll be vulnerable to its use by the powerful.

More: “On Bullshit” by Harry Frankfurt

Update: Fake news is a terrible bucket term which is used to cover everything from incorrection information to state propaganda, via all sorts of misinformation.

Related: Podcast: YANSS 157 – The psychology behind why people don’t speak out against, and even defend, norms they secretly despise – which covers, inter alia, norm engineering

intellectual self-defence

Seeking perspectives: how to arbitrate among our desires?

Because our values have arisen in a process of debate, inference and generalisation, they are no longer even distant consequences of our basic needs. Our nature arises from choices that were not determined by our biological make-up. It is enabled, but not determined, by biology.

So writes Ronnie de Sousa in an essay for Aeon called ‘Natural-born existentialists

I accept that nature doesn’t tell us what we ought to desire. I also think that some of what feel like our most essential desires are social products. Here’s an extreme example of that: If I was born a viking warrior I would want to die in battle (or at least want to want that). Viking-me would recognise that desire as defining of my self, but from our modern standpoint we also recognise that it is entirely an accident of history. Those of us who want to die in battle, or to avenge their honour, we seek to rehabilitate, not encourage.

If some of our most strongly held desires are social products, and there is no divine or biological nature which tells us which desires are good or bad, how do we know which desires should be the foundation for our selves, and which we should disavow, and so strive to un-want?

Possibly related: moral dumbfounding. When we cannot explain our intuitions, when is this evidence that they are, or should be axiomatic, and when is it evidence that we have no sound basis for what we belief and we should abandon our beliefs?

Clues, perspectives, historical and empirical notes welcome!

intellectual self-defence quotes

Popper on scientific objectivity

…science and scientific objectivity do not (and cannot) result from the attempts of an individual scientist to be ‘objective’, but from the friendly-hostile co-operation of many scientists. Scientific objectivity can be described as the inter-subjectivity of scientific method…

Two aspects of the method of the natural sciences are of importance in this connection. Together they constitute what I may term the ‘public character of scientific method’. First, there is something approaching free criticism. A scientist may offer his theory with the full conviction that it is unassailable. But this will not impress his fellow-scientists and competitors; rather it challenges them : they know that the scientific attitude means criticizing everything, and they are little deterred even by authorities. Secondly, scientists try to avoid talking at cross-purposes…In the natural sciences this is achieved by recognizing experience as the impartial arbiter of their controversies. When speaking of ‘experience’ I have in mind experience of a ‘public’ character, like observations, and experiments, as opposed to experience in the sense of more ‘private’ aesthetic or religious experience; and an experience is ‘public’ if everybody who takes the trouble can repeat it…

This is what constitutes scientific objectivity. Everyone who has learned the technique of understanding and testing scientific theories can repeat the experiment and judge for himself.

Karl Popper, The Open Science and Its Enemies, Chapter 23 (p217-218, Vol. II)

intellectual self-defence

Dan Ariely teaches you how to say “no”

Dan Ariely has this autoreply keyed up, so that by typing a single word he can send you this email:


This is a very interesting and important topic and close to my heart. But sadly, my life is so full these days that I don’t even have time for the things I’ve already promised to do. I even have a few of my own projects that I haven’t been able to find the time to work on. Not to mention that I have projects with close collaborators that I promised to work on and haven’t been able to.

So while this sounds exciting, I know at the end of the day, it would just put me in a deeper spiral of obligation and place even more constraints than I have now. I hope you understand but I have to decline.

Wishing you all the best and lots of luck.


intellectual self-defence

Please no rhetorical questions

Rhetorical questions are generally difficult to deploy effectively in a piece of writing. When debating with someone over the Internet their use is particularly ill-advised, for at least three reasons.


Reason #1: Would clarity benefit from rephrasing as a statement?. Clarity benefits if you rephrase as a statement.

You can probably rephrase what you mean as a statement, which will be clearer and more explicit. Go on, try it. Rephrase your rhetorical question, and add your answer. The person you are arguing with now has less chance of missing what you are really trying to say. Communication is hard. Communicating with someone who disagrees with you is extra-hard.

Clarity benefits if you rephrase rhetorical questions as statements.


Reason #2: Isn’t it obvious what I mean?. It isn’t obvious what you mean.

In our face-to-face communication we rely on a bunch of non-verbal channels to convey additional information. This is crucial when what we mean is different from exactly what we say. In written language these channels – tone of voice, gesture, etc – are missing. Moreover you can be less certain of what your audience knows or assumes. For these reasons, things like sarcasm are notoriously difficult to pick up on over the Internet. Like sarcasm, rhetorical questions assume a shared understanding to allow you to convey what you mean without stating it explicitly. So, by using a rhetorical question you are taking a risk that your real meaning will be understood by your reader. That is an unnecessary risk, don’t take it.

It isn’t obvious what you mean. Rhetorical questions ask your reader to infer your real meaning. Be kind, just tell them.


Reason #3: Don’t we all know the answer to this one?. We don’t all share the same assumptions.

Rhetorical question assume a shared framework, something you cannot assume when you are disagreeing with someone. If you want to focus on the substantive disagreement you have with someone, don’t dance around it with rhetorical questions. Say what you mean so your reader can respond based on that, rather on what they have to guess about the inspiration for your questions.

The clue is in the name – rhetorical questions are for rhetoric, persuading an audience, a passive mass rather than a particular individual who has the right of reply. If you are arguing on the Internet you should have the courtesy not to treat a singular reader or small group of readers who want a right of reply as an audience. They have a considered opinion which you need to listen to. They may even want to understand what your considered opinion is. You do them – and your opinion – a disservice if you deploy rhetorical questions, which obscure your meaning in favour of self-satisfied point-scoring.

No, we don’t all the know the answer to your rhetorical question. We don’t agree, so we probably don’t share common assumptions. Rhetorical questions assume we do, so you should particularly avoid them in argument.

intellectual self-defence

physics as necessary but not sufficient to explain causation in complex systems

Ellis (2008) has an account of the nature of top-down causation in complex systems. He says there are at least five established forms of top-down causation – ways in which the higher level properties of a system can have causal power over the lower level elements (in contrast to the reductive view which would say that all behaviour of a system can be explained by the lower level properties : the physics is primary approach).

For me, the value of the paper is a single thought: physics provides the necessary, but not sufficient conditions, for explaining human behaviour. Without our physical natures – our undeniable existence as material beings, governed by four primary physical forces – we would not be, but to understand our being you need more than an account of the component materials and their governing forces.

Probably an obvious point, but succinctly put and useful to have on hand when faced with reductionists, or when trying to figure out the proper role of human agency in a strictly physical universe.

Ellis, G. F. (2008). On the nature of causation in complex systems. Transactions of the Royal Society of South Africa, 63(1), 69-84.

intellectual self-defence

The Moral Arc

I am seeking suggestions for things to read on a specific topic, which I am struggling to articulate. I would like to read an analysis of how individuals understand their own moral development. Moral philosophers have accounts of what is moral, how it should be understood. This lacks the first person perspective I want to explore – I want to read something that takes seriously the subjective moral life as it is, not as it should be. Experimental philosophers have accounts of differences in people’s responses to moral dilemmas. This is too static – I want to read something that takes seriously our ability to change morally, and particularly to be agents of our own changes in belief. Biographies, particularly of spiritual or political figures, have first person accounts of moral change – why people lost their faith, or changed faith, in deities, parties or principles – but these don’t allow the comparison across people that I’d like.

I wonder if such a book exists. Something like “In a different voice“, but with more emphasis on adult development, or The Intellectual Life of the British Working Classes, with a specific focus on moral change.

The motivation is to escape the implicit model of many psychological accounts, which portray people as passive information processors; at their worse stimulus response machines, but even at their best as mere suboptimal rational agents. I’d like to think more about people as active moral agents – as having principles which are consciously developed, seriously considered, subject to revision, passionately defended and debated. Then, of course the trick is to design empirical psychology research which, because it takes this perspective seriously, allows this side of people to manifest rather then denying or denigrating it.

intellectual self-defence technical notes

Permanent Zero

‘Email overload’ is one of those phrases everyone thinks they know the meaning of: “I get too many emails!”. Last autumn I met Steve Whittaker, who has a reasonable claim to have actually coined the phrase, way back in 1996. He explained to me that the point wasn’t to say that we get to much email, but that email is used for too many different things. We’re using it to send messages, receive messages, get notifications, schedule tasks, chat, delegate tasks, archive information and so on forever.

Shifting the focus from email as number of individual messages (too many!), to email as functions (still too many!) lets you see why the ‘Inbox Zero‘ idea doesn’t quite work. Inbox Zero appeals to my sense of being in control over my email, and it is better for me than not having a righteous scheduling system for my email, but it doesn’t split the multiple functions for which I use email.

Now, for you today, I’d like to share my newest strategy for managing my email, which is inspired by Whittaker’s ‘Email overload’ distinction.

The first thing to do is to separate off the single largest function of email – receiving messages – from the others. You need to stop emails arriving in your inbox, leaving you free to send and search without distraction. Create a filter and have all incoming mail moved to that folder. Now stare in satisfaction at “You have no new email!” in your inbox. Schedule a time to go to your received mail folder and kill as many emails as you can, using your favourite inbox zero strategies (protop: if you send emails at 4.30 you minimise the chances of someone replying that day). Now your workflow which only involves sending messages and dealing with old messages isn’t tangled up with the distraction of receiving new messages.

Next, separate off all email that isn’t personal correspondence. Set a second filter which removes all email without your email address in the ‘to’ or ‘cc’ fields. These are circulars. You can scan the titles and delete en mass.

If you are using gmail, you can import these filters (after editing to make relevant adjustments).
remove from inbox, unless sent to ‘exception’ address
remove all circulars
Right click to ‘save as’, they won’t show up in a browser. Note that my new folders begin with ‘A_’ so they are top of my alphabetised folder list.

intellectual self-defence

Values vs Finances

I attended a University meeting recently, an open forum to discuss our strategy and vision. My small group spend most of its time talking about the conflict between values and finances. Values we might aspire to – things like helping fight climate change – and finances – the constraints from ‘the bottom line’, the need to recognise the costs of different actions. Something about how the group settled on this dichotomy disturbed me. It wasn’t that there weren’t intelligent people in the group, who make good points, but I left with the inarticulate feeling that there was something wrong with the framing around the discussion we had. I’ve been thinking about it for over a week, and I’m now a bit closer to figuring out some of the problems with the idea that values come into conflict with finances.

The first problem with this false opposition is that it positions values as a luxury, something we can only afford to think about if we service the necessity of finances. Rather, values are the necessity – and prior to any consideration of finances. How can you decide on any action unless you know what you want, and what you value? This is impossible for a person, or an institution. Sure, we have some givens – Universities teach and do research – but I’d argue they reflect implicit values which we need to articulate. Only once we know what values we share can we then start to decide what we want to do, and only then can we start to cost those actions.

The second problem with putting finances in opposition to values is that it reifies an abstract notion and gives the false impression that ‘finances’ are somehow simple and concrete. In fact, even if the University unwisely adopted the corporate directive to maximise profits that does not unpack into a clear decision strategy. Over the complex space of possible timescales, and possible strategies, and possible changes in the environment, it isn’t clear at all what actions will maximise profits. You need a sense of your mission even if you are trying to maximise profits – which we aren’t.

My sense is that in the discussion people referred to ‘finances’ as a proxy for external constraints. We’d like to teach for free, but lecturers and buildings cost money etc. My objection to vaguely referring to ‘finances’ is that it stops detailed discussion of specific external constraints – not all of which are financial (for example we’d like to recruit the best research staff from around the world, but visa restrictions hamper this).

My third and final issue with the opposition of values and finances is that it positions values as flexible – things we’ll set within whatever wiggle room finances affords us – but finances as fixed. But Universities are big enough players to change the environment within which they operate. We all are, especially though the power of collective action. Fees, funding, visa restrictions are all negotiable. We, as a society, and as a University which should play a role in shaping society, decide on how these things work. We should articulate our values and take part in doing that. I reject a fatalistic submission to the way the world is – which is often what homage to finances reflects. A ‘there is no alternative’ nihilism which promotes passivity.

idiocy intellectual self-defence

The Natures of Explanation

(Attention conservation notice: mostly me trying to work out what I mean. If you know, feel free to get in touch)

Explanation is not a zero-sum game. You can add additional explanations without negating existing explanations. The loss of life after the flooding of New Orleans was due to Hurricane Katrina. And it was due to climate change. And under-investment in the levees. And a history of social exclusion based on race and class. All these explanations are true, there is no explanatory exclusivity.

I am reading Bruno Latour’s “Science in Action” where he gives the best (only?) account I have seen of how any explanation can be countered or superseded by subsequent explanations. Scientists seek to settle claims – to generate “black boxes” of fact, in Latour’s terms – but the process of scientific debate sees a flux of competing explanations. An experiment by A said X. But two experiments by Y said not-X. But Y isn’t using the correct equipment, of course his experiments give the wrong results. But X’s equipment is biased to give the answer X, Y has to use non-standard equipment. But Z has shown not-X with A’s equipment for sub-case Z. And so on. Explanations seek to settle, but can always be weakened by subsequent explanations which qualify, reframe or negate. It is not just that subsequent claims diminish our confidence that X is the case, on some linear scale where 0>confidence>1. Instead, there is a fundamental uncertainty in the very metrics we are judging.

We seek to define or find (domains) where exclusivity applies. Responsibility and blame feels like a domain where exclusivity applies – almost by definition, because we want it to apply. If it was my fault it is not your fault. We want blame to sum to 1, so that even in complex cases we sort through the responsibility of all involved an apportion a limited amount of blame to each party.

Obviously, when non-exclusive explanations originating from science are used in the moral domain, it is natural for people to interpret them exclusively. If your brain or your environment made you commit a crime, it is not your fault. In a similar way – perhaps essentially similar – freedom of the will is often talked about as an exclusive property. Is your choice at the moment free OR is it pre-determined? This is a fundamental misconception, in my opinion.

You need a tolerance for ambiguity to deal in non-exclusive explanations. Usually we seek to find a restricted domain where we can argue over explanations which are, temporarily, exclusive. Is it nature or nurture? Is dyslexia caused by cerebellar dysfunction or magnocellular pathway dysfunction? For the non-restricted domain the ground can always shift underneath you. Someone can come along a redefine any element of what you are arguing about, including the tools of argument themselves.

intellectual self-defence quotes

The Principles of Newspeak

The purpose of Newspeak was not only to provide a medium of expression for the world-view and mental habits proper to the devotees of IngSoc, but to make all other modes of thought impossible. It was intended that when Newspeak had been adopted once and for all and Oldspeak forgotten, a heretical thought — that is, a thought diverging from the principles of IngSoc — should be literally unthinkable, at least so far as thought is dependent on words. Its vocabulary was so constructed as to give exact and often very subtle expression to every meaning that a Party member could properly wish to express, while excluding all other meaning and also the possibility of arriving at them by indirect methods.

From the foregoing account it will be seen that in Newspeak the expression of unorthodox opinions, above a very low level, was well-nigh impossible. It was of course possible to utter heresies of a very crude kind, a species of blasphemy. It would have been possible, for example, to say Big Brother is ungood. But this statement, which to an orthodox ear merely conveyed a self-evident absurdity, could not have been sustained by reasoned argument, because the necessary words were not available. Ideas inimical to Ingsoc could only be entertained in a vague wordless form, and could only be named in very broad terms which lumped together and condemned whole groups of heresies without defining them in doing so.

George Orwell, ‘The Principles of Newspeak’, the appendix to his ‘1984’ (1949)

books intellectual self-defence

books that make you feel like a genius

There’s a nice paragraph in Camilla Power’s book review in the time Times Higher Ed:

While there are interesting ideas here in a random scatter of cases and anecdotes, the trouble is that it makes the reader feel equally random: scatterbrained, as if you’ve been doing idle searches on Google or browsing Wikipedia all day. The kind of theoretical coherence found in the elegant, simple propositions of Richard Dawkins’ The Selfish Gene or Amotz and Avishag Zahavi’s The Handicap Principle: A Missing Piece of Darwin’s Puzzle – books that made you feel like a genius, armed with a new perspective on the world – is not evident.

Power has captured what is wrong with so much popular science writing, and what is right with those books I really value

intellectual self-defence politics

what the world needs: buyafuckingshovel for news

My worry is that there is so much news that that it is hard to keep attention on the things we think are important. Or, rather, the deluge of information means that we’re more likely to accept someone else’s prioritisation of what is important. (Did you notice how the Guardian’s liveblog of the financial crisis was dropped to make room for the liveblog of events in Libya?)

Similarly, there’s a dark thought I sometimes have, where everybody will know the truth about some of the great ills of our society but nobody will have time or will to do anything about them. Something like a cross between a “somebody else’s problem field” and a late-modernity version of 1984 (it won’t be that everybody knows we have always been at war with Oceania, or really does love big brother; instead everybody will know these are lies, but they will be lies we live by rather than fight). In this grim meathook future we’ll know we’re decimaiting non-human species, or that Tony Blair lied about Iraq, or that our government is eroding vital civil liberties and we’ll care, but we just won’t get round to doing anything for some reason (oh look at this!)

A small example would be the family served with an eviction notice by Wandsworth Council because the son was arrested in the recent London riots. Not even found guilty, just charged! Now I heard this, and I thought it was scandal and symptomatic of a deep problem with the way some of us think about justice. But apart from the excuse of writing this blog post, I never followed it up, never found out if the reported headline reflected the truth. I hoped, vaguely, that someone was fighting this decision. Thought it was probably illegal. Thought a bunch of things, but basically got on with my life. Wouldn’t it be nice if, once something in the news caught my attention, something I decided was important to me, there was an easy mechanism for bringing follow ups on that story to my attention.

What I think I want is something like for news stories. I want a personalised sidebar on the news website I use, and I want it filled with follow ups on the stories that I tag as important to me. This tagging should be as easy as ‘liking’ something on facebook. Now I can get joined up news, which will give me a more coherant and less vulnerable to bias view of the world, and also ensure that I can work with the media to focus on what really matters to me, rather than just the enless succession of the ten thousand things.

Dear, can you fix it for me?

intellectual self-defence psychology

For a theatre of the mind

“Neuro” is fashionable these days, from neuroethics to behaviour change (one of which is philosophy, and the other of which is psychology, but both are promoted on their connection with neuroscience). Something which is under-discussed is that psychology has a rich set of fundamental, and different, perspectives on how we ought to think about the mind and brain. These compete with, and complement, each other. The one you adopt will dramatically affect how you read a situation and the “psychological” solutions you are inspired to propose. Probably most people are aware of the neuroscience perspective, and the associated worldview of the mind as a piece of biochemical machinery. From this we get drugs for schizophrenia and brain scans for lie detection. This is the view of the mind which is ascendant. Probably, also, most people are vaguely aware of the Freudian perspective, that dark territory of the undermind with its repressed monsters and tragic struggles. From here we get recovered memory therapy and self-esteem workshops for young offenders. Although people will be aware of these perspectives, will they also be aware of the contradictions between them, and the complements, or the fact that both are viewed by some professionals in psychology as optional, or even harmful, ways of thinking about the mind? And what about the chorus of other perspectives, not all necessary contradictory, but all catalysing insights into mind and behaviour; evolution, cybernetics, cognitivism, situationism, narrative approaches, dynamic systems theory. Each of these will not just give you different answers, but promote entirely different classes of questions as the central task of psychology.

I’d love to work on an theatre or exhibition piece about conceptions of the mind, something which dramatised the different understandings of mind. I think it could be a freshing change from a lot of “art-science” pieces about psychology, which unthinkingly accept the cog-neuro consensus of anglo-US psychology and/or see their purpose as bludgoning the public with a bunch of information they have decided “people should know”. Something about perspectives, rather than facts, would inherantly lend itself to art-dramatic intepretations, and open a space for people enage with how they understand psychological science, rather than being threatened, as is so common, with what scientists thing they should understand.

intellectual self-defence

Enemy Leaflets Fall from Above (Leunig, 1982)

Click for the full cartoon

intellectual self-defence systems

choice is not preference

There is a beauty to the arrangement whereby a cake is shared by one of us dividing it and the other choosing which part they want. The person dividing doesn’t know which part they’ll get so they have every incidentive to make fair shares. They say that John Rawls took this as inspiration for his philosophy of how a just society should be organised (but I don’t know enough about that).

But the cake cutting example only works for a world where the cake is homogeneous and the two cake-eaters have identical preferences (in this case, to have as much as possible). Imagine a world where the cake has a fruit half and a nut half, say, and I have two cake-eaters, A and B. A likes fruit and nut equally, she doesn’t care. B is allergic to nuts. Now the game of “one cuts, one chooses” doesn’t work. If A cuts she will slice the cake in half and be happy with whichever half she’s left with, but B better hope that A makes a half which is entirely fruit, otherwise she’ll be forced to make a choice between two bits of cake, some of which she can’t eat. B is at no risk of losing out, A is at substantial risk. If B cuts first, she might consider cutting the cake into a nut half and a fruit half, but then she has to hope A chooses the fruit half. And she might cut the cake into mixed halves an put up with a portion she can’t eat (but ensuring B only gets half the cake). The game-theoretic solution is probably to cut the cake into a larger, nut-plus-small-amount-of-fruit, half and a smaller, just-fruit, half. A will choose the larger half. A definitely wins, B loses out.

The solution whereby A and B both have half, and both enjoy their halves equally (ie B gets the fruit half) is simple, but enreachable via this sharing game.

I’m reminded of an experiment I think I read about in George Ainslie’s Breakdown of Will (don’t have the book to hand to check, so apologies for inaccuracies. We can pretend it is a thought experiment and I think it still makes the point). There’s a large long cage with a lever that opens a door at the other end. If you are a pig it take 15 seconds, say, to run from the lever to the door. After 20 seconds the door closes, so you get to eat your fill for 5 seconds. One pig on her own gets regular opportunities to feed, as well as plenty of exercise running backward and forth. Now imagine a big pig and a small pig. The big pig is a bully and always pushes the small pig off any food. In a cage with normal feeding arrangements the big pig gets all the food (poor small pig!). But in this bizarre long cage with the lever-for-food arrangement, a funny thing happens. The big pig ends up as a lever pressing slave for the small pig, who gets to eat all the foot.

To see why, we need a game-theory analysis like with the cake example. If the little pig pressed the lever, the big pig would start eating the food and the little pig wouldn’t be able to budge her. There’s no incentive for the little pig to press the lever, she doesn’t get any food either way! The big pig, however, has a different choice : if she presses the lever then she can charge down to the food and knock the little pig out of the way, getting 5 seconds of food. It’s worth it for big pig, but the outcome is that she does all the running and only gets a quarter of the food.

This suprising result is none the less a ‘behaviourally stable strategy’, to bastardise a phrase from evolutionary game theory.

Bottom line: minimally complex environments and heteogenities in agents’ abilities and preferences break simple fairness games. In anything like the real world, as Tom Slee so convincingly shows, choice is not preference.

intellectual self-defence science

How Science Really Works

I want to make a radio documentary about how science really works. The popular imagination has been captured by a model of science which is incomplete and unhelpful. Science doesn’t produce neutral facts, it is process whose very nature is contested within the institution of science as well as from outside. Science is a complex social process, and may not even be a single unified thing.

This documentary I’m imagining would start in a University bar on a friday night, were we could hear some scientists talk about work in the lab in the way scientists all over the world do, not in the language of journal papers, grant applications and popular TV features, but as the work which they know intimately, with its set-backs, rivalries and esoteric rewards. We’d then visit a few important thinkers to get some vital alternative perspectives on how science works:

Steve Fuller from Warwick could tell us about the social construction of knowledge, about how science rewrites the history of discoveries to present an ideal of its process as logical and inevitable when in fact is it accidental and contingent. Someone could outline Feyerabend’s “Against Method” and we could see some scientists get irate at his deconstruction of the sacred cows of the naive, traditional model of how science works (which, in my experience, is what tends to happen when you throw Feyerabend at them).

Terence Kealey, VC of Buckingham University and author of “Sex, science and profits” will explode the myth that publicly funded research is good for the economy and outline his idea that “there’s no such thing as science, just scientists”.

Ben Goldacre will take us into the murky world of pharmaceutical research and show us the ways industry funding can distort “pure science”.

Finally, we tackle science and politics, talking to the climate researchers at the centre of the “Climate Gate” email scandal and show how the mistaken ideal of “science as objective” gets in the way of a proper understanding of the role of science in political debate. (Basically, my argument is that an overly idealised model of science leaves open the rhetorical space for an unhelpful cultural relativism, whereby the critical theorists can claim that science is just a social construction and the political fringes feel they can contest scientific consensus with a GCSE biology and the will to believe). We’ll talk to Jim Manzi who will outline his idea of causal density, showing why applying the scientific method to problems of society will not be as straightforward as the cheerleaders of scientific rationalism assume.

Now, who would like to make this documentary with me?

(NB I have not sought the involvement/permission of the people named in this post!)

See also
Emotional Cartography book launch talk
The Reality of Culture
The Choice of Facts

intellectual self-defence misc

Sorry No Ducks (a cartoon by Michael Leunig)

books intellectual self-defence quotes

Moral Clarity by Susan Neiman

Susan Neiman’s “Moral Clarity – a guide for grown-up idealists” (2009) is a passionate and literary book about moral reasoning and the achievements of the Enlightenment (especially Kant). The book contains fantastic and acute re-readings of the myths of Job and Odysseus, as well as plenty of examples of Neiman’s own moral clarity – she has a great analyst’s knack of being able to articulate clearly and succinctly exactly what was so pernicious about many of the arguments and actions of the neocon government under Bush. Recommended.

“The Enlightenment gave reason pride of place, not because it expected absolute certainty, but because it sought a way to live without it”

books intellectual self-defence politics psychology

Ad Nauseam

adnauseam I am reading Ad Nauseam: A Survivor’s Guide to American Consumer Culture, edited by Carried McLaren and Jason Torchinsky. The book is a funny, smart and sometimes shocking collection of articles from Stay Free Magazine and blog. I first came across Stay Free when I was researching the psychology of advertising and was impressed by their sophisticated take on how adverts affect consumers’ decision making. They discuss in Ad Nauseam how advertising is often misunderstood, with people relying on an intuitive ‘Advertising doesn’t effect me’ view or swinging to the opposite extreme of the ‘Sinister Advertisers Manipulate Consumers with their Mind Control Tricks’ position. Both positions distract from the very real, but not magical, power of advertising.

The book has a great discussion of Wilson Bryan Key’s Subliminal Seduction, the book that launched the idea that subliminal, and often sexual, figures are embedded in random features of adverts such as in ice cube shadows. The idea of these ’embeds’ is nonsense, of course, but great fun to look for and a great distraction from the real persuasive content of the advert. The book also has a chapter on the origins of modern advertising practice in 19th century pharmaceutical advertising (the manufacturing of ailments for which ready made ‘cures’ can be sold has been covered by Vaughan on before, in relation to the mental health). Packed with critical analysis of the advertising industry, more informative history and some shocking examples of how consumerism has worked its way into many aspects of our daily lives, this book is essential intellectual self-defense, managing to be critical and aware without ever being sanctimonious or hysterical.

Cross-posted at

academic advertising intellectual self-defence

Manipulation in Education

There is a fundamental power asymmetry in education. Teachers understand what they are teaching, learners do not. Learners, by definition, cannot have a full appreciation of what they are about to learn, of its value and of how it will change them. If they did, they would not be in the position of learners. The only way you can ignore this is if you are mislead into accepting the banking metaphor of education (education as information transmission, teachers as content providers, students as receptacles to be filled).

One implication of this power asymmetry is that the authority and responsibility of the teacher cannot be abdicated. Students should not be left to ‘decide for themselves’. Sure, students can pursue their own path of inquiry, but teachers should be there to persuade and guide them. For a teacher to pretend that they are letting students ‘make up their own minds’ is simply a denial of their role, an obfuscation. Of course it is important that people can decide for themselves, but students’ autonomy is not icreased by a lack of teacher manipulation of their choices. All our choices are conditioned by our past, our environment and by other people. Free choices are still conditioned choices.

A discussion of this in relation to the morality of manipulation is provided by Buss, S. (2005). Valuing Autonomy and Respecting Persons: Manipulation, Seduction, and the Basis of Moral Constraints. Ethics, 115(2), 195-235. doi: 10.1086/426304. Buss only touches on the topic of education in a footnote (no. 71) but the implications of the general argument for education are clear: we cannot avoid affecting other people, manipulation without informed consent of their free chocies is inevitable, and so we cannot pretend that it is possible for teachers to avoid making choices for their students, or that full informed consent of students in the content of their education is desirable or possible.

academic intellectual self-defence psychology

Orality and academia

Academia is a quintessentially literate culture. Studying is reading, the outputs of research are papers and books. Your are judged on your ability to express yourself in writing, and when you do this you must reference the written works of others. This is what defines scholarship. More than this, the habits of thought are patterned by literacy. Literate thought is analytic, dissective. The business of science is that of ordering into lists, breaking into parts, assigning subordinate and superordinate.

And yet there is also a non-literate part to being an academic, a part closely alighted with the praxis of the discipline. This is giving lectures, discussing in seminars, attending conferences. The paper outputs of academia can disguise this component, but it is essential.

In a high-education system dominated by production-line and consumption values, by a receptacle-model of education (students as containers, education as a substance) it is easy to denigrate the ‘live’ oral component of academia, but in doing so we deny students contact with an essential part of scholarly life. Additionally we deny ourselves a cognitive model which can augment literate thinnking.

Walter Ong (Orality and Literature, 1982/2002) has written convincingly about the psychological dynamics of oral vs literate culture. Literate culture encourages finished works, whereas the knowledge of oral cultures is always live, an act of telling rather than knowing. As such it is part of a commons, rather than copyrighted (and plagarised). Oral knowledge is situational, empathetic and participatory, rather than abstracted and objectively distanced, it is contested rather than autonomous and tends towards holism rather than the progressive analytic deconstruction of literate thought. Oral thought is thematic compared to literate thought’s ability to dictate strict chronologies and linear narratives / list structures (just think of the memory constraints on oral culture which lacks written aids to see why pure abstract lists are impossible).

It is clear that academia needs to take elements from both of sides of these distinctions. Oral cognition can be impressionistic rather than precise, conservative rather than innovative and amnesic rather the hypermnemonic. Nonetheless the lived aspects of oral thought are a vital part of disciplinary practice and exposure to them is essential if students are to get a true view of their subjects.

A similar thing is argued by Kevin McCarron, a English literature lecturer and part-time stand-up comedian who argues for the imporance of improvisation in teaching. He says that an overreliance on preparation (the script or text of the class) as getting in the way of the (living) interaction of student and teacher:

(article in the Times Higher Education)

“If we don’t put ourselves under pressure, nothing interesting or exciting is going to happen. How could it? In fact, what we’ve done is spent three hours the previous night making sure that it doesn’t happen.

“Then we have the gall to offer these hours of preparation as morally sound. Self-protection is being offered to the world as a moral value. That preparation has been done to protect the teacher from the students. Teachers spend hours and hours preparing because they are terrified of bring caught out.”

intellectual self-defence science

The choice of facts

Robert Park’s article ‘The Seven Warning Signs of Bogus Science is a result of his attempt to help judges faced with expert witnesses making scientific arguments. He has attempted to come up with heuristics of bad science: ‘indicators that a scientific claim lies well outside the bounds of rational scientific discourse’

Here they are:

  • 1. The discoverer pitches the claim directly to the media.
  • 2. The discoverer says that a powerful establishment is trying to suppress his or her work.
  • 3. The scientific effect involved is always at the very limit of detection.
  • 4. Evidence for a discovery is anecdotal.
  • 5. The discoverer says a belief is credible because it has endured for centuries.
  • 6. The discoverer has worked in isolation.
  • 7. The discoverer must propose new laws of nature to explain an observation.

It’s a good list. Sadly, however, telling the difference between sense and nonsense is never going to be easy. Even the best of us, when we get out of our field, can feel at a loss. It feels to me that your position on the classic controversial science debates (global warming, alternative medicine, creationism) is utterly removed from the facts either way, but instead depends on the pre-theoretical commitments you have made. So, for example, a preference for conventional vs alternative medicine, or creation science vs evolution, is in fact impossible to refute from within the frame of reference of the person with that preference (this will be obvious to any creationist who has tried talking to an evolutionist, or vice-versa). Rather than a choice which can be faulted on facts, it is really a case of choices about what kinds of information define facts. All views of the world have biases in them, the distinction between a scientist and a pseudoscientist is not about which each believes to be true, but rather about what set of systemmatic biases each has decided to place their faith in.

intellectual self-defence science

statistical self-defence

How to lie with Statistics is a classic. A ‘sort of primer in ways to use statistics to deceive’, says the author Darrell Huff. Why teach the world how to lie with statistics? Because, he says, ‘crooks already know these tricks; honest men must learn them in self-defence’.

The bulk of the book is worked examples of classic statistical slights-of-hand: graphs with missing sections on the axes, different kinds of averages, post-hoc observation of correlations. What I want to do here is just review the last chapter ‘How to talk back to statistics’, which gives some rules of thumb on how to ‘look a phoney statistic in the eye and face it down’ whilst recognising ‘sound and usable data in the wilderness of fraud’. Huff gives the reader five simple questions with which to arms themselves, which i summarise and then provide some commentary on at the end.

Huff’s five questions with which to arm yourself against statistics:

1. Who says so?

Can we suspect deliberate or unconscious bias in the originator of the statistic? Huff recommends looking for an “O.K Name” – e.g. a university – as some slim promise of reliability. Second to this he recommends being careful to distinguish the originator of the ‘data’ from the originator of the conclusion or intepretation.

2. How Does He Know?

Is the sample biased? representative? large enough?

3. What’s Missing?

Statistics given without a measure of realiability are ‘not to be taken very seriously’. What is the relevant base rates / appropriate comparison figure? Do averages disguide important variations?

4. Did somebody change the subject?

E.g. More reported cases are not the same as more cases, what people say they do (or will do) is not the same as what they actually do (or will do), association (correlation) is not causation.

5. Does it make sense?

Is the figure spuriously accuracy? Convert percentages to real numbers and convert real numbers to percentages, compare both with your intuitions.

Commentary by tom

Who says so?

Two things interest me about the recommendation to base judgements of credibility, even if just in part, on authority. Firstly, by doing this Huff is conceeding that we are simply not able to make a thorough independent evaluation of the facts ourselves. This is in contradiction to the idea that science is Best because if we doubt something we can check it out for ourselves. The pragmatic response to this is obviously ‘well, you won’t check out everything, but you could check out any individual thing if you wanted’. Is this much consolation in a world where everyone, including the authorities, are assaulted by too much information to check out personally? Scientific authority then becomes a matter of which social structures, which use which truth-heuristics, you trust rather than a matter of direct proof (“it says so in the bible!” verses “it says so in a respectable academic paper!”?).

The second thing that interests me is that the advice to rely on authorities becomes problemmatic for those who either don’t know who the authorities are, or who distrust the usual authorities. What proportion of the population knows that the basic unit of scientific authority is the peer-reviewed journal paper? You can see that if you don’t know this you immediately lose a vital heuristic for evaluating the credibility of research you are told about. In a similar vein, even experts in one domain may be ignorant of the authorities in another domain — leading to similar problems with judging credibility. If you know about but simply don’t trust the established authorities you are similarly lost at sea when trying to evaluate incoming evidence (a reason, I’ll bet, for the mixed quality of information available from, variously, conspiracy theorists and alternative medicine practicioners).

How Does He Know?

This is perhaps the most important question you can ask in my opinion. Often all that is required to dispell the superficially-convincing fog that accompanies some statistic or factoid is to ask How did they find out? What would actually be involved in gathering that information? Could it possibly be correct? For example, ‘if you die in your dream, you really die’ How do they know?! Dead people aren’t exactly available for comment.

What’s Missing?

Knowing what is missing is the hardest trick, in my opinion. It’s a mark both of expertise and of genuine intelligence to be able to pick up on what isn’t being said, to notice when the intepretation of what you’re being told could be fundamentally altered by something you aren’t being told (because, of course, this involes imaging a bunch of counter-factuals). Outside the realm of statistics the idea of frame-analysis speaks to this idea of making invisible what isn’t talked about.

Did somebody change the subject?

Does it make sense?

Both good checks to carry out when challenged by a statistic. It is unfortunate that statistics seem to have an inherant air of authority – a kind of wow factor – and these questions are good tools with which to start dismantling it. I think this wow factor is because statistics seem to imply rigourous, unbiased, comprenhensive investigation, even though they may in fact arise from nothing of the sort. In the same way that evolution will produce imitators who have the colouration of being poisonous, or whatever, without actually bothering to have to produce the poison, and most social situations will attract free-riders who want to get the benefits without paying the costs, so there is an evolution of rhetoric strategies to include things which carry the trappings of credible information without going through the processes which are actually causal in making the information from these sources credible. So we get statistics because everybody knows science uses statistics, we get figures quoted to the second decimal place when the margin of error is a hundred times larger than this level of accuracy, and we get nonsensical arguments supported using citations, even though the studies or works cited are utterly without credibility, because having citations in your arguments is an established form of credible arguments which is easy to reproduce for any argument, whatever the level of credibility.

Reference: Darrell Huff (1954) How to Lie with Statistics