Quick Note on the Recent Buzzfeed Study

A couple people asked me to expand on comments made in my recent Familiarity = Truth post. In it I say this about the Buzzfeed finding that over 50% of Clinton supporters who remember fake anti-Clinton headlines believed them:

[A] number like “98% of Republicans who remember the headline believed it” does not mean that 98% of Republicans who saw that headline believed it, since some number of people who saw it may have immediately discounted it and forgotten all about it.

What does that mean? And how does it mitigate the effect we see here?

Well, for one, it means that despite Buzzfeed’s excellent and careful work here, they have chosen the wrong headline. The headline for the research is “Most Americans Who See Fake News Believe It, New Survey Says”. But the real headline should be the somewhat more subtle “Most Americans Who Remember Seeing Fake News Believe It, New Survey Says”

Why is that important? Because you can see the road to belief as a series of gates. Here’s the world’s simplest model of that:

Sees it > Remembers it > Believes it

So, for example, if we start out with a thousand people, maybe only 20% see something. This is the filter bubble gate.

But then, a certain amount of those people who see it process it enough to remember it. And this is not a value neutral thing — many decades of psychological research tells us we notice things which confirm our beliefs more than things that don’t. Noticing is the first part of remembering. So we should guess that people that remember a news story are more likely to believe it than those that don’t. Hence, when we read something like “Over 50% of people who remembered a fake headline believed it” this does not mean that 50% of people who read it believed it, because remembering something predicts (to some extent) belief.

Let’s call this the “schema gate” since it lets through things that fit into our current schemas.

So how big is this effect? From the data we see, it’s smaller than I would have thought. I say this because when we look at the numbers of people who remember a headline, the Trump and the Clinton numbers are not that far off. For instance, 106 Clinton supporters saw the famous murder-suicide headline, compared to 165 Trump supporters. While that certainly is quite a bit more Trump supporters (and even more on a percentage basis) we have to assume a good percentage of that difference is due to different networks of friends and filter bubble effects. If you assume that highly partisan Republicans are going to have 50% or 75% more exposure to Anti-Clinton stories, then there isn’t much room left for much of a schema gate effect.

This leads to an interesting question — if we are really attached to a schema gate effect, then we have to dial down our filter bubble effect. Maybe filter bubbles impact us less than we think, if this many Democrats are seeing this sort of story in their feed?

There’s a couple other far out ways to make the math work, but for the most part you either get evidence of a strong filter bubble gate and a weaker than expected schema gate, or vice versa. Or you get both things somewhat weaker than expected.

In any case, it’s one of the more fascinating results from the study, and if Buzzfeed or anyone else is looking to put a shovel into a new research question, this is where I’d suggest to dig.



4 thoughts on “Quick Note on the Recent Buzzfeed Study

  1. hello
    there is also the enormous issue of memory effects and biases; rather than a psychological model of a schema gate we could also consider a philosophical definition that knowledge is justified true belief; here the Democrats who did not believe may have decided that their initial belief that it was true was not reliable or justified via reading other sources;

    Mike in your analysis so far of fake news in general I still think a moral manic argument still holds some water?

    • Moral panic — I see we’re you’re coming from, but in general “panic” denotes the fact that the panic is unfounded — Elvis in 50s, young women reading in the 19th c., satanic worship in the 1980s, etc. I’d say that while the causal argument is murky that we are seeing massive effects of *some* set of things lately, and we’re really working backward from massive known social effects and saying “Is this a possible input?” That to me seems the opposite of moral panic logic which says — it seems trivial and harmless now, but think of what it will cause down the road!

      In other words, I fully believe the influence of this *might* be overblown, but not in the normal sense of a moral panic, which has no real effects to focus on.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s