I’ve been working over the break to boil down how to check Internet claims into something short and active. Short, because longer prescriptions don’t work. Active, because we are trying to build habits.
Here’s what I’ve got so far.
- See if someone has already done the work. Some people call this the “Check Snopes First” rule, but there’s actually a broader array of sites you can check as well. My guess is if people checked Snopes and other sites first then 80% of the most pernicious stuff would disappear from our feeds overnight.
- Go upstream. Here’s the second maddening thing with claim-checking. I have watched in my long career student after student looking at an article on some third-tier clickbait aggregation site and trying to determine the validity of an article by evaluating that particular site. That site doesn’t matter. Go upstream and get as close to the original source as possible before starting your analysis.
- Look at what others say about the source. Once you find your site upstream, it’s time to get off it again. See what other people say about your source. Use tools such as whois, Google Scholar, and SourceWatch to find out who is behind the information on the site, what their agenda is, and what expertise they bring to the table.
- Get a second opinion. But not from the same doctor! So your final step is to look for corroborating (and disproving!) evidence. But here’s what happens a lot of times: people see a claim (“Ford Motor Company supports Black Lives Matter group”). They trace it to the source (rare, but sometimes happens). Then to verify it they search on the claim and find there are dozens of stories out there talking about this. The thing is, if all these stories stem from the same original source, they can’t be used to verify that source. So as you scan search results, be looking for a source that is going to bring in additional information, or approach the question from a different angle.
There’s some bigger understandings that inform these actions. One thing I’m thinking about a lot nowadays is how the level of syndication and rampant “reporting on reporting” creates the appearance of broad consensus within hours of an original claim. I mean, you’re seeing the same claim on literally hundreds of sites. It must be true, or at least valid, right?
Jon Udell is working on a Chrome extension that encodes some of the process we’re discovering works most consistently; you can see that work here. As I said, we’re still trying to get this down to something that almost becomes muscle memory — we don’t believe you’ll be able to fully investigate a site off of a recipe, but to borrow a term from Jon, I think we can make some “strategies for internet citizens” partially encode as habits.
3 thoughts on “Checking Internet-Based Claims”
Hmm. I think the process itself is valuable, but understandably not everyone has time to investigate every single piece of news, so I will check Jon Udell’s blog.
But I was recently trying to investigate a particular scientific claim (something related to nutrition, which gets loads of ridiculousness). And here’s some of what i was thinking
A. The only reason i investigated it is that even tho a doctor said it to me, I asked where she learned it and she said via WhatsApp (ok, via a nutritionist, but still via WhatsApp). I thought the info looked suspicious so I looked it up. Meaning – we usually don’t investigate things if we think it sounds reasonable. Otherwise we would go nuts. And I am guessing people have different thresholds of what they count as reasonable in different domains. And different capacities to judge credibility beyond that in the different domains [what if the inaccurate or fake news item is framed so cleverly it doesn’t tickle or suspicion radar?]
B. Trying to find sources, coincidentally there are lots of sources around the topic. Very few were “with” (only one or two) and many against or talking from a different angle. None of it links to peer-reviewed work or strong sources like NIH or Mayo Clinic or even known newspapers. So nothing to report here
C. Searched Google Scholar and found nothing. So it probably hasn’t been investigated scientifically yet. Maybe. But also: keep in mind that I could have found several peer-reviewed pieces with *different* conclusions and you would need to be domain-knowledgeable to judge credibility of those by investigating strength of research methodology, reputation of journal, researchers, their affiliations, and how their article is written
I didn’t know about Snopes btw. Will check that out also
Not everybody has to check anything. We need to develop what epidemiologists call a “herd immunity” — enough resistance to this stuff that the spread slows. I think that translates, actually, into a relatively small percentage of people verifying things before they retweet them.
I’ll also say that most of this stuff takes a lot of time to narrate, but less than a minute to do for someone who is skilled.
I also agree with you, mostly. Different people find different things weird, and hopefully between us all we’ll get things covered. I’d add though that the other set of things I check are things that seem to be just too “perfect”: “First Trump Visit Will Be to Russia, Spokesman Says”. Etc.
Oh and thanks for narrating your process — we need more of that.