The “Always Check” Approach to Online Literacy

One of the things I’ve been trying to convince people for the past year and a half is that the only viable literacy solution to web misinformation involves always checking any information in your stream that you find interesting, emotion-producing, or shareable. It’s not enough to check the stuff that is suspicious: if you apply your investigations selectively, you’ve already lost the battle.

Once you accept that, certain things become clear. Your methods of checking have to be really quick. They have to be habitual, automatic. They can’t be cognitively expensive. And those who teach media literacy have to be conscious of this trade-off between depth and efficacy and act accordingly.

What do I mean by that? Let’s use an analogy: which technique do you think would prevent more car accidents?

  • A three-second check every time you switch lanes
  • A twenty-second check executed every time you think a car might be there

There are some hard problems with misinformation on the web. But for the average user, a lot of what goes wrong comes down to failure to follow simple and quick processes of verification and contextualization. Not after you start thinking, but before you do.

I can’t get these processes down to a two second mirror-and-head-check, but I can get them close. What follows are some of the methods we teach students in our work. It will seem like there is a lot of stuff to learn here, but you’ll notice that it comes down to the same strategies repeated in different contexts. This repetition is a feature, not a bug.

Is This the Right Site?

Today’s news reveals that Russian-connected entities were trying to spoof sites like the Hudson Institute for possible spear-phishing campaigns. How do we know if the Hudson Institute site we are on is really the real site? Here’s our check:


The steps:

  • Go up to the “omnibar”
  • Strip off everything after the domain name, type wikipedia and press enter
  • This generates a Google search for that URL with the Wikipedia page at the top
  • Click that link, then check in the sidebar that the URL matches.
  • Forty-nine out of fifty times it will. The fiftieth time you may have some work to do.

In this case, the URL does match. What does this look like if the site is fake? Here’s an example. A while back a site at impersonated the Bloomberg News site. Let’s see what that would look like:


You do the same steps. In this case Bloomberg News is not the top result, but you scroll down and click the Bloomberg News link, and check the URL and find it is different. If you’re lazy (which I am) you might click that link to get to the real site.

What Is the Nature of This Site?

Let’s stick with the Wikipedia technique for a moment, because it’s useful for a few other questions. As an example, let’s take one that got past both a Washington Post reporter and the WaPo fact-checkers a month or so ago. Question: Is this article really by the lead singer of Green Day?


Let’s check:


Again, same process. Now does this mean that you are 100% sure that it’s not Billie Joe that wrote that article? No — there’s a slight slight chance that maybe somehow the lead singer of Green Day wrote a —

Nah, you know what? It’s not him. Or if it is, the chances are so infinitesimal it’s not worth spending any more time on it. Find another source.

How about this site, and its searing commentary on Antifa and journalists?


Maybe you agree with this article. I don’t, but maybe you do. And that’s okay. But do you want to share from this particular site to your friends and family and co-workers? Let’s take a look!


You can dig into this if you want, and look through the numerous links in that Wikipedia page that support this description. Maybe have a little mini-forum in your head about the differences between white nationalism and white supremacy.

Or maybe — here’s a thought — find a similar article from some other site that hasn’t been called a white supremacist organization by half a dozen mainstream groups. Because no matter what you think of the article, funneling friends and family to a site that has published such sentences as “When blacks are left entirely to their own devices, Western civilization — any kind of civilization — disappears” is not ethical — or likely to put you in the best light.

Is This Breaking News Correct?

Here’s some breaking news.


More people than you would think believe that the blue checkmark = trustworthy. But all the blue checkmark really does is say that the person is who they say they are, that they are the person of that name and not an imposter.

Your two-second “mirror and head-check” here is going to be to always, always hover, and see what they are verified for. In this case the verification means something: this person works for, a legitimate news site, and she covers a relevant beat here (the White House):


But maybe you don’t know CNBC, or maybe you see this news from someone not verified, or verified but not as a reporter. How will you know whether to share this? Because you know you’re DYING to share it and you can’t wait much longer

Use our “check for other coverage” technique:


When a story is truly breaking, this is what it looks like. Our technique here is simple.

  • Select some relevant text.
  • Right-click or Cmd-click to search Google
  • When you get to Google don’t stop, click the “News” tab to get a more curated feed
  • Read and scan. Investigate more as necessary.

Scan the stories. If you want to be hypervigilant, scan for sources you recognize, and consider sharing one of the stories featuring original reporting instead of the tweet.

I’m going to state this again, but if you look at that loop above you’ll see this is about a seven second operation. You can absolutely do this every time before you share. And given it is so easy, it’s irresponsible not to. I’m not going to tell you you are a bad person if you don’t do these checks, but I think in your heart you already know.

Teach This Stuff First Already

Maybe you think you do this, or you can really “recognize” what’s fake by looking at it. I am here to tell you that statistically it’s far more likely you’re fooling yourself.

If you’re a human being reading this on the internet and if you’re not a time traveler from some future, better world, there is less than a one in a hundred chance you do the sort of checks we’re showing regularly. And if you do do this regularly — and not just for the stuff that feels fishy — then my guesstimate is you’re about two to three standard devs out from the mean.

Now imagine a world where checking your mirrors before switching lanes was rare, three standard-deviations-out behavior. What would the roads look like?

Well, it’d probably look like the Mad Max-like smoking heap of collisions, car fires, and carnage that is our modern web.

I get worried sometimes that I am going to become too identified with these “tricks”. I mean, I have a rich history of teaching students digital literacies that predates this work. I’ve been doing the broader work intensively for ten years. (Here’s a short rant of mine from 2009 talking about web literacy pedagogy.) I’ve read voraciously on these subjects and can talk about anything from digital redlining to polarization models to the illusory truth effect. I’m working on a project that looks to document the history of newspapers on Wikipedia. I worked on wiki with Ward Cunningham. I ran my first “students publish on the web” project in 1997.

But I end up coming back to this simple stuff because I can’t shake the feeling that digital literacy needs to start with the mirror and head-checks before it gets to automotive repair or controlled skids. Because it is these simple behaviors, applied as habitand enforced as norms, that have the power to change the web as we know it, to break our cycle of reaction and recognition, and ultimately to get even our deeper investigations off to a better start.

I have underlying principles I can detail, domain knowledge I think is important, issues around identity and intervention we can talk about. Deeper strategies for the advanced. Tips to prevent a fragility of process. Thoughts about the relationship between critical thinking and cynicism.

But for the love of God, let’s start with the head check.




QAnon and Pinterest Is Just the Beginning

I have been talking about Pinterest as a disinformation platform for a long time, so this article on QAnon memes on Pinterest is not surprising at all:

Many of those users also pinned QAnon memes. The net effect is a community of middle-aged women, some with hundreds of followers, pinning style tips and parfait recipes alongside QAnon-inspired photoshops of Clinton aide John Podesta drinking a child’s blood. The Pinterest page for a San Francisco-based jewelry maker sells QAnon earrings alongside “best dad in the galaxy” money clips.

Pinterest’s algorithm automatically suggests tags with “ideas you might love,” based on the board currently being viewed. In a timely clash of Trumpist language and Pinterest-style relatable content, board that hosts the Podesta photoshop suggests viewers check out tags for “fake news” and “so true.”

The story is a bit more complex than that, of course. It’s not clear to me that the users noted here are not spammers (as we’ll see below). It’s quite possible many of these accounts are people mixing memes and merchandise as a marketing amplification strategy. We don’t know anything about real reach, either. There are no good numbers on this.

But the threat is real, because Pinterest’s recommendation engine is particularly prone to sucking users down conspiracy holes. Why? As far as I can tell, it’s a couple of things. The first problem is that Pinterest’s business model is in providing very niche and personalized content. It’s algorithm is designed to recognize stuff at the level of “I like pictures of salad in canning jars”, and as Zeynep Tufekci has demonstrated with YouTube, engines of personalization are also engines of radicalization.

But it’s more than that: it’s how it goes about recommendation. The worst piece of this, from a vulnerability perspective, is that it uses “boards” as a way to build its model of related things to push to you, and that spammers have developed ways to game these boards that both amplify radicalizing material and and provide a model for other bad actors to emulate.

How Spammers Use Pinterest Boards as Chumbuckets

The best explanation of how this works comes from Amy Collier at Middlebury,  whose post on Pinterest radicalization earlier this year is a must-read for those new to the issue. Drawing on earlier work on Pinterest manipulation, Collier walks through the almost assuredly fake account of  Sandra Whyte, a user who uses boards with extreme political material to catch the attention of users. Here’s her “American Politics” board:


These pins flow to other users’ home pages with no context, which is why the political incoherence of the board as a whole is not a problem for the user. People are more likely to see the pins through the feed than the board as a whole.

Once other users like that material, they are more likely to see links to TeeSpring T-shirts this user is likely selling:


The T-Shirts are print-on-demand through a third-party service, so hastily designed that the description can’t even be bothered to spell “Mother” right.


So two things happen here. When Moms like QAnon content, they get t-shirts, which provides the incentive for spammers to continue to make these boards capitalizing on inflammatory content. Interestingly, when Moms like the T-shirts, they get QAnon content. Fun, right?

How Pinterest’s Aggressive Recommendation Engine Makes This Worse

About a year ago I wrote an article on how Pinterest’s recommendation engine makes this situation far worse.  I showed how after just 14 minutes of browsing, a new user with some questions about vaccines could move from pins on “How to Make the Perfect Egg” to something out of the Infowarverse:


What was remarkable about this process was that we got from point A to B by only pinning two pins on a board called vaccination.

I sped up the 14 minute process into a two and a half minute explanatory video. I urge you to watch it, because no matter how cynical you are it will shock you.

I haven’t repeated this experiment since then, so I’m unable to comment on whether Pinterest has mitigated this in the past year. It’s something we should be asking them, however.

I should note as well that the UI-driven decontextualization that drove Facebook’s news crisis is actually worse here. Looking at a board, I have no idea why I am seeing these various bits of information at all, or any indication where they come from.


Facebook minimized provenance in the UI to disastrous results. Pinterest has completely stripped it. What could go wrong?

Pinterest Is a Major Platform and It’s Time to Talk About It That Way

Pinterest has only 175 million users, but 75 million of those users are in the United States. We can assume a number of spam accounts pad that number, but even accounting for that, this is still a major platform that may be reaching up to a fifth of the U. S. population.

So why don’t we talk about it? My guess is that its perceived as a woman’s platform, which means the legions of men in tech reporting ignore it. And the Silicon Valley philosopher-king class doesn’t bring it up either. It just sounds a bit girly, you know? Housewife-ish.

This then filters down to the general public. When I’ve talked about Pinterest’s vulnerability to disinformation,  the most common response is to assume I am  joking. Pinterest? Balsamic lamb chops and state-sponsored disinfo? White supremacy and summer spritzers?

Yup, I say.

I don’t know how compromised Pinterest is at this point. But everything I’ve seen indicates its structure makes it uniquely vulnerable to manipulation. I’d beg journalists to start including it in their beat, and researchers to throw more resources into its study.

A Provocation for the Open Pedagogy Community

Dave Winer has a great post today on the closing of These are sites run by Berkman, some dating back to 2003, which are being shut down.

My galaxy brain goes towards the idea of federation, of course. The idea that everything referencing something should store a copy of what it references connected by unique global identifiers (if permissions and author preferences permit), and that we need a web that makes as many copies of things as the print world did, otherwise old copies of the Tuscaloosa News will outlast anything you are reading today on a screen. Profligate copying, as Ward Cunningham has pointed out, is biology’s survival strategy and it should be ours as well.

(I know, nature is not teleological. It’s a metaphor.)

But my smaller provocation, perfectly engineered for Friday twitter outrage at me and my sellout-ness, is this:

All my former university hosted sites are gone. We built up a WPMU instance at Keene in 2010, and the lack of broad adoption meant when I left in 2013 we shut it down. I ran some wiki on university servers here and at Keene, and those are gone too.

All my self-hosted sites are corrupted from hacks or transfer errors in imports. Go back into this blog and you’ll find sparse posting schedule for some years between 2010 and 2012 and it’s because those posts got nuked in a 2012 hack. I had to go out to the Wayback Machine and reconstruct the important ones by hand.

Transfer errors, let me tell you:  Go back to 2007 and look at all the images that failed imports and moves on this blog when it was self hosted. There’s also this weird “” character that pops up in all of them like this:

Hold on, you say, these Metro signs look different! There’s no BRAND!

The entire Blue Hampshire community I co-founded, over 15,000 posts and 100,000 comments, originally self-hosted on SoapBlox and then WordPress? Gone. It’s probably OK, I said a lot of stupid stuff. But of course it was also a historically important site, one of the most successful state political blogging communities, one of the first communities to be syndicated by Newsweek, one of the first to feature news stories that cross-posted — as news stories — to Huffington Post. One of the first sites to get individual statements from all the Democratic presidential candidates in a weekly forum. Gone, gone, gone.

I know, this doesn’t seem to be provocative, but here’s the thing:

My Blogger sites from 2005 forward? They’re up and they are pristine.


I mean, I’m not sure that’s a great thing — it was where I put little experiments too little to be worth setting up another BlueHost domain. But it also did me a solid in Keene Scene, where the 12-year old images of Keene life have stayed up unmolested and without any maintenance. (I’d quite forgotten about it, really).


Same holds — as I’ve mentioned before — for projects students put up on Google Sites. The BlueHost server (and later the Rackspace account) was long ago shut down but Google Sites is still up.

I’m not making a specific case here. But I do want to point out a big reason I moved to self-hosted and institutional solutions was this idea that commercially hosted stuff was too fickle. In 2006, it seemed that every week a new site shut down. For better or worse (mostly worse) monopoly consolidation has changed that dynamic a bit. There are other good reasons for self-hosting or doing institutional hosting, but durability is more downside than upside of these options, and we might want to let our students know that if they want something to stay up, self-hosting may not be the best choice.

Newspapers On Wikipedia Update: Initial Wikidata Pass

Thanks to initial work by folks at Wellesley and Wikidata work from 9of99 on Wikipedia, the Newspapers on Wikipedia project has both created an initial Wikidata set of extant U.S. Newspapers and mapped that to needs for page and infobox creation.

The full set is here and can be queried in multiple ways:

Visually these maps overstate needs in high density areas, since the red dots (needs page) take precedent over blue dots (has page) in a conflict, and the data has a geolocation that is only as granular as the town (hence Chicago has one geolocation). And the data will need continued cleanup — I’ve spotted a few issues just screenshotting regions. But this initial data set will be developed alongside the rest of the project, and even when papers don’t make it into Wikipedia, we’ll make sure the Wikidata on them is accurate, and try to match them with other sets of data as we go forward.

According to the data here (which again, is imperfect) the current counts are:

  • Has Wikipedia page and Infobox: 957
  • Needs Infobox: 84
  • Needs Page: 3775

(We’ve already put a dent in some of the work before this, so we’ll go back and manually tally up a baseline.)

Anyway, some maps. Keep in mind this is very preliminary.

A Note about Cognitive Effort and Misinfo (Oh, and also I’m a Rita Allen Misinformation Solutions Forum Finalist)

So I forgot to report this, but I put together a team and submitted a proposal to the Rita Allen Misinformation Solutions Forum contest, and our project was chosen out of all the submissions as one of five finalists. I’ll be going to D.C. in October to pitch it in a competition for one of two prizes.

The project is named  Let me fact-check that for you”: a semi-automated, personalized guide generator for the “Wait! You’re wrong about that!” responder.

The tool is meant to empower current Snope-sers to not just post links to alternative articles, but to post short, customized guides that show how they went about fact-checking the particular link, story, or image. Too often when someone in a comment thread debunks or contextualizes something, it’s just dueling links. No one learns how to check things any better. Our hope is to make a website service where you plug in a URL or image and click through a couple decisions. Out the other end comes a sharable little five second, screenshot-based guide showing how you might check that specific link or image.

The idea is loosely (very loosely) inspired  by the old joke site “Let me Google that for you”, where you could plug in a question someone had asked you and it would create a little video of the process to Google the answer. The idea with LMGTFY was partially to shame people into checking Google before bother people, but the other piece of it was to demonstrate that the time cost of consulting Google first was minimal. People were overestimating the cost of consulting Google, the little links were reminders.

People outside misinfo may not be aware of this, but there is a critique of the “people won’t fact check because they love their own point of view” that posits that people — to some extent — aren’t just choosing things they agree with because they like being right, but because it just requires less effort than engaging in more accuracy-oriented behavior. Gordon Pennycook and David Rand, for instance, have an interesting paper on this idea, showing that people that score high on cognitive reflection (an appetite for effort) also show better headline discernment, even when headlines are ideologically aligned.

I’m not necessarily sold on the Pennycook and Rand version of this idea, but I’m interested in the broader insight. I know it doesn’t explain the worst offenders, but I’ve found with those I work with that cynicism (“Pick what you want, it’s all bullshit!”) is often driven by the cognitive exhaustion of sorting through conflicting information. This insight also aligns with Hannah Arendt’s work — totalitarianism wins the information war by deliberately overwhelming the capacity of a population to reconcile endless contradictions. The contradictions are a tool to increase the cost of pursuing truth relative to other options.

If this is the case, one approach might be to encourage people to be more effortful when looking at online media. (Meh.) But the approach I favor is to reduce both the real and perceived cost of sorting through the muck through finding cheap, good enough methods and popularizing them. Doing that — while fostering a culture that values accuracy — might cause a few more people to regard the cost of checking something to be worth it relative to other seemingly more economical options like partisan heuristics, conspiracy thinking, or cynical nihilism.

As such, the methods that our tool will demonstrate will be useful (at decreasing real cost, since our methods fall back on some cognitively inexpensive methods). But the bigger impact is just letting people see that they probably imagine the cost of weeding out the worst information as being much higher than it actually is. By resetting these expectations, we can influence the behavior they choose.

As they say, it’s a theory. Anyway, let me know if you’ll be at the forum in October. I’d love to meet up. And if you’re working on something similar, let me know.

Unintended Consequences to Google Context Cards on Conspiracy Videos?

I was putting together materials for my online media literacy class and I was about to pull this video, which has half a million views and proposes that AIDS is the “greatest lie of the 21st century.” According to the video, HIV doesn’t cause AIDS, retrovirals do (I think that was the point, I honestly began to tune out).

But then I noticed one of the touches that Google has added recently: a link to a respectable article on the subject of AIDS. This is a technique that has some merit: don’t censor, but show clear links to more authoritative sources that provide better information.

At least that’s what I thought before I saw it in practice. Now I’m not sure. Take a look at what this looks like:


I’m trying to imagine my students parsing this page, and I can’t help but think without a flag to indicate this video is dangerously wrong that students will see the encyclopedic annotation and assume (without reading it of course) that it makes this video more trustworthy.  It’s clean looking, it’s got a link to Encyclopedia Britannica, and what my own work with students and what Sam Wineburg’s research has shown is that these features may contribute to a “page gestalt” that causes the students to read this as more authoritative, not less — even if the text at the link directly contradicts the video. It’s quite possible that the easiness on the eyes and the presence of an authoritative link calms the mind, and opens it to the stream of bullshit coming from this guy’s mouth.

Maybe I’m wrong. It seems a fairly easy thing to test, and I assume they tested it. But it’s also possible that when these things get automated the things you thought were edge conditions turn out to be much more the norm than anticipated. In this case, the text that forms that paragraph from Britannica is on “AIDS”, not “AIDS denialism”, and as such the text rebuttal probably has less impact than the page gestalt.

I get the same feeling from this one about the Holocaust:


What a person probably needs to know here is not this summary of what the Holocaust was. The context card here functions, on a brief scan, like a label, and the relevant context of this video is not really the Holocaust, but Holocaust denialism, who promotes it, and why.

Again, I hope I’m wrong. Subtle differences in implementation can matter, and maybe my gut on this is just off. It really could be — my job involves watching a lot of people struggle with parsing web pages, and that might warp my perspective.

But it should be easy enough for a researcher to take these examples and see how it works in practice, right? Does anyone know if someone has done that?