False Positives and Fake News

I keep losing this information, so I thought I’d put it here on the blog. Some notes to thinking about disinfo in a different way. I realize this is a lot of stuff from everywhere and nothing is directly comparable. But some trends emerge.

First from Pew most people think they are good or pretty good at spotting fake news:


That’s partially because people tend to believe they are slightly above average at almost everything, but there you go.

From that same report (late 2016) many people tended to overestimate how often they saw fake news. A reasonable answer for most people would be sometimes given what we know about overall frequency. Again, maybe this is just a difference in definitions or psychological salience:


Ok, and then there’s this from Harvard’s IOP, on 18-29 year-olds, from March of 2017. I don’t like the term “fake news” here, because it could also trigger Trump identification and responses based on that (e.g. CNN shows up in feed, that’s fake news, right?). But fascinating, right? The average answer to what percentage is fake is about 50%! (h/t Joshua Benton for pointing me here):


Then there’s this from a Politico poll from October 2017.


Followed by this chilling result:


And this from September 2016:


Anyway, all this leads me wonder if most people who think they are very good at spotting fake news are actually generating a lot of false positives. Again, this isn’t meant to argue that point necessarily — just wanted to collect these charts in one place.

300+ Web Searches for Your Online Literacy Class

Sometimes in online media literacy we need a Google search that will turn up a mixture of high quality and low quality information for students to sort through. But it’s surprisingly hard to come up with a large array of unique queries on the spot.

I generated this list of questions to ask Google, Bing, or whatever from a combination of Buzzsumo results and Google autocomplete suggestions. Quality and complexity of results may vary, and what Google returns is ever-shifting. So please check the search results before assigning them to your students as mini research projects to make sure the difficulty is right for your purpose.

Here we go:

  1. Are “Hail Satan” license plates now available in Tennessee?
  2. Are Americans flocking to Mexico for dental care?
  3. Are President Trump’s vacations bankrupting the Secret Service?
  4. Are Tasmanian tigers extinct?
  5. Are baby boomers squeezing millennials out of the housing market?
  6. Are birth control pills in our water supply causing “transgender” fish?
  7. Are children of divorce more likely to fail exams?
  8. Are children of divorced parents more likely to divorce?
  9. Are children of divorced parents more likely to leave religion?
  10. Are children reading less?
  11. Are e-cigarettes as bad for people as smoking?
  12. Are earth temperatures cooler than when Al Gore won the Nobel Prize?
  13. Are illegal aliens in Canada complaining about a lack of free housing?
  14. Are learning styles real?
  15. Are men who marry “chubby” women happier?
  16. Are millennials finding it hard to transition into adulthood?
  17. Are more terrorists right wing or left wing?
  18. Are most convicted terrorists in the U.S. citizens?
  19. Are opioids killing more people than AIDS at its peak?
  20. Are people who are late more likely to live longer?
  21. Are people who are late more successful?
  22. Can a “raw diet” cure eczema?
  23. Can adding water to whisky increase its flavor?
  24. Can an algorithm be racist?
  25. Can an algorithm catch a serial killer?
  26. Can an algorithm predict risk of suicide with 92 percent accuracy?
  27. Can baking soda cure cancer?
  28. Can cannabis cure opioid addiction?
  29. Can celery prevent chronic inflammation?
  30. Can colloidal silver cure cancer?
  31. Can dandelion weed cure cancer?
  32. Can embracing your darkest emotions improve mental health?
  33. Can magnets be used to retrieve short and long term memories?
  34. Can magnets cure cancer?
  35. Can water bottles set things on fire?
  36. Could a recent sperm count drop make humans extinct?
  37. Does happiness reduce stress?
  38. Did 2,000 old seeds grow into an extinct biblical tree?
  39. Did 200 hundred protestors demand that New York City take down a statue of Teddy Roosevelt?
  40. Did Belgium ban halal butchers?
  41. Did Belgium ban the kosher slaughter of animals?
  42. Did Black Lives Matter activists interrupt a Montreal Pride parade?
  43. Did China buy Volvo?
  44. Did China buy Walmart?
  45. Did China hack the U.S. Seventh Fleet?
  46. Did Common Core kill music programs?
  47. Did Donald Trump pay actors to cheer during his announcement he would run for president?
  48. Did France ban plastic plates and cups?
  49. Did France ban the use of Gardasil?
  50. Did George Clooney donate millions to tear down Confederate statues?
  51. Did German schools ban pork to accommodate Muslim students?
  52. Did Harvard researchers find Exxon misled public on climate science?
  53. Did Hitler execute “fake news” journalists that were telling the truth?
  54. Did Idaho health premiums spike 81 percent?
  55. Did Illinois ban travelling elephant acts?
  56. Did Iran ban Zumba?
  57. Did John McCain approach Russia for a campaign contribution?
  58. Did Joy Ann Reid retweet a hoax about President Trump hiring black actors for a Phoenix rally?
  59. Did Millennials kill Applebee’s?
  60. Did NASA predict earth will experience 15 days of darkness in November 2017?
  61. Did Nancy Pelosi’s father dedicate confederate statues?
  62. Did Obama deport more people than any president before him?
  63. Did Obama pardon 1970s terrorist Elizabeth Anna Duke?
  64. Did Obama pardon a Puerto Rican domestic terrorist?
  65. Did Obama pardon a crack dealer who went on to brutally murder a family?
  66. Did Obama refuse to pardon and army officer who killed jihadists?
  67. Did Oberlin, Ohio replace Columbus Day with Indigenous Peoples Day?
  68. Did President Trump eliminate a plastic bottle ban in National Parks?
  69. Did Putin ban fluoride in Russia?
  70. Did Robert E. Lee’s descendant denounce white supremacy at the VMAs?
  71. Did Russia ban cryptocurrencies (such as Bitcoin)?
  72. Did Russia ban the Neo-Nazi site The Daily Stormer?
  73. Did Russia ban the use of VPNs?
  74. Did Russia classify Jehovah’s Witnesses as dangerous extremists?
  75. Did Salt Lake City’s Mayor go undercover as a homeless man?
  76. Did Secretary Devos end Common Core?
  77. Did Switzerland ban animal tested cosmetics?
  78. Did Switzerland ban the import of halal meat?
  79. Did Switzerland ban the import of kosher meat?
  80. Did Switzerland ban the wearing of the burqa?
  81. Did Switzerland band certain deodorants due to cancer risk?
  82. Did Sylvia Plath not return to America because of the lack of affordable health care?
  83. Did Trump rescind Obama’s flood risk rules weeks before Harvey?
  84. Did U.S. GDP set a new record under Donald Trump?
  85. Did Wikileaks turn down leaks on Russia during the 2016 campaign?
  86. Did a Facebook AI invent its own language?
  87. Did a London police chief say that non-English speakers would get priority treatment?
  88. Did a Memphis theater stop showing Gone With the Wind due to racial insensitivity?
  89. Did a Nigerian student develop a cure for breast cancer?
  90. Did a cargo ship cross the arctic without an icebreaker for the first time ever?
  91. Did a reporter who exposed a BBC pedophilia cover-up die under suspicious circumstances?
  92. Did a severely mentally ill sperm donor father 36 kids after lying on application?
  93. Did an ancient map show Antarctica without its ice cap?
  94. Did an op-ed in the New York Times argue that raping children shouldn’t be a crime?
  95. Did hackers breach dozens of voting machines brought to a conference?
  96. Did illegal border crossings decrease by 40% in Trump’s first month as President?
  97. Did over 500 Android apps contain spyware?
  98. Did people in Baltimore vandalize a Christopher Columbus monument?
  99. Did police in Seattle bust a Satanic pedophile ring in August 2017?
  100. Did protestors in Atlanta tear down a “Peace Monument” after mistaking it for Confederate Statue?
  101. Did representative John Conyers introduce a bill for reparations?
  102. Did scientists just successfully “edit” the first human embryo ever in the U.S.?
  103. Did teens in Hawaii kill albatrosses, setting conservation efforts back 10 years?
  104. Did the Australia Weather Bureau tamper with climate numbers?
  105. Did the American Cancer Society cancel a Mar-a-Lago event?
  106. Did the Catholic Church claim child sex abuse victims ‘consented’?
  107. Did the Every Student Succeeds Act end Common Core?
  108. Did the National Cancer Institute confirm that cannibis cures cancer?
  109. Did the University of Southern California misspell Shakespeare’s name on a new statue?
  110. Did the first-ever “Customized Lipstick” store open in downtown Montreal?
  111. Did the head of Starbucks say it was OK to commit violence against whites?
  112. Did the two day care workers spend 21 years in prison over fake Satanic ritual charges?
  113. Did two Douglas County men get 33 years in prison for flying Confederate flag?
  114. Did waitress Brianna Siegel receive a $1,200 tip on a $20 bill?
  115. Do 11 California counties have more registered voters than voting age citizens?
  116. Do 96% of Trump voters stand by their decision?
  117. Do Concord grapes improve your memory?
  118. Do DNA test results change health habits?
  119. Do acts by Muslim terrorists get less press attention?
  120. Do bilingual speakers experience time differently?
  121. Do birth control pills cause brain tumors?
  122. Do changes in the sun have influence on global temperature?
  123. Do children of divorce have higher stroke risk?
  124. Do cleanses work?
  125. Do detox diets work?
  126. Do drugs kill more people than guns?
  127. Do lemons cure cancer?
  128. Do men who eat more vegetables smell sexier to women?
  129. Do nitrates cause cancer?
  130. Do nitrates cause heart attacks?
  131. Do people with an X on both palms make better leaders?
  132. Do people with blood type O get bit more by mosquitoes than blood type A?
  133. Do police arrest more people for marijuana use than all violent crimes combined?
  134. Do sanctions work?
  135. Do sanctuary cities have significantly higher crime rates?
  136. Do sanctuary cities violate federal law?
  137. Do six Baltimore schools have zero students who are proficient in math and English?
  138. Do unattractive people out earn average looking people?
  139. Do vaccines cause autism?
  140. Do vegetables have more nitrates than hot dogs?
  141. Do video games improve your memory?
  142. Do vinegar baths cure fevers?
  143. Do women have more stamina than men?
  144. Does “science say” it’s in our nature to social media “creep”?
  145. Does Facebook keep deleted posts?
  146. Does Facebook now allow users to flag things as “Literally Hitler”?
  147. Does fracking cause air pollution?
  148. Does Russia have more prostitutes than doctors, farmers, and firemen combined?
  149. Does SPAM cause cancer?
  150. Does Social Security add to the national debt?
  151. Does Social Security have a surplus?
  152. Does U.S. solar power employ more people than oil, coal, and gas combined?
  153. Does aerobic exercise increase oxygen levels?
  154. Does alcohol cure cancer?
  155. Does avocado cure eczema?
  156. Does bad weather make us horny?
  157. Does being near the ocean “change your brain”?
  158. Does caffeine trigger a taste for sweets?
  159. Does climate change cause earthquakes?
  160. Does climate change cost India $10 billion per year?
  161. Does diet soda cause heart attacks?
  162. Does dietary fiber affect blood sugar?
  163. Does dietary fiber have calories?
  164. Does drinking alcohol improve memory?
  165. Does drinking champagne prevent Alzheimers?
  166. Does eating soy cause cancer?
  167. Does exercise improve COPD?
  168. Does exercise improve your memory?
  169. Does fracking cause cancer?
  170. Does fracking cause climate change?
  171. Does fracking cause droughts?
  172. Does fracking cause earthquakes?
  173. Does franking cause sinkholes?
  174. Does lipstick cause cancer?
  175. Does loneliness cause dementia?
  176. Does loneliness cause heart disease?
  177. Does loneliness kill you?
  178. Does one child in the U.S. disappear every 40 seconds?
  179. Does oregano oil kill warts?
  180. Does oxytocin cure xenophobia?
  181. Does peanut oral immunotherapy work?
  182. Does physical activity reduce stress?
  183. Does smoking reduce obesity?
  184. Does smoking weed improve your memory?
  185. Does sniffing rosemary improve your memory?
  186. Does soda kill hundreds of thousands of people a year?
  187. Does stevia improve focus?
  188. Does stevia improve memory?
  189. Does sugar kill more people than cigarettes?
  190. Does swearing improve stamina?
  191. Does taking birth control pills increase your risk of death?
  192. Does the U.S. Navy see climate change as a threat?
  193. Does the creator of Minecraft believe Pizzagate is real?
  194. Does the president of France spend $30,000 on makeup?
  195. Does toothpaste cure acne?
  196. Does toothpaste help a burn?
  197. Does visiting the beach change your brain?
  198. Has Cuba found a cancer vaccine?
  199. Has Melania Trump issued a ban on Monsanto products in White House?
  200. Has a one trillion dollar lawsuit been filed against media for “staging” Sandy Hook?
  201. Has chlorpyrifos, a pesticide, been linked to autism?
  202. Has the U.S. killed more than 20 million people in “victim nations” since World War II?
  203. Have 59% of Millennials raised in a church dropped out?
  204. Have Greenland’s polar ice caps melted past the point of no return?
  205. Have Monarch butterfly numbers been dropping?
  206. Have anti-white hate crimes in LA nearly doubled from the previous year?
  207. Have neuroscientists discovered a song that reduces anxiety by 65 percent?
  208. How dangerous is Cancun right now?
  209. How dangerous is mold?
  210. How does childhood poverty in the U.S. compare to other nations?
  211. How have American views of Nazis changed over time?
  212. How many Indians are at risk from rising sea levels?
  213. How many Indians are in the U.S.?
  214. How many black slaves were there in 1850?
  215. How many people does air pollution kill each year?
  216. How much does U.S. life expectancy vary by county?
  217. How much more do white men make than black women?
  218. How much more expensive are elections now than the late 1800s?
  219. In June 2017, was a first grader’s artwork hung at the Met?
  220. Is Alaska’s permafrost thawing?
  221. Is America’s biggest bank stealing land from indigenous tribes?
  222. Is California about to tax clean drinking water?
  223. Is China ahead of the U.S. in robots?
  224. Is China ahead of the U.S. in solar?
  225. Is China now a majority stockholder in Walmart?
  226. Is China working on climate change?
  227. Is FIFA considering banning players from making “cross” sign?
  228. Is Fukushima dumping hundreds of tons of dangerous water into the Pacific?
  229. Is Osama Bin Laden’s son about to take over Al Qaeda?
  230. Is Sweden safe to visit?
  231. Is Vladmir Putin richer than Bill Gates and Jeff Bezos combined?
  232. Is a Kentucky coal company planning to build the state’s largest solar farm?
  233. Is alcohol a carcinogen?
  234. Is being annoyed at others chewing a sign of genius?
  235. Is being jobless better for your health than a bad job?
  236. Is cancer a fungus?
  237. Is cigar smoking safer than cigarette smoking?
  238. Is dietary fat linked to a low sperm count?
  239. Is divorce contagious?
  240. Is sitting the new smoking?
  241. Is sodium glutamate dangerous?
  242. Is sodium glutamate gluten free?
  243. Is the Spix’s macaw extinct?
  244. Is the earth flat?
  245. Should kids get anesthesia for dental work?
  246. Was Common Core a part of No Child Left Behind?
  247. Was Common Core passed by Congress?
  248. Was Common Core repealed?
  249. Was Common Core started by Obama?
  250. Was Dr. Phil cancelled in Denmark for exposing a pedophile ring?
  251. Was Hitler financed by the Federal Reserve?
  252. Was Huma Abedin’s family business owned by a violent extremist group?
  253. Was a Las Vegas school counselor caught on camera kicking an ADHD student?
  254. Was a Maryland teen barred from graduation due to “immoral” pregnancy?
  255. Was a border patrol vehicle sprayed with manure after an argument?
  256. Was a first grade student punished for “misgendering” transgender classmate?
  257. Was an Egyptian academic accused of ‘glorifying Satan’ after teaching Milton’s Paradise Lost?
  258. Was celery the “avocado toast” of the Victorian era?
  259. Were HPV vaccine deaths hidden deliberately by researchers?
  260. Were Samuel L. Jackson and Magic Johnson mistaken for ‘lazy migrants’ in Italy?
  261. Were there early democratic societies in the Americas?
  262. Were two Indian engineers recently shot in Kansas as part of a suspected hate crime?
  263. What country has the highest STD rate?
  264. What county has the highest STD rate?
  265. What county has the highest crime rate in Florida?
  266. What county in the U.S. has the highest life expectancy?
  267. What county in the U.S. has the lowest life expectancy?
  268. What happened to Sylvia Plath’s missing novel?
  269. What is the chance of an asteroid killing most human life on earth?
  270. What is the lowest approval rating a President has had?
  271. What is the most spoken language in Africa?
  272. What percentage of America’s wealth do the richest one percent hold?
  273. What percentage of American children live in poverty?
  274. What percentage of French children live in poverty?
  275. What percentage of India’s wealth do the richest one percent hold?
  276. What percentage of Japanese children live in poverty?
  277. What percentage of motorists use a mobile phone when driving?
  278. What percentage of teen cases of HIV involve gay teens?
  279. What percentage of worldwide terrorism deaths occur in Europe and America?
  280. What percentage of San Franciscans can’t afford rent?
  281. Which state had the most slaves per captia in 1850?
  282. Who runs CNN?
  283. How many people does coal kill each year?
  284. Who runs FEMA?
  285. Who runs the inquirer.net site?
  286. Why are so many Millennials living with their parents?
  287. Why has the Japanese birth rate plummeted?
  288. Will eating more salt make you healthier?
  289. Will troops be deployed to halt illegal immigration?
  290. Would 36% of Brits consider having sex with a robot?
  291. Did Black Lives Matter block emergency crews from reaching Harvey victims?
  292. Did Oregon’s Governor sign a “gun confiscation” law?
  293. Were confederate soldiers considered U.S. veterans under law?
  294. Were pyramids discovered in Antarctica?
  295. Did John McCain admit to being a “war criminal”?
  296. Did a Johns Hopkins Scientist expose risks of flu vaccine?
  297. Was Bill Clinton expelled from Oxford over a rape incident?
  298. Was Robert E. Lee opposed to Confederate monuments?
  299. Is an Ohio cemetery exhuming bodies of Confederate soldiers?
  300. Were significant numbers of Irish slaves in America?
  301. Was a cop who arrested Malia Obama found dead?
  302. Is collecting rainwater illegal in some states?
  303. Will House Bill 610 defund education?
  304. Did an elementary school in California force students to cross dress for LGBT Week?
  305. Were 98 million Americans given cancer through Polio shots?
  306. Is wrapping string around a baby’s toes dangerous?
  307. Does eating carrots improve you vision?
  308. Do Ramen noodles use a wax that causes cancer?
  309. Was the body of missing White House intern Margaret Sanguay found?
  310. Did 179 kids go missing in Indiana in the first 100 days of 2017?
  311. Is a Facebook Drug Task Force now monitoring all Facebook posts?
  312. Have crime rates in Australia increased since its gun ban?
  313. Is high salt intake associated with double the risk of heart failure?
  314. Will there be more plastic in the ocean than fish by 2050?
  315. Does lemon aroma reduce blood pressure?
  316. Does lemon scent improve concentration?
  317. Do lemons reduce blood sugar?
  318. Do lemons help diabetes?
  319. Why do so many millennials have tattoos?
  320. What percentage of millennials have tattoos?
  321. Did German almost become the official language of the U.S.?
  322. Would America have won the Revolutionary War without French support?
  323. Can vigorous coughing prevent a heart attack?
  324. Did 234 people die eating contaminated Patti Labelle sweet potato pies?
  325. Does a single volcanic eruption contribute more to global warming than 100 years of humans?
  326. Are polar bears going extinct?
  327. Is the rainforest in danger?
  328. Can wrapping yourself in plastic wrap increase weight loss?
  329. Can robots have emotions?












Publishers and Platforms Need to Label Genres. Now, Please.

Today, from Medium. News about Trump!


See down there at the bottom? The headline about Trump? It’s yet another satirical headline showing up as like hard news. In 2018. A year and a half after we were supposed to fix this sort of thing. What’s going on?

So here’s a way to think about this. Think about my blog.

People have been coming to this blog for over a decade now, and the return visitors know what this blog is about. It’s analysis and opinion. It’s commentary. It’s not original research, it’s not hard news. It’s not satire.

For most of publishing history if you read an article you’d have a pretty good expectation of what genre to expect. If you picked up MAD Magazine you didn’t expect hard news. If you picked up the Washington Post, you could find various things in it, but they were mostly labeled to distinguish them and put in separate sections where possible.

Platforms tore away that context by taking everything and throwing it into one big bucket of content, platformatized and monetized. So now you get pieces in an endless feed from places you don’t even know; and where they are from places you know you don’t know *where* in the publication they are from or have any expectations about them. Which is why the avante garde of our current hoax and disinformation cycle was a bunch of liberals sharing unfunny Andy Borowitz columns they had had mistaken for New Yorker reporting. Again, if you had a subscription to the New Yorker, you’d know that section as the unfunny joke section towards the back. And on the site, it is clearly marked as satire, right up top:



But in Facebook’s endless homogenized feed, Borowitz looks like this:


I imagine it’s something similar with Medium above. The aggregation and recommendation tools strip this context out and repeatedly cause confusion.

It’s true that people do like having a standard interface for the feed, but the feed needs to figure out ways to parse this information and add the genre labels and indicators back in.

Sometimes you hear platforms companies complaining “Well, what do you want us to do?” Adding genre labels — and ideally then letting people the set the desired genre mix for their feed — is a simple solution that should have been done in 2014. The fact that it is 2018 and we’re still having this conversation is bizarre. Why not just get it done?

How To Read Laterally: A Lesson for New York Times Columnists Including But Not Limited to Bari Weiss

Today in the New York Times, a Bari Weiss column links to an OFFICIAL ANTIFA ACCOUNT that calls gay man Dave Rubin an anti-LGBT fascist. This is supposed to prove, according to Weiss, that the Left is out of control:

Dave Rubin, a liberal commentator who favors abortion rights, opposes the death penalty and is married to a man, yet is denounced as an “Anti-L.G.B.T. fascist” and a “fascist lieutenant” for criticizing identity politics.

This links to explosive tweets that show how civility has declined and — as she points out — everyone is being called a fascist now by liberals. Shocking example cited:


And …


Linking to things people said on Twitter to prove a broad sociological point is pretty 2016, but the bigger problem is that the account she links to — from the pages of the New York Times — is a troll/hoax account. It’s a fake account designed to take in gullible readers and outrage them into spreading classic disinformation, stirring up hate against real antifa, either for political reasons or lulz or some combination of the two.

There are numerous ways you can guess that fact — the name “Official Antifa” being the first hint — but there is also a simple way to check this one: read laterally. So let me show NYT columnists visiting this blog a thirty-second maneuver that can prevent further ridicule and degradation of public discourse by hoax accounts. Here we go. It takes less than thirty seconds so pop your headphones in from the start or you’ll miss it:


In this case, we select the Twitter handle, right-click to search on it, click the Google News tab to get a curated set of links/sources, and…

Actually that’s it. We’re done.


Now it’s not always this easy. Sometimes it takes a full minute, or 90 seconds. Occasionally it takes more than that and you have to use different, more complex methods. But it’s a pretty short process overall and one that you should be doing with every tweet you share, never mind ones you link to in the national newspaper of record.

We follow Sam Wineburg and Sarah McGrew’s groundbreaking media literacy research and call this technique reading laterally. It’s one of the four moves in the free eTextbook Web Literacy for Student Fact-Checkers, and a core technique in the online media literacy project we’re rolling out with the American Democracy Project, the Digital Polarization Initiative. Ninety-second fact-checks, based on new principles, that help students quickly verify and contextualize information.

I know I’ve written this glibly. Believe me, it’s because if I am not glib I’ll just get angry. This is work from the major national newspaper of record, not a gullible uncle posting on Facebook. And while Weiss claims to care about the disintegration of public discourse, she demonstrates the exact lack of care and skill that had led to it, allowing herself to be used, easily and fruitfully, by the crudest sort of manipulation.

The book is free. It’s about a two hour read. Please read it and stop linking to trolls.




Media Literacy Is About Where To Spend Your Trust. But You Have To Spend It Somewhere.

A lot of past approaches to online media literacy have highlighted “debunking” and present a large a portion of cases where students debunk tree octopuses and verifiably false things. And show students how they are manipulated, etc.

And this is good in the right amounts. There’s a place for it. It should comprise much of your curriculum.

But the core of media literacy for me is this question of “where you spend your trust.” And everything has to be evaluated in that framework.

There’s not an option to not trust anyone, at least not an option that is socially viable. And societies without trust come to bad ends. Students are various, of course, but what I find with many students is they are trust misers — they don’t want to spend their trust anywhere, and they think many things are equally untrustworthy. And somehow they have been trained to think this makes them smarter than the average bear.

A couple stories will illustrate the problem. I was once working with a bunch of students and comparing Natural News (a health supplements site which specializes in junk science claims) and the Mayo Clinic, one of the most respected outfits out there. OK, I say, so what’s the problem with taking advice from Natural News?

Well, says a student, they make their money selling supplements, and so they have an incentive to talk down traditional medicine.

I beam like a proud papa. Good analysis!

“And,” the student continues, “the Mayo Clinic is the same way. They make money off of patients so they want to portray regular hospitals as working.”

Houston, we have a problem.

I was in an upper division class another time and we were looking at an expert in a newspaper cited for his background in the ethnobiology of issues around the study of birds. I did what I encourage students to do in such cases: as a sanity check, make sure that the person being quoted as an academic expert has a publication record in the relevant area, preferably with a cite or two. (There are other varieties of expertise, of course, but in this case the claimed expertise was academic).

The record comes up. This guy’s top article on birds, biologists, and indigenous knowledge has something like 34 citations in Google Scholar. “So what do you think?” I ask them.

“Eh,” they say. “Not great.”

This was, mind you, not a room full of published ethnobiologists. And the ethnobiologist quoted in the article was not claiming to overturn the fundamental insights of ethnobiology, or anything requiring extraordinary evidence.

So 34 other experts had considered this person’s niche work worth talking about but hey, we’re still not sure this guy’s worth listening to on a subject we know nothing about and in which he is making rather moderate claims…


Another class, looking at Canadian paper the National Post, noted that while it was a “real” paper with a real staff, the Wikipedia page on it noted a controversy about some wrong information they published in 2006, where the editor had to actually pen an apology. “So kind of half-and-half, right?”

I’ve referred to this before as trust compression, the tendency for students to view vastly different levels of credibility of sources all as moderately or severely compromised. Breitbart is funded by the Mercers, who are using it directly to influence political debate, but the Washington Post is also owned by Jeff Bezos who donated to Democrats. So it’s a wash. And yes, we have the word of an expert in a subject where she has multiple cites against the word of a lobbying group but neither one is perfect really. Everyone’s got an agenda, nobody knows everything, and there’s not 100% agreement on anything anyway.

You see this in areas outside of expertise as well, incidentally. With quotes I often ask students (and faculty!) to source the quote and then say if the quote was taken out of context. The answer? You’ll always get a range from “completely taken out of context” to “somewhat taken out of context”. That upper register of “Nope, that quote was used correctly” is something you really have to coax the students into.

I don’t quite know how to square this with the gullibility often on display, except to say that very often that gullibility is about not being able (or willing) to distinguish gradations of credibility.

This should scare you, and it has to be at the core of what we teach — to teach students they need to decompress their trust, get out of that mushy middle, and make real distinctions. And ultimately, put their trust somewhere. Otherwise we end up with what Hannah Arendt so accurately described as the breeding ground of totalitarianism:

In an ever-changing, incomprehensible world the masses had reached the point where they would, at the same time, believe everything and nothing, that everything was possible and that nothing was true… Mass Propaganda discovered that its audience was ready at all times to believe the worst, no matter how absurd, and did not particularly object to being deceived because it held every statement to be a lie anyhow…

I do believe this insight — that trust has to be spent somewhere and that our problem is not gullibility, but rather the gullibility of cynics — has to be at the core of what we teach and how we teach it. You have some trust, and you have to be willing to spend it somewhere. So enough of the “this isn’t great either”, enough of the “eh”. What’s your best option for spending that trust? Why?

If everything is compromised, then everything can be ignored, and filtering is simply a matter of choosing what you want to hear. And students will economize that lesson in a heartbeat. In fact, I’m worried they already have, and it’s up to us to change that.

The Three Acts of Online Media Literacy Lessons: A First Pass

Some years ago, Dan Meyer pioneered and promoted a structure of math lessons based on three “acts” that fit together in a way that gave lessons a momentum and rhythm in the way that three act structure in film gives films (or TV shows or whatever) a structure and a rhythm. The acts as I understand them are:

  • an engaging and perplexing Act One,
  • an information and solution seeking Act Two,
  • a solution discussion and solution revealing Act Three.

I’ve been trying to explain how we structure our lessons in Digipo, and while this may be a slight (or substantial?) perversion of the Three Act structure I feel like we have three acts as well.

  • An initial focused investigation (with our 90-second fact-checks)
  • A broader look at structural issues and social impact
  • A move to personal and civic interventions

This is very raw, and nascent. I haven’t refined the terms here, and maybe this doesn’t work. Yadda yadda yadda. But let’s give this a try.

Act One: The Initial Case

We take an artifact on as a case, and run it through our fact-checking techniques. So for example, we can take this picture of Carl Sagan that appeared after Elon Musk launched a Tesla into space as a promotional stunt.


It’s like Carl Sagan was a prophet, foreseeing the commercialization of space from the 1970s.

So is it true? We put the students immediately on it, and see what they come up with.

After giving the students five or so minutes to work this out. We walk around the classroom and have discussions about their individual strategies. For the students that finish early we come up with some additional detail they can track down.

Then we have the students share their methods with the class, and show our solution, which is usually (but not always) a bit more efficient than the student’s methods.

In this case the solution starts with a reverse image search, which immediately suggests this is the Pioneer Plaque that has been altered:

pioneer plaque.PNG

We see there is another image, widely disseminated of Sagan holding up this Pioneer Plaque. It’s out there earlier than the other version. If you read up on the Pioneer Plaque even briefly you realize this was a major effort of Sagan’s — this message from earth to space aliens that attempted to communicate using a language an alien culture might be able to decipher.


If you want to get real CSI on it, you can even note that the depiction of the solar system on the bottom of the Pioneer Plaque was not even erased to make the new one:


Case solved, right?

Yeah. Kind of.

Act Two: The Story Behind the Story

Maybe I watch too many police procedurals. But the way procedurals in a long arc series work is often that there is a small case that comes across the desk of a detective. Someone commited suicide, or wandered out into traffic. Made a public disturbance. Broke into an office. The initial case is solved but a larger story is uncovered. The first act gives you the what. Maybe the how. But Act Two is about the why.

The larger story for us is usually the structural issues or the social impact. In this case there’s a number of stories we can pursue, but here’s one I particularly like.

Was it originally humor? Does it matter? The original image seems to have been a photoshop that has been around since at least 2012, and was recently posted to an anti-Elon Musk subreddit on Reddit:


Have students look at the page and decide whether the people who originally shared this knew it was fake or not. The signals on the page make it pretty clear people saw it as a Photoshop job and were using it as a joke with a point. 

So why did it get shared as real? Get your students to think about how meaning and context changes when a photo or quote moves from one subculture to another. What would have been different about the audience and context on that subreddit (knowledge of Sagan, knowledge of the ‘OP’ — the ‘original poster’ on Reddit, the way that comments stay with the photo, a common culture of joking through photoshopping) versus the other contexts (Instagram, Twitter, Facebook) to which it propagated?

There are some other questions I put in the guide to this. You could look at the phenomenon of “sign-holding” in social media, and how this technique, which is meant to prove authenticity, is perverted by Photoshopping. You could talk about this compulsion to attach our current thoughts to dead figures of the past, via the fake quotes that are ever popular. In each of these cases, the question is “What is the larger social, psychological, and political situation that drives this phenomenon?”

My experience is that doing this for Act Two after a very analytical and dry Act One just works in some magical way that is hard to describe. The initial investigation grounded them in a context and got them to think analytically, and now they bring that into the larger conversation.

I’ll also say that pedagogically you can’t do one without the other. The second act imbues the first act with a social meaning that straight fact-checking and source-verification lacks.

Act Three: The Plan for Action

In Digipo we have three goals for our program. Students will be able to:

  • Perform basic verification and contextualization tasks.
  • Understand the larger social structures behind and social impact of mis-, dis-, and mal-info.
  • Design personal, civic, and political interventions that improve our shared information environment.

That last part has been key for us from the beginning. So you see a mistake online, or someone being harrassed, or learn about the ways in which trolling is used to silence people. What do you do?

In Act Three, with the students fresh from talking about the larger issues, we focus on both small and large interventions for these problems.

In this case we ask the students to imagine that this has been posted by a friend who seems to think it is real. Do you correct them? Do you do it publicly? Through DM?

Do you maybe do it subtly, pretending that they know that the picture is a photoshop job. “Cool picture! Have you ever seen the original?” along with a link to the original.

Maybe you reshare it yourself, but provide the context “I know this is fake, but it’s how I feel right now.” What do they think would work best? Or does it even matter? Why or why not?

Interventions aren’t always personal or technological. We can talk about that a bit more with some other examples, but sometimes there’s a policy discussion that has to happen. Sometimes it’s a discussion about what a better platform interface might look like — you can have your students design platform UI solutions to an issue, to think through the way in which a UI creates sometimes harmful behavior.

But there’s the sequence:

  • The Initial Case
  • The Story Behind the Story
  • The Plan for Action

I admit this is clumsy and a bit bloated compared to Dan’s work. But I think there’s something here we can get to.

Let’s take a few quick examples and sketch out how they might work.

More Examples

In the Now

In the Now is a hip (or trying to be hip) video channel for politically engaged 20-somethings available on multiple platforms. Here’s a promo for it:

Act One: The Initial Case

  • Show the In the Now video
  • Question: who runs this? What’s their agenda?
  • (Students investigate)
  • Reveal methods: Either Wikipedia search, or a search for news mentions gets you there really quickly.
  • Answer: Turns out it’s the Russians actually. 😦

Act Two: The Story Behind the Story

  • Drill down on the style of In the Now, which bills itself as a site for free-thinking people. People going against the grain.
  • Discuss how this technique is often used to pull people into conspiracy theories. Solicit student experiences with this. Why is this so attractive?
  • Add some complexity: sometimes going against the grain is good. How do you tell the difference?
  • Bring up the BBC comparison — why is Russian media different from the BBC or NPR? Those are state-funded too, right?

Act Three: The Plan for Action

  • Note the new disclaimer on the video — this is state-funded media. Discuss whether YouTube do more to indicate the funding sources of videos. Also: is it unfair to single out state-funded entities?
  • Does this deception even matter, given the low view counts on this video? Look through and see if this channel has had breakout successes. What subjects seem to have worked?


Harvard Domestic Violence Study


Act One: The Initial Case

  • Show the above image
  • Question: Is this statistic true? Is the source credible?
  • (Students investigate)
  • Reveal methods: Taking selected text and putting into Google, scanning for authoritative URL.
  • Answer: It is a correct statistic, but slightly decontextualized as it does not separate minor incidents and serious injury. Ask the students if they thought it was correct before checking it. Why or why not?

Act Two: The Story Behind the Story

  • In this case let’s focus on the photo. It’s not from the study, and the study said nothing about race, so why this photo of a black couple?
  • Discussion of the Angry Black Woman meme. Is this image harmful? Does it reinforce stereotypes? What is the overall social impact?
  • Can we think of some other harmful stereotypes that we see in photos? Which do you think impacts us more? Why do images with these photos often go more viral? What examples do students see? Can they think of some “from the left”?

Act Three: The Plan for Action 

  • Let’s assume you want to reshare this article because you think it’s interesting. What’s a better way to share it? What would be better text?
  • For a bigger project, have students actually design a Pinterest Pin image that summarizes an important research article in a way that promotes sharing while not distorting the findings or using stereotypes. (One thing to note here — we used to push students into long written projects — wiki and the like — but are embracing smaller more various projects  now.)


These examples only scratch the surface of the things we want to talk about. Clickbait, adtech, bots, othering, tribalism, identity, trust, journalistic integrity  — all these things can make a good Act Two, and if you look through Four Moves, we have the activities to support them.

Anyway — this is a work in progress, but gets at both the rhythm of the class sessions and the three necessary elements of online media literacy (skills, structural perspective, action for change). Curious to see what people think. I’ll eventually take a better pass at laying this all out.




Recognition Is Futile: Why Checklist Approaches to Information Literacy Fail and What To Do About It

The following is a provocation for #EngageMOOC. Thanks to Bonnie Stewart and the rest of the #EngageMOOC crew for inviting me to contribute.

Whooping Cough

When I was in my twenties I went to the doctor with a cough I believed was whooping cough due to the tell-tale “whoop” intake of breath that occurred after I had coughed myself blue. It came sporadically towards the end of the day, so when I went to the doctor one morning I had to describe it without him actually seeing what I was experiencing.

He asked me a lot of questions: how it felt, my medical history and habits. And one of the many questions he asked was whether I smoked. I said I did.

And though he asked other questions, that was basically it for the doctor. As far as he was concerned it was smoker’s cough and I should quit smoking. He prescribed some cough syrup and sent me on my way. I stared in amazement but took the script.

I went home, and it got worse.  I went back in, and this time got a physician’s assistant.

But this time it went very differently. The assistant asked me what was wrong, and I said I thought I had whooping cough. Rather than proceed to other questions, he stopped and left the room, returning in a couple minutes.

“I just checked our notices,” he said, “And it looks like there was an outbreak of pertussis a little north of here. I’m going to put you on a broad spectrum antibiotic, and check back with you next week.”

And with that he wrote me out a script for erythromycin, and I was on my way. In a short time I was better.

“Recognizing” Fake News

Most educational approaches promoted as solutions to fake news look decidedly like the first doctor’s method. Take in everything you can about the item you are looking at, and see if you can recognize it for what it is. Take the Newseum’s “E.S.C.A.P.E. Junk News” method. Students are asked to look at stories and evaluate them along six multi-faceted dimensions:

Do the facts hold up?
Look for information you can verify:

  • Names
  • Numbers
  • Places
  • Documents

Who made this and can I trust them?
Trace who has touched the story:

  • Authors
  • Publishers
  • Funders
  • Aggregators
  • Social Media Users

What’s the big picture?
Consider if this is the whole story and weigh other force surrounding it.

  • Current events
  • Cultural trends
  • Political goals
  • Financial pressures

Who is the intended audience?
Look for attempts to appeal to specific groups or types of people.

  • Image choices
  • Presentation Techniques
  • Language
  • Content

Why was this made?
Look for clues to the motivation

  • The publisher’s mission
  • Persuasive language or images
  • Moneymaking tactics
  • Stated or unstated agendas
  • Calls to action

How is this information presented?
Consider how the way it’s made affects the impact.

  • Style
  • Grammar
  • Tone
  • Image choices
  • Placement and layout

Now, you might think a person filling out this exhausting battery of questions would make a good decision on what is credible and what is not. But research suggests otherwise. In fact, what we know from studies of expertise in many fields is such exhaustive holistic assessments can make the evaluator more prone to error.

Why? Because in the end, any such list of attributes is going to point in many different and contradictory directions, and your exhausted mind — which cannot hold this much information in working memory at one time — will find a way to take a shortcut. Maybe it will choose to notice more salient features over less salient ones. Maybe it will fall back on racism, bigotry, stereotypes. Or resort to confirmation bias. Maybe it will just give up entirely.

In medicine, this can have deadly consequences, which is why physicians today are less likely to write down all available symptoms and look for the magic connect-the-dots disease, and more likely to walk down simple decision trees. “Do you have a family history of heart disease?” ends up early in that sequence if you walk in with chest pain, because depending on that answer, the questions will change. The questions have to change. If there is a family history of heart disease, and a patient is presenting with chest pain, the doctor is not going to spend much time messing around with questions about acid reflux. They are going to rule out some other simple causes and get you off to some tests.

Likewise, the physician’s assistant in my introductory story heard symptoms that might be pertussis and might be smoker’s cough. But rather than collecting as much information as possible and evaluating that holistically he sought an answer to the one question that mattered: had there been an outbreak? Pertussis outbreaks are still fairly rare. If there was an outbreak, there was a decent chance what I had was whooping cough. If there was no outbreak, the chance that it was whooping cough was vanishingly small.

I’ve talked a lot in the past about how recent research by Sam Wineburg and Sarah McGrew demonstrated that effective fact-checkers “got off the page” they were evaluating,  “using the network to check the network.” Historians and students in their study tried to evaluate whether something was credible by reading it closely. Fact-checkers, on the other hand, immediately opened other tabs and saw what Wikipedia or Google Scholar had to say about the source. The students and historians performed poorly using their method, with many unable to distinguish material put out by blatant political advocacy groups from material put out by widely respected professional groups. The fact-checker’s methods, on the other hand, got them to the right answers in a fraction of the time.

But it’s not just about getting off the page — it’s also about asking the most important questions first, and not getting distracted by salient yet minor details, or becoming so overloaded by evaluation your bias is allowed free rein.

The Fast and Frugal Logic of the Four Moves

Based on these concerns, I came up with the “four moves” of fact-checking and source verification. For students trying to ascertain the truth of a story on the web less is often more, and what students need are not long lists of attributes to weigh in some complex holistic calculus, but quick and directed moves that solve simple scenarios quickly and complex scenarios in a reasonable amount of time. The four moves I came up with were:

  • Check for previous work. Most stories you see on the web have been either covered, verified, or debunked by more reputable sources. Find a reputable source that has done your work for you. If you can find that, maybe your work is done.
  • Go upstream to the source. If you can’t find a rock-solid source that has done your verification and context-building for you, follow the story or claim you are looking at to it’s origin. Most stories shared with you on the web are recoverage of some other reporting or research. Follow the links and get to the source. If you recognize the source as credible, your work may be done.
  • Read laterally. If you have traced the claim or story or research to the source and you don’t recognize it, you will need check the credibility of the source by looking at available information on its reliability, expertise, and agenda.
  • Circle back. A reminder that even when we follow this process sometimes we find ourselves going down dead ends. If a certain route of inquiry is not panning out, try going back to the beginning with what you know now. Choose different search terms and try again.

They are structured so that you can quickly eliminate hoaxes and egregiously wrong stories. Here’s an example of how that might work. Say you see a story that says that Jennifer Lawrence has died:


Well, has she died?

The ESCAPE method, like the RADCAB and CRAAP methods before it ask you to look at the evidence, think about the source, consider the context, audience, purpose, tone, grammar, style and so forth to figure out whether this is reliable.

The four moves, on the other hand, proposes to answer simple questions first. If Jennifer Lawrence has died, there should be wall to wall coverage of that, right? So check for previous work, and look to see if reliable outlets are covering this. If she is dead, they will be. If she is not, they won’t. Lo and behold, if we type [[Jennifer Lawrence dead]] into Google News we don’t find any stories about her dying:


That’s done in five seconds. But of course not all questions are that easy. Consider the following image found floating around Pinterest:


A short search of Google News doesn’t show any relevant coverage of the study. So let’s go to the next level and see if we can find the source.

There’s a link typed on the image, but rather than deal with that we do a Google Search and immediately hit the jackpot, immediately seeing a Google result on the issue that comes from Harvard.


We can go to that page and find the study is more nuanced than what is presented in the image (and also that there is no race data in the study — the image here is purely an attempt to use the trope of the “angry black women” to get more clicks).

Consider the difference had we spent time looking at the provider of the image for bias, tone, evidence, purpose and the like (a futile process we call “fact-checking the mailman”). By following the moves we quickly get to the most authoritative source for the fact and work from there, in the original context.

Finally, if we don’t trust the original source, or have never heard of it, we can “read laterally” (a term borrowed from Wineburg’s group) and find out more about the organization or publisher. Here we do a quick read on the source of the above study:

public health.PNG

We find that the URL is correct and that the school is considered a leading school of public health in the United States.

Recognition is Futile

These are just a couple of the most simple examples. As you will see in the text of Web Literacy for Fact-Checkers, they can be used in more complex scenarios as well. Reading laterally can involve checking out an expert’s publication record, for example. Tracing a photo upstream to the source might involve reverse image search. Certain site-specific searches can make finding previous work easier.

But all these techniques tend to avoid any complex tasks of recognition or similarity. Recognition is futile for a number of reasons. It overloads cognitive capacity,  draws attention to easily manipulable surface features,  and makes solving even the simplest problems time-consuming. Additionally, as surface features become easier and easier to fake, recognition as a model will become less and less useful.

We have only touched the surface of the Four Moves approach here, and it currently being somewhere just short of 2 a.m. on a Saturday night we’ll have to stop here for now.  I hope you’ll explore both the textbook and the activities blog.

But the broad point here is to move away from the idea that we want students to “recognize” fake news. Instead we want to give students a process to quickly verify and contextualize news. That’s a very different approach than what we’ve typically done in the past, but it’s a shift we have to make if we want to empower our students to make the fast and accurate assessments the current information environment requires.