Media Literacy Is About Where To Spend Your Trust. But You Have To Spend It Somewhere.

A lot of approaches to online media literacy highlight “debunking” and present too large a portion of cases where students debunk tree octopuses and verifiably false things. And show student how they are manipulated. And there’s a place for that. It should comprise much of your curriculum.

But the core of media literacy for me is this question of “where you spend your trust.” And everything has to be evaluated in that framework.

There’s not an option to not trust anyone, at least not an option that is socially viable. And societies without trust come to bad ends. But what I find with many students is they are trust misers — they don’t want to spend their trust anywhere, and they think many things are equally untrustworthy. And somehow they have been trained to think this makes them smarter than the average bear.

A couple stories will illustrate the problem. I was once working with a bunch of students and comparing Natural News (a health supplements site which specializes in junk science claims) and the Mayo Clinic, one of the most respected outfits out there. OK, I say, so what’s the problem with taking advice from Natural News?

Well, says a student, they make their money selling supplements, and so they have an incentive to talk down traditional medicine.

I beam like a proud papa. Good analysis!

“And,” the student continues, “the Mayo Clinic is the same way. They make money off of patients so they want to portray regular hospitals as working.”

Houston, we have a problem.

I was in an upper division class another time and we were looking at an expert in a newspaper cited for his background in ethnobiology of cultural intersections with birds. I did what I encourage students to do which is make a sanity check and make sure that the person that was being quoted as an academic expert had a publication record in the relevant area, preferably with a cite or two. (There are other varieties of expertise, of course, but in this case the claimed expertise was academic).

The record comes up. This guy’s top article on birds, biologists, and indigenous knowledge has something like 34 citations in Google Scholar. “So what do you think?” I ask them.

“Eh,” they say. “Not great.”

This was, mind you, not a room full of published ethnologists. And the ethno-biologist quoted in the article was not claiming to overturn the fundamental insights of ethnobiology.

So 34 other experts had considered this person’s niche work worth talking about but hey, we’re still not sure this guy’s worth listening to on a subject we know nothing about and in which he is making rather moderate claims…

Another class, looking at Canadian paper the National Post, noted that while a “real” paper with a real staff, the Wikipedia page on it noted a controversy about some wrong information published in 2006, where the editor had to actually pen an apology. “So kind of half-and-half, right?”

I’ve referred to this before as trust compression, the tendency for students to view vastly different levels of credibility of sources all as moderately compromised. Breitbart is funded by the Mercers, who are using it directly to influence political debate, but the Washington Post is also owned by Jeff Bezos who gave to Democrats. Yes, we have an expert in a subject with multiple cites against the word of a lobbying group but neither one is perfect really.

You see this in areas outside of expertise as well, incidentally. With quotes I often ask students (and faculty!) to source the quote and then say if the quote was taken out of context. The answer? You’ll always get a range from “completely taken out of context” to “somewhat taken out of context”. That upper register of “Nope, that quote was used correctly” is something you really have to coax the students into.

I don’t quite know how to square this with the gullibility often on display, except to say that very often that gullibility is about not being able (or willing) to distinguish gradations of credibility.

This should scare you, and it has to be at the core of what we teach — to teach students they need to decompress their trust, get out of that mushy middle, and make real distinctions. Otherwise we end up with what Hannah Arendt so accurately described as the breeding ground of totalitarianism:

In an ever-changing, incomprehensible world the masses had reached the point where they would, at the same time, believe everything and nothing, that everything was possible and that nothing was true… Mass Propaganda discovered that its audience was ready at all times to believe the worst, no matter how absurd, and did not particularly object to being deceived because it held every statement to be a lie anyhow…

I do believe this insight — that trust has to be spent somewhere and that our problem is not gullibility, but rather the gullibility of cynics — has to be at the core of what we teach and how we teach it. You have some trust, and you have to be willing to spend it somewhere. So enough of the “this isn’t great either”, enough of the “eh”. What’s your best option for spending that trust? Why?

If everything is compromised, then everything can be ignored, and filtering is simply a matter of choosing what you want to hear. And students will economize that lesson in a heartbeat. In fact, I’m worried they already have.

Advertisements

The Three Acts of Online Media Literacy Lessons: A First Pass

Some years ago, Dan Meyer pioneered and promoted a structure of math lessons based on three “acts” that fit together in a way that gave lessons a momentum and rhythm in the way that three act structure in film gives films (or TV shows or whatever) a structure and a rhythm. The acts as I understand them are:

  • an engaging and perplexing Act One,
  • an information and solution seeking Act Two,
  • a solution discussion and solution revealing Act Three.

I’ve been trying to explain how we structure our lessons in Digipo, and while this may be a slight (or substantial?) perversion of the Three Act structure I feel like we have three acts as well.

  • An initial focused investigation (with our 90-second fact-checks)
  • A broader look at structural issues and social impact
  • A move to personal and civic interventions

This is very raw, and nascent. I haven’t refined the terms here, and maybe this doesn’t work. Yadda yadda yadda. But let’s give this a try.

Act One: The Initial Case

We take an artifact on as a case, and run it through our fact-checking techniques. So for example, we can take this picture of Carl Sagan that appeared after Elon Musk launched a Tesla into space as a promotional stunt.

unnamed

It’s like Carl Sagan was a prophet, foreseeing the commercialization of space from the 1970s.

So is it true? We put the students immediately on it, and see what they come up with.

After giving the students five or so minutes to work this out. We walk around the classroom and have discussions about their individual strategies. For the students that finish early we come up with some additional detail they can track down.

Then we have the students share their methods with the class, and show our solution, which is usually (but not always) a bit more efficient than the student’s methods.

In this case the solution starts with a reverse image search, which immediately suggests this is the Pioneer Plaque that has been altered:

pioneer plaque.PNG

We see there is another image, widely disseminated of Sagan holding up this Pioneer Plaque. It’s out there earlier than the other version. If you read up on the Pioneer Plaque even briefly you realize this was a major effort of Sagan’s — this message from earth to space aliens that attempted to communicate using a language an alien culture might be able to decipher.

11230972_876352435765604_1496484349038422435_n.jpg

If you want to get real CSI on it, you can even note that the depiction of the solar system on the bottom of the Pioneer Plaque was not even erased to make the new one:

picture.jpg

Case solved, right?

Yeah. Kind of.

Act Two: The Story Behind the Story

Maybe I watch too many police procedurals. But the way procedurals in a long arc series work is often that there is a small case that comes across the desk of a detective. Someone commited suicide, or wandered out into traffic. Made a public disturbance. Broke into an office. The initial case is solved but a larger story is uncovered. The first act gives you the what. Maybe the how. But Act Two is about the why.

The larger story for us is usually the structural issues or the social impact. In this case there’s a number of stories we can pursue, but here’s one I particularly like.

Was it originally humor? Does it matter? The original image seems to have been a photoshop that has been around since at least 2012, and was recently posted to an anti-Elon Musk subreddit on Reddit:

https://www.reddit.com/r/EnoughMuskSpam/comments/7vxx4h/carl_sagan_on_using_space_to_sell_teslas/

Have students look at the page and decide whether the people who originally shared this knew it was fake or not. The signals on the page make it pretty clear people saw it as a photoshop and were using it as a joke with a point. 

So why did it get shared as real? Get your students to think about how meaning and context changes when a photo or quote moves from one subculture to another. What would have been different about the audience and context on that subreddit (knowledge of Sagan, knowledge of the ‘OP’ — the ‘original poster’ on Reddit, the way that comments stay with the photo, a common culture of joking through photoshopping) versus the other contexts (Instagram, Twitter, Facebook) to which it propagated?

There are some other questions I put in the guide to this. You could look at the phenomenon of “sign-holding” in social media, and how this technique, which is meant to prove authenticity is perverted by photoshopping. You could talk about this compulsion to attach our current thoughts to dead figures of the past, via the fake quotes that are ever popular. In each of these cases, the question is “What is the larger social, psychological, and political situation that drives this phenomenon?”

My experience is that doing this for Act Two after a very analytical and dry Act One just works in some magical way that is hard to describe. The initial investigation grounded them in a context and got them to think analytically, and now they bring that into the larger conversation.

I’ll also say that pedagogically you can’t do one without the other. The second act imbues the first act with a social meaning that straight fact-checking and source-verification lacks.

Act Three: The Plan for Action

In Digipo we have three goals for our program. Students will be able to:

  • Perform basic verification and contextualization tasks.
  • Understand the larger social structures behind and social impact of mis-, dis-, and mal-info.
  • Design personal, civic, and political interventions that improve our shared information environment.

That last part has been key for us from the beginning. So you see a mistake online, or someone being harrassed, or learn about the ways in which trolling is used to silence people. What do you do?

In Act Three, with the students fresh from talking about the larger issues, we focus on both small and large interventions for these problems.

In this case we ask the students to imagine that this has been posted by a friend who seems to think it is real. Do you correct them? Do you do it publicly? Through DM?

Do you maybe do it subtly, pretending that they know that the picture is a photoshop job. “Cool picture! Have you ever seen the original?” along with a link to the original.

Maybe you reshare it yourself, but provide the context “I know this is fake, but it’s how I feel right now.” What do they think would work best? Or does it even matter? Why or why not?

Interventions aren’t always personal or technological. We can talk about that a bit more with some other examples, but sometimes there’s a policy discussion that has to happen. Sometimes it’s a discussion about what a better platform interface might look like — you can have your students design platform UI solutions to an issue, to think through the way in which a UI creates sometimes harmful behavior.

But there’s the sequence:

  • The Initial Case
  • The Story Behind the Story
  • The Plan for Action

I admit this is clumsy and a bit bloated compared to Dan’s work. But I think there’s something here we can get to.

Let’s take a few quick examples and sketch out how they might work.

More Examples

In the Now

In the Now is a hip (or trying to be hip) video channel for politically engaged 20-somethings available on multiple platform. Here’s a promo for it:

Act One: The Initial Case

  • Show the In the Now video
  • Question: who runs this? What’s their agenda?
  • (Students investigate)
  • Reveal methods: Either Wikipedia search, or a search for news mentions gets you there really quickly.
  • Answer: Turns out it’s the Russians actually. 😦

Act Two: The Story Behind the Story

  • Drill down on the style of In the Now, which bills itself as a site for free-thinking people. People going against the grain.
  • Discuss how this technique is often used to pull people into conspiracy theories. Solicit student experiences with this. Why is this so attractive?
  • Add some complexity: sometimes going against the grain is good. How do you tell the difference?
  • Bring up the BBC comparison — why is Russian media different from the BBC or NPR? Those are state-funded too, right?

Act Three: The Plan for Action

  • Discuss whether YouTube do more to indicate the funding sources of videos.
  • What would that look like? Would it make a difference?
  • Does it even matter, given the low view counts on this video?

 

Harvard Domestic Violence Study

557003d2790c480d1cb6aac3e5e7198a1

Act One: The Initial Case

  • Show the above image
  • Question: Is this statistic true? Is the source credible?
  • (Students investigate)
  • Reveal methods: Taking selected text and putting into Google, scanning for authoritative URL.
  • Answer: It is a correct statistic, but somewhat decontextualized as it does not separate minor incidents and serious injury.

Act Two: The Story Behind the Story

  • In this case let’s focus on the photo. It’s not from the study, and the study said nothing about race, so why this photo of a black couple?
  • Discussion of the Angry Black Woman meme. Is this image harmful? Does it reinforce stereotypes? What is the overall social impact?
  • Can we think of some other harmful stereotypes that we see in photos? Which do you think impacts us more? Why do images with these photos often go more viral? What examples do students see?

Act Three: The Plan for Action 

  • Let’s assume you want to reshare this article because you think it’s interesting. What’s a better way to share it? What would be better text?
  • For a bigger project, have students actually design a Pinterest Pin image that summarizes an important research article in a way that promotes sharing while not distorting the findings. (One thing to note here — we used to push students into long written projects — wiki and the like — but are embracing smaller more various projects  now.)

Conclusion

These examples only scratch the surface of the things we want to talk about. Clickbait, adtech, bots, othering, tribalism, identity, trust, journalistic integrity  — all these things can make a good Act Two, and if you look through Four Moves, we have the activities to support them.

Anyway — this is a work in progress, but gets at both the rhythm of the class sessions and the three necessary elements of online media literacy (skills, structural perspective, action for change). Curious to see what people think. I’ll eventually take a better pass at laying this all out.

 

 

 

Recognition Is Futile: Why Checklist Approaches to Information Literacy Fail and What To Do About It

The following is a provocation for #EngageMOOC. Thanks to Bonnie Stewart and the rest of the #EngageMOOC crew for inviting me to contribute.

Whooping Cough

When I was in my twenties I went to the doctor with a cough I believed was whooping cough due to the tell-tale “whoop” intake of breath that occurred after I had coughed myself blue. It came sporadically towards the end of the day, so when I went to the doctor one morning I had to describe it without him actually seeing what I was experiencing.

He asked me a lot of questions: how it felt, my medical history and habits. And one of the many questions he asked was whether I smoked. I said I did.

And though he asked other questions, that was basically it for the doctor. As far as he was concerned it was smoker’s cough and I should quit smoking. He prescribed some cough syrup and sent me on my way. I stared in amazement but took the script.

I went home, and it got worse.  I went back in, and this time got a physician’s assistant.

But this time it went very differently. The assistant asked me what was wrong, and I said I thought I had whooping cough. Rather than proceed to other questions, he stopped and left the room, returning in a couple minutes.

“I just checked our notices,” he said, “And it looks like there was an outbreak of pertussis a little north of here. I’m going to put you on a broad spectrum antibiotic, and check back with you next week.”

And with that he wrote me out a script for erythromycin, and I was on my way. In a short time I was better.

“Recognizing” Fake News

Most educational approaches promoted as solutions to fake news look decidedly like the first doctor’s method. Take in everything you can about the item you are looking at, and see if you can recognize it for what it is. Take the Newseum’s “E.S.C.A.P.E. Junk News” method. Students are asked to look at stories and evaluate them along six multi-faceted dimensions:

Evidence
Do the facts hold up?
Look for information you can verify:

  • Names
  • Numbers
  • Places
  • Documents

Source
Who made this and can I trust them?
Trace who has touched the story:

  • Authors
  • Publishers
  • Funders
  • Aggregators
  • Social Media Users

Context
What’s the big picture?
Consider if this is the whole story and weigh other force surrounding it.

  • Current events
  • Cultural trends
  • Political goals
  • Financial pressures

Audience
Who is the intended audience?
Look for attempts to appeal to specific groups or types of people.

  • Image choices
  • Presentation Techniques
  • Language
  • Content

Purpose
Why was this made?
Look for clues to the motivation

  • The publisher’s mission
  • Persuasive language or images
  • Moneymaking tactics
  • Stated or unstated agendas
  • Calls to action

Execution
How is this information presented?
Consider how the way it’s made affects the impact.

  • Style
  • Grammar
  • Tone
  • Image choices
  • Placement and layout

Now, you might think a person filling out this exhausting battery of questions would make a good decision on what is credible and what is not. But research suggests otherwise. In fact, what we know from studies of expertise in many fields is such exhaustive holistic assessments can make the evaluator more prone to error.

Why? Because in the end, any such list of attributes is going to point in many different and contradictory directions, and your exhausted mind — which cannot hold this much information in working memory at one time — will find a way to take a shortcut. Maybe it will choose to notice more salient features over less salient ones. Maybe it will fall back on racism, bigotry, stereotypes. Or resort to confirmation bias. Maybe it will just give up entirely.

In medicine, this can have deadly consequences, which is why physicians today are less likely to write down all available symptoms and look for the magic connect-the-dots disease, and more likely to walk down simple decision trees. “Do you have a family history of heart disease?” ends up early in that sequence if you walk in with chest pain, because depending on that answer, the questions will change. The questions have to change. If there is a family history of heart disease, and a patient is presenting with chest pain, the doctor is not going to spend much time messing around with questions about acid reflux. They are going to rule out some other simple causes and get you off to some tests.

Likewise, the physician’s assistant in my introductory story heard symptoms that might be pertussis and might be smoker’s cough. But rather than collecting as much information as possible and evaluating that holistically he sought an answer to the one question that mattered: had there been an outbreak? Pertussis outbreaks are still fairly rare. If there was an outbreak, there was a decent chance what I had was whooping cough. If there was no outbreak, the chance that it was whooping cough was vanishingly small.

I’ve talked a lot in the past about how recent research by Sam Wineburg and Sarah McGrew demonstrated that effective fact-checkers “got off the page” they were evaluating,  “using the network to check the network.” Historians and students in their study tried to evaluate whether something was credible by reading it closely. Fact-checkers, on the other hand, immediately opened other tabs and saw what Wikipedia or Google Scholar had to say about the source. The students and historians performed poorly using their method, with many unable to distinguish material put out by blatant political advocacy groups from material put out by widely respected professional groups. The fact-checker’s methods, on the other hand, got them to the right answers in a fraction of the time.

But it’s not just about getting off the page — it’s also about asking the most important questions first, and not getting distracted by salient yet minor details, or becoming so overloaded by evaluation your bias is allowed free rein.

The Fast and Frugal Logic of the Four Moves

Based on these concerns, I came up with the “four moves” of fact-checking and source verification. For students trying to ascertain the truth of a story on the web less is often more, and what students need are not long lists of attributes to weigh in some complex holistic calculus, but quick and directed moves that solve simple scenarios quickly and complex scenarios in a reasonable amount of time. The four moves I came up with were:

  • Check for previous work. Most stories you see on the web have been either covered, verified, or debunked by more reputable sources. Find a reputable source that has done your work for you. If you can find that, maybe your work is done.
  • Go upstream to the source. If you can’t find a rock-solid source that has done your verification and context-building for you, follow the story or claim you are looking at to it’s origin. Most stories shared with you on the web are recoverage of some other reporting or research. Follow the links and get to the source. If you recognize the source as credible, your work may be done.
  • Read laterally. If you have traced the claim or story or research to the source and you don’t recognize it, you will need check the credibility of the source by looking at available information on its reliability, expertise, and agenda.
  • Circle back. A reminder that even when we follow this process sometimes we find ourselves going down dead ends. If a certain route of inquiry is not panning out, try going back to the beginning with what you know now. Choose different search terms and try again.

They are structured so that you can quickly eliminate hoaxes and egregiously wrong stories. Here’s an example of how that might work. Say you see a story that says that Jennifer Lawrence has died:

Rip+in+piece+so+4chan+started+some+project+with+the_7a88fa_5283146.png

Well, has she died?

The ESCAPE method, like the RADCAB and CRAAP methods before it ask you to look at the evidence, think about the source, consider the context, audience, purpose, tone, grammar, style and so forth to figure out whether this is reliable.

The four moves, on the other hand, proposes to answer simple questions first. If Jennifer Lawrence has died, there should be wall to wall coverage of that, right? So check for previous work, and look to see if reliable outlets are covering this. If she is dead, they will be. If she is not, they won’t. Lo and behold, if we type [[Jennifer Lawrence dead]] into Google News we don’t find any stories about her dying:

jlaw.PNG

That’s done in five seconds. But of course not all questions are that easy. Consider the following image found floating around Pinterest:

557003d2790c480d1cb6aac3e5e7198a1

A short search of Google News doesn’t show any relevant coverage of the study. So let’s go to the next level and see if we can find the source.

There’s a link typed on the image, but rather than deal with that we do a Google Search and immediately hit the jackpot, immediately seeing a Google result on the issue that comes from Harvard.

harvard.PNG

We can go to that page and find the study is more nuanced than what is presented in the image (and also that there is no race data in the study — the image here is purely an attempt to use the trope of the “angry black women” to get more clicks).

Consider the difference had we spent time looking at the provider of the image for bias, tone, evidence, purpose and the like (a futile process we call “fact-checking the mailman”). By following the moves we quickly get to the most authoritative source for the fact and work from there, in the original context.

Finally, if we don’t trust the original source, or have never heard of it, we can “read laterally” (a term borrowed from Wineburg’s group) and find out more about the organization or publisher. Here we do a quick read on the source of the above study:

public health.PNG

We find that the URL is correct and that the school is considered a leading school of public health in the United States.

Recognition is Futile

These are just a couple of the most simple examples. As you will see in the text of Web Literacy for Fact-Checkers, they can be used in more complex scenarios as well. Reading laterally can involve checking out an expert’s publication record, for example. Tracing a photo upstream to the source might involve reverse image search. Certain site-specific searches can make finding previous work easier.

But all these techniques tend to avoid any complex tasks of recognition or similarity. Recognition is futile for a number of reasons. It overloads cognitive capacity,  draws attention to easily manipulable surface features,  and makes solving even the simplest problems time-consuming. Additionally, as surface features become easier and easier to fake, recognition as a model will become less and less useful.

We have only touched the surface of the Four Moves approach here, and it currently being somewhere just short of 2 a.m. on a Saturday night we’ll have to stop here for now.  I hope you’ll explore both the textbook and the activities blog.

But the broad point here is to move away from the idea that we want students to “recognize” fake news. Instead we want to give students a process to quickly verify and contextualize news. That’s a very different approach than what we’ve typically done in the past, but it’s a shift we have to make if we want to empower our students to make the fast and accurate assessments the current information environment requires.

 

From Precinct to Voter

A summary of some reading from an old Wikity page. One way of thinking about current political trends is to see them as continuations of of trends brought about by other channels and uses of data dating back to the 1960s. In this telling, data and direct access to voters first erodes the precinct level in favor of national analysis and contact campaigns, available only through national party infrastructure. The commodification and availability of these channels online now starts to erode the power of parties as well.

Original article follows

——————-

Early political processes focused on precincts and wards as the unit of allocating campaign effort. You would get out the vote in the areas where you had broad support, and sometimes, less honorably, suppress the vote in those places where you didn’t.

As canvassing became supplemented with phone calls, direct mail, and other pieces, the unit increasingly became the individual voter. Campaigns were less concerned about getting out the vote of Ward 8 and more concerned about flushing out demographics such as “College educated under-25s”

As early as the 1960s Democrats began systematically assessing which precincts should be allocated campaign resources using statistics aggregated over fairly wide geographic areas. By the 1990s, the precinct was being supplanted by the individual voter as the unit of analysis, just as wall maps and clipboards were giving way to web applications and Palm Pilots. (Source)

As campaigns became more focused on these units of analysis, many traditional GOTV efforts became less privileged. And even where traditional methods were used, they were used under the guidance of the new approach — a ward captain in 1960 would get out the vote for his or her ward; by the 2000s they would be armed with data on what specific doors they needed to knock on, based on likely percentages of support and number of previous touches tracked.

Computational Propaganda and Totalitarianism (A Thread)

It Can Take As Little As Thirty Seconds, Seriously

 

I talk about 90-second fact-checks and I think people think I’m a bit unhinged sometimes. What can students possibly do in that short amount of time that would be meaningful?

A lot, actually.

For example, this press release on some recent research was shared with me today:

eureka alert.PNG

Now I want to re-share this with people, but I’d like to be a good net citizen as well. Good net citizens:

  • Source-check what they share
  • Share from the best source possible
  • Provide source/claim context to people they share with when necessary

To do that in this case we need to get to the source of the press release, on a site controlled by the American Psychological Association directly, and share that version of this. We also need to check that the American Psychological Association is the credible organization that we think it is. How long will this take?

Literally thirty seconds, if you know how to do it:

  • Select the headline, search on it.
  • First result up is from apa.org, that looks promising
  • Go there, look to make sure it’s the same release
  • Search Wikipedia for the site address. Find the article on the APA.
  • Check to make sure the APA is a real organization.
  • Check to make sure the APA web address matches

And you’re done. That may sound like a lot of steps, but each one is simple, fast, and fluid. Here are those steps executed in real time (video intentionally silent). I really encourage you to watch the video to see how ridiculously easy this is for someone with some training.

There’s really no excuse not to do this for things you share. It not only allows you to share from a more authoritative source, which is good for society and the economics of publishing, but it allows you provide your readers helpful context. Compare this:

aaas.PNG

To this:

apa2.PNG

I used to focus on students on writing longer research pieces on issues, and we still do that in various classes we work with. But just this behavior alone improves the world:

  • Check what you share
  • Share from the better source
  • Provide a context blurb to share your own source verification with others

You don’t need to write an essay. And most any student (or teacher!) can learn the techniques. Think of it as information hygiene, the metaphorical handwashing you engage in to prevent the spread of misinformation.

Learn the skills and make the world a better place. There may be good excuses for not doing this, but time is not one of them.

(Oh, and here’s that APA press release — it’s really interesting!)

 

Instead of letting people vote on news, Facebook should adopt Google’s rater system

A message I sent to a newsgroup on Facebook’s recent proposal that users could rate sites as a solution. To my surprise, I find myself suggesting they should follow Google’s model, which, while often faulty, is infinitely better than what they are proposing.

==============

(Regarding the announcement), I think there’s a better, time-tested way of doing this that doesn’t deal with individual ratings but benefits from expert analysis and insight. Use a modified version of the Google system.

Most people misunderstand what the Google system looks like (misreporting on it is rife) but the way it works is this. Google produces guidance docs for paid search raters who use them to rate search results (not individual sites). These documents are public, and people can argue about whether Google’s take on what constitutes authoritative sources is right — because they are public.

The raters rate search quality off the documents, and coders try to code to get the score up, but the two pieces are separate.

It’s not a perfect system, but it provides so many things this Facebook proposal doesn’t even touch:

  • It defines a common set of standards as to what a “good” result looks like, without going into specific sources.
  • It provides a degree of public transparency that Facebook doesn’t even come close to
  • It provides incentives for publishers to act in ethical ways, e.g. high quality medical or financial advice (“your money or your life” categories) must be sourced to professionals, etc.
  • It separates the target from the method of assessing whether the target is being hit
I’m not saying it doesn’t have problems — it does. It has taken Google some time to understand the implications of some of their decisions and I’ve been critical of them in the past. But I am able to be critical partially because we can reference a common understanding of what Google is trying to accomplish and see how it was falling short, or see how guidance in the rater docs may be having unintended consequences.

In such a system Facebook would hire raters who would rate feed quality — not individual sites — for a variety of criteria which experts have decided is characteristic of quality news feeds (and which readers by and large agree with). That would probably mean ascertaining whether sites included in the feed had the following desirable attributes:

  • Separation of opinion and analysis and news content, with opinion in particular clearly marked
  • Sponsored content clearly marked, and comprising small portion of overall site
  • Syndicated content clearly identified
  • Satire pieces marked unmistakably as satire
  • News stories clear about the process and methods by which they verified news in an article (e.g. “Kawczynski declined to be interviewed Sunday, but in posts on his website and on Gab…”)
  • A retraction policy, and an email address to send noted errors to
  • Headlines that match in tone and meaning the content of the attached article
  • Descriptive blurbs in Facebook that accurately describe the content of the article
  • Pictures which are either related to the event or marked as stock or file footage with descriptive and accurate captions
  • Links where appropriate to supporting coverage from other news outlets
  • A clear and accurate about page which defines who runs the paper
  • A lack of plagiarism — e.g. does the content pass the “Google paste test” for material not marked as syndicated.
Raters would rate the quality of the sources showing up in their feed and Facebook engineers would work on improving feed quality by getting the ratings up. No one gets banned or demoted by name. Or promoted by name.
The place of experts and the public would be to clarify what they trust in news. In fact, the Trust Project has already done much of the work that would go into feed quality rating docs. I summarize their work and my simplification of it here:
Again, the rater guidance documents get published. We continue to argue over whether the guidance is correct and whether the implementation is meeting the guidance or being gamed. We still raise holy hell about misfires and get them to rethink guidance and code.
The approach Facebook is currently proposing, on the other hand, is essentially nihilistic, and like many nihilistic things it may have current utility (and may even work temporarily), but provides a lousy foundation for dealing for problems to come.
Mike.

P.S. By and large I think you will find both that the public would rather trust expert opinion on what  constitutes quality than trust their neighbor, and that the public more or less agrees — both right and left — with the practices in the bulleted list above.