Hapgood

Mike Caulfield's latest web incarnation. Networked Learning, Open Education, and Online Digital Literacy


How to Analyze a News Claim and Publish the Analysis on Digipo.io

First, pick a claim. It will be associated with at least one news article, but note that you are analyzing the claim, not the article.

You can get a claim by going to the Open Claims page.

capture

The fact the link is red means the page hasn’t been created yet. When you click it you will have an option to create the page. Click the create button.

capture

When you create the page you get a form to fill out.

Please note — fact-checking is hard, and laborious and this is going to take you some time to do. So while this is a form, trust me, you are going to be here a while. Plan to take about two hours for this activity end to end. Anyway, the form looks  like this:

capture

Here’s one really important thing. DO NOT FILL OUT THE FORM IN ORDER. The order here is for the convenience of the reader, not the writer. Here is the order you will fill out the form in (with a couple loop-backs).

  • Claim
  • Prevalence
  • Origin
  • History Summary
  • Issues and Analysis
  • Status
  • Summary
  • Title
  • Further Reading

Ready? Here we go….

Claim

Add the claim in. You may end up editing this claim as you do the next step, but make a try at what the claim is. If you’ve come from the open claims page, look at whatever claim the link made.

Claim: In September 2016, a respected European physics journal published an article concluding that the World Trade Center Towers collapsed due to controlled demolition.

Note that the claim should be something falsifiable. Note too that you want to avoid “straw man” claims — as you investigate the claim make sure this is a claim that some people are actually making and that many others are actually hearing.

This claim is good because we can see if a journal published this article, if the article made that claim, and if the journal was respected. If all three things are true, then we can call the claim true. If all are false it’s false. If some are true it’s somewhere in between.

Prevalence

This is the most boring and laborious part. But it’s important. Here is where you look to see how widespread this claim is.

To do this, search for an document a few instances of your claim.  If you get absolutely bored you can stop at two or three instances, and let some other wiki author find more later.

How do you find them? Google is good. Bing works. There’s a lot of Google magic I can show you here, but let’s keep it simple for now.

capture

OK, so we found one (maybe two!) Now we go to the page and see if it claims this is supported by a physics journal. Note that right now we don’t know if this claim is true or not — we’re just trying to find examples of people making this claim.

We go to the page. Remember, what we’re collecting is claims a prestigious physics journal has come out for the 9/11 truther case of demolition.  We fire up hypothesis, since as we gather evidence we are going to track it.

capture

In hypothesis we’ll do a page level annotation, tagging this as:

  • digipo:claim, and
  • analysis:wtc_physics_journal (the name of our page)

capture

When you type the tags in and hit return they do this thing. You can get more information from hypothes.is on how to use their product.

You do this with a number of claims, and then you go and add them to your prevalence section:

capture

Here I’ve done something extra, which is added research on how many Facebook shares each one of these have, in order to show how widely this spread, but you don’t have to do that.

Whether you do a couple or a dozen, the point here is to see the claim in context. You want to do that to make sure you aren’t making the claim up. Our claim here is that people say a prestigious physics journal has come out for the “controlled implosion” theory of the World Trade Center collapse. By the end of this we see that this claim is widespread, and we even jot down some notes about the words each site uses.

Origin and History Summary

You won’t always have a clear origin to a claim. Sometimes is will be diffuse. Here the claim is not the journal per se, but the first sites to make the claim that a journal said this. Here we keep it short and sweet:

One of the earliest pieces of coverage was on 911 Blogger, although it’s reasonable to assume it hit the 9/11 Truth community pretty much at the same time.

We choose 911 Blogger as the first because in our research it has the earliest dates. If we find something earlier we can replace it.

We then write a short history summary that gives the sense of what we discovered about the way this spread:

Summary: The claim originated when Europhysics News published a critique of the traditional explanations for the WTC 9/11 building collapses. A network of “9/11 truther” sites quickly spread the claim that a scientific journal had confirmed that the collapses were the result of a controlled demolition.

We also scan the articles we have put together and tweak our claim a bit.  The more common claim is “prestigious European scientific journal” so we’ll go with that.

Claim: In September 2016, a prestigious European scientific journal published an article concluding that the World Trade Center Towers collapsed due to controlled demolition.

Issues and Analysis

Now, finally we get to the claim.  At some point we’ll list the tricks of the trade to quickly ascertain a claim. One of the keys, however, is to not forget what the claim *is*. In this case we could get dragged down a real hole of conspiracy theory if we engage with the demolitions claim. But our claim isn’t about demolitions or airplanes or the temperature at which steel melts. Our claim is that these fringe theories recently gained prestige by achieving publication in a prestigious scientific journal.

I won’t go through the whole process, but I will mention one thing — as I research, I am constantly highlighting evidence for and against the claim in articles I find and tagging it with

  • digipo:evidence, and
  • analysis:wtc_physics_journal (the name of our page)

capture

By the end of that process, I’ve got a lot of evidence I can access through my hypothesis dashboard. Above you see evidence I’ve gotten from a public letter from Europhysics News (the actual magazine, not journal, that ran the story). Each piece of evidence has a short note about what it shows.

I then write up the Issues and Analysis piece, which is admittedly the fun part. But now I can add links to to the hypothesis highlights, so that each claim I make can directly link to its verification.

Here’s our issues and analysis. The links work if you want to try them.

While the issue of the 9/11 building collapse is difficult to understand, we are dealing with the narrower claim of whether a prestigious physics journal published an article giving credence to a fringe theory. Key to the claim is the idea of authority: no one is claiming that new evidence has come to light here; the claim is that the scientific community is finally coming around to this conclusion, with journal publication being a first step.

On that count, the claim is mostly false. While it is true that Europhysics News published this article, they are not a peer-reviewed journal, but rather a magazine loosely associated with a prestigious society. The article was not submitted to any formal review process, and was in fact an invited article, the topic of which the editors were unaware. They have stated that they did not mean to endorse the ideas in the article, and have since put into place new processes designed to limit the freedom of authors writing invited articles.

The most popular articles went so far as to claim that the article was published in the actual journal of the European Physics Society, which is entirely wrong. Still, the fact that the article got even this level of acceptance is newsworthy. So we are rating this claim mostly false.

This is a close call, and you’ll notice it requires a judgement call about whether the spirit of the claim is supported or not. This is one of the reasons why it’s important to do your prevalence work first: it’s very clear doing that work that the major point of this claim in context is prestige.

Status and Summary

We go up and change the status from Under Investigation to Mostly False.  We want people to see this at a glance.

We then write the summary. The summery should introduce no new information, and should //not// be linked — all the links supporting it are in the sections below. This is meant to be a quick read for someone landing at the page.

This one I wrote for this is a bit long — I want to eventually cut it by a third to a half. But it gives an idea of what a summary is like. (Again, remember that the summary should not introduce facts not supported and linked in the investigation below):

Summary: The claim here is about the existence of such an article, not 9/11. Such an article was published, but it was published in a magazine associated with the European Physical Society, not a scientific journal. The magazine later explained that they have no review process in place for such articles and had published this as an invited article without being aware of the claims that would be made. They have stated that they have instituted new procedures that give them more oversight of such submissions. Since the core of this claim regards the credibility that a true journal article might provide to the controlled demolition theory, we rate this claim mostly false.

Further Reading

In the course of doing your research you’re likely to have found and read a bunch of things that give the bigger picture, the larger context. Put them here. Be careful not to introduce too much bias. I was tempted to add a great essay to further reading on why people believe conspiracy theories, but felt that might be too prejudicial. I chose instead to include two other sites that had debunked this claim, including Snopes.

Title

Finally, go up and give it a title. We’re split on this issue right now — what an appropriate title is — but our current thinking is that the title should not make a claim, since a) the claim may change, and b) unless you are careful with wording, headlines that rebut claims can reinforce belief in those claims.

We go with something that maybe hints at the result but forces the reader to read a bit more. Again, we’re looking at the best way to do this for the reader, and this convention may change.

capture

And that’s it. Here’s the page.

There’s other ways you can build out the page, ways you can organize your hypothesis annotations, add screenshots, and so on. But this is the skeleton of the activity:

  • Choose a claim.
  • Research the claim in context.
  • Document its history and prevalence.
  • Research the validity of the claim.
  • Document the sources which support and undermine the claim.
  • Write the issues and analysis, assembling links to evidence, and set the status of the claim.

That’s it. On average a claim will take anywhere from an hour to half a day to debunk. In general, the more precise the claim is, the more work it is: e.g. “Trump supporter threatens decorated cop in hijab.” takes longer to research than “Trump supporter threatens cop in hijab”,  and that takes longer than “Person threatens cop in hijab”.  Each adjective and noun is another verification challenge. So when starting out if it feels a bit overwhelming, start with simpler claims.

I’m curious what people’s experience with this process is. I realize as I typed this up that this was what I did as a political blogger back when I ran a political community — my posts were most often research projects of this sort, proving or disproving various claims made by politicians. I think what I’ve laid out here is pretty straightforward, but I’d be interested to know if I have underestimated the difficulty for people. Let me know.

And if you want a login to digipo, contact me!



3 responses to “How to Analyze a News Claim and Publish the Analysis on Digipo.io”

  1. […] through exercises to analyze and fact-check news stories. The pedagogical approach Mike describes here is evolving; in parallel I’ve been evolving a toolkit to help students research and organize […]

  2. By programmers for programmers.
    Yes yes yes of course blow me off. You know that I’m a know-nothing because you’re know all. Am I really that far off?

    I support officers and engineers because their egos are so fragile their as though allergic to substantial criticism. When the bully / charismatic psychopath pisses on them of course they tolerate … cuz they’re flexible and adaptive … right?

    Long/short: my cohort abandons / shit cans / monetizes even the slightest understanding of #SocialCognition (which me and mine were chatting about on the hippie bus late-60s) and validate their efforts by … well … what do careerists do? I can’t say. I avoided it like the plague. Pretending to be weak and cowardly.

    What I know about your projects for sure: they effectively elbow me aside … still … as always.
    Again, I’m a know nothing.
    CBC, Motorola, NORAD/SAC, ComInt … none of those credentials “matter” … they have no “weight” … I am not a person of “gravity” … why?
    #SocialCognition: you notice I didn’t suck up to you. (True. Intentional.) You project a septic heap of judgments on me. (Compulsive. On your part.) And therefore despise me. (Valence … not logic … not reason … valence. Like / Not Like. Us / Them. You’re successful careerists! It’s adaptive! You dance with the one who brung you! It’s merely thoughtlessness … no big deal.)

    Those who //talk/// about this stuff suck all the oxygen out of the room … as they always have. (You don’t think the cult leaders reward that sorta thing? of course you know it. And you get rewarded for denying it. Awww c’mon … you’ll pretend that “oligarch” is too arcane a term for you. [Dishonesty; false presence; negotiating in bad faith.])

    I’m a technologist. You are the priests of a religion that brought us Trump as PotUS.
    Like Clinton and Dems … you just can’t wrap your arms around reality.
    You sailed under the flag of piracy … and have been beaten by those more experience …
    … you really think we peons are gonna save your bacon?
    pfffffffffffffft

    NB:
    a) discourse-based decision support system … which you strangle in the cradle;
    b) deliberative politics … to which you pay lip service (You monetize like cocaine whores. Oh-w000ps no … like crack whores.)
    c) participatry deliberation … which I’ve been working on since 1975 … which you strain to ignore

    p.s. you aren’t “nice” people. you use “nice” as a way to validate suck-up game-players.

Leave a comment