Plenary Workshop at NELIG: What is Critical Thinking, and Why Is It So Hard to Teach?

I call this a plenary workshop, but as I learned after I agreed to do it, it was not only a plenary session, but it was the only session. Apparently NELIG, at least in its quarterly meetings, is structured as one giant workshop. No pressure there, then… ūüėČ

In any case, I think it worked out (The abstract is here). This was a reformulation of some of the material covered in the Critical Skills Workshop over break, but redirected to issues of information literacy. If there’s one big idea in it, it’s that when we think critically we don’t often do the computation-intense sort of processing we tend to conceptualize as critical thinking. The most important pieces of critical thinking (as practiced in daily life) happen before you start to “think” — they come from the conceptual frameworks that formulate our intuitive responses. To address problems in critical thought, you have to understand the conceptual frameworks in use by students, work with the student to actively deconstruct them, and provide more useful frameworks to replace them.

If you can’t do that — if your idea is that the students will just learn to think harder — you’re lost.

The participants were great — actively engaged, great thinkers asking all the right questions. I want library faculty in all my presentations from now on: you really can’t do better. In the activity, they identified the differences between the conceptual frameworks librarians use to parse results lists, and the frameworks used by students — students use “familiarity” and “match” as their guideposts — to them, the act of choosing a resource is like that of choosing a puzzle piece. Librarians look at genre and bias — what sort of document is this (journal article, news story, conference proceeding, blog post) and what markers of bias can we spot (URL, language, title, etc). For librarians, this is an exercise of seeking out construction materials, not finding puzzle pieces.

We talked a little about how to students these processes may appear the same: librarians talk about bias, and students hear “use familiar sources”. Librarians talk about genre, and students hear “fit” or “match” — “How many journal articles do I need to collect? How many news stories?”, which is really just a different way of asking what shape the puzzle piece should be in. Until you address the underlying conceptual misunderstanding directly through well-structured activities, students will continue to plug what you teach them into a conceptual framework that undermines the utility of the new knowledge.

Slides are here. There’s some good stuff in there, but much is incomprehensible without the activities and narration.

To all NELIG participants, thanks for a great Friday morning. It was a pleasure to talk with you all!


Comparing Porn Prosecutions

One of the things I like about the COMPARABLE framework is how nicely it can be used not only to evaluate existing comparisons, but to think through what a fair comparsion would look like where none is provided. For instance, today I saw this:

‚ÄúWell you have to look at the proof that‚Äôs in the prosecution. Under the Bush administration, pornographers were prosecuted much more rigorously under existing law than they are under the Obama administration,‚ÄĚ Santorum said. ‚ÄúMy conclusion is they have not put a priority on prosecuting these cases, and in doing so, they are exposing children to a tremendous amount of harm. And that to me says they‚Äôre putting the un-enforcement of this law and putting children at risk as a result of that.‚ÄĚ

The first thing is the habit of mind — when a student sees the word “more”, hopefully that triggers comparison mode. Honestly, getting that bell to ring is the hardest bit of this. Once we are there, we use our framework:

C: Comparison groups are the prosecutions under Bush vs under Obama. Fair enough.

O: Santorum talks about rigor, and priority — but the key claim here seems to be “un-enforcement” so the best variable seems like it might be number of prosecutions. But we’d also probably have to find some way to take into account severity of crime. A small prosecution on a¬†fine-able¬†offense should not equal a large prosecution with jail terms, etc.

M: Mental experiment on this is hard, since there are no numbers to run through. So we’ll skip it.

P: Again, without numbers, there’s nothing to do here.

A: We probably want to look at this not only from raw numbers, but as a percent of the DOJ’s total effort. There might also be other factors. Since there are limited resources of the DOJ, any large operation in a non-porn area that requires the same people might make resources scarce. If we do raw numbers, we probably also want to make sure we are taking into acount Obama has only been in office three years — a good comparison might be the last three years of the Bush administration to the first three years ¬†of Obama. Finally, we might look at action controlled for the size of the porn industry — a bigger industry might require more regulation.

R: Randomness is not such an issue here as we’re not sampling. But there might be some year to year randomness in the number of prosecutable offenses.

A: Alternative measures might include looking at this as a trend. For example, were prosecutions declining year over year in the Bush administration, a decline under Obama might be a continuation of a historical trend. If they were increasing under Bush, even a stabilization under Obama would look like a redirection of resources.

B: Base rates — I’m not sure what relevant base rates are here. Again, we don’t have numbers. But obviously understanding whether a percentage increase or decrease is meaningful will require absolute numbers and an idea of how prosecutions relate to offenses. It might also be useful to see if the Bush administration is the historical anomaly here.

L: This is a longitudinal comparison. It might be interesting to go the other direction too, and look at how a country like 2011 Canada prosecutes this stuff.

E: Not sure how distribution effects this, although subpopulations is obvious — we’d want to look at how different types of crime account for the whole of prosecutions in each administration — the likelihood is that that breakdown would tell us far more than the headline statistic of prosecutions.

One thing to notice about COMPARABLE is it avoids going directly down the association rabbit-hole — in this case the “kids exposed to harm” piece. While that’s certainly important, I find that those questions end up being too nuanced for many of our students. The comparison question here is complex, certainly, but there is a certain concreteness to it that is helpful to the beginning student of QR.

Number Needed to Treat!

Number Needed to Treat is an aggregate measure of clinical benefit that medical study geeks love because it has a comprehensibility lacking in odds ratios and relative benefit percentages.

It represents the number of patients that would need to receive a treatment for one of the patients to avoid an adverse outcome (death, stroke, development of diabetes, high cholesterol). For instance, say we want to want to put people on a Mediterranean Diet after a first heart attack. How many heart patients would we have to put on a Mediterranean Diet to save one life? Here’s the answer from 30!

That’s pretty good, especially with no known harm. And it’s easy to conceptualize.

I was reminded of how few people ever come into contact with the NNT when I read the excellent thread in this NPR post.

If you don’t know the NNT metric, click that link and learn about it. It’s a great way of conceptualizing the benefit of interventions, and it may even help you think about your own work in a different way.

More Confused Credit Card vs. Student Debt Reporting

Student debt is a problem, I think. I’m pretty sure about that. But reporting like this isn’t helpful:

But this is a bigger problem than many realize. A recent study by the¬†Federal Reserve Bank¬†of New York found that outstanding student loans have surpassed the nation’s $693-billion credit card balance.

Even more eye-opening, nearly 80% of Americans held credit cards as of 2008, compared with 15% of consumers who now hold student debt. That illustrates just how small of a pool of Americans holds this huge pile of debt.

I don’t know where to start. Here’s a couple quick hits:

  • Student Debt is non-revolving debt. It gets borrowed in a big chunk and paid off over years. Credit card debt is revolving debt — expanding and contracting on a monthly basis. If student debt is similar to anything, it is probably similar to other non-revolving debt, like mortgages and car loans.
  • Student Debt has been rising, but the big reason it passed credit card debt recently is that credit card spending contracted massively during the recession.
  • According to the article, student debt is a problem because of it’s scale. But when it’s revealed that it is only held by 15% of the population, that’s… more proof it’s huge?
  • Supposedly the large amount combined with the small amount of holders shows something about how much debt the average loan holder owes. Except we already know how much the median holder owes — it’s around $13,000. Which is too high, sure, but also a directly citable figure that the article could provide.

In terms of the COMPARABLE framework, I think “M” is particularly important here. Do a mental experiment. Imagine instead the following headline, which inverts the comparison:

Credit Card Debt Soon to Exceed Student Loan Debt

Would we see that as a good thing? Or would we look at that and wonder why people don’t spend money on things with tangible long-term benefits, instead of buying junk?

Or consider this headline, which chooses an alternate comparator (“C” in the framework):

Student Loan Debt Exceeds Auto Loan Debt for First Time

Is that shocking? Or does it actually sound pretty sane? Because here’s the thing — auto loan debt is the same as credit card debt right now. And it’s hands down a better comparator — non-revolving debt that focuses on something of direct utility. It wouldn’t have *just* been exceeded by student loans, but the amount is functionally equivalent to credit card loans right now:

Straw Men

Occasionally when I argue against the claim that higher education is on the verge of catastrophic collapse, and warn that MOOCs are being advanced by many as a replacement for higher education instead of a supplement to it I’m told I’m arguing against straw men. Where are these people? Show me the articles!

The truth is the sentiment is so widespread no one bothers to to write an article about it. ¬†It’s the assumed subtext of most of what you read. However, I couldn’t help but notice this comment on that article I just critiqued:

[Student debt] may not be a ‘bomb’ on the overall economy, but it is a precision guided device that will blow up higher education. ¬†The sub-prime problem resulted in price inflation and excessive borrowing… sound familiar? ¬†Sub-prime killed the housing market just as student debt will start a series of disruptive changes to what we ¬†now recognize as higher education. ¬†Higher education is over subscribed and over priced while powerful alternatives are visible on the horizon. ¬†See article on Massive Open ¬†Online Courses “MOOCs” in the NYT or last week’s 60 minute piece on Khan Academies. ¬†The disruptive technologies are in place while student indebtedness is the trip wire. ¬†It used to be that the quality of higher education could be correlated with brick & mortar square footage per student… does that make any sense today? ¬† An education Tsunami is coming and the globe is flatter than it was… hold on!

This isn’t some brilliant education visionary showing up in the comments. This is a guy who saw a show on TV, read an article in the Times, and is parroting back their implicit meaning to whoever will listen. It’s not a novel interpretation, it’s a banal one. And it should bother anyone who believes the truth is more subtle than that.