Number Needed to Treat is an aggregate measure of clinical benefit that medical study geeks love because it has a comprehensibility lacking in odds ratios and relative benefit percentages.

It represents the number of patients that would need to receive a treatment for one of the patients to avoid an adverse outcome (death, stroke, development of diabetes, high cholesterol). For instance, say we want to want to put people on a Mediterranean Diet after a first heart attack. How many heart patients would we have to put on a Mediterranean Diet to save one life? Here’s the answer from thennt.com: 30!

That’s pretty good, especially with no known harm. And it’s easy to conceptualize.

I was reminded of how few people ever come into contact with the NNT when I read the excellent thread in this NPR post.

If you don’t know the NNT metric, click that link and learn about it. It’s a great way of conceptualizing the benefit of interventions, and it may even help you think about your own work in a different way.

Student debt is a problem, I think. I’m pretty sure about that. But reporting like this isn’t helpful:

But this is a bigger problem than many realize. A recent study by the Federal Reserve Bank of New York found that outstanding student loans have surpassed the nation’s $693-billion credit card balance.

Even more eye-opening, nearly 80% of Americans held credit cards as of 2008, compared with 15% of consumers who now hold student debt. That illustrates just how small of a pool of Americans holds this huge pile of debt.

I don’t know where to start. Here’s a couple quick hits:

• Student Debt is non-revolving debt. It gets borrowed in a big chunk and paid off over years. Credit card debt is revolving debt — expanding and contracting on a monthly basis. If student debt is similar to anything, it is probably similar to other non-revolving debt, like mortgages and car loans.
• Student Debt has been rising, but the big reason it passed credit card debt recently is that credit card spending contracted massively during the recession.
• According to the article, student debt is a problem because of it’s scale. But when it’s revealed that it is only held by 15% of the population, that’s… more proof it’s huge?
• Supposedly the large amount combined with the small amount of holders shows something about how much debt the average loan holder owes. Except we already know how much the median holder owes — it’s around $13,000. Which is too high, sure, but also a directly citable figure that the article could provide.

In terms of the COMPARABLE framework, I think “M” is particularly important here. Do a mental experiment. Imagine instead the following headline, which inverts the comparison:

Credit Card Debt Soon to Exceed Student Loan Debt

Would we see that as a good thing? Or would we look at that and wonder why people don’t spend money on things with tangible long-term benefits, instead of buying junk?

Or consider this headline, which chooses an alternate comparator (“C” in the framework):

Student Loan Debt Exceeds Auto Loan Debt for First Time

Is that shocking? Or does it actually sound pretty sane? Because here’s the thing — auto loan debt is the same as credit card debt right now. And it’s hands down a better comparator — non-revolving debt that focuses on something of direct utility. It wouldn’t have *just* been exceeded by student loans, but the amount is functionally equivalent to credit card loans right now:

Occasionally when I argue against the claim that higher education is on the verge of catastrophic collapse, and warn that MOOCs are being advanced by many as a replacement for higher education instead of a supplement to it I’m told I’m arguing against straw men. Where are these people? Show me the articles!

The truth is the sentiment is so widespread no one bothers to to write an article about it.  It’s the assumed subtext of most of what you read. However, I couldn’t help but notice this comment on that article I just critiqued:

[Student debt] may not be a ‘bomb’ on the overall economy, but it is a precision guided device that will blow up higher education.  The sub-prime problem resulted in price inflation and excessive borrowing… sound familiar?  Sub-prime killed the housing market just as student debt will start a series of disruptive changes to what we  now recognize as higher education.  Higher education is over subscribed and over priced while powerful alternatives are visible on the horizon.  See article on Massive Open  Online Courses “MOOCs” in the NYT or last week’s 60 minute piece on Khan Academies.  The disruptive technologies are in place while student indebtedness is the trip wire.  It used to be that the quality of higher education could be correlated with brick & mortar square footage per student… does that make any sense today?   An education Tsunami is coming and the globe is flatter than it was… hold on!

This isn’t some brilliant education visionary showing up in the comments. This is a guy who saw a show on TV, read an article in the Times, and is parroting back their implicit meaning to whoever will listen. It’s not a novel interpretation, it’s a banal one. And it should bother anyone who believes the truth is more subtle than that.

There’s a great article in The Atlantic on student debt, and it’s well worth a read. But it makes what I believe is a common mistake on student loan debt. Check out this paragraph…

 When teenagers are forced to take out loans in order to pay for their education — the median graduate leaves school $12,800 in debt — it acts as a tax on their future wages. It postpones their ability to settle down, buy a home, and have children. That’s tragic for them, and it’s tragic for us, because it means less money will flow into other, more productive parts of the economy.

That figure comes from the Fed’s Liberty Street Economics:

The average outstanding student loan balance per borrower is $23,300. Again, there is substantial heterogeneity in balances of individual borrowers. The median balance of $12,800 is roughly half the average level…..

Notice, however, the difference.  This is the median balance — this is the median value of loans, not what the median student owes. That’s an important distinction, since only about 66% of students take out loans.

To consider how that might affect things, consider this set of numbers, where the median is 5.5:

{1,2,3,4,5,6,7,8,9,10}

Now what happens if we add our 33% of zero values to this? Well, if this set represents 2/3 of the values, then there are five zeroes to add in to fill it out:

{0,0,0,0,0,1,2,3,4,5,6,7,8,9,10}

Our median now drops from 5.5 to three. Now admittedly, our example is not a normal distribution, but we’d still expect a drop from the 50th percentile to the 33rd or so, which should be substantial.

This seems trivial — I agree that student loan debt is an issue, and the the number of students graduating with debt is a problem. But if we are to address the problem effectively, knowing the precise scale and shape of the debt is important.

Part of the thing I like about this is it shows that it’s really not very hard. Start with this: what are the actual figures? What was actually counted? Do the figures even make sense?

You, too, can take the three minutes it takes before re-pinning or re-sharing the latest piece of idiocy…

From here:

The amount of student loans taken out last year crossed the $100 billion mark for the first time and total loans outstanding will exceed $1 trillion for the first time this year. Americans now owe more on student loans than on credit cards, reports the Federal Reserve Bank of New York, the U.S. Department of Education and private sources.

From here:

For almost three generations, debt has been a nearly inescapable part of becoming a doctor. Over 80 percent of each medical student class will graduate in debt; and while that percentage has remained unchanged for 25 years, the increase in the total amount owed has leapfrogged over all other economic reality checks, like inflation and the consumer price index.According to the Association of American Medical Colleges, which has been trying to address the problem for nearly a decade, young doctors who graduated from medical school last year had an average debt of $158,000, or$2.3 billion for the group as a whole. Almost a third of students owed more than $200,000, a number that will only increase with the addition of interest over payback periods of 25 to 30 years.

There’s a per year/per education problem with comparing any numbers here. But the scale is such that medical student debt probably accounts for a nontrivial amount of the difference between mean and median student debt. Law school debt is may be another factor, though I don’t have any good numbers on that yet.

Update: I actually wrote this before Anya K. confirmed it for me (I generally write a ton over nights and the weekend, but publish into a queue that spaces them out over the week). But thank you Anya for your reply — it’s good to get confirmation from someone that has been over the numbers in detail. What I’d really like to know is excluding this sort of debt, what does the debt picture look like? Sure I can find that somewhere….

Because this seems as good a place as any to note it, I think one person who doesn’t get credit enough on learning badges is Roger Schank. When I went to work for his company in 2000, he’d already been talking about a “merit badge” approach to assessment for years. And for certain projects with Harvard Business School Publishing and others, we essentially implemented those approaches.

Here’s Schank talking in 1999 about it:

“We won’t get rid of certification but perhaps we can contemplate new kinds of certification. Students should be certified as having accomplished something or as being able to do something. Like Boy Scout merit badges or Karate black belts or Truck Driver’s licenses, the proof should be in the pudding. A student should show his stuff, he should be able to do something and the attestation to the doing should be the certification.”

And if that leaves you thinking that “badges” was only one of many ways he expressed this idea, be assured by the time I joined the company in 2000, the merit badge metaphor was *the* metaphor used.

There were probably others at the time saying the same thing. I don’t know. But it seems like someone should give Schank some credit.

Incidentally, I have mixed feelings about badges. I was bullish on the concept even as late as 2007, but since then I have become more ambivalent.  I think I’ve become aware that while much of the badge talk centers around student learning, cognitive science, and motivation, the real reason badges are being pushed relentlessly is a lot of companies want to tap into some of that free taxpayer money for education, but don’t have the means or the patience to buy and run an accredited institution. The end game of all of this is that Facebook finds a way to skim money out of the Pell Program, or some Silicon Valley startup gets to take money from government-backed student loans.  That’s what badges are really about.

I’m not saying that we shouldn’t have new players in higher education, and I’m not saying badges aren’t a useful tool in our instructional design toolbox. It just makes me nervous when we discuss dismantling accreditation barriers through proxy issues like badges.

I found this surprising (emphasis mine):

The average outstanding student loan balance per borrower is $23,300. Again, there is substantial heterogeneity in balances of individual borrowers. The median balance of $12,800 is roughly half the average level, which indicates that a small fraction of people have balances significantly higher than the median. About one-quarter of borrowers owe more than $28,000; about 10 percent of borrowers owe more than $54,000. The proportion of borrowers who owe more than $100,000 is 3.1 percent, and 0.45 percent of borrowers, or 167,000 people, owe more than $200,000. The distribution also varies by age group: for example, borrowers between the ages of thirty and thirty-nine have the highest average outstanding student loan balance, at $28,500, followed by borrowers between the ages of forty and forty-nine, whose average outstanding balance is $26,000 (see chart below).

Actually, there’s a surprise for me in almost every sentence here. First, the mean loan amount we hear tossed around all the time averages in only borrowers — students who do not borrow are not averaged in. To me, that’s deceptive. I can’t find the number right now, but I think between 25% and 33% of undergraduates take out no loans. If it is 25%, you can average in those zeroes by knocking 25% off that figure (I think that works, right?) and you get something like $17,500 — which isn’t as impressive.

Second, that is a huge difference between mean and median loan amount. That tends to indicate a very skewed distribution overall, with a few higher borrowers bringing up the average dramatically. That’s a different (and easier) problem than a high median debt.

Third, I don’t know what to make of the high borrowers (the top 3%). Maybe these are for medical programs, or something else I don’t understand. Could they even be for undergraduate education? $100,000? Maybe I’m missing something? Is this accumulated debt + (a lot of) unpaid interest?

If it is for normal, undergraduate programs, should we be encouraging debt of this sort?

Finally, the average debt of the different age groups. Actually, the heavily indebted forty year olds are not  that surprising. Your prototypical debt would be paid off between 24 and 34 years old.  I imagine a high percentage of cases in those older age groups are defaulters, people that couldn’t pay down the debt over twenty years, and that would drag the average up. The amount in the thirty year old age bracket surprises me though, because I would think that people almost done paying down debt would offset people that are defaulting. So there’s something weird going on there….will look into that….

The new Chronicle College Completion Site may be met with groans from people who think the wrong metrics have been chosen or that necessary detail has been omitted, but I think it’s wonderful.

Sure, I’m not sure that cost per degree should be the central figure in the debate. I think cost per degree probably does not account enough for selectivity issues (take only students you *know* will graduate, and your cost per degree will go down). But even with that caveat, it provides a fascinating example of what many people have been trying to communicate to the press and public for years, to no avail: there really is no single educational productivity crisis, and measures which lump Dartmouth in with Podunk State are dangerously deceptive.

To understand why that is, take a bookstore (bookstores, which have been hit by the End of Retail phenomenon, are often used as an analogy for higher education). Bookstores have a given cost of operation, and that cost gets put into the markup on the book. Borders has failed, Barnes and Noble has not — but essentially the economics of both are pretty similar. We wouldn’t expect Barnes and Noble to have a cost of 32 cents per book sold and Borders to have a cost of three dollars per book sold. We’d expect these prices to track pretty close.

Education does not work like that. Education is more like a hospital system that routes the most difficult cases to hospitals that operate on pennies a day, and routes hangnails to research centers. Yale University spends $500,000 per degree. And they do that while being highly selective (96% complete within six years, median SAT 1472), which should technically reduce the cost. Fort Hays State University spends $29,000 per degree, with only 40% completing in four years (median SAT score: 990).

Then we have places like UMW, which spends $46,000 per completion, with a high completion rate (6 years: 75%), but a mid-range selectivity (SAT: 1170). Or Keene State, which is less selective than UMW (median SAT score: 1000), and pays for that with lower completion (56%) that results in a a higher cost per degree ($64,000).

We could go further into the data, but that’s probably enough to get the point. The obvious point is that the distribution of spending in education makes the U.S. health care system look positively socialist.

The less obvious point is that is that, once you get away from the Dartmouths in the equation we are actually pretty close to a workable system.

Let’s start with some idealism. If Fort Hays could double its completion rate while keeping costs down, the cost for a four-year degree would be $15,000. That’s the full cost, not just the student-side or subsidy side. At that rate we could give 40% of today’s 18 year olds a chance to earn a degree at a cost of less than 40 billion. And that’s with us picking up the total tab. Have students pay $2,000 a year, and the tab is $9,000. Not per year, mind you. For the entire degree.

At $9,000 per degree the cost to the U.S. government is about 22 billion to graduate 40% of the population. Another 20% would be served by two-year technical colleges and trade schools at even lower cost.

To give you an idea of the scale of 22 billion in terms of federal spending, the current Pell program is over 30 billion dollars a year.

People will jump on, I’m sure — hey, I’ve seen Fort Hays, I don’t like it! Or, better, how would you double the graduation rate? Well, if you don’t like Fort Hays, pick somewhere else. If we can’t double graduation rates, then increase them by 50%, or 40% and redo the math.

Or work the other way. Pick a place like Keene State or UMW, keep graduation rates constant, but reduce cost through a hybrid online/face-to-face approach. This is what University of Central Florida has been doing at a huge scale. It can be done elsewhere too.

Change the numbers in any reasonable way, and you’re still in the same ballpark. We are not at the point where we have to pursue zero-marginal cost options as end-to-end solutions.

People talk a lot about disruptive technology, but then assume that the disruption will look like MITx giving certificates out. When I look at these figures, that’s not what disruption looks like to me. Change the reward structure of higher education, and disruption looks like the University of Central Florida, or a souped-up UMW. There are admittedly huge institutional and cultural barriers to this — read The Innovative University for a summary of why we all aspire to be non-productive Dartmouths instead of efficient state universities — but evolution does not require that all institutions evolve. Given the right reward structure we only need a few institutions to show the way. I continue to think that’s possible.