This is from the University of Edinburgh. It’s being heavily circulated right now in edtech. I agree with most of it.

I find point two particularly a) true, and b) problematic. And if the truth of the manifesto is to be useful truth, that’s the piece that has to be solved institutionally first.

So we have this COMPARABLE framework I’ve been working on, where COMPARABLE is an acronym for the sorts of things you want to look at when presented with a comparison.

The “M” in the acronym stands for “Mental Experiment”, and it’s a reminder that a lot of sanity checking claims is about taking some guesstimate numbers and running them through the claim to see if they make sense.

A good example is this recent quote from candidate Rick Santorum:

He claimed that “62 percent of kids who go into college with a faith commitment leave without it,” but declined to cite a source for the figure. And he floated the idea of requiring that universities that receive public funds have “intellectual diversity” on campus.

There’s a lot of definition problems here — but given that there is no source, we we have to fill this stuff in. I am going to read it the way I think he meant it to be taken:

“college”: Any college, including community college
“leave”: Any way of leaving, including not finishing
“w/o it”: I’m assuming a loss of faith (not a change in denomination)

The premise of a mental experiment, in this case at least, is to ask what the world would look like if this was true. Let’s be charitable, and say that every single kid that goes to college is a person of faith. In that case, this model would predict that 62% of people who have attended college are atheists or agnostics.

So what rate of atheism would that predict in America? Here’s some data from the 2000 census from 25 year-olds and over:

• 21% of Americans had taken some college courses but had not earned a degree in 2000, compared with 18.7% 10 years earlier.
• 15.5% had earned a bachelor’s degree but no higher, compared with 13.1% in 1990.
• 8.9% earned graduate or professional degrees, compared with 7.2% earlier.

Rounding up, 21+16+9 = 46% of Americans who have “left college” in one way or another. Again, assuming that absolutely no kids went to college unreligious, 46 * 0.62 = 28.5. So what we should see in populations that have graduated college in the last 20 years is an atheism/agnosticism rate of about 28.5%.

Looking here, we find that about 22% of the 18-29 set is non-religious (if you include Deism, which is kind of suspect — isn’t a belief in God a faith commitment? But again, will give a charitable reading here).

So 22% does not equal 28.5%. And it’s not really that close. We’re doing some apples to oranges here (gen population to 18-29 year olds, etc.), but nothing that could possibly account for that gap, considering how generous we’ve been in other areas of the model. So we should be highly skeptical of this claim.

There’s one final nail in the coffin of this claim — remember that in order to get that 22%/30% comparison we had to attribute every nonreligious conversion to a college education (30% was just the level of non-religion predicted by college education, if nonreligion existed without college education, the level would be much higher).

So is that what we find?

No:

“Nones” in this context are people who identify under the “religion” question on the American Religious Identification Survey as “No Religion”. As you can see, there is a small association between a college education and nonreligion, but it’s very small. College graduates are 20% of the nones, and 17% of the population. That’s not much of a difference. And people who have “some college” college are actually more likely to be religious than the general population (24% of nones, 26% of population).

Actually, if you needed to answer the Santorum question, you’d really have to just show that graph above (and if I was writing a political post on this, rather than a Stat Lit post, that’s what I’d do). But I wanted to show the process of thought that leads one to ask the questions that leads to finding the graph. It’s a messy imprecise process that has a ton of stuff in it that is pretty questionable.

But that’s what mental experiments are supposed to be  — they don’t provide definitive answers, but they give you a handle on the questions raised — and once you know the right questions, you’re most of the way there. A mental experiment is your first fumbling attempt to get a grab on something solid.

For the stats text, I’ve been trying to think of/find rules that apply across a wide array of disciplines. Here’s one: control for cyclical effects. It applies here (summer gas spikes):

And here, with voting in elections — presidential elections (a cyclical event) boosts turnout every second congressional election):

 2008:  a bit over 340,000 come out to vote:

 And here is 2010:

The left-most column, incidentally, is Republicans, followed by Democrats in the second column. So did Democratic support collapse, dropping by nearly 50%? Of course not. It’s very difficult to make any comparison of a mid-term to a presidential cycle election, because in mid-terms different sorts of people tend to turn out. In general, midterms tend to pull out more anti-incumbency voters than pro-incumbency, and in this case the incumbency in 2010 was Democratic.

If you want to use the mid-term results to predict the general election you are going to have to come up with some model to account for the difference. Maybe you could look at subpopulations or some other measure. But the point is, like the gas prices, you have to control for cyclical effects.

I’d love to hear about additional examples of cyclical effects from your academic disciplines or the jobs you work. Shoot me an email, or post to twitter, facebook, google…

Gas prices are up again. What’s going on? Who’s to blame?

Of course big long term driver is China. China needs more gas, and its exploding demand has put pressure on the market. But what about the recent spike?

Apparently the big villian is… Chemistry:

Gas prices fluctuate seasonally — up in spring, down in fall — for a very specific reason: Butane. As Rapier wrote last year “Butane is a cheap ingredient in gasoline that boils at low temperatures. In winter, this isn’t a problem. But in summer, butane evaporates from gas, polluting the air while leaving us with less fuel in the tank than we paid for. As temperatures rise, refineries replace butane with more costly ingredients and draw down winter inventories just as beach season begins. Chemistry, not corporate conspiracy, limits supply.”

This happens every single year. You’d think people would have caught on by now. But sadly, no.

Prepare yourself for the shock when prices go down right before the elections. It’s a conspiracy!

Decent graph from NYT showing quintiles over time, in this case, the declining portion of government benefits the poorest 20% of the population receives.

These sorts of graphs are very in right now, as the format is perfect for showing change of distribution over time, and so much of our political discourse is dealing with questions of distribution. I used to see these relatively rarely, but in the last couple of years  they are all over the place. Which is good! It’s an extremely compact way to show a trend and control for it.

One note on teaching graphs — I think there is a tendency with students to fall back on the idea that “Percentages should be done in pie charts”. As this shows, nothing could be further from the truth. Pie charts are a lousy form which people expect now, but there are so many more elegant was to deal with part whole relationships, especially ones we track over time.

Making some progress on the Making Fair Comparisons textbook. The preface is below.

One thing I’ve learned from reading cheesy self-help books: If you believe a skill will change a person’s life, you should say it. At the end of the book, the reader will know if their life is changed or not. There’s time to be cynical later. At the beginning of the book, let your passion show.

So anyway, here’s the cheesy intro to the text. I love it.

Why we compare

Which intersection in town is the most dangerous?
How much more expensive will college be if I graduate a year late?
Which product line has given our business the best overall return in the past two years?
How much more campaign money was spent in the election of 2008 compared to previous elections?

Comparisons don’t happen in a vacuum. Usually when someone is comparing things, they are comparing them for a reason. In the case of the intersection question above, maybe there is an action pending – if we are going to upgrade one intersection, which one should it be? Businesses may want to know what products have been the most profitable so they can pursue profitable avenues at the expense of the less profitable ones. A political scientist may be investigating the influence of money on elections, and trying to determine if that influence has increased over time.

Ultimately, comparisons have real world consequences. If you rightly determine which intersection is the most dangerous as an urban planner, perhaps you can save a life. Knowing which product lines have given a company a good return could be the key to keeping a business afloat, saving your job and the jobs of others.  Determining whether money in elections is out of control or in line with historical trends can help us plot a course of action for our country that fixes what is wrong with our system while preserving what is right.

Depending on what profession you go into, you may use algebra or you may not. Some of you may use calculus or trigonometry. Some of you will be asked to use advanced statistical methods. Most won’t.

But every single one of you will be asked to compare things as an employee, consumer, and citizen. And whether you are able to compare things adequately will have dramatic effects on the success of your business, your family, and your community.

This is a book about how to use very simple statistical techniques to compare things. It is not so much about formulas as it is about critically thinking about numbers. We honestly believe this skill will be one of the most important skills you acquire in your college career.  Mastering it will change your life for the better, and get you closer to being the sort of person you want to be.

I’m writing a intro stats book right now (a small one for students). This lecture really brings home the problem of the “garden path” solution, and how small changes in presentation could make a big difference.

Everyone knows that campaign spending is out of control, right?

Except it’s not. In raw numbers, of course, the amount just keeps getting bigger, but controlled for inflation, it’s exactly what you would expect, and no more expensive than it was at the turn of the century, as the graph above from Mother Jones shows.

And if you control for eligible voters (remember that women did not get the right to vote until 1920, and 18-21 year olds did not get to vote until the 1970s), Jonathan Bernstein points out that we are spending far less on elections per voter than we did early in the century.

So what’s going on? Dana Houle notes at Rooted Cosmopolitan that the election reforms of 1973 capped contributions at $1,000. Because the cap was not indexed to inflation the cap very quickly moved from being relatively generous to exceedingly tight. You can see the result – as inflation skyrocketed, the real worth of $1,000 plummeted. In 2004, the cap was reset to $2,000 (which frankly only begins to adjust for post ’73 inflation), then in 2008 it was raised to $2,300, and what you see is the amount finally catching up to historical norms after its post-Watergate reform lows.

In the course we’re putting together on Making Fair Comparisons, one of the rules we give students is to control for inflation and population wherever you can, if only to see what happens. It’s hard to figure out how to control for something more complex, but you can get per capita numbers and inflation adjusted dollars pretty easily from WolframAlpha – it takes under 10 seconds to control a couple numbers and see what happens. Yet, no one does. We’d rather throw around “Most expensive campaign ever” nonsense, because it fits our intuitions or our politics.

I’ll make one more comment – it’s often mentioned by Lessig and others how much time politicians have had to spend raising money – a disciplined candidate will spend ten or more hours a week doing direct fundraising calls and as many hours doing fundraising events. Lessig claims that congressmen will spend 30% to 70% of their time raising money, implying this is a new phenomenon – looking at these charts I wonder if the increased fundraising time is a result of the non-indexed cap. In 1973, if you wanted to raise $1 million dollars for a general election race, you’d have to get maximum donations from 200 of those names on your call list. By 2004, you’d have to get more than four times that many maximum donations. No wonder congress members that served through the seventies forward have said that the amount of calls they had to make for money has increased – inflation was pushing them to it.

Incidentally, I favor publicly funded elections, for a variety of reasons. But we should be suspicious of claims that the current spending is exponentially larger than it was in the past. And as we tell our students, we should be ruthless with commentators and pundits who don’t at least attempt to control for relevant variables.

In a Moneybox post I mostly agree with, Matt Yglesias says this:

In order to do well in courses on 19th Century British Literature or Social Anthropology or Philosophy or American History in a properly running American college, what you need to do is get pretty good at reading and writing documents in the English language…. [And] If you can compose an email that’s 10 percent clearer in 90 percent of the time as the other guy, you’re going to get ahead in a wide range of fields. Outside of office work, a big part of the difference between a hard-working individual who’s pretty good at his job and a person who’s able to leverage his skills and hardwork into an entrepreneurial or managerial role is precisely the ability to research things and write up plans. Everyone knows that a kid growing up in rural India is obtaining valuable skills if he gets better at English, but this is equally true for a kid growing up in Indiana.

To which I would say, yes – but there is not as much transfer in that area as you might think. In many ways, academic writing trains one in habits that have to be unlearned in a business environment (as a former language and linguistics grad student I can attest to this). The quality of business writing among faculty, who have practiced writing more intensely than most, is not any better than that in the general population.

Liberal Arts *can* be very useful to business, but it has to be taught for transfer, in an integrative way.  It requires getting beyond the term paper conception of writing and into something less formal but more regular. And maybe more varied — again, transfer (for many students) requires practice across multiple domains with explicit explanation of how the domains relate. The kid at the top of your class is going to move from Milton to competitive analysis reports just fine, but the kids in the middle need guidance.