One of the things I like about the COMPARABLE framework is how nicely it can be used not only to evaluate existing comparisons, but to think through what a fair comparsion would look like where none is provided. For instance, today I saw this:

“Well you have to look at the proof that’s in the prosecution. Under the Bush administration, pornographers were prosecuted much more rigorously under existing law than they are under the Obama administration,” Santorum said. “My conclusion is they have not put a priority on prosecuting these cases, and in doing so, they are exposing children to a tremendous amount of harm. And that to me says they’re putting the un-enforcement of this law and putting children at risk as a result of that.”

The first thing is the habit of mind — when a student sees the word “more”, hopefully that triggers comparison mode. Honestly, getting that bell to ring is the hardest bit of this. Once we are there, we use our framework:

C: Comparison groups are the prosecutions under Bush vs under Obama. Fair enough.

O: Santorum talks about rigor, and priority — but the key claim here seems to be “un-enforcement” so the best variable seems like it might be number of prosecutions. But we’d also probably have to find some way to take into account severity of crime. A small prosecution on a fine-able offense should not equal a large prosecution with jail terms, etc.

M: Mental experiment on this is hard, since there are no numbers to run through. So we’ll skip it.

P: Again, without numbers, there’s nothing to do here.

A: We probably want to look at this not only from raw numbers, but as a percent of the DOJ’s total effort. There might also be other factors. Since there are limited resources of the DOJ, any large operation in a non-porn area that requires the same people might make resources scarce. If we do raw numbers, we probably also want to make sure we are taking into acount Obama has only been in office three years — a good comparison might be the last three years of the Bush administration to the first three years  of Obama. Finally, we might look at action controlled for the size of the porn industry — a bigger industry might require more regulation.

R: Randomness is not such an issue here as we’re not sampling. But there might be some year to year randomness in the number of prosecutable offenses.

A: Alternative measures might include looking at this as a trend. For example, were prosecutions declining year over year in the Bush administration, a decline under Obama might be a continuation of a historical trend. If they were increasing under Bush, even a stabilization under Obama would look like a redirection of resources.

B: Base rates — I’m not sure what relevant base rates are here. Again, we don’t have numbers. But obviously understanding whether a percentage increase or decrease is meaningful will require absolute numbers and an idea of how prosecutions relate to offenses. It might also be useful to see if the Bush administration is the historical anomaly here.

L: This is a longitudinal comparison. It might be interesting to go the other direction too, and look at how a country like 2011 Canada prosecutes this stuff.

E: Not sure how distribution effects this, although subpopulations is obvious — we’d want to look at how different types of crime account for the whole of prosecutions in each administration — the likelihood is that that breakdown would tell us far more than the headline statistic of prosecutions.

One thing to notice about COMPARABLE is it avoids going directly down the association rabbit-hole — in this case the “kids exposed to harm” piece. While that’s certainly important, I find that those questions end up being too nuanced for many of our students. The comparison question here is complex, certainly, but there is a certain concreteness to it that is helpful to the beginning student of QR.