Informal Statistical Inference

What we are trying to do in our Stat Lit class is to develop good intuitions about data, rather than create mini-statisticians. Our belief is that everyone, in almost any job or civic task, has to make inferences from data without having access to complex data crunching tools or methods, and, as such, it is this “skimming data” approach that is most useful for students to acquire.

This paper introduces me to the useful term Informal Statistical Inference. The informal part is obvious enough — the statistical inference piece requires:

  1. A statement of generalization “beyond the data”
  2. Use of data as evidence to support this generalization
  3. Probabilistic language that expresses a level of uncertainty around the generalization

That’s narrower than Statistical Literacy, but still provides a nice skeleton for some meaningful Stat Lit outcomes. Worth passing on, I thought….

Generational Learning Styles Are Bunk, Part 28

Via Downes, this 2008 summary of the research on whether today’s students really do learn differently. The answer is probably best summed up in these two excerpts:

“In contrast to the dubious bromides provided by the “experts” quoted above, a review of educational research reveals that there are virtually no research-based findings or evidence drawn from robust learning theory that supports the differential effectiveness of different instructional designs or strategies across the generations. Nor is there a compelling case for the development of a new instructional design model to accommodate generational differences.”

and towards the end

The major question addressed in this review is whether generational difference is a variable important enough to be considered during the design of instruction or the use of different educational technologies. At this time, the weight of the evidence is negative.”

No one reading this blog is surprised by this, I’m sure. It was a ridiculous idea on it’s face to start with. 

It’s time to kill this zombie myth though. Next time you see someone cite this idea that Millenials “learn differently” please take the lecturer aside afterwards and gently inform him or her that there is not a single high-quality study that supports this — and there are a number of decent studies that refute it. Maybe in time we can undo this ridiculousness…

What Nick Carr (maybe) Misses About Memory and Integration

What Nick Carr (maybe) Misses About Memory and Integration

Nick Carr on the recent Science article on the effect of Google on memory:

If a fact stored externally were the same as a memory of that fact stored in our mind, then the loss of internal memory wouldn’t much matter. But external storage and biological memory are not the same thing. When we form, or “consolidate,” a personal memory, we also form associations between that memory and other memories that are unique to ourselves and also indispensable to the development of deep, conceptual knowledge. The associations, moreover, continue to change with time, as we learn more and experience more. As Emerson understood, the essence of personal memory is not the discrete facts or experiences we store in our mind but “the cohesion” which ties all those facts and experiences together. What is the self but the unique pattern of that cohesion?

I suppose the thing that occurs to me is that there’s two sides to this question of storage. The first is that, yes, to the extent we know something is easily retrievable, we may spend less effort in trying to integrate it. I remember reading an article once about professors who announced that lecture PowerPoints would be available after class (after having lectured for years without sharing the lecture) — the result was not an increase in student test scores, but a dramatic drop. Students stopped trying to actively process presented material and make mental connections while listening. And ultimately, this hurt student development.

I think though there is another side to the issue — to the extent we are accessing external knowledge, someone has to externalize it, right? And the process of externalizing information is also a powerful form of integration.

So in other words if we are Googling more but also microblogging more, things could (perhaps) move to balance out. The person that agonizes over how to capture a speakers’ thought in 140 characters may be going through as rigorous a process of consolidation and integration as the person who is try to remember what the speaker is saying.

I don’t know that, of course, any more than Nick knows for sure the patterns on display in the Science article will lead to the effects he fears — but it’s worth keeping somewhat of a larger focus, looking at the overall impact a set of behaviors has on these processes rather than just one behavior in isolation.

Spaced retrieval: Absolute spacing enhances learning regardless of relative spacing.

Spaced retrieval: Absolute spacing enhances learning regardless of relative spacing.

Sorry, no online version. But the abstract says it all — spaced retrieval matters, but these systems to perfectly control the spacing (through gradually increasing it) may be hooey:

Repeated retrieval enhances long-term retention, and spaced repetition also enhances retention. A question with practical and theoretical significance is whether there are particular schedules of spaced retrieval (e.g., gradually expanding the interval between tests) that produce the best learning. In the present experiment, subjects studied and were tested on items until they could recall each one. They then practiced recalling the items on 3 repeated tests that were distributed according to one of several spacing schedules. Increasing the absolute (total) spacing of repeated tests produced large effects on long-term retention: Repeated retrieval with long intervals between each test produced a 200% improvement in long-term retention relative to repeated retrieval with no spacing between tests. However, there was no evidence that a particular relative spacing schedule (expanding, equal, or contracting) was inherently superior to another. Although expanding schedules afforded a pattern of increasing retrieval difficulty across repeated tests, this did not translate into gains in long-term retention. Repeated spaced retrieval had powerful effects on retention, but the relative schedule of repeated tests had no discernible impact.

Don’t Show, Don’t Tell? Direct Instruction Can Thwart Independent Exploration

Don’t Show, Don’t Tell? Direct Instruction Can Thwart Independent Exploration

New study, to be published in Cognition. Far from being a salvo in the never-ending direct instruction vs. discovery learning war, this a cautionary tale about the subtleties of presentation, especially with small children:

So what’s a teacher or parent to do? Schulz is quick to point out that the study is not an argument against instruction. “Things that you’re extremely unlikely to figure out on your own — how to read, how to do calculus, how to drive a car — it would make no sense to try to learn by exploration,” she says.

Rather, the study underscores the real-world trade-offs between education and exploration, and the importance of acknowledging what is unknown even while imparting what is known. Teachers should, where possible, offer the caveat that there may be more to learn.

“Teachers can say things like, ‘I’m showing you what we think is true, but there are a lot of other possibilities you should consider,’” Schulz says.

In short, she says, “a little humility can go a long way.”

Read through the article and see what a profound effect a slight difference in presentation can have on students. It’s pretty neat.

Cockroach Performance Anxiety

Cockroach Performance Anxiety

Via Ariely, this great experiment on the social facilitation effect from the 60’s: cockroaches do better on simple tasks in the presence of other cockroaches, but worse in the presence of other cockroaches when the task is difficult: 

However, research by Zajonc, Heingartner, and Herman (1969) argued that such conscious, cognitive processes weren’t necessarily an important component, as cockroaches, which presumably do not have the same conscious processes as humans, showed the same social facilitation effect.

In this same experiment, Zajonc and colleagues also found that the cockroach’s performance decreased in the presence of other cockroaches when the task was particularly difficult. In the landmark article, “Social facilitation”, Robert Zajonc offered the theoretical explanation that the presence of others increases physiological arousal. During this state of arousal, he argued that performance increases when task is easy or familiar, while performance decreases when a task is difficult or unknown. As such, social facilitation effects came to be understood based on two components: the presence of others and the ease of the task for the individual.

I’d love to link to the original article, but it is locked up in some Elsevier safe somewhere, despite being a classic article that is at this point over forty years old. 

Large Stakes and Big Mistakes

Large Stakes and Big Mistakes

I’m on vacation, and catching up with some reading. Dan Ariely’s book on the Upside of Irrationality is so far decent, though a little too chatty at times (I like a little less of the human interest backstory, YMMV). 

In any case, one thing I plan to do over the next couple days is find and read some of the studies I hit in it that I find interesting. This one is pretty neat — an experiment with motivation through money and social stress that appears to confirm the existence of  Yerkes-Dodson Law in humans — as potential monetary/social reward increases, performance increases then decreases. In the case displayed above, the small rewards appear to have been set too high, so you don’t get the full curve here, but the deterioration at higher levels of payment is pronounced. 

And yes, I know you’ve probably seen Dan Pink cover a portion of this study — but do you want to spend your life watching TED videos, or do you want to read some research? Right. I thought so. 

[Note: the tables are not presented in-text, but scroll to the end of the paper and you’ll find them there.]