Semantic Mapping vs. Pictorial Cues

From A Theory-Based Meta-Analysis of Research on Instruction by RJ Marzano:

The next two techniques displayed in Table 7.2 employed the information processing function of idea representation.  Techniques that provided students with metacognitive strategies for using visual memory had an effect size of 1.04, indicating a percentile gain of 35 points.  Presumably, these strategies help students represent information they are reading in nonlinguistic form…  From these findings, one might infer that idea representation is a key aspect of the reading process…

[R]eading studies [that] addressed techniques that attempt to enhance the idea representation information processing function during reading using pictorial aids…were considered as a group in themselves (as opposed to grouping them with the idea representation techniques in Table 7.2) because they did not employ the metacognitive system.  Rather, they were considered manipulations of the environment designed to stimulate idea presentation in students.  These studies had an average effect size of .46 (n=16; SD=.20), indicating a percentile gain of 28 points.  Table 7.4 displays the differential effects of techniques within this category

I may be misreading this, but what it seems to say is that interventions that use pictures as cues presented to students while reading underperform interventions that teach students to make up their own pictorial representations by a lot (0.46 is a fairly average effect size seen in numerous interventions, whereas 1.0 and above is a fairly rare effect size).

The key seems to be that information organized for the students into pictorial form allows them to disengage from metacognitive strategies, whereas organizing their ideas into pictorial form themselves engages metacognitive processing. These findings are in line with Hattie’s findings on audio-visual aids and the like. 

But I’ve only just skimmed this at this point, and the focus is not higher education here, so one must tread carefully. I’ll come back to this tomorrow I guess. 

This all relates to a crazy idea for the Quantitative Literacy course next semester…

Stanovich on Conflict and Critical Thinking

Well, actually the Hitchcock review of Stanovich:

What types of people succeed in overriding interactional intelligence in conflict situations? As one might expect, subjects with greater cognitive ability (as measured by SAT Total scores) were more likely to do so. But so were those with the dispositions characteristic of an ideal critical thinker: even after controlling for differences in cognitive ability, reasoning performance correlated with degree of open-mindedness and epistemic flexibility (cultivating reflectiveness rather than impulsivity, seeking and processing information that disconfirms one’s belief, being willing to change one’s beliefs in the face of contradictory evidence). Further, these dispositions tended to cut across different domains.

For those unfamiliar with Stanovich, his model of the mind rewrites the typical intuition/logic model with a intuitive mind/algorithmic mind/reflective mind model. The main implication is that intelligence is not enough — in practice, many people who are highly intelligent have dispositions that shirk from the hard work of interrogating intuitions, and use rationality only to confirm gut instinct. For Stanovich, intelligence and rationality are related but separable terms. 

I like the model, though I’m still slogging through his work, and probably don’t grasp the details fully (the above is surely a simplification, and possibly wrong). What it gets at, by empirically demonstrating the gap between cognitive power and the ability and drive to interrogate intuitions, is a version of “critical thinking” that might actually mean something useful…

Openness as a Privilege Multiplier and the MIT Certificates

This is pretty huge news:

Millions of learners have enjoyed the free lecture videos and other course materials published online through the Massachusetts Institute of Technology’s OpenCourseWare project. Now MIT plans to release a fresh batch of open online courses—and, for the first time, to offer certificates to outside students who complete them.

The credentials are part of a new, interactive e-learning venture, tentatively called MITx, that is expected to host “a virtual community of millions of learners around the world,” the institute will announce on Monday.

Here’s how it will work: MITx will give anyone free access to an online-course platform. Users will include students on the MIT campus, but also external learners like high-school seniors and engineering majors at other colleges. They’ll watch videos, answer questions, practice exercises, visit online labs, and take quizzes and tests. They’ll also connect with others working on the material.

We’re going to see these projects roll out pretty fast now, and it’s going to be combined, I think, with a growing backlash against the money spent on traditional education. Why are we spending so much money on Random State College, the argument will go, when anyone can get a credential from MIT for free (or nearly free)? 

And it’s a good question, really. With multiple measures (CLA, PP, CAAP) showing almost no gain in critical thinking skills or quantitative reasoning among undergraduates during college years what’s the point of supporting these institutions? 

On the face-to-face side, it’s really time to start using our face-to-face resources to greater effect. The bar will be set by free or nearly-free online options. If we can’t outperform them, we won’t survive, at least not in our current form.

On the online side, it’s worth confronting the openness as a privilege multiplier question now, rather than later. It’s comforting to think of these experiments as an addition to the current range of options available to students, and therefore existing outside the normal ethical space of college. The approach to OER so far has been well, it works for some people, and not for others — we don’t worry about the failures, we just headcount successes. It’s been education as bonus points, or worse, education as a Google product — hey, we gave it to you for free, if it doesn’t work for you, lump  it.

That was fine during the broad experimental period of Open Learning. As Open Learning becomes posited more and more as a broadly applicable solution, however, such nonchalance becomes more dangerous. Any system of education can “succeed” if student failure is seen purely a reflection on the student and not the learning design. But if Open Learning is pitched as a solution to the current economic crisis of higher education, we need to do whole a lot better than that. 

Base Rate, Revisited

Reading Dan Kahneman’s Thinking Fast and Slow, and I can tell very early in it’s going to be excellent.

The following Kahneman insight is an old saw of research on statistical intuition by now, but was revolutionary when he and Tversky came up with it in the early 70s. I thought I’d share it for those not familiar with it:

As you consider the next question, please assume that Steve was selected at random from a representative sample:

An individual has been described by a neighbor as follows: “Steve is very shy and withdrawn, invariably helpful but with little interest in people or in the world of reality. A meek and tidy soul, he has a need for order and structure, and a passion for detail.”

Is Steve more likely to be a librarian or a farmer?

The resemblance of Steve’s personality to that of a stereotypical librarian strikes everyone immediately, but equally relevant statistical considerations are almost always ignored. Did it occur to you that there are more than 20 male farmers for each male librarian in the United States? Because there are so many more farmers, it is almost certain that more “meek and tidy” souls will be found on tractors than at library information desks. However, we found that participants in our experiments ignored the relevant statistical facts and relied exclusively on resemblance. We proposed that they used resemblance as a simplifying heuristic (roughly, a rule of thumb) to make a difficult judgment. The reliance on the heuristic caused predictable biases (systematic errors) in their predictions.

This seems a bit of a game when guessing occupations, but of course replace “Steve” with “an unknown medical condition” that resembles X, and the stakes become much more serious. Classic heart disease symptoms at 30 are far less likely to be heart disease than fuzzier, more ambiguous symptoms at age 70. One hopes one one’s doctor knows that — if they do, it’s through training and education, not untutored intuition.


You can read their 1974 paper, “Judgment under Uncertainty: Heuristics and Biases”, where they first introduced these concepts, here.

The book, just out this year and covering Kahneman’s work and the more recent work of others in the field, is here.

Moonwalking with Einstein

Just finished Joshua Foer’s book Moonwalking with Einstein, one of the most amusing books I’ve read in a while. I’d highly recommend it to anyone, just based on the style of his writing alone, which strikes me as Jonah Lehrer as written by Sarah Vowell (of The Wordy Shipmates period, not Assassination Vacation). But that probably doesn’t really capture it either. 

You know what — you just have to read it. 

In any case, a lot of it is about memory tricks and the like, but the thread that runs through it to the end is the power of deliberate practice (he becomes a test subject during his training for Ericsson himself). We tend to not get better at things after achieving base-level competency because we know enough to get by. Unstructured experience, as Ericsson has shown again and again, can be a really poor teacher. Conversely, when we make a commitment to approach our performance more critically and strategically, even small amounts of effort  can pay off big. Working smart at learning something will tend to beat working long.

All good lessons, I think. 

Higher Education is Already a Voucher System

Saw this about the K-12 online space today in NYT:

Some teachers at K12 schools said they felt pressured to pass students who did little work. Teachers have also questioned why some students who did no class work were allowed to remain on school rosters, potentially allowing the company to continue receiving public money for them. State auditors found that the K12-run Colorado Virtual Academy counted about 120 students for state reimbursement whose enrollment could not be verified or who did not meet Colorado residency requirements. Some had never logged in.

“What we’re talking about here is the financialization of public education,” said Alex Molnar, a research professor at the University of Colorado Boulder School of Education who is affiliated with the education policy center. “These folks are fundamentally trying to do to public education what the banks did with home mortgages.”

Sound familiar? These practices are already widespread in higher education’s for-profit space, and to a lesser extent in the non-profit space. 

We’ve been a voucher system for decades, and the corrosion is extensive….



As Rowin points out that badges ‘draw upon widespread use of badges and achievements in gaming‘ and as somebody who has many badges and achievements in various game systems I can’t help but wonder if some of the problems that have cropped up in games might cross over into the Open Badge Initiative.

David goes on to outline some historical problems with badges. I think badges may be useful in a lot of circumstances, but would like to see more posts like this, talking about the problems with them.

(via @josiefraser)