Hapgood

Mike Caulfield's latest web incarnation. Networked Learning, Open Education, and Online Digital Literacy


A Simple, Less Mathematical Way to Understand the Course Signals Issue

I’m re-reading my Course Signals post, and realizing how difficult it is to follow the numbers. So here’s an example that might make it clearer.

From this desk here, without a stitch of research, I can show that people who have had more car accidents live, on average, longer then people that have had very few car accidents.

Why? Because each year you live you have a chance of racking up another car accident. In general, the older you live, the more car accidents you are likely to have had.

If you want to know whether people who have more accidents are more likely to live longer because of the car accidents, you have to do something like take 40 year-olds and compare the number of 40 year-olds that make it to 41 in your high and low accident groups (simple check), or use any one of a number of more sophisticated methods to filter out the age-car accident relation.

The Purdue example is somewhat more contained, because the event of taking a Course Signals class or set of classes happens once per semester.  But what I am asking is whether

  1. the number of classes a student took is controlled, for, and more importantly,
  2. whether first to second year retention is calculated as
    1.  the number of students that started year two / the number of students who started year one (our car accident problem), or
    2.  the number of students who started year two / the number of students who finished year one (our better measure in this case).

I think it is the “car accident problem” statistic that is being quoted in the press. If it is, then it’s possible the causality is reversed: students are taking more Course Signals courses because they persist, rather than persisting because they are taking more Signals courses.



6 responses to “A Simple, Less Mathematical Way to Understand the Course Signals Issue”

  1. […] The Purdue researchers didn’t appear to have controlled for the number of courses taken. He explained it with an analogy: the older you live, the more car accidents you are likely to have had, and so people who have […]

  2. […] Put another way, “students are taking more … Signals courses because they persist, rather than persisting because they are taking more Signals courses,” Caulfield wrote. […]

  3. […] of this technology we’ve got a problem. Why? Well, because a lot has happened since 2010. Mike Caulfield pointed out six months ago, and Michael Feldstein re-iterated, the research claims of the effectiveness of […]

  4. […] was pushback about these claims, with Mike Caulfield suggesting that perhaps “students are taking more Course Signals courses because they persist, rather than […]

  5. […] was pushback about these claims, with Mike Caulfield suggesting that perhaps “students are taking more Course Signals courses because they persist, rather than […]

Leave a comment