On Sex After Prostate Surgery, Confusing Data [Problems with Term Definition]
A classic problem of term definition from the NYT (somewhat older article):
A notable study in 2005 showed that a year after surgery, 97 percent of patients were able to achieve an erection adequate for intercourse. But last month, researchers from George Washington University and New York University reviewed interim data from their own study showing that fewer than half of the men who had surgery felt their sex lives had returned to normal within a year.
So which of the studies is right? Surprisingly, they both are.
Basically, the first number hinges on whether the patient occasionally acheives an erection “adequate for intercourse”. The article goes on to say that this definition is pretty inadequate from the patient’s viewpoint:
“That definition is misleading,” said Dr. Jason D. Engel, director of the urologic robotic surgery program at the George Washington University Hospital. “It doesn’t mean it was good intercourse, and it doesn’t even mean your penis was hard. That man is going to say, ‘I’m impotent.’ But in the surgeon’s eyes, that man had an erection adequate for intercourse.”
The better question for men is whether they can have sex when they want to, with or without drugs like Viagra. In a recent series of patients, Dr. Engel found that after a year 47 percent of men who had robotic prostatectomy were able to have regular sex.
Although he could cite statistics to give men a more hopeful view, he said that did not help the patient.
What we stress in the statistical literacy course is that such faulty definitions are not wrong — just ill-suited to the questions they are trying to answer. A man who asked this question before surgery is likely asking a question for which definition #1 is clearly unsuited.
If your answer was that war is the wrong metaphor, you win the prize, I suppose. Still, I found this exercise from a medical stats textbook rather interesting:
17.1. A major controversy has occurred about apparent contradictions in biostatistical data as researchers try to convince Congress to allocate more funds for intramural and extramural investigations supported by the NIH. Citing improved survival rates for conditions such as cervical cancer, breast cancer, and leukemia, clinicians claim we are “winning the war” against cancer. Citing increased incidence rates for these (and other cancers), with minimal change in mortality rates, public-health experts claim that the “war” has made little progress, and we should focus on prevention rather than cure.
17.1.1. What explanation would you offer to suggest that the rising incidence of cancer is a statistical consequence of “winning” rather than “losing” the battle?
17.1.2. What explanation would you offer to reconcile the contradictory trends for survival and mortality rates, and to suggest that both sets of results are correct?
The pessimistic explanation is that early detection detects more benign cancers, leading to inflated five-year survival stats. But the question is looking for the optimistic answer, which seems harder to grok.
Off the top of my head, one thing that comes to mind is that people can get multiple types of cancer. If you survive thyroid cancer at 60, you might die of colon cancer at 70. Since we all die of something, greater survival rates would also lead to greater incidence, but wouldn’t necessarily reduce mortality. In a world of miracle cancer cures, for instance, you might start racking up cancer later in life the way people rack up colds in their youth.
Of course, I don’t have a teacher’s edition here, so any input would be welcome — alternate theories?
How Visa Predicts Divorce
Hunch then looks for statistical correlations between the information that all of its users provide, revealing fascinating links between people’s seemingly unrelated preferences. For instance, Hunch has revealed that people who enjoy dancing are more apt to want to buy a Mac, that people who like The Count onSesame Street tend to support legalizing marijuana, that pug owners are often fans of The Shawshank Redemption, and that users who prefer aisle seats on planes “spend more money on other people than themselves.”
Stuff like this is usually overblown a bit (writers always get a case of Gladwell-itis when talking statistics) but it’s also the future as more and more data about us gets logged in ways that allow for association.
On thing that occurs to me reading this is that the shift in statistics consumption is likely to mirror the post-internet shift in traditional publishing — strong associations, like books, used to be hard and expensive to churn out, so a lot of filtering went into the process up front — people would run statistics on things they thought might matter for other reasons.
With the advent of total life-logging via credit card, smartphones, and social media and the with rise of large and extensive cohort databases, associations become cheap — but the significance filter is passed on to the consumer.
In other words, if your publication filter is insufficient for the modern world (as Shirky claims) just imagine how inadequate your statistics filter is for the deluge about to come…
We often talk of social statistics, especially those that seem as straightforward as age, as if a bureaucrat were poised with a clipboard, peering through every window, counting; or, better still, had some machine to do it for them. The unsurprising truth is that, for many of the statistics we take for granted, there is no such bureaucrat, no machine, no easy count, we do not all clock in, or out, in order to be recorded, there is no roll call for each of our daily activities, no kindergarten 1, 2, 3.
What there is out there, more often than not, is thick strawberry jam, through which someone with a bad back on a tight schedule has to wade—and then try to tell us how many strawberries are in it.
I’m reading Blastland and Dilmot’s The Numbers Game right now, and it is brilliant so far. I love that it starts with one of the fundamental quantitative reasoning questions: What did you count and how did you count it?
The book is tangentially related to the long running BBC radio show More or Less, which you can can listen to for free here.
Milo Schield’s short paper Teaching the Social Construction of Statistics deals with “strawberry jam” issues, and is well worth a read.
Researching my health statistics class, and found this great walk-through of the issues of sensitivity and specificity in medical test design and interpretation. Clear, easy to read, and suitable for everyone.
Everyone that gets medical tests done or will get medical tests done (which, let’s face it, is everyone) should be familiar with this stuff, but it’s often hard to visualize. The author’s technique for representing this makes it wonderfully simple. Click through!
If you are interested in these issues, Kaiser Fung’s Numbers Rule Your World has a great discussion of how they relate to the sham that is steroids testing in sports.
The best book on health statistics I have read is Know Your Chances. For some reason there is a free PDF of it here. It’s short, you can read it in an afternoon, and it’s one of the most useful things you could spend your time on, honest.
I’ve been playing around with cognitive disfluency in slide design for my class lately, trying to solve a conundrum.
The problem is this — we know from research that reading materials that introduce “desirable difficulties” (such as presenting information in a difficult to read font) are recalled better than reading materials with a cleaner, more fluent presentation. This has been referred to as the “Comic Sans Effect”, after the notoriously hard to read font that is also apparently one of the more memorable. But the research shows that anything which disturbs fluency can have positive effects on recall — printing pages with a low toner cartridge, or producing deliberately bad photocopies.
(There’s a lot of caveats to this research, which I’ll deal with later — particularly around the issue of whether we are testing “difficulty” or “novelty”, but also it is a relatively new finding and it’s unclear how it transfers to something like slide design…)
The problem is there’s a natural tension between your need as a presenter to have your slides represent you as a professional, and your desire to introduce desirable difficulties into slide-reading. The slidesets linked below represent my attempt to strike that balance. They are heavily influenced by mid-90s graphic design and perhaps also by Leigh Blackall’s presentation style from five or six years ago (Leigh’s slides in that 2006 Networked Learning presentation seared themselves into my brain forever, a perfect example of this working well).
Anyway, here’s some attempts by me to do this. Viva disfluency!
Association & Causation
Observable/Unobservable, Inference, and Claims
From A. N. Whitehead’s An Introduction to Mathematics, a brilliant early reflection on what we now see as a System 1/System 2 problem: “It is a profoundly erroneous truism, repeated by all copy-books and by eminent people when they are making speeches, that we should cultivate the habit of thinking of what we are doing. The precise opposite is the case. Civilization advances by extending the number of important operations which we can perform without thinking about them.”