The Worried Well Whipped Into A Frenzy
Transcript
BOB GARFIELD: Gary Schwitzer has devoted his life to reviewing how health news is reported and, more often than not, misreported. We last spoke to him in 2009 when, in the face of continuously ham-handed health reports, he decided to stop examining network television coverage of health altogether. Here’s Gary at the end of our last interview.
GARY SCHWITZER: I believe that we have given over the airwaves to these companies that are abusing that privilege and misinforming the public inaccurately, in an imbalanced way and incompletely. Yeah, that’s, I think, how you’d define a disgrace.
BOB GARFIELD: But now, Gary’s back! His website, healthnewsreview.org, gathered a team of experienced reviewers to evaluate 1,899 health stories covering innovations in health science. His study, reviewing stories from 2006 to 2013, appeared earlier this month in the Journal of the American Medical Association. Schwitzer and his team judged the success or failure of these stories about new stuff in healthcare by developing and asking a set of very focused questions.
GARY SCHWITZER: In reporting about how wonderful something was, did it actually discuss the cost, did it adequately explain the size of the potential benefit and the size of the potential harm? Did it evaluate the quality of the evidence? Did it have more than a single source? And did it look at conflicts of interest in the source? Did the story rely solely or largely on a news release? Was it reporting on something that was truly available, or was this a phase one study that is years away? In reporting on the new, which is what our job is in journalism, did it also put the “new” into the context of existing alternatives, which, by default, have a longer, more proven track record?
BOB GARFIELD: Let’s talk about some of the more common errors. On the subject of statistical literacy, one of them that you divined was the confusion of risk reduction in relative terms versus absolute terms. Tell me what that looks like when it goes wrong.
GARY SCHWITZER: It's like having a 50% off coupon at Macy's, but you don't know what that 50% off coupon can be applied to. It can be applied to a diamond necklace or to a pack of chewing gum. Until you answer that, you don't know the absolute value; you only know the relative value. So let's not talk about a drug reducing the risk of something by 50%, 50% of what? If we’re talking about a change from 2 in 100 having a problem down to 1 in 100, that, indeed, is a 50% relative risk reduction, but to people with this condition, it's a difference of 1 in 100. And it means that everybody else who didn’t benefit had to take the drug, pay for it, run the risk of side effects and didn't get any result.
BOB GARFIELD: Another big problem is the failure to explain the limits of observational studies. What do you mean by that?
GARY SCHWITZER: I’m now into my second cup of coffee, and coffee stories lead the league in observational studies, it seems.
[BOB LAUGHING]
So, if I take half of your station and have them drink coffee for five years and keep a diary of what happens to ‘em, and another half doesn’t, I cannot say at the end of that observation that coffee raised or lowered their risk of stroke, because you cannot make a causal link from an observational study. It's not a true experiment. And over and over and over again we use active causal verbs, like “raised risk, lowered risk, boosted protection” to describe observational studies, and it’s simply inaccurate.
BOB GARFIELD: Because correlation is not causation.
GARY SCHWITZER: Yes.
BOB GARFIELD: The reporters, me among them, love to put a human face to an otherwise abstract story, you know, give it a little flesh and blood. But the anecdote bedevils you as a reviewer of health news. Why?
GARY SCHWITZER: Because that is possibly the leading category of imbalance. We tell the positive patient stories, but we don't tell about the dropouts from the trials, the people who couldn't maintain the regimen, why couldn't they stay with it. I think you got to feel the hand of the medical marketing mavens, when you see just these positive glowing success stories which dominate American health news.
BOB GARFIELD: One of the almost universal problems was new technology as being reported kind of uncritically as the next big thing. Why do you think journalists get all wide-eyed when the subject is medicine and health?
GARY SCHWITZER: When you're talking about proton beam radiation therapy machines that require a linear accelerator the size of a football field, it is, gee whiz, it’s all the kind of stuff that gets you on page 1. It also should be the kind of stuff that we’re reporting, whoa, let’s put the brakes on and look at where we have the evidence and where we don't for how this is any better than what we’re already using.
BOB GARFIELD: Is it the triumph of hope over journalistic skepticism that drives this, or something more sinister?
GARY SCHWITZER: Look, Bob, nobody who covers this stuff rolls out of bed in the morning and says, how can I go in and hurt people today. But, as we've shown, we harm people when we consistently deliver inaccurate, imbalanced, incomplete coverage of new stuff. We are whipping the worried well into a frenzy.
I’m going to talk squarely out of both sides of my mouth right now [LAUGHS], Bob. I want to acknowledge that we simultaneously are seeing some of the best healthcare journalism we've ever seen, with special in-depth investigative data-driven projects by – often by foundation-supported efforts, like ProPublica and Kaiser Health News. We are seeing some of the best. But this daily drumbeat of dreck that occurs in between these special projects, we would be better off with far less news coverage.
BOB GARFIELD: With the same mistakes made over and over and over, do you not tire of your efforts? Are you the Don Quixote of healthcare reporting?
GARY SCHWITZER: If I didn't think that this could affect meaningful change, I wouldn’t keep doing it. But I should tell you that, in fact, an end has been enforced on me, at least temporarily, but maybe permanently, because the eight years of foundation funding that I had to do this has ended. I've kept my site alive by my own individual blogging. I’ve not been able to find another source to fund this kind of work.
I think one of the reasons for that is that almost every day, with almost everything we write, we upset somebody in journalism. But we also upset somebody in healthcare, who liked the way that shoddy journalism was making their idea look better than it really may have been. So you tell me. Who is gonna fund this kind of stuff?
BOB GARFIELD: Good luck, and thank you.
GARY SCHWITZER: Well, thank you again for your continued interest in our work.
BOB GARFIELD: Gary Schwitzer is publisher of the website, healthnewsreview.org.
BOB GARFIELD: Gary Schwitzer has devoted his life to reviewing how health news is reported and, more often than not, misreported. We last spoke to him in 2009 when, in the face of continuously ham-handed health reports, he decided to stop examining network television coverage of health altogether. Here’s Gary at the end of our last interview.
GARY SCHWITZER: I believe that we have given over the airwaves to these companies that are abusing that privilege and misinforming the public inaccurately, in an imbalanced way and incompletely. Yeah, that’s, I think, how you’d define a disgrace.
BOB GARFIELD: But now, Gary’s back! His website, healthnewsreview.org, gathered a team of experienced reviewers to evaluate 1,899 health stories covering innovations in health science. His study, reviewing stories from 2006 to 2013, appeared earlier this month in the Journal of the American Medical Association. Schwitzer and his team judged the success or failure of these stories about new stuff in healthcare by developing and asking a set of very focused questions.
GARY SCHWITZER: In reporting about how wonderful something was, did it actually discuss the cost, did it adequately explain the size of the potential benefit and the size of the potential harm? Did it evaluate the quality of the evidence? Did it have more than a single source? And did it look at conflicts of interest in the source? Did the story rely solely or largely on a news release? Was it reporting on something that was truly available, or was this a phase one study that is years away? In reporting on the new, which is what our job is in journalism, did it also put the “new” into the context of existing alternatives, which, by default, have a longer, more proven track record?
BOB GARFIELD: Let’s talk about some of the more common errors. On the subject of statistical literacy, one of them that you divined was the confusion of risk reduction in relative terms versus absolute terms. Tell me what that looks like when it goes wrong.
GARY SCHWITZER: It's like having a 50% off coupon at Macy's, but you don't know what that 50% off coupon can be applied to. It can be applied to a diamond necklace or to a pack of chewing gum. Until you answer that, you don't know the absolute value; you only know the relative value. So let's not talk about a drug reducing the risk of something by 50%, 50% of what? If we’re talking about a change from 2 in 100 having a problem down to 1 in 100, that, indeed, is a 50% relative risk reduction, but to people with this condition, it's a difference of 1 in 100. And it means that everybody else who didn’t benefit had to take the drug, pay for it, run the risk of side effects and didn't get any result.
BOB GARFIELD: Another big problem is the failure to explain the limits of observational studies. What do you mean by that?
GARY SCHWITZER: I’m now into my second cup of coffee, and coffee stories lead the league in observational studies, it seems.
[BOB LAUGHING]
So, if I take half of your station and have them drink coffee for five years and keep a diary of what happens to ‘em, and another half doesn’t, I cannot say at the end of that observation that coffee raised or lowered their risk of stroke, because you cannot make a causal link from an observational study. It's not a true experiment. And over and over and over again we use active causal verbs, like “raised risk, lowered risk, boosted protection” to describe observational studies, and it’s simply inaccurate.
BOB GARFIELD: Because correlation is not causation.
GARY SCHWITZER: Yes.
BOB GARFIELD: The reporters, me among them, love to put a human face to an otherwise abstract story, you know, give it a little flesh and blood. But the anecdote bedevils you as a reviewer of health news. Why?
GARY SCHWITZER: Because that is possibly the leading category of imbalance. We tell the positive patient stories, but we don't tell about the dropouts from the trials, the people who couldn't maintain the regimen, why couldn't they stay with it. I think you got to feel the hand of the medical marketing mavens, when you see just these positive glowing success stories which dominate American health news.
BOB GARFIELD: One of the almost universal problems was new technology as being reported kind of uncritically as the next big thing. Why do you think journalists get all wide-eyed when the subject is medicine and health?
GARY SCHWITZER: When you're talking about proton beam radiation therapy machines that require a linear accelerator the size of a football field, it is, gee whiz, it’s all the kind of stuff that gets you on page 1. It also should be the kind of stuff that we’re reporting, whoa, let’s put the brakes on and look at where we have the evidence and where we don't for how this is any better than what we’re already using.
BOB GARFIELD: Is it the triumph of hope over journalistic skepticism that drives this, or something more sinister?
GARY SCHWITZER: Look, Bob, nobody who covers this stuff rolls out of bed in the morning and says, how can I go in and hurt people today. But, as we've shown, we harm people when we consistently deliver inaccurate, imbalanced, incomplete coverage of new stuff. We are whipping the worried well into a frenzy.
I’m going to talk squarely out of both sides of my mouth right now [LAUGHS], Bob. I want to acknowledge that we simultaneously are seeing some of the best healthcare journalism we've ever seen, with special in-depth investigative data-driven projects by – often by foundation-supported efforts, like ProPublica and Kaiser Health News. We are seeing some of the best. But this daily drumbeat of dreck that occurs in between these special projects, we would be better off with far less news coverage.
BOB GARFIELD: With the same mistakes made over and over and over, do you not tire of your efforts? Are you the Don Quixote of healthcare reporting?
GARY SCHWITZER: If I didn't think that this could affect meaningful change, I wouldn’t keep doing it. But I should tell you that, in fact, an end has been enforced on me, at least temporarily, but maybe permanently, because the eight years of foundation funding that I had to do this has ended. I've kept my site alive by my own individual blogging. I’ve not been able to find another source to fund this kind of work.
I think one of the reasons for that is that almost every day, with almost everything we write, we upset somebody in journalism. But we also upset somebody in healthcare, who liked the way that shoddy journalism was making their idea look better than it really may have been. So you tell me. Who is gonna fund this kind of stuff?
BOB GARFIELD: Good luck, and thank you.
GARY SCHWITZER: Well, thank you again for your continued interest in our work.
BOB GARFIELD: Gary Schwitzer is publisher of the website, healthnewsreview.org.
Hosted by Bob Garfield
Produced by WNYC Studios