BOB: In the coverage of gun incidents in America, there is a recurring refrain...

CLIP: We’ve got all kinds of data about the benefits of firearms, defensive gun uses, ranging into the millions per year.

CLIP: Almost all national survey estimates indicate that defensive gun use by victims are at least as common as offensive uses by criminals.

CLIP: In fact, researchers have estimated, that there’s as many as 2 and a half million defensive uses of firearms in the United States each year.

BOB: So it seems gun owners use firearms to defend themselves from crime 2 and a half million times each year. It’s hard to summon an argument against such straightforward evidence. The problem is: it isn’t true. Or even close to being true. Evan Defilippis co-wrote an essay for Politico about how this faulty statistic spawned an unshakeable myth.

DEFILIPPIS: Well, this number comes from a study in 1992 conducted by professors at Florida State University, Gary Kleck and Marc Gertz. What happened was that these researchers called a random sub-sample of 5,000 Americans. 66 of them reported that they had used a gun in self-defense, so that's a little bit more than 1 percent. And then from that number, the researchers simply scale it up to the entire American population to get around 2.5 million defensive gun uses a year.

BOB: Now it can be legitimate to take a sample of data and project it based on larger populations, but there are statistical protocols for doing that. Was the sample size too small?

DEFILIPPIS: Well the real problem wasn't so much the sample size as much as it was the fact that defensive gun uses is an inherently rare phenomenon. And the real concern is that there just needs to be a small handful of people that are making defensive gun use claims when there isn't, and it completely biases this entire statistic.

BOB: You cite an example of another dubious survey...I'm speaking of course of alien abductions.

DEFILIPPIS: That's right. This is actually one of my favorite demonstrations of how absurd it is to scale up rare events in the way that Kleck and Gertz did. And this example comes from a paper done by David Hemingway at the Harvard School of Public Health. He mentions an ABC poll that was conducted in 1994 that surveyed 1500 adults. Well the survey asked whether respondents had ever seen an alien spacecraft and 150 people reported that they had in fact seen an alien spacecraft. Now if we were to use precisely the same methodology employed by Kleck and Gertz, we would be forced to conclude that something like 20 million americans have seen a spacecraft

BOB: And some of them are in Congress

DEFILIPPIS: [Laughs] I mean this is why respected social scientists aren't using this poll as definitive evidence of the existence of aliens, because they recognize that some people are simply confused about what it is that they're observing. And if you examine responses from surveys that inquire into defensive gun uses, for example, you have a situation where someone  hears some sort of spooky sound outside. They fire a warning shot into the air, and then they report that as an effective defensive gun use. There's one respondent who claimed that there was a group of kids surrounding his car and he brandished his weapon and they scattered. And he reported he had deterred some sort of potential crime. So we have all of these potential issues of simply misinterpretation.

BOB: Beyond the statistical risk, this subject of questioning in a telephone survey, you say will inevitably yield false positives -- why?

DEFILIPPIS: There was this article written by Dr. David Hemenway where he mentions that there's a host of potential biases. One of them is known as the social desirability bias. People will either lie or more probably embellish stories in order to appear better in front of an interviewer. Gun owners often feel that they need to justify ownership through the use of these defensive gun use examples, real or not. And another large problem is known as telescoping: a respondent may be reporting a legitimate self defense use that is outside the timeframe of the question asked in the survey. We know that this is an enormous problem because the National Crime Victimization survey which is considered the gold standard in criminal victimization surveys, they find that between 30-40% of defensive gun use incidents are simply an artifact of this telescoping bias.

BOB: So there's misremembering there's telescoping, and there's another not insignificant fact that the defensive gun uses reported by the survey takers are often, you know, crimes themselves.

DEFILIPPIS: That's right, and this is the part of the story that gun advocates actually don't like to talk about. Because even Gary Clerk admits that between 36-64% of defensive gun uses in his own survey were likely illegal. And Hemenway attempted to substantiate this claim. He did 2 random digit dial surveys in 1996 and 1999 where he asked open ended questions about defensive gun use incidents to respondents. He then took their detailed responses and gave them to 5 criminal court judges. And the judges determined that the majority of defensive gun uses were illegal, and dangerous to society. If this 2.5 million number has any credibility at all it would show an epidemic of massive proportions.

BOB: This study was published 23 years ago. It's been pretty thoroughly debunked, has it not?

DEFILIPPIS: That's correct  And if the claim is true, that guns are used in self defense 2.5 million times a year, we should be able to find at least a fraction in police reports and media reports. Well, there are a litany of right wing websites which scour the internet reporting any defensive gun use they can find. And then at the same time there are non partisan organizations, like the Gun Violence Archive, dedicated to scanning media and police reports. Well, if you put together the work of all of these organizations, they've barely been able to scrape together 1,600 defensive gun uses on an annual basis.

BOB: Now you've referred to the NRA and to right wing media that indiscriminately tosses around this bogus statistic. What about the mainstream media?

DEFILIPPIS: This number's trotted out in conservative media, in centrist media, as a fact. The issue with a lot of centrist sources is that in an effort to be fair and balanced, the author simply presents studies from both sides and comes to the conclusion that it's difficult to make a definitive judgement one way or the other, and the problem is that we have a public health tragedy on our hands, and claiming that all evidence carries the same weight appears to me to just be lazy journalism.

BOB: Now I just want to be clear about this. It's not as though you and your partners at Armed With Reason said, “Golly, I wonder what the effects are on public health of widespread gun ownership; let's just start at zero and accumulate the evidence we can.” No, your presumption was that easy access to guns creates a public health menace. That's your founding premise, is it not?

DEFILIPPIS: Well, we didn't actually have any founding premise prior to starting the website. It was only after researching the subject for a year that we became invested in this perspective that there is probably a positive association between gun prevalence and homicide, suicides, and accidents, and since then we've been writing for the last 2 years describing the exact nature and content of that relationship.

BOB: So if you weren't an advocate to begin with, you've become one.

DEFILIPPIS: That's right.

BOB: Evan, thank you so much.

DEFILIPPIS: Thanks so much for having me on the show.

BOB: Evan Defilippis is the co-founder of ArmedWithReason.com.