BOB: This is On the Media, I’m Bob Garfield.

 

BROOKE: And I’m Brooke Gladstone. So we end another week in this endless election cycle, and you’re inundated with polls. But they don’t all send the same message.

 

CLIP: Senator Ted Cruz leads Donald Trump 31 percent to 21 percent in the new Des Moines Register/Bloomberg politics polls. Cruz surging 21 points since August, into the lead.”

CLIP:…including this one from Monmouth University which shows Trump at 41% - I think 27 points ahead of Ted Cruz”

 

BROOKE: : Trump is winning and losing? What does that mean? And ten months out from the general election, who cares?  When do polls matter?

For guidance on that and many other questions, we bring you another installment of our Breaking News Consumers Handbook. This time: Election Poll Edition.

Harry Enten is a senior political writer and analyst at the data news site Fivethirtyeight, our partner this season in probing the polls.  Harry, welcome to On the Media.

 

ENTEN: Thanks for having me.

 

BROOKE: What's going on?

 

ENTEN: Chaos! Chaos is going on. No. When you look at a primary, right, there are many different ways you can poll it. You can poll state by state or nationally. Donald Trump is performing very well in the national polls right now but in Iowa he's actually trailing in the majority of the polls as he has throughout his entire candidacy.

 

BROOKE: Let's talk about what a national primary poll is. there is no national primary, so it measures nothing, right?

 

ENTEN: That's exactly what it is. Primary polls that are taken nationally are measuring something that doesn't actually exist. There is no national primary. If you were to take a national primary poll average at this point and compare it in the past to how predictive it was, the margin of error is something like plus or minus 28 percentage points, so I ran a headline this past week where I said "Donald Trump may end up with anywhere between 8 and 64 percent of the vote." Primaries occur by a state by state basis, and the first primary is actually the Iowa caucus. And then you have the New Hampshire primary, so I would be paying attention to polls that are coming from both of those. At this point they're stll not all that predictive, but they get more predictive as we get close to the primary but they are certainly more predictive of those contests than the national polls are of the national primary.

 

BROOKE: So, when should we start paying attention maybe 6 weeks before the caucus? Are we going to get anywhere near predictability at that point?

 

ENTEN: I mean, if somebody is leading by a very large margin, then yes, you know, whoever's leading 6 weeks out would have a pretty good shot of winning. But it's all a chain of events, right, you have to at least do well in Iowa or New Hampshire, and then if you do so you can go on to win later primaries and win the nomination.

 

BROOKE: What about the general election. When should we start paying attention to national polls?

 

ENTEN: We know from the past that the general election polls right now are just not that predictive. pay attention to national general election polls much further out from where the actual voting occurs. If there's someone who has, you know, a three or four or five point lead after both major party conventions, that person usually goes on to win the presidency.

 

BROOKE: You and other poll watchers pay close attention to how polls are conducted. What are we to look for?

 

ENTEN: Who is this pollster, have we actually heard of this pollster, do they have a good track record? The Des Moines Register poll by Ann Selzer has a tremendous track records. While somebody like Zogby and Ractif has a horrible record. The American research group out of New Hampshire has not been tremendously accurate. Now that doesn't mean they won't be or that they're not trying to improve, but at least in the past, they haven't been particularly good.

BROOKE: What else?

 

ENTEN: Look at who you are sampling. Primaries tend to attract lower participation rates than presidential general elections. So we don't want to just take a poll of all the possible voters, we want to take a poll of likely voters, the people who are actually going to show up and vote in that primary.

BROOKE: And midterms.

 

ENTEN: And midterms as well. Basically those elections in which we'll have a smaller turnout than in presidential general elections. Polls that use quote unquote likely voters tend to be more accurate than those who use registered voters.

 

BROOKE: What else.?

 

ENTEN: How are you conducting that poll, are you conducting it over the phone? Are you conducting it via the internet? Are you conducting it over the phone but with an automated dialer and then someone presses one for Mitt Romney, two for Barack Obama, three for undecided? Live interview polls have tended to be more accurate, especially in primaries. At least the United States.

 

BROOKE: Four years ago, 21% of polling of the GOP primaries was done online. This year it's half. I assume because it's way cheaper.

 

ENTEN: Yeah. I don't have to tell you that media budgets are decreasing, newspapers which traditionally have paid for state polling, they're going, goodbye, see ya later. Media is looking for any way that they can cut costs and internet polls are simply cheaper. That doesn't necessarily mean that they'll end up less accurate, but it could end up being the case.

 

BROOKE: You've observed that Donald Trump does a lot better when the interviewer is not a real person on the line, and this year, no live interview pollsters are producing weekly results, so the type of poll that favors Trump is in the news a lot more often.

 

ENTEN: It makes my job more difficult and it makes the media consumer's job more difficult because they are being told constantly Donald Trump is at x percentage when the live interviewer polls actually have him at x percentage minus say two or three or four or five depending on the exact pollster.

 

BROOKE: Let's talk about the one bit of qualification that we the news consumer get every single time:

 

CLIP MONTAGE

She has a six point lead and the margin of error in that poll is 5.9%

That poll has a margin of error of plus or minus 5 points...

It has a margin of error of 4.9%

 

BROOKE: What the margin of error is, we're not always sure what that means. And you suggest that maybe it doesn't matter that much anyway?

 

ENTEN: There are any number of problems with this. The biggest problem is that if we go back to primary polls, very few people turnout for primaries. The margin of error assumes that you are polling the correct universe of actual primary voters. In fact, you may be polling people who are never going to show up for the Iowa caucus for example, And then your margin of error you can basically throw it out the window. I would much rather poll 350 people that I know are going to show up for the Iowa caucus, than poll 1,000 people and not have any sort of clue whether or not they're going to show up.

 

BROOKE: Another way to play the averages here, is as you note, pay attention to averages, and trends.

 

ENTEN: Because even the best pollsters, Anne Seltzer for instance, produced a poll in the middle of 2012 which had Barack Obama beating Mitt Romney by double digits, when pretty much all of the other polls showed a much closer race. And Selzer rightfully stood by her pool, she said I did the best methods, but sometimes you have outlier,s and in this particular case even the best had an outlier. So paying attention to the averages generally helps.

 

BROOKE: Let's talk about outliers. HOw are they often portrayed in the media?

 

ENTEN: Look, news media is always looking for a story, and the best story that you can get is an outlier -- oh my god, we have this brand new result and it shows something completely different!!!

 

CLIP MONTAGE:

A stunning new poll sure to shock the Clinton campaign this morning

Just minutes away a shocking new poll

And some new polling numbers make a stunning forecast for the midterm elections

 

ENTEN: So stay tuned to your local news next, or whatever.

 

BROOKE: It's a stunner!

 

ENTEN: It's a stunner!

 

BROOKE: It's a bombshell!

 

ENTEN: So, outliers tend to get outsized media attention and most times outliers are outliers for a reason and that is that they are incorrect.

 

BROOKE: This whole discussion reminds me of a personal bugaboo, which I bring up every time, and that is that stupid question always asked from the very beginning: who would you vote for if the election were tomorrow? Obviously, most people answer with a guess because the vast majority of people are still undecided. Which would be obvious if you were to ask the real question, who are you going to vote for in November? The answer would be so full of undecideds it wouldn't be worth a headline.

 

ENTEN: Maybe, or they could offer undecided as a third option, right, instead of just saying are you gonna vote for x or y, you could say are you going to vote for x, y or are you undecided at this point.

 

BROOKE: But they don't, do they.

 

ENTEN: They don't.

 

BROOKE: Another place where I think they try and drum up headlines is when they poll on hypothetical match ups. I mean, for a long time they were matching Joe Biden with everybody under the sun. We have some others --

 

[CLIP MONTAGE]

Quinnipiac University puts Ben Carson ahead of Hillary Clinton if the general election happened today...

Clinton versus Trump, she would win if that election were held today...

Hypothetical general election match up: Bernie Sanders, Democratic nominee, Republican nominee, by 8 points.

 

BROOKE: Isn't this business with the hypothetical match ups more about donors and less about the public?

 

ENTEN: You want to be the quote unquote electable candidate, and that's something that these hypothetical matchups supposedly can tell you, though if I were a donor I wouldn't buy it for a second.

 

BROOKE: So, On the Media and FiveThirtyEight are collaborating now to look at this dance between polling and media coverage. We already know that the polls fuel media narratives, and that those media expectations do matter. You could lose Iowa and still win Iowa!

 

ENTEN: You can actually see this statistically whereby a person who underperforms in Iowa even if they win the Iowa caucus then does worse than expected in New Hampshire because the media portrays that quote unquote win as a loss and somebody who actually loses but performs better than expectation the exact opposite. So I go back to the 1984 Democratic primary, where Walter Mondale basically whipped Gary Hart in the state of Iowa. The media portrayed Hart as the winner because the media said that Hart performed better than expectations  in Iowa even though he finished only at I think 17% of the vote, and then he was able to go on and win in New Hampshire --

 

HART: Tonight in New Hampshire, we buried the label dark horse...

 

ENTEN: And then it was a race because the media is always looking to create a race.

 

BROOKE: Many have told us there's too much noise in the system for the polls to be productive at this point, even some pollsters have warned that these debate lineups probably shouldn't be based so much on these noisy numbers, but the polls keep coming out, the media keep paying for them and reporting on them, why?

ENTEN: People like to know what's going to happen and they believe rightly or wrongly that polls can tell us what's going to happen even though probably more wrongly in primaries at least this far out.

BROOKE: Thanks a lot, Harry.

ENTEN: Thank you.

 

BROOKE: Harry Enten is a senior political writer and analyst for Fivethirtyeight. You can find their election coverage at fivethirtyeight dot com slash politics. And there’s a handy, one-page edition of our breaking news consumers handbook at onthemedia.org.