[CLIP]

 

NEWS REPORT We've known the truth about the president in the Ukraine almost from the very beginning.  He said on the White House lawn to reporters that he did it.  [END CLIP]

 

BROOKE GLADSTONE So that's settled, right? Not so fast. 

 

[CLIP]

 

SEAN HANNITY This was the day the Democrats, Trump, Ukraine, Quid pro quo, coup, impeachment attempt and hoax officially died. [END CLIP]

 

BROOKE GLADSTONE Obviously, disagreement is the point. From WNYC in New York, this is On the Media. I'm Brooke Gladstone. So facts are up for grabs. At least we can rely on numbers to tell the story. No. Turns out the algorithms that run our lives are just opinions embedded in math.

 

[CLIP]

 

CATHY O’NEIL Applying for credit, applying for a job, applying to go to college, applying for a loan. All those things are now algorithmic. So it's really, really important to remember that it's not necessarily an opinion that you have to share. [END CLIP]

 

BROOKE GLADSTONE It's all coming up after this.

 

From WNYC in New York, this is On the Media. Bob Garfield is out this week. I'm Brooke Gladstone and here's EU Ambassador Gordon Sutherland famously repeating

 

[CLIP]

 

GORDON SOTHERLAND Everyone was in the loop. Was there a quid pro quo? The answer is yes. Everyone was in the loop. [END CLIP]

 

BROOKE GLADSTONE He spoke at the impeachment inquiry this week over which Democrat Adam Schiff presided, serving up an evidence-based narrative countered by Republican Devin Nunes’ unfounded one. I make this assessment on facts available to all. And here's Ukraine diplomat David Holmes.

 

[CLIP]

 

DAVID HOLMES The three priorities of security, economy and justice and our support for Ukrainian Democratic resistance to Russian aggression became overshadowed by a political agenda promoted by former New York City Mayor Rudy Giuliani and a cadre of officials operating with a direct channel to the White House. [END CLIP]

 

BROOKE GLADSTONE and former White House Russia expert Fiona Hill.

 

[CLIP]

 

FIONA HILL Some of you on this committee appear to believe that Russia and its security services did not conduct a campaign against our country and that perhaps somehow for some reason, Ukraine did. This is a fictional narrative that has been perpetrated and propagated by the Russian security services themselves. [END CLIP]

 

BROOKE GLADSTONE The so-called legacy media saw testimony confirmed by many of a quid pro quo for personal gain. Or was there? Trump's defenders had nothing. Or did they? Fox News commentator Jesse Waters.

 

[CLIP]

 

JESSE WATERS The hoax is over. I mean, you guys tried. You failed. You didn't have one witness to testify that Trump directed a quid pro quo involving military aid. Not one. [END CLIP]

 

BROOKE GLADSTONE Fox News host, Sean Hannity

 

SEAN HANNITY This was the day that Democrats, Trump Ukraine, Quid pro quo, coup impeachment attempt and hoax officially died. [END CLIP] 

 

DAVID ROBERTS The odd thing about this entire affair is that we've known the truth about the president in the Ukraine almost from the very beginning. I mean, he said on the White House lawn to reporters that he did it.

 

BROOKE GLADSTONE That's David Roberts, who covers energy and politics for Vox, who set out to answer the eternal question, ‘How can the president's defenders on Capitol Hill and on Fox News and such like media keep denying the obvious, keep repeating narratives and timelines and outright b.s. that is continually, definitively debunked? An NPR/PBS NewsHour Marist poll published earlier this week found that two thirds of Americans say nothing they hear in the impeachment inquiry would cause them to change their minds about whether Trump ought to be impeached. David Roberts calls this a pivotal moment in our epistemic crisis, meaning the crisis in how we acquire knowledge?

 

DAVID ROBERTS Right. The crisis in how we come to know things, how we learn things. What we count as true, who we trust. And by the crisis, I just mean we're sort of splitting into two worlds, not just two different sets of values or priorities or sort of visions for the country, but two different fact sets. And you get people dug in to where they're saying things to pollsters like, look, I picked my side. I don't care what anyone says anymore. 

 

BROOKE GLADSTONE The Democrats have the field, they have the talent, they have the facts, they have the format, but you argue that it is very much in doubt that they could win.

 

DAVID ROBERTS By any sort of objective standard, if there were a referee, the referee would have stepped in and said, this is over. You guys have established your case. But it is the lack of referees that is precisely the problem. You would have to have a referee whose word is respected by both sides of the contest. And that's what we don't have anymore. So one side's referees can say, hey, we think it's over and obvious in the other side's referees can just say no. We refuse to acknowledge it. And in that circumstance, there's just no way to finish the game.

 

BROOKE GLADSTONE So you argued in a piece last weekend on Vox that this moment is defined by tribal epistemology and that it's kind of like tribal morality.

 

DAVID ROBERTS Right. So it's tribal morality, I think is is something people are more familiar with. It's when you decide the welfare of my tribe, my group takes precedence over principles. So, for instance, in the mid 2000s, we had a big national debate about torture and the sort of two sides. One was saying, ‘No, don't torture it’s a principle. It applies to everyone, right? Us and them.’ But the other side was saying, ‘look, no, we're good people. We're on the side of right. So if we torture, it's OK because our tribe is good. So what we do is good.’ And tribal epistemology is just the same thing. You put your tribal interests ahead of sort of epistemological principles of evidence and self correction and self-criticism and peer review, all these sort of mechanisms we have for determining truth, you just put your tribe's interests ahead of those. So you end up saying we're gonna believe whatever is useful for us in the moment to believe.

 

BROOKE GLADSTONE I was thinking about the ejection by the president of members of the tribe for betrayal. And as you might put it, epistemic transgressions. 

 

DAVID ROBERTS Yes. 

 

BROOKE GLADSTONE For example, we have a tape of Representative Jim Himes questioning Jennifer Williams on Tuesday. She worked for Pence. She also worked for Obama and the Bush administration before that and for the Bush-Cheney reelection campaign.

 

[CLIP]

 

JIM HIMES Ms. Williams on Sunday, the president personally targeted you in a tweet. It reads, tell Jennifer Williams, whoever that is, to read both transcripts of the presidential calls and see the just released statement from Ukraine. Then she should meet with the other “Never Trumpers” who I don't know and mostly never even heard of and work out a better presidential attack. [END CLIP] 

 

DAVID ROBERTS Two things are notable about that. As you say, the entire weight of her experience and previous commitments when she crossed the tribe, it was tissue paper. It meant nothing. And secondly, Trump is sort of like exacerbated all this in the sense that his behavior is so erratic in his sort of assertions or sort of changing day by day. So there's no way to even kind of pretend anymore that you're basing your defense of him on principles. He makes that impossible. So if you want to be a member of the tribe in good standing, you have to go along with all of Trump's veering back and forth. The fact that Mike Pence did nothing to step up and defend her shows that he knows the game like, well, if this is the way Trump veered today. Oh, well, I got, I'm following him come what may.

 

BROOKE GLADSTONE People generally are too busy to pay close attention to politics. They pay attention to fragments and they use shorthand. And I wanted you to talk about those heuristics.

 

DAVID ROBERTS I think it's perfectly normal and it's not in any way derogatory to say that most people just sort of see headlines here and there, watch the nightly news, occasionally scroll past it on Facebook. And so they have to use heuristics, shortcuts. And one heuristic that's very strong in all people, I think is this sense that if something is valid, if it makes sense, it will command at least some agreement from both sides of a controversy. Right? If you can get consensus, it's a signal of legitimacy. To take an example, if we see a Republican come forward and say something's wrong with this behavior that Trump did, we sort of grant it a little bit more weight. But the problem is what Mitch McConnell, the Senate Republican leader, figured out when Obama was elected. It's fully in my power to just give him consensus and agreement across party lines on nothing. The voting public won't understand the mechanisms of Washington well enough to know that it's me doing it. They just look up and see the president and they say, gosh, everything Obama does just causes a big fight. So if the public is just grumpy and disgusted and tuned out because the whole thing seems like a ridiculous squabble that never ends, that is absolutely to the Republicans benefit. Our base will fight alongside us every step of the way. And this sort of unengaged middle is going to see this mess and just tune out.

 

BROOKE GLADSTONE So when we talk about Trump's base, we're talking about under a quarter of registered voters. But what's going on in the Hill now isn't about them. They're already on board. This is about that 30 percent, according to the NPR/PBS Marist poll that says their minds are still open to information coming to light in the hearings. So what are the president's defenders need, this 30 percent of us to do or not do?

 

DAVID ROBERTS As I said, the way to make people dismiss something is just to make it look partisan, which is why they keep attacking the process itself. And what they want is for that 30 percent to look at this spectacle and say, oh, it's just more of this, I don't care, I'm going to tune out. That's what they want and need is for nothing to break through to that fuzzy middle.

 

BROOKE GLADSTONE I also think that there's this effort to make these proceedings incomprehensible.

 

DAVID ROBERTS Yes. Well, it's just the starkest dichotomy as you're watching now of these sort of somber Democrats asking factual questions and the Nunes coming in with these sort of random conspiracy theories that if you aren't closely following Fox in the kind of right wing bubble, you wouldn't even recognize what he's referring to, like 

 

[CLIP]

 

DEVIN NUNES Democrats on this very committee negotiated with people who they thought were Ukrainians in order to obtain nude pictures of Trump. [END CLIP] 

 

DAVID ROBERTS Believe me, you don't need to know. But it's like there's this whole menagerie of conspiracy theories that you won't even know what they are unless you're in that bubble. And those are the people that Republicans are speaking to. So if you're just a member of the general public, they’re literally not going to make sense to you.

 

BROOKE GLADSTONE We saw a perfect example of this on Tuesday when Nunes said this.

 

[CLIP]

 

DEVIN NUNES Well, ambassador and  Mr. Morrison, I have some bad news for you. TV ratings are way down, way down. I don't. Don't hold it personally. I don't think it's you guys. Whatever drug deal the Democrats are cooking up here on the dais, American people aren't buying it. [END CLIP] 

 

DAVID ROBERTS What's amazing about that to me is it shows that Trump's mentality is infusing the party now to someone like me or you, who cares, what are the facts of the case, whether or not it's getting good ratings seems very obviously irrelevant. But if there is no such thing as truth, if there's only one side's truth and the other side's truth, then the performance of the argument is the only thing. And so to Nunes, the fact that the Democrats performance isn't drawing ratings is entirely relevant. In his mind, there is nothing but competing performances going on here.

 

BROOKE GLADSTONE You wrote that Democrats are attempting to do something that arguably nothing since the 9/11 attacks has done unite Americans in a clear understanding of a threat and a clear will to action. And you wrote that at this point it seems entirely possible that it won't happen. So what then?

 

DAVID ROBERTS Well, I want to predict that it's going to fall apart because you can't operate a movement in a country based on fantasies. But on the flip side, I kind of thought that a long time ago and sort of defied gravity sort of the other way. You know, the other way things can go is if this sort of cultish, increasingly authoritarian movement takes over the country. In Russia and Turkey and Poland, right it's a disturbingly longer and longer list, we see countries that we thought were democracies devolve into this. In the U.S, so much has happened in the last few years that we thought would never happen. I think we should really loosen up our imaginations as to what can happen when a movement that is convinced that everything it knows and loves is in danger of falling apart. A movement that's thinking like that unconnected anymore to facts or reality and got its hands on the power of the federal government is the basic recipe for democracies falling apart.

 

BROOKE GLADSTONE Hannah Arendt once said, “No matter how large the tissue of falsehood that an experienced liar has to offer, it will never be large enough to cover the immensity of factuality.” But I guess she never reckoned with cable news and the Internet.

 

DAVID ROBERTS Right, I mean, she didn't know about Facebook. I mean, this all these things work, right? Like historians are gonna look back and they're gonna say, whoa, that was an unpleasant and ridiculous episode. And there's gonna be, you know, sort of hearings. And we right ourselves eventually, it seems like. But it would be nice if for once when we're in the middle of one of these episodes, we could just stop it while it's happening. You have to think what could intervene and stop what looks like a spiral toward a really grim conclusion. And that's kind of my impulse in writing the column, is just sort of smack people upside the head and get him to start thinking like these are not normal times. What would be sufficient to break through this frozen partisan drift toward collapse? And I really feel like the general public needs to get engaged here because there are real dangers lurking around every corner here.

 

BROOKE GLADSTONE David, thank you very much. 

 

DAVID ROBERTS Thank you, Brooke. 

 

BROOKE GLADSTONE David Roberts writes about energy and politics for Vox.

 

[MUSIC]

 

Coming up, how algorithms deepen our dangerous misconceptions about how the world works.

 

[CLIP]

 

If I get to define success for myself, that's one thing. But if Facebook is defining success for me, I don't trust it. [END CLIP]

 

BROOKE GLADSTONE Good times. This is On the Media.

 

This is On the Media. I'm Brooke Gladstone. So obviously politics has divided us into two camps where we don't share any facts, but numbers, they’re unassailable right? They're neutral. Well, to continue on the path we began this hour about how we fool ourselves for fun and profit. Let's talk tech. Want to predict who's going to default on a loan or commit a crime or a washout at work? We've got an algorithm for that. Mostly they stink, but now they're taking over.

 

[CLIP]

 

NEWS REPORT Algorithms are replacing human advisers and brokers. [END CLIP]

 

[CLIP]

 

NEWS REPORT Our military uses an algorithm in their Skynet program to decide who should be on the terrorism kill list. [END CLIP]

 

[CLIP]

 

NEWS REPORT They are being used for everything from diagnosing illnesses to helping police predict crime hotspots. [END CLIP]

 

BROOKE GLADSTONE Cathy O'Neil is a mathematician, data scientist and investigative journalist. She founded the consulting firm O R C A A, ORCAA, which audits algorithms for racial, gender, economic inequality, an all around bad science. She loves math. She used to be a Wall Street quant, but something about the financial meltdown of 2008 turned her off the use of algorithms for the purposes of prediction, something about how no one actually check to see if they really worked and what happens when they don't. And even when they do. She's the author of Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy. I started with the basics. What's an algorithm?

 

CATHY O’NEIL It’s just a set of directions. Long division that you learn in fourth grade is an algorithm. I use the word algorithm. It's short for predictive algorithm, and that's a way of predicting the future based on the past. And we use the training data that is all around us.

 

BROOKE GLADSTONE When you say training data

 

CATHY O’NEILThe information we've collected from the past, like it's memories

 

BROOKE GLADSTONE They are used to select to whom to give or deny a loan

CATHY O’NEIL That's right. 

 

BROOKE GLADSTONE Who gets hired and who doesn't? 

 

CATHY O’NEIL Who goes to prison and for how long? 

 

BROOKE GLADSTONE There are algorithms. And there are weapons of math destruction. What's the difference?

 

CATHY O’NEIL Most algorithms are totally benign. Like I could build an algorithm in my basement on my computer. I could be trying to predict the stock market and nobody uses it. It doesn't matter. So I guess the shorthand version is algorithms that are important, that are destructive and that are secret. That's the weapon of math destruction.

 

BROOKE GLADSTONE Let's talk about some algorithms that you note in your book that are maybe one of those things, but not the others. For instance, sports algorithms predicting how teams or players may behave, fed data that actually reflect the behavior that they're trying to predict. They are regularly updated and though they are widely used, you wouldn't regard them as a weapon of math destruction.

 

CATHY O’NEIL Correct. So they're very important, like in the sense that there's a lot of money behind them and people really care if they're right or wrong. But if they make a mistake, that gets learned. So if we don't trade for a player and they go to another team and do really well, the algorithm learns that they made a mistake. And that's often not the case for weapons of math destruction. But the real thing that distinguishes that is that it doesn't wreak havoc.

 

BROOKE GLADSTONE Can you give me an example of using proxies? 

 

CATHY O’NEIL Mmm.

 

BROOKE GLADSTONE  You can't actually use the real thing like an athlete's behavior in the league. You have to use something that might be an indicator of something else. That's a proxy that characterizes a lot of your WMD. 

 

CATHY O’NEIL Absolutely.

 

BROOKE GLADSTONE Give me an example of that.

 

CATHY O’NEIL Well, the most pernicious example of that is, in my opinion, the predictive policing or the crime rate scores, the recidivism rate scores.

 

[CLIP]

 

NEWS REPORT Police in Los Angeles are trying to predict the future.

 

We know where crime happened yesterday, but where's it going to happen tomorrow and the next day? 

 

And they're not alone. More and more departments are using data driven algorithms to forecast crime. [END CLIP]

 

CATHY O’NEIL So they predict locations of arrests. To say that's where the crime must be, rather than acknowledging that police act differently in certain neighborhoods than they act in other neighborhoods, we don't really have crime data. If you think about it, there's lots of crimes that go on that do not lead to arrests. There's lots of pot smoking among white people that never get arrested. So there's a lot more sort of non arrested white crime than people of color. So when we use arrests as a proxy for crime, we are really over burdening those people who are already profiled by the police.

 

BROOKE GLADSTONE So in that case, arrests are used instead of criminal behavior?

 

CATHY O’NEIL Yes. When I say something like arrests are a bad proxy for crime. A lot of I'm sure a lot of your listeners are like, but people get arrested for crimes. You know, I just want to make a point that, you know, I've talked to a lot of police chiefs and a lot of judges about these kinds of algorithms. And one of things they keep on coming back to is almost no real mental health care in this country, so people get arrested very consistently for addiction problems or untreated mental health problems. That's not crime. And the police don't think it's crime. The judges don't really want to think of it as crime. And yet these scoring systems are basically suggesting since this person is much more likely to be rearrested in the future because guess what, they're still going to be addicted or they're still going to a mental health problem or they're still gonna be poor. And there's all sorts of crime.

 

BROOKE GLADSTONE There's still going to be living in the neighborhood where behavior that could go on pretty much unchallenged in a frat house is going to send you into the system.

 

CATHY O’NEIL Exactly. So those very predictable things show up quite well statistically. So that's that is how your score goes up. That's how your you get to that point where you're considered high risk and you're actually sentenced to prison for longer. And the judges don't like it. The judges want to be putting people into prison because they're actually a public health risk, not because they're addicts.

 

BROOKE GLADSTONE That's a proxy that characterizes a lot of your WMD.

 

CATHY O’NEIL Everybody will understand hiring algorithms. So let's just say you have a big company, Brooke. You're just like, oh, my God, I'm getting so many applications for these 10 positions. And like, I get a thousand applications. How am I going to sort through them all? I would like somebody to help me. And I don't want it to be a person because they're too expensive. I want it to be an algorithm. And so you hire me. I'm your data scientist and I come and build you an algorithm to sort through these applications. Well, you're going to say, Kathy, I want to hire people that will be successful at my firm. And I'll say, OK, well, what do you mean by success? This is where the proxy comes in. You'll say, well, I don't measure directly whether someone's good at their job, b

 

BROOKE GLADSTONE Cuz you can't know that 

 

CATHY O’NEIL Because, how do you know that? Right. I mean, what does it mean for someone to do well at your company? What do you think?

 

BROOKE GLADSTONE OK. They stay. 

 

CATHY O’NEIL They stay a long time. 

 

BROOKE GLADSTONE They generate a lot of good ideas. 

 

CATHY O’NEIL How do you measure that, though? 

 

BROOKE GLADSTONE Let's see. They get promoted.

 

CATHY O’NEIL Excellent. OK. So we have this kind of triumvirate of data points about each employee, like how long do they stay? How many promotions? How many raises? This is exactly the way that people measure success at companies. And this is exactly the kind of algorithm that gets built. So I'm training your data on 20 years of your past practice and hiring people. Boom. Implicit bias that we know exists in who gets promoted, who gets hired, who gets raises, who stays for a long time, who feels welcome enough to stay for a long time, gets baked in to the algorithm that I just wrote.

 

BROOKE GLADSTONE And you observe in the book that almost all of these algorithms predict behavior, not on what you do or what you've done, because that's so hard to measure, but who you are.

 

CATHY O’NEIL And I'll just add one last point to emphasize how invisible this might look from the perspective of the employer. Certain kinds of mistakes are much more obvious when you do it this way, namely false positives, which is to say I've hired someone they didn't work out. That is easy to spot because what a pain. But you don't see are the people that you could have hired that would have worked out but were filtered out. And that's where we see the sort of narrowing bottleneck of who is deemed future successful.

 

BROOKE GLADSTONE Another thing we don't really understand how it works, but we have to worry about its implications is the facial recognition technology. 

 

CATHY O’NEIL Yes. 

 

BROOKE GLADSTONE What's the problem there?

 

CATHY O’NEIL Well, there's a bunch of different levels of problems. One of them is that it sometimes just doesn't work. So my friend Joy Buolamwini at MIT Media Lab, was the first to come out with an audit. She's done a few audits now, Amazon most recently, where she found that at a technical level, they weren't working very well.

 

[CLIP]

 

JOY BUOLAMWINI All companies perform better on males than females. And all companies also perform better on lighter subjects than on darker subjects. We saw that all companies performed worst on darker females. In fact, as we tested women with darker and darker skin, the chances of being correctly gendered came close to a coin toss. [END CLIP]

 

BROOKE GLADSTONE So why? Why does facial recognition work better on white men than black women?

 

CATHY O’NEIL It doesn't have to. It just happens to because of the training data, literally the corpus of pictures that were used to train the algorithm was much more white and much more male. I think Joy calls them the pale male data problem and believe it or not, they weren't thinking carefully enough before deploying it to the world to say, hey, does this work as well on black faces as white faces? Why don't these companies get ahead of this a little bit and test this and sort of have evidence in advance that this is not going to be unfair. 

 

BROOKE GLADSTONE Who determines if the algorithms are working and how? 

 

CATHY O’NEIL That's kind of the craziest part. And I'm so glad you asked. Nobody. There is no standard. A large company says, oh, we don't want to build these algorithms. We want to rent them. Essentially, we licensed them from some data vendor and the data vendor says, oh, you can trust this, but we're not going explain it at all. It's a black box.

 

BROOKE GLADSTONE Proprietary. 

 

CATHY O’NEIL That's part of the licensing agreement. You don't get to know how this works, but you can use it to hire people. You can use it if you're a police department to find people, you can use it if you're a department of education to fire your teachers, blah, blah, blah. There is no particular standard.

 

BROOKE GLADSTONE Which brings us to the issue of how do you determine when an algorithm is successful? What is your definition of success? I was really moved by your discussion of “Clopenings.”.

 

CATHY O’NEIL Mmmm. Yeah. This is a great example of where the definition of success for the people using the algorithm is the opposite of the definition of success for the people who are targeted by the algorithm. So clopenings is the concept where you are basically a minimum wage worker is probably working, you know, in a large store and you close on one evening and then you open the store the next morning and you probably don't have enough time to even go home and see your kids. Right? Barely enough time to sleep. And the crazy thing is that these scheduling algorithms will for one week make you clopen three days in a row and then the next week you don't have any work at all. I looked into the research that was developing these algorithms. One of them made me cry. I mean, it was so brutal. It was like, you have the option if you use this algorithm to toggle the switch to make sure that none of your employees get enough hours a week to qualify for benefits. You can just turn this little switch on and like all of your employees, will be wage slaves forever. They will not be able to go to night school because their hours change every day. They will not be able to put their kids in daycare regularly. It is such a small benefit for the employer if you compare it to the wrecking of the life of the employee, it's maddening, but it's not actually technically illegal. So the algorithm exploits it.

 

BROOKE GLADSTONE You posed the question should we as a society be willing to sacrifice a little efficiency in the interest of fairness? And you talk about Starbucks. Starbucks wants to have a good image. Its scheduling algorithm was exposed?

 

CATHY O’NEIL Yeah.

 

BROOKE GLADSTONE It said that it was going to improve it. No more clop innings, no more employing people. Short of triggering some benefits. 

 

CATHY O’NEIL Right. 

 

BROOKE GLADSTONE But the trouble was that the incentives to managers to be efficient were so irresistible that they never actually made any changes.

 

CATHY O’NEIL Yeah, I mean, it's a philosophical question. It's basically you're saying do we have any answer to capitalism? All these algorithms that they're using in these corporate settings are about profitability, not about happiness. So if we wanted to address that, we would actually have to change the incentive structure of corporations. It's a big ask.

 

BROOKE GLADSTONE How would you assess the way that we the general public view algorithms?

 

CATHY O’NEIL I want us to learn to be skeptical. I want us to say, I don't need a math pHD to ask you why I'm getting fired. The power that we give to the algorithms is the thing we have the most control over.

 

BROOKE GLADSTONE Do we?

 

CATHY O’NEIL Let me give you another example. The U.S. News and World Report College Ranking Model. Who gives that power? Us.

 

BROOKE GLADSTONE You describe the impact of that ranking colleges turn themselves into pretzels. Students spend tons of money in order to fit the parameters that colleges have adapted to because of the rankings in U.S. News and World Report. A a pernicious feedback loop you call it. 

 

CATHY O’NEIL Yes. They’re bogus and their game-able. A college knows that if they look exclusive, then they look better for the ranking. So they just get a bunch of kids they know will never make it to apply. And yeah, of all the stupid things at the U.S. News and World Report pays attention to it doesn't pay attention to the costs. 

 

BROOKE GLADSTONE It's not one of the criteria. 

 

CATHY O’NEIL Exactly. When college admissions officers are crazily gaming the algorithm, which is what their job seems to be nowadays, they don't care if the tuition goes up. Why do we keep giving these questionable, stupid algorithms so much power?

 

BROOKE GLADSTONE If people listening to this interview only take one thing away. This is what I'd want them to take away. You say that an algorithm is an opinion embedded in math.

 

CATHY O’NEIL Right. I mean, there's so many choices that go into every algorithm. And the most important one being what do we mean by success? If I get to define success for myself, that's one thing. But if Facebook is defining success for me, I don't trust it. As algorithms proliferate, which they are applying for credit, applying for a job, applying to go to college, applying for a loan, applying for housing, all those things are now algorithmic. So they all define success for them, not for us. That's their opinion, right? It's really, really important to remember that it's not necessarily opinion that you have to share.

 

BROOKE GLADSTONE Thank you so much. 

 

CATHY O’NEIL Thank you so much, Brooke. 

 

BROOKE GLADSTONE Cathy O'Neil is a mathematician, data scientist and founder of the consulting firm ORCAA.

 

Coming up, I'm not shrill, I just sound that way. 

 

[CLIP]

 

HILLARY CLINTON And if we work together, if we…

 

NEWS REPORT We’ve heard the criticism before, that she sounds shrill, that she shouts. 

 

PRESIDENT TRUMP Who’s become very shrill. You know the word shrill? She's become shrill. [END CLIP]

 

TINA TALLON We see so much more of that now that we have more women running for president rather than talking about the content of their policy proposals, we're focused on the timbre of their voices.

 

BROOKE GLADSTONE This is On the Media.

 

This is On the Media. I'm Brooke Gladstone.

 

[CLIP]

 

FIONA HILL I hate to say it, but often when women show anger, it's not fully appreciated. It's often pushed on to emotional issues that perhaps are deflected onto other people. [END CLIP]

 

BROOKE GLADSTONE Former White House Russia expert Fiona Hill may have been angry, but she wasn't shrill. She had the accent and spoke from the diaphragm. Some of us don't know that can make moments behind the mike. A challenge not because of our voices, because of the mike and the frequency band. Women who appear in the public eye or rather in the public ear often are derided for the way they sound. But how they sound is also a product of how they're heard. And technology has a hand in that.

 

In a recent article for The New Yorker, Clark University professor and musician Tina Tallon noted that, “women who speak publicly and challenge authority have long been dismissed as shrill or grating.” We've often heard that women's voices are no good for the radio. Tallon says that's been the case since the invention of radio.

 

TINA TALLON Right. So there was a great article published in the radio broadcast magazine in 1924 in which one of their editors actually interviewed a number of station managers. It was a female editor, by the way, pretty revolutionary for the time. And so she basically went through all of these various complaints that people had. And they ranged from everything regarding audio quality, saying that female voices or people with higher voices sounded distorted and nasal and tinny to things about their personality and their senses of humor, saying that they're just not affable, they don't manage to really connect with listeners or they sound inauthentic and affected. All of these criticisms have been established for nearly a century now and have associated with women in the media.

 

BROOKE GLADSTONE All the things that were said about Hillary Clinton when she was running.

 

[CLIP]

 

HILLARY CLINTON And if we work together, if we…

 

NEWS REPORT We’ve heard the criticism before, that she sounds shrill, that she shouts, she doesn’t smile enough. 

 

NEWS REPORT Like a goose bumpy feel every time I hear Hillary's shrill voice rising and even higher and higher

 

PRESIDENT TRUMP And the Hillary who has become very shrill. You know the word shrill? She's become shrill. [END CLIP]

 

TINA TALLON We see so much more of that now that we have more women running for president rather than talking about the content of their policy. Proposals were focused on the timbre of their voices. 

 

BROOKE GLADSTONE And of course, voices are among the most intimate ways that we encounter people. A person's voice is like a person's face.

 

TINA TALLON The voice is the sight of so much when it comes to identity construction, not just for the person speaking, but also for the listener. And there are so many patterns that we learn about who makes certain sets of sounds and how. And so some of it is based upon the way that we observe the world. And some of it is also culturally conditioned in the way that we're looking for who is allowed to speak in what contexts. 

 

BROOKE GLADSTONE There was technology used for government purposes and all of that, but radio really began in this country in the mid 20s. That's where we need to peg the beginning of this tale. 

 

TINA TALLON Exactly. So in 1924, about 5 percent of households had radio receivers. By 1932, 60 percent of households had a receiver and they were now in place in cars as well. And of course, the government needed to step in and start to regulate because you ended up with frequent signal interference, especially in a lot of urban environments. And so finally, in 1927, the Federal Radio Commission went and decided to allot each station its own little 10000 hertz slice of bandwidth real estate. So there's a segment before you take the signal and modulate it into something that can be transferred in between stations known as the base band or the pre- modulated signal that had to be actually limited to 5000 hertz because amplitude modulation actually doubles the bandwidth of the signal. So initially they said, OK, we're gonna take all of our baseband signals and limit them to 5000 hertz. What that meant was all of the microphones and all of the equipment that people were using to record didn't need to go above 5000 hertz because none of that information would get transmitted.

 

BROOKE GLADSTONE Right. So what are we as human beings most attuned to? And then what were we able to hear on the radio? 

 

TINA TALLON Right. So there was some great research that happened in 1933 that basically looked at what are the frequencies that the human hearing apparatus is most sensitive to. And what they found is that we hear frequencies between about 1000 hertz and 7000 hertz more loudly than we hear frequencies at 100 hertz. 

 

BROOKE GLADSTONE So in real life, we hear higher voices, louder than lower voices. And on the radio, what happens? 

 

TINA TALLON What happened was there was a common thought at the time that women spoke more softly than men. A lot of this came from the result that the equipment that they were using to measure voices were optimized for lower voices by the people developing the technology who were exclusively men. In the early 1910s. And so they actually were missing a lot of the information at higher frequencies.

 

BROOKE GLADSTONE And so when you're not getting the same volume for women's voices, as you note in your article, you get less definition. The consonants are heard in the higher frequency ranges and you can't compensate by pumping up the volume for women's voices because we already perceive them as louder. The evolutionary theory goes that higher pitched voices were voices of warning or something like that.

 

[CRYING]

 

[SIRENS]

 

TINA TALLON And so because the microphones were not as good at picking up these high frequencies, it was all or nothing, right? You either don't pick it up at all or you pick up the maximum amount and it's distorted and has that harsh edge on it. A lot of engineers thought, well, women are quieter. As soon as a woman would sit behind the microphone, many of them would automatically crank the dial up, which meant that you were going to get those distortions in whatever higher ranges that the microphone could pick up. And this led to a lot of issues with women being called unintelligible because we just couldn't figure out what the words they were saying were. All we could hear were the vowels.

 

BROOKE GLADSTONE There's an engineer you quote in your piece who tested his technology with women's voices and came to an interesting conclusion.

 

TINA TALLON Yes, his name is R.C. Steinberg and he was a researcher at Bell Labs. He was actually one of the more outspoken engineers who would present his research publicly. And so the first mention of this disparity was actually in the 1927 Bell Laboratories record. And he wrote a little brief and it was entitled Understanding Women. 

 

BROOKE GLADSTONE Not their voices, just women.

 

TINA TALLON Exactly. Just, women. He quips that man's traditional inability to understand women may have a basis of fact if one so wishes to interpret certain recent experiments in our laboratories. And he talks about this disparity with confidence and the fact that people with higher voices tend to have consonants that lie between 5000 and 7000 hertz. The equipment at the time simply couldn't pick those up. And so there are actually these experiments where they figured out the “intelligibility percentages” at different cutoff levels and they found that women were significantly harder to understand when everything above 5000 hertz was cut off. But what's fascinating here is rather than advocating for changing the technology, he ends the article by saying nature has so designed women's speech that it is almost always most effective when it is of soft and well modulated tone. So here he is just saying, listen, ladies,

 

BROOKE GLADSTONE You're not built for the radio. 

 

TINA TALLON Exactly. Nature has just designed you this way. That's a bummer. Unfortunately, that stuck. There's no change and a lot of this technology and this government regulation in terms of the band limiting on these signals really, really influenced how manufacturers started making their equipment. And it stays that way. And so what that means is that if something isn't quite up to the quality that you're used to seeing, then we just throw away that material. Right? This audio doesn't quite sound as good. And so, therefore, I'm just not going to include this. Unfortunately, we either completely exclude people or we include them in a way that diminishes them. And so this lack of not only lack of representation overall, but lack of accurate or lack of quality representation is a huge problem. We're leaving out the stories of entire demographics, ignoring the contributions of people simply because our technology is limiting their their ability to exist in our mediums.

 

BROOKE GLADSTONE When I began work as an editor at NPR in the late 80s, I talked to a lot of women who were given kind of basic training before they went on the air. And this is NPR. They were told they sounded too young. They needed to lower their voices.

 

TINA TALLON Right. The first time I've seen it discussed was 1906. There actually was a telephone switchboard operators handbook where they talked about training the voice to become low and melodious and soft. And part of the reason, again, for that was because the microphones that were used to pick up sounds and telephone receivers were utter garbage. 

BROOKE GLADSTONE Right. 

 

TINA TALLON In the early nineteen hundreds, they were terrible. And so all of this kind of started very, very early on. 

 

BROOKE GLADSTONE You noted something across the ocean, the Maggie Thatcher effect. 

 

TINA TALLON Yes. So when she came to power, a lot of people dismissed Margaret Thatcher as a “shrill housewife.” And so between the 60s and the 80s, she actually worked with coaches to refine her vocal image. And she actually dropped her voice approximately 60 hertz. That's about half an octave in that range. And she was very transparent about this. Many scholars have written about this. Her biographer has written about this. And so the fact that she was able to become so successful, I think a lot of people started looking at that as an example.

 

[CLIP]

 

MARGARET THATCHER Very much so. I've done a good deal of other speaking, but speaking in the House of Commons is quite...

 

We shall have to make up our minds about the cabinet very quickly, because otherwise the press will discuss it all for me. [END CLIP] 

 

BROOKE GLADSTONE Elizabeth Holmes, the founder of Theranos, which purported to have a great way to test people's blood. It turned out to be a lot of hooey, the saga turns very dark, but there was a lot of commentary about her voice. 

 

TINA TALLON And this was one of the most bizarre of the deceptions involved in this whole saga.

 

[CLIP]

 

NEWS REPORT Her voice, 

 

ELIZABETH HOLMES what she could figure out, 

 

NEWS REPORT a surprising baritone, 

 

ELIZABETH HOLMES was that it would likely cost her a few thousand dollars to get these tests done.

 

NEWS REPORT In this interview with NPR from 2005, we hear a very different sounding Elizabeth.

 

ELIZABETH HOLMES No, it hasn't. Well, if I use traditional words to describe what we're doing, it's hard.

 

TINA TALLON That was actually raw audio from an interview that she did back in 2005. She had kind of let her guard down is the theory here. And so you suddenly hear something that's a little bit more natural. Of course, she realizes it and dips back into this deep baritone. But if you go and analyze this recording, there's actually a disparity of about 100 hertz, which is a huge amount. That's even more than Margaret Thatcher. That's something that surely doesn't just happen naturally.

 

BROOKE GLADSTONE Although her family did say she never changed her voice. 

 

TINA TALLON Exactly. And certainly there is a wide dynamic range that can occur as we get excited, you know, and we raise our voices to communicate enthusiasm or intensity. But what's very interesting, of course, is we're picking apart her voice and that that has no substance when it comes to the actual facts of the court case. Sure. It kind of fits into this narrative of deception. And so I think it is an interesting thing to criticize. And everything about her image was criticized from the black turtlenecks to the way she wore her hair. All of that became a part of the saga.

 

BROOKE GLADSTONE Yeah, which brings us back to the candidates. The word shrill certainly was often applied to Hillary Clinton last time around. Here, we hear Tucker Carlson doing precisely that in 2016.

 

[CLIP]

 

NEWS REPORT A bunch of men both running against her. And also in the pundit class who are saying things about her, that she's shrill, that she's shouting. 

 

TUCKER CARLSON Maybe she is shrill, maybe she is shouting, what does that have to do with her gender?

 

NEWS REPORT Because when was the last time you heard a man be accused of being shrill? That is such a code word. 

 

TUCKER CARLSON I don’t know, I think a lot of men are shrill. 

 

NEWS REPORT I don't think I've ever heard a man be called shrill.

 

NEWS REPORT I call Ted Cruz shrill.

 

TUCKER CARLSON Can I just call bs on this? No man on television has called Hillary Clinton shrill, because that is a felony offense and everybody knows it. [END CLIP]

 

TINA TALLON I haven't actually heard that. I think one of the one of the most hilarious parts of that clip is the fact that Tucker Carlson is yelling and the female host actually is maintaining a very level tone. And this is something we see time and time again. So many male candidates stand up there and literally yell, if a female candidate were to do the same thing, she would be viewed as hysterical, can't keep her temper in check. And so this is another aspect of this. There's a scholar who has coined the term “animatedness” about this, the idea that if you do nothing, you're viewed as doing something. And if you do something, it's viewed as a hyperbolic expression.

 

BROOKE GLADSTONE So what do you think? We're in a certainly a better spot than we were. There are a lot more women out there. I mean, have we finally gotten over the technological obstacles of the 1920s or maybe the social cultural ones?

 

TINA TALLON Well, I think we've certainly made a lot of progress both on the technical and the cultural fronts. But one of the things that we deal with a lot now is compression algorithms. Now that we have digital technology and streaming is actually one of the main modes of consumption now for any sort of audio or video. And what that means is that to reduce the amount of data and get it to your computer fast enough or get it to your mobile device fast enough, it has to be compressed. We have to get rid of some of the information. And a lot of these algorithms disproportionately affect higher frequencies. Again, similar to the old microphones and the old transmission modes of the 1920s, for some reason we haven't been able to transcend that, all based on a lot of the same not great research from the 1920s. So it's still haunts us to this day. Now, culturally, I think we're definitely getting to a point where we're more critically examining the ways that we talk about women and people with higher voices and people with different identities in the media. And we're examining how that influences the way that their stories are told, if they're told it all. I've noticed a lot of people on Twitter ironically or sarcastically calling men shrill, but it's still kind of feels like this very tongue in cheek type of application because there's still this underlying knowledge of the long and gendered history of this insult.

 

BROOKE GLADSTONE Do you think this figures into this campaign? 

 

TINA TALLON I'm not sure yet. We keep running up against this word likeability or electability. And I think that that has become the new shrill where there are gendered notions of who has what it takes to actually be in a position of authority. We're still not quite to the point where we're able to disentangle biases about gender from notions about who can lead and who is fit to lead.

 

BROOKE GLADSTONE Tina, thank you very much. 

 

TINA TALLON Thank you so much for having me. 

 

BROOKE GLADSTONE Tina Tallon is a composer, musician and professor at Clark University.

 

[MUSIC] 

That’s it for this week’s show. On the Media is produced by Alana Casanova-Burgess, Micah Loewinger, Leah Feder, John Hanrahan, and Asthaa Chaturvedi. We had more help from Charlotte Gartenberg and our show was edited by. Our technical director is Jennifer Munson, our engineer this week was Sam Bair. Katya Rogers is our executive producer, On the Media is a production of WNYC Studios. I'm Brooke Gladstone.