BROOKE GLADSTONE:  Over the last decade, many courts also have started using algorithm-based predictive tools to determine how long or even whether a convict should be sentenced to jail. Julia Angwin, a senior reporter at ProPublica, scrutinized one of the most used risk assessment tools, Northpointe. A self-described math geek, she was seeking some high-stakes numbers to explore.

JULIA ANGWIN:  What I realized was the highest stakes for an algorithm was this one that was basically used to determine whether you were risky enough to be set free.

BROOKE GLADSTONE:  So why Northpointe's algorithm, specifically?

JULIA ANGWIN:  Yeah, so when I started looking into risk assessment scores, the first thing I realized was there were dozens of them. States build their own, academics build their own. Some of them are open source. There were two sort of national commercial vendors, Northpointe and another one, Multi-Health Systems. For the story to be national in scope, I wanted to pick one of those two, and so I chose Northpointe.

BROOKE GLADSTONE:  How does it work?

JULIA ANGWIN:  The way risk assessments work is you’re arrested and they ask you a whole bunch of questions. Some of them are about your criminal record, your work history, your employment status, your family - have they been in jail – what kind of neighborhood do you live in? A lot of these reflect theories of criminality, which is that if you live in an unstable environment, you're more likely to commit crime.

BROOKE GLADSTONE:  They do not ask about race.

JULIA ANGWIN:  No, they haven't asked about race since, I think, about the ‘70s. There were early risk-assessment tools that did ask about race, but that became very uncool [LAUGHS] during the civil rights era.

BROOKE GLADSTONE:  Aha. They do ask some sort of intriguing questions like, agree or disagree with this statement: A hungry person has a right to steal.

JULIA ANGWIN:  Yeah, the one that you’re reading from is the Northpointe Criminal Thinking Score. The thinking behind that [LAUGHS] was the idea that the judge might choose, if you have a high criminal thinking score, to offer you some mental health counseling.

BROOKE GLADSTONE:  Mm-hmm.

JULIA ANGWIN:  There's an idea behind the risk assessment movement that these are going to be used for treatment.

BROOKE GLADSTONE:  A hungry person has a right to steal, agree or disagree. I don’t know whether that suggests you have a criminal mind or, or that you're just a liberal.

JULIA ANGWIN:  Correct. You could definitely see it that way.

  [BROOKE LAUGHS]

Also, my question is, let’s say I'm sitting in a cell and somebody [LAUGHS] comes to me with a little clipboard and they say, agree or disagree, a hungry person has a right to steal. Perhaps it's an IQ test because [LAUGHS] I think the answer you should give is no.

BROOKE GLADSTONE:  Okay, now you found in your study, which you made of Broward County, Florida, that they were equally inaccurate, whether you're talking about a white defendant or a black one.

JULIA ANGWIN:  That's right. They were right about 60 percent of the time, for both blacks and whites, when it came to predicting whether they would commit another crime during the next two years. However, they were twice as likely to misclassify a black defendant as high risk when they weren't, and they were twice as likely to misclassify a white defendant as low risk when they weren't.

BROOKE GLADSTONE:  I mean, obviously, anecdotes are just anecdotes, but you offer a couple of them to sort of illustrate your findings, which are kind of mind blowing. You want to share one?

JULIA ANGWIN:  Sure. We chose two people who had a very similar crime, which was petty theft.

BROOKE GLADSTONE:  They’d both stolen about 80 bucks’ worth of stuff.

JULIA ANGWIN:  Yes. So the girl, a young 18-year-old black girl, she had picked up a kid’s Huffy bicycle and Razor scooter, she and her friend, out of somebody's yard, tried to ride them, realized that they were too big, gave them back to the mother who had come to ask for them back. But the neighbor had called the cops, so she was arrested and she was given a high-risk score, 10 out of 10, the highest possible risk. We compared her case with this guy who was a white man who was older. He had shoplifted about $80 worth of stuff, and he had already served a five-year term for armed robbery. He had another armed robbery under his belt. And he was given a score of 3 out of 10, low risk.

So we followed what happened in the two years after they were scored. He was arrested for stealing thousands of dollars’ worth of electronics from a warehouse and he’s serving a nine-year sentence right now in state prison. And she has not been arrested at all.

BROOKE GLADSTONE:  And you found that only 20 percent of people predicted to commit violent crimes actually went on to do so.

JULIA ANGWIN:  Nobody would say that's a, a good result. In fact, there was a time when psychologists were brought in to assess future violence, and they only had a success rate of about 53 percent. And so, actually, that was used to justify not having psychologists do this. So the idea that this is a measure that has replaced that, that is only 20 percent accurate, is shocking.

BROOKE GLADSTONE:  But the driving force behind these risk assessments wasn’t to help in sentencing, right?

JULIA ANGWIN:  Right. The people who create them talk about how they really want to identify the issues that people who are entering the criminal justice system face and allow them to get treatment for those issues. The Northpointe is a very good example of that. That founder said that's what he came up with it for. But it is used in sentencing throughout Wisconsin and other states, much of upstate New York. In Wisconsin, the Northpointe score was actually challenged in a case recently, where the defendant said it wasn't fair that it was used in his sentencing. And the founder of Northpointe testified that he had not intended it to be used for sentencing.

BROOKE GLADSTONE:  And it isn't just Northpointe. There are lots of states that are using all kinds of risk assessments in sentencing, including Arizona, Louisiana, Washington. You cite the case of Paul Zilly. The first time he knew he had a score was when he was in court and [LAUGHS] he was about to be sentenced. That was back in 2013 in Barron County, Wisconsin.

He'd been accused of stealing a push lawnmower and some tools.

JULIA ANGWIN:  Yes. The prosecutor and his attorney had actually come to a plea deal, which would have had no prison time. And they presented it to the judge. The judge said, you know, I'm looking at the risk score here that shows him at very high risk of being a violent criminal, and so I'm gonna overturn your plea deal and actually give him two years in prison. That is when Paul Zilly learned that he had this score and that it was gonna be such an impact on his life. So from prison, he appealed his score, attempted to get it overturned. Eventually, he got it reduced from two years to a year and a half. But it’s worth pointing out that he wouldn't have served any prison time if the judge had not overruled the plea deal, to begin with, based on these scores.

BROOKE GLADSTONE:  In fairness though, these scores sometimes do work as originally intended, to direct defendants to programs that could help them and to keep them out of jail.

JULIA ANGWIN:  Yeah, Virginia and other states use risk scores to justify diverting people from prison to treatment programs, and that's proven really successful at reducing prison populations without increasing crime.

The question I really have is, if this score is only 60 percent correct under the best circumstances and 20 percent correct when you talk about violent crime, is that the right way to choose who gets that one spot in the treatment program? That's my question. [LAUGHS]

BROOKE GLADSTONE:  But how do you do it, because these algorithms were supposed to be designed to bypass our personal biases in making those judgments? How do you do that kind of triage?

JULIA ANGWIN:  Right. I just think that if the judge got the score with a little thing saying, this thing is only 60 percent accurate, and the defendant had a chance to challenge it with independent tests and everyone felt like they’d had a full hearing - what I'm describing is just what we call due process –

  [BROOKE LAUGHS]

- I think that I wouldn't have any problem with it being used. But that’s what's happening, right?

What needs to be done is what we did, which is independent assessment which validates it. The problem is that no one's doing that. New York State - they were the first buyer of Northpointe score – they then went and did their own validation study. They didn’t believe their study until 12 years after they had started using the score. And when they did it, they didn’t test race or the violent score. The only independent validations that occur are by the buyers of the score after they've implemented it, because it takes years.

But here’s what I would say:  That’s where you get into this question of accuracy. So imagine if you just said, look, we are a racist system. Policing is racist, prosecution is racist, sentencing is racist. So then you have to start making moral judgments. You know, it's not all math. That’s how I view it.

BROOKE GLADSTONE:  Yeah.

JULIA ANGWIN:  You know, math can only take you so far. Eventually, math is really just a cover story for morality.

BROOKE GLADSTONE:  Julia, thank you so much.

JULIA ANGWIN:  Thank you.

BROOKE GLADSTONE:  Julia Angwin is a senior reporter at ProPublica.

  [MUSIC/MUSIC UP & UNDER]

DEBORAH AMOS:  That’s it for this week’s show. On the Media is produced by Meara Sharma, Alana Casanova-Burgess, Jesse Brenneman and Dasha Lisitsina. We had more help from Emma Stelter, Isabel Cristo and Micah Loewinger. And our show was edited by Brooke. Our technical director is Jennifer Munson. Our engineer this week was Casey Holford.

BROOKE GLADSTONE:  Katya Rogers is our executive producer. Jim Schachter is WNYC's vice president for news. On the Media is a production of WNYC Studios. Bob Garfield will be back next week. I’m Brooke Gladstone.

DEBORAH AMOS:  And I’m Deb Amos.

BROOKE GLADSTONE:  So Deb, did you see Game of Thrones last week?

DEBORAH AMOS:  Yes! But you know what? The Brexit vote could mess everything up for the show.

BROOKE GLADSTONE:  You are such a news junkie. How do you make that connection?

DEBORAH AMOS:  So it’s the EU that funded the production in Northern Ireland. It’s where the last battle scene happens. It took them 25 days to shoot it. That’s paid for by the EU. With Britain out of the European Union, they could lose their funding! And Winterfell could fall.

BROOKE GLADSTONE:  [LAUGHS] Not again! Thanks, Deb.

DEBORAH AMOS:  Thanks, Brooke.

 [GAME OF THRONES CLIP]:

RAZDAHL MO ERAZ:  I imagine it's difficult adjusting to the new reality. Your reign is over.

DAENERYS TARGARYEN:  My reign has just begun.

  [END CLIP]

  [FUNDING CREDITS]