Micah Loewinger: This is On the Media. I'm Micah Loewinger. In early February, The Guardian ran a story about CNN staffers, who say that the news outlets internal editorial policies around the conflict in Israel and Palestine amount to "journalistic malpractice." Six staffers spoke anonymously and shared dozens of internal memos and emails, which showed that--

Staffer: Daily news decisions are shaped by directives from the CNN headquarters in Atlanta, setting strict guidelines on coverage. Additionally, every story on the war must be green-lit by their Jerusalem bureau before it's published or makes air.

Micah Loewinger: The staffers claim that the very guardrails that CNN put up around reporting on this conflict to protect against perceptions of bias are actually perpetuating a pro-Israel bias. It's not just CNN. There are calls of bias from all quarters.

Female Speaker 4: There's a dehumanization, vilification, and demonization of the state of Israel, that is manifested in media bias against Israel.

Activist: The New York Times is manufacturing consent for genocide, and we're here to say, "Shame."

Crowd: Shame.

Jonathan Greenblatt: I love this show and I love this network, but I've got to ask, "Who is writing the scripts? Hamas?"

Micah Loewinger: In the first week after October 7th, the BBC received over 1,500 audience complaints about its coverage. Those complaints were evenly split between those who felt the coverage was biased against Israel and biased against Palestine, according to The Guardian. There is no shortage of outrage and critiques. What's lacking is a ton of credible data sets. There are some, but they're just snapshots of the media universe. It's difficult to get a complete picture of bias because the reporting is ongoing. It's happening as we talk about it.

We decided to look at a couple of the available studies. I began by speaking to William Youmans, a professor of Media and Public Affairs at George Washington University. Youmans reviewed 51 hours of television from NBC's Meet the Press, CBS's Face the Nation, ABC's This Week, and Fox News's Sunday, between October 8th and January 14th. He says he plans to submit his study for peer review, but decided the need for his findings was too urgent to wait upwards of a year for an academic journal to share it with the public.

William Youmans: Part of why I rushed to publish these findings is because I saw so much chatter about bias that's very impressionistic, anecdotal, one-offs, but I wanted to conduct a systematic study.

Micah Loewinger: He wanted to understand what kind of people these shows were featuring to discuss the war.

William Youmans: Were they mostly pro-Palestinian, neutral, or pro-Israel? What was the framing that they used to talk about this issue? I wanted to see if the balance of framing was related to the balance of who gets invited as guests on these programs.

Micah Loewinger: What did you find?

William Youmans: I found that overwhelmingly, most of the guests were American, 120 out of 140. Of those guests, not one of them was Arab or Palestinian-American. The driver of the pro-Israeli arguments in these shows was US officials.

Martha Raddatz: For the latest on the conflict, we're joined now by Secretary of State, Antony Blinken.

Presenter 1: We go now to White House National Security Advisor, Jake Sullivan.

Presenter 2: Now, Jon Finer, Deputy National Security Advisor with the Biden administration.

Presenter 3: Joining me now is John Kirby, National Security Council Coordinator for Strategic Communications. John, welcome.

William Youmans: Well, who were the 20 that were not American in terms of guests? Of those, half of them were Israelis, officials for the most part, and some of these were repeat guests, people from the Netanyahu administration or cabinet, or official spokespeople.

Micah Loewinger: Like Israeli diplomat, Michael Herzog.

Michael Herzog: We are not after innocent civilians. We are after the terrorists who've been hiding behind civilians.

William Youmans: I was surprised to find out that there was only one guest who was Palestinian, Husam Zomlot, who is the PA's representative to the UK.

Husam Zomlot: This is not A war against Hamas. It's clear since it started, it's a war against our people.

Micah Loewinger: He found that the guests were 4.6 times more likely to be sympathetic to Israel than Palestine. I asked him, "How do you measure something like that?"

William Youmans: Well, basically I use a five-point scale. That's acknowledging that someone could be pro-Israeli unconditionally. That means everything that they were saying was considered pro-Israeli. That would be something like, "Israel has a right to self-defense," or, "Israel's mission is to destroy Hamas." These are comments that are really aligned with what the Israeli government's saying. Something that would be very pro-Palestinian would say, "Palestinians are resisting against the military occupation." I would code that as very pro-Palestinian. Then there's neutral statements that are just descriptions of fact, that aren't evaluative statements that are meant to be more sympathetic to one side or the other.

Micah Loewinger: I'm sure there's some listeners who heard, "Israel's mission is to destroy Hamas," and they would say, "That's a fact."

William Youmans: Who is mostly getting killed in Gaza are Palestinians who are not members of Hamas. To assume that this is a correct framing and a factual framing embeds a certain degree of bias that reflects the Israeli government's talking points. Now, to be honest with you, I know that there would be concerns about this particular part of the study. Aware of that, I was very conservative in that I over-counted, if anything, what would be considered pro-Palestinian discourse.

For example, if they mentioned how many Palestinian casualties there were, I counted that as being pro-Palestinian, in sympathy, even though that would arguably be neutral. The reason why I coded it as that is because there is some debate about the statistics that have been given. There's been a demand on the Israeli side to identify the source is Hamas-run Gaza Health Ministry, even though the State Department relies on these statistics as well.

Micah Loewinger: Using the schema that you've described, you did an analysis of seven key words, occupation, hostages, Israel, Palestine or Palestinians, Hamas, ceasefire, and genocide. What did you find?

William Youmans: One key word, especially in the Palestinian narrative and the Palestinian explanation of what's happening, is the word occupation, in reference to Israel's military occupation, which began in 1967. A lot of pro-Israel advocates denied that Gaza was still under Israeli occupation.

Micah Loewinger: The Israeli government says that the occupation in Gaza ended in 2005 when Israel withdrew its troops. This week The Hague finished hearing testimony on this very subject, and will offer its opinion in due course. Though many organizations, including the Red Cross and the European Union say Gaza is still an occupied territory. Youmans agrees.

William Youmans: Israel still controlled what goes in and outside of Gaza, enforces through military means the defenses, and also doesn't let Palestinian fishermen go out into the water beyond a certain distance from the shoreline, and the airspace is completely controlled by Israel. Very effectively, it's a territory that does not have complete control over itself.

The word "occupation," which would be a key frame for understanding this was mentioned only 15 times in about 51 hours of television that I analyzed. This can only happen if you don't have any Palestinian guests on. There's a direct connection between who's booked as guests and what gets talked about.

I was also interested in the difference of distribution between talking about hostages versus talking about prisoners. The groups that took hostages explicitly said that they were taking hostages in order to have the prisoners released. There are thousands of Palestinian prisoners under Israeli custody. A lot of them aren't charged. They're just held in detention without any formal legal procedure or process. This is something that's been pointed out by human rights organizations.

Then a good number of those Palestinian prisoners were under the age of 18. Some of them are being held with charges. Some of them did commit crimes and were subject to legal process. Of course, those were mostly in military courts, which have a lot of fundamental problems with them in terms of rights to defendants, but Palestinians have used hostage-taking in order to try to secure freedom for Palestinian prisoners. What we heard though was talking about hostages on these shows in isolation, hostages were talked about 529 times but Palestinian prisoners were only mentioned 17 times.

Micah Loewinger: Youmans says that of the 51 segments he studied, the term ceasefire came up 94 times, and genocide was mentioned 23 times. He says only one of those times was it in reference to Gaza.

William Youmans: 20 of those times, it was guests talking about protest slogans for Palestinian freedom being equivalent to calls for Jewish genocide. In reference to "From the river to the sea, Palestine will be free," or "Free Palestine," guests on these programs equated those to calls for genocide.

Micah Loewinger: Youmans notes that his study didn't span the entire first news cycle of the International Court of Justice hearings when coverage focused on the question of genocide became more common.

William Youmans: Now ICJ preliminary rulings came out more on the tail end of my study, to be fair, but there were certainly claims circulating in the public sphere that there might be an unfolding genocide in Gaza. Yet it was completely absent from the discussion on these programs.

Micah Loewinger: Youmans sent me a link to a 2012 book by Shraga Simmons, co-founder of HonestReporting, a pro-Israel media watchdog group. The book titled David and Goliath, alleges that liberal American media have used an underdog narrative to highlight the plight of Palestinians and vilify Israelis, which speaks to one of the major challenges of studying bias in media, especially on this topic, what's known as the hostile media effect.

William Youmans: Hostile media effect is basically a theory about perceptions of bias and it says that those who perceive bias are themselves biased in their own way and that affects how they perceive bias.

Micah Loewinger: The term comes from a 1985 study in which pro-Israel and pro-Arab students at Stanford were asked to review news stories about the conflict. After watching the same news coverage, both groups of students said that the coverage was biased against their side. Youmans says by studying media coverage in the aggregate and encouraging other scholars to scrutinize his research, he thinks it's possible to overcome the hostile media effect.

William Youmans: That's how you protect against your bias shaping research too much but I would also point out that there is still an underlying reality and sometimes the underlying reality is itself in a way biased against the perspectives of one group or the other.

Micah Loewinger: What do you say to listeners right now who might be thinking that your study is, in fact, anti-Israel?

William Youmans: It's not anti-Israel to say that there needs to be more perspectives than just the Israeli perspective. To me, this is about are the US media doing what's journalistically appropriate in presenting a diverse array of viewpoints or are they overwhelmingly presenting one set of standpoints that would be more sympathetic to Israel. I found, unfortunately, overwhelming evidence for the latter.

Micah Loewinger: William, thank you very much.

William Youmans: Thank you.

Micah Loewinger: William Youmans is an associate professor of Media and Public Affairs at George Washington University. So far, we've heard about the Sunday morning talk shows. According to data journalist and illustrator Mona Chalabi, the language and framing used in straight news reports likely plays the greatest role in shaping impressions of the war.

Male Speaker 7: The Pulitzer Prize for illustrative reporting and commentary, Mona Chalabi, contributor, The New York Times.

Micah Loewinger: When Chalabi took the stage on October 19th, to accept her Pulitzer Prize, she called out other journalists who had been referring that evening to the Israel-Gaza war.

Mona Chalabi: I don't think it's the Israel-Gaza war. I think it's the Israel-Palestine war. No one in this room is willing to mention the P word and I think it's really important. I'm sorry, I know I'm not supposed to make speeches here but it felt important.

Micah Loewinger: How did they respond when you said that?

Mona Chalabi: It was awkward. It was really awkward. Why isn't this being framed as the Israel-Palestine war? To do so, would be to use the word Palestine, which is not recognized in journalistic style books as a legitimate word for journalists to use. That in itself is a deeply bias choice.

Micah Loewinger: Mere hours before the Pulitzer ceremony, she posted a chart on Instagram where she now has half a million followers, an illustration depicting mentions of Israeli and Palestinian deaths in The New York Times. The data came from Holly Jackson, a computer science PhD at UC Berkeley. You created visualizations of two main trends from the Holly Jackson data. The first is a graph showing that The New York Times consistently mentioned Israeli deaths more often than Palestinian deaths from October 7th to the 18th, even as more Palestinians were dying.

In the graphs, it's striking. You can see the mentions of Israeli and Palestinian deaths following similar trajectories. We see a trend of when there's an article talking about deaths in the war. The articles appear to be at the same time mentioning Israeli deaths and Palestinian deaths, even as Palestinian deaths way outpaced Israeli deaths. Can you just tell me a little bit about how you approached illustrating this data?

Mona Chalabi: The issue here is disproportionality. To cite the fact that Israeli deaths are mentioned more often than Palestinian deaths isn't inherently in itself problematic. It's problematic because Palestinian deaths are occurring at a far higher rate and pace right now than Israeli deaths. The chart was trying to show both things at once, which is relatively complex. I was actually worried when I would publish this, that it might not fully resonate.

It's two-line charts. It's both boring and confusing. I don't normally use these kind of styles of visual depiction but it felt also really important to me to not use my hand-drawn methods that I use elsewhere that imply sometimes, in precision, and they play in precision deliberately. But actually, counterintuitively, this research is much more robust than, for example, I don't know I'm thinking of another piece that I'm working on right now, which is just surveying how many Americans take acid or take hallucinogens. That polling data has a high degree of imprecision. The research that Holly Jackson has done is undeniable. She has literally taken all of the articles that appear on The New York Times website and counted all of these mentions. It felt really important to me to actually rely on more traditional methods of data visualization.

Micah Loewinger: She says she pitched this analysis to The New York Times and was rejected. In late December, she illustrated more research, this time from Dana Najjar and Jan Lietava, who scraped language from 600 articles and 4,000 posts on the BBC website.

Mona Chalabi: In BBC reporting, Israelis were far more likely to be described as murdered, massacred, slaughtered than Palestinians. There were 23 times that Israelis were described as being massacred and only occurred once for Palestinians. The term slaughtered was used 20 times to describe Israeli deaths and zero times to describe Palestinian deaths.

Micah Loewinger: The Intercept found a similar pattern in its review of The New York Times, Washington Post, and LA Times. "The term slaughter was used by editors and reporters to describe the killing of Israelis versus Palestinians, 60 to 1, and massacre was used to describe the killing of Israelis versus Palestinians, 125 to 2. Horrific was used to describe the killing of Israelis versus Palestinians 36 to 4."

Mona Chalabi: Similarly, by the way, when BBC reporting, Israelis were more likely to be humanized in the reporting. Israelis would be described as mothers, grandmothers, daughters, fathers, husbands, sons.

Micah Loewinger: I'm looking at your post here. The term mother or grandmother was used 51 times to 32 times for Israelis versus Palestinians. Daughter and granddaughter was used 35 times to describe Israelis, 15 for Palestinians. The term father and grandfather was used 33 times for Israelis, 9 for Palestinians. Husband 30 times to 5 times for Palestinians. Son/grandson was used 25 times to 11 times for Palestinians.

Mona Chalabi: Those numbers might not sound that high, which was my first instinct when hearing them but when you take into account that by the end of December, maybe 20 times more Palestinians had been killed than Israelis and yet the numbers are still lower for the Palestinian side to use these descriptors that basically lend humanity to somebody who has died.

The nature of the reporting on this has also been that Israel has stopped journalists from entering Gaza to be able to report on this. There is fundamentally an informational asymmetry that is baked in that replicates and mirrors these patterns of bias that aren't necessarily about the nefarious intent of the person that has the byline. It's also that they have access to a different degree of information in reporting on these deaths.

Micah Loewinger: CNN's Clarissa Ward is one of the only reporters who was granted brief access to Gaza without an IDF escort. She wrote a Washington Post op-ed in January calling on Israel to allow other journalists in.

Clarissa Ward: This was our first time being able to gain access into Gaza.

Micah Loewinger: Clarissa Ward on CNN.

Clarissa Ward: You have a perfect storm here with massive bombardment, an inability to create safe zones, an inability to get humanitarian access where it's needed, and incredibly brave journalists who are doing everything they can to tell the stories and bring the reality to the world but the frustration of international journalists who can't get in to try to complement and supplement their efforts.

Micah Loewinger: This week, more than 55 foreign correspondents signed a letter asking for Israel and Egypt to lift their restrictions and allow journalists into Gaza. When asked about [unintelligible 00:30:10] peace on Israel's Channel 13, a reporter said this.

Male Speaker 8: [foreign language]

Micah Loewinger: If indeed Western reporters begin to enter Gaza, then for sure, this will be a big headache for Israel and Israeli Hasbara. Hasbara, by the way, means explanation in Hebrew, but in practice, it's shorthand for pro-Israel propaganda.

Micah Loewinger: All this at a time when Palestinian journalists in Gaza are being killed at an alarming rate, Mona Chalabi told me she stands behind her decision to criticize the coverage she's seeing in mainstream outlets, though it's come at a professional cost.

Mona Chalabi: Yes. I haven't been commissioned once since October except to write a single opinion piece, which was canceled after I refused to write from the perspective of being an Arab Muslim woman. That is not the perspective that I'm doing this journalism from. I'm doing this journalism from a perspective, which is my perspective on every single article that I write, which is how can I report on this accurately, honestly, and fairly.

Micah Loewinger: Mona, thank you very much.

Mona Chalabi: Thank you.

Micah Loewinger: Mona Chalabi is a data journalist and illustrator. We requested comment from The New York Times, The Washington Post, The Wall Street Journal, the LA Times, the BBC, NBC, CBS, ABC, and Fox. Most did not respond. A New York Times spokesperson said that, "We have documented the death toll. The Israeli response has visited on the civilians of Gaza. Our coverage bears witness to the unfolding atrocities and brings readers firsthand accounts from Palestinians and Israelis."

A Washington Post spokesperson responded that, "The Washington Post has consistently published balanced and sophisticated coverage of the complex Middle East War over the past months, in both its news and opinion reporting." A BBC spokesperson said that, "The BBC has made clear the devastating human cost to civilians living in Gaza and Israel. We do not think coverage can be assessed solely by counting particular words used and do not believe this analysis demonstrates bias." For the full statements, head to onthemedia.org.

Coming up, journalist Mehdi Hasan on how to change up the guest list on Cable News. This is On The Media.