Brooke Gladstone: This is On The Media, I'm Brooke Gladstone.

Micah Loewinger: I'm Micah Loewinger. A couple weeks back ahead of the Democratic primary in New Hampshire--

Male Speaker 6: An AI-generated call is falsely telling Democratic voters not to vote in tomorrow's primary. Here's part of that false AI-generated call.

AI Joe Biden: What a bunch of malarkey. Do you know the value of voting Democratic on our votes count? It's important that you save your vote for the November election.

Micah Loewinger: This bogus call, which reached as many as 25,000 phones across the state, prompted the Federal Communications Commission this week to outlaw such AI phone fakery. The episode highlights how effective and effortless these AI tricks have become, and how those charged with combating them are always one step behind. This is especially true at TikTok, where videos of conspiracy theories, really dumb conspiracy theories are reaching millions of eyeballs and generating serious money.

Abbie Richards is a misinformation researcher and a senior video producer at Media Matters, a left-leaning watchdog group. She's been studying the viral tactics behind this growing cottage industry.

Abbie Richards: You start off by saying something that is utterly unhinged.

AI: Government just captured a vampire and tried to keep it a secret. On October 10--

Abbie Richards: Then what you do is you create usually a fake main character, typically an explorer or a scientist.

AI: Alejandro Suarez was an explorer from West Palm Beach.

Abbie Richards: You describe the adventure through which they make this discovery.

AI: Alejandro walked for an hour in the woods until he reached a large rusted security fence.

Abbie Richards: Then it all turns out to be a coverup. At that point, the goal is to really just waste time and tell a long story because you want it to be over 60 seconds long.

Micah Loewinger: My favorite one that you identified in your piece is the quote-unquote, "Joe Rogan" clip of him talking about some scientist who overheard a conversation about an asteroid that's going to destroy planet Earth or something and the government doesn't want us to know about it.

AI Joe Rogan: We are all probably going to die in the next few years. Did you hear about this? There's this asteroid that is on a collision course with Earth. Pull it up, Jamie. Apparently this--

Abbie Richards: My favorite thing about the AI Joe Rogan conspiracy theories, they almost always start with a clip of him talking into the mic. They're not even trying to dub it, so they put the captions right over his mouth. [laughs]

Micah Loewinger: Part of the reason those videos work is that, yes, the content is really absurd but it's also something you could imagine Joe Rogan being like, "Oh my god, dude, I just read this crazy--" It works, you know?

Abbie Richards: Oh, it does. I saw one that was him saying that the US stayed in Iraq because they were looking for a Stargate. I was like, you know what, I could imagine him saying this.

[laughter]

Micah Loewinger: As you mentioned, the fact that these videos are over 60 seconds is important to the people who are trying to monetize the videos because it plays a role in TikTok's creativity program. Can you describe that?

Abbie Richards: You have to be in an eligible country. You have to be at least 18 years old. You have to have at least 10,000 followers and you have to have at least 100,000 video views in the last 30 days. Once you join the creativity program, the videos that you produce that are over 60 seconds long are eligible for monetization.

Micah Loewinger: Tell me a little bit about the kinds of accounts that are sharing these videos and how many views they're getting.

Abbie Richards: I found accounts that were getting 20 million, 30 million views on some of these videos. We identified these two accounts. One was English language and one was Spanish language, both of which were receiving millions of views. They appeared to be affiliated, they had the same name translated in English and Spanish and they have the same profile picture. The English language account had received over 342 million views since it began posting in February of last year. Then the Spanish language one had received over 329 million views and it only started posting in September. That's just one account in each language and it's doing really well posting this AI voice conspiracy theory content. This account in particular very obsessed with Megalodons, fun fact.

Micah Loewinger: When we say AI-generated, there are multiple generative AI tools that are being used on each of these videos.

Abbie Richards: Yes. It varies depending on the creator and the video itself. All of the videos that I'm pointing to here are definitely using an AI text-to-speech program. That's how we're getting an AI Joe Rogan. That's also how we get this voice that I'm sure everybody has heard. His name is Adam. He's in the 11 Lab Software. He's reading a lot of these.

AI: Micah Loewinger is a chill guy who loves hanging out.

Abbie Richards: Then on top of that there's often AI-generated images in the video because just listening to AI Joe Rogan wouldn't really be that interesting. Sometimes it's all AI-generated images. Sometimes they are mixed with just regular images in the Discord servers where they talk about how to make this content and they share tips. They often will recommend using AI to help you write the script or come up with the ideas.

Micah Loewinger: When you say discord server, you're referring to the kind of cottage industry that rests on top of the actual videos and channels themselves. There are, as you said, whole Discord servers, medium articles, YouTube channels, and these hustle bro guru influencers who claim that they can help other TikTokers make it big. What are they preaching and what are they hawking?

Abbie Richards: It seems like a pretty classic get-rich scheme to me. They're offering courses or coaching one-on-one advice and feedback on your content, teaching you how to essentially create content that will go as viral as possible that you can monetize and then make money off of.

Micah Loewinger: You actually hung out in some of the Discord servers. What did you find?

Abbie Richards: They're talking about how to essentially make more money. One person said, for example, if it's a conspiracy channel post podcast clips about how they're poisoning the food supply and then link an affiliate product that is meant to detoxify the body. I'm like, I love when they just say what they're doing. It makes my job easy.

Micah Loewinger: They just spell it out. Putting aside the fact that a lot of these videos fall into this dumb occult genre of like vampires and like wendigos and these kinds of things, what are the major tells that some of these videos are AI-generated?

Abbie Richards: The voice is the first giveaway often, but then there's small details that are just wrong, like the wrong number of fingers or asymmetry distortion. If there's ever any text in the image, it's usually not any language that we've ever seen before. AI is still pretty bad at language. Also, they have a certain look to them. Like when you look at art and it makes you feel nothing.

Micah Loewinger: Yes, that's this. If it's so obviously fake, a lot of the people sharing them probably think that they're funny or they just think it's a captivating story and let's just be charitable here and assume that a lot of people are not convinced, then what's the harm?

Abbie Richards: The harm is that we are essentially pushing out content that teaches people to think about the world in a way that's really broken. It's a really unhelpful framework for understanding the world. Even if they know that the AI is AI, we still have a problem with viral conspiracy theories. It pulls us away from understanding how our world actually functions. I'm less concerned about what comes off as real and more concerned about just how easy it is to make this misinformation at scale

Micah Loewinger: And make money from it.

Abbie Richards: Yes. It's super profitable and you can just pump it out. The people that are making this content, a lot of them probably aren't even really deep believers in conspiracy theories. They're just following the money. We need to make sure that pumping out conspiracy theory content just isn't profitable.

Micah Loewinger: The 2024 presidential election is approaching fast and researchers have been voicing concerns over AI-generated misinformation and disinformation. There was of course that AI-generated Biden robocall. The FCC has just been granted the power to start pursuing legal actions against people who might be creating this stuff. On TikTok, how much political AI-generated content are you seeing? Have users been digging into this particular niche as a potential business model as well?

Abbie Richards: The type of people who make content about a dragon being discovered in Antarctica, they probably aren't as interested in niche political conspiracy theories because that's much more likely to be demonetized and it has a smaller audience. They're really going for just scraping as many people as possible. That's not to say I haven't seen a lot of political ones. Did you see the Biden tap water one?

Micah Loewinger: No.

AI: Joe Biden controls you through the water you drink. Yes, you heard that right.

Abbie Richards: It uses an AI-generated image of Joe Biden over a sink.

Micah Loewinger: If I drink Joe Biden's water, then he gets to control my actions or something? [laughs]

Abbie Richards: I think so. Honestly didn't follow the plot that much. They have laid out an entire framework and provided a vast amount of resources and YouTube instructional videos on how to make this sort of content that goes as viral as possible.

Micah Loewinger: You're saying the infrastructure that these people have created, the educational materials, the Discord servers, the how-to guides all over the place could be used by anyone for anything?

Abbie Richards: Yes. That is concerning when we pair that with an electorate that's already primed for lots of conspiracy theories and then we're mixing that with AI that can just create this content at a scale that like we've never seen before.

Micah Loewinger: You said that election misinformation has been a problem on TikTok in the past. Do you think the platform has learned any lessons and is equipped to moderate itself this time around?

Abbie Richards: Maybe they've learned some lessons, but I don't think that any platform should be walking into this election thinking that they're safe and that they have all their bases covered.

Micah Loewinger: Abbie, thank you very much.

Abbie Richards: Thank you so much for having me.

Micah Loewinger: Abbie Richards is a video producer at Media Matters. Her latest piece is titled TikTok Has an AI Conspiracy Theory Problem.

AI: Numerous studies have concluded that [unintelligible 00:49:45] actually causes depression and disrupts other harmonies in people's bodies, making them more susceptibly to suggestion and thus easier to manipulate.

Micah Loewinger: That's it for this week's show, On the Media is produced by Eloise Blondiau, Molly Rosen, Rebecca Clark-Callender, and Candice Wang with help from Shaan Merchant.

Brooke Gladstone: Our technical directors, Jennifer Munson, our engineers this week were Andrew Nerviano and Brendan Dalton. Katya Rogers is our executive producer. On the Media is a production of WNYC Studios. I'm Brooke Gladstone.

Micah Loewinger: I'm Micah Loewinger.

[music]

Copyright © 2024 New York Public Radio. All rights reserved. Visit our website terms of use at www.wnyc.org for further information.

New York Public Radio transcripts are created on a rush deadline, often by contractors. This text may not be in its final form and may be updated or revised in the future. Accuracy and availability may vary. The authoritative record of New York Public Radio’s programming is the audio record.