☰ SectionsBig-city news, all in one place.

AI-Generated TikTok Persona Recited a Real Creator's Exact Words in Alligator Alcatraz Rumor Video

Forensic experts say a realistic-looking TikTok speaker was created with artificial intelligence and delivered word for word the script of another creator's video, a tactic researchers describe as a new form of deception.

By Juan Gonzalez ·

A video that appears to show a person describing a rumor about incinerators at "Alligator Alcatraz" is fake, according to forensic media experts, even though every word in it came from a real TikTok creator. The rumor concerns a state-run immigration detention facility in the Florida Everglades and has no evidence behind it, yet versions of it have been viewed millions of times in recent days. One video pushing the claim drew nearly 20 million views.

The clip in question stands out because the speaker's face and voice appear to have been produced with artificial intelligence tools, according to two forensic media experts consulted by NPR, whose reporting KJZZ is drawing on. The script matches, down to the wording, a video posted days earlier by a different account. The copied version collected more than 200,000 views.

Researchers who study deepfakes, the AI-made images and videos convincing enough to pass as real, say the case points to a new way the technology is being used to mislead: having AI recite a real person's actual words, including their stumbles, "ums" and "uhs."

Ali Palmer, a 33-year-old Dallas creator who posts as @aliunfiltered_, had a video about a father who jumped off a Disney cruise ship to save his child reproduced this way. She said copycat accounts commonly repost entire videos, but having an AI-generated person speak her words was new to her. "It's upsetting. It feels like a violation of privacy," she said, adding that she has reported copying to TikTok without result.

Hany Farid, a digital forensics professor at the University of California, Berkeley, analyzed the incinerator video, the Disney cruise video and other posts from the same account at NPR's request and concluded they were AI products. He noted that identity theft by AI typically targets famous people or influencers, such as in crypto scams or bogus cancer cures, whereas here an average person's words were taken. Such content, he said, "would easily slip through the content moderation cracks."

Using AI to copy videos does not appear to violate TikTok's rules, though the platform requires users to label realistic AI-generated content. After NPR's story was published, labels indicating AI-generated media appeared on videos from the two accounts it identified. TikTok and most of the affected creators did not respond to requests for comment, and NPR could not determine who runs the accounts or why.

The two accounts share traits: each has about 10,000 followers and follows no one, and many videos feature Black personas that Farid described as "low-quality deepfakes." They have drawn on viral topics ranging from a face-lift in Guadalajara, Mexico, to a woman wearing a dog collar to the former Meghan Markle dancing in a hospital delivery room. Darren Linvill of Clemson University's Media Forensics Hub, who also reviewed the videos, said the persona's voice changes from one video to the next, which is a major giveaway.

Linvill studies state-backed propaganda from nations including China, Russia and Iran, and said AI personas such as fake news anchors are used in such operations. NPR found no sign these accounts belong to one. "Stealing content on social media is as old as social media," he said, but AI makes the process "cheaper, faster and easier."

Alex Hendricks, a 32-year-old retail worker in Montana, noticed two back-to-back videos on the rumor with identical monologues in different voices and posted a video pointing it out, which drew few views. He urged viewers to cross-reference what they see before sharing it, though he said he is unsure people will listen.

Source: KJZZ

More from Phoenix