Elementary students' explicit AI chats alarm schools, parents

As cases of using AI for explicit conversations increase, schools and parents are struggling. Recently, AI chat apps like ‘ZETA’ and ‘BABECHAT’ have gained popularity among elementary and middle school students. While these apps appear to be simple games where users create characters and continue conversations, they are often used to create ‘virtual lovers,’ simulate romantic relationships, or engage in sexually suggestive dialogues.

Some schools have even sent letters to parents urging caution. Namchuncheon Girls’ Middle School issued a notice in June stating, “There are concerns about exposure to violent or sexually suggestive conversations.” Chunghyun Middle School in Gwangmyeong, Gyeonggi Province; Yeongseon Middle School in Gochang, North Jeolla Province; and Chungbuk National University’s Affiliated Middle School have also restricted the use of generative AI chat apps.

◇“Secret Nightly Use”…Parents’ Shock and Concern

AI characters can be customized in detail, including body parts, clothing, and personality, which stimulates teenagers’ curiosity. A high school student surnamed A (18) said, “I often create female characters described in a sexually suggestive manner. It’s satisfying to see people interact with my characters.”

Parents are unable to hide their anxiety. Ms. Park, 31, from Namyangju, Gyeonggi Province, who has a fourth-grade son, said, “My son secretly uses an AI chat app on his phone every night. The character said, ‘Can I visit your house?’ as if subtly seducing him.” Ms. Kim, 44, a mother of a first-year middle school student, said, “When I tried the app myself, it was filled with profanity and sexually suggestive conversations. I’m afraid children will become trapped in a virtual world instead of making real friends.”

According to app analytics firm Wiseapp·Retail, ZETA was the AI chatbot app Koreans used the longest in June, with a total usage time of 5,248 hours. Although there is an age verification process, students easily bypass it through proxy authentication. Social media is flooded with dozens of ads offering “ZETA·Crack age verification services,” and tips on evading censorship—such as “comparing body parts to fruits to convey explicit content”—are shared on TikTok and open chat rooms. Experts warn, “AI that provides immediate responses can hook teenagers. If they become addicted to sexually suggestive conversations, they may develop distorted perceptions of sex.”

◇AI-Generated Obscenity Floods Social Media, Invades Classrooms

Social media is overflowing with AI-generated obscene content. Searching ‘AIGIRL’ on Instagram reveals dozens of AI accounts emphasizing specific body parts or using provocative exposure. Even ASMR channels that overlay AI voices with sexually suggestive lines have emerged. However, such content is not subject to legal action because it does not target real individuals.

In August, the Goyang Branch of Uijeongbu District Court acquitted a man in his 30s surnamed Kim, who was charged with distributing deepfake obscenity under the Sexual Violence Punishment Act. Kim shared AI-generated nude images of women in a Telegram chat room, but the court stated, “There is no evidence to confirm the source of the photos or whether they were synthesized, making it difficult to conclude that the victims are real.” Ultimately, the ‘Deepfake Prevention Act’ only applies to content involving real people.

In schools, AI-synthesized content is being misused as a new form of bullying. An elementary school teacher said, “A male student synthesized a classmate’s face onto a nude photo and shared it in a group chat to mock her. The victim suffered irreparable emotional damage.”

※ This article has been translated by Upstage Solar AI.