The dark side of artificial intelligence: South Korea to combat 'fake sexual content' problem

The Asian nation of South Korea is facing a 'fake sexual content' problem. Security authorities in South Korea have announced that they will develop a detection system to combat 'fake sexual content' that has caused victimization for many people.

In a statement made by the South Korean police, it was stated that an investment of approximately 6 million dollars will be made to establish a system capable of detecting deepfake content.

According to a report by Yonhap, efforts are underway in South Korea to curb the rise in crimes involving sexually explicit images and audio generated by artificial intelligence. In the officials' statement, it was noted that the project is expected to be completed by 2027 and that the system will accelerate the detection of deepfake videos.

In 2024, 573 suspects were detained in South Korea in connection with 1,094 deepfake-related crimes.

It was understood that 'deepfake' images were being spread via Telegram in South Korea. Authorities had launched an investigation into the messaging application.

THE DEEPFAKE INCIDENT IN SOUTH KOREA

It was determined that 'sexually explicit' images containing audio and visuals manipulated via 'deepfake' methods, featuring many victims—the majority of whom were women and girls—were being spread in certain chat groups linked to schools and universities in South Korea.

Head of the National Office of Investigation Woo Jong-soo had noted that a preliminary investigation had been opened into Telegram, where these images were being spread, on the grounds of 'aiding and abetting a crime'.

Telegram officials, who met with representatives from the Korea Communications Standards Commission (KCSC), stated that they were aware of the seriousness of the problem and promised to implement a 'zero-tolerance policy', including strict measures against such illegal material.