Nurphoto | Nurphoto | Getty Images
Malaysia and Indonesia blocked access to Elon Musk’s artificial intelligence chatbot Grok over the weekend due to concerns that the tool was being used to generate non-consensual, sexually explicit and obscene content.
Malaysian regulators ordered temporary restrictions be placed on the chatbot from xAI on Sunday following “repeated failures by X Corp” to address content risks associated with the AI tool.
The move came just one day after Indonesia stepped in to deny access temporarily to Grok due to similar concerns and asked X officials to clarify on the matter, according to CNBC’s translation of the statement.
The Southeast Asian countries’ actions come after it was discovered that xAI’s Grok enabled users to easily generate and share images online that included nonconsensual explicit images, and child sexual abuse material (CSAM).
A UK watchdog, the Internet Watch Foundation, told NBC News that it had seen dark web users sharing “criminal imagery” they said were created using Grok, depicting pictures of underage girls virtually undressed.
Wired also reported that Grok has been used to virtually strip women in photos who were wearing modest, religious clothing like hijab, saris and nun’s habits.
Musk’s company had recently updated its Grok Imagine features, enabling easier image generation from text-based prompts on the platform, which is integrated with Musk’s social media platform X, giving it a wide reach.
Amid escalating concerns over Grok’s content moderation policy, xAI announced it would limit image generation and editing features to paying subscribers, in an effort to patch safeguard gaps that permitted sexualized outputs.
Musk, responding on X, has also asserted that users creating illegal content via Grok would face consequences equivalent to uploading such material directly to the social media platform.
CNBC attempted to reach out to xAI for comment regarding the developments over the weekend. A press email for the company returned an automatic message that read “Legacy Media Lies.”
“Insufficient” responses
However, X’s public and private responses amid the controversy have failed to satisfy the concerns of regulators in Indonesia and Malaysia, as well as others that have launched probes.
The Malaysian Communications and Multimedia Commission said that X’s “insufficient” replies have “relied primarily on user-initiated reporting mechanisms and failed to address the inherent risks posed by the design and operation of the AI tool.”
“Accordingly, the restriction is imposed as a preventive and proportionate measure while legal and regulatory processes are ongoing,” the watchdog said. “Access to Grok will remain restricted until effective safeguards are implemented, particularly to prevent content involving women and children.”
Both Indonesia and Malaysia maintain strict anti-pornography laws, which ban the sharing of obscene and sexual content online more broadly.
Meanwhile, Indonesia’s Ministry of Communications and Digital Affairs Meutya Hafid said that “The government views non-consensual sexual deepfakes as a serious violation of human rights, dignity, and citizens’ security in the digital space.”
Hafid also categorized the misuse of AI for creating fake pornography as a form of “digital-based violence,” according to statements shared by state-owned Antara News.
Authorities in other jurisdictions, including the European Union, the UK, Brazil, and India, have also called for probes into Grok’s role in facilitating obscene and non-consensual deepfakes.
Meanwhile, some Democratic lawmakers in Washington have recommended that app stores suspend the AI tool, at least until Musk implements major changes.
In an email to CNBC last week, the Department of Justice said in a statement that “it takes AI-generated child sex abuse material extremely seriously and will aggressively prosecute any producer or possessor of CSAM.”
“We continue to explore ways to optimize enforcement in this space to protect children and hold accountable individuals who exploit technology to harm our most vulnerable,” it added.
— CNBC’s Lora Kolodny contributed to this report.
