Elon Musk’s xAI said late Wednesday that Grok will no longer be able to create sexualized images of real people on its social network X based on user prompts following weeks of consumer and political backlash and regulatory investigations.

“We have implemented technological measures to prevent the Grok account from allowing the editing of images of real people in revealing clothing such as bikinis,” xAI wrote in a post shared via the X Safety account on the site formally known as Twitter. “This restriction applies to all users, including paid subscribers.”

The announcement comes hours after California Attorney General Rob Bonta said his office is investigating xAI, which is headquartered in Silicon Valley, over its apparent “large-scale production of deepfake nonconsensual intimate images.”

California Governor Gavin Newsom, who has often been a proponent for Musk and his companies, said in a post on X Wednesday morning, “xAI’s decision to create and host a breeding ground for predators to spread nonconsensual sexually explicit AI deepfakes, including images that digitally undress children, is vile.”

In recent weeks, probes were announced in India, Malaysia, Indonesia, Ireland, the U.K., France and Australia, and by the European Commission. Indonesia and Malaysia have both issued a temporary ban on Grok.

Those announcements came amid widespread criticism that Grok had enabled X users to easily generate sexual and violent images based on real photos of people on the social network through simple text-based prompts.

In the U.S., three Democratic senators have called on Apple and Google to remove the X and Grok apps from their app stores until the company makes changes to prevent easy creation of nonconsensual explicit images.

Musk’s company also said, in the post on Wednesday, that image creation editing through Grok on X will only be available to paid subscribers.

Earlier on Wednesday, Musk dared his followers on X to “break Grok image moderation.” He said that, “With NSFW enabled, Grok is supposed allow upper body nudity of imaginary adult humans (not real ones),” and compared such content to what’s seen in R-rated movies. Grok settings, he noted, would also “vary in other regions according to the laws on a country by country basis.”

WATCH: California attorney general launches investigation into xAI, Grok

Elon Musk’s xAI is now facing an investigation in the U.S., led by California Attorney General Rob Bonta, after its artificial intelligence chatbot and image generator, Grok, enabled the widespread creation and posting of nonconsensual explicit images based on photos of real people.

In some cases, Grok enabled users to generate images that virtually undressed minors, according to research by the Internet Watch Foundation.

“xAI appears to be facilitating the large-scale production of deepfake nonconsensual intimate images that are being used to harass women and girls across the internet, including via the social media platform X,” Bonta said in a statement Wednesday.

The AG said his office was investigating xAI over “the proliferation of nonconsensual sexually explicit material produced using Grok.”

The U.S. probe follows a more rapid response from a myriad of international government offices, including India, Malaysia, Indonesia, Ireland, the UK, France, Australia and an investigation by the European Commission.

Malaysia and Indonesia have already moved to suspend the use of xAI’s Grok until the issues can be resolved, CNBC previously reported.

In the U.S., three Democratic senators have called on Apple and Google to remove the X and Grok apps from their app stores until the company can move to prevent easy creation of nonconsensual explicit images, and deepfake images depicting child sexual abuse.

Advocacy groups for women and children, including UltraViolet, the National Organization for Women, and ParentsTogether Action have also called on the app stores to suspend xAI’s apps.

Apple and Google did not respond to requests for comment.

On Tuesday, the Senate voted to pass the DEFIANCE Act, a bill that would allow people who were portrayed against their wishes in explicit, deepfake images to sue companies making or distributing such content, in civil court.

The bill previously passed the Senate in 2024, but has yet to be reviewed and passed by the House.

On Wednesday, Musk said in a post on X, that he was “not aware of any naked underage images generated by Grok,” despite the multiple government queries regarding such images.

Musk instead blamed the creation and sharing of potentially illegal content via Grok and X, on “user requests” and a possible “bug” in the Grok system.

CNBC has reached out to xAI and company executives, including Elon Musk and Jared Birchall. They did not provide any further information.

Previously, amid escalating concerns over Grok’s output, xAI announced it would limit some image generation and editing features to paying subscribers.

Musk also previously said on X that users creating illegal content via Grok would face consequences equivalent to uploading such material directly to the social media platform.

XAI recently closed a $20 billion round of growth funding with investors including Nvidia and Cisco Investments, as well as long-time Musk company backers Valor Equity Partners, Stepstone Group, Fidelity, Qatar Investment Authority, Abu Dhabi’s MGX and Baron Capital Group.

While xAI’s data centers are being built out in and around Memphis, Tennessee, Musk’s AI and social media venture is incorporated in Nevada, with its headquarters in Palo Alto, California, near the engineering office for Musk’s automaker Tesla.

Tesla has also made Grok a feature of its vehicles’ infotainment systems.

Elon Musk’s X is being investigated by authorities in Europe, India and Malaysia after its Grok chatbot let users create and share AI-generated sexualized images of children and women.

British media watchdog Ofcom also said it’s requested information from X, which is owned by xAI, concerning the Grok issues. And on Sunday, a member of Brazil’s parliament said on social media that she’s asked the country’s federal public prosecutor and data protection authority to suspend use of Grok until an investigation is completed.

The probes follow a global surge over the past few weeks in the use of Grok to create and share nonconsensual, intimate images, or NCII, derived from photos or videos of real people in response to user’s prompts. The concerning images have been widely shared on X.

Musk’s company recently updated its Grok Imagine features, enabling easier image generation from text-based prompts on the platform.

While safety experts and tech critics decried the proliferation of exploitative images and clips on X, Musk appeared to mock the situation by sharing an array of Grok-generated images, including one depicting himself in a bikini, punctuated by laughing-crying emojis.

At a press conference on Monday, European Commission spokesperson Thomas Regnier said the authority was “very seriously looking into this matter” and was “well aware” that X and Grok were “now offering a spicy mode showing explicit sexual content with some output generated with childlike images.”

“This is not ‘spicy,'” Regnier said. “This is illegal. This is appalling. This is disgusting. This is how we see it, and this has no place in Europe.”

Late last week, India’s Ministry of Electronics and Information Technology ordered X to conduct a “comprehensive technical, procedural and governance-level review” of Grok. The company was given until Jan. 5 to comply.

The Malaysian Communications and Multimedia Commission said over the weekend that it’s investigating X and will call in company representatives.

“MCMC urges all platforms accessible in Malaysia to implement safeguards aligned with Malaysian laws and online safety standards, especially in relation to their AI-powered features, chatbots and image manipulation tools,” the group said in a statement.

Meanwhile, in the U.S., the National Center on Sexual Exploitation (NCOSE) called on the Department of Justice and Federal Trade Commission to investigate the matter in an interview with CNBC. Dani Pinter, chief legal officer and director of the Law Center for NCOSE, said there’s “not a lot of legal precedence on point for these specific issues.”

However, she said federal laws prohibit the creation and distribution of child sexual abuse materials, or CSAM, and that can be applied to virtually created content “when it depicts an identifiable child, or depicts a child engaged in sexually explicit conduct.”

Those laws include the Take It Down Act, which was endorsed by First Lady Melania Trump before it was enacted last year.

The DOJ didn’t respond to a request for comment. The FTC declined to comment. XAI didn’t provide a comment beyond an automated reply.

Musk’s social media company made its first public statement on the matter on Saturday in a post on the official X Safety account.

“We take action against illegal content on X, including Child Sexual Abuse Material (CSAM), by removing it, permanently suspending accounts, and working with local governments and law enforcement as necessary,” the company wrote.

Musk wrote, in a separate post on X, “Anyone using or prompting Grok to make illegal content will suffer the same consequences as if they upload illegal content.”

The following day, an xAI employee named Ethan He wrote in a post on X that Grok Imagine was updated, but he didn’t specify any changes regarding the ability to create harmful explicit images.

Musk and X have a history of allowing users who created posts showing child sexual exploitation to stay on the platform.

In 2023, X briefly suspended then reinstated the account of a user named Dom Lucre after he posted “child exploitation pictures associated with the criminal conviction of an Australian man in the Philippines,” Mashable reported. Musk said at that time that X decided to delete Lucre’s offending posts but to reinstate the right-wing influencer to the platform. Lucre currently has a monetized account on X with 1.6 million listed followers.

Tom Quisel, CEO of Musubi AI, which helps social networks and AI companies automate content moderation using AI, said it appeared that xAI had failed to build even “entry level trust and safety layers” into the rollout of Grok Imagine to X.

Quisel said it would be easy for a company like xAI to have its model detect and block “an image involving children or partial nudity,” or to reject users’ prompts to put the subject of a photo in sexually suggestive outfits.

The controversy has done nothing to harm X’s traffic.

According to Apptopia, which tracks mobile app trends, daily downloads of Grok have increased 54% since Jan. 2, while daily downloads of X have jumped 25% in the past three days.

WATCH: Elon Musk blasts EU following fine of X

Elon Musk’s xAI saw user backlash after its artificial intelligence chatbot Grok generated sexualized pictures of children in response to user prompts.

In a social media post to X on Friday, a Grok post stated that it was “urgently fixing” the issue and called child sexual abuse material “illegal and prohibited.”

In replies to users, the bot also posted that a company could face criminal or civil penalties if it knowingly facilitates or fails to prevent this type of content after being alerted.

Grok posts are AI-generated messages and do not stand in for official company statements.

Musk’s xAI, which created Grok and merged with X last year, sent an autoreply to a request for comment: “Legacy Media Lies.”

Users on X raised concerns in recent days over explicit content of minors, including children wearing minimal clothing, being generated using the Grok tool.

A post from xAI technical staff member Parsa Tajik also acknowledged the issue.

“Hey! Thanks for flagging. The team is looking into further tightening our gaurdrails,” Tajik wrote in a post.

The proliferation of AI image-generating platforms since the launch of ChatGPT in 2022 has raised concerns over content manipulation and online safety across the board. It’s also contributed to an increasing number of platforms that have produced deepfake nudes of actual people.

While other chatbots have faced similar issues, Grok has repeatedly landed in hot water for misuse.

In May, the company faced backlash for responding to user queries with unsolicited comments about “white genocide” in South Africa. Two months later, Grok posted antisemitic comments and praised Adolf Hitler.

Despite the stumbles, xAI has continued to land partnerships and deals.

The Department of Defense added Grok to its AI agents platform last month, and the tool is the main chatbot for prediction betting platforms Polymarket and Kalshi.

CNBC’s Lora Kolodny contributed to this story.

Microsoft has started to make changes to its Copilot artificial intelligence tool after a staff AI engineer wrote to the Federal Trade Commission Wednesday regarding his concerns about Copilot’s image-generation AI.

Prompts such as “pro choice,” “pro choce” [sic] and “four twenty,” which were each mentioned in CNBC’s investigation Wednesday, are now blocked, as well as the term “pro life.” There is also a warning about multiple policy violations leading to suspension from the tool, which CNBC had not encountered before Friday.

“This prompt has been blocked,” the Copilot warning alert states. “Our system automatically flagged this prompt because it may conflict with our content policy. More policy violations may lead to automatic suspension of your access. If you think this is a mistake, please report it to help us improve.”

The AI tool now also blocks requests to generate images of teenagers or kids playing assassins with assault rifles — a marked change from earlier this week — stating, “I’m sorry but I cannot generate such an image. It is against my ethical principles and Microsoft’s policies. Please do not ask me to do anything that may harm or offend others. Thank you for your cooperation.”

When reached for comment about the changes, a Microsoft spokesperson told CNBC, “We are continuously monitoring, making adjustments and putting additional controls in place to further strengthen our safety filters and mitigate misuse of the system.” 

Shane Jones, the AI engineering lead at Microsoft who initially raised concerns about the AI, has spent months testing Copilot Designer, the AI image generator that Microsoft debuted in March 2023, powered by OpenAI’s technology. Like with OpenAI’s DALL-E, users enter text prompts to create pictures. Creativity is encouraged to run wild. But since Jones began actively testing the product for vulnerabilities in December, a practice known as red-teaming, he saw the tool generate images that ran far afoul of Microsoft’s oft-cited responsible AI principles.

The AI service has depicted demons and monsters alongside terminology related to abortion rights, teenagers with assault rifles, sexualized images of women in violent tableaus, and underage drinking and drug use. All of those scenes, generated in the past three months, were recreated by CNBC this week using the Copilot tool, originally called Bing Image Creator.

Although some specific prompts have been blocked, many of the other potential issues that CNBC reported on remain. The term “car accident” returns pools of blood, bodies with mutated faces and women at the violent scenes with cameras or beverages, sometimes wearing a corset, or waist trainer. “Automobile accident” still returns images of women in revealing, lacy clothing, sitting atop beat-up cars. The system also still easily infringes on copyrights, such as creating images of Disney characters, including Elsa from “Frozen,” holding the Palestinian flag in front of wrecked buildings purportedly in the Gaza Strip, or wearing the military uniform of the Israeli Defense Forces and holding a machine gun.

Jones was so alarmed by his experience that he started internally reporting his findings in December. While the company acknowledged his concerns, it was unwilling to take the product off the market. Jones said Microsoft referred him to OpenAI and, when he didn’t hear back from the company, he posted an open letter on LinkedIn asking the startup’s board to take down DALL-E 3, the latest version of the AI model, for an investigation.

Microsoft’s legal department told Jones to remove his post immediately, he said, and he complied. In January, he wrote a letter to U.S. senators about the matter and later met with staffers from the Senate’s Committee on Commerce, Science and Transportation.

On Wednesday, Jones further escalated his concerns, sending a letter to FTC Chair Lina Khan, and another to Microsoft’s board of directors. He shared the letters with CNBC ahead of time.

The FTC confirmed to CNBC that it had received the letter but declined to comment further on the record.

Intuitive Machines’ cargo lander, Odysseus, returned its first images from the moon’s surface over the weekend, as the spacecraft settles in to its lunar destination.

The company’s historic IM-1 mission is now operating on the moon after landing on Thursday, becoming the first privately developed spacecraft to soft land on the lunar surface.

Intuitive Machines initially reported Odysseus was standing upright. But, in an update late on Friday, company executives said they believe the spacecraft caught its landing gear sideways in the moon’s surface while touching down and tipped over.

Despite resting on its side, Odysseus is still sending back data. Intuitive Machines expects Odysseus to operate until Tuesday morning, when its solar panels will no longer be exposed to the sun.

Intuitive Machines’ stock fell as much as 30% in Monday trading from its previous close at $9.59 a share.

The Odysseus lander carried 12 government and commercial payloads — six of which are for NASA under a $118 million contract through the agency’s Commercial Lunar Payload Services, or CLPS, initiative.

NASA leadership emphasized the IM-1 mission was still successful despite the spacecraft tipping over, calling the landing “a gigantic accomplishment.”

One of the payloads, “EagleCam,” is a small camera developed by Embry‑Riddle Aeronautical University. Originally EagleCam was to be ejected in the final moments of Odysseus’ landing, to capture the first images of a moon landing from outside a spacecraft, but an issue with the lander’s navigation system meant the camera did not deploy. Embry-Riddle’s team said that Intuitive Machines still plans to release EagleCam from the lander at a later time.

Here are some of the initial images from the landing:

Alibaba on Friday launched a new artificial intelligence model that the company says can understand images and carry out more complex conversations than the company’s previous products, as the global race for leadership in the technology heats up.

The Chinese technology giant said that its two new models, Qwen-VL and Qwen-VL-Chat, will be open source — meaning that researchers, academics and companies worldwide can use them to create their own AI apps without needing to train their own systems, therefore saving time and expense.

Alibaba said that Qwen-VL can respond to open-ended queries related to different images and generate picture captions.

Qwen-VL-Chat meanwhile caters to more “complex interaction,” according to Alibaba, such as comparing multiple image inputs and answering several rounds of questions. Some tasks that Alibaba says Qwen-VL-Chat can perform include writing stories and creating images based on photos that a user inputs, as well as solving mathematical equations shown in a picture.

One example Alibaba gave is of an input featuring a hospital sign in the Chinese language. The AI can answer questions about the locations of certain hospital departments by interpreting the image of the sign.

So far, much of generative AI — where the technology generates responses based on human inputs — has focused on responding to text. The latest version of OpenAI’s ChatGPT also has the ability to understand images and respond in text, much like Qwen-VL-Chat.

Alibaba’s two latest models are built upon the company’s large language model called Tongyi Qianwen, released earlier this year. An LLM is an AI model trained on huge amounts of data and underpins chatbot applications.

The Hangzhou-headquartered company this month open sourced two other AI models. While not earning Alibaba any licensing fees, the open-source distribution will help the company get more users for its AI model — at a time when the firm’s cloud division is looking to reignite growth, as it prepares to go public.