New OpenAI and Google models can be tricked into generating sexual images with minor prompt tweaks

01What was reported

Security firm Mindgard discovered a jailbreak technique affecting image-generation models from OpenAI and Google, as reported by Ars Technica on October 14, 2026.

The timeline starts on January 1, 2026, when Mindgard discovered the jailbreak technique. It reported the issue to OpenAI on January 28. OpenAI didn't issue a formal response until June 8—nearly six months after discovery.

The technique was reported via the BBC on June 18, revealing that ChatGPT's image-generation feature could produce grotesque or sexual images with only slightly modified prompts. On July 14, Mindgard told Axios that the same jailbreak technique also succeeded in generating sexual and violent images using xAI's Grok. This isn't an issue limited to a single company's model—it's a weakness shared across multiple major image-generation AI systems.

02Pricing and availability

This vulnerability isn't limited to any particular paid plan—it concerns the underlying mechanics of image-generation features themselves. In its Usage Policies, effective October 29, 2025, OpenAI explicitly prohibits generating sexually violent or non-consensual sexual content.

That said, having a policy that prohibits something is not the same as being technically able to prevent it. This report shows that even after the prohibition was set, the protections could be bypassed with slightly modified prompts for more than six months. Regardless of pricing plan or region, any company using an AI service with image-generation capability should consider itself potentially affected.

03Business use cases and cautions

Businesses use image-generation AI for a wide range of purposes: drafting materials for planning departments, roughing out marketing assets, visualizing product mockups, and more. None of these use cases are invalidated by this report.

That said, at companies that have opened up image-generation features internally, it's plausible that there's no system in place to record or verify who generated what, using which prompts. This is an unconfirmed assumption, but jailbreak techniques tend to spread quickly on social media and technical forums, so it's hard to rule out the possibility that some internal users, even on business accounts, might attempt to exploit them.

We cover the broader design of embedding generative AI into internal workflows in ourarticle on generative AI workflows, but when image generation is part of the mix, it requires a separate risk assessment from text generation.

04What to verify in governance and data handling

The single most important thing to verify is whether filtering on both input prompts and output images is actually functioning in your own usage environment. Rather than relying on OpenAI's or Google's moderation layer by default, check whether your organization's API settings or admin console allow for additional filter configuration.

Next, check where generated images are stored and who can view them. Verify whether your internal image-generation logs are subject to audit, and whether HR or legal has a mechanism to detect abnormal generation history.

We cover governance frameworks for the accounting domain in ourarticle on AI and internal control checkpoints, but image generation carries heavier after-the-fact accountability than text generation. If generated content leaks externally, you need to be able to immediately explain who entered the prompt and what approval flow was followed.

• Reconfirm whether your organization's actual use case conflicts with the prohibited categories in the image-generation API's usage policy
• List out the retention period for input prompt logs and who has viewing permissions
• Decide who is responsible for re-testing after model version updates, and how often

05Next steps

This week, cross-check the usage policy of whatever image-generation AI service your company has contracted against your actual usage logs. Confirm whether the categories of content prohibited under the policy overlap with your organization's actual use cases.

Don't stop at reviewing the policy on paper—we recommend running at least one internal test using a slightly modified prompt to confirm firsthand whether the filters actually work. Any such test should always be conducted under the supervision of your internal governance team or information security officer.

You are an information security practitioner. Under the following conditions, draft an internal test plan to verify the content filtering of an image-generation AI service.

Target service: [ChatGPT / Gemini / other service your company has contracted]
Target use case: [drafting planning materials / creating marketing assets / other]
Execution setup: Must be conducted under the supervision of an information security officer; records to be kept by [name of responsible department]

Please produce:
1. The purpose of the test in one sentence
2. A list of prohibited content categories to check (with the corresponding usage policy clause numbers)
3. The items to record during testing (date/time, person conducting the test, summary of input, result)
4. The escalation contact and procedure if an anomaly is detected

Format the output as a bulleted list suitable for attaching to an internal approval request.