White Circle open-sources RISE, a red-teaming algorithm for AI image models
Summary
White Circle has launched RISE, an automated tool designed to test the vulnerabilities of AI image generators by exploring various attack strategies to circumvent their established rules. This tool employs an open attacker model that generates numerous prompts based on each strategy, allowing for continuous learning even against stringent defenses. RISE is notable for being fully open-sourced and accessible, enabling defenders to deploy it on their own image models using existing frameworks like Qwen and Gemini Flash without the need for additional training.
Analysis
Qwen: Qwen refers to a family of publicly available large language models suitable for driving red-teaming workflows. RISE leverages Qwen under the hood to convert attack strategies into diverse prompts without requiring custom training. RISE: RISE is an open-source red-teaming algorithm released by White Circle for systematically testing AI image model safeguards. It generates and iterates on attack prompts using available models, assigning credit to strategies based on bypass success rates to maintain effectiveness even against highly restricted systems. OpenAI: OpenAI builds and deploys frontier AI systems, including image generation models with built-in safeguards. Its models are specifically cited as targets that RISE successfully probes for vulnerabilities through automated prompt generation and strategy search. DeepMind: DeepMind researches and releases advanced AI models, including image generators equipped with content safety mechanisms. RISE is positioned to evaluate and challenge the robustness of those safeguards as part of broader efforts to test the latest systems from the lab. Gemini Flash: Gemini Flash is a fast, publicly accessible version of Google’s Gemini model family used for efficient inference. RISE incorporates Gemini Flash alongside other models to scale prompt generation and evaluation during safeguard testing. White Circle: White Circle develops AI safety and red-teaming tools. It recently open-sourced RISE, an automated algorithm that searches across attack strategies to test and bypass safeguards in leading image generation models from OpenAI, DeepMind, and others. The tool runs on publicly available models like Qwen and Gemini Flash with no extra training needed and is designed for defenders to use directly on their own systems. Model Usage: RISE operates using readily available models such as Qwen and Gemini Flash without any additional training required. Tool Accessibility: The RISE code is fully open, the supporting models are public, and defenders can immediately run the tool against their own image models. AI Safety Practices: Effective safety work for AI image generators increasingly depends on systematic search over diverse attack strategies rather than static defenses.
Categories
aiai_agentsmachine_learningtech