Google DeepMind study reveals flaws in LLMs' random sampling
Summary
A new study from Google DeepMind reveals that Large Language Models (LLMs), such as Gemini, Qwen3, and OLMo, cannot produce truly random choices independently. Although these models can convert provided random numbers into desired outcomes, they struggle to select actions uniformly across distributions when left to their own devices, often favoring specific numbers like 7 or 42. The research demonstrates that without the input of external random values, LLMs fail to meet the stochastic sampling requirements needed for agentic systems, leading to significantly higher success rates for attackers in security contexts. This underscores the importance of generating randomness through code rather than relying on LLMs' inherent randomness.
Analysis
OLMo: OLMo is the Open Language Model series developed by the Allen Institute for AI to promote open research into large-scale language models. The study included OLMo in its empirical analysis, confirming that its sampling behavior also failed statistical tests for matching specified distributions. Qwen3: Qwen3 is part of Alibaba's Qwen family of open and proprietary large language models designed for general reasoning and tool use. In the DeepMind paper, Qwen3 models were evaluated on stochastic sampling benchmarks and exhibited the same inability to produce outputs matching requested probability distributions without external randomization. Gemini: Gemini refers to Google's series of multimodal large language models developed and scaled by DeepMind. The recent study tested Gemini alongside other models on repeated distribution sampling tasks and found consistent deviations from target probabilities across multiple configurations. Google DeepMind: Google DeepMind is an AI research organization focused on advancing machine intelligence through fundamental breakthroughs and applied systems. The lab recently released a paper analyzing how large language models fail at reliable stochastic sampling, a core requirement for agentic applications. This work demonstrates that even advanced models cannot reliably emulate random draws from distributions on their own. AI Agents: Agentic systems frequently require LLMs to sample actions from probability distributions derived from observations or policies. Model Behavior: Frontier LLMs can accurately convert externally supplied random values into target distributions but cannot generate equivalent stochastic outputs independently.
Categories
aimachine_learningtech