OpenAI tests model compliance with laws of war for US military
Summary
OpenAI is currently testing its models to ensure their responses conform to the laws of war, a process aimed at preparing these models for use by the U.S. military. Sarah Yager, who took on a new role focused on human rights after 30 years in military issues, is leading this initiative. She noted that international experts have been inputting high-stakes combat scenarios into the models, allowing engineers to adjust the models based on adherence to the Geneva Conventions. This marks a pioneering backward finetuning approach that OpenAI has never implemented before.
Analysis
OpenAI: OpenAI is an artificial intelligence research and deployment company developing advanced models for various applications. The organization is now evaluating whether its models' responses adhere to the laws of war, including the Geneva Conventions, in preparation for potential use by the U.S. military. It recently established its first human rights and responsible deployment role to manage this alignment process through expert testing and model adjustments. Sarah Yager: Sarah Yager is a professional with three decades of experience on military issues who joined OpenAI in June as its head of human rights and responsible deployment. She is overseeing a novel evaluation method where international experts input high-stakes combat prompts into OpenAI models and engineers refine them based on compliance with legal standards. Military AI Alignment: OpenAI is conducting specialized testing of its models against international humanitarian law standards for defense applications. Responsible AI Practices: The company introduced a dedicated human rights role to guide ethical model adjustments in high-stakes scenarios.
Categories
aitechmachine_learningpolitics