Microsoft's Coding-Agent Skill Distillation outperforms GEPA in prompt optimization
Summary
A new Microsoft paper highlights that a coding agent, when provided with an agent's old logs, can create better prompts than traditional trial-and-error tuning tools. This approach, termed Coding-Agent Skill Distillation (CASD), allows the coding agent to analyze all available data collectively, identifying systematic failures and optimizing prompts without the need for repeated environment interactions. Results indicate that CASD outperformed the state-of-the-art GEPA tool on three out of four benchmarks, all while being significantly more cost-effective at around $1.60 per prompt. This research suggests that offline analysis of agent trajectories could be a more efficient method for improving prompt optimization in AI systems.
Analysis
GEPA: GEPA is a state-of-the-art reflective prompt optimizer that improves prompts through iterative search over edits and validation rollouts. The Microsoft paper directly benchmarks CASD against GEPA, showing superior results on three of four agent benchmarks using the same static logs. SkillOpt: SkillOpt is a validation-gated reflective search approach for prompt optimization that relies on iterative evaluation. The paper demonstrates that the proposed offline CASD method outperforms SkillOpt across all four tested benchmarks while requiring far less computation. Microsoft: Microsoft is a major technology company with extensive AI research efforts across its labs and product teams. In this news, Microsoft researchers authored the paper introducing Coding-Agent Skill Distillation (CASD) as an efficient alternative to iterative prompt optimization methods. Agent Evaluation: Benchmarks involving simulated environments and task-specific scenarios are commonly used to measure advances in agent prompt optimization. AI Research Methods: Offline analysis of agent trajectories is emerging as a lower-cost path to prompt improvement compared with repeated environment interactions.
Categories
techaimachine_learningai_agents