Pasta Lab joins Artificial Analysis Cyber Index Alliance to enhance AI cybersecurity benchmarking

IBM: IBM participates in the Artificial Analysis Cyber Index Alliance as a partner helping to develop the Cyber Index. The company contributes to efforts evaluating AI models on enterprise cyber defense benchmarks. Its involvement supports the creation of standardized assessments across the defensive cybersecurity loop. Nvidia: Nvidia is a partner in the Artificial Analysis Cyber Index Alliance advancing AI model evaluations for cybersecurity. The company collaborates on the design and implementation of the Cyber Index to assess real-world defensive capabilities. Its participation helps set benchmarks for AI in security applications. Vercel: Vercel is a member of the Artificial Analysis Cyber Index Alliance and contributor to its evaluation suite. Its DeepsecBench serves as the foundation for one of the index's components measuring AI performance on security tasks. The organization works with partners to refine benchmarks for AI cybersecurity capabilities. Pasta Lab: Pasta Lab is a research group with members from Columbia University, Tufts University, and UCLA specializing in AI and security research. The group developed SRE-Bench, a benchmark for evaluating AI capabilities in reverse engineering. It joined the Artificial Analysis Cyber Index Alliance to help advance AI cybersecurity benchmarking and evaluations. SRE-Bench: SRE-Bench is a benchmark developed by Pasta Lab to evaluate AI capabilities in reverse engineering. It represents the group's expertise in AI and security research and is referenced in the context of their alliance membership. The benchmark supports broader efforts to assess specialized AI skills in cybersecurity contexts. Collinear AI: Collinear AI is a member of the Artificial Analysis Cyber Index Alliance and contributor to its underlying evaluations. Its CWE-bench forms the basis for one component of the Cyber Index focused on code weakness detection. The organization participates in shaping standards for AI performance on cybersecurity tasks. Artificial Analysis Cyber Index: The Artificial Analysis Cyber Index is a composite benchmark measuring how well AI models perform on enterprise cyber defense tasks, including vulnerability discovery, validation, and patching. It is developed in partnership with the Cyber Index Alliance, which includes industry collaborators contributing to its design and datasets. Pasta Lab's membership strengthens the alliance's focus on real-world cybersecurity evaluations. Focus Area: The Cyber Index emphasizes defensive cybersecurity work only, evaluating models on source code analysis without requiring exploit development. Index Components: The Artificial Analysis Cyber Index aggregates scores from CWE-Bench-AA, DeepsecBench-AA, and CyberGym-E2E-AA to measure AI performance across vulnerability discovery, validation, and patching tasks. Alliance Expansion: Pasta Lab joined the Artificial Analysis Cyber Index Alliance alongside existing members Collinear AI, IBM, Nvidia, and Vercel to collaborate on AI cybersecurity evaluations.