Separating signal from noise in coding evaluations
中文摘要
OpenAI分析指出编程基准测试SWE-Bench Pro存在问题，引发对其评估AI模型可靠性和准确性的担忧。
English Summary
OpenAI's analysis of the SWE-Bench Pro coding benchmark reveals issues, raising concerns about the reliability and accuracy of AI model evaluations.
原文节选
A new analysis from OpenAI reveals issues in SWE-Bench Pro, a popular coding benchmark, raising concerns about reliability and accuracy in evaluating AI models.