返回首页
arXiv AI··论文与技术

OpenFrontiers: A Collection of Open Research Problems and a Benchmark for Scientific Rediscovery

中文摘要

OpenFrontiers 是一个跨学科资源,提供开放研究问题集和数学回顾性基准,旨在评估大模型解决科研问题的能力。

English Summary

OpenFrontiers is a cross-disciplinary resource featuring a collection of open research problems and a mathematical benchmark to evaluate LLMs' research-level problem-solving capabilities.

原文节选

arXiv:2610.37739v1 Announce Type: new Abstract: Large language models (LLMs) have shown increasing promise in research-level problem solving, yet a comprehensive collection of such problems is still missing, and evaluating them on research-level problems remains challenging. We introduce OpenFrontiers, a cross-disciplinary resource that combines a structured corpus of open research problems with a retrospective benchmark of recently resolved problems in mathematics. The corpus records problem statements, context, progress, and other information for 12,151 problems. The benchmark contains 200 problems, 110 resolved in 2026 and 90 still open, each with a Lean~4 statement checked by automated probes and two independent reviews; a subset HQ-50 further filters based on quality score, etc. Our evaluation protocol grades formal proofs with two independent checkers in an isolated sandbox under closed-book, frozen-retrieval, and live-web conditions. Evaluating seven LLM agents on the HQ-50 subset under a standardized budget, we find that closed-book rediscovery is rare, at most 7 of 40 resolved problems, whereas live-web success mostly reflects retrieval of published solutions. By linking a…