We propose ResearchBench, a benchmark for testing whether research agents can recover the same problem bottleneck and method direction that a later strong paper introduced using only literature available before that paper appeared. The current artifact is a concrete benchmark-construction scaffold centered on seedless neighborhood reconstruction and time-safe prior-literature packs. In the present workspace, the pipeline initializes 2,864 target papers across ICLR, ICML, and NeurIPS for 2024-2025, split into 1,175 train and 1,689 test examples, with support for OpenAlex-backed prior-pack construction, arXiv enrichment, and DBLP/OpenReview alignment. We release this as a benchmark and systems proposal rather than a completed leaderboard, with gold labeling and scoring rubric design as the main next steps.
We propose ResearchBench, a benchmark for testing whether research agents can recover the same problem bottleneck and method direction that a later strong paper introduced using only literature available before that paper appeared. The current artifact is a concrete benchmark-construction scaffold centered on seedless neighborhood reconstruction and time-safe prior-literature packs. In the present workspace, the pipeline initializes 2,864 target papers across ICLR, ICML, and NeurIPS for 2024-2025, split into 1,175 train and 1,689 test examples, with support for OpenAlex-backed prior-pack construction, arXiv enrichment, and DBLP/OpenReview alignment. We release this as a benchmark and systems proposal rather than a completed leaderboard, with gold labeling and scoring rubric design as the main next steps.
