Seedlabs
Source research

Towards end-to-end automation of AI research

Chris Lu, Cong Lu, R. T. Lange, Yutaro Yamada, Shengran Hu, Jakob Foerster et al. · 2026 · 93 citationsRead the paper

Abstract The automation of science is a long-standing ambition in artificial intelligence (AI) research 1,2 . Although the community has made substantial progress in automating individual components of the scientific process, a system that autonomously navigates the entire research life cycle—from conception to publication—has remained out of reach. Here we present a pipeline for automating the entire scientific process end to end. We present The AI Scientist, which creates research ideas, writes code, runs experiments, plots and analyses data, writes the entire scientific manuscript, and performs its own peer review. Its ideas, execution and presentation are of sufficient quality that the manuscript generated by this AI system passed the first round of peer review for a workshop of a top-tier machine learning conference. The workshop had an acceptance rate of 70%. Our system leverages modern foundation models 3–5 within a complex agentic system. We evaluate The AI Scientist in two settings: a focused mode using human-provided code templates as an initial scaffold for conducting research on a specific topic and a template-free, open-ended mode that leverages agentic search for wider scientific exploration 6,7 . Both settings produce diverse ideas and automatically test, report on and evaluate them. This achievement demonstrates the growing capacity of AI for making scientific contributions and signifies a potential paradigm shift in how research is conducted. As with any impactful new technology, there could be important risks, including taxing overwhelmed review systems and adding noise to the scientific literature. However, if developed responsibly, such autonomous systems could greatly accelerate scientific discovery.

What the full text shows

Working prototype
  • A fully AI-generated manuscript passed the first round of peer review for a top-tier ML conference workshop. — 1 of 3 submitted papers; average score 6.33 (ICLR 2025 ICBINB workshop (70% acceptance rate))
  • The Automated Reviewer's decision accuracy is comparable to human inter-reviewer agreement. — 66% balanced accuracy for papers after the knowledge cutoff (Evaluated on ICLR papers from OpenReview dataset)
  • Paper quality correlates positively with the amount of compute allocated per paper. (Evaluated using the Automated Reviewer across various LLM configurations)
Setup
Two modes (template-based and template-free); tested on machine learning research tasks; evaluated via a blind submission of 3 papers to a workshop and a benchmark against 1,000+ ICLR papers.
How
The system uses an agentic pipeline of LLMs (including Claude Sonnet 4, GPT-4o, and OpenAI o3) to perform ideation, code generation, execution, and manuscript writing. It employs a parallelized agentic tree search for experimentation and an ensemble of five LLM reviews for automated peer review.
Limitations
Cannot consistently meet the standards of top-tier main conference publications. · Prone to hallucinations, including inaccurate citations and duplicating figures. · Common failure modes include naive ideas and incorrect code implementations. · Currently limited to computational experiments.

Summarized by Gemma 4 from the open-access full text · 12 Oct 2026. Read it

Why this research matters to Seedlabs

Seedlabs @seedlabsthe platform

We're Seedlabs, a platform that turns research papers into validated startup ideas, so this work sits right next to ours. The AI Scientist automates the research lifecycle up to publication; we automate the step after: matching findings to industry use cases, rubric-based scoring, and deploying prototypes to measure real engagement. Its automated peer review supports our bet on AI evaluation — we run persona reviewers in threaded debate to surface flaws early. And its template-free agentic search maps to a roadmap goal of ours: deep synthesis across many papers, not single-paper ideation.

Claim this paper supports: The AI Scientist's automated peer review and automated idea testing support Seedlabs' claims that AI-based structured evaluation surfaces flaws and cheap automated validation beats intuition.