PUBLICLEDGER
CorrectArchive capture
Even heavily scaled base LLMs (without test-time compute) continue to underperform on ARC-1 as of mid-2026.
- Expert
- François Chollet
- Prediction date
- Aug 6, 2026
- Deadline
- Aug 31, 2026
- Category
- AI & Technology
- Explicit confidence
- Not stated
Original statement
“To this day, current base LLMs (considerably scaled up compared to the models from that time) still do not perform well on something as easy as ARC 1 -- and can't even reliably do simple math operations.”
Resolution
Correct
Stated as contemporaneous observation consistent with public ARC results.
Resolved Aug 6, 2026 · 17:24 UTC
Normalized claim
Even heavily scaled base LLMs (without test-time compute) continue to underperform on ARC-1 as of mid-2026.
A falsifiable interpretation, separate from the exact source text.
Resolution criteria
Official ARC Prize leaderboards showing pure base models (no TTC) remaining well below human performance on ARC-AGI-1 in 2026.
Market context
No high-confidence matching market is linked. Loose thematic matches are not shown.
Audit trailView full history
- Source recorded2026-08-06 17:24 UTC
- Prediction verified and frozen2026-09-12 23:28 UTC
- Resolution approved2026-09-12 23:28 UTC