Frontier MLLMs pass up to 96% of conventional CAPTCHAs, yet fall far behind humans on interactive spatial ones.
pass@1 (%) of humans andthe best model, GPT-6 Astra
best pass@1 of GUI agents,even CAPTCHA-trained ones
of top-model failures comefrom misreading the screen
of failed open-model attemptsever drop their wrong belief
pass@1 (%) of Qwen3.5-9Bafter training in the Gym
Why interactive spatial CAPTCHAs?
Spatial understanding is a loop
People infer the 3D layout behind a view, act on it, look at the result and correct themselves.
Conventional CAPTCHAs are solved
Frontier models pass up to 96% of ReCAP and 82% of Open CaptchaWorld puzzles.
Most spatial benchmarks never ask to act
Most use multiple-choice questions; TuringSpace has the model click and drag through 125 task types.
Every task is a program, so the benchmark never runs out
A seed turns each task into a new puzzle with its own checker and solution. Three held-out seeds per task form TuringSpace-Bench; every other seed is training data in TuringSpace-Gym, drawn from separate asset pools and never repeating a test layout.
The reference solution of one instance per category, played in the browser one action at a time.
The best model still fails one puzzle in four
Why models fail, and how they learn
Stronger models do not think more; they know where to think
GPT-6 Astra uses 75 times fewer thinking tokens than Kimi K3 per attempt, and solves 2.3 times as many tasks.
We rerun all 125 tasks on one test seed at every reasoning effort for four models, and compare the thinking tokens of ten models at medium effort.
Reasoning defends a wrong belief instead of fixing it
The reasoning doubts its wrong belief in 89.4% of failed attempts, but gives it up in only 7.5%, almost always after the puzzle is already lost.
We take all 293 failed attempts on one test seed of the three models whose full reasoning is visible (Kimi K3, Qwen3.8-Max, Qwen3.8-27B). Claude Opus 5.5 finds the wrong belief that decides each failure and labels how the reasoning treats it; an author agrees with 92% of 50 sampled labels.
Three failed attempts, in the models' own words
One case for each way the reasoning keeps the wrong belief.
Models click where they intend; most failures come from misreading the screen
We label 380 failed attempts of GPT-6 Astra, Claude Opus 5.5 and Gemini 4 Argon with one of ten root causes in three groups.
Closed-loop and planning tasks show a new screenshot after every action, yet perception errors still cause 85 of their 129 failures. No failure comes from clicking at the wrong place.
A 9B model trained in the Gym improves on all seven out-of-distribution benchmarks
We fine-tune Qwen3.5-9B with thinking off and a frozen vision encoder on 55K verified oracle trajectories of TuringSpace-Gym, with no robot data, and test it zero-shot with each benchmark's official prompt.
- +31.5In domain: 1.6% → 33.1%, above Kimi K3, Qwen3.8-Max and Gemini 3.8 Flash and close to Claude Sonnet 5.5 (37.1%).
- +17.2Spatial CAPTCHAs teach conventional ones: +17.2 on ReCAP and +10.0 on Open CaptchaWorld, while CAPTCHA-trained agents reach ≤ 3.5% on ours.
- +19.8Real scenes: +9.9 on the room videos of VSI-Bench and +19.8, +7.4 and +12.2 on three robot benchmarks whose photos never appear in training.
Cite TuringSpace
@misc{ding2026turingspacegroundingspatialintelligence,
title={TuringSpace: Grounding Spatial Intelligence in Action with Interactive CAPTCHAs},
author={Xuwei Ding and Skylar Zhai and Linxin Song and Jiarui Feng and Dongqi Fu and Huajun Xi and Junnan Li and Jingcheng Liang and Jieyu Zhao and Yinglong Xia and Hong Li and Jian Kang},
year={2026},
eprint={2611.25387},
archivePrefix={arXiv},
primaryClass={cs.LG},
url={https://arxiv.org/abs/2611.25387},
}