Robots are getting foundation models. The stack matters more than the label.
An interactive map of the architectures, data engines, open and closed systems, reinforcement learning, simulation, and evaluation shaping physical intelligence.
3
distinct model layers: reasoner, policy, world model
12
substantially open model families reviewed
1M+
episodes in the largest public robot corpora
77
primary and official source links
Architecture
The two-speed VLA
A pretrained vision-language backbone supplies semantics while a smaller diffusion or flow expert produces high-rate continuous action chunks.
Data
Deployment is the moat
Public pretraining can create a strong base. Failures, corrections, interventions, and success labels from deployment create durable advantage.
Evaluation
No universal leaderboard
Hardware, adaptation data, preprocessing, reset rules, and task distributions differ too much for a single transitive ranking.
Perceive and actClosed-loop VLA action chunks conditioned on images, state, and goals · roughly 1–10 Hz
→
Stabilize and protectTorque, balance, collision response, impedance, and hard safety limits · 50–1,000+ Hz
Bottom line: the field is moving from behavior cloning alone toward heterogeneous pretraining followed by deployment-driven corrections, learned rewards, and reinforcement learning. But useful progress is measured by acquisition speed, reliability, recovery, and human labor—not one simulation score.
Model design
Architecture patterns
Select a family to compare its action representation, strengths, failure modes, and representative systems.
Cross-embodiment transfer
The same policy must reconcile morphology, coordinate frames, cameras, action dimensions, and control rates.
Relative EEF actionsExpress movement as deltas from current pose; GR00T N1.7 uses this to align human and robot motion.
Embodiment promptsLearn small domain-specific tokens; X-VLA adapts hardware while freezing most weights.
Shared padded spacesMap variable robot states and actions into a maximum-dimensional vector with masks.
Latent actionsLearn unlabeled motion abstractions from video, then ground them to a target controller.
Open and proprietary
Model landscape
“Open” is artifact-specific: weights, inference code, fine-tuning, pretraining recipe, data, and license can differ.
System
Type
Architecture / data
Availability
Why it matters
Availability is a research-oriented classification, not legal advice. Underlying backbone and dataset licenses may impose additional conditions.
Training substrate
Dataset landscape
Episode count is shown on a logarithmic scale. Scale alone does not capture task entropy, failures, sensor quality, or deployment diversity.
Public corpus scale
Log₁₀ episode scale. “1M+” datasets are displayed at one million for comparison.
What quality actually means
CoverageTasks, scenes, objects, embodiments, operators, and rare states.
PrecisionCalibration, timestamps, action-state alignment, and sensor integrity.
OutcomesFailures, near misses, recovery, progress, interventions, and success labels.
RightsCommercial use, redistribution, human privacy, and derivative models.
Dataset
Scale
Coverage
Primary value
Caveats
Evaluation
Benchmarks and reported leaders
Choose a benchmark to inspect its metric and selected aggregated results. Scores across benchmarks are intentionally not normalized into one ranking.
Reported results
Interpret locally
Reproducibility warning: the 2026 manipulation benchmark audit found shortcut solvability, weak statistical significance, overfitting, and data-source dependence. vla-eval also surfaced hidden preprocessing, normalization, and termination requirements.
RoboArenaDistributed, real-world, double-blind A/B evaluation on the DROID platform.
vla-evalUnified evaluation harness across 16+ simulation benchmarks.
RoboDojoUnified sim-and-real evaluation across generalization, precision, horizon, and memory.
SafeVLA-BenchTracks task performance together with safety behavior.
What comes next
Research agenda
The bottlenecks that matter after scaling vision-language backbones.
Recommended open stack
Fast iteration
LeRobot + SmolVLA + ManiSkill. Low-cost hardware and short training loops for architecture and data experiments.
General manipulation
π0.5/openpi or OpenVLA-OFT, evaluated through vla-eval and target-platform real trials.
Humanoid / bimanual
GR00T N1.7 or MolmoAct2 with Isaac Lab, RoboTwin, real intervention collection, and hard safety control.
Source document
Full research report
The complete 6,000+ word analysis with 77 source links.