OverviewRobotic foundation models for physical intelligence
Trung Pham ↗
Landscape report · 2026

Robots are getting foundation models. The stack matters more than the label.

An interactive map of the architectures, data engines, open and closed systems, reinforcement learning, simulation, and evaluation shaping physical intelligence.

3
distinct model layers: reasoner, policy, world model
12
substantially open model families reviewed
1M+
episodes in the largest public robot corpora
77
primary and official source links
Architecture

The two-speed VLA

A pretrained vision-language backbone supplies semantics while a smaller diffusion or flow expert produces high-rate continuous action chunks.

Data

Deployment is the moat

Public pretraining can create a strong base. Failures, corrections, interventions, and success labels from deployment create durable advantage.

Evaluation

No universal leaderboard

Hardware, adaptation data, preprocessing, reset rules, and task distributions differ too much for a single transitive ranking.

Reason and planLanguage, spatial understanding, memory, tools, task decomposition · roughly 0.1–2 Hz
→
Perceive and actClosed-loop VLA action chunks conditioned on images, state, and goals · roughly 1–10 Hz
→
Stabilize and protectTorque, balance, collision response, impedance, and hard safety limits · 50–1,000+ Hz
Bottom line: the field is moving from behavior cloning alone toward heterogeneous pretraining followed by deployment-driven corrections, learned rewards, and reinforcement learning. But useful progress is measured by acquisition speed, reliability, recovery, and human labor—not one simulation score.
Model design

Architecture patterns

Select a family to compare its action representation, strengths, failure modes, and representative systems.

Cross-embodiment transfer

The same policy must reconcile morphology, coordinate frames, cameras, action dimensions, and control rates.

Relative EEF actionsExpress movement as deltas from current pose; GR00T N1.7 uses this to align human and robot motion.
Embodiment promptsLearn small domain-specific tokens; X-VLA adapts hardware while freezing most weights.
Shared padded spacesMap variable robot states and actions into a maximum-dimensional vector with masks.
Latent actionsLearn unlabeled motion abstractions from video, then ground them to a target controller.
Open and proprietary

Model landscape

“Open” is artifact-specific: weights, inference code, fine-tuning, pretraining recipe, data, and license can differ.

SystemTypeArchitecture / dataAvailabilityWhy it matters

Availability is a research-oriented classification, not legal advice. Underlying backbone and dataset licenses may impose additional conditions.

Training substrate

Dataset landscape

Episode count is shown on a logarithmic scale. Scale alone does not capture task entropy, failures, sensor quality, or deployment diversity.

Public corpus scale

Log₁₀ episode scale. “1M+” datasets are displayed at one million for comparison.

What quality actually means

CoverageTasks, scenes, objects, embodiments, operators, and rare states.
PrecisionCalibration, timestamps, action-state alignment, and sensor integrity.
OutcomesFailures, near misses, recovery, progress, interventions, and success labels.
RightsCommercial use, redistribution, human privacy, and derivative models.
DatasetScaleCoveragePrimary valueCaveats
Evaluation

Benchmarks and reported leaders

Choose a benchmark to inspect its metric and selected aggregated results. Scores across benchmarks are intentionally not normalized into one ranking.

Reported results

Interpret locally

Reproducibility warning: the 2026 manipulation benchmark audit found shortcut solvability, weak statistical significance, overfitting, and data-source dependence. vla-eval also surfaced hidden preprocessing, normalization, and termination requirements.
RoboArenaDistributed, real-world, double-blind A/B evaluation on the DROID platform.
vla-evalUnified evaluation harness across 16+ simulation benchmarks.
RoboDojoUnified sim-and-real evaluation across generalization, precision, horizon, and memory.
SafeVLA-BenchTracks task performance together with safety behavior.
What comes next

Research agenda

The bottlenecks that matter after scaling vision-language backbones.

Recommended open stack

Fast iteration

LeRobot + SmolVLA + ManiSkill. Low-cost hardware and short training loops for architecture and data experiments.

General manipulation

π0.5/openpi or OpenVLA-OFT, evaluated through vla-eval and target-platform real trials.

Humanoid / bimanual

GR00T N1.7 or MolmoAct2 with Isaac Lab, RoboTwin, real intervention collection, and hard safety control.

Source document

Full research report

The complete 6,000+ word analysis with 77 source links.

Loading report…