π₀.₅: a Vision-Language-Action Model with Open-World Generalization
1. Paper overview
In one sentence: A shared policy transfers heterogeneous supervision into real-home control through discrete pre-training and a continuous action expert, while the incremental value of explicit runtime hierarchy remains uncertain. e02e03e04e05e11e12e17
| At a glance | What to know |
|---|---|
| Research problem | Source description Cleaning an unfamiliar home requires both physical skill transfer and scene-level decisions about what to manipulate next. The paper asks whether heterogeneous co-training can make long-horizon mobile manipulation generalize beyond training environments, using about 400 hours of directly relevant mobile-robot demonstrations alongside much larger additional data sources. e02 |
| Core mechanism | |
| A key reported result | Dishes in Sink — VLA comparison: Approximately 90–95% for π0.5, read from Figure 10’s bar rather than a numeric table. Average task progress, percent of rubric points; higher is better.. Held-out mock homes; same robot action dataset across models; roughly ten trials per task under Appendix C. Approximately 55% for π0-FAST+Flow, 40% for π0 300k, and 20% for π0 80k, also plot estimates. A large observed gain on executed manipulation. π0-FAST+Flow lacks HL and WD supervision; the comparison changes the training recipe as well as inference behavior. e15e12 |
| Reading caution | Source description Unfamiliar or mechanically difficult handles, occluded spills and repeated drawer opening/closing remain failure modes. Simple prompts, modest context and limited memory constrain richer preferences and navigation between rooms. e20 |
Core contributions
Figure 3. One shared policy learns with discrete tokens, then gains a continuous action expert. Original paper, p. 4 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the left half as a collection of training example types: a prompt and images can lead to a subtask label, FAST action tokens, or a caption. These examples share the pre-trained vision-language backbone. The dashed divider separates this stage from post-training and inference on the right. Follow the predicted “pick up the pillow” subtask down to the low-level command: it becomes the condition for producing the action chunk. Noise enters the separate action-expert block, whose output is continuous action values. Section 3.1 makes this dependency precise: the low-level distribution sees the subtask and observation, rather than the original high-level goal directly. e04e05e06e09
What it supports. The architecture lets diverse supervision train a common policy before continuous action generation is introduced. The token-only stage uses α = 0; post-training adds the flow loss with α = 10. At deployment, text decoding and ten denoising steps connect semantic selection to physical control.
Where the evidence stops. This diagram is schematic, not a timing or gradient-flow specification. Its Gemma label says 2.6B, whereas Appendix B.1 calls the VLM 2B; the supplied text does not reconcile that parameter-count difference.
2. Motivation
2.1 The problem and the proposed response
Cleaning an unfamiliar home requires both physical skill transfer and scene-level decisions about what to manipulate next. The paper asks whether heterogeneous co-training can make long-horizon mobile manipulation generalize beyond training environments, using about 400 hours of directly relevant mobile-robot demonstrations alongside much larger additional data sources. e02
2.2 What this reading follows
An unfamiliar kitchen poses two coupled problems: deciding what should happen next and moving the robot accurately enough to make it happen. π0.5 connects them through a textual subtask predicted by the same model that supports low-level control. Its central design choice is a training recipe that brings robot demonstrations, semantic annotations, web examples and verbal guidance into one policy, then equips that policy with a continuous action expert. This edition follows the information flow, the physical evaluation and the ablations. Read its percentages as rubric-based task progress; the most revealing hierarchy comparison separates subtask training from explicit subtask prediction at deployment. e02e03e04e05e11e12e17
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | VLA |
| Architecture | Dual-system |
| Prediction paradigm | Not applicable |
| Quadrant | Not applicable |
3.1 Evidence-based assessment
Conflicts with the recorded classification
VLA, flow matching and hierarchical behavior are supported. However, Dual-system conflicts if it means separate high- and low-level models: Sections 2 and D.1 explicitly describe the same model, with specialized action-expert weights. No future-observation predictor or inverse-dynamics action extraction is specified, so the recorded Not applicable world-model paradigm/quadrant is appropriate. This assessment preserves the catalog snapshot. e03e04e05e07e17
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Turn diverse supervision into one training language
Begin with the supervision interface rather than the robot hardware. A robot demonstration can supply FAST action tokens; a web example can supply a caption, answer or object location; a labeled household episode can supply a semantic subtask and its associated actions. All become prediction tasks for the shared backbone. Appendix B explains why the data groups are complementary: fixed robots can be transported into more homes, laboratory robots supply varied manipulation tasks, and semantic annotations teach the connection between a broad instruction and a concrete next step. Pre-training uses this broad mixture. Post-training then omits laboratory CE data, filters relevant robot episodes by success and duration, and adds verbal demonstrations from experts steering the learned low-level policy. Those verbal examples supervise subtask decisions, not a separate human-controlled execution mode used for the final autonomous evaluation. e08e09e11
Figure 11. Two action encodings share an observation prefix without reading each other’s targets. Original paper, p. 17 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read vertically to choose a query token and horizontally to identify the keys it can attend to. The dark blocks denote permitted attention, as confirmed by Appendix B.1. Image, prompt and state queries see the full prefix. The triangular FAST block permits only causal token dependencies. In the bottom row, action-expert queries can read the prefix and all expert embeddings, while the light block under FAST prevents access to discrete action targets. Conversely, the light upper-right region prevents VLM queries from attending to the action expert. This query/key orientation matters: the resulting information flow is from the VLM prefix into the expert, not the reverse. e05e07e14e17
What it supports. The mask makes joint discrete and continuous action training compatible with inference that uses only the continuous branch for control. In particular, the action expert cannot solve its training task by reading FAST labels that would be unavailable along this inference path. Both branches still share observation information.
Where the evidence stops. An attention mask specifies permitted forward dependencies; it does not itself establish a stop-gradient boundary or frozen backbone. The paper does not provide a dedicated mask ablation demonstrating how much performance this particular restriction contributes.
5.2 Follow a command from observation to executed movement
At runtime, the high-level pass observes the scene and overall request and predicts a textual subtask. Section 3.1 then conditions the action distribution on that subtask and the observation. This is a shared-model hierarchy, even though the action tokens use specialized expert weights. The action expert refines a noisy action chunk through ten denoising steps; FAST tokens remain part of the training formulation rather than the deployed continuous-action decoding path. Appendix B specifies 50-action chunks and a control system that sends pose and base-velocity targets at 50 Hz to PD controllers. High-level inference uses all four cameras; low-level inference uses the front and wrist views. The paper says high-level updates are less frequent, but does not supply their exact cadence or a complete action-chunk execution schedule. e03e04e05e06e07e10
5.3 Separate transfer, scoring and hierarchy effects
Reader analysis: three different questions sit behind the result plots. Figure 5 tests whether familiar task types transfer to unfamiliar homes; it does not test arbitrary new household goals. Appendix C scores intermediate achievements, so high progress can coexist with incomplete episodes. Figure 8 then asks which training sources support that progress, revealing large aggregate losses when other robot data is removed. Figure 16 asks a narrower question: does explicitly predicting subtasks add value beyond learning from those labels? The implicit-HL condition retains much of the performance, and its difference from the full model has p = 0.144. Finally, inspect the task breakdowns before generalizing aggregate conclusions: web data and high-level policies affect different tasks differently. Together, these results support heterogeneous transfer more decisively than a claim that runtime hierarchy is indispensable. e11e12e14e17e18e19
5.4 Training and inference
During training
Pre-training starts from VLM weights and uses next-token prediction on MM mobile data, ME non-mobile robots in varied homes, CE laboratory/OXE data, HL subtask/bounding-box supervision and WD captioning, VQA and localization. It runs 280k steps at batch 2048. e08e09
Post-training adds a randomly initialized action expert and jointly optimizes token and flow losses for 80k steps, batch 384, α = 10. It retains successful MM/ME episodes below the 90th-percentile task length, relevant HL and WD, and adds VI demonstrations from experts directing the learned low-level policy through language. CE is omitted. e08e09
During inference
Autoregressive subtask decoding is followed by ten flow denoising steps. High-level inference runs less frequently than low-level inference; its exact cadence is not specified. All four cameras support high-level inference, while low-level inference uses the front and wrist cameras. e03e05e10
The robot receives targets at 50 Hz with action chunking. PD controllers track arm, gripper and lift poses and base velocities; no additional trajectory planner or collision detection is used. The policy is conditioned on current observations, without an explicit predicted-world rollout. e04e10
5.5 Implementation flow
- Encode observations
PaliGemma processes image patches, language and discretized proprioception. The 300M-parameter action expert uses separate transformer weights, noisy actions and timestep conditioning through adaptive RMSNorm. Appendix B.1 specifies a 50-action horizon, written H = 49 for the inclusive action interval. e04e06
- Choose and condition on a subtask
The same model first predicts a semantic instruction from the observation and overall goal. Its low-level action distribution then depends on that subtask and the observation, with no direct dependence on the original goal in the stated factorization. e03e04
- Keep action representations separate
The observation/prompt/state prefix has full internal attention. FAST tokens have causal attention; action-expert embeddings attend to the prefix and one another, but not FAST tokens. VLM embeddings cannot attend to the expert. This is an attention restriction, not a stated stop-gradient operation. e07
6. Experiments & results
π0.5 transfers heterogeneous robot, semantic and web supervision into a shared vision-language-action policy for household manipulation in unseen homes. Discrete action tokens support pre-training; a flow-matching expert supplies continuous control after post-training. The experiments demonstrate executed robot task progress, while the hierarchy ablation separates training benefits from the less certain incremental benefit of explicit subtask inference (e02, e04, e05, e11, e17).
The supplied paper presents quantitative results and ablations as original plots, not tables. It contains no quantitative table to crop, so four result/ablation figures supply the empirical visuals alongside two mechanism figures. Means quoted from bar heights are explicitly approximate; the source does not supply an exact numeric table or fully define the error bars. e11e12e13e14e15e16e17e18e19
6.1 Read the original evidence
Figure 5b. The real-home evaluation measures partial task completion in unseen scenes. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start with the groups along the bottom: three real homes appear in green shades, followed by the mock environments in yellow. Read the task printed inside each bar before comparing heights; the paper does not plot every task in every home. The vertical scale is average task progress, so a bar measures earned rubric points rather than the fraction of episodes completed perfectly. Figure 5b says each plotted task/environment pair averages ten trials. Appendix C clarifies the distinction: dishes earn separate points for picking up and placing each object, and the drawer task also scores opening and closing. Thus partial manipulation success contributes to the displayed average. e11e12e08
What it supports. The real-home means span approximately 65–95% task progress, while the mock bars lie around 80–95%, based on visual estimates. This supports the authors’ claim of meaningful executed behavior in unfamiliar homes. The lower kitchen scores in Home 3 also show that transfer quality varies across scenes.
Where the evidence stops. Three homes do not establish unrestricted household reliability, and mock bars are not paired trials in identical settings. B.2’s reference to Figure 5 as showing training homes conflicts with the held-out description in Section 4.1 and the caption; that cross-reference remains unresolved.
Figure 10. The full recipe improves manipulation progress over action-only VLA baselines. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read one task group at a time, matching the four bar colors to the legend. The yellow bar is π0.5, the pale bar adds FAST plus flow matching to π0 without the full semantic co-training recipe, and the green bars are the two π0 training-duration settings. All models receive the same robot action dataset according to Section 4.4, but the full model also benefits from other supervision. The vertical axis is task progress under Appendix C, not binary task success. The dishes group is especially easy to separate visually; laundry has a smaller gap between π0.5 and FAST+Flow, and bed-making remains relatively difficult. e15e12
What it supports. For Dishes in Sink, π0.5 reaches roughly 90–95% progress versus about 55% for FAST+Flow, 40% for π0 300k and 20% for π0 80k. These are bar-height estimates. The observed ordering supports the full recipe’s benefit beyond extending the baseline’s action training.
Where the evidence stops. The comparison changes multiple ingredients, including HL/WD supervision and inference behavior, so it cannot assign the improvement solely to hierarchy. No exact numeric result table or definition of the plotted error bars is supplied.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| Dishes in Sink — VLA comparison Held-out mock homes; same robot action dataset across models; roughly ten trials per task under Appendix C. | Approximately 90–95% for π0.5, read from Figure 10’s bar rather than a numeric table. Average task progress, percent of rubric points; higher is better. | Approximately 55% for π0-FAST+Flow, 40% for π0 300k, and 20% for π0 80k, also plot estimates. A large observed gain on executed manipulation. π0-FAST+Flow lacks HL and WD supervision; the comparison changes the training recipe as well as inference behavior. e15e12 |
| Household cleanup in three unseen real homes Figure 5b reports ten trials per plotted task/environment pair; not every task is plotted in every home. | Real-home bars span approximately 65–95%; the three mock-environment bars span approximately 80–95%. Average task progress (%), not whole-episode success. | Mock settings provide context, not a paired baseline on identical objects and layouts. Supports transfer of familiar task types to new scenes. Section 4.1 describes many evaluated tasks as lasting two to five minutes. e11e12 |
| Training-mixture ablation across four household tasks Held-out mock homes; four tasks with nominally ten trials each. | Approximately 78% for full π0.5. Average rubric task progress (%). | About 72% without WD (p = 0.385), 51% without CE, 54% without ME, and 40% without both CE and ME (each p < 0.001). Values are plot estimates; p-values are printed. Robot-data transfer matters strongly. The aggregate WD comparison is not significant, although object-language and task-specific diagnostics show benefits. e14e18e12 |
| Explicit versus implicit high-level inference Appendix D.1 compares high-level policies using the π0.5 low-level inference process; implicit HL keeps high-level training but omits runtime subtask inference. | Approximately 78% for full π0.5. Average rubric task progress (%) and reported two-sided t-test. | Implicit HL is about 71% (p = 0.144); no HL is about 62% (p = 0.011). Removing VI or WD gives about 60% (p = 0.009 and 0.008). Means are plot estimates. The best observed mean favors explicit hierarchy, but its advantage over implicit HL is not significant at 0.05. Training supervision carries a substantial part of the benefit. e17e12 |
| Location scaling and object-language generalization Separate scaling recipe: pre-training excludes mobile data; post-training uses 3, 12, 22, 53, 82 or 104 locations for 40k steps with matched unique sample counts. | Generally improves with location diversity; the largest setting approaches the in-domain control in Figure 6. Rubric progress, correct-object selection and placement success. | No-pre-training controls fare substantially worse. ID and OOD language tasks differ and must be compared within category. Supports diversity under the stated sample control, not a universal scaling law. Appendix D’s direct-command language evaluation omits high-level inference. e13e16 |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Figure 8. Removing other robots’ data causes the largest aggregate losses. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Use the yellow full-model bar as the reference, then read each removal label. WD means web captioning, question answering and localization data. CE is laboratory cross-embodiment data, while ME supplies non-mobile robot demonstrations from varied home environments. The last bar removes both CE and ME, despite the label’s compact “or” wording. All bars average progress across the four household tasks, with nominally ten trials per task. The printed p-values compare each ablation with the full model. The aggregate is useful for judging the recipe, but Appendix E’s per-task breakdown is necessary to see whether an ingredient helps all tasks similarly. e08e12e14e18
What it supports. The full mean is about 78%, compared with roughly 51% without CE, 54% without ME and 40% without both; all three comparisons print p < 0.001. Removing WD gives about 72%, with p = 0.385. The clearest aggregate evidence therefore concerns transfer from other robot datasets.
Where the evidence stops. A nonsignificant aggregate WD result is not evidence that web data is useless. Figures 9 and 17 show object-language and task-specific effects. Dataset removals also change the training mixture; they do not isolate a single learned representation.
Figure 16. High-level training matters; the extra gain from explicit runtime hierarchy is less certain. Original paper, p. 22 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. First compare the yellow model with “implicit HL,” which retains subtask supervision during training but sends the task prompt directly to low-level inference at runtime. Then inspect “no HL,” which also removes that training supervision. These are different interventions. The remaining bars test verbal-instruction data, web data, or alternative high-level policies while using the π0.5 low-level inference process described in D.1. Read the p-values above the bars alongside their heights. The human policy is called an oracle in the text, but it is still an evaluated condition whose observed mean need not be an upper bound for the learned interface. e17e12e19
What it supports. The full model is about 78% versus roughly 71% for implicit HL. Its printed p = 0.144 does not establish an explicit-hierarchy gain at a 0.05 threshold. No HL is about 62% with p = 0.011; removing VI or WD also significantly reduces the aggregate. The evidence favors the training recipe more strongly than a necessity claim for runtime subtask prediction.
Where the evidence stops. Nominal trial counts are small and cancelled trials are excluded under Appendix C. Figure 18 also shows task-dependent effects. Neither the aggregate ranking nor the human-policy comparison demonstrates universal planning superiority or that every task requires explicit hierarchy.
7. Analysis & limitations
7.1 What the evidence leaves open
Unfamiliar or mechanically difficult handles, occluded spills and repeated drawer opening/closing remain failure modes. Simple prompts, modest context and limited memory constrain richer preferences and navigation between rooms. e20
Task progress awards intermediate credit. Trials cancelled for robot failures, time limits or other causes are excluded; exact per-comparison counts and error-bar definitions are not fully reported. This limits reliability and uncertainty interpretation. e12
The source has unresolved inconsistencies: Figure 3 labels Gemma 2.6B while B.1 says 2B; B.2 refers to Figure 5 when describing training homes despite its held-out evaluation caption. Appendix E calls Dishes in Sink both sensitive and less sensitive to high-level policy choice. These are preserved, not silently reconciled. e04e06e08e11e19
7.2 Questions for discussion
- Would a matched-compute explicit-versus-implicit hierarchy comparison reproduce the small observed gain (e17)?
- How much does task progress change when cancelled episodes count as failures (e12)?
8. Reproducibility audit
8.1 Requirements and known gaps
Reproduction needs the heterogeneous robot datasets, manual subtask/bounding-box annotations, web mixture and verbal demonstrations, plus the two mobile platforms. Preserve per-dataset 1st/99th-percentile action normalization to [−1,1], zero-padding and joint/end-effector control-mode prompts. e08e09e10
Appendix B specifies architecture, augmentation, batch sizes and step counts but leaves optimizer/learning-rate schedules, full mixture weights, accelerator budget, software versions and exact inference scheduling unspecified. It gives no explicit freezing schedule. These prevent an exact training-cost or implementation reconstruction. e06e07e08e09
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Separate the benefit of subtask labels from runtime subtask inference
Reader-proposed check, not performed: reproduce full, implicit-HL and no-HL conditions on identical held-out drawer and dishes scenes. Match robot data, training steps, low-level decoding budget and object layouts; record any remaining training-mixture changes. Interleave policies and report both rubric progress and strict episode completion, with confidence intervals and all cancellations accounted for. Log subtask switches and action latency. If implicit HL remains comparable to full while no HL degrades, that supports a supervision benefit without establishing a runtime hierarchy requirement. A repeatable full-model advantage under the matched controls would strengthen the explicit-inference claim. e17e12e19e10
Check 2: Test whether web supervision improves object selection or physical placement
Reader-proposed check, not performed: train matched full and no-WD variants, retaining identical robot and HL/VI data, action normalization and training budgets. Use Appendix D’s five-object setup with target objects farther away than distractors; evaluate ID and OOD categories separately on the same drawer task to avoid the original task mismatch. Measure correct-object selection and successful placement separately, retaining cancelled trials in a sensitivity analysis. A larger OOD selection loss than placement loss conditional on correct selection would support a semantic-transfer explanation; uniform motor degradation would challenge that explanation. e14e16e08e09e12
8.3 Reading coverage
Visual audit: Visually inspected the title/author page, every original figure (1–18), Equation (1), the architecture and query/key mask, training/data appendices, robot/control details, task rubrics, language protocols and per-task results. All six final original crops were viewed; the model-comparison crop was corrected to retain the full y-axis label. Figure 3’s subtask arrow and Figure 11’s mask agree with the stated conditioning and attention rules. Parameter-count, Figure 5 cross-reference and Appendix E wording inconsistencies are disclosed rather than repaired. Reference-only pages 10–15 were read in the complete text pass but not rendered; no scientific claim here depends on those references as independently inspected works. Separate external supplements, videos, code and datasets remain outside this reading.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9, 16, 17, 18, 19, 20, 21, 22, 23, 24. Appendix coverage: reviewed.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Abstract and title page
- 1 Introduction
- 2 Related Work
- 3 The π0.5 Model and Training Recipe, including 3.1–3.3
- 4 Experimental Evaluation, including 4.1–4.4
- 5 Conclusions
- 6 Limitations and Future Work
- References
- A Preliminaries
- B Supplemental for Section 3, including B.1–B.4
- C Task evaluation rubric
- D Language following experiment setup and D.1
- E Per-task performance breakdown
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Separate supplemental material availability has not been fully verified.
- All eight supplied text chunks were read individually, covering all 24 PDF pages. Text extraction does not reconstruct figure images; the retained PDF was therefore inspected separately for every figure and all claim-supporting pages.
- Separate supplemental material availability has not been fully verified. No external website, code, dataset or video was inspected, and no experiments were reproduced.
- Identity/version: the inspected title page identifies the CoRL 2025 edition. The observed title is π0.5: a Vision-Language-Action Model with Open-World Generalization; the catalog uses subscript Unicode typography for the same title. The title page lists Michael Equi rather than the catalog’s Michael Robert Equi, and Brian Ichter rather than brian ichter. The remaining author sequence agrees. No revision number, revision date or comparison with another edition is established by the supplied material (e01).
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e01PDF p. 1, title, author block and conference footer
Title and 35-author sequence identify π0.5 and Physical Intelligence, with the CoRL 2025 footer. Michael Equi and Brian Ichter are the observed spellings; no numbered revision is printed.
Go to primary source ↓e02PDF p. 2, Section 1, paragraphs 3–4
The system combines about 400 hours of mobile manipulation with heterogeneous sources; 97.6% of first-phase training examples are not household mobile-manipulation examples.
Go to primary source ↓e03PDF p. 3, Section 2, Robot reasoning and planning with language
The same model performs high- and low-level inference; high-level inference operates at lower frequency.
Go to primary source ↓e04PDF p. 4, Figure 3 and Section 3.1 factorization
Two training stages and subtask-to-action conditioning are described. The action distribution conditions on the subtask rather than the overall goal. Figure 3 visibly labels SigLIP 400M, Gemma 2.6B and an action expert of 300M.
Go to primary source ↓e05PDF p. 5, Section 3.2, Equation (1) and surrounding paragraphs
FAST token cross-entropy and flow-matching squared error are combined. The printed interpolation is τa+(1−τ)ω and target ω−a. Pre-training sets α=0; inference uses autoregressive text and ten denoising steps.
Go to primary source ↓e06PDF pp. 16–17, Appendix A and B.1
PaliGemma initialization, 50-action chunks with H=49, adaptive RMSNorm timestep injection, and a 300M expert are specified. The text describes a 2B VLM, unlike Figure 3’s 2.6B label.
Go to primary source ↓e07PDF p. 17, Figure 11 and Appendix B.1 attention-mask paragraph
Rows are queries and columns keys. The prefix is fully connected; FAST is causal; expert tokens attend to the prefix and each other, never FAST. No VLM embedding attends to the expert. Subsequent paragraphs specify timestep sampling and image augmentations.
Go to primary source ↓e08PDF pp. 17–19, Appendix B.2 and Figure 12
MM, ME, CE including OXE, HL labels/bounding boxes and WD are defined. Figure 12 separates pre- and post-training mixtures. B.2’s MM paragraph refers to Figure 5 as showing some training homes; the report preserves the inconsistency with that figure’s held-out caption.
Go to primary source ↓e09PDF pp. 5–6, Section 3.3; PDF p. 19, Appendix B.2–B.3
Pre-training uses 280k steps, batch 2048; post-training uses 80k steps, batch 384, α=10 and a randomly initialized expert. Successful MM/ME episodes below the 90th-percentile task length are retained; WD, relevant HL and VI are included. Action normalization and padding are specified.
Go to primary source ↓e10PDF p. 19, Appendix B.4; PDF p. 20, Figure 13
Two mobile platforms have 18/19-dimensional state/action spaces, dual six-DoF arms, grippers, torso lift, holonomic base and four RGB cameras. Low-level inference omits the rear camera. Targets run at 50 Hz using PD tracking without added planning or collision detection.
Go to primary source ↓e11PDF p. 6, Section 4.1; PDF p. 7, Figure 5a–b and caption
Three unseen real homes are evaluated. Each plotted pair averages ten trials; green real-home bars range approximately 65–95% progress and yellow mock bars approximately 80–95%. Many evaluated tasks last two to five minutes.
Go to primary source ↓e12PDF pp. 19–21, Appendix C
Scores are percentages of task-specific points: dishes 8, drawer 4, laundry 3 and bed 5. Standard comparisons nominally use ten episodes per task and 40 per policy, interleaved. Cancelled episodes are removed; plots report two-sided t-tests with variable trial numbers. Exact exclusions and error-bar definitions are not fully supplied.
Go to primary source ↓e13PDF pp. 6–7, Section 4.2 and Figures 6–7
The scaling experiment uses six location counts from 3 to 104, mobile-free pre-training, and 40k post-training steps with matched unique samples. Progress and language metrics generally improve; the largest setting is comparable to the in-domain control.
Go to primary source ↓e14PDF p. 8, Section 4.3 and Figures 8–9
Figure 8 shows roughly 78/72/51/54/40% for full/no WD/no CE/no ME/no CE or ME. Printed p-values are 0.385 for no WD and <0.001 for each robot-data removal. Figure 9 separates ID/OOD follow and success rates and warns that their tasks differ.
Go to primary source ↓e15PDF p. 8, Section 4.4 and Figure 10, all task groups, especially Dishes in Sink
For Dishes in Sink, π0.5 is roughly 90–95%, versus about 55/40/20% for FAST+Flow/π0 300k/π0 80k. The full model has the highest mean in each group; its gap over FAST+Flow is smaller for laundry, and bed-making scores are lower. All receive the same robot action data; FAST+Flow omits HL and WD. Values are plot estimates, not exact tabulated means.
Go to primary source ↓e16PDF pp. 21–22, Appendix D and Figures 14–15
Two unseen kitchen scenes test selection among five objects with the target farther away; ID tasks include drawer and sink, OOD uses novel drawer objects. Selection and successful placement are separate metrics. The model-class language comparison does not use high-level inference.
Go to primary source ↓e17PDF pp. 22–23, Appendix D.1 and Figure 16
Full π0.5 averages about 78%; implicit HL about 71%, no HL about 62%, no VI/no WD about 60%. Printed p-values versus full are 0.144/0.011/0.009/0.008; GPT-4 HL and human HL have p=0.002/0.016. VI is about 11% of high-level mobile examples. Training and runtime HL removals are distinguished.
Go to primary source ↓e18PDF p. 23, Figure 17 and Appendix E, Co-training recipe ablations
Effects vary by task. In Figure 17, removing WD hurts Items in Drawer but not Dishes in Sink; removing ME/CE markedly reduces kitchen-task performance. The caption and analysis report cross-embodiment benefits.
Go to primary source ↓e19PDF p. 24, Figure 18 and Appendix E final paragraph
Per-task high-level effects vary. Laundry scores are relatively similar, while kitchen and bed tasks show different sensitivities. The text describes Dishes in Sink as both strongly affected by no HL and less sensitive to high-level policy choice.
Go to primary source ↓e20PDF p. 9, Section 6 Limitations and Future Work
The authors describe difficult handles, partial observability, distracted subtask selection, simple prompts and limited context/memory as remaining limitations.
Go to primary source ↓8.5 Primary sources
π0.5: a Vision-Language-Action Model with Open-World Generalization ↗
PDF · 14,099 extracted words
Source fingerprint
8eadc5ada5ab7f9d1ba6b472125eff391249f5403b8e0fa22c976c9ed00bbfc4