UniVLA: Learning to Act Anywhere with Task-centric Latent Actions
1. Paper overview
In one sentence: UniVLA learns task-oriented action tokens from videos to share policy knowledge across embodiments, while retaining supervised control adaptation and a fixed temporal bottleneck. e-identitye-probleme-lame-policye-decodee-limitations
| At a glance | What to know |
|---|---|
| Research problem | Source description Robot action labels depend on embodiment, while raw video reconstruction also rewards encoding camera shifts, lighting and other agents. UniVLA asks whether a compact, task-oriented action vocabulary can let one policy learn useful behavior across robot, navigation and human videos. e-problem |
| Core mechanism | Source description Language-conditioned latent learning first allocates a codebook to residual visual changes, then learns a separate task-centric codebook used as policy supervision. e-lam |
| A key reported result | LIBERO four-suite manipulation: Full: 95.2; Bridge-only: 92.5; Human-only: 88.7. Average task success (%). Suite-specific supervised adaptation; ten tasks and fifty demonstrations each; 500 trials per suite, averaged over three seeds. OpenVLA: 76.5; LAPA: 65.7. Full exceeds OpenVLA by 18.7 percentage points from the cells. Pretraining excludes LIBERO; this is not zero-shot task execution. Wrist-view access differs for some baselines. e-libero |
| Reading caution | Source description Fixed latent granularity/codebook capacity may limit dexterous or multi-arm control. Most instructions describe short actions; extending to coarse goals is less established. These limitations are explicitly acknowledged. e-limitations |
Core contributions
- Source description
Language-conditioned latent learning first allocates a codebook to residual visual changes, then learns a separate task-centric codebook used as policy supervision. e-lam
Figure 2. Language supplies task context before a second codebook learns to replace it. Original paper, p. 4 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read each stage from the frame pair at the top to the spatial decoder below. Blue tokens carry DINOv2 features; snowflakes mark frozen DINOv2 and T5 encoders. In Stage 1, yellow language tokens enter both encoder and decoder, while green tokens pass through latent quantization. The authors intend the green bottleneck to represent residual, task-irrelevant variation. The weight-copy arrows lead to Stage 2, where red task-centric queries replace language. Section III-A explicitly freezes the old codebook; it does not say that every copied transformer weight is frozen. Both types of quantized code appear in the Stage-2 decoder. e-lame-policye-latent-ablation
What it supports. The central design allocates representational capacity before policy learning: only the task-centric labels become the policy’s action targets. The graphic illustrates the intended division of information, while the actual downstream usefulness of the resulting codes is tested separately in Table III. It is not itself evidence of perfectly separated causes of motion.
Where the evidence stops. Section III-A first says the decoder receives no historical frames, yet its equations explicitly include current features and the figure shows a blue context path. We follow the equations and disclose this unresolved contradiction; the two blue output groups do not establish an additional loss beyond the specified future-feature objective.
2. Motivation
2.1 The problem and the proposed response
Robot action labels depend on embodiment, while raw video reconstruction also rewards encoding camera shifts, lighting and other agents. UniVLA asks whether a compact, task-oriented action vocabulary can let one policy learn useful behavior across robot, navigation and human videos. e-problem
2.2 What this reading follows
A video contains more motion than a robot should imitate: the camera can shake, another agent can move, and objects unrelated to the instruction can appear. UniVLA tries to separate those changes before learning its policy. Language first helps a latent model allocate residual visual information to one codebook; a second codebook then takes over the task-related role. A vision-language policy learns to predict the second set of codes, and a supervised action head turns its internal features into controls. The six excerpts below connect this training recipe to deployment results and ablations, while preserving disagreements within the supplied v3 paper. e-identitye-probleme-lame-policye-decodee-limitations
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | Not assigned |
| Architecture | Not assigned |
| Prediction paradigm | Not assigned |
| Quadrant | Not assigned |
This table preserves the labels recorded at reading time. The current major category is VLA. View the current classification.
3.1 Evidence-based assessment
Insufficient evidence to decide
The recorded taxonomy is unassigned, so there is no quadrant assertion to confirm. UniVLA uses a separately learned inverse/forward latent-action model, an autoregressive VLM and control heads. Joint future/action prediction within one deployed model is not demonstrated; world-model planning is future work. This inverse-dynamics-derived supervision should be distinguished from both joint prediction and inference-time world-model control. e-lame-policye-decodee-limitations
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 1. Give the nuisance codebook something useful to do
Reader interpretation: the first training stage makes language a competing information channel. If the decoder already knows which action the instruction describes, its limited codebook has less reason to spend capacity on that same information. The authors therefore expect it to retain residual variation, including camera motion and unrelated scene changes. Stage 2 removes the language channel while preserving the old codebook and adds a new one. The intended pressure is for the new codes to supply the missing task information. This is a training bias, not a guarantee that each code has a single causal meaning. Figure 2 explains the construction; Table III supplies the stronger behavioral test by using either code family as policy supervision. The historical-frame inconsistency remains relevant to any implementation. e-lame-latent-ablation
Figure 3. A shared latent vocabulary still needs visual context and an embodiment-specific control head. Original paper, p. 5 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Follow the diagram upward from third-view RGB and the task instruction. DINOv2 and SigLIP supply visual features, while the tokenizer supplies language tokens. The transformer produces latent-action outputs, and the detokenizer branch exposes the discrete ACT vocabulary. The purple action decoder receives blue visual information at its Q arrow and red latent embeddings at its KV arrow. Section III-C makes this precise: pool the visual embeddings first, use that context in an attention query over action embeddings, then project to an action chunk. Thus the head uses contextualized features, not merely an integer code looked up in a fixed motor table. e-policye-decodee-historye-real-architecturee-probleme-decoder-ablation
What it supports. The architecture separates a vocabulary shared across data sources from controls specialized to a robot. Four predicted latent tokens summarize the next action interval; the control head can output a chunk appropriate to the target system. The Q/KV directions agree with the two attention equations and explain why the visual-query ablation matters.
Where the evidence stops. Figure 3 omits the additional proprioception branch used in physical experiments, documented in Appendix C.2. It also omits historical prompt tokens. The introduction’s 10.8M decoder count conflicts with the 12.6M head reported later; neither should be presented as a universal reconciled configuration.
5.2 2. Separate knowing the action from specifying its controls
Reader interpretation: a shared action code can be useful across embodiments precisely because it need not encode one robot’s motor coordinates. That leaves a grounding problem at deployment: the same task-level motion can require different controls in different scenes. UniVLA addresses this using the final-layer visual embeddings as context for attention over latent-action embeddings, then projects the result into the target action space. The policy continues to predict four latent tokens, while the supervised head learns action chunks using low-level labels. Appendix C adds proprioception for physical experiments. This explains why the visual-query ablation is informative: removing scene context from the query lowers average LIBERO success even when the proposed decoding head remains. It also limits the action-free claim to the pretraining stages. e-policye-decodee-real-architecturee-decoder-ablatione-pretrain
5.3 3. Trace the feedback loop without inventing a planner
The evaluated rollout carries one previous group of latent-action tokens into the next prompt, alongside the current observation and instruction. During post-training, the historical codes come from observed frame pairs; at inference, the policy uses its earlier latent outputs. Reader interpretation: this compact history may communicate task progress without repeatedly encoding past images, but it also creates a possible mismatch between annotated history and mistaken model outputs. Table V supports the practical value of the design: Long success rises from 88.1% to 92.0%, and R2R oracle success from 30.6% to 47.1%. These results do not establish new-skill learning from demonstrations in context. Nor does the policy use the feature decoder to search imagined futures: that extension appears in the future-work discussion. e-historye-nave-limitations
5.4 Training and inference
During training
Pretraining uses Open X-Embodiment, GNM and Ego4D frames/instructions, excluding available actions and proprioception. Policy modules are jointly optimized for 20,000 steps with batch 1,024 and learning rate 2e-5: about 30 hours on 32 A100s. The 960 versus 21,500 A100-hour comparison concerns reported policy pretraining; it does not establish matched total development cost. e-pretraine-policy
Downstream LoRA adaptation jointly minimizes latent-token negative log-likelihood and L1 error against ground-truth low-level actions. LIBERO uses suite-specific behavioral cloning; thus action-free pretraining does not imply action-free deployment. e-decodee-libero
During inference
At each step, process current RGB and the instruction augmented with the previous four-token latent output, except initially. Generate latent actions, decode a control chunk, execute controls and repeat with feedback. Physical experiments report 10 Hz on an RTX 4090 with chunk size 12. The exact executed-prefix/replanning schedule is not specified. e-historye-decodee-real-setup
The evaluated policy does not roll out the forward model to rank imagined futures. Planning trees and world-model-assisted refinement are future work. e-limitations
5.5 Implementation flow
- Extract temporal features
Choose frames about one second apart, adjusting the frame interval to dataset frequency. Frozen DINOv2 encodes spatial patches; a spatial-temporal inverse-dynamics encoder maps paired features and learnable action queries to quantized latents. A spatial decoder learns feature reconstruction. e-lam
- Separate residual and task information
Stage 1 provides frozen T5 language to encoder and decoder, encouraging limited-capacity latents to carry remaining visual variation. Stage 2 copies model weights, removes language, freezes the task-irrelevant codebook and adds a task-centric codebook. Reader analysis: semantic separation is an inductive bias, not a proved identifiability result. e-lame-latent-ablation
- Predict codes and decode controls
Prismatic-7B fuses DINOv2/SigLIP features with instructions and predicts codebook-index ACT tokens. The deployment head pools visual embeddings, uses them to query final-layer latent embeddings, then projects into action chunks. The real-world variant also projects proprioceptive state. e-policye-decodee-real-architecture
6. Experiments & results
UniVLA learns a discrete action vocabulary from language-annotated videos, teaches a vision-language policy to predict that vocabulary, and adapts a visual-conditioned head to physical controls. Its distinguishing idea is to separate task-relevant motion from distracting changes before policy pretraining. Strong manipulation and navigation results support transfer, but deployment still needs action-labeled adaptation; future-observation prediction trains the latent representation rather than serving as the evaluated inference-time planner (e-lam, e-policy, e-decode, e-libero, e-nav, e-limitations).
6.1 Read the original evidence
Table I. Transfer from video pretraining improves all four LIBERO suites after supervised adaptation. Original paper, p. 6 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the four task-suite columns before the Average column. Spatial changes layout, Object changes objects, Goal changes objectives, and Long requires multi-step manipulation. The three UniVLA rows vary pretraining data: human videos, Bridge, or the full mixture. They all undergo downstream supervised training, so the Human row does not represent a policy executing LIBERO directly from human videos. The footnotes also matter: MDT and MaIL receive additional wrist-camera inputs, while LAPA was reproduced using Prismatic-7B. Section IV-A.1 reports 500 evaluation trials per suite and averages three seeds; it excludes LIBERO from both latent-model and policy pretraining. e-liberoe-pretraine-lam
What it supports. The Full row averages 95.2% compared with OpenVLA’s 76.5%, a calculated 18.7-percentage-point gap. Bridge-only reaches 92.5% and Human-only 88.7%. These comparisons support the usefulness of pretraining across data sources, including human video, followed by target-suite adaptation. They do not establish zero-shot task execution.
Where the evidence stops. The introduction says 18.5 for the OpenVLA gap; the cells support 18.7 percentage points. No per-seed uncertainty is displayed. Different pretraining corpora, input modalities and policy heads mean this table alone cannot isolate the latent-separation mechanism.
Table A-V. Physical transfer is substantial, but task completion and partial progress tell different stories. Original paper, p. 18 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Each task has two columns: Succ. is the reported success percentage, while Score measures progress using the three subgoals listed immediately above in Table A-IV. Compare these separately rather than treating the score as a percentage. The bottom three rows expand UniVLA’s pretraining sources from Bridge to OXE to the full mixture. The physical setup uses a Piper arm, third-view RGB and a proprioception branch in the control head. The generalist models are adapted jointly across tasks; Diffusion Policy is trained separately for each task. Reading the towel columns reveals an exception that the overall average hides. e-real-resultse-real-setupe-real-architecturee-scaling
What it supports. Full UniVLA reports 81.7% success and 2.63 average score, versus LAPA’s 45.0% and 1.95. Thus the reported 36.7-point success and 0.68-score gains are over LAPA. Diffusion Policy has higher towel completion, 53.3% versus 46.7%, despite lower partial-progress score, 2.33 versus 2.47.
Where the evidence stops. Per-task rollout counts and uncertainty are absent. Cleaning is listed as 100% successful with only 2.33/3 average score, an unexplained mismatch between the metrics. Figure 9 also shows 2.64 for the full-model robot score; this excerpt preserves Table A-V’s 2.63.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| LIBERO four-suite manipulation Suite-specific supervised adaptation; ten tasks and fifty demonstrations each; 500 trials per suite, averaged over three seeds. | Full: 95.2; Bridge-only: 92.5; Human-only: 88.7. Average task success (%) | OpenVLA: 76.5; LAPA: 65.7. Full exceeds OpenVLA by 18.7 percentage points from the cells. Pretraining excludes LIBERO; this is not zero-shot task execution. Wrist-view access differs for some baselines. e-libero |
| R2R continuous navigation 10,819 training examples; 1,839 val-unseen examples; RGB-only. | 47.1 Oracle success within three metres (%) | OpenVLA: 17.5; NaVid: 49.1. Oracle arrival does not establish reliable stopping or path efficiency. NaVid uses historical images; LLaVA-Nav uses a smaller evaluation subset. e-nav |
| Physical Piper manipulation Four tasks after 20–80 adaptation trajectories per task; third-view RGB plus proprioception in the head. | 81.7 and 2.63. Mean success (%) and subgoal score (out of 3) | LAPA: 45.0/1.95; OpenVLA: 38.3/1.63. The 36.7-point gain is over LAPA. Towel success remains below single-task Diffusion Policy, 46.7 versus 53.3. Generalization variants average 68.9%/2.49; physical rollout counts and intervals are absent. e-real-setupe-real-architecturee-real-resultse-generalization |
| Latent-supervision ablation Ego4D-only policy pretraining followed by LIBERO adaptation. | Task-centric: 88.7 / 79.4. LIBERO average / Long success (%) | Genie-style: 82.3 / 69.6; task-irrelevant: 56.5 / 0.2. The 6.4-point average gain supports the training representation; it does not separately isolate language, feature space and every quantization choice. e-latent-ablation |
| Decoder and history ablations LIBERO decoding comparison; instruction-only versus historical-latent prompts in LIBERO and R2R. | Full decoder average 95.2; history raises Long 88.1 to 92.0 and R2R 30.6 to 47.1. Success (%) | Decoder without visual queries: 92.5 average; autoregressive controls: 73.6. Decoder cells imply 2.7 average points from visual queries and 43.0 Long points over autoregression, differing from prose. History gains do not by themselves prove in-context skill acquisition. e-decoder-ablatione-history |
| CALVIN ABC-to-D instruction chains Train environments A/B/C; test 1,000 five-task chains in D. | 56.5 / 3.80. Five-task success (%) / mean completed chain length | OpenVLA: 43.5 / 3.27; RoboDual: 54.4 / 3.66. Training steps and observation modalities differ. The prose’s prior-best CLOVER comparison overlooks the stronger RoboDual row. e-calvin |
| SimplerEnv WidowX/Bridge manipulation Four simulated tasks; 24 trials per task stated; Bridge is present in pretraining. | Table A-III: 47.9; decoder-only: 35.4. Conflicting prose: 42.7. Aggregate task success (%) | OpenVLA: 1.0; block-stacking success is only 2.8 for full UniVLA. These are table-reported values with unresolved prose disagreement, not unseen-domain transfer or physical success. Trial aggregation behind fractional rates is unspecified. e-simplere-pretrain |
| LIBERO downstream demonstration efficiency Use 10% of LIBERO-Goal demonstrations for adaptation. | 86.3 Task success (%) | OpenVLA with full demonstrations: 79.2. This supports the specific data-efficiency comparison; it does not show that every task needs one tenth as much data. e-data-efficiency |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Table III. Useful video supervision depends on which latent information is predicted. Original paper, p. 10 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. The rows name ways of constructing latent labels for policy pretraining on Ego4D. Here, Genie identifies the compared latent-action construction approach; the table is not a benchmark of interactive video generation. Each column measures later LIBERO task success. Read the task-irrelevant row as a diagnostic intervention: making nuisance codes the prediction target gives a much weaker policy, especially on Long. Then compare the task-centric row against the Genie-style row to see whether separating useful motion adds value beyond learning undifferentiated latent actions. This comparison connects the two colored codebooks in Figure 2 to executed simulated manipulation. e-latent-ablatione-lame-libero
What it supports. Task-centric supervision reaches 88.7% average success versus 82.3% for Genie-style supervision, a 6.4-point gain. On Long, the corresponding scores are 79.4% and 69.6%; nuisance-only supervision falls to 0.2%. The result supports choosing task-relevant labels for policy learning, particularly when task progress spans multiple actions.
Where the evidence stops. The table shows downstream utility, not a direct measurement of semantic disentanglement. It does not independently isolate language conditioning, DINO features and codebook design. Unreported optimization or capacity differences would need control in a reproduction before assigning the gain to one component.
Table IV. Visual-conditioned chunk decoding contributes beyond the latent vocabulary. Original paper, p. 10 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Begin with what changes between the rows. Auto-regressive means generating discretized low-level action bins sequentially, following the baseline decoding approach. It does not mean that the proposed policy stops predicting latent tokens autoregressively. The middle row keeps the proposed control head but removes visual embeddings as query inputs. The final row restores the visual context used in Figure 3. Read across the Long column to see the large control-decoder difference, then compare the two proposed-head rows to assess the more specific contribution of the visual query. These are downstream success rates, not token accuracy or measured latency. e-decoder-ablatione-decodee-policy
What it supports. The full head reaches 95.2% average success, versus 92.5% without visual queries and 73.6% for autoregressive controls. The table-derived visual-query gain is 2.7 percentage points. Long rises from 49.0% to 92.0% when comparing autoregressive controls with the full head, a 43.0-point difference.
Where the evidence stops. The paragraph beneath the table instead gives gains of 2.2 and 42.1. Those values do not match the displayed cells. Changing the control parameterization can also change timing and optimization, so the large gap is not solely evidence about code semantics.
7. Analysis & limitations
7.1 What the evidence leaves open
Fixed latent granularity/codebook capacity may limit dexterous or multi-arm control. Most instructions describe short actions; extending to coarse goals is less established. These limitations are explicitly acknowledged. e-limitations
Source inconsistencies remain: decoder conditioning versus the no-history sentence; 10.8M versus 12.6M head counts; introduction LIBERO gain 18.5 versus table-derived 18.7; decoder gains 2.2/42.1 versus 2.7/43.0; and SimplerEnv 42.7 versus 47.9. SimplerEnv prose also claims reduced grasp rates contrary to all four grasp columns. e-lame-probleme-real-architecturee-liberoe-decoder-ablatione-simpler
Figure 9’s full-model R2R value 49 and robot score 2.64 conflict with 47.1 and 2.63 elsewhere. Table A-V’s cleaning success 100% alongside score 2.33/3 is unexplained. Selected same-code images and attention maps support plausibility, but do not quantify disentanglement or establish causal attention. e-scalinge-nave-real-resultse-qualitative
7.2 Questions for discussion
- Would task-centric codes remain useful when short action instructions are replaced by coarse goals (e-limitations, e-lam)?
- Does history help through task-progress information or simply correlated previous controls (e-history)?
8. Reproducibility audit
8.1 Requirements and known gaps
A reproduction needs Table A-I’s mixture, time-aligned frame sampling, DINOv2/T5/Prismatic weights, both latent-training stages and action-labeled adaptation. Latent-model optimization, loss weighting and LoRA rank are not fully specified. Codebook size 16 is illustrated, not supplied as a complete experiment configuration. e-pretraine-lame-policye-decode
For the real-world head, retain both eight-head attention blocks, 64-dimensional heads, hidden size 512, MLP ratio four, one-layer action projection and two-layer proprioception projection. Human/Bridge recipes report batch 258 on eight GPUs and approximately 200 A100-hours; preserve 258 as printed pending clarification. e-real-architecturee-pretrain
Proposed checks: hold data and downstream head fixed while replacing latent supervision; separately compare correct, shuffled and absent history tokens with matched control timing. Report per-task uncertainty and token-prediction accuracy. These test representation usefulness and history dependence, without presuming successful reproduction. e-latent-ablatione-historye-decoder-ablation
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Hold the policy pipeline fixed while changing its video labels
Reader-proposed check, not performed: use the same Ego4D split, frame intervals, policy initialization, training budget, code length and downstream visual decoder for Genie-style, task-irrelevant and task-centric labels. Evaluate the same LIBERO tasks and seeds, recording per-task success intervals and held-out token-prediction accuracy. Add a matched variant with shuffled Stage-1 instructions to test whether language alignment contributes beyond extra training. The central prediction is that correctly aligned task-centric labels outperform both undifferentiated and nuisance labels, particularly on Long. If matched controls erase that advantage, Table III would provide weaker support for the proposed separation mechanism. e-lame-policye-latent-ablatione-libero
Check 2: Test whether history carries progress information or brittle correlations
Reader-proposed check, not performed: compare absent, correct previous-output and shuffled previous-output histories on identical LIBERO-Long and R2R episodes, holding observation access, model weights, decoder and control schedule fixed. Repeat training with each history condition to separate inference corruption from adaptation effects. Measure Long completion and R2R oracle arrival separately from stopping success, and stratify episodes following a failed action. If correctly ordered history reliably helps but shuffled history does not, temporal information is implicated. If erroneous past outputs systematically worsen recovery, the result would expose a boundary of feeding predictions back into the policy. e-historye-decodee-nave-libero
8.3 Reading coverage
Visual audit: Visually inspected the original title/byline and v3 margin on p. 1; all main-paper method, evaluation, ablation and limitation pages through p. 11; and all appendix pages 16–18, including compute, mixture, CALVIN, SimplerEnv, physical scoring and proprioception details. Figures 1–10, A-1/A-2 and Tables I–V, A-I–A-VI were viewed. Figure 2 conditioning paths were checked against both Section III-A equation systems; Figure 3 Q/KV arrows were checked against Section III-C. Every final crop was viewed, including the corrected Table I crop retaining both footnotes. Bibliography-only pages 12–15 were read in text, not visually inspected. Separate video demonstrations, earlier editions and code remain outside this reading. All numerical and method evidence cited by the report and this edition lies on the declared inspected pages.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 16, 17, 18. Appendix coverage: reviewed.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- PDF pp. 1–2: title/byline, abstract and I. Introduction
- PDF pp. 2–3: II. Related Work, A–C
- PDF pp. 3–5: III. Methodology, A–C, all displayed equations
- PDF pp. 5–11: IV. Evaluations, A–C
- PDF p. 11: V. Conclusion; VI. Limitations and Future Work; Acknowledgment
- PDF pp. 11–15: References
- PDF pp. 16–18: Appendix A. Implementation Details; B. Additional Results; C. Real-world Robots
Outside the original text pass
- Identity note: the catalog and supplied observedTitle are “UniVLA: Learning to Act Anywhere with Task-centric Latent Actions”; the inspected title page prints “Learning to Act Anywhere with Task-centric Latent Actions” without the model-name prefix. The exact arXiv identifier, eight-author byline and named UniVLA framework agree. Metadata retains the supplied observedTitle, with the printed-title display difference preserved here; no alternative work was substituted.
- Edition note: the reviewed artifact explicitly identifies arXiv:2505.06111v3, 3 November 2025, later than the catalog submission date of 9 May 2025. Earlier versions and a primary revision history were not supplied, so no claim is made about when the title display changed or about scientific equivalence between revisions.
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Separate supplemental material availability has not been fully verified.
- All eight supplied text chunks were read individually, covering the entire 18-page artifact. Original PDF pages 1–11 and 16–18 were rendered and visually inspected, resolving the extraction-only figure gap. Bibliography-only pages 12–15 were read as text.
- No separate supplements or demonstration videos were supplied for inspection. The linked code was not inspected, and no experiments were run or reproduced.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e-identityPDF p. 1, title, byline, affiliations, arXiv margin and Figure 1
The printed title is Learning to Act Anywhere with Task-centric Latent Actions. The margin identifies arXiv:2505.06111v3 [cs.RO], 3 Nov 2025. The eight authors are Qingwen Bu, Yanting Yang, Jisong Cai, Shenyuan Gao, Guanghui Ren, Maoqing Yao, Ping Luo and Hongyang Li. The byline lists The University of Hong Kong, OpenDriveLab and AgiBot; Figure 1 and the abstract name the framework UniVLA.
Go to primary source ↓e-problemPDF pp. 1–2, Abstract and Section I
The paper motivates learning transferable actions from heterogeneous videos without ground-truth action labels, while filtering camera motion and other task-irrelevant changes. The introduction describes three stages and names a 10.8M-parameter decoder.
Go to primary source ↓e-lamPDF pp. 3–4, Section III-A, both unnumbered Encode/Decode systems and Figure 2
Frames are separated by approximately one second using dataset-specific frame intervals. DINOv2 patch features are inputs and reconstruction targets; the encoder is a spatial-temporal transformer and the decoder a spatial transformer. Stage 1 conditions on T5 language and learns the task-irrelevant codebook; Stage 2 replaces language with new task-centric tokens and freezes the old codebook. Figure 2 marks DINOv2 and T5 with snowflakes and copies encoder/decoder weights between stages. Despite the p. 3 statement that no historical frames enter the decoder, both displayed decoding systems condition on O_t and Figure 2 contains a blue context path. The specified reconstruction error targets future features.
Go to primary source ↓e-policyPDF p. 4, Section III-B, next-latent-action loss and final paragraph
Prismatic-7B combines SigLIP and DINOv2 visual encoders, a projection and LLaMA-2. New ACT tokens index a latent codebook. Autoregressive next-token negative log-likelihood trains four action tokens. The text illustrates a codebook size of 16 and compares 960 A100-hours of policy pretraining with 21,500 for OpenVLA.
Go to primary source ↓e-decodePDF p. 5, Figure 3, Section III-C and attention-pooling equations
Visual embeddings provide pooled queries to attention over latent-action embeddings from the final VLM layer, followed by linear projection into low-level action chunks. LoRA adaptation jointly optimizes latent-token prediction and L1 low-level action loss. Section III-C reports a 12.6M action head and approximately 123M trainable parameters. Figure 3 labels visual features Q and action features KV, consistent with the equations.
Go to primary source ↓e-historyPDF p. 5, Section III-C, Learn from history outputs; p. 10, Table V; p. 11, History latent actions
Post-training uses latent actions inferred from historical frame pairs in the prompt. Inference adds the preceding latent output group, four tokens, except at the first step. Table V reports instruction-only versus history: LIBERO-Goal 95.0/95.6, LIBERO-Long 88.1/92.0, R2R 30.6/47.1. Longer histories are said to yield diminishing returns without a detailed sweep.
Go to primary source ↓e-pretrainPDF p. 16, Appendix A.1 and Table A-I
The mixture contains single-arm Open X-Embodiment data, indoor/off-road GNM videos and Ego4D. Actions and proprioception are excluded in pretraining. Policy encoders, LLM and prediction head are jointly optimized: global batch 1,024, per-device batch 32, learning rate 2e-5, 20,000 steps, approximately 30 hours on 32 NVIDIA A100 GPUs. Human/Bridge variants are reported with global batch 258 across eight GPUs and approximately 200 A100-hours. SimplerEnv is explicitly excepted from the unseen-environment assertion.
Go to primary source ↓e-liberoPDF p. 6, Figure 4, Table I with footnotes, Section IV-A.1; p. 7, Results; p. 2, contribution bullet
Each LIBERO suite has ten tasks and fifty demonstrations per task; evaluation uses 500 trials per suite and averages three seeds. No LIBERO samples enter latent-model or policy pretraining; downstream suite-specific supervised adaptation is used. Full UniVLA scores 96.5/96.8/95.6/92.0 and 95.2 average; Human 88.7, Bridge 92.5, OpenVLA 76.5 and LAPA 65.7 average. MDT and MaIL use extra wrist views; LAPA is reproduced with Prismatic-7B. The displayed 95.2 minus 76.5 is 18.7 percentage points, unlike the introduction’s 18.5 figure.
Go to primary source ↓e-navPDF p. 7, Figure 6 and Section IV-A.2; p. 8, opening continuation
R2R uses 10,819 training and 1,839 val-unseen examples; evaluation is RGB-only without depth or odometry. The metric is oracle success with a three-metre goal threshold. Figure 6 gives UniVLA 47.1%, OpenVLA 17.5%, NaVid 49.1%, and LLaVA-Nav 14%. LLaVA-Nav is specifically reported on a 100-episode subset. UniVLA uses current RGB plus historical latent action; NaVid uses historical images.
Go to primary source ↓e-real-setupPDF pp. 8–9, Section IV-A.3; p. 18, Appendix C.1 and Table A-IV
Real-world tasks use an AgileX Piper arm, a seven-dimensional action space and third-view Orbecc DABAI camera RGB. The authors collect 20–80 adaptation trajectories per task. Four tasks are assessed with success and up to three subgoal points. Diffusion Policy is single-task; generalist policies are trained jointly across tasks. The paper reports 10 Hz closed-loop inference on an RTX 4090 and a chunk size of 12. Per-task rollout counts and uncertainty are not specified here.
Go to primary source ↓e-real-resultsPDF p. 18, Tables A-IV and A-V; p. 7, Figure 5; pp. 8–9, deployment results
Table A-V reports Full UniVLA success/score 81.7%/2.63, LAPA 45.0%/1.95 and OpenVLA 38.3%/1.63. Full UniVLA task successes are 93.3%, 100.0%, 46.7%, 86.7%. Diffusion Policy exceeds UniVLA on towel success, 53.3% versus 46.7%, but has a lower towel subgoal score, 2.33 versus 2.47. The 36.7-point and 0.68-score improvements refer to LAPA. Cleaning receives 100% success but 2.33/3 average score; the source does not reconcile these metrics.
Go to primary source ↓e-generalizationPDF p. 8, Table II and Figure 7; p. 9, Generalizability Analysis
For changed lighting, visual distractors and a screwdriver-to-marker substitution, UniVLA obtains successes of 66.7%, 53.3% and 86.7%, averaging 68.9% with score 2.49. LAPA averages 28.9%/1.36 and OpenVLA 20.0%/0.98. The novel-object test also changes the language instruction.
Go to primary source ↓e-latent-ablationPDF pp. 9–10, Section IV-B Quantitative analysis and Table III
Policies pretrained on Ego4D with Genie-style, task-irrelevant or task-centric latent labels attain LIBERO average success 82.3%, 56.5% and 88.7%, respectively. Long-suite success is 69.6%, 0.2% and 79.4%. The task-centric gain over Genie-style labels is 6.4 percentage points overall.
Go to primary source ↓e-decoder-ablationPDF p. 10, Table IV, caption and Latent action decoder paragraph
Autoregressive low-level decoding, the proposed decoder without visual queries, and the complete decoder yield LIBERO average success 73.6%, 92.5% and 95.2%; Long scores are 49.0%, 86.0% and 92.0%. Arithmetic from the cells gives a 2.7-point visual-query gain and 43.0-point Long gain over autoregressive decoding. The accompanying prose instead says 2.2 and 42.1, respectively.
Go to primary source ↓e-data-efficiencyPDF p. 10, Figure 10 and Data efficiency paragraph
The horizontal axis is the percentage of downstream training demonstrations. UniVLA’s LIBERO-Goal successes at 10%, 20%, 50% and 100% are 86.3%, 90.4%, 93.1% and 95.6%; corresponding Long values are 62.4%, 71.4%, 87.0% and 92.0%. The text compares Goal at 10% with OpenVLA at full data, 79.2%.
Go to primary source ↓e-scalingPDF p. 9, Figure 9; p. 10, Data scalability; p. 18, Table A-V
Figure 9 gives Bridge/OpenX/full LIBERO values 92.5/94.2/95.2, real-robot scores 2.05/2.35/2.64 and R2R values 38/44/49. Table A-V instead gives the full real-robot score as 2.63; Figure 6 and Table V give full R2R as 47.1. These displays cannot be silently combined into a single consistent numerical scaling series.
Go to primary source ↓e-calvinPDF p. 16, Appendix B.1 and Table A-II; p. 17, opening continuation
CALVIN trains on A/B/C and evaluates 1,000 five-task instruction chains in D. UniVLA uses third-view RGB and language; its five-task success and mean completed length are 56.5% and 3.80. OpenVLA with action chunks scores 43.5% and 3.27; RoboDual scores 54.4% and 3.66. UniVLA trains 100k steps, OpenVLA 200k. The prose calls CLOVER at 45.4% the prior best, although RoboDual is higher in the same table.
Go to primary source ↓e-simplerPDF p. 17, Table A-III and Appendix B.2
The WidowX/Bridge evaluation covers four tasks, with 24 trials per task stated but no further aggregation detail. Bridge overlaps pretraining. Decoder-only adaptation freezes the rest of the model; the other UniVLA row uses LoRA fine-tuning on Bridge. Table A-III reports aggregate task success 35.4% decoder-only, 47.9% UniVLA and 1.0% OpenVLA. The prose instead reports 42.7%. UniVLA’s four grasp columns all exceed decoder-only values, contradicting prose that describes reduced grasp performance; green-block task success is only 2.8%, versus decoder-only 4.2%.
Go to primary source ↓e-limitationsPDF p. 11, Section VI, all four subsections
The authors acknowledge fixed codebook size and temporal granularity, predominantly single-arm evaluation, and reliance mainly on fine-grained short-horizon language. More complex embodiments remain future work. World-model planning trees, reinforcement-learning alignment and zero-shot learning from tokenized demonstration videos are proposed future directions, not evaluated capabilities.
Go to primary source ↓e-real-architecturePDF p. 18, Appendix C.2 and Table A-VI; p. 5, Section III-C; p. 2, Introduction
The real-world head adds proprioceptive-state processing. It has separate visual and latent attention pooling blocks, each with eight heads, head dimension 64, hidden size 512 and MLP ratio four; action projection has one layer and proprioception projection two layers. Table A-VI lists 12.6M parameters, whereas the introduction names 10.8M without reconciling the counts.
Go to primary source ↓e-qualitativePDF p. 9, Figure 8 and Section IV-B; p. 17, Figures A-1/A-2 and Appendix B.3; p. 18, opening continuation
Selected paired frames assigned the same latent code show semantically similar motion across data sources and views. Attention visualizations concentrate on grippers and manipulated objects. These are qualitative examples; no population-level code-purity or attention-faithfulness metric is supplied.
Go to primary source ↓8.5 Primary sources
UniVLA: Learning to Act Anywhere with Task-centric Latent Actions ↗
PDF · 12,621 extracted words
Source fingerprint
1b8551db322a0e31514601d9d84879f85a5c391e4911e40752ea9df7567196f5