RynnVLA-002: A Unified Vision-Language-Action and World Model
1. Paper overview
In one sentence: A shared action-and-image predictor improves robot learning through joint supervision, while a separate continuous head makes action chunks practical without imagined-image rollouts during control. identityarchitecturecontinuousinferenceliberocontinuous-ablationreal-ablation
| At a glance | What to know |
|---|---|
| Research problem | Source description Can predicting action-conditioned visual change improve a language-conditioned robot policy, while action supervision improves the same model's image prediction? The authors target weak action representations and foresight in conventional VLAs; their stronger language about learning physics remains an interpretation of the reported tasks. motivationarchitecture |
| Core mechanism | Source description Unify both conditional tasks in a shared token vocabulary and backbone, with a continuous Action Transformer added for real-world control. architecturecontinuous |
| A key reported result | LIBERO four-suite manipulation: Continuous: 97.4 average; Spatial 99.0, Object 99.8, Goal 96.4, Long 94.4. Success rate (%). Four suites; 50 deployment rollouts per task from different initial states; no large-scale robot-data pretraining. Discrete RynnVLA: 93.3 average. Pretrained OpenVLA-OFT: 97.1; X-VLA: 98.1; EO-1: 98.2. Competitive rather than best overall. Heterogeneous backbones and pretraining prevent treating the leaderboard as an isolated test of world-model supervision. liberosim-protocol |
| Reading caution | Source description Physical testing covers one SO100 platform and two pick-and-place tasks. Other embodiments, mobile long-horizon manipulation and deformable-object tasks remain untested; one real-world task reportedly takes roughly four days to train. scope-cost |
Core contributions
- Source description
Unify both conditional tasks in a shared token vocabulary and backbone, with a continuous Action Transformer added for real-world control. architecturecontinuous
- Author claim
Mask cross-action attention for discrete chunks, and retain discrete supervision alongside continuous regression. The authors report better chunk robustness and faster continuous-policy convergence. maskdiscrete-ablationauxiliary
Figure 2. One shared backbone learns two conditional tasks, with two ways to decode actions. Original paper, p. 3 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Follow the arrows upward from the tokenizers. On the left, the instruction, proprioceptive state and image history enter the shared backbone. The upper left separates a parallel continuous Action Transformer from autoregressive discrete action detokenization. M marks the image-history count and K the action-chunk length. On the right, an image and an action condition the next image, with N denoting repetitions of that prediction sequence. The repeated image arrow belongs to this world-model branch. Figure 2 and Section 3.1 establish shared parameters; the explicit inference description on pages 5–6 establishes that one query uses only one of these task modes. architectureformulationtrainingcontinuousinference
What it supports. The benefit being tested is transfer through a shared representation. During training, discrete action targets and future-image targets teach the same backbone; the continuous head also receives regression supervision. During control, actions can be obtained directly from that learned context without spending inference time generating the image sequence shown on the right.
Where the evidence stops. The figure is a training overview, not a closed-loop planner. Its displayed state/action vectors are not accompanied by a complete coordinate-convention specification. Do not infer a rollout-selection mechanism or an action execution schedule from the repetition arrows.
2. Motivation
2.1 The problem and the proposed response
Can predicting action-conditioned visual change improve a language-conditioned robot policy, while action supervision improves the same model's image prediction? The authors target weak action representations and foresight in conventional VLAs; their stronger language about learning physics remains an interpretation of the reported tasks. motivationarchitecture
2.2 What this reading follows
RynnVLA-002 asks whether learning what actions do to images can improve learning which actions to take. It places both problems in one Chameleon backbone, then gives the policy a continuous Action Transformer alongside its discrete token output. The useful distinction is between training and deployment: future-image targets shape the shared representation, but a policy call does not imagine a video before acting. Read the architecture first, then the attention mask, and finally the comparisons that separate leaderboard performance from controlled ablations. The supplied v3 reports strong LIBERO results and narrower SO100 evidence, with several qualifications that are clearer in its tables than in its headline claims. identityarchitecturecontinuousinferenceliberocontinuous-ablationreal-ablation
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | Not assigned |
| Architecture | Not assigned |
| Prediction paradigm | Not assigned |
| Quadrant | Not assigned |
This table preserves the labels recorded at reading time. The current major category is WAMs. View the current classification.
3.1 Evidence-based assessment
Insufficient evidence to decide
The catalog is unassigned, so there is no substantive label to affirm or reject. Architecture evidence supports one shared action/world backbone plus a continuous head. Training jointly learns forward visual prediction and direct action prediction, but one call does not jointly emit a future/action pair. This is neither inverse-dynamics action extraction nor inference-time rollout planning; any quadrant assignment must preserve those distinctions. architectureformulationcontinuousinference
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Use two conditional tasks to teach one representation
Start with the two conditionals in Section 3.1. A policy needs an instruction, robot state and image history to decide an action; a world model needs observations and actions to predict visual change. RynnVLA makes both problems available to one backbone by expressing images, text, states and discrete actions in a common vocabulary. During the default finetuning stage, the task types are sampled equally. Discrete action targets and next-image targets both use cross-entropy, while the continuous action branch adds L1 regression. This construction explains what the paper means by unification: shared parameters and complementary supervised tasks. It does not require a robot to generate imagined images before acting. The inference paragraph explicitly separates the queries, so the world-model contribution to the deployed policy is carried through training. formulationtokenstrainingcontinuousinference
Figure 3. The discrete policy blocks attention between actions while keeping causal structure inside each action. Original paper, p. 5 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read each matrix by rows: a gray cell allows the row token to attend to the column token. Compare the Action_1 rows in panels (a) and (b). The proposed mask turns the Action_0 columns white, removing a route by which an earlier action can influence the next action. Text and image columns remain available. The triangular gray blocks within Action_0 and Action_1 remain, so this is not a fully independent-token decoder. Panel (c) preserves conventional causal attention for image prediction, allowing the next image to depend on the preceding image and supplied action. These markings agree with the cross-action restriction described in Section 3.3. maskformulationtrainingdiscrete-ablationcontinuous
What it supports. The mask targets error propagation across actions rather than the entire autoregressive token mechanism. Table 3 supplies a relevant controlled comparison: with chunking enabled and world-model training absent, average success rises from 54.0% with naive attention to 76.6% with the proposed mask, a reader-computed gain of 22.6 percentage points.
Where the evidence stops. State tokens are omitted from this schematic although Equation (1) and the policy sequence include state. The source's language about independent actions should not erase the visible within-action causal blocks. The mask also does not guarantee smooth physical trajectories.
5.2 Understand why the discrete fix does not finish the control problem
The discrete mask addresses a specific dependency: one generated action should not make the next action drift away from the observed scene. Figure 3 removes that cross-action path while preserving token-level causal structure inside each action. Table 3 and the chunk-length curves support the usefulness of this restriction in simulation. The authors then identify a different physical problem: isolated actions can form a discontinuous trajectory, and a large discrete decoder can overfit limited robot demonstrations. Their continuous head uses learnable queries and parallel chunk prediction, with bidirectional attention described in the introduction. My reading is that the two designs target different failure modes: dependence on erroneous earlier actions versus coordination across a trajectory. The continuous head retains discrete supervision during training, so changing the execution output need not discard the action-token learning signal. maskdiscrete-ablationcontinuousauxiliary
5.3 Separate mutual benefit from a claim of learned physics
Read the empirical argument in two directions. Table 4 asks whether future-image supervision improves executed continuous policies; its middle pair gives a three-point average gain with the listed inputs held fixed. Table 6 asks whether action supervision improves visual prediction; most metric comparisons improve, with Goal PSNR providing an explicit exception. Together they support mutual training benefit within the evaluated data and tasks. They do not directly identify why the representation improved. The authors' explanation that moving objects receive harder, more useful predictive supervision is plausible, but the paper does not measure a physics representation or control for every alternative auxiliary-learning effect. The SO100 table strengthens the case for practical benefit while narrowing its scope to a small physical setup with incomplete reporting of uncertainty and implementation details. continuous-ablationworld-resultsqualitativereal-protocolreal-ablationscope-cost
5.4 Training and inference
During training
The default is one finetuning stage with VLA/world-model examples sampled 1:1. 'Without pretraining' means no additional large-scale robot-manipulation pretraining; it does not mean random initialization. Whether particular inherited modules are frozen is unspecified. tokenstraining
LIBERO removes failed trajectories and no-operation actions. The world-model evaluation uses a 90/10 training/validation split. Settings are M=2, K=10 for Spatial/Long and K=5 for Object/Goal, N=1, and continuous-loss weight alpha=10. sim-protocol
A separate experiment pretrains the world model on the same data source before VLA training; Table 8 reports gains across all suites. This is an alternative ablation, not the main joint recipe. pretraining-ablation
During inference
Each call answers one query. Policy mode generates an action chunk from language, state and visual history; it does not generate future images for control. World-model mode predicts observations from images and actions. Thus visual supervision transfers through shared parameters, without an image-based planner in the execution loop. inference
Longer chunks reduce opportunities to adapt the policy. Continuous decoding is faster in the reported efficiency study, but neither the exact execution/replanning schedule nor the measurement hardware is specified. discrete-ablationefficiency
5.5 Implementation flow
- Represent all discrete modalities
Initialize from Chameleon. VQ-GAN image tokens and BPE text tokens share a 65,536-entry vocabulary with state/action tokens. Each continuous state/action dimension uses 256 bins based on training-data ranges. Continuous head outputs bypass tokenization. tokens
- Learn two conditional sequences
Policy examples place instruction, state and M image observations before K action targets. World-model examples place images and actions before next-image targets, with up to N repetitions. They omit task-specific language beyond a generic prediction prompt. training
- Separate action-generation mechanisms
The discrete mask removes access to previous actions but preserves within-action causal structure in Figure 3. The continuous head consumes contextual features and uses learnable queries to predict the whole chunk in parallel. The authors associate this smaller head with reduced overfitting and smoother motion. maskcontinuous
6. Experiments & results
RynnVLA-002 finetunes a shared Chameleon backbone for action prediction and action-conditioned image prediction, then adds a parallel continuous-action head. Its strongest evidence is mutual training benefit: better executed policies and better held-out visual predictions. Policy inference uses no imagined-image rollout. The reported 97.4% LIBERO average is competitive, while physical evidence is limited to SO100 pick-and-place.
6.1 Read the original evidence
Table 1. Continuous RynnVLA reaches 97.4% average without large-scale robot-data pretraining. Original paper, p. 6 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Begin with the bottom two rows, then read horizontally across Spatial, Object, Goal and Long before consulting Average. Continuous RynnVLA scores 99.0, 99.8, 96.4 and 94.4, yielding the reported 97.4 average; the discrete variant averages 93.3. The Pretraining column has a narrow meaning from the original caption: large-scale robot manipulation data. RynnVLA still inherits Chameleon initialization. Check the stronger pretrained rows as well: X-VLA and EO-1 have higher averages, while OpenVLA-OFT is slightly below RynnVLA. Dashes for Seer and UVA represent missing suite results, not zero success, and their averages remain unreported. liberotokenstrainingsim-protocol
What it supports. The source demonstrates a competitive policy without additional large-scale robot pretraining. It does not establish an overall leaderboard win: EO-1 reports 98.2% and X-VLA 98.1%, compared with RynnVLA's 97.4%. The continuous/discrete comparison also indicates why the action-output mechanism matters alongside the shared training objective.
Where the evidence stops. The rows combine different backbones and pretraining resources; they do not isolate world-model supervision. The paper reports 50 deployment rollouts per task, but this table gives no uncertainty intervals. Preserve its missing entries rather than filling in averages.
Table 6. Action supervision broadly improves held-out visual prediction, with a Goal PSNR exception. Original paper, p. 10 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Within each suite, compare World Model with Action World Model and use the arrows rather than assuming every larger number is better. FVD and LPIPS favor lower values; PSNR and SSIM favor higher ones. The Object block is a clear example: FVD falls from 1141.6 to 877.2 while PSNR rises from 20.31 to 22.18. Inspect Goal carefully before generalizing: its FVD, SSIM and LPIPS improve, but PSNR falls slightly from 22.25 to 22.13. The original printed SSIM and LPIPS values are retained here because the paper does not explain their scaling. Evaluation uses the held-out portion of cleaned LIBERO video-action pairs. world-resultssim-protocolqualitativeinference
What it supports. These results support the other direction of mutual enhancement: action learning can benefit the same model's future-image prediction. FVD improves in all four suites, and Object improves on all four metrics. The evidence concerns predictive visual fidelity under supplied actions; it is separate from the policy deployment success reported in the other tables.
Where the evidence stops. The source does not specify video evaluation horizon, detailed metric implementation or SSIM/LPIPS scaling. No uncertainty accompanies the pairs. Improved rendered futures, including the selected Figure 8 examples, do not establish autonomous planning or general physical correctness.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| LIBERO four-suite manipulation Four suites; 50 deployment rollouts per task from different initial states; no large-scale robot-data pretraining. | Continuous: 97.4 average; Spatial 99.0, Object 99.8, Goal 96.4, Long 94.4. Success rate (%) | Discrete RynnVLA: 93.3 average. Pretrained OpenVLA-OFT: 97.1; X-VLA: 98.1; EO-1: 98.2. Competitive rather than best overall. Heterogeneous backbones and pretraining prevent treating the leaderboard as an isolated test of world-model supervision. liberosim-protocol |
| SO100 pick-and-place across six settings 248 block and 249 strawberry demonstrations; single-target, multi-target and distractor scenarios; ten tests per task stated. | Block: 90/90/80; strawberries: 80/80/50, in scenario order. Success rate (%) | Best baseline entries: block 100/70/80; strawberries 80/70/70. Baselines use pretrained checkpoints finetuned on the same task data. Gains depend on scenario; strawberry distractor performance trails GR00T's 70%. Success requires placing at least one target, not clearing every target. real-protocolreal-results |
| Discrete attention-mask ablation Table 3, chunked discrete policies without world-model training, rows 3–4. | 76.6 with the proposed mask. Average LIBERO success rate (%) | 54.0 with naive chunking; adding world-model training to the masked model gives 78.1. The mask comparison improves by 22.6 percentage points. This ablation's 78.1 is not the main discrete model's 93.3; the source does not fully reconcile their configurations. discrete-ablationlibero |
| World-model supervision for continuous control Table 4 rows 2–3: wrist camera present, proprioceptive state absent. | 94.6 with world-model training. Average LIBERO success rate (%) | 91.6 without it; adding state subsequently yields 97.4. The listed controlled world-model gain is 3.0 percentage points; the final state-input gain is a separate 2.8 points. continuous-ablation |
| SO100 world-model and input ablations Table 5; its specific pick-and-place task is not named. | Full continuous model: 80/80/50. Single-target / multi-target / distractor success (%) | Without world modeling: 30/10/0. Discrete generation, missing state, or missing wrist camera: 0/0/0. The table supports large configuration-dependent gains. It does not support the nearby blanket 'over 80%' description or define the abstract's overall '50%' improvement. real-ablation |
| Held-out action-conditioned visual prediction LIBERO cleaned-data validation split; joint action-world model versus world-model-only training. | Object: 877.2 / 22.18 / 65.03 / 22.60. FVD ↓, PSNR ↑, SSIM ↑, LPIPS ↓, as printed | World-only Object: 1141.6 / 20.31 / 59.59 / 27.30. Goal PSNR instead falls from 22.25 to 22.13. Visual prediction improves broadly, not universally. The source does not explain SSIM/LPIPS scaling; these are image/video metrics, not executed robot success. world-resultssim-protocol |
| Action-generation efficiency Table 7 rows 6 and 9, wrist camera enabled, history length 1. | Continuous: 7.75 for chunk 5; 15.78 for chunk 10. Reported frequency (Hz) | Discrete chunking: 2.74 for either chunk size. Supports faster parallel action generation in the reported setup. Frequency normalization and hardware are unspecified, so these values do not establish replanning or image-rollout rates. efficiency |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Table 4. Rows 2–3 isolate the listed world-model-training change; rows 3–4 add state. Original paper, p. 9 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Treat this table as a sequence of different questions. Rows 1–2 add the wrist camera; rows 2–3 add world-model training while keeping the listed sensory inputs fixed; rows 3–4 add proprioceptive state. For the paper's central training claim, the middle comparison is the most informative: the average changes from 91.6 to 94.6. Looking across that pair shows improvements in every suite, including Long from 81.4 to 85.8. The last row reaches 97.4 only after state is added. It is therefore essential to keep the world-model and state contributions separate when explaining the final benchmark score. continuous-ablationsim-protocol
What it supports. The table supports a 3.0-percentage-point average gain from the world-model objective in the listed wrist-camera, no-state configuration. Adding state then yields another 2.8 points. This is more specific evidence for the joint-learning mechanism than the heterogeneous leaderboard, while also showing the importance of the policy's input information.
Where the evidence stops. The four rows are not a complete factorial experiment: there is no world-model-off row with both wrist camera and state present. The source describes a common training recipe, but it does not report multi-seed uncertainty or detailed compute matching.
Table 5. Physical performance depends on the continuous head, sensory inputs and joint training together. Original paper, p. 9 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read rows 4–5 first, because they keep continuous actions, wrist camera and state enabled while changing the world-model column. Success changes from 30/10/0 to 80/80/50 for single-target, multi-target and distractor conditions. Then compare row 5 with rows 1–3: the discrete version and the versions missing either state or wrist camera each record zero in all three columns. These are tested configurations, not universal impossibility results. The table does not name its particular pick-and-place task, so its 80/80/50 row should not be silently assigned to strawberries merely because those values match one row in Table 2. real-ablationreal-protocolreal-results
What it supports. World-model supervision produces a large gain in this physical ablation, but it works within a particular continuous-action and sensor configuration. The table also makes a source discrepancy visible: the nearby prose says performance rises to 'over 80%', whereas the best displayed cells are exactly 80%, and distractor success is 50%.
Where the evidence stops. The real-world protocol states ten tests per task and does not quantify its time budget. Table 5's task identity is unspecified. Neither this table nor the abstract defines the aggregation behind the headline '50%' overall improvement.
7. Analysis & limitations
7.1 What the evidence leaves open
Physical testing covers one SO100 platform and two pick-and-place tasks. Other embodiments, mobile long-horizon manipulation and deformable-object tasks remain untested; one real-world task reportedly takes roughly four days to train. scope-cost
No uncertainty intervals or multi-seed summaries accompany the main tables. Small physical trial counts and selected qualitative trajectories limit robustness claims. The proposed object-focused physics explanation is not directly measured. real-protocolliberocontinuous-ablationqualitative
7.2 Questions for discussion
- Would world-model supervision still help after matching total updates, VLA examples and compute across training conditions?
- Does removing cross-action attention reduce sensitivity to corrupted earlier actions while worsening physical trajectory continuity?
8. Reproducibility audit
8.1 Requirements and known gaps
A reproduction needs the Chameleon initialization/tokenizers, cleaned LIBERO trajectories or teleoperated SO100 demonstrations, front/wrist/state synchronization, both token objectives, and the parallel L1-trained head. Preserve the stated split, mixing ratio, chunk settings and success criteria; the no-operation threshold and physical time budget are not specified. tokenstrainingcontinuoussim-protocolreal-protocol
The supplied paper omits exact checkpoint variant, optimizer, learning rate, batch size, full training schedule, Action Transformer dimensions, freeze policy, hardware and software versions. Table 7's history-length convention is not explicitly reconciled with M=2. Reported four-day training cost alone is not a reproducible compute budget. tokenstrainingcontinuoussim-protocolefficiencyscope-cost
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Does correct action-to-image alignment cause the policy gain?
Reader-proposed check, not performed: reproduce the Table 4 rows 2–3 comparison using the same wrist-camera, no-state inputs and fixed cleaned LIBERO split. Add a diagnostic joint-training arm that permutes only the conditioning actions in world-model examples while retaining their images and targets; leave VLA examples intact. Match VLA exposures, joint-arm task ratio, update count, token lengths and initialization, and report training compute and multiple seeds. Evaluate policy success and held-out visual prediction separately. If correctly aligned world supervision beats both VLA-only and shuffled-action supervision, that supports action-conditioned dynamics as a useful signal. If shuffled supervision preserves the policy gain despite worse action-conditioned prediction, generic auxiliary regularization becomes a stronger alternative explanation. trainingsim-protocolcontinuous-ablationworld-resultsqualitative
Check 2: Does the mask block error propagation without solving trajectory continuity?
Reader-proposed check, not performed: train discrete policies with default versus Figure 3 attention on identical data and settings, keeping world-model supervision fixed. Evaluate the source's chunk lengths 5, 10, 15 and 20 with matched initial states. In an offline decoder probe, perturb the generated Action_0 tokens while fixing text, state, images and sampling randomness; measure the resulting change in later-action outputs. Then test unperturbed execution for success and action discontinuity, comparing with a continuous-head reference under matched observations. The proposed mask should reduce sensitivity to preceding-action corruption; failure to do so would challenge its claimed isolation. If it reduces that sensitivity but still produces discontinuous physical actions, the continuous head addresses a distinct limitation rather than merely duplicating the mask. maskcontinuousdiscrete-ablationsim-protocolreal-ablation
8.3 Reading coverage
Visual audit: Visually inspected the title/authors/affiliations/version on p. 1; Figures 1–9 and Tables 1–8 on pp. 2–12; all supporting method, training, evaluation, efficiency and reproduction-detail pages 4–13; and the scope/cost statement on p. 13. Every final crop was viewed, including a tightened Table 6 crop rendered directly at 400 DPI. Figure 2 arrows and Figure 3 row/column semantics were checked against their captions and Sections 3.1–3.3. Figure 3 omits the state input and retains within-action causal blocks; the guides preserve these distinctions. Reference-only pages 14–17 were read as text, not visually inspected. No appendix is present; separate supplements, external code and videos were not inspected, and no experiments were run.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13. Appendix coverage: not present.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Abstract and title block (p. 1)
- 1 Introduction (pp. 1–3)
- 2 Related Work, including 2.1–2.2 (pp. 3–4)
- 3 Methods, including 3.1 Overview, 3.2 Data Tokenization and 3.3 Action Chunk Generation (pp. 4–7)
- 4 Experiments, including 4.1 Simulation Results, 4.2 Real-World Robot Results and all 4.3 ablations (pp. 7–13)
- Real-World Scope and Cost; 5 Conclusion (p. 13)
- References (pp. 13–17)
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Identity/version: the supplied artifact is arXiv:2511.17502v3, stamped 30 May 2026, with a title-block date of June 2, 2026. Its exact title and fourteen authors match the catalog. The catalog's November 21, 2025 submission date refers to an earlier record; earlier editions were not supplied or compared, and their contents are not inferred.
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout. This acquisition limitation was addressed by visually inspecting PDF pages 1–13 and all six final crops.
- Separate supplemental material availability has not been fully verified.
- No appendix appears in the supplied 17-page PDF. All six text chunks, including references, were read; reference-only pages 14–17 were not visually inspected.
- Linked code and external material were not inspected; no experiments were reproduced.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
identityPDF p. 1, title block and arXiv margin
The title and all fourteen authors match the catalog. The margin identifies arXiv:2511.17502v3, 30 May 2026; the title block says June 2, 2026 and names DAMO Academy, Alibaba Group; Hupan Lab; Zhejiang University.
Go to primary source ↓motivationPDF pp. 1–2, Section 1 and Figure 1
The authors motivate shared action and image understanding/generation and contrast this with separate action-only and action-conditioned image models. Physics understanding is their interpretation.
Go to primary source ↓architecturePDF p. 3, Figure 2; p. 4, Section 3.1
One shared backbone is queried as a VLA or world model. Figure 2 shows language, state and image inputs for action generation, discrete and continuous action branches, and an image/action-conditioned image-generation branch.
Go to primary source ↓formulationPDF p. 4, Section 3.1, Eqs. (1)–(2)
The policy conditions on language, proprioception and observation history; the world model conditions on preceding observations and actions. Parameters are shared through the integrated model.
Go to primary source ↓tokensPDF pp. 4–5, Section 3.2, Tokenizers
Chameleon initialization supplies VQ-GAN image and BPE text tokenizers. Image compression is 16, codebook size 8192; 256×256 and 512×512 images yield 256 and 1024 tokens. State/action dimensions use 256 bins from training-data ranges; the combined vocabulary has 65536 entries. Continuous actions are untokenized.
Go to primary source ↓trainingPDF p. 5, Section 3.2, VLA Model Data, World Model Data and Training Objective
VLA sequences contain instruction, state, front/wrist images and K action targets; world-model sequences contain a generic prediction prompt, images, actions and next-image targets. Task types are sampled 1:1 in one finetuning stage without extra large-scale robot-data pretraining. Discrete action and image targets use cross-entropy.
Go to primary source ↓inferencePDF pp. 5–6, Section 3.2, Inference Modes
One call performs one queried task. Policy calls generate discrete or continuous actions without rolling out future images for control; world-model calls predict images conditioned on images and actions.
Go to primary source ↓maskPDF p. 5, Figure 3 and caption; p. 6, Section 3.3, Attention Mask for Discrete Action Chunk
Gray cells denote row-token access to column tokens. Panel (b) removes Action_1 access to Action_0 while retaining text/image access and within-action triangular blocks; the world-model mask remains causal. State is omitted from this schematic.
Go to primary source ↓continuousPDF p. 2, Section 1, hybrid-head paragraph; pp. 6–7, Section 3.3
The authors attribute discrete-policy failure to overfitting and discontinuity. A smaller Action Transformer uses full language/image/state context, learnable action queries and parallel chunk output; the introduction specifies bidirectional attention. It is supervised with L1 regression added to both discrete losses.
Go to primary source ↓sim-protocolPDF p. 7, Section 4.1, Benchmark through Metrics
LIBERO has Spatial, Object, Goal and Long suites. Failed trajectories and no-operation actions are removed. World-model data are split 90/10 for training/validation. Policy evaluation uses 50 deployments per task with different initial states. M=2, K=10 for Spatial/Long, K=5 for Object/Goal, N=1 and alpha=10.
Go to primary source ↓liberoPDF p. 6, Table 1, RynnVLA-002 and comparison rows; p. 7, Benchmark Results
Continuous RynnVLA reports Spatial/Object/Goal/Long scores 99.0/99.8/96.4/94.4 and 97.4 average; discrete reports 93.3 average. EO-1 and X-VLA report 98.2 and 98.1 averages with robot pretraining; OpenVLA-OFT reports 97.1. Pretraining in this table means large-scale robot manipulation pretraining.
Go to primary source ↓real-protocolPDF p. 7, Figure 4; p. 8, Section 4.2, Datasets, Baselines and Evaluation
SO100 teleoperation datasets contain 248 block and 249 strawberry demonstrations. Baselines are finetuned on the same data from official pretrained checkpoints. Three scenarios test single targets, multiple targets and distractors. Success requires placing at least one target within a time budget; timeout, more than five consecutive failed grasps, or attempted distractor manipulation causes failure. Ten tests per task are stated; the time budget is not quantified.
Go to primary source ↓real-resultsPDF p. 8, Table 2 and Section 4.2 Results
RynnVLA block success is 90/90/80 and strawberry success is 80/80/50 for single/multiple/distractor settings. The best baseline entries are 100/70/80 for blocks and 80/70/70 for strawberries. The text's 20% block multi-target improvement is a 20-percentage-point difference.
Go to primary source ↓discrete-ablationPDF p. 8, Table 3, rows 1–5; p. 9, Section 4.3 first paragraph; p. 10, Figure 6
With no world-model objective, naive chunking scores 54.0 average and masked chunking 76.6. Adding world modeling gives 78.1; without chunking, adding it gives 62.8 to 67.2. Figure 6 plots masked, naive and no-chunking policies across chunk sizes and shows that excessively long chunks can reduce success.
Go to primary source ↓continuous-ablationPDF p. 9, Table 4, rows 1–4 and Section 4.3
Averages are 84.5 without wrist/state/world modeling, 91.6 with wrist, 94.6 with wrist plus world modeling, and 97.4 after adding state. Rows 2–3 hold the listed inputs fixed while adding world-model training; rows 3–4 change proprioception.
Go to primary source ↓real-ablationPDF p. 9, Table 5, rows 1–5 and Section 4.3; p. 1, abstract
Table 5 does not name its specific pick-and-place task. Its full continuous model scores 80/80/50, versus 30/10/0 without world modeling; discrete actions or missing wrist/state each score zero. Adjacent prose says below 30% and over 80%, while the table includes 30 and peaks at 80. The abstract's overall 50% gain has no explicit aggregation definition.
Go to primary source ↓world-resultsPDF p. 10, Table 6 and VLA Model Enhances the World Model; p. 7, Metrics
On held-out LIBERO, Object FVD/PSNR/SSIM/LPIPS changes from 1141.6/20.31/59.59/27.30 to 877.2/22.18/65.03/22.60. Joint training lowers FVD in all suites; Goal PSNR declines from 22.25 to 22.13. Directions are lower FVD/LPIPS and higher PSNR/SSIM. SSIM and LPIPS scaling is not explained.
Go to primary source ↓qualitativePDF p. 9, Figure 5 and final paragraph; pp. 10–11, world-model discussion and Figure 8
Figure 5 contrasts policy trajectories with/without world-model training. The authors attribute improved retries to object-focused supervision. Figure 8 shows generated front/wrist views for two bowl tasks and is used to argue improved grasp and cross-view consistency; these are selected image sequences.
Go to primary source ↓auxiliaryPDF p. 10, Figure 7; p. 11, Discrete Actions Accelerate the Convergence continuation
The learning curves and discussion compare continuous policies trained with/without discrete action tokens; the authors report faster convergence and improved final performance when retaining the discrete targets.
Go to primary source ↓efficiencyPDF p. 12, Table 7, Figure 9 and Efficiency Analysis
Table 7 row 9, continuous with wrist camera and history length 1, reports 7.75 Hz for chunk 5 and 15.78 Hz for chunk 10; discrete row 6 reports 2.74 Hz for both. The text attributes speed to parallel decoding and discusses the image-context speed tradeoff. Hardware and exact frequency normalization are not given here.
Go to primary source ↓pretraining-ablationPDF p. 12, Table 8 and World Model Pretraining for VLA Model
A separate same-data world-model pretraining experiment improves Goal/Object/Spatial/Long from 67.3/82.9/77.8/23.0 to 73.1/84.0/79.8/30.2. The main recipe instead uses one joint training stage.
Go to primary source ↓scope-costPDF p. 13, Real-World Scope and Cost and Section 5
The authors limit physical evaluation to SO100 pick-and-place, explicitly leaving other platforms, long-horizon mobile manipulation and deformable objects untested. They report roughly four days to train a single real-world task because of image-token generation cost.
Go to primary source ↓8.5 Primary sources
RynnVLA-002: A Unified Vision-Language-Action and World Model ↗
PDF · 8,466 extracted words
Source fingerprint
2788b0a36abb737af6c59511f7d2d00bcc0012570deb33a519afbe8e8b3fbb94