WorldArena: A Unified Benchmark for Evaluating Perception and Functional Utility of Embodied World Models
1. Paper overview
In one sentence: WorldArena separates the quality of imagined robot videos from their usefulness for training, evaluating and executing policies, revealing that one video score cannot establish functional competence. e02e03e10e11e14e15e16e17
| At a glance | What to know |
|---|---|
| Research problem | Source description Perceptually convincing prediction does not establish that a world model provides useful training data, ranks policies accurately, or yields executable actions. WorldArena therefore measures these roles separately while retaining automated video assessment and human judgment. The empirical setting is robotic manipulation, so broader embodied competence remains outside the demonstrated scope. e02e03e11 |
| Core mechanism | |
| A key reported result | Direct action planning: adjust bottle and click bell: WoW: 20% / 21%; TesserAct: 1% / 35%. Task success rate (%). Table 5; six world models paired with an IDM; predicted actions executed in RoboTwin, using the data-engine task setting. π0.5: 77% / 66%. The best world-model result is 20% for adjust bottle and 35% for click bell, achieved by different models. These are simulator execution results, not physical-robot trials. e11e14e16 |
| Reading caution | Reader analysis Functional results cover two manipulation tasks and smaller model subsets than video evaluation. Table 4–5 point estimates lack repeated-training uncertainty; Figure 4–5 shaded bands have no stated confidence level. The evidence cannot establish broad physical-robot generalization or a causal benefit of any architecture. e14e15e16e17 |
Core contributions
Figure 3. Three uses of a world model require three different endpoints. Original paper, p. 6 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Begin at the upper left: the world model feeds an inverse dynamics model (IDM), whose action labels supply a VLA policy's training data; that policy is then tested in the simulator. At upper right, an initial observation and instruction instead drive video prediction, the IDM decodes actions, and the simulator executes them. Along the bottom, follow each observation upward into the policy and its action downward into the world model. Generated observations continue the rollout, and a VLM judges the resulting video. Section 3.2 specifies the frozen world-model features and VPP-style action head used for data synthesis. e03e11e14e15e16
What it supports. The benchmark does not equate three superficially similar successes. A generated trajectory may help a separately trained policy, support a plausible virtual evaluation, or produce executable actions. These endpoints test different combinations of the world model, IDM, policy and judge; success in one branch does not establish success in another.
Where the evidence stops. Figure 3's caption calls the comparison real-world evaluation, but Sections 3.2 and 4 use RoboTwin. Treat the results as simulator evidence. The planner branch shows no explicit feedback arrow; the text calls execution closed-loop without specifying replanning cadence.
2. Motivation
2.1 The problem and the proposed response
Perceptually convincing prediction does not establish that a world model provides useful training data, ranks policies accurately, or yields executable actions. WorldArena therefore measures these roles separately while retaining automated video assessment and human judgment. The empirical setting is robotic manipulation, so broader embodied competence remains outside the demonstrated scope. e02e03e11
2.2 What this reading follows
A convincing video of a robot completing a task leaves several practical questions unanswered. Could it teach a policy, predict which policy will succeed, or be converted into actions that actually complete the task in a simulator? WorldArena builds separate tests for these questions around RoboTwin manipulation. Its sixteen video metrics describe appearance, motion, consistency, physics, geometry and controllability; EWMScore averages those measurements. The functional tests then expose what that average misses. Read the pipeline first, compare the two kinds of success-rate table, and finally use the correlation plots to distinguish perceptual agreement, policy ranking and calibrated prediction. e02e03e10e11e14e15e16e17
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | Benchmarks & simulators |
| Architecture | Not applicable |
| Prediction paradigm | Not applicable |
| Quadrant | Not applicable |
3.1 Evidence-based assessment
Supports the recorded classification
The recorded benchmark, evaluation-protocol and composite-score categories fit the source. WorldArena evaluates several separate world models, policies, IDMs and judges. It does not propose one shared architecture or one prediction paradigm, so architecture, prediction paradigm and quadrant are appropriately not applicable. e02e11
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Follow the action labels before interpreting a success rate
WorldArena's data-engine and planner paths share a world model and an inverse dynamics model, but terminate at different agents. For synthesis, the paper first fine-tunes the world model, then freezes it and conditions a diffusion action head on intermediate features, following the VPP design. This creates paired videos and actions for π0.5 training. Table 4 measures that trained policy. For planning, the predicted video and IDM instead lead directly to simulator actions; Table 5 measures this combined system. WoW's click-bell results illustrate the distinction: synthetic-data training reaches 71%, while its planner reaches 21%. The figures describe these pathways, but do not supply a complete action-head optimization recipe or planner feedback schedule. Reproducing the comparison requires those details rather than treating either percentage as intrinsic video-model accuracy. e11e14e16
5.2 Understand what the composite rewards
EWMScore's aggregation is simple only after substantial measurement design. Raw flow, trajectory, photometric and smoothness scores receive empirical clipping; depth error is direction-inverted. The sixteen normalized metrics are then scaled and averaged. Reader analysis: this weights metrics equally, not the six dimension groups equally. It also inherits each metric's shortcuts. The source explicitly counters static-video consistency scores by multiplying them by a dynamic-degree penalty; below the threshold gamma, apparent persistence earns less credit. That protects against one failure mode, but motion intensity and CLIP diversity still do not independently establish correct contact or correct instructions. Appendix A.17 also leaves the eight-model calibration cohort unresolved against the fourteen-model evaluation. A reproduction should preserve published bounds and expose implementation choices before interpreting small changes in the final ranking. e02e05e06e09e10
5.3 Separate ranking agreement from trustworthy simulation
In the evaluator pathway, a policy sees generated observations and returns actions to the world model. Success is then assigned from the generated rollout, not measured by executing that rollout in a physical environment. Appendix B compares five reference frames with five generated frames and emphasizes arm selection, final-state similarity and intent; it allows some rendering hallucinations. Figure 4 asks whether this proxy agrees with RoboTwin across five policies. Reader analysis: CtrlWorld's high correlation supports ranking within that set, while its inflated success estimates caution against using the proxy as an absolute reliability forecast. The authors suggest overfitting to successful trajectories, but their experiment does not separate transition-model optimism from judge permissiveness. Comparing ranking, calibration and independently reviewed success labels would distinguish these explanations more clearly. e11e15e18
5.4 Training and inference
During training
RoboTwin supplies 2,500 videos across fifty tasks: 2,000 train and 500 test videos for video evaluation. Models with available training code are post-trained using official implementations; equivalent adaptation of every commercial model is not established. e03
For data synthesis, fine-tune the world model, freeze it, and use a VPP-style diffusion action head conditioned on intermediate world-model features as the IDM. Generated video–action pairs train π0.5. The setup mentions 10%, 20%, 30%, 50% and 100% policy-data fractions, but Table 4's reported experiment uses 25 synthetic trajectories per task. e03e11e14
During inference
As a data engine, the world model supplies training examples; the downstream π0.5 policy is tested in the simulator. As an action planner, the world model and IDM produce actions from an observation and instruction, and those actions are executed in RoboTwin. The paper calls this closed-loop execution but does not specify the replanning cadence. e11e16
As a policy evaluator, alternate policy actions and action-conditioned generated observations until the rollout exceeds the reference length by 20%. A VLM judges success; Appendix B compares five reference frames with five rollout frames, prioritizing arm choice, final-state completion and action intent. Compare the resulting success rates against RoboTwin. e11e15e18
5.5 Implementation flow
- Measure appearance, motion and persistence
MUSIQ and LAION assess frames; V-JEPA feature distributions supply JEPA similarity. RAFT supports dynamic degree and flow intensity; VFI-Mamba interpolation supplies motion smoothness. DINO, CLIP and optical-flow reconstruction measure subject, background and photometric consistency. Dynamic-degree penalties discourage static-video shortcuts. e04e05e06
- Measure interactions, geometry and conditioning
Qwen3-VL-8B judges interaction quality, perspectivity and instruction following. SAM3 arm-box centers, interpolated through missing detections, support trajectory alignment. Depth-Anything predictions are median-aligned over forty sampled frames. Qwen2.5-VL descriptions encoded with CLIP assess semantic alignment; CLIP dissimilarity across differently instructed videos measures action-following diversity. e07e08e09
- Aggregate video metrics
Apply the specified direction alignment and empirical bounds, then scale normalized video metrics to 0–100 and average all sixteen. Equal metric weighting does not give equal weight to the six dimensions, because they contain different numbers of metrics (reader deduction). e02e10
6. Experiments & results
WorldArena evaluates whether generated robot videos are useful for learning and control. It pairs sixteen video metrics with three functional tests: synthetic-data policy training, world-model policy evaluation, and action execution through an inverse dynamics model. Its EWMScore summarizes video quality, while separate RoboTwin experiments reveal weaker alignment with downstream performance. This is a benchmark contribution, not a newly unified robot policy.
The paper provides benchmark comparisons, correlation diagnostics and selected good/bad metric examples, but no controlled removal of a model component or individual score term. Figures 4–5 therefore serve as diagnostics in the ablation section; they cannot establish a causal contribution for an architecture or metric. e15e17e19
6.1 Read the original evidence
Tables 2 and 3. Appearance and trajectory agreement can favor different models. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read down one metric column before comparing across columns: the normalized scores represent different measurements, not interchangeable success probabilities. The upper table covers visual quality, motion quality and content consistency; the lower table covers physics adherence, 3D accuracy and controllability. Compare Wan 2.6 and CtrlWorld in Image Quality, then find the same rows under Trajectory Accuracy. Their ordering reverses. The latter uses detected arm trajectories aligned to reference demonstrations, whereas image quality is a no-reference frame-quality prediction. The Action Following column measures diversity across differently instructed videos, so it should be interpreted alongside instruction correctness, not substituted for it. e03e04e07e09e13
What it supports. Wan 2.6 scores 0.6824 for image quality against CtrlWorld's 0.3522, but only 0.1182 for trajectory accuracy against 0.4766. This concrete reversal demonstrates why a polished image is insufficient evidence of faithful manipulation dynamics. It does not establish which model would execute a new task successfully.
Where the evidence stops. Section 4.2.1 praises WoW's action following, but Table 3's maximum is Cosmos-Predict 2.5 (text), at 0.1418 versus WoW's 0.0434. Retain the table values. Training-code availability also limits how uniformly the compared models were adapted.
Table 4. Useful synthetic data depends strongly on the task. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Here the row names identify the source of policy-training data. They do not identify the policy executing a generated video directly. Section 4.2.2 maps Task 1 to adjust bottle and Task 2 to click bell, trains π0.5 on 25 synthetic trajectories per task, and evaluates each task 100 times. Use the zero-shot row to see whether synthetic training helps at all, then compare with the row labeled trained with real data. WoW is the strongest synthetic source in both columns, but its relationship to the reference-data baseline changes substantially between the two tasks. e03e11e14
What it supports. WoW-trained π0.5 reaches 45% on adjust bottle and 71% on click bell. The reference-data baseline reaches 77% and 66%, respectively. Thus WoW exceeds that baseline by 5 percentage points on click bell while remaining 32 points below it on adjust bottle; a single aggregate would obscure this task dependence.
Where the evidence stops. The paper does not pair these rows with uncertainty or specify a matched reference-data trajectory count. RoboTwin supplies the benchmark setting, so the table's real-data label is not evidence of a physical-robot evaluation. The observed gain is not a controlled data-efficiency conclusion.
Table 5. Converting imagined futures into actions remains much weaker than the policy baseline. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Reuse the task mapping from the data-engine experiment: Task 1 is adjust bottle and Task 2 is click bell. This table changes the execution pathway. Instead of training π0.5 on synthetic examples, each world model is paired with an IDM and its predicted action sequence is executed in RoboTwin. The first row is therefore a policy baseline, not another synthetic-data source. Scan each column separately: WoW has the best world-model result on Task 1, while TesserAct leads Task 2. The source describes the setting as similar to the data-engine experiment but provides no detailed replanning schedule. e11e14e16
What it supports. WoW achieves 20% on adjust bottle; TesserAct achieves 35% on click bell. The corresponding π0.5 values are 77% and 66%. Even the strongest world-model row in each column remains substantially below the policy baseline. These numbers concern actions executed in a simulator, not the visual appearance of successful generated videos.
Where the evidence stops. This comparison measures the combined world-model and IDM pathway. It does not isolate whether errors come from prediction, action decoding or execution timing, and the source supplies no controlled component ablation or per-row uncertainty.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| Synthetic-data policy training: adjust bottle and click bell Table 4; π0.5 trained with 25 synthetic trajectories per task; each task evaluated 100 times in the RoboTwin setting. | WoW: 45% / 71%; RoboMaster: 7% / 68% (adjust bottle / click bell). Task success rate (%) | Zero-shot π0.5: 2% / 5%; reference-data-trained π0.5: 77% / 66%. WoW is strongest among synthetic-data rows on both tasks. Its click-bell gain over reference-data training is 5 percentage points, but its adjust-bottle deficit is 32 points. No uncertainty or matched reference-data trajectory count accompanies these rows. e03e14 |
| Direct action planning: adjust bottle and click bell Table 5; six world models paired with an IDM; predicted actions executed in RoboTwin, using the data-engine task setting. | WoW: 20% / 21%; TesserAct: 1% / 35%. Task success rate (%) | π0.5: 77% / 66%. The best world-model result is 20% for adjust bottle and 35% for click bell, achieved by different models. These are simulator execution results, not physical-robot trials. e11e14e16 |
| Policy evaluation agreement with RoboTwin Figure 4; five π0.5 policies of differing competence evaluated through action-conditioned rollouts. | CtrlWorld: r = 0.986. Pearson correlation of success rates | Cosmos-Predict 2.5: r = 0.483. CtrlWorld tracks policy ordering much better, but both proxies overestimate success. High correlation therefore does not imply calibrated probabilities. The authors suggest overfitting to successful trajectories; this explanation is not isolated experimentally. e15 |
| Video quality versus functional utility Figure 5; fourteen models for human comparison; six for each functional comparison. | Human evaluation: r = 0.825. Pearson correlation with EWMScore | Data-engine performance: r = 0.600; action-planner performance: r = 0.360. The composite aligns more strongly with human video judgments than with execution. Different model subsets and small functional samples limit direct comparison of correlation strength. e12e14e16e17 |
| Per-metric video evaluation Tables 2–3; RoboTwin video evaluation with the reported 2,000/500 train/test split. | Wan 2.6: image quality 0.6824, trajectory accuracy 0.1182. Normalized image-quality and trajectory-accuracy scores | CtrlWorld: image quality 0.3522, trajectory accuracy 0.4766. The same pair reverses ordering between appearance and reference-trajectory alignment. Neither score is a task success percentage. e03e13 |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Figure 4. An evaluator can rank policies well while overstating their success. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Each point is one of five π0.5 policies evaluated both in RoboTwin and through an action-controllable world model. Follow the horizontal axis to compare actual simulator performance, then read upward to find the proxy estimate. CtrlWorld's points closely track the fitted line; Cosmos-Predict 2.5's ordering is less consistent. Now compare the numerical scales on the two axes rather than the slope alone: proxy success lies above simulator success in both panels. Section 3.2 uses repeated policy–world-model rollouts, while Appendix B supplies the reference-based VLM judgment used to label generated executions successful or unsuccessful. e11e15e18
What it supports. CtrlWorld's reported correlation is 0.986, versus 0.483 for Cosmos-Predict 2.5. This supports stronger policy-ranking agreement for CtrlWorld in the tested set. Yet the authors also observe systematic optimism in both proxies. Selecting a better policy and estimating its true probability of success are distinct evaluation requirements.
Where the evidence stops. Only five policies are plotted, and the shaded bands have no specified confidence level. The authors' successful-trajectory-overfitting explanation is a hypothesis, not an isolated ablation. A VLM's tolerance for rendering artifacts may also affect proxy success judgments.
Figure 5. EWMScore tracks human video judgments more closely than functional performance. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. All three panels put EWMScore on the horizontal axis, but their vertical axes ask different questions. The left panel uses human video judgments; the center uses performance of policies trained from generated data; the right uses performance of world-model-plus-IDM planners. Read the legends as well as the regression lines: the human comparison includes fourteen models, whereas the functional comparisons include six. EWMScore itself is computed from the sixteen video metrics in Section 3.4. Human ratings and functional success are external comparisons with that score, not ingredients averaged into it. Keep those evaluation roles separate when reading the downward progression in reported correlations. e10e12e14e16e17
What it supports. The reported Pearson correlations are 0.825 for human evaluation, 0.600 for data-engine performance and 0.360 for action planning. The paper's central gap is therefore visible within one figure: a video-quality composite agrees with perceptual judgments better than it predicts downstream utility on these tested model sets.
Where the evidence stops. The functional panels contain only six models, and their populations differ from the human panel. Correlation does not prove that better appearance causes better control. The source gives no confidence level for the shading and no controlled test of metric contributions.
7. Analysis & limitations
7.1 What the evidence leaves open
Functional results cover two manipulation tasks and smaller model subsets than video evaluation. Table 4–5 point estimates lack repeated-training uncertainty; Figure 4–5 shaded bands have no stated confidence level. The evidence cannot establish broad physical-robot generalization or a causal benefit of any architecture. e14e15e16e17
Several measurements are proxies: CLIP diversity does not itself certify instruction correctness, monocular depth alignment does not measure true scene geometry, and sparse-frame VLM success judgments explicitly tolerate some rendering hallucinations. e08e09e18
The source leaves internal inconsistencies unresolved: Appendix A.17 names eight normalization models versus fourteen evaluated; it calls JEPA similarity an error although Eq. (3) makes it higher-is-better. Eqs. (8) and (10) print sums where prose describes averages. Section 4.2.1 praises WoW action following, but Table 3's maximum is Cosmos-Predict 2.5 (text). e03e04e06e10e13
7.2 Questions for discussion
- Would CtrlWorld retain policy ranking accuracy on failures and held-out tasks after correcting its optimism?
- How stable are EWMScore rankings to the calibration cohort, motion penalty and instruction-diversity confounds?
8. Reproducibility audit
8.1 Requirements and known gaps
Reconstruction needs the exact RoboTwin split, model checkpoints and official adaptation recipes, IDM feature taps and action training, π0.5 settings, judge and feature-extractor versions, and Table 6 bounds. The PDF gives no complete optimizer/loss schedule, hardware budget, seed list, action horizon or baseline data-count match. e03e11e14e16e10
Metric implementation also needs the dynamic sigmoid steepness and penalty threshold, VLM frame count and decoding settings, SAM3 confidence/NMS thresholds, semantic scaling, and reciprocal-error safeguards. These are not numerically fixed in the relevant definitions. Preserve the published normalization bounds rather than silently choosing a new calibration cohort. e05e06e07e08e10
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Test whether policy ranking survives a stricter success judge
Reader-proposed, not performed: freeze the five π0.5 checkpoints, initial states, actions and rollout-length rule when comparing CtrlWorld and Cosmos-Predict 2.5. Score the same saved rollouts with the Appendix B five-frame protocol and with blinded human review of the full videos; also score simulator videos with both judges against simulator success labels. Report Pearson and rank correlation, signed success-rate bias and false-positive rates. If changing only the judge removes most optimism while preserving ranking, that implicates the measurement layer. Persistent optimism across judges would instead motivate a transition-model or rollout-data investigation. Keep simulator tasks and policy sampling fixed throughout. e11e15e18
Check 2: Check EWMScore for motion and instruction-diversity shortcuts
Reader-proposed, not performed: use held-out RoboTwin clips to construct matched controls consisting of a repeated initial frame, an intact successful clip, and a clip with unrelated motion or a wrong instruction. Hold video length, resolution, reference set, feature extractors and Table 6 bounds fixed. Report all sixteen scores and EWMScore, with and without the documented dynamic penalty, using a preregistered gamma sensitivity sweep because its value is unspecified. Independently label task correctness. The static control should lose consistency credit, while unrelated motion or diversity should not outrank correct execution as a functional signal. A reversal would identify a concrete limit of the composite without claiming to reproduce the full benchmark. e05e06e09e10e19
8.3 Reading coverage
Visual audit: Visually inspected the title and author page; every scientific main-text page; all metric definitions, equations, prompts and normalization bounds in Appendices A–B; and every Appendix C example. Figure 3's arrows were checked against Section 3.2; its real-world caption wording is qualified using the RoboTwin protocol. Figures 4–5 supply diagnostic evidence, with unspecified band confidence levels. Tables 2–5 and all six final crops were inspected at their final resolution; neighboring text fragments were removed by correcting two crop bounds. References on pages 10–11 were read in the complete text. No separate supplement or full-motion video was supplied or inspected.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22. Appendix coverage: reviewed.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Abstract; Sections 1–2: Introduction and Related Works
- Section 3: WorldArena, including all video metrics, three functional roles, human evaluation and EWMScore
- Section 4: Experimental setup, all results and inter-metric analysis
- Section 5: Conclusion and Future Work; References
- Appendix A.1–A.17: All metric definitions, equations and normalization bounds
- Appendix B: Policy-success judge prompt
- Appendix C: All qualitative comparisons (Figures 6–11)
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Separate supplemental material availability has not been fully verified.
- The supplied source is arXiv:2602.08971v2 [cs.CV], dated 11 February 2026 in the margin; its title-page preprint date is 12 February 2026. The catalog submission date is 9 February 2026. Title and all 21 authors match; v1 was not supplied or compared.
- The acquisition manifest notes that text extraction does not reconstruct figure images. This was addressed by inspecting the retained PDF: all scientific pages (1–9 and 12–22) and every final crop were visually reviewed; reference-only pages 10–11 were read as text.
- Separate supplemental material availability has not been fully verified; no separate supplement was supplied.
- Code, project website and leaderboard were not inspected. No experiments were reproduced.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e01PDF p. 1, title, author block, affiliations, arXiv margin and preprint date
Exact title and 21 author names match the catalog. Margin: arXiv:2602.08971v2 [cs.CV], 11 February 2026; printed preprint date: 12 February 2026. Eight institutions are listed; Xin Jin has no numbered institutional marker.
Go to primary source ↓e02PDF pp. 2–3, Sections 1–3; Figure 1 and Table 1
The benchmark combines sixteen video metrics in six dimensions, three embodied roles and human evaluation. It evaluates fourteen models and distinguishes generative quality from functional utility.
Go to primary source ↓e03PDF p. 6, Section 4.1, Dataset and Tested Models
RoboTwin 2.0 has fifty scenarios and 2,500 videos; video train/test counts are 2,000/500. Policy-data fractions are 10/20/30/50/100%. Models with training code are post-trained using official implementations; complete training settings and compute are not specified here.
Go to primary source ↓e04PDF p. 12, Appendix A.1–A.3, Eqs. (1)–(4)
MUSIQ and LAION frame scores are averaged. JEPA similarity uses V-JEPA features and exp(-alpha times squared polynomial-kernel MMD), alpha 40; larger similarity is better.
Go to primary source ↓e05PDF pp. 13–14, Appendix A.4–A.6, Eqs. (5)–(7)
RAFT top-five-percent flow drives a sigmoid dynamic-degree score with resolution-adaptive threshold; steepness is symbolic. Overall flow averages pixel displacement. VFI-Mamba reconstruction uses SSIM weighted by log motion difference.
Go to primary source ↓e06PDF pp. 14–15, Appendix A.7–A.9, Eqs. (8)–(13)
DINO subject, CLIP background and reciprocal photometric reconstruction scores receive min(1, dynamic-degree/gamma) penalties. Gamma is not numerically specified. Eqs. (8) and (10) show sums despite averaging language.
Go to primary source ↓e07PDF pp. 15–17, Appendix A.10–A.11, VLM rubric and Eqs. (14)–(16)
Qwen3-VL-8B judges interaction quality from sampled frames and instruction. SAM3 box-center trajectories use NMS, confidence filtering and missing-point interpolation, then reciprocal NDTW. Sampling count, filtering thresholds and a zero-distance safeguard are not specified.
Go to primary source ↓e08PDF pp. 17–18, Appendix A.12–A.15, Eq. (17)–(18)
Depth-Anything estimates forty sampled frames and aligns depth medians before relative error. Qwen3-VL-8B judges perspective/instruction following. Qwen2.5-VL descriptions feed CLIP text features for semantic alignment, with unspecified scale w.
Go to primary source ↓e09PDF p. 5, Section 3.1.6, Action Following; p. 18, Appendix A.16, Eq. (19)
Main text uses three instructions from the same initial frame. The appendix generalizes to N videos and defines action-following score as average pairwise CLIP dissimilarity.
Go to primary source ↓e10PDF p. 5, Section 3.4; pp. 18–19, Appendix A.17, Eqs. (20)–(21), Table 6
EWMScore is the arithmetic mean of sixteen normalized video metrics scaled to 0–100. Empirical clipping uses first/99th percentiles; depth error is inverted. Table 6 gives five pairs of bounds. Appendix text names eight calibration models and inconsistently describes JEPA similarity as an error.
Go to primary source ↓e11PDF pp. 5–6, Section 3.2 and Figure 3 with caption
Data engine: fine-tune then freeze the WM, use a VPP-style IDM and train downstream π0.5. Policy evaluator: action-conditioned policy–WM rollouts and VLM success. Action planner: WM–IDM actions execute in RoboTwin. Rollout stopping exceeds reference length by 20%. Figure 3's caption says real-world evaluation, whereas the experiment uses RoboTwin; no replanning cadence is given.
Go to primary source ↓e12PDF p. 5, Section 3.3
Seventy annotators evaluate 3,500 videos. Overall quality, instruction following and physical adherence receive 1–5 ratings normalized to 0–100; pairwise choices supply win rates.
Go to primary source ↓e13PDF pp. 6–7, Section 4.2.1 and Tables 2–3, named metric columns
Wan 2.6 image/trajectory scores are 0.6824/0.1182; CtrlWorld 0.3522/0.4766. Veo instruction following is 0.9328 and interaction quality 0.7872. Action-following maximum is Cosmos-Predict 2.5 (text), 0.1418, versus WoW 0.0434.
Go to primary source ↓e14PDF pp. 7–8, Section 4.2.2, Embodied Data Engine; Table 4, all rows
Task 1 is adjust bottle; Task 2 click bell. Each synthetic-data policy uses 25 trajectories per task; tasks are executed 100 times. Zero-shot 2/5%, reference-data 77/66%, Genie 7/21%, TesserAct 1/35%, RoboMaster 7/68%, Vidar 13/53%, WoW 45/71%, Wan 2.2 15/41%. No uncertainty or reference-data count is attached to the rows.
Go to primary source ↓e15PDF p. 7, Figure 4, both panels; p. 8, Section 4.2.2, Embodied Policy Evaluator
Five policies are compared. CtrlWorld correlation is 0.986; Cosmos-Predict 2.5 is 0.483. Both yield success rates above the simulator; authors suggest partial overfitting to successful trajectories. Shaded bands are not assigned a confidence level.
Go to primary source ↓e16PDF p. 8, Table 5 and Section 4.2.2, Embodied Action Planner
Action sequences are executed in RoboTwin using the data-engine task setting. Task 1/2 rates: π0.5 77/66%, Genie 10/20%, TesserAct 1/35%, RoboMaster 8/20%, Vidar 2/19%, WoW 20/21%, Wan 2.2 12/20%. No per-row uncertainty is supplied.
Go to primary source ↓e17PDF p. 8, Figure 5 and Section 4.3
EWMScore correlates with human evaluation at r=0.825, data-engine performance at 0.600 and action-planner performance at 0.360. Legends show fourteen versus six-model comparisons. No band confidence level or causal ablation is specified.
Go to primary source ↓e18PDF p. 19, Appendix B, policy-success judge description and complete prompt
Five reference and five generated frames are compared for arm choice, final state and action intent. The prompt permits object deformation, color shifts, minor trajectory differences and length differences; success is binary.
Go to primary source ↓e19PDF pp. 20–22, Appendix C, Figures 6–11 and captions
Selected good/bad examples illustrate all six dimensions, including static motion, identity changes, contact violations, trajectory deviations, depth distortions and wrong-object instructions. These are diagnostic examples, not controlled ablations.
Go to primary source ↓e20PDF p. 9, Section 5, Conclusion and Future Work
The authors conclude that strong visual generation is insufficient for reliable embodied decision-making and propose expanding evaluated model coverage.
Go to primary source ↓8.5 Primary sources
WorldArena: A Unified Benchmark for Evaluating Perception and Functional Utility of Embodied World Models ↗
PDF · 12,098 extracted words
Source fingerprint
e41ab0f9ce75adad027e4e77402e1bbd77a50ab12340a6517a300d2edc94a6e6