PAIWorld: A 3D-Consistent World Foundation Model for Robotic Manipulation
1. Paper overview
In one sentence: PAIWorld couples camera-aware attention with frozen-teacher relation distillation to improve multi-view video generation, while leaving the connection to executed robot success unmeasured. e02e03e04e15e17
| At a glance | What to know |
|---|---|
| Research problem | Source description Robot cameras observe the same manipulation from different positions. Individually plausible videos can disagree about object location, depth or appearance. The target is a set of future views compatible with one evolving 3D scene. The authors attribute inconsistency to inadequate inter-view communication and absent geometric supervision, particularly in sparse, wide-baseline robotic views. e02e04 |
| Core mechanism | |
| A key reported result | AgiBot-World text-conditioned multi-view generation: 14.20; 0.7683; 0.1844; 45.0389; 175.7778; 0.9041; 0.4056, respectively. MEt3R ↓; SSIM ↑; LPIPS ↓; FID ↓; FVD ↓; Scene Consistency ↑; Geometric error ↓. Table 3 compares generated videos on AgiBot-World; test partition and sample count are not given. Genie-Envisioner: MEt3R 15.75, SSIM 0.7445, Scene Consistency 0.9231; Wan2.1: FID 56.4735, FVD 184.2186. Best reported MEt3R with a 1.55 reduction, while Genie-Envisioner leads Scene Consistency. Geometric error follows the lower-is-better prose despite Table 3’s upward arrow. e13e14 |
| Reading caution | Reader analysis Figure 1 and the conclusion advertise planning, action modeling and policy post-training, but Section 4 reports generation benchmarks and ablations. No downstream robot-success comparison substantiates those application gains. The conclusion leaves contact dynamics, deformable objects, fluids and extended-horizon planning for future work. e17 |
Core contributions
Figure 2. Camera geometry shapes the attention pathway; frozen geometric features supervise intermediate representations. Original paper, p. 5 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start at the first-frame images on the left. The upper branch enters Depth Anything 3 (DA3), producing geometric features and camera estimates; the middle branch supplies VAE latents and a view embedding. Follow the camera branch through the yellow Geo-RoPE embedder into view-spatial cross-attention. The blue dotted connection compares intermediate DiT representations with teacher relations. At the bottom, encoded future images provide training targets. Snowflakes identify frozen encoders and the flame marks the trainable DiT. Text/actions enter above the DiT; Section 3.2 clarifies that text modulates normalization while action maps concatenate with noisy latents. e03e04e05e06e07e08e17
What it supports. The teacher and cross-view attention have different jobs: one supplies a geometric training signal, while the other lets views exchange features during generation. Figure 2 therefore supports the two-pillar interpretation. It does not supply a planner, inverse-dynamics decoder or executable robot-action head.
Where the evidence stops. The schematic says “Predicted latent,” whereas Eq. (3) specifies velocity prediction. It also draws DA3 on first frames only, while Eqs. (8)–(11) require relations over frames and the clip. Use the equations for training; the drawing leaves temporal teacher-data routing incomplete.
2. Motivation
2.1 The problem and the proposed response
Robot cameras observe the same manipulation from different positions. Individually plausible videos can disagree about object location, depth or appearance. The target is a set of future views compatible with one evolving 3D scene. The authors attribute inconsistency to inadequate inter-view communication and absent geometric supervision, particularly in sparse, wide-baseline robotic views. e02e04
2.2 What this reading follows
A robot’s wrist and external cameras can show very different images of the same event. A useful video simulator must preserve that shared event as it predicts each view. PAIWorld addresses this with two cooperating changes: attention lets camera views exchange features, and a geometric teacher shapes what those features encode during training. The figures below trace that information flow, compare action- and text-conditioned results, and examine the ablation behind the claimed interaction. Read the leaderboard scores as generation evidence. The paper sketches planning and policy applications, but its reported experiments do not measure their downstream success. e02e03e04e15e17
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | Foundational work |
| Architecture | Not applicable |
| Prediction paradigm | Not applicable |
| Quadrant | Not applicable |
This table preserves the labels recorded at reading time. The current major category is Benchmarks & simulators. View the current classification.
3.1 Evidence-based assessment
Supports the recorded classification
The foundational neural-world-simulator classification is supported: the detailed architecture predicts camera observations and learns geometry-aware latent features. The recorded action-architecture, prediction-paradigm and quadrant fields remain Not applicable. A shared multi-view DiT is not sufficient evidence for a One Model world/action system; no joint action-prediction head or inverse-dynamics action decoder is specified. The application sketches do not establish either architecture. e03e04e07e17
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Identify what the simulator predicts before imagining a controller
Start with Section 3.1’s conditional distribution: observed camera frames, camera parameters and a command determine a distribution over future images. An action sequence enters as conditioning; it is not the output of the detailed model. Section 3.2 makes this concrete by projecting action information into spatial maps beside the noisy latent video. Training compares predicted velocity with the velocity of a known interpolation between clean latents and noise. Reader deduction from the endpoint convention: generation must traverse that path from noise toward data, then decode the resulting latents. Neither that deduction nor the application diagram supplies a numerical sampler or robot controller. A planning system would additionally need candidate actions, a selection criterion and a feedback protocol; this paper’s experiments evaluate the video-generation component. e03e04e17
5.2 Separate a route between cameras from knowledge about geometry
The communication route begins with each camera’s own query and key geometry. Geo-RoPE reserves half of their channels for pixel-ray direction and half for a pose feature shared across that view. Cross-View Attention then lets a query access keys and values from all views at the same time. The zero-initialized residual gate allows this new branch to start without contributing to the pretrained output, while periodic spatial-concat attention offers another cross-view route. These operations make information exchange possible and geometrically biased. The paper’s second pillar determines what should be represented along that route: frozen DA3 features supervise intermediate DiT relations. Reader interpretation: learned camera-dependent similarities should be treated as an inductive bias, rather than an exact geometric correspondence solver or a guarantee against inconsistent predictions. e05e06e07e08
5.3 Use relational supervision to interpret the four-row experiment
Latent 3D-REPA compares relationships between tokens instead of requiring student features to equal teacher features coordinate by coordinate. A random anchor subset supplies cosine similarities from every token to selected references. The spatial loss samples within a frame across views and positions; the temporal loss samples across the entire clip. SmoothL1 aligns these student and DA3 similarity matrices, and the combined REPA loss is weighted by 0.5 alongside flow matching. Table 4 then asks whether this supervisory signal and the architectural pathway work better together. Their joint MEt3R reduction exceeds the summed individual reductions. Reader interpretation: this supports a useful interaction, but the bundled pathway switch and absent uncertainty leave open whether Geo-RoPE, extra attention capacity or optimization differences account for the size of the effect. e07e08e15
5.4 Training and inference
During training
Training uses approximately 2.5M clips: AgiBot-World 35%, RoboMIND 20%, and Galaxea, RoboTwin and RoboCOIN 15% each. Action-conditioned variants receive further task-specific fine-tuning on WorldArena and AgiBot-Challenge2026. e09
The paper reports 30,000 iterations, AdamW, 3,000 warmup iterations to a peak learning rate of 3 × 10⁻⁵, cosine decay and approximately 30,000 NVIDIA H200 GPU-hours. The batch size is described only as proportional to GPU count. DA3 stays frozen; Figure 2 also marks the VAE frozen. The DiT is trainable and REPA projection heads are randomly initialized. e03e08e09
During inference
The requested output is future video conditioned on observations, camera geometry and text or action maps. REPA is an auxiliary training objective, not an inference-time reward or action-selection rule. The supplied method gives no robot feedback controller or action-extraction algorithm. e04e07e08e17
Reader inference from the flow convention: sampling must move from noise at s = 1 toward data at s = 0, followed by VAE decoding. The PDF does not specify the numerical solver, sampling steps or deployment-time camera-estimation schedule; it cannot support a precise latency or closed-loop rollout recipe. e03e04e07e09
5.5 Implementation flow
- Compress and condition
The approximately 14B model uses Cosmos-Predict2.5, a Wan2.1 spatial-temporal VAE and Cosmos-Reason1 text embeddings. Context and future video are encoded into latents. Text modulates DiT blocks through AdaLN; action maps, such as projected end-effector trajectories, concatenate with noisy latents along channels. e03e04e09
- Encode each camera geometrically
Geo-RoPE splits query/key channels equally into ray and pose subspaces. Rays use intrinsics and inverse camera rotation; the 12-dimensional pose vector contains Euler angles, translation, camera position and optical axis. Each view uses its own camera geometry. e05
- Exchange features across views
At selected layers, each view queries keys and values concatenated from all views at the same frame. A residual gate starts at zero. Periodic spatial-concat attention additionally spans view and spatial positions. Reader interpretation: these are learned attention biases, not hard epipolar constraints. e06
- Distill relations during training
A 3D convolutional projector maps intermediate DiT features into the teacher-related representation space. Random anchors provide token-to-anchor cosine relations; SmoothL1 aligns student and frozen DA3 relations within frames and across the clip. DA3 also supplies camera estimates. e07
6. Experiments & results
PAIWorld adds camera-aware communication and geometric representation distillation to a video diffusion transformer. It predicts future camera views from context plus text or supplied actions. Its strongest mechanism evidence is a four-setting ablation in which the combined additions reduce MEt3R more than either alone. The experiments establish generation improvements; claims about better robot planning and policies lack corresponding downstream evaluations in this PDF (e03, e04, e15, e17).
6.1 Read the original evidence
Table 1. The aggregate WorldArena lead coexists with substantial differences between component scores. Original paper, p. 10 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Locate PAIWorld in the last row, then compare one column at a time. All arrows point upward; bold denotes the best result and underlining the second best. EWMScore is the aggregate, while the remaining columns describe different generation properties. The nearest aggregate competitor is UNIS, which also has the strongest Motion Quality. Moving rightward to 3D Accuracy changes the leading method to Pelican-Unify. This matters for a geometry-focused paper: the overall rank should be read alongside the metric most directly relevant to the proposed mechanism, not treated as evidence of winning every quality dimension. e10e11
What it supports. PAIWorld reports EWMScore 72.31 versus UNIS 72.16, a 0.15 score-point margin. Its Motion Quality is 80.45 versus UNIS 81.60, while 3D Accuracy is 91.51 versus Pelican-Unify 97.65. These values establish the reported aggregate lead but a mixed component profile. No confidence intervals are reported.
Where the evidence stops. The introduction calls Motion Quality best; this table and the results paragraph place it second. The paper’s “near the top across every axis” interpretation is also stronger than these rows support. No uncertainty accompanies the small aggregate margin.
Table 2. PAIWorld trades some reconstruction and trajectory score for the highest reported scene-consistency score. Original paper, p. 12 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read across PAIWorld’s row before comparing it with NeoVerse-ABot and Loop. EWMScore summarizes benchmark quality; PSNR concerns reconstruction fidelity. The paper defines Scene Consistency as DINOv2 feature similarity over time, and nDTW as alignment between generated and ground-truth trajectories. Their names should not obscure their different meanings: semantic stability, visual reconstruction and trajectory alignment need not rank models identically. The dashes for Wild Path and VIPL-GENUN are missing reports, so those entries cannot be ranked on component scores. Preserve the decimal scales printed here rather than converting PSNR into an assumed decibel value. e12
What it supports. PAIWorld’s EWMScore is 0.8245, behind NeoVerse-ABot’s 0.829 and ahead of Loop’s 0.8241. Its Scene Consistency is 0.9041 versus NeoVerse-ABot’s 0.8974, an absolute difference of 0.0067. PSNR 0.6161 and nDTW 0.9531 remain below NeoVerse-ABot’s 0.6246 and 0.9651. These are the source’s reported generation metrics.
Where the evidence stops. DINOv2 temporal similarity is not a direct test of epipolar geometry or executed manipulation. The source does not explain the PSNR normalization, and missing competitor components limit the scene-consistency ranking to reported values.
Table 3. Text-conditioned results provide the clearest quantitative geometric comparison, with an important header error. Original paper, p. 14 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Begin with MEt3R on the right: the metric uses point-cloud cross-projection and lower is better. Then compare the adjacent Geometric column, defined in the text as Sampson epipolar error. Its printed upward arrow conflicts with that definition and the lower-is-better discussion on page 15; retain the original header but interpret the error using the prose. On the left, SSIM measures structural correspondence, LPIPS perceptual similarity, and FID/FVD distributional differences. Scene Consistency instead uses temporal semantic features. Comparing these groups separates visual fidelity, temporal semantics and geometry, which do not give identical rankings. e13e14
What it supports. PAIWorld’s MEt3R is 14.20 versus Genie-Envisioner 15.75. It also reports SSIM 0.7683, LPIPS 0.1844, FID 45.0389 and FVD 175.7778. Its Scene Consistency of 0.9041 is below Genie-Envisioner’s 0.9231, so the geometric advantage does not mean uniform superiority across all metrics.
Where the evidence stops. The Cosmos row says Predict2, while Section 4.3 says Predict2.5; that baseline version is unresolved. The prose also misidentifies the second-best LPIPS baseline. The exact test split, sample count and uncertainty are not supplied.
Figure 7. The qualitative comparison invites tracking the same objects through viewpoints and time. Original paper, p. 15 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Compare the left and right halves at the same time label, then move downward through T = 0, 5, 10, 15 and 20. Each method groups prediction images, a reconstruction rendering and depth images. In the upper scene, follow the orange bottle and blue bag across the prediction panels; inspect whether the reconstruction keeps their arrangement coherent as time advances. In the lower cluttered scene, compare the structure of the reconstructed display with the corresponding image views. This paired reading is more informative than judging individual frames for sharpness, because the paper’s target is agreement across cameras. e16e13e17
What it supports. The authors present these selected scenes as visual support for reduced object drift and improved reconstruction consistency. The visible PAIWorld reconstructions appear more stable in these examples, consistent with the direction of the reported MEt3R improvement. The image provides examples of the claimed behavior rather than an estimate of its frequency.
Where the evidence stops. The caption says two viewpoints, but the upper scene visibly shows three prediction/depth panels. The depth colors have no calibrated scale, and selected renderings cannot establish reconstruction accuracy, physical contact correctness or robot task success.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| WorldArena action-conditioned video generation Reported benchmark entries after task-specific fine-tuning; exact evaluation split and sample count are not specified. | 72.31; 80.45; 91.51, respectively. EWMScore ↑; Motion Quality ↑; 3D Accuracy ↑ | UNIS: EWMScore 72.16 and Motion Quality 81.60; Pelican-Unify: 3D Accuracy 97.65. Best reported aggregate, a 0.15 score-point lead over UNIS. Motion Quality is second, contradicting the introduction’s best-motion claim. Aggregate leadership does not establish leadership in 3D accuracy. e09e10e11 |
| AgiBot-Challenge2026 action-conditioned video generation Reported challenge table; component scores are missing for Wild Path and VIPL-GENUN. | 0.8245; 0.9041; 0.6161; 0.9531, respectively. EWMScore ↑; Scene Consistency ↑; PSNR ↑; nDTW ↑ | NeoVerse-ABot: 0.829; 0.8974; 0.6246; 0.9651. Second EWMScore, highest reported Scene Consistency. The latter is DINOv2 temporal semantic similarity, not a direct geometric measurement. Preserve the reported PSNR scale; its normalization is unspecified. e12 |
| AgiBot-World text-conditioned multi-view generation Table 3 compares generated videos on AgiBot-World; test partition and sample count are not given. | 14.20; 0.7683; 0.1844; 45.0389; 175.7778; 0.9041; 0.4056, respectively. MEt3R ↓; SSIM ↑; LPIPS ↓; FID ↓; FVD ↓; Scene Consistency ↑; Geometric error ↓ | Genie-Envisioner: MEt3R 15.75, SSIM 0.7445, Scene Consistency 0.9231; Wan2.1: FID 56.4735, FVD 184.2186. Best reported MEt3R with a 1.55 reduction, while Genie-Envisioner leads Scene Consistency. Geometric error follows the lower-is-better prose despite Table 3’s upward arrow. e13e14 |
| AgiBot-World pathway and REPA ablation Flat-concatenation backbone, pathway only, REPA only and both; CVA bundles Cross-View Attention with Geo-RoPE. | 16.84 → 15.91 / 16.12 → 14.20. MEt3R ↓ | Individual reductions 0.93 and 0.72 sum to 1.65; joint reduction is 2.64. The 0.99 excess over summed gains supports interaction on this metric. It does not isolate Geo-RoPE from attention, and no seed variability is reported. e15 |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Table 4. The joint reduction in geometric error exceeds the sum of the two isolated reductions. Original paper, p. 16 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Treat the first two columns as switches. Both crosses give the flat-concatenation backbone. CVA alone activates the communication pathway, including Geo-RoPE; REPA alone adds geometric relation supervision. The final row enables both. Follow MEt3R downward, remembering that smaller is better, and use delta as the reduction relative to the first row. The source caption defines this grouping, so CVA should not be read as attention with camera encoding held fixed. Finally, check SSIM, LPIPS and FID: each is also best in the final row, showing that the tested geometric improvement accompanies better visual metrics. e15
What it supports. MEt3R falls from 16.84 to 15.91 with CVA and to 16.12 with REPA. Their reductions sum to 1.65, while both reach 14.20, a reduction of 2.64. The joint gain exceeds that sum by 0.99 MEt3R units, supporting a beneficial interaction in this experiment.
Where the evidence stops. This bundled ablation cannot distinguish Geo-RoPE’s contribution from attention’s, and it reports no seed variation or confidence intervals. A super-additive difference on one metric is evidence of interaction, not proof that the two ingredients guarantee 3D consistency.
7. Analysis & limitations
7.1 What the evidence leaves open
Figure 1 and the conclusion advertise planning, action modeling and policy post-training, but Section 4 reports generation benchmarks and ablations. No downstream robot-success comparison substantiates those application gains. The conclusion leaves contact dynamics, deformable objects, fluids and extended-horizon planning for future work. e17
No confidence intervals, seed counts or detailed evaluation partitions are given. Because AgiBot-World is both a training source and evaluation benchmark, the unspecified partition prevents assessing overlap. This is an unresolved protocol issue, not evidence that leakage occurred. e09e13e15
Source inconsistencies remain: Table 3 labels the Cosmos baseline Predict2 while Section 4.3 names Predict2.5; its Geometric arrow contradicts the metric definition. Page 15 calls Genie-Envisioner second-best LPIPS although the table gives Cosmos a lower LPIPS. The exact baseline version remains unresolved. e13e14
7.2 Questions for discussion
- Does the ablation interaction persist with identical training budgets and multiple seeds? (e15)
- How sensitive is Geo-RoPE to camera-estimation error or moving wrist cameras? (e05, e07)
- Do lower MEt3R scores improve executed manipulation under a fixed planner? (e13, e17)
8. Reproducibility audit
8.1 Requirements and known gaps
A faithful implementation needs the specified pretrained modules, synchronized camera data, action-map construction and teacher features. Missing configuration includes effective batch/GPU count, image resolution, clip length/rate, selected attention/alignment layers, anchor counts, fine-tuning schedules and sampler settings. Section 3.5 also says “VGGT feature dimension” while naming DA3 as teacher; projector shape needs clarification. e03e04e06e07e09
Proposed checks should first repeat the four-way ablation under matched data, compute and seeds, then perturb camera geometry while controlling action-map inputs. These discriminate repeatable pathway–objective interaction from gains that do not depend on the claimed camera information. No such new check was run here. e05e06e15
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Repeat the pathway–objective interaction with controlled training
Reader-proposed experiment: reproduce all four CVA/REPA settings from one backbone initialization using the same training clips, optimizer updates, evaluation clips and sampler. Establish and publish a held-out AgiBot-World partition that excludes training episodes. Run at least three seeds and record parameter counts and GPU-hours for each setting. Measure MEt3R plus SSIM, LPIPS and FID, and estimate uncertainty on the joint reduction minus the sum of the isolated reductions. A reliably positive excess would support the claimed interaction; an excess indistinguishable from zero would weaken it. This is a proposed mechanism check, not a claim of reproducing the reported 14B training run. e09e13e15
Check 2: Test whether correct camera geometry actually drives the improvement
Reader-proposed experiment: hold trained weights, initial frames, text/actions, noise seeds and sampler fixed. Compare correct camera parameters with view-permuted parameters and controlled small pose perturbations only at the Geo-RoPE input. Keep rendered action maps unchanged so action-conditioning errors do not explain the result. Evaluate MEt3R and Sampson error, with SSIM and Scene Consistency as complementary controls; stratify by camera pair and motion. Geometry-dependent routing predicts worse geometric scores after mismatching cameras, with graded sensitivity to perturbation. Little change would challenge the claim that correct camera encoding drives the gain. This tests input sensitivity; it does not replace a separate retrained Geo-RoPE ablation. e04e05e06e13e15
8.3 Reading coverage
Visual audit: PDF pages 1–17 were rendered and actually viewed: title/version/affiliation on page 1, contributor credits on page 17, Figures 1–7, Tables 1–4, complete method/equation pages 5–9, training configuration on page 10 and evaluation/conclusion pages 10–16. All six final original crops were individually viewed and retain table headers or figure legends. Figure 2 was checked against Eqs. (2)–(12): frozen markers and attention routing were inspected, while its latent-output label and first-frame-only DA3 drawing were disclosed as schematic mismatches with the training formulation. Table 3’s Geometric arrow and baseline-name discrepancy, Table 1’s motion-ranking discrepancy and Figure 7’s viewpoint-count discrepancy are preserved and explained. Pages 18–20 contain references and were read as text; no empirical claims here rely on their page images. No appendix is present in the supplied PDF and no separate supplement was supplied. No code was inspected or executed, and no experiments were reproduced. The title-page byline is retained exactly. PDF page 17 supports the observed contributor transcription; the separate catalog-credit comparison in base-report coverage explicitly reproduces the supplied author snapshot and does not attribute that metadata to the PDF or infer a revision history.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17. Appendix coverage: not present.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Abstract (p. 1)
- 1 Introduction (pp. 1–3)
- 2.1 World Foundation Models for Physical AI (pp. 3–4)
- 2.2 Multi-View and 3D-Aware Visual Generation (p. 4)
- 2.3 3D Representations and Geometric Reconstruction (pp. 4–5)
- 3 Method, including 3.1–3.7 and Eqs. (1)–(12) (pp. 5–9)
- 4 Experiments, including 4.1–4.4 (pp. 9–16)
- 5 Conclusion and future directions (p. 16)
- Contributions (p. 17)
- References [1]–[60] (pp. 17–20)
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Separate supplemental material availability has not been fully verified.
- The extraction-only image limitation was addressed by visually inspecting PDF pages 1–17, every original figure and table, and all six final crops. References on pages 18–20 were read as text.
- No separate supplement was supplied. No appendix is present in the 20-page PDF. Code and external benchmark documentation were not inspected; no experiments were reproduced.
- Observed identity/version (e01): the PDF title exactly matches the supplied observed title. The title page displays arXiv:2606.18375v3 [cs.RO], 23 Jun 2026, and credits The PAIWorld Team, with the affiliation Institute of AI for Industries, Chinese Academy of Sciences. Page 17 separately lists core contributors, contributors and corresponding author Kai Xu. Earlier versions were not supplied or compared; no revision-to-revision author change is established.
- Supplied catalog author snapshot used for the identity comparison (bibliographic metadata, not a statement from the PDF): Yuhang Huang; Xuan Lv; Junyan Xu; Zhiyuan Yu; Jiazhao Zhang; Ruizhen Hu; Wancheng Feng; Shilong Zou; Hewen Xiao; Ziqiao Zhou; Kaiyun Huang; Zhiyu Peng; Juzhan Xu; Hang Zhao; Chenyang Zhu; Renjiao Yi; Yifei Huang; Douhui Wu; Yan Zhang; Kexu Cheng; Chunhe Song; Yunzhi Xue; Xiuhong Zhang; Leitao Guo; Yunji Chen; Bin Wu; Haibin Yu; Kai Xu.
- Reader comparison of that supplied catalog snapshot with the observed contributor groups transcribed in e01: all catalog-listed individuals appear on PDF page 17. The group ordering differs: Jiazhao Zhang is second among core contributors, and Kai Xu is listed with the core contributors rather than at the end of the combined credits. Zhibin Zhu appears in the PDF contributor group but is absent from the supplied catalog string. These are differences between the supplied metadata and observed credits; they do not establish that a contributor was added in a particular revision. Metadata for this edition preserves the verified title-page byline.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e01PDF p. 1, title/author block and arXiv margin; p. 17, Contributions
The title is PAIWorld: A 3D-Consistent World Foundation Model for Robotic Manipulation. The byline is The PAIWorld Team; affiliation is Institute of AI for Industries, Chinese Academy of Sciences. The margin reads arXiv:2606.18375v3 [cs.RO], 23 Jun 2026. Core contributors are Yuhang Huang, Jiazhao Zhang, Xuan Lv, Junyan Xu, Zhiyuan Yu, Ruizhen Hu and Kai Xu. Contributors are Wancheng Feng, Shilong Zou, Hewen Xiao, Ziqiao Zhou, Kaiyun Huang, Zhiyu Peng, Juzhan Xu, Hang Zhao, Zhibin Zhu, Chenyang Zhu, Renjiao Yi, Yifei Huang, Douhui Wu, Yan Zhang, Kexu Cheng, Chunhe Song, Yunzhi Xue, Xiuhong Zhang, Leitao Guo, Yunji Chen, Bin Wu and Haibin Yu. Kai Xu is corresponding author.
Go to primary source ↓e02PDF pp. 2–3, Section 1; p. 4, Section 2.2
The authors motivate cross-view object, depth and texture consistency and argue for an information pathway plus geometric supervision; robotic cameras are characterized as sparse, wide-baseline scene views.
Go to primary source ↓e03PDF p. 5, Figure 2 and caption
The diagram routes images to frozen-marked DA3 and spatial-temporal VAEs, camera geometry toward Geo-RoPE/cross-attention, and intermediate DiT features to relation alignment. The DiT is marked trainable. Future-image latents feed a flow-matching loss. The drawing labels its output Predicted latent and shows DA3 receiving first-frame images; the exact velocity target and clip-wide teacher relation loss are specified separately in Eqs. (3), (8)–(11).
Go to primary source ↓e04PDF p. 6, Sections 3.1–3.2, Eqs. (1)–(3)
Conditional future multi-view video uses context, intrinsics/extrinsics and text/actions. A Wan2.1 VAE produces latents; linear interpolation and velocity loss are explicit. Text uses AdaLN; actions are spatial maps concatenated along channels. No solver or action-output rule is specified here.
Go to primary source ↓e05PDF p. 7, Section 3.3, Eqs. (4)–(6)
Queries/keys split into equal ray and pose subspaces. Rays use inverse intrinsics and transposed rotation. The 12-dimensional camera feature includes Euler angles, translation, camera position and optical axis. Geo-RoPE rotates each subspace separately.
Go to primary source ↓e06PDF pp. 7–8, Section 3.4, Eq. (7) and Spatial-Concat Self-Attention
View-specific rotated queries attend to concatenated keys/values across all views at a frame. A residual AdaLN-Zero gate initializes to zero. Periodic attention flattens V × H × W positions. Specific insertion layers and period are not listed.
Go to primary source ↓e07PDF pp. 8–9, Section 3.5, Eqs. (8)–(11)
Frozen DA3 supplies geometric features and camera estimates. A 3D convolutional projector processes intermediate DiT features; the prose inconsistently calls its target the VGGT feature dimension. Token-to-anchor cosine relations are aligned with SmoothL1 within frames and across the clip. Anchor sampling has O(MK) cost; numeric anchor counts and alignment layer are unspecified.
Go to primary source ↓e08PDF p. 9, Sections 3.6–3.7, Eq. (12)
Total loss adds REPA to flow matching with lambda 0.5. DA3 is frozen; cross-view gates initialize to zero. The authors describe complementary architecture and supervision.
Go to primary source ↓e09PDF p. 10, Section 4.1, Base Model / Dataset / Training Configuration
Cosmos-Predict2.5 backbone, approximately 14B parameters, Cosmos-Reason1 text embedder and random REPA heads are specified. Approximately 2.5M clips use proportions 35/20/15/15/15 across AgiBot-World/RoboMIND/Galaxea/RoboTwin/RoboCOIN. Action variants receive task-specific fine-tuning. Training reports 30,000 iterations, AdamW, 3,000-step warmup, peak 3 × 10⁻⁵, cosine schedule and approximately 30k H200 GPU-hours; batch size only scales with unspecified GPU count.
Go to primary source ↓e10PDF p. 10, Table 1, PAIWorld / UNIS / Pelican-Unify rows and metric columns
PAIWorld reports EWMScore 72.31, Visual Quality 63.04, Motion Quality 80.45, Content Consistency 57.85, Physics Adherence 61.66, 3D Accuracy 91.51 and Controllability 87.16. UNIS reports EWMScore 72.16, Motion Quality 81.60 and Controllability 90.19. Pelican-Unify has 3D Accuracy 97.65. Bold marks best and underlining second-best.
Go to primary source ↓e11PDF p. 3, contribution bullet 3; p. 11, Section 4.2.1 metric list and results paragraph
The introduction says best Motion Quality, but the results paragraph correctly calls 80.45 second to UNIS 81.60. EWMScore is described as aggregate quality. The claim of near-top performance on every axis must be checked against Table 1, which shows several substantially lower component scores.
Go to primary source ↓e12PDF p. 12, Table 2 and Section 4.2.2 metric definitions; p. 13, nDTW definition/results paragraph
PAIWorld EWMScore/PSNR/Scene Consistency/nDTW are 0.8245/0.6161/0.9041/0.9531. NeoVerse-ABot reports 0.829/0.6246/0.8974/0.9651; Loop EWMScore is 0.8241. Wild Path and VIPL-GENUN omit component metrics. Scene Consistency is DINOv2 temporal semantic similarity; nDTW measures trajectory alignment. PSNR normalization is not explained.
Go to primary source ↓e13PDF p. 14, Table 3, all rows; Section 4.3 metric definitions
PAIWorld SSIM/LPIPS/FID/FVD/Scene Consistency/Geometric/MEt3R are 0.7683/0.1844/45.0389/175.7778/0.9041/0.4056/14.20. Genie-Envisioner has 0.7445/0.3345/83.7847/207.2025/0.9231/0.5327/15.75. Cosmos-Predict2 LPIPS is 0.3251 and MEt3R 17.47; Wan2.1 FID/FVD/MEt3R are 56.4735/184.2186/16.59. Geometric is defined as Sampson epipolar error, lower better; MEt3R uses point-cloud cross-projection.
Go to primary source ↓e14PDF p. 13, Section 4.3 baseline list; p. 14, Table 3 header and metric list; p. 15, Section 4.3 interpretation
Section 4.3 names Cosmos-Predict2.5 while Table 3 names Cosmos-Predict2. Table 3 prints an upward Geometric arrow, but the metric definition and results prose say lower is better. The prose identifies Genie-Envisioner as second-best LPIPS, whereas Cosmos has the lower baseline LPIPS. These discrepancies are retained rather than silently corrected in the crops.
Go to primary source ↓e15PDF pp. 15–16, Section 4.4; p. 16, Table 4, all four CVA/REPA rows
CVA means Cross-View Attention with Geo-RoPE. Neither/pathway-only/REPA-only/both have MEt3R 16.84/15.91/16.12/14.20. Corresponding SSIM is 0.6912/0.7204/0.7156/0.7683, LPIPS 0.2783/0.2361/0.2447/0.1844 and FID 53.17/50.02/49.88/45.04. Joint improvement 2.64 exceeds 0.93 + 0.72. No uncertainty or separate Geo-RoPE ablation is provided.
Go to primary source ↓e16PDF p. 15, Figure 7 and caption
Two scene groups compare Genie-Envisioner and PAIWorld using prediction, reconstruction and depth panels, with time labels T = 0, 5, 10, 15, 20. The authors interpret these as improved cross-view consistency. The top group visibly contains three prediction/depth view panels although the caption says two viewpoints; no calibrated depth color scale is supplied.
Go to primary source ↓e17PDF p. 2, Figure 1; pp. 9–16, Section 4 and Tables 1–4; p. 16, Section 5
Figure 1 sketches world generation, world action models, robotic planning and policy post-training. The empirical section evaluates action/text-conditioned video generation and a generation ablation, without corresponding planner/policy success tables. The conclusion asserts downstream benefits but lists contact/deformable/fluid modeling, longer planning horizons and a world-model/WAM data loop as future directions.
Go to primary source ↓8.5 Primary sources
PAIWorld: A 3D-Consistent World Foundation Model for Robotic Manipulation ↗
PDF · 8,729 extracted words
Source fingerprint
aa24ba7f72937015c1ac295bae45fd574bdea1ed8768364dc4d64687ff4bd9cf