Generative World Modelling for Humanoids: 1X World Model Challenge Technical Report
1. Paper overview
In one sentence: State-conditioned video generation and autoregressive token prediction win two forecasting tracks, while their inference choices expose a gap between benchmark scores and useful imagined futures. e-taskse-sampling-methode-compression-modele-leaderboarde-sampling-ablatione-compression-diagnostic
| At a glance | What to know |
|---|---|
| Research problem | Source description Given observed video and supplied past and future robot states, forecast visual consequences. Sampling scores the 512×512 final frame two seconds ahead by PSNR; compression scores future Cosmos tokens by top-500 cross-entropy. Future states are privileged conditioning within these protocols, not actions selected by either predictor. e-taskse-compression-data |
| Core mechanism | Source description Adapt a pretrained flow-matching video model through fixed context latents and state-driven adaLN-Zero modulation; average stochastic predictions to improve the pixel-based challenge score. e-sampling-methode-sampling-inference |
| A key reported result | Sampling challenge leaderboard: 23.00 dB; rank 1. Final-frame PSNR (higher is better). Public challenge test set; first submission additionally trained on train plus validation raw data. Duke: 21.56 dB; Michael: 18.51 dB. A leaderboard advantage under submitted systems' differing training and inference choices. The associated 26.62 dB validation score is contaminated by training exposure. e-leaderboarde-sampling-protocol |
| Reading caution | Reader analysis The authors leave the best sampling strategy for downstream decision-making open. Frame and token scores establish neither action selection nor physical execution success; supplied future states make the evaluation conditional on information a controller would need to produce. e-conclusione-tasks |
Core contributions
- Source description
Adapt a pretrained flow-matching video model through fixed context latents and state-driven adaLN-Zero modulation; average stochastic predictions to improve the pixel-based challenge score. e-sampling-methode-sampling-inference
- Source description
Train a separate Transformer from scratch with factorized spatial and causal temporal attention, then use greedy autoregressive token prediction. e-compression-modele-compression-inference
Figure 1. The two challenge tracks differ in representation, horizon and scoring. Original paper, p. 1 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read each row from left to right: context plus robot states, model generation, and comparison with the target. The upper row represents RGB sampling. Its caption specifies seventeen observed frames and evaluation at the seventy-seventh frame. The lower row inserts a tokenizer before prediction: three context grids lead to three future grids, with each block corresponding to seventeen RGB frames. The double-headed arrows on the right mark evaluation comparisons, not a robot feedback loop. Sections 2 and 3 clarify that the state input includes future as well as past states. e-taskse-compression-data
What it supports. This overview separates two conditional prediction tasks that should not be merged into one score. Sampling evaluates a future image with PSNR; compression evaluates discrete predictions with top-500 cross-entropy. The shared humanoid imagery establishes the application setting, while the target representation determines each model's output.
Where the evidence stops. The supplied future-state trajectory is an input to prediction. The figure does not demonstrate that a policy selects that trajectory, that a robot executes it, or that visual prediction improves task success.
2. Motivation
2.1 The problem and the proposed response
Given observed video and supplied past and future robot states, forecast visual consequences. Sampling scores the 512×512 final frame two seconds ahead by PSNR; compression scores future Cosmos tokens by top-500 cross-entropy. Future states are privileged conditioning within these protocols, not actions selected by either predictor. e-taskse-compression-data
2.2 What this reading follows
A humanoid world model can be evaluated by the pixels it predicts or by the discrete video tokens it assigns probability to. Team Revontuli builds a different system for each setting: a pretrained Wan video generator adapted to robot states, and a Transformer trained from scratch on tokenized interactions. Both receive future robot states as part of the benchmark input. This distinction matters when interpreting the leaderboard: the experiments test conditional forecasting, while robot decision-making remains a separate question. The most revealing evidence comes from inference choices—averaging improves PSNR but worsens perceptual metrics, and generated token histories expose errors hidden by teacher forcing. e-taskse-sampling-methode-compression-modele-leaderboarde-sampling-ablatione-compression-diagnostic
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | Not assigned |
| Architecture | Not assigned |
| Prediction paradigm | Not assigned |
| Quadrant | Not assigned |
This table preserves the labels recorded at reading time. The current major category is Benchmarks & simulators. View the current classification.
3.1 Evidence-based assessment
Insufficient evidence to decide
The catalog records no classification to confirm or dispute. Architecture evidence shows two separate conditional world predictors: latent flow matching and autoregressive token modeling. Neither jointly predicts executable actions nor implements inverse dynamics. A One Model world-action label is unsupported; under an action-output quadrant definition, this is outside that scope, not an unverified claim of integration. e-sampling-methode-compression-modele-compression-inferencee-tasks
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Turn an image-conditioned generator into a state-conditioned forecaster
Start with what the sampling model is allowed to know: past video and a complete robot-state trajectory. Wan already supplies a strong latent video generator, so the adaptation changes its conditioning rather than training visual dynamics from scratch. Multiple observed latents remain fixed during generation. The state branch downsamples and embeds the trajectory, temporally compresses it, and produces modulation aligned with latent slices. Adding that modulation to the flow-timestep modulation lets the same DiT blocks respond to both generation time and robot state. Training uses rank-32 LoRA on the backbone; inference generates futures from the fixed context. The paper's 77-to-21 frame preprocessing arithmetic remains unresolved, so the high-level conditioning mechanism is clearer than the exact indexing recipe. e-taskse-sampling-methode-sampling-training
Figure 2. Robot-state modulation joins flow-time modulation inside each Wan block. Original paper, p. 2 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Follow the latent path across the top: normalized and modulated self-attention, retained cross-attention, then a modulated MLP, each with a residual connection. At the bottom, the state-embedding arrow enters the same summation node as the timestep embedding. Its output supplies the scale/shift and scaling branches drawn above. Section 2.1 explains the distinction hidden by this compact diagram: flow-time modulation is shared over the latent sequence, but robot-state modulation varies with the corresponding temporal slice. The repeated-block label applies to the thirty-layer Wan backbone. The separate video-conditioning operation fixes observed latents and is described in the text rather than drawn here. e-sampling-methode-sampling-traininge-sampling-ablatione-sampling-protocol
What it supports. The adaptation supplies robot information through modulation of an existing video generator. It preserves the backbone's attention structure while making its latent transformations depend on aligned states. Empty prompts retain the cross-attention path, but the challenge data provide no text descriptions for the model to exploit.
Where the evidence stops. The diagram is consistent with the additive conditioning described in Section 2.1. It does not resolve the reported preprocessing frame-count mismatch, and the experiments do not isolate the benefit of state conditioning with a state-removal ablation.
5.2 Understand why the winning image can be blurrier
The cleanest sampling experiment holds the non-CFG model's inference-step count fixed and changes how many predictions are averaged. PSNR and SSIM improve, but LPIPS and FID worsen. The authors explain averaging as selective blurring in uncertain regions, such as moving arms. Reader interpretation: when plausible futures disagree about precise image content, their mean can reduce pixel error without resembling one coherent sampled future. The experiment supports that metric tradeoff; it does not demonstrate which output is most useful for planning. Separately, adding validation data to training changes the data protocol, and increasing steps changes inference compute. Those rows should inform deployment choices individually rather than being folded into a single universal improvement claim. e-sampling-inferencee-sampling-ablatione-sampling-protocole-conclusion
Figure 3a. Factor attention across space and time, then predict the next token grids. Original paper, p. 4 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Begin with the embedded token grids at left. The robot-state branch passes through an MLP and convolution, then enters the addition node before the repeated ST blocks. Within each block, spatial attention connects positions in one frame, while temporal attention connects a fixed spatial position across frames; normalization and residual paths surround these operations and the MLP. The colors illustrate attention groupings, not learned semantic classes. The caption and Section 3.1 specify causal temporal attention, although this schematic does not explicitly draw the mask. The output linear layer provides token predictions; Section 3.3 supplies the autoregressive rollout procedure. e-compression-modele-compression-datae-compression-inference
What it supports. Factoring attention avoids applying one joint attention operation over every space-time token pair. The source says the spatial-attention bottleneck then scales linearly with frame count. This architecture supports a separately trained discrete world predictor, with additive state conditioning and an output distribution over future tokens.
Where the evidence stops. The drawing's H labels spatial height, whereas the problem statement uses H for context length. The text also gives T=5 for a dataset of six grids without fully specifying the alignment. These notation and indexing details require clarification for reproduction.
5.3 Separate token learning from token rollout
Compression begins with three observed token grids and predicts three future grids conditioned on supplied states. During teacher-forced training, the model sees correct prior tokens, which permits parallel loss computation across time under causal attention. At inference, it must consume its own earlier predictions. Each spatial position receives a categorical distribution, and greedy decoding supplies the next grid used as context. The orange-versus-green curves make this change visible: teacher-forced validation is more optimistic than autoregressive validation. The implementation also changes numerical precision, using bfloat16 training but float32 inference because the former degraded prediction performance at inference. Scheduled sampling did not meaningfully close the rollout gap in the authors' experiments, though the report omits the settings needed to assess that negative result. e-compression-datae-compression-modele-compression-traininge-compression-inferencee-compression-diagnostic
5.4 Training and inference
During training
Sampling uses rank-32 LoRA on the DiT for 23,000 AdamW steps at constant learning rate 4×10⁻⁴ and effective batch 1,024. The reported cluster has four nodes with eight NVIDIA B200 GPUs each. Runs with and without classifier-free guidance are compared. e-sampling-training
Compression uses approximately 306,000 samples, each containing six grids (6,144 tokens) plus aligned states. Teacher forcing minimizes future-grid cross-entropy. Training lasts 80 epochs with fused AdamW, β₁=0.9, β₂=0.95, matrix-only weight decay 0.05, tied input/output embeddings and batch 160. After 2,000 warmup steps, learning rate decays linearly from 8×10⁻⁴ to zero. Training uses bfloat16 AMP on the same cluster. e-compression-datae-compression-training
During inference
Sampling keeps observed latents fixed, generates multiple stochastic futures and averages predictions. Table 2 varies ensemble size, denoising steps and CFG scale. The challenge imposes no inference-time compute restriction; this does not establish real-time operation. e-sampling-methode-sampling-inference
Compression predicts categorical distributions per spatial token and greedily feeds selected grids into later steps. Inference uses float32 because bfloat16 degraded performance. No inverse-dynamics module, action decoder or execution feedback loop is described. e-compression-traininge-compression-inferencee-tasks
5.5 Implementation flow
- Align and fix visual context
Wan 2.2 TI2V-5B supplies a 30-layer DiT and Wan2.2-VAE latents. The source reports downsampling 77 frames to 21, with five conditioning frames and sixteen targets. Multiple context latents remain fixed during generation. Empty text prompts retain the original cross-attention path. The frame-count inconsistency is unresolved below. e-sampling-method
- Inject time-aligned robot information
Angle and velocity features receive sinusoidal augmentation. An MLP projects states to dimension 256; a two-layer 1D convolution compresses time to align with video latents. Another MLP produces adaLN-Zero modulation, added to flow-timestep modulation. State modulation varies across corresponding temporal slices; timestep modulation is shared. e-sampling-method
- Factor token attention
Compression alternates attention within each 32×32 grid and causal attention across time at each spatial coordinate, followed by an MLP. The model uses 24 layers, eight heads, dimension 512, sequence length T=5, pre-LayerNorm, QKNorm, learned absolute positions and dropout 0.1. State embeddings pass through an MLP and convolution before addition to video embeddings. e-compression-model
6. Experiments & results
Team Revontuli uses two separate, state-conditioned predictors: a LoRA-adapted Wan video model for future RGB frames and a spatio-temporal Transformer for future discrete tokens. Both lead the reported challenge leaderboard, but their scores measure conditional prediction rather than executed humanoid control.
6.1 Read the original evidence
Table 1. First place in both tracks, under distinct prediction metrics. Original paper, p. 1 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start with the Benchmark column, because the numerical columns mean different things in the two row groups. Sampling uses PSNR in decibels, with separate test and validation columns; higher is better. Compression uses cross-entropy, where lower is better, and only its test header explicitly says Top-500. Dashes indicate absent or inapplicable entries and should remain missing. Compare submitters within one track and split. Then cross-check Table 2: the 23.00 test PSNR and 25.53 validation PSNR displayed together here belong to different sampling configurations there, so this row is a leaderboard summary rather than a fully specified run. e-leaderboarde-sampling-protocole-tasks
What it supports. Revontuli reports test PSNR 23.00 dB against Duke's 21.56 dB and test top-500 CE 6.64 against Duke's 7.50. The abstract gives the latter more precisely as 6.6386. These comparisons establish the reported leaderboard ordering for the two challenge tracks.
Where the evidence stops. This is not a matched-compute or matched-training ablation. Table 2 identifies training on validation data for the first sampling submission; the leaderboard supplies neither uncertainty intervals nor evidence of executed robot success.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| Sampling challenge leaderboard Public challenge test set; first submission additionally trained on train plus validation raw data. | 23.00 dB; rank 1. Final-frame PSNR (higher is better) | Duke: 21.56 dB; Michael: 18.51 dB. A leaderboard advantage under submitted systems' differing training and inference choices. The associated 26.62 dB validation score is contaminated by training exposure. e-leaderboarde-sampling-protocol |
| Sample-averaging ablation Table 2 first three rows: no CFG, 20 inference steps, 1/5/20 averaged samples; validation and test reported separately. | Validation PSNR: 22.63/24.52/24.88 dB; test: 21.05/22.11/22.42 dB. Validation SSIM: 0.707/0.750/0.762; LPIPS: 0.137/0.165/0.201; FID: 40.23/71.46/90.71. PSNR, SSIM, LPIPS and FID | One sample is the control; larger averages improve PSNR and SSIM but worsen LPIPS and FID. The metric tradeoff contradicts any blanket claim that all visual-quality measures improve with averaging. e-sampling-ablation |
| Post-deadline sampling configuration Table 2 last row: 100 inference steps, 20 samples, CFG scale 1.0. | 25.53/23.04 dB; test result explicitly obtained after the deadline. Validation/test PSNR | At 20 steps and CFG 1.5: 24.59/22.53 dB; at 100 steps and CFG 1.5: 25.07/22.55 dB, also post-deadline on test. Keep the 23.04 dB result separate from the official 23.00 dB submission. Table 1 pairs validation 25.53 with test 23.00, which Table 2 assigns to different configurations. e-sampling-protocole-leaderboard |
| Compression challenge leaderboard Public challenge test set; validation CE is a separate column. | 6.6386 in the abstract, rounded to 6.64 in Table 1; rank 1. Validation CE: 4.92. Test top-500 cross-entropy (lower is better) | Duke: test 7.50, validation 5.60; a27sridh: test 7.99. Supports stronger token prediction on this benchmark. No uncertainty intervals or executed-task success are reported. e-leaderboarde-compression-data |
| Teacher-forcing versus rollout diagnostic Figure 3b: compression training and validation curves. | Autoregressive validation remains above teacher-forced validation; no exact endpoint is tabulated in this plot. Cross-entropy loss | Same model evaluated using generated versus ground-truth history. Teacher-forced validation is optimistic. Scheduled sampling reportedly gave no meaningful improvement, but its settings and quantitative ablation are absent. e-compression-diagnostic |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Table 2. Averaging raises pixel accuracy while exposing a perceptual tradeoff. Original paper, p. 3 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Use the first three rows for the clearest comparison: inference steps remain twenty, CFG is absent, and the number of averaged samples increases from one to five to twenty. The first header says 'Num. Inf. Samples'; Sections 2.3–2.4 discuss that setting as inference or sampling steps, distinct from the adjacent ensemble-size column. Read the arrows beside each metric: PSNR and SSIM improve upward, LPIPS and FID downward. Keep the starred first submission separate because it also trains on validation data. The final block increases inference steps to one hundred; its daggered test scores were obtained after the deadline. Both footnotes are retained in the crop. e-sampling-ablatione-sampling-protocole-sampling-inference
What it supports. From one to twenty averaged samples, validation PSNR rises from 22.63 to 24.88 dB and SSIM from 0.707 to 0.762. Yet LPIPS increases from 0.137 to 0.201 and FID from 40.23 to 90.71, both worse. Averaging helps the challenge's pixel objective without improving every measure of image quality.
Where the evidence stops. The starred 26.62 dB validation value is not a held-out result. The 23.04 dB test score is post-deadline. Neither should replace the official 23.00 dB submission or be pooled with ordinary validation rows.
Figure 3b. Ground-truth history makes validation easier than autoregressive rollout. Original paper, p. 4 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the horizontal axis as optimizer steps and the vertical axis as cross-entropy loss. Blue is teacher-forced training; orange and green evaluate the validation set under different histories. Orange supplies ground-truth tokens at previous steps, while green uses greedy predictions as subsequent context. Their separation therefore concerns the evaluation protocol, not just train-versus-validation generalization. The caption explicitly identifies green as the inference-time procedure. Section 3.4 reports that scheduled sampling was tried to reduce the gap, without meaningful improvement. The graph provides a qualitative trajectory rather than a table of exact endpoints or uncertainty. e-compression-diagnostice-compression-inferencee-compression-traininge-leaderboard
What it supports. The model's conditional predictions look stronger when previous future grids are supplied correctly. Greedy rollout exposes a persistent loss gap as prediction errors enter later context. This is the most direct diagnostic of the compression model's training-to-inference mismatch in the report; no exact gap is claimed from the plotted curves.
Where the evidence stops. The authors describe teacher-forced validation as an idealized lower bound, but do not prove a general bound. Scheduled-sampling settings and numerical comparisons are absent, and this plot should not be equated with the separate top-500 test leaderboard.
7. Analysis & limitations
7.1 What the evidence leaves open
The authors leave the best sampling strategy for downstream decision-making open. Frame and token scores establish neither action selection nor physical execution success; supplied future states make the evaluation conditional on information a controller would need to produce. e-conclusione-tasks
Preprocessing says factor-four downsampling of indices 0–76 yields 21 frames, although those regularly spaced indices yield 20. Padding or endpoint handling is unspecified. The compression objective also switches from H to an undefined K; the reported T=5 alignment with six dataset grids is not explained. e-sampling-methode-compression-datae-compression-modele-compression-training
The paper reports neither uncertainty across seeds nor detailed split construction. Its optimistic teacher-forced loss is called a lower bound, but no general bound is proved. Leaderboard ranking and qualitative learning curves should not be read as controlled evidence for every architectural component. e-leaderboarde-sampling-protocole-compression-diagnostic
7.2 Questions for discussion
- When does sample averaging preserve enough detail for downstream decisions despite its LPIPS/FID cost?
- How much of the compression rollout gap comes from generated-history errors rather than distribution calibration?
8. Reproducibility audit
8.1 Requirements and known gaps
Required artifacts include the challenge raw/tokenized data, Wan 2.2 TI2V-5B and its VAE, and the Cosmos tokenizer. Exact split identifiers, state normalization, LoRA target modules, frozen-module inventory, sampler/scheduler configuration and software versions are not specified. Resolve temporal alignment before claiming faithful reproduction. e-sampling-methode-sampling-traininge-compression-datae-compression-modele-compression-training
The authors report reaching the sampling leaderboard lead in 36 hours and training compression in under 17 hours. These are author-reported timings on the described cluster, not controlled hardware-normalized comparisons or independently reproduced costs. e-conclusione-sampling-traininge-compression-training
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Test averaging against a controlled blur baseline
Reader-proposed check, not performed: freeze one non-CFG checkpoint and evaluate the same held-out clips with twenty inference steps and ensembles of one, five and twenty samples. Add a single-sample Gaussian-blur control, selecting its blur strength on a separate tuning subset. Report PSNR, SSIM, LPIPS and FID on identical targets, together with generation cost and paired uncertainty estimates. Keep all training exposure to validation clips excluded. The reported tradeoff predicts higher PSNR but worse LPIPS/FID as ensemble size grows; failure to reproduce those directions would challenge the averaging explanation for this checkpoint. e-sampling-ablatione-sampling-protocole-sampling-inference
Check 2: Measure error accumulation under matched token inference
Reader-proposed check, not performed: first resolve the documented grid/state indexing and confirm the temporal mask excludes future video tokens. On one fixed held-out set and checkpoint, compare teacher-forced and greedy autoregressive validation using identical states and float32 precision. Record CE separately for each future grid, keeping full-distribution validation CE distinct from the top-500 test metric. Then repeat both modes in bfloat16 without changing other settings. A rollout gap that grows with prediction depth would support accumulated-history errors; a similar precision penalty in both modes would identify a separate numerical effect. Neither pattern alone establishes control usefulness. e-compression-datae-compression-modele-compression-traininge-compression-inferencee-compression-diagnostic
8.3 Reading coverage
Visual audit: All six original PDF pages were rendered and visually inspected, including the title/author/version block, all method and training text, equations, evaluation protocols, results and references. Figure 1, Figure 2, Figure 3a, Figure 3b and Tables 1–2 were inspected as final original crops. Table 2 retains its train-plus-validation and post-deadline footnotes. Figure 2's additive modulation arrows agree with Section 2.1; Figure 3a's causal temporal behavior is specified by the caption and text, not an explicit mask drawing. Pages 1–4 cover every scientific claim and proposed-check premise. No appendix is present; separate supplements, cited works, code and experiments remain outside this reading.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6. Appendix coverage: not present.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Title, authors, affiliations and version (p. 1)
- Abstract and 1. Introduction (p. 1)
- 2. Sampling Challenge: problem and preprocessing (p. 2)
- 2.1. Model; 2.2. Training; 2.3. Inference (pp. 2–3)
- 2.4. Results (p. 3)
- 3. Compression Challenge: problem and data (p. 3)
- 3.1. Model; 3.2. Training (pp. 3–4)
- 3.3. Inference; 3.4. Results; 4. Conclusion (p. 4)
- References [1]–[41] (pp. 5–6)
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Identity: the supplied catalog title matches the main title and technical-report subtitle. The printed subtitle additionally ends '- Team Revontuli'. All nine authors match in order. The inspected artifact is arXiv:2510.07092v1 [cs.LG], 8 October 2025; no other edition was supplied or compared.
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout. This extraction limitation was addressed by inspecting all six PDF pages and every final crop.
- Separate supplemental material availability has not been fully verified.
- No appendix is present in this six-page PDF. No external references, code, checkpoints or supplementary files were inspected; no experiments were reproduced.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e-identityPDF p. 1, title block, author superscripts, equal-contribution footnote and arXiv margin
Main title and technical-report subtitle include Team Revontuli; the nine named authors and five affiliations are visible. Margin identifies arXiv:2510.07092v1 [cs.LG], 8 Oct 2025. Mereu and Scannell have equal-contribution marks.
Go to primary source ↓e-tasksPDF p. 1, Figure 1 and caption; p. 2, Section 2 Problem Statement; p. 3, Section 3 Problem Statement
Sampling observes 17 frames and past/future states, scoring the 77th frame at 512×512 by PSNR. Compression uses three context grids and predicts three more; both tracks condition on supplied states.
Go to primary source ↓e-sampling-methodPDF p. 2, Data Pre-processing, Section 2.1 and Figure 2/caption
Wan 2.2 TI2V-5B has 30 DiT layers. Context latents are fixed; empty text keeps cross-attention. State sinusoidal features, 256-dimensional projection, two-layer temporal convolution and an MLP produce adaLN-Zero modulation added to timestep modulation. The text states 77-to-21 downsampling, five context frames, sixteen targets and VAE temporal length 1+(L−1)/4.
Go to primary source ↓e-sampling-trainingPDF p. 2, Section 2.2 Training
Reports rank-32 LoRA, AdamW, 23k steps, constant learning rate 4×10⁻⁴, effective batch 1024 and four nodes each with eight NVIDIA B200 GPUs; compares CFG and non-CFG training.
Go to primary source ↓e-sampling-inferencePDF p. 2, Section 2.3; p. 3, Section 2.4
Inference compute is unrestricted. Authors describe averaging predictions to blur uncertain regions, tuning ensemble size, inference steps and CFG, and contrasting averaging with Gaussian blur.
Go to primary source ↓e-sampling-ablationPDF p. 3, Table 2 first block, first three rows; Section 2.4 first-block discussion
For 1/5/20 samples at 20 inference steps without CFG: validation PSNR 22.63/24.52/24.88, test PSNR 21.05/22.11/22.42, SSIM 0.707/0.750/0.762, LPIPS 0.137/0.165/0.201 and FID 40.23/71.46/90.71.
Go to primary source ↓e-sampling-protocolPDF p. 3, Table 2 caption, starred submission row, CFG blocks and daggered test cells; Section 2.4
Starred first submission trains on train+validation, reporting validation/test PSNR 26.62/23.00. At 20 steps, 20 samples, CFG 1.5: 24.59/22.53. At 100 steps, 20 samples, CFG 1.5: 25.07/22.55; CFG 1.0: 25.53/23.04. Both 100-step test scores carry post-deadline daggers.
Go to primary source ↓e-leaderboardPDF p. 1, Abstract and Table 1, sampling and compression rows
Table 1 lists Revontuli sampling test/validation PSNR 23.00/25.53, Duke 21.56/25.30, Michael test 18.51; compression test top-500 CE/validation CE 6.64/4.92, Duke 7.50/5.60, a27sridh test 7.99. Revontuli ranks first in both; abstract gives compression 6.6386.
Go to primary source ↓e-compression-dataPDF p. 3, Section 3 Problem Statement and Eq. (1)
Cosmos 8×8×8 tokenizer supplies six 32×32 grids per approximately 306,000 samples, with 64×25 robot states; H=M=3. Each three-grid block corresponds to 17 RGB frames at 256×256. Test metric considers top-500 logits per token.
Go to primary source ↓e-compression-modelPDF p. 3, Section 3.1; p. 4, Figure 3a and caption
Spatial attention covers one grid; causal temporal attention covers the same coordinate across time. Model uses 24 layers, eight heads, dimension 512, T=5, dropout 0.1, pre-LayerNorm, QKNorm and absolute positions. States pass through an MLP and 1D convolution (kernel 3, padding 1) before additive conditioning.
Go to primary source ↓e-compression-trainingPDF p. 3, Section 3.2; p. 4, Training Objective and Implementation
Training is teacher-forced future-grid cross-entropy for 80 epochs with fused AdamW β₁=0.9, β₂=0.95, matrix-only decay 0.05 and tied embeddings. Peak learning rate 8×10⁻⁴ decays to zero after 2000 warmup steps; batch 160, bfloat16 AMP and same B200 cluster. Inference switches to float32. The objective writes K without explicitly defining it.
Go to primary source ↓e-compression-inferencePDF p. 4, Section 3.3, autoregressive factorization and greedy-decoding equation
Future-grid probabilities factor over time conditioned on earlier grids and the complete robot-state sequence. Each step predicts categorical spatial-token distributions; greedy decoding selects argmax outputs to build deterministic sequences.
Go to primary source ↓e-compression-diagnosticPDF p. 4, Figure 3b, caption and Section 3.4
Blue is teacher-forced training, orange teacher-forced validation and green greedy autoregressive validation. Orange is persistently below green. Authors interpret teacher-forced validation as optimistic and an idealized lower bound; scheduled sampling reportedly did not meaningfully improve results.
Go to primary source ↓e-conclusionPDF p. 4, Section 4 Conclusion
Authors report a sampling leaderboard lead in 36 hours and compression training in under 17 hours. They state that averaging optimizes PSNR but the appropriate inference strategy for downstream decision-making remains open.
Go to primary source ↓8.5 Primary sources
Generative World Modelling for Humanoids: 1X World Model Challenge Technical Report ↗
PDF · 4,367 extracted words
Source fingerprint
cd3751aed78b95a5a9e57d679cf3e6881f86f6deaa93aaad61cbc2420ce7f2e4