Programmable World Model
1. Paper overview
In one sentence: An executable box world maintains persistent facts, while a video renderer completes their appearance; strong global state scores leave entity-specific fidelity and component contributions unresolved. e-probleme-worlde-controlse-renderere-tablee-metricse-reporting
| At a glance | What to know |
|---|---|
| Research problem | Source description A plausible next frame need not preserve who exists, what happened off-screen, or which rules govern later interactions. The authors seek an editable, persistent state representation that supports entity-level programming while leaving detailed appearance and motion to a generative renderer. The central design problem is deciding how much geometry the explicit system must construct and evolve. e-probleme-tradeoffs |
| Core mechanism | Source description The framework separates executable state evolution, deterministic projection of state-augmented 3D boxes, and learned observation generation. Its interface exposes identity, category and object-motion controls while the engine stores nonvisual attributes and relations. e-architecturee-worlde-controls |
| A key reported result | Visible alive-character count consistency: 94.00 Count Accuracy (%). 50 CombatStateBench clips; eight randomly sampled frames per clip, 400 total. Qwen3.6-27B judges RGB without privileged annotations. Baselines receive the same initial observations and benchmark transitions through native prompt switching. LingBot-World-V2: 40.75; YUME: 32.00. Gains: 53.25 and 62.00 percentage points. Strong aggregate count agreement under this protocol. The comparison changes conditioning interfaces as well as systems and does not isolate an individual component. e-benchmarke-baselinese-metricse-table |
| Reading caution |
Core contributions
- Source description
The framework separates executable state evolution, deterministic projection of state-augmented 3D boxes, and learned observation generation. Its interface exposes identity, category and object-motion controls while the engine stores nonvisual attributes and relations. e-architecturee-worlde-controls
- Reader analysis
An automatic video annotation pipeline supplies renderer training controls, and CombatStateBench evaluates agreement between rendered observations and engine-maintained counts and death states. The claim of broader programmable worlds exceeds the narrow combat measurements. e-datae-benchmarke-metrics
Figure 2. Boxes mark the authors' chosen boundary between explicit geometry and generated detail. Original paper, p. 3 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the four representations from left to right, then read the three labels beneath the shared arrow. The figure positions boxes between semantic descriptions and richer geometry: they specify world-space position, extent and orientation without detailing every surface or joint. Section 2 explains why this matters differently during training and inference. Training can recover structure from motion that has already occurred. At inference, a program must construct that structure from a requested transition before the renderer can produce the motion. More detail therefore increases the amount of animation or dynamics that the explicit system must supply. e-tradeoffse-worlde-renderer
What it supports. The proposed advantage is an intermediate abstraction: programs maintain entity geometry and persistent attributes, while the video prior fills in articulation and secondary dynamics. This explains the choice of boxes as a practical interface, but the graphic supplies a design rationale rather than numerical evidence that boxes optimize the trade-off.
Where the evidence stops. The arrow has no measured scale. The paper does not benchmark these four representations under matched training data, compute or control tasks, so its ordering should be read as the authors' conceptual argument.
2. Motivation
2.1 The problem and the proposed response
A plausible next frame need not preserve who exists, what happened off-screen, or which rules govern later interactions. The authors seek an editable, persistent state representation that supports entity-level programming while leaving detailed appearance and motion to a generative renderer. The central design problem is deciding how much geometry the explicit system must construct and evolve. e-probleme-tradeoffs
2.2 What this reading follows
Imagine a character being defeated while the camera turns away. A programmable environment must retain the event even when no image shows it, then render the appropriate state when the camera returns. This paper places that responsibility in an explicit engine. A coding agent defines rules and entity attributes; a compiler projects state-augmented 3D boxes into visual controls; a pretrained video model supplies detailed appearance. The design offers a compact interface between executable facts and generated observations. Its combat benchmark provides strong evidence for global count and death-state agreement, while leaving the fidelity of particular identities and the causes of improvement less settled. e-probleme-worlde-controlse-renderere-tablee-metricse-reporting
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | Not assigned |
| Architecture | Not assigned |
| Prediction paradigm | Not assigned |
| Quadrant | Not assigned |
This table preserves the labels recorded at reading time. The current major category is Benchmarks & simulators. View the current classification.
3.1 Evidence-based assessment
Classification assessment not applicable
The supplied catalog is entirely unassigned, so there is no positive classification to confirm or contradict. Architecturally this is an explicit programmed engine plus a learned generative renderer. It neither jointly predicts actions and future observations in one learned model nor recovers actions through inverse dynamics. Treat it as outside an action-prediction quadrant unless that taxonomy explicitly includes programmable rendering systems. e-architecturee-worlde-renderer
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Treat a death as an engine update before treating it as an image
Begin with the state transition, not the renderer's next frame. The initial detector gives the orchestrator geometric entities and identifiers. The orchestrator then attaches attributes, relations and rules, such as factions, health and valid attacks. At an interaction step, the engine checks whether the action is allowed, applies the programmed effects and resolves resulting events. An entity's health can therefore affect later interactions even when it is outside the camera view. Only after recording the new canonical state does the compiler extract information relevant to the next observation. The paper's lightweight box world avoids specifying every joint or surface: it retains entity-scale geometry and interaction semantics while delegating fine visual realization to the video prior. This is executable state evolution, not a learned policy selecting physical actions. e-worlde-architecturee-tradeoffs
Figure 3. State execution, control compilation and memory-conditioned rendering have different responsibilities. Original paper, p. 6 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start with the left execution row: validate the action, apply rules and resolve events. The middle stage turns projected boxes into identity, direction and semantic signals. Their display order is schematic; Equation (10) specifies concatenation as identity, semantic, direction. On the right, the main transformer receives camera conditioning and memory tokens, while the copied branch processes structured controls; Equation (12) specifies additive feature injection. Follow the lower RGB-to-depth-to-point-cloud-to-reprojection arrows to understand spatial memory. Temporal history instead retains recent, mid-range and long-range latent context plus the initial image. Section 4.4.2 restricts both memory paths to observations available before the next chunk. e-architecturee-worlde-controlse-renderere-memorye-reporting
What it supports. The architecture makes engine state authoritative and uses the learned model to realize its visible consequences. Temporal and spatial memories support appearance continuity; they do not replace the executable rule store. A correct health update and a faithful depiction of its consequence remain separate stages of the system.
Where the evidence stops. The figure does not itself specify a full training schedule, loss or attention mask. The prose establishes bidirectional denoising within chunks and causal memory across them; it supplies no component ablation establishing each branch's contribution.
5.2 Tell the renderer where, which and how the entity moves
A projected rectangle moving across an image is ambiguous: the object may move, the camera may move, or both may move. The compiler uses world-space velocity to distinguish these cases, rotates it into camera coordinates and selects one of seven motion states. A static entity keeps the static label even as the camera shifts its projection. Identity and semantic maps answer different questions. An identity slot tracks a particular instance across frames; a category embedding can be shared by several entities or help depict an entity lacking an appearance reference. Training randomizes identity-slot assignments across samples while keeping them stable within a sequence. The concatenated maps guide a trainable ControlNet, and the inherited frozen camera pathway supplies the matching viewpoint trajectory. e-controlse-renderer
5.3 Separate persistent facts from persistent appearance, then evaluate both carefully
Reader interpretation: this architecture has two different forms of persistence. The engine stores world facts such as rules and health, while renderer memories preserve previously generated appearance. Recent latent history supplies detailed continuity; more distant history is compressed, and depth-based reprojection brings past observations into a new view. The initial frame remains an anchor. The source explicitly permits only completed past chunks in these memory inputs, although denoising within the current chunk is bidirectional. None of this guarantees that the rendered character corresponding to a persistent identifier is always correct. CombatStateBench's count metric checks a global total, and its death metric accepts any dead character in the sampled post-event frames. The high reported scores therefore motivate, rather than replace, an identity-aware test of persistent appearance and interaction outcomes. e-worlde-memorye-renderere-metricse-table
5.4 Training and inference
During training
The pretrained main branch and inherited camera pathway remain frozen while the new spatial-control branch is optimized. Identity slots are distinct within a sample, randomized across samples and fixed through each sequence. Cross-chunk conditioning introduces trainable parameters; source-video history is degraded with noise, corruption and partial dropping. e-controlse-renderere-memory
HUD-free Cyberpunk 2077, Forza Horizon 6 and GTA V gameplay supplies paired videos and controls. ViPE estimates cameras/depth; Qwen3-VL discovers categories; SAM3 segments and tracks instances; WildDet3D recovers OBBs. World-coordinate box displacements determine motion labels. Small visible masks are filtered, and a small displacement threshold identifies static objects. e-datae-benchmark
During inference
The first chunk uses I_0, zero temporal history and no spatial memory. Later chunks attend to recent, mid-range and long-range history plus the initial-image anchor. Completed RGB frames are lifted with estimated depth/cameras and reprojected into future views for spatial memory. Denoising is bidirectional within a chunk, but both memories use only completed past chunks. e-renderere-memory
5.5 Implementation flow
- Initialize a box world
An off-the-shelf 3D detector recovers visible entities, OBB geometry and persistent identifiers. The orchestrator combines this layout with the user's description to define attributes, relations, supported actions, event triggers and objectives. State comprises entities and poses, semantic/functional attributes, inter-entity relations and executable rules. e-world
- Execute rules
For each player action, the engine validates it, applies its effects and resolves triggered events. Health, inventory and faction membership can affect future transitions without being directly visible. The engine records the authoritative update before rendering; the method does not extract a robot action from a predicted video. e-worlde-architecture
- Compile visible controls
Project OBBs into the target camera and rasterize persistent identity embeddings, text-encoded categories and motion-direction embeddings. Depth-aware rasterization resolves overlaps. World-space object velocity is rotated into camera coordinates and quantized into forward, backward, left, right, static, up or down; camera-induced image displacement alone does not imply object motion. e-controls
- Condition the renderer
A Structured Spatial ControlNet processes the compiled controls and noisy video latent, then adds projected layer-wise features into the LingBot-World-v1 backbone. Its inherited camera pathway handles viewpoint changes. The generative prior completes shape, texture, articulation, illumination and secondary motion left unspecified by the boxes. e-renderer
6. Experiments & results
Programmable World Model makes a rule-executing engine authoritative for world facts and uses a conditioned video model to render them. State-augmented 3D oriented bounding boxes connect these components without requiring detailed animated assets. On CombatStateBench, the system reports 94% visible-alive-count accuracy and 98% death-state accuracy, but these permissive global metrics do not establish correct entity-specific interactions or autonomous action selection (e-world, e-controls, e-table, e-metrics).
The supplied paper contains one quantitative table and qualitative Figures 4–5, but no component ablation or quantitative long-horizon diagnostic curve. Figure 2 is conceptual and Figure 4 compares complete systems. This edition therefore uses those original diagnostics and states their scope; it does not present an invented ablation or treat five sampled frames as a complete video. e-tradeoffse-death-visuale-qualitativee-reporting
6.1 Read the original evidence
Table 1. Global state agreement improves substantially under CombatStateBench's permissive evaluation. Original paper, p. 12 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. First separate the rightmost state columns from the four video-quality columns. Count Accuracy checks exact agreement between the engine's visible-alive count and an RGB-only Qwen3.6-27B judge over 400 sampled frames. State Accuracy checks 50 death events, accepting an event when any of three post-transition samples depicts a dead character. The first four columns describe imaging quality, foreground consistency, background consistency and temporal flicker. All values are percentages. Baselines share initial observations and benchmark transitions but use prompt switching through their native interfaces, whereas Ours receives engine-derived structured controls. e-tablee-benchmarke-baselinese-metricse-qualitye-reporting
What it supports. Ours reaches 94.00% Count Accuracy versus 40.75% for LingBot-World-V2 and 32.00% for YUME, and 98.00% State Accuracy versus 8.00% and 58.00%. It also leads the four quality columns, although the imaging-quality difference from LingBot-World-V2 is only 0.16 percentage points. The largest gains concern coarse state agreement.
Where the evidence stops. The death score does not require the intended victim or location. These are system comparisons with different control interfaces, without reported uncertainty or judge calibration; they cannot isolate the effect of any one module or establish complete entity-level fidelity.
Figure 4. Selected death-event sequences expose errors that visual plausibility alone can conceal. Original paper, p. 12 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Follow each panel horizontally through its five selected time steps. The input row displays the prescribed box-world evolution; the remaining rows show LingBot-v2, YUME and Ours. In panel (a), the target box changes from upright to horizontal while other boxes remain upright, and the corresponding Ours row shows a prone figure at later times. Panel (b) adds camera and entity motion, making apparent movement and disappearance harder to interpret. Compare surviving entities and the event outcome together. The input visualization is the proposed renderer's structured guidance, not a control channel shared identically by both baselines. e-death-visuale-baselinese-worlde-metricse-reporting
What it supports. The samples illustrate why a convincing shooting response can still violate the intended world state. In these displayed cases, the proposed system better reflects the specified death while retaining other entities. The authors use the figure to diagnose baseline disappearance and unrealized death, complementing the aggregate count and state table.
Where the evidence stops. This is a qualitative system comparison, not a component ablation. Five selected frames cannot establish continuous temporal correctness, event timing or failure frequency, and the illustrated baseline interfaces differ from the proposed structured controls.
Figure 5. Appearance changes, newly visible entities and longer rollouts are demonstrated through selected frames. Original paper, p. 13 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read each pair of rows as control followed by generated observation. Panels (a) and (b) show the global box world, whereas (c) through (e) show camera-view box maps with a ground grid. This coordinate distinction matters: entities behind the initial camera can exist in global state before appearing in RGB. Panel (b) illustrates their later reveal; panel (e) increases the visible population over a reported 897-frame autoregressive sequence. Section 5.3 also distinguishes initial-image provenance: (b) and (e) use GTA V first frames unseen in training, while (a), (c) and (d) start from GPT Image 2 images. e-qualitativee-memorye-reporting
What it supports. The examples broaden the illustrated behavior beyond one combat appearance: the renderer depicts novel-looking characters, vehicle scenes and mixed categories. The camera-rotation example reveals three characters placed behind the initial view. The long sequence's sampled frames suggest continued scene organization as NPCs arrive, motivating a systematic persistence test.
Where the evidence stops. The 897-frame length is stated in the caption; only five selected time steps are visible here. No matched generalization baseline or quantitative long-horizon curve is supplied, and these stills do not establish uninterrupted stability, measured latency or autonomous gameplay.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| Visible alive-character count consistency 50 CombatStateBench clips; eight randomly sampled frames per clip, 400 total. Qwen3.6-27B judges RGB without privileged annotations. Baselines receive the same initial observations and benchmark transitions through native prompt switching. | 94.00 Count Accuracy (%) | LingBot-World-V2: 40.75; YUME: 32.00. Gains: 53.25 and 62.00 percentage points. Strong aggregate count agreement under this protocol. The comparison changes conditioning interfaces as well as systems and does not isolate an individual component. e-benchmarke-baselinese-metricse-table |
| Visual realization of death events The same 50 clips; 50 death events, with three post-transition frames per event and the RGB-only Qwen3.6-27B judge. | 98.00 State Accuracy (%) | LingBot-World-V2: 8.00; YUME: 58.00. Gains: 90.00 and 40.00 percentage points. Measures detection of some dead character after an event, not correct victim identity, precise timing, location or continued persistence throughout the rollout. e-tablee-metrics |
| Perceptual and temporal video quality VBench metrics on the 50 CombatStateBench clips; higher is better, with all table values expressed as percentages. | 67.62 / 94.74 / 96.98 / 99.00 Imaging / Subject Consistency / Background Consistency / Temporal Stability | LingBot-World-V2: 67.46 / 81.87 / 91.89 / 96.85; YUME: 64.10 / 92.35 / 93.63 / 98.76. Ours leads every reported column. Temporal Stability uses the temporal-flickering score. These measures concern generated appearance; no uncertainty or significance tests are reported. e-tablee-quality |
| Qualitative long-horizon and appearance generalization Figure 5 shows five selected time steps per scenario: novel appearance, large-angle rotation, racing, humans with vehicles, and increasing NPC count. | Panel (e) samples an 897-frame autoregressive sequence; panel (b) reveals three initially behind-camera characters. Qualitative examples; no aggregate generalization metric | No matched quantitative generalization or long-horizon baseline is supplied for Figure 5. Panels (b,e) use unseen GTA V first frames; (a,c,d) use GPT Image 2 initial images. These selected examples do not establish uninterrupted stability or autonomous gameplay. e-qualitative |
6.3 Ablations and diagnostic examples
No ablation or diagnostic visual is included in this edition. The source-specific visual limitations are recorded above.
7. Analysis & limitations
7.1 What the evidence leaves open
Count and death scores are deliberately permissive. They can miss identity swaps or a wrong victim, and the source does not report human calibration of the VLM judge. A correct engine state therefore should not be equated with completely faithful rendering. e-metricse-table
No component ablation isolates identity slots, direction conditioning, OBB representation or either memory. Figure 2 is a conceptual trade-off, not a measured representation comparison; Figures 4–5 provide selected diagnostics. Training–inference mismatch is motivated but not quantified. e-tradeoffse-reportinge-qualitative
The reported tasks concern generated game observations. The paper supplies no robot execution, learned action policy, physical-dynamics validation or measured interactive latency. Program correctness and faithful image synthesis are separate dependencies. e-worlde-renderere-reporting
7.2 Questions for discussion
- How often can the global metrics pass while the wrong entity changes state (e-metrics)?
- Which memory or control component explains performance under camera revisitation (e-controls, e-memory, e-reporting)?
8. Reproducibility audit
8.1 Requirements and known gaps
Required assets include the gameplay corpus, its camera/depth/track annotations, engine programs, control embeddings, LingBot-World-v1 and memory implementation. The source does not specify corpus size, loss objective, optimizer, learning rate, training schedule, hardware, inference resolution/rate, chunk length values or memory capacities. e-renderere-memorye-datae-reporting
Reconstruction also needs the unspecified visible-mask and static-displacement thresholds, benchmark verifier tolerances, sampling seeds and judge prompts. Benchmark construction retains only sequences passing geometry, motion and transition-persistence checks; report rejection rates when rebuilding it. Test/train separation is explicit for two qualitative first-frame examples, not fully documented for CombatStateBench. e-datae-benchmarke-metricse-qualitative
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Isolate the world-velocity direction signal
Proposed check, not run: construct paired camera/object trajectories with similar projected-box displacement but different true object motion. Hold initial images, box paths, identities, categories, camera controls, sampling seeds and memory settings fixed within each comparison. Compare the complete renderer against a matched retraining that removes only the direction map, and an explicitly labeled inference-time direction-swap stress test. Score static-versus-moving behavior and motion direction with blinded human annotations, alongside the paper's metrics. A direction-specific benefit should concentrate in camera/object ambiguity cases; no such interaction would weaken the proposed mechanism. Report retraining separately because inference corruption is a distribution shift. e-controlse-renderere-datae-reporting
Check 2: Test whether global death success hides the wrong victim
Proposed check, not run: make two same-category characters visually distinguishable, designate one victim in the engine, and include a surviving distractor through a camera turn and return. Retain the original RGB-only count/death protocol, but separately annotate victim identity and persistence over the complete returned-view segment using blinded human raters. Compare full rendering with a matched retraining without identity conditioning, keeping geometry, categories, motion and memory fixed. The falsifiable concern is high global State Accuracy despite wrong-victim or identity-swap failures. Report both score families, rater agreement and clip-level uncertainty; a single aggregate death score cannot resolve this discrepancy. e-worlde-controlse-metricse-qualitativee-reporting
8.3 Reading coverage
Visual audit: All 17 supplied PDF pages were rendered and visually inspected, including title metadata, every method/training/evaluation page, Figures 1–5, Table 1 and references. The five final original crops were individually viewed. Figure 3's execution and memory arrows were checked against Eqs. (3), (10), (12) and (14) and their prose; map display order is distinguished from formal concatenation order. The table crop retains its percentage and sample-count note. Full-page inspection covers the caption and method evidence needed for crop explanations. No appendix is present; separate supplements, linked code and continuous videos were not inspected.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17. Appendix coverage: not present.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Title, authors and abstract (p. 1)
- 1 Introduction
- 2 Representation Trade-offs
- 3 Related Work, including 3.1–3.3
- 4 Method, including 4.1–4.5 and 4.4.1–4.4.2
- 5 Experiments, including 5.1–5.3
- 6 Conclusion
- References (pp. 15–17)
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Separate supplemental material availability has not been fully verified.
- All six supplied text chunks and all 17 PDF pages were read; Figures 1–5 and Table 1 were visually inspected, resolving the extraction-only visual omission. No appendix is present in this PDF.
- Identity: the title and all eleven authors match the catalog. Page 1 identifies arXiv:2609.10540v1 [cs.CV], 9 September 2026; its internal date is September 10, 2026. These distinct dates are preserved; no other revision was supplied or compared.
- Alaya Lab is the only named affiliation. Jiacheng Lin, Yu-Lun Liu and Yung-Yu Chuang have no affiliation marker; their affiliations are not inferred.
- Linked code, project material and videos were not accessed. No experiments were reproduced. Selected PDF frames do not constitute viewing complete videos.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e-identityPDF p. 1, title/byline, affiliation marker, date block and arXiv margin
Title and eleven-author sequence match the catalog. The margin identifies 2609.10540v1, 9 September 2026; the internal date is September 10, 2026. Alaya Lab is attached to triangle-marked authors only.
Go to primary source ↓e-problemPDF p. 2, Section 1, motivation and representation question
The paper motivates direct entity control, persistent global/nonvisual state and executable world rules, connected to visual generation through state-augmented 3D OBBs.
Go to primary source ↓e-tradeoffsPDF pp. 3–5, Figure 2 and Section 2, especially Eqs. (1)–(2) on p. 4
The conceptual spectrum connects greater structural detail with control and construction/annotation burden. Training extracts structure from realized dynamics; inference constructs structure from state transitions before generation.
Go to primary source ↓e-architecturePDF p. 1, Figure 1; p. 6, Figure 3 and Section 4.1, Eq. (3)
World programming, deterministic control compilation and generative rendering form distinct stages. Figure 3 shows execution steps, control maps, the camera adapter and temporal/spatial memory paths.
Go to primary source ↓e-worldPDF pp. 6–7, Section 4.2, Eqs. (4)–(5); p. 7, Section 4.3, Eq. (6)
A detector initializes visible OBBs/identifiers. The orchestrator programs attributes, relations and rules; the engine validates actions, applies effects and resolves events, including nonvisual attributes, before state compilation.
Go to primary source ↓e-controlsPDF pp. 7–9, Section 4.3, especially p. 8 Eqs. (7)–(10)
Control maps encode persistent identity slots, text-encoded semantic labels and seven camera-relative object-motion states. Slot assignments are randomized across samples but fixed in-sequence. Depth resolves overlaps; world velocity rather than image displacement defines motion.
Go to primary source ↓e-rendererPDF p. 9, Section 4.4.1, Eqs. (11)–(13), and opening of 4.4.2
Structured Spatial ControlNet adds features to frozen LingBot-World-v1 main blocks and retains its frozen camera pathway. The renderer completes unspecified visual details; denoising is bidirectional within chunks.
Go to primary source ↓e-memoryPDF p. 10, Section 4.4.2, Eq. (14), Temporal history and Geometry-aligned spatial memory
Later chunks use trainable cross-chunk conditioning with multiscale temporal history, initial-image anchoring and AlayaWorld-style depth/camera reprojection memory. Training degrades source history; inference updates memory from completed chunks only.
Go to primary source ↓e-dataPDF pp. 10–11, Section 4.5, Eq. (15), camera estimation through conditioning-map generation
ViPE, Qwen3-VL, SAM3 and WildDet3D generate camera/depth, categories, tracked masks and OBBs. World-coordinate displacements produce direction labels. Mask-area filtering and a static threshold are described without numerical settings.
Go to primary source ↓e-benchmarkPDF p. 11, Section 5.1, Training Data and Benchmark
Training uses HUD-free Cyberpunk 2077, Forza Horizon 6 and GTA V footage. CombatStateBench contains 50 verified clips with synchronized controls/states; an agent evolves box worlds, death states persist, and automatic geometry/motion checks filter sequences.
Go to primary source ↓e-baselinesPDF pp. 11–12, Section 5.2 Baselines; p. 14, Eq. (16) and preceding paragraph
LingBot-World-V2 and YUME share initial observations and benchmark transitions but retain native conditioning. Because they lack external instance-state interfaces, death events are conveyed by environment/action prompt switching.
Go to primary source ↓e-metricsPDF p. 14, Section 5.2 VLM-based Evaluation, Count Accuracy and State Accuracy, Eqs. (17)–(18); p. 12, Table 1 caption
RGB-only Qwen3.6-27B evaluates 400 sampled frames and 50 death events. Counts require exact equality; each death event needs a dead character in any of three sampled post-transition frames, without victim identity or location checks.
Go to primary source ↓e-tablePDF p. 12, Table 1, all three method rows and six metric columns
Ours: 67.62/94.74/96.98/99.00/94.00/98.00. LingBot-World-V2: 67.46/81.87/91.89/96.85/40.75/8.00. YUME: 64.10/92.35/93.63/98.76/32.00/58.00. Column order: imaging, subject, background, temporal stability, count, state; all percentages.
Go to primary source ↓e-qualityPDF pp. 14–15, Section 5.2 Video Quality
The four VBench measures concern imaging, foreground appearance consistency, background consistency and temporal flickering. Higher temporal-stability values mean less flicker.
Go to primary source ↓e-death-visualPDF p. 12, Figure 4 panels (a)–(b) and caption; p. 15, Section 5.3 first paragraph
Five selected frames compare input state visualizations and three generated methods for static and dynamic camera/entity settings. The authors diagnose disappearing live characters and unrealized death states in baseline outputs.
Go to primary source ↓e-qualitativePDF p. 13, Figure 5 panels (a)–(e) and caption; p. 15, Section 5.3
Examples cover a minotaur scene, large-angle rotation revealing three behind-camera characters, racing, humans with vehicles and selected frames of an 897-frame NPC rollout. First frames in (b,e) are unseen GTA V samples; (a,c,d) use GPT Image 2. Input coordinates differ between (a,b) and (c–e).
Go to primary source ↓e-reportingPDF pp. 9–11, Sections 4.4–5.1; pp. 11–15, Sections 5.2–6; pp. 15–17, References
The supplied method/setup gives no complete training recipe or numerical compute/latency configuration. Experiments comprise Table 1 and qualitative Figures 4–5, without component ablations, uncertainty estimates or quantitative long-horizon curves. The PDF ends with references and contains no appendix.
Go to primary source ↓8.5 Primary sources
Programmable World Model ↗
PDF · 8,739 extracted words
Source fingerprint
d9d8313f851ed82dc1bc248b739cce4c07cbbb08cbd1b2c8743ca33c0bcb9c77