Building Pretraining Data for World Models: An Unreal Engine-Based Pipeline for Action-Conditioned Video Generation
1. Paper overview
In one sentence: Recording physics-resolved trajectories before offline rendering enables aligned multi-view data production, while the learning value of its perceptual curation remains unmeasured. e-scopee-workflowe-scalee-viewere-failurese-limits
| At a glance | What to know |
|---|---|
| Research problem | Source description Interactive video models need controls aligned with visual transitions. Ordinary video rarely supplies the actions that caused those transitions. Unreal Engine exposes controls and geometry, but real-time physics and offline Movie Render Queue (MRQ) evaluation have different execution requirements. Heterogeneous assets also introduce collision, loading and rendering failures. e-scopee-workflowe-assets |
| Core mechanism | Source description A trajectory representation connects physics execution to replay, producing synchronized views with frame-level actions, character transforms and camera states. e-samplee-workflowe-render |
| A key reported result | Production of action-conditioned multi-view video: 2,691 h at 1080p; 6,076 h at 720p. Reported produced video duration by resolution. Production inventory: 429 retained levels from 2,384 purchased packs, 40 humanoid characters and five views; no benchmark split. No competing pipeline baseline. Figures 2–3 and the conclusion use approximately 2,700 h and 6,000 h. Production totals, not prediction accuracy. View aggregation and unique physical-trajectory hours are not explicitly defined; avoid inferring unique trajectory counts or a pack-to-level retention percentage. e-scalee-workflowe-diversitye-limits |
| Reading caution | Source description Perceptual filtering may remove structurally useful scenes. The authors leave controlled downstream calibration outside scope; visual quality alone does not establish dynamics-learning utility. e-limits |
Core contributions
- Source description
A trajectory representation connects physics execution to replay, producing synchronized views with frame-level actions, character transforms and camera states. e-samplee-workflowe-render
- Source description
Persistent scene pools, local process scheduling and asynchronous upload support production across 25 nodes with eight RTX 5090 GPUs each. e-schedulinge-scalee-recovery
- Source description
Scene screening and case-specific failure diagnostics document data-quality and reliability problems, without establishing an optimal training-data distribution. e-filtere-failurese-limits
Figure 2. The recorded trajectory separates determining motion from rendering its observations. Original paper, p. 5 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Follow the solid arrows from asset preparation through validation and scheduling to the green Stage I column. PIE resolves collision, gravity and slopes; the trajectory log carries actions, poses and camera states into the purple Stage II column. Replay reconstructs character and camera states before MRQ produces image sequences. The bottom band supports the production stages with recovery and monitoring. The scoring and eligibility inequalities match Equation (2) and Section 4.2. The synthetic-action box abbreviates the key list: Equation (1) also includes diagonal combinations. The diagram therefore summarizes the implementation; it is not a learned network or a training graph. e-workflowe-samplee-filtere-rendere-scalee-recovery
What it supports. The verified mechanism is a fixed-motion handoff. Stage I supplies realized motion, while Stage II spends rendering computation on that motion without selecting new actions or rerunning character physics. This arrangement addresses the authors’ difficulty combining runtime simulation with MRQ’s repeated timeline evaluation.
Where the evidence stops. The diagram rounds output totals to 2,700 h and 6,000 h; Table 1 gives 2,691 h and 6,076 h. Its recovery shorthand does not establish frame-level checkpoint resumption: Section 8.2 describes salvaging completed trajectories.
2. Motivation
2.1 The problem and the proposed response
Interactive video models need controls aligned with visual transitions. Ordinary video rarely supplies the actions that caused those transitions. Unreal Engine exposes controls and geometry, but real-time physics and offline Movie Render Queue (MRQ) evaluation have different execution requirements. Heterogeneous assets also introduce collision, loading and rendering failures. e-scopee-workflowe-assets
2.2 What this reading follows
The useful starting point is the trajectory file: it connects an executed sequence of controls to the images later rendered from that sequence. This paper explains how Unreal Engine scenes become action-conditioned pretraining data through physics execution, camera construction, offline rendering and production recovery. The visual tour follows that information flow, then separates reported output volume from evidence of learning utility. Its tables describe dataset scale and diagnosed failures; they do not evaluate a trained world model. Read the camera examples as illustrations of the representation, and the inspection interface as a debugging aid whose presence does not certify every generated trajectory. e-scopee-workflowe-scalee-viewere-failurese-limits
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | Not assigned |
| Architecture | Not assigned |
| Prediction paradigm | Not assigned |
| Quadrant | Not assigned |
This table preserves the labels recorded at reading time. The current major category is Datasets. View the current classification.
3.1 Evidence-based assessment
Classification assessment not applicable
The catalog snapshot is unassigned. This work supplies upstream data infrastructure without a proposed learned world-action architecture. PIE and MRQ are execution stages, not evidence for One Model/Multiple Models or joint prediction/inverse-dynamics quadrants. Its use in EchoWM does not transfer that system’s classification to this paper. e-scopee-workflow
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 1. Make recorded state the contract between physics and images
The separation starts with an execution mismatch. PIE must advance the simulated world so that collision, gravity and contact determine character motion; MRQ may evaluate a timeline multiple times to produce an output frame. The source therefore records realized motion before rendering it. Stage I’s per-frame callback applies controls and reads transforms without blocking engine advancement. Stage II reconstructs a timeline from those records and does not reissue the controls. The implementation details reveal what this contract must preserve: scale must be written explicitly, and vertical mesh placement must use the capsule height actually realized by the engine. Otherwise a replay can retain the general path while showing an incorrectly sized, floating or sunken character. These corrections preserve the intended representation; they are not evidence of a learned dynamics model. e-workflowe-stage1e-render
Figure 1. Different camera conventions expose different observations of locomotion. Original paper, p. 4 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Use the original caption to read the montage: the top row contains top-down views, the bottom row back views, the center four images first-person views, and the side images the corresponding left/right views. These examples deliberately come from different environments. They illustrate camera configurations and scene variety, rather than five simultaneous observations of one event. Section 6.3 explains how a fixed character trajectory supplies each view: external desired positions use character-relative offsets, followed by ray-based occlusion correction. The first-person view sits near the character’s head. Camera distance contracts quickly around obstacles and returns outward more slowly. e-samplee-cameras
What it supports. The pipeline offers five synchronized observation streams per trajectory, with viewpoint changes computed after physical character motion is fixed. The montage makes their visual differences tangible: external views expose the character, whereas first-person examples emphasize the environment ahead. This supports an explanation of the data representation.
Where the evidence stops. A collection of stills cannot verify temporal synchronization, camera smoothness or physical validity across complete sequences. The cross-scene montage also cannot establish cross-view consistency for a single trajectory.
5.2 2. Separate a key label from the trajectory it produces
The main system uses character-relative controls. Sampling W or a diagonal combination is only one part of trajectory generation: an action is held for a random duration, facing can turn toward a point of interest, and repeated obstruction can trigger escape rotation or respawning. Section 6.2 says that guidance leaves action-sampling probabilities unchanged. Reader interpretation: that statement concerns the marginal sampler, not invariance of world-space paths or temporal transition statistics. Camera trajectories add another transformation because their desired external positions follow character orientation and then undergo occlusion correction. The reported 46.6% action share therefore cannot certify either spatial coverage or predictable screen-space motion. Section 9’s newer camera-coordinate implementation addresses a different action convention and should be evaluated separately from the five-view character-centric system. e-samplee-actionse-camerase-scalee-limits
5.3 3. Follow a sample through three different notions of success
First, a scene must be renderable: short clips are manually inspected for crashes, invalid walkable regions, hovering and interpenetration. Second, individual views must meet perceptual thresholds, and enough sampled outputs must pass for the level to enter production. Third, completed output must survive operational failures: recovery requires both a completion marker and the expected video files, while upload waits until directories have been quiet. These gates answer different questions. Reader analysis: a recovered, visually acceptable video can still be unhelpful for learning dynamics; none of these gates is a downstream prediction test. The authors explicitly acknowledge that their perceptual thresholds are not calibrated against world-model performance. Table 1 measures production, Figure 5 supports spot checks, and Table 2 records diagnoses. A controlled learning comparison remains the missing link. e-filtere-recoverye-scalee-viewere-failurese-limits
5.4 Training and inference
During training
No predictive architecture, training objective, optimization schedule or downstream train/test split is introduced. A pretrained image-quality model supplies aesthetic scores for curation; this does not establish world-model training or joint action/video learning in this report. e-scopee-filtere-limits
During inference
There is no learned inference-time controller. Sampled controls execute in engine physics; their recorded consequences are rendered afterward. This is synthetic data generation, not demonstrated robot execution or learned planning. e-stage1e-actionse-render
5.5 Implementation flow
- Screen and calibrate assets
Inventory actual asset directories; exclude incomplete sublevels, showcases and problematic large partitioned worlds. Configure character animations, scale, collision and locomotion. Manually inspect five 10-second 540p renderability clips per candidate level, then evaluate twenty 10-second 540p clips for visual quality. Score individual views, excluding the tiled preview. e-assetse-filter
- Preserve scene-cache locality
Persistently bind each node to a scene pool and reshuffle between passes. The allocator assigns pools; node schedulers manage slots, retries and cleanup. Copy scenes into active slot projects and share characters locally. Multiple processes may share a GPU, with concurrency tuned empirically. e-scheduling
- Execute and record motion
Stage I uses a per-frame callback state machine in Play In Editor (PIE), returning control so physics advances. Spawn candidates fall until stable contact. Hold sampled actions for random durations and orient toward points of interest. Persistent movement input with negligible displacement triggers escape rotation, then respawning if needed. e-stage1e-actions
- Compute cameras after motion
Construct back, left, right, elevated top-down and first-person views from the fixed character trajectory. External cameras ray-cast toward desired offsets and move inward before obstacles. Distance contracts quickly and expands slowly to reduce popping. These corrections do not change character motion. e-cameras
- Replay for offline rendering
A fresh process reconstructs translation, rotation and explicit scale tracks; observed engine capsule height determines mesh vertical offset. Remove auto-playing cinematic actors and adjust texture residency before MRQ. Encode rendered image sequences into per-view videos, then delete raw images. Actions and character physics are not rerun. e-rendere-viewer
- Recover completed outputs
Append-only markers communicate completion. Interrupted instances retain trajectories only when completion markers and expected videos both exist. Upload waits for a quiet period; local output is deleted only after successful transfer. Failure snapshots and external monitoring support diagnosis. The visual inspector is separate from automatic acceptance. e-recoverye-viewer
6. Experiments & results
This paper documents the Unreal Engine synthetic-data component used in EchoWM: simulate character motion once, record controls and states, then replay the trajectory for high-quality multi-view rendering. Its contribution is a production system combining scene curation, cache locality and failure recovery. Reported video volume establishes production scale; downstream learning utility remains untested here.
This production-focused source supplies a quantitative inventory/throughput table, qualitative output examples and diagnostic material, but no downstream model benchmark or controlled component-ablation figure/table. The reported cache timing is a prose production observation. Figure 5 and Table 2 serve as diagnostics in the ablation section; neither measures an ablation effect or failure frequency. All six selected visuals are original inspected excerpts. e-scopee-scalee-schedulinge-viewere-failurese-limits
6.1 Read the original evidence
Table 1. Production statistics quantify the reported corpus and cluster. Original paper, p. 6 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the table in groups. The first rows describe the asset and representation inventory: 2,384 purchased packs, 429 production levels, 40 humanoid characters, five synchronized views and nine action states. The next rows specify a standard trajectory of 1,800 frames over 60 seconds and a cluster of 25 nodes with eight RTX 5090 GPUs per node. The final rows report video output, five-view throughput and a combined action share. Keep asset packs distinct from levels, and production duration distinct from learning accuracy. The table gives exact output totals, unlike the rounded labels elsewhere in the paper. e-scalee-diversitye-limits
What it supports. The authors report 2,691 hours at 1080p and 6,076 hours at 720p, plus five-view throughput of 33 min/node-hour. Diagonal, backward and lateral movement together account for 46.6% of actions. These establish reported production volume and a coarse description of action coverage.
Where the evidence stops. No uncertainty, throughput resolution breakdown, baseline or downstream benchmark is supplied. The minute/hour accounting does not explicitly define camera-view aggregation, so it cannot safely determine unique physical-trajectory hours or total compute cost.
Figure 3. The retained scenes span several semantic categories, with uneven shares. Original paper, p. 6 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Begin with the inner ring, which assigns scenes to major categories; the outer ring names finer groups such as medieval settlements, forests, science-fiction facilities and commercial interiors. Historical & Fantasy is the largest category at 29.4%. Nature & Landscape and Sci-Fi & Futuristic each show 21.7%, Urban & Architecture 19.1%, and Other 8.2%. These percentages describe the scene pool, not hours of video or action frequencies. The center identifies 429 scenes and summarizes output with rounded hour totals. Section 4.3 uses the distribution to argue that no single scene type dominates the retained pool. e-diversitye-scalee-schedulinge-limits
What it supports. The chart supports semantic variety among retained production scenes: four major categories each contribute roughly one-fifth to three-tenths of the pool. It does not show equal representation. As reader analysis, repeated rendering means a scene-count distribution need not equal the distribution of frames used for training.
Where the evidence stops. Rounded percentages need not sum exactly to 100%. The plot contains no held-out split, per-category learning result or coverage of rejected assets; it cannot demonstrate generalization or absence of filtering bias.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| Production of action-conditioned multi-view video Production inventory: 429 retained levels from 2,384 purchased packs, 40 humanoid characters and five views; no benchmark split. | 2,691 h at 1080p; 6,076 h at 720p. Reported produced video duration by resolution | No competing pipeline baseline. Figures 2–3 and the conclusion use approximately 2,700 h and 6,000 h. Production totals, not prediction accuracy. View aggregation and unique physical-trajectory hours are not explicitly defined; avoid inferring unique trajectory counts or a pack-to-level retention percentage. e-scalee-workflowe-diversitye-limits |
| Five-view production throughput Production cluster: 25 nodes, each with eight RTX 5090 GPUs. | 33 min / node-hour. Five-view production throughput | No baseline, resolution-specific timing breakdown, sampling window or uncertainty is supplied. Preserve the table unit. Whether minutes aggregate camera streams or denote synchronized sequence duration is not clarified; per-GPU and total-cluster cost conversions would require assumptions. e-scale |
| Repeated-scene rendering with persistent pools Repeated visits reuse scene-specific caches; cold starts can exceed ten minutes. | Author-reported production observation: less than half of cold-start time. Per-scene rendering time relative to cold start | Repeated warm scenes versus cold starts; no controlled ablation table or timing distribution. Supports locality operationally, while leaving workload comparability and separate scheduling/cache effects unresolved. e-scheduling |
| Locomotion action coverage Current generated action distribution across nine states. | 46.6%. Combined diagonal, backward and lateral action share | No per-state histogram or alternative-policy evaluation. Substantial non-forward locomotion is present; this does not establish balanced actions, spatial coverage or generalization. e-scalee-diversity |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Figure 5. The inspector connects an observed frame to its controls and spatial record. Original paper, p. 11 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read across the panels rather than treating the screenshot as a score. The left side displays a tiled multi-view preview and a playback scrubber. The upper right identifies the current action as W + A at frame 891, while the lower right plots character and camera trajectories. Section 8.1 describes synchronized frame-by-frame scrubbing to inspect action alignment, unexpected camera changes, abnormal character motion and spatial confinement. This is a qualitative diagnostic interface. The paper explicitly keeps it outside automatic acceptance and partial-output recovery, which instead depend on scoring or completion markers and expected files. e-viewere-filtere-recovery
What it supports. The interface makes several otherwise separate records jointly inspectable: images, actions and geometry. That is useful for locating alignment or camera errors that may be invisible in execution logs. The source presents a debugging capability, not a measured reduction in alignment errors or a model ablation.
Where the evidence stops. Only the original static screenshot was inspected. Its visible panels do not establish successful playback, sequence-wide correctness or dataset-wide error rates. The tiled preview is also excluded from automatic aesthetic/luminance scoring.
Table 2. Operational diagnoses explain why correct-looking output needs more than a completed process. Original paper, p. 16 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read each row horizontally: observed failure, diagnostic signal, identified cause, then mitigation. Engine cases include stale skeletal state after character changes, partitioned-world initialization, cinematic camera takeover and animation root-motion conflicts. Container cases distinguish GPU access failure inside a container from memory-limit exits; the hardware row describes node-level driver/GPU failure. Storage rows concern interrupted cleanup and accumulated logs. Appendix D explicitly says these symptom-to-cause mappings are case-specific. The intermediate diagnostic column matters: it supplies the evidence used to distinguish causes, rather than suggesting that every superficially similar failure has the same remedy. e-failurese-rendere-recovery
What it supports. The table documents concrete failure mechanisms that can affect production or corrupt its visual interpretation. Camera ownership and animation-root conflicts are especially relevant because a process may produce images while representing unintended behavior. These cases explain implementation safeguards, without quantifying their reliability benefit.
Where the evidence stops. This is a diagnostic case collection, not a controlled ablation. No incidence rates, recovery-latency measurements or comparative success percentages are reported. The listed mitigations describe the authors’ environment and were not executed during this reading.
7. Analysis & limitations
7.1 What the evidence leaves open
Perceptual filtering may remove structurally useful scenes. The authors leave controlled downstream calibration outside scope; visual quality alone does not establish dynamics-learning utility. e-limits
The repertoire omits jumping, running, crawling, attacking and complex interactions. A newer single-camera, camera-coordinate implementation is discussed as an extension; it should not replace the main character-centric five-view system in interpreting this report. e-limits
Engine physics and screened synthetic assets do not establish real-world transfer. Action holding, guidance and recovery shape trajectories; the marginal action share cannot isolate their consequences. e-actionse-limits
Table 2 supplies case-specific diagnoses without failure frequencies or controlled before/after measurements. Similar symptoms can have different causes. e-failures
7.2 Questions for discussion
- How does perceptual filtering affect dynamics prediction when scene and action coverage are controlled? [e-filter, e-limits]
- Which replay invariants guarantee that rendered transitions retain recorded control semantics? [e-render, e-actions]
8. Reproducibility audit
8.1 Requirements and known gaps
Reproduction needs usable Unreal scenes, calibrated character animations/collision, PIE automation, MRQ replay, local storage and scheduling. Appendices A–B list configuration groups and storage conventions. The reported cluster is production infrastructure, not a demonstrated minimum. e-assetse-stage1e-rendere-confige-scale
The source does not specify an Unreal/software version, exact asset inventory or redistribution terms, complete action/holding distributions, recovery thresholds, camera smoothing constants, MRQ sample counts or final concurrency settings. Appendix A lists parameters rather than a complete configuration. A public dataset split and access/license protocol are not established. e-assetse-actionse-camerase-rendere-confige-scope
Reader-proposed checks: compare replay transforms with recorded states under rendering-setting changes, then compare matched-size filtered and renderable-only training corpora on held-out scenes. These experiments are not reported results. e-rendere-filtere-limits
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Proposed check: does MRQ replay preserve the recorded trajectory?
On one renderable level, record a fixed action sequence with a calibrated character, retaining transforms, actions and actual capsule height. Replay the same intermediate under two MRQ sampling/warm-up configurations with identical output timestamps. Compare character/camera translation, orientation and scale against Stage I, and inspect ground contact and projected landmarks. Pre-register numerical tolerances appropriate to the engine representation. Include deliberately omitted scale and requested-versus-observed capsule height as separate fault controls. The preservation claim predicts invariant frame-indexed states across rendering settings; systematic drift, scaling changes or vertical offsets would falsify that invariant. This is a reader-proposed test, not an experiment in the paper. e-stage1e-rendere-assets
Check 2: Proposed check: do perceptual thresholds improve learning at fixed data volume?
Create equal-sized corpora from the same renderable candidate pool: one using the paper’s aesthetic/luminance and level-pass thresholds, the other sampled before that automatic filter. Match scene-category and action shares where both pools permit, and disclose unmatched strata. Keep the chosen predictive architecture, optimization budget, horizon and random-seed schedule fixed. Evaluate held-out levels, keeping all synchronized views of a trajectory in the same split, with predeclared future-frame prediction and action-response metrics. If the filter improves aesthetic scores without improving those metrics, its assumed learning benefit is unsupported under this controlled setting. Neither training setup nor result is supplied by this paper. e-filtere-samplee-diversitye-limits
8.3 Reading coverage
Visual audit: All 16 original PDF pages were rendered and actually viewed, covering the title/author block, complete body, references and Appendices A–D. Figures 1–5, Tables 1–2 and both equations were visually checked. All six final original crops were viewed. Figure 2’s action shorthand, thresholds and stage arrows were checked against Equations (1)–(2) and Sections 3.2, 4.2, 6 and 7; exact table totals are distinguished from rounded figure labels. Supporting pages for scheduling, replay, recovery, limitations and appendix configuration/diagnostics are included even where not cropped. No external code, complete generated videos or separate supplements were inspected.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16. Appendix coverage: reviewed.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Abstract
- 1 Introduction
- 2 Related Work (2.1–2.2)
- 3 Data Generation Overview (3.1–3.2)
- 4 Assets and Dataset Curation (4.1–4.3)
- 5 System Architecture (5.1–5.3)
- 6 Trajectory Generation (6.1–6.3)
- 7 Offline Rendering (7.1–7.2)
- 8 Reliability and Operations (8.1–8.4)
- 9 Limitations and Discussion
- 10 Conclusion
- References
- Appendix A Key Configuration Parameters
- Appendix B Directory Conventions
- Appendix C Common Troubleshooting Procedure
- Appendix D Representative Production Failure Cases
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Separate supplemental material availability has not been fully verified.
- All five supplied text chunks and all 16 PDF pages were read. The extraction does not reconstruct figure images; the retained PDF supplied the inspected original Figures 1–5 and Tables 1–2.
- Separate supplemental material availability has not been fully verified; no separate supplement was supplied.
- Identity note: the observed title and all six authors match the catalog. PDF p. 1 shows arXiv:2609.03557v1 dated 3 September 2026 and an internal report date of September 4, 2026. These distinct date labels are preserved; no other edition was supplied or compared.
- No code, asset collection, generated video files, external project page or referenced paper was inspected. The static viewer figure was inspected, not an interactive viewer or complete video. No experiments were reproduced.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e-identityPDF p. 1, title, author/affiliation block, arXiv margin and report date
Title: Building Pretraining Data for World Models: An Unreal Engine-Based Pipeline for Action-Conditioned Video Generation. Authors: Haoyu Wang; Songchun Zhang; Haoran Li; Haoyang Huang; Zeyue Xue; Nan Duan. Affiliations: Joy Future Academy, JD; Tsinghua University; The Hong Kong University of Science and Technology. Margin: arXiv:2609.03557v1, 3 Sep 2026; internal date: September 4, 2026.
Go to primary source ↓e-scopePDF pp. 1–3, Abstract, Section 1 and Section 2.2; p. 13, Section 10
The paper addresses controls missing from ordinary video and describes the Unreal synthetic-data component used in EchoWM. It explicitly focuses on infrastructure rather than a new world-model architecture or demonstrated downstream improvement.
Go to primary source ↓e-samplePDF pp. 3–4, Section 3.1, Equation (1), Figure 1 and caption
A standard trajectory has 1,800 frames over 60 seconds, five synchronized views and nine discrete actions. Figure 1 samples different scenes: top/bottom rows are top-down/back views, center four images first-person, and side images left/right views.
Go to primary source ↓e-workflowPDF pp. 4–5, Section 3.2 and Figure 2
Stage I runs physics in PIE and records trajectories. Stage II uses a new process to replay states and render through MRQ without resampling actions or another physics run determining motion. Figure 2 links curation, scheduling, both stages and output management; its totals are rounded.
Go to primary source ↓e-assetsPDF pp. 5–6, Section 4.1
Fab assets are stored centrally and staged locally. Inventory excludes incomplete sublevels, showcases and failure-prone partitioned worlds. Character configuration includes animations, collision, scale, orientation, speed and locomotion. For non-humanoid extensions, actual visual height can differ from clamped collision size.
Go to primary source ↓e-filterPDF pp. 6–7, Section 4.2 and Equation (2); p. 5, Figure 2 scoring/eligibility blocks
Manual screening uses five 10-second 540p clips per candidate; quality screening uses twenty such clips per renderable level. Individual views pass when aes > 5.5 and 25 < lum < 140; level pass rate must exceed 25%. Tiled previews are excluded. Aesthetic scoring uses a pretrained image-quality model.
Go to primary source ↓e-scalePDF p. 6, Table 1, all Statistic/Value rows
Reported: 2,384 packs; 429 levels; 40 humanoid characters; five views; nine actions; 1,800 frames/60 seconds; 25 nodes with eight RTX 5090 GPUs/node; 2,691 h at 1080p; 6,076 h at 720p; 33 min/node-hour five-view throughput; 46.6% diagonal + backward + lateral actions. Timing protocol and uncertainty are not provided.
Go to primary source ↓e-diversityPDF p. 6, Figure 3 inner/outer rings and center; p. 7, Section 4.3
Scene shares: Historical & Fantasy 29.4%, Nature & Landscape 21.7%, Sci-Fi & Futuristic 21.7%, Urban & Architecture 19.1%, Other 8.2%. Center labels round the video hours. Section 4.3 reports 46.6% combined diagonal/backward/lateral actions.
Go to primary source ↓e-schedulingPDF pp. 7–8, Figure 4 and Sections 5.1–5.3
Central assignment is coarse-grained; nodes manage slots locally. Persistent pools reuse caches: cold starts can exceed ten minutes, and reported repeated-scene time is below half of cold-start time. Concurrency is empirically tuned; scenes are copied into slot projects and virtual displays are isolated.
Go to primary source ↓e-stage1PDF p. 8, Section 6.1, callback and state-machine sequence; p. 9, character preparation/spawning
Stage I uses nonblocking per-frame callbacks to apply controls and read resulting transforms. Starts/anchors or large objects guide spawn candidates, which fall to stable contact or are resampled after failing to settle.
Go to primary source ↓e-actionsPDF p. 9, Section 6.2
Actions are sampled from a predefined distribution and held for random durations. Point-of-interest guidance adjusts facing without changing sampling probabilities. Persistent negligible displacement under movement input triggers escape rotation and eventually respawn. Numeric distributions/recovery settings are not supplied.
Go to primary source ↓e-camerasPDF p. 9, Section 6.3
Five camera trajectories are computed after character motion is fixed. External offsets follow character orientation. Ray-based occlusion correction contracts camera distance quickly and expands slowly without affecting character motion.
Go to primary source ↓e-renderPDF pp. 9–10, Sections 7.1–7.2; p. 11, texture-residency continuation and encoding
The authors report unresolved aliasing in their SceneCapture setup and choose MRQ supersampling. Replay writes translation, rotation and scale, uses observed capsule height for mesh offsets, removes cinematic actors and adjusts texture residency. Image sequences are rendered without repeating character physics.
Go to primary source ↓e-viewerPDF p. 11, Figure 5, caption, encoding paragraph and Section 8.1
Five videos and a tiled preview accompany action, pose and execution metadata. Figure 5 pairs video, action state and 3D trajectories for qualitative scrubbing; it shows W + A at frame 891. The viewer supports debugging, not automatic acceptance or recovery.
Go to primary source ↓e-recoveryPDF p. 12, Sections 8.2–8.4
Trajectory markers track progress; an instance marker follows normal shutdown. Partial recovery requires both a trajectory marker and expected videos. Upload waits for a quiet period and retains local files until transfer succeeds. Separate failure snapshots and external GPU/disk monitoring support recovery.
Go to primary source ↓e-limitsPDF pp. 12–13, Section 9; p. 13, Section 10
The authors discuss competing control, diversity, physical, visual and scale objectives. Empirical perceptual thresholds lack downstream calibration. The main character-relative scheme differs from a newer camera-relative single-camera extension. Richer actions are absent; downstream experiments remain outside scope. Conclusion totals are rounded.
Go to primary source ↓e-configPDF pp. 14–15, Appendices A–C
Appendix A names rendering, character and scheduler parameters, including concurrency, resolution/frame count, retries and disk thresholds, without a full numeric configuration or software version. Appendix B documents storage conventions; Appendix C orders scheduler, engine, progress, GPU and snapshot diagnostics.
Go to primary source ↓e-failuresPDF p. 15, Appendix D introduction; p. 16, Table 2, Engine/Container/Hardware/Storage rows
Table 2 pairs symptoms, diagnostic signals, case-specific causes and mitigations: skeletal state, premature partition loading, cinematic camera takeover, animation root motion, container GPU mappings, memory limits, driver failures, cleanup and logging. Similar symptoms may have different causes; no failure frequencies are reported.
Go to primary source ↓8.5 Primary sources
Building Pretraining Data for World Models: An Unreal Engine-Based Pipeline for Action-Conditioned Video Generation ↗
PDF · 7,581 extracted words
Source fingerprint
d38d35cee66e97863bf102ba0919fe596c97f936f77dbfeb8087e991babd3cd7