GameWAM: A World Action Model for Video Games
1. Paper overview
In one sentence: GameWAM turns video co-training into native game control through shared visual context, short action commitments and mode-specific decoding, while sampled sources remain a control vulnerability. architecturemaskcontrolmcuablationslasi
| At a glance | What to know |
|---|---|
| Research problem | Source description Native keyboard–mouse control must handle rapid first-person motion, concurrent keys, persistent state and switches between camera and GUI cursor control. The authors seek dynamics-aware policy learning while keeping feedback frequent and memory bounded. problem |
| Core mechanism | Source description The system couples visual prediction and action learning through shared realized context, synchronized gameplay/GUI data, per-step action specialization and block-cycle history. architecturemaskdatacontrolhistory |
| A key reported result | MCU task completion: Mini 50.7%; All 46.6%. All Embodied/GUI/Combat: 47.5±36.0 / 60.0±38.6 / 32.2±30.2%; steps 138/155/203. Average success rate (%) and native steps per successful episode. Mini: 30 tasks, ten runs/task; All: over 800 tasks, five runs/task; final checkpoint. OpenHA Mini 36.8%; Game-TARS All 42.5%, category ASRs 50.4/39.1/38.1% and steps 373/406/372. Highest reported overall averages, but Game-TARS leads two All categories. The ± values are across-task standard deviations, not confidence intervals; successful-only steps and heterogeneous training prevent a controlled efficiency claim. mcuprotocol |
| Reading caution | Source description The controller lacks explicit symbolic task graphs or recipes. Evaluation largely concerns atomic tasks in digital games; visual memory lacks direct task-retrieval supervision. Dataset-composition effects are not fully isolated. limitations |
Core contributions
- Source description
The system couples visual prediction and action learning through shared realized context, synchronized gameplay/GUI data, per-step action specialization and block-cycle history. architecturemaskdatacontrolhistory
- Author claim
The authors claim the first WAM for native closed-loop gameplay and GUI control; this reading does not independently establish priority. problem
- Source description
LASI identifies structured low-frequency source influence using fixed-condition interventions, beyond observing diverse stochastic outputs. lasi
Figure 2. Video supervision shapes the visual context that the action branch uses at deployment. Original paper, p. 4 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read each panel from its inputs at the bottom toward its outputs at the top. During training, ground-truth future observations and actions are separately corrupted, while history and current observations supply context. The Video DiT and Action DiT have distinct prediction streams. At the top, the router chooses gameplay or GUI action predictions. On the inference side, follow the current observation through the VAE encoder and the cached visual path into action generation. The missing future-video output is consequential: Appendix A.4 specifies that the default model omits future-video denoising but retains Video-DiT processing of realized observations. architecturemaskrouting
What it supports. The method can benefit from visual prediction during learning without generating an imagined video at every decision. Both objectives shape the clean visual context consumed by action generation. The two action heads specialize the same native control representation for gameplay and GUI, with a route selected independently at each action step.
Where the evidence stops. The schematic labels an interaction “Joint Attn” and an output “Mixture.” Equations (5)–(6) and Appendix A.4 provide the operational details: noisy future modalities are decoupled, and rollout routing uses hard per-step selection. The picture alone should not determine the inference interpretation.
2. Motivation
2.1 The problem and the proposed response
Native keyboard–mouse control must handle rapid first-person motion, concurrent keys, persistent state and switches between camera and GUI cursor control. The authors seek dynamics-aware policy learning while keeping feedback frequent and memory bounded. problem
2.2 What this reading follows
A game controller must decide both how to move and when to look again. GameWAM approaches this through two generative branches trained on aligned observations and native actions, with a shared representation of the world already observed. The important deployment detail is that its default policy denoises actions without generating future video. It then executes only part of each forecast and updates its context from real feedback. The figures below connect that design to benchmark results and component tests. They also expose a second story: low-frequency structure in the random action source can systematically steer camera motion, making source handling part of the controller’s behavior. architecturemaskcontrolmcuablationslasi
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | WAMs |
| Architecture | Dual-system |
| Prediction paradigm | Joint prediction |
| Quadrant | Q3 · Dual-system × Joint prediction |
3.1 Evidence-based assessment
Supports the recorded classification
Dual-system × Joint prediction is supported by separate Video/Action DiTs with modality-specific states and parameters, trained on visual/action targets. Joint prediction here means shared-context co-training; default masking decouples noisy futures and deployment omits future-video denoising. Actions are generated directly, not through inverse dynamics. Gameplay/GUI heads are modes inside the action branch, not the basis for the dual-system judgment. architecturemaskrouting
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 How video learning reaches an action-only controller
Begin with a completed training trajectory and a decision boundary. Earlier observations form clean context; future observations and actions remain prediction targets. The two modalities are independently corrupted for flow training. Under the default mask, each target stream reads the clean prefix and its own noisy variables, so action prediction cannot obtain an answer from a simultaneously denoised future frame. The connection instead runs through the Video DiT’s clean-prefix K/V states: both objectives backpropagate through the path that constructs those states. At deployment, the same visual path encodes actual observations and supplies the Action DiT. Future-video denoising can therefore be omitted without removing learned visual context. This also explains why the action-only-supervision ablation asks a different question from action-only inference: one removes a training signal, while the other removes an online computation. maskarchitectureablations
Figure 3. Predict beyond the committed prefix, then rebuild the next decision from actual feedback. Original paper, p. 6 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start with the red and blue legend on the left: a red forecast extends beyond the blue action prefix that actually executes. Each new observation arrives after execution and supports another overlapping forecast. At the boundary between cycles, history is compressed and the transient K/V cache is reset. The right panel distinguishes that cache from persistent recent and long history. Recent history stores compressed executed segments; overflow supplies attention-pooled long-term memory. Section C.2 instantiates the drawing with P=16 predicted actions, E=8 executed actions and three committed blocks per cycle. Its configured cross-cycle memory holds at most 40 tokens. controlhistoryconfigurationablations
What it supports. The controller separates how far it forecasts from how long it acts without new feedback. Persistent memory survives a cache reset, while fine-grained current-cycle context remains available until that reset. Crucially, only observations produced by executed actions are admitted into these stores; a predicted but unexecuted future never becomes factual history.
Where the evidence stops. This diagram establishes information flow, not a uniform task benefit from memory. Table 2 reports a higher Embodied point estimate when history is removed, alongside worse GUI and Combat performance. Bounded persistent context therefore needs task-specific evaluation.
5.2 One mouse interface, two conditional action distributions
Camera control and cursor control reuse physical mouse channels, but the meaning and scale of a movement change when a GUI opens. GameWAM keeps a shared action representation and specializes its prediction rather than introducing a separate high-level policy. The Action DiT supplies gameplay and GUI flow predictions together with a mode logit for every native action step. During training, the observed mode chooses the supervised head and supplies the router target. During rollout, the predicted mode selects the entire action vector for that step; the route may change within a forecast. Continuous controls use the corresponding normalization, while binary coordinates retain their shared treatment. The reported unified-distribution ablation evaluates this package of specialization choices, so it does not tell us how much of the gain comes from routing versus normalization alone. routingobjectiveablations
5.3 Forecasts, committed actions and stochastic continuity
Consider the reported 16-action forecast with an eight-action commitment. After eight actions execute, new observations update the clean context and the previous unexecuted suffix is discarded. A later forecast may thus respond to a state that differs from what the earlier prediction anticipated. Across three committed blocks, the cache accumulates realized context; at the cycle boundary, persistent visual memory carries information into a fresh cache. Reader interpretation: this design addresses two different forms of continuity—retaining useful past observations and avoiding a long commitment before feedback. LASI adds another consideration. Reusing one sampled action source can repeat a directional camera tendency across decisions even as context changes. Refreshing the source reduces that coherent accumulation, but the fixed-condition experiments show why this is a mitigation rather than evidence that the underlying source sensitivity has disappeared. controlhistoryconfigurationlasilimitations
5.4 Training and inference
During training
Minecraft mixes Event-Anchored VPT, Regular VPT and scripted GUI samples at 80/5/15%. Event construction has a 200,000-window budget with 96-frame windows; training separately samples densely near anchors. APPO experts supply ViZDoom trajectories. Teacher-forced overlapping plans expose only causally complete observations. datacontrol
Wan2.2-TI2V-5B initializes the visual branch; adapted weights initialize the narrower Action DiT. Both use 30 layers and 24 heads, widths 3072/1024. VAE/text encoder remain frozen; DiTs, router, history and state encoder train. configuration
Video/action flow losses combine with router BCE and history prediction of a stop-gradient visual feature. Invalid coordinates are masked; continuous/discrete errors are reduced separately. Video/action/mode/history weights are 1/1/0.05/0.5. Training reports 21,900 AdamW steps, batch 352 and approximately 22 hours on eight H200s. objectivetraining
During inference
Default deployment encodes realized observations with the Video DiT, then denoises only actions using ten first-order steps from Gaussian sources. Future-video denoising is omitted. Fresh sources at every replan reduce repeated-source bias; evaluation uses the final second-epoch checkpoint. masktraininglimitations
5.5 Implementation flow
- Encode causally available observations
The Video DiT builds clean-prefix K/V states from observations and history. Video and Action DiTs have distinct hidden states and parameters. Both losses shape the shared prefix; each noisy future stream sees that prefix and itself, excluding the other noisy modality. architecturemask
- Decode heterogeneous native control
Minecraft uses 22 coordinates; continuous mouse channels serve camera or cursor control. Ground-truth mode selects the trained head; predicted mode selects the full action vector at rollout. Continuous normalization is mode-specific; binary controls are thresholded and hotbar selection constrained. ViZDoom uses a masked 9-D interface. routing
- Commit, observe and retain history
Forecast 16 actions, execute eight, discard the suffix and replan after real feedback. Three blocks form a 24-action cycle. At its end, reset transient K/V state and compress executed observations. Two recent segments plus gated long-term slots retain at most 40 cross-cycle tokens. controlhistoryconfiguration
6. Experiments & results
GameWAM learns native game control through parallel video and action flow models, then deploys action-only denoising conditioned on learned visual context. It combines per-step gameplay/GUI routing, overlapping forecasts and bounded visual history. Results support the complete controller on Minecraft and ViZDoom; source interventions reveal that sampled noise can impose coherent camera bias (architecture, mask, mcu, vizdoom, lasi).
6.1 Read the original evidence
Table 1. GameWAM leads the reported overall averages while category-level comparisons remain mixed. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Within each task category, read Steps alongside ASR Mini and ASR All. Steps average native environment actions only over successful episodes; they do not count denoising calls. Mini uses ten runs per task and All uses five. Section C.6 identifies the displayed uncertainty as standard deviation across per-task success rates. The two rightmost columns summarize the reported category averages. Game PT. identifies interaction pretraining spanning many games; a cross does not mean an absence of foundation-model pretraining. The paper uses red and blue for its first- and second-ranked comparable entries, so inspect actual values as well as color. mcuprotocolconfiguration
What it supports. GameWAM reports 50.7% Mini and 46.6% All average success, compared with OpenHA’s 36.8% Mini and Game-TARS’s 42.5% All. It uses 138, 155 and 203 steps per successful Embodied, GUI and Combat episode. The All category values also show why average leadership is narrower than universal leadership: Game-TARS exceeds GameWAM on Embodied and Combat.
Where the evidence stops. These systems differ in training data, supervision, available inputs and action representations. Successful-only step counts can also compare different subsets of solved episodes. Read this as a system benchmark, not a controlled estimate of architecture or training-data efficiency.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| MCU task completion Mini: 30 tasks, ten runs/task; All: over 800 tasks, five runs/task; final checkpoint. | Mini 50.7%; All 46.6%. All Embodied/GUI/Combat: 47.5±36.0 / 60.0±38.6 / 32.2±30.2%; steps 138/155/203. Average success rate (%) and native steps per successful episode | OpenHA Mini 36.8%; Game-TARS All 42.5%, category ASRs 50.4/39.1/38.1% and steps 373/406/372. Highest reported overall averages, but Game-TARS leads two All categories. The ± values are across-task standard deviations, not confidence intervals; successful-only steps and heterogeneous training prevent a controlled efficiency claim. mcuprotocol |
| MCU Mini component ablations Same Mini task categories as the full controller. | Full 50.7; action-only supervision 35.7; coarser sampling 36.7; no event sampling 38.0; unified action distribution 38.3; P=E 41.3; no history 46.7. Average success rate (%) | Removing video supervision gives the largest drop, 15.0 percentage points. Video co-training matters here. Without history, Embodied rises 70.0→75.0 while GUI/Combat decline. These rows do not establish statistical significance. ablations |
| MCU masking and model-side execution frequency Otherwise matched variants; frequency measured separately on one H200 over ten episodes. | Default: 50.7/46.6%, 12.51 Hz. Mini/All ASR (%) and execution frequency (Hz) | Joint future-video/action attention: 46.3/39.6%, 8.12 Hz. Default masking raises All ASR by 7.0 percentage points and model-side frequency about 1.54×. Timing excludes environment simulation/execution; GUI Mini slightly favors joint attention. mask-resultsprotocol |
| ViZDoom four-map control 50 episodes/map; corresponding final checkpoint; Battle 1/2 and Defend Line/Center. | 43.12 / 19.02 / 23.52 / 15.36. Average episode reward | Game-TARS: 18.87 / 10.04 / 15.24 / 14.00. Improves over Game-TARS on all maps; plotted means include no uncertainty intervals. vizdoomprotocol |
| Zero-shot Minecraft-to-VoxeLibre gameplay Unchanged Minecraft checkpoint; deterministic interface mapping; six tasks × 20 episodes; GUI excluded. | 71/120 successes (59.2%); task range 10–85%. Micro-averaged success rate | No comparative baseline reported. Selected low-level transfer between related voxel games, not broad game or physical-control generalization. transfer |
| LASI fixed-condition camera-source intervention 24 conditions × 16 sources; final Minecraft checkpoint; temporal DCT modes 0–2. | r=0.890; donor-follow 94.8%; zeroing removes 99.25%. Yaw DCT0 association, donor-follow rate and source-induced variance removed | 95% CIs: [0.835,0.936], [92.2,97.1]%, [98.93,99.50]%, respectively; donor/base or zeroed/base comparisons hold context fixed. Supports causal source influence on coarse generated motion. These diagnostic statistics are not task-success gains or evidence that zeroing is a deployable cure. lasilimitations |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Table 6. Removing direct noisy-future coupling improves overall control and reduces online computation in this comparison. Original paper, p. 28 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Compare the two rows within each column before looking at the far-right frequency measurement. Both variants use the same realized clean prefix; the joint alternative additionally exchanges noisy future-video and action information, requiring future-video denoising online. The default can omit that future-video stream. Mini and All are separate evaluations, while Steps again includes successful episodes only. Section C.6 measures frequency separately on one H200, averaging ten episodes and including model-side replanning computation while excluding environment simulation and execution. It should therefore be kept distinct from the native-action counts beside each success-rate column. maskmask-resultsprotocol
What it supports. Default masking reports 50.7% versus 46.3% Mini ASR, 46.6% versus 39.6% All ASR, and 12.51 versus 8.12 Hz. The latter is approximately 1.54 times the model-side execution frequency. The joint variant still has a slightly better GUI Mini result, 45.0% versus 43.0%, so direct coupling is not worse in every column.
Where the evidence stops. The joint variant’s smaller successful-episode step counts on GUI and Combat accompany different success rates. They cannot by themselves show a better controller. The reported comparison also does not identify a unique internal mechanism causing the overall performance gap.
Table 2. Video co-training and temporal sampling have the largest measured ablation effects. Original paper, p. 9 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Use the full model as the reference row, then compare each variant’s rightmost average before inspecting its category pattern. “Action-only supervision” changes training: it removes the future-video learning signal, unlike the default action-only deployment path that still benefits from video co-training. “Matched prediction–execution horizon” sets P=E, eliminating forecast extension beyond the committed interval. “Unified action distribution” removes the mode-specific formulation. Finally, inspect the history row across all three categories rather than reading its average alone. Its Embodied result rises, while GUI and Combat fall. The table’s uncertainty values should not be read as confidence intervals on the average differences. ablationsroutingmaskcontrolprotocol
What it supports. Removing video supervision lowers average Mini ASR from 50.7% to 35.7%, a 15.0 percentage-point drop. Coarser sampling gives 36.7%; removing event-anchored clip sampling gives 38.0%. Matching prediction and execution horizons gives 41.3%. Together these results support the training and timing choices, while the history result shows that component benefits can depend on task category.
Where the evidence stops. These are reported component comparisons, not significance tests or a complete factorial study. In particular, the unified-action row evaluates a combined formulation involving prediction and normalization; it does not isolate their individual contributions.
Figure 5. Controlled source interventions change the coarse camera trajectory even when conditioning stays fixed. Original paper, p. 9 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Follow one row across all three panels, such as Y0. P0–P2 and Y0–Y2 denote pitch and yaw temporal DCT modes 0–2 in eight-step camera chunks. The first panel measures association while varying sources within each fixed condition. The middle panel replaces only source modes 0–2 with a donor’s coefficients; donor-follow means the new output is closer to the donor output than the base output. The last panel zeros those source modes and measures how much within-condition source-induced output variance disappears. The controlled suite uses 24 conditions and 16 sources per condition on the final Minecraft checkpoint. lasilimitations
What it supports. Yaw DCT0 has correlation 0.890, donor-follow rate 94.8%, and 99.25% variance removal under zeroing. The interventions support a causal effect of low-frequency source structure on coarse generated motion. They sharpen the diagnosis beyond correlation alone, since visual and other conditioning are held fixed while the source coefficients change.
Where the evidence stops. Variance removal is not task success. These diagnostics neither establish that zeroing preserves useful camera control nor identify the internal cause of LASI. The deployed mitigation refreshes sources between replans; the paper states that underlying sensitivity remains.
7. Analysis & limitations
7.1 What the evidence leaves open
The controller lacks explicit symbolic task graphs or recipes. Evaluation largely concerns atomic tasks in digital games; visual memory lacks direct task-retrieval supervision. Dataset-composition effects are not fully isolated. limitations
Source resampling mitigates accumulated coherent bias but leaves LASI. Diagnostics do not localize its internal cause; attempted training corrections did not robustly remove it without trade-offs. limitations
7.2 Questions for discussion
- Would task-aware memory supervision improve the categories harmed by removing history?
- Can LASI be reduced while preserving legitimate smooth camera motion and successful closed-loop behavior?
8. Reproducibility audit
8.1 Requirements and known gaps
Reproduction needs aligned VPT/MineStudio streams, APPO demonstrations, pretrained modules and environment interfaces. Preserve mode labels, masks, event sampling, P/E geometry and fresh-source inference. Artifact release is promised; fitted normalization statistics, numeric binary thresholds and the precise width-adaptation recipe are not supplied. dataconfigurationtrainingrelease
Proposed checks: compare masks with equal data and evaluation seeds while timing observation encoding separately; compare reused versus refreshed sources on paired tasks, measuring both camera drift and success. Neither check was run here. mask-resultslasilimitations
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Separate the value of video supervision from the cost of noisy-future coupling
Reader-proposed check, not performed: train the default decoupled model, the joint-mask variant and the action-only-supervision variant with the same data mixture, initialization procedure, update count and task protocol. Fix P=16, E=8 and fresh-source inference. Evaluate paired MCU Mini tasks and seeds, reporting category success and uncertainty across independent training runs. Time realized-observation encoding separately from action and future-video denoising on one H200. The paper’s interpretation would be weakened if the default’s advantage over action-only supervision disappears across runs, or if removing future-video denoising fails to explain the measured model-side timing difference. maskconfigurationablationsmask-resultsprotocol
Check 2: Test whether refreshing sources reduces drift without sacrificing useful motion
Reader-proposed check, not performed: use the final checkpoint with paired task states and compare one fixed source reused throughout an episode against independently refreshed sources at each replan. Keep the ten-step deployment sampler, task budget and P/E geometry identical; repeat across many source seeds. Measure task success, cumulative signed camera rotation, and low-DCT source/action association, including failed episodes. Use low-band zeroing only as a diagnostic third condition, not an assumed remedy. The mitigation claim predicts fewer persistent rotation failures after refreshing, even if source/action association remains. A decline in task success or loss of needed smooth turns would expose a practical trade-off. traininglasilimitationsprotocol
8.3 Reading coverage
Visual audit: Visually inspected the v1 title/author page; Figures 2–5; Tables 1–2 and 5–7; and the declared method, objective, implementation, evaluation and LASI pages needed to interpret the six crops. All final crops were viewed, including revised table crops. The complete supplied text and appendices were read; other figure images, external artifacts and separate supplements were not visually inspected or verified.
PDF pages inspected for this edition: 1, 4, 5, 6, 7, 8, 9, 20, 21, 25, 26, 27, 28, 37, 38, 43. Appendix coverage: reviewed.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Abstract and Sections 1–6 (PDF pp. 1–10)
- AI Use, Ethics and Reproducibility Statements; References (pp. 10–14)
- Appendix A.1–A.6: extended method (pp. 16–22)
- Appendix B.1–B.6: alignment, interfaces and datasets (pp. 22–25)
- Appendix C.1–C.6: implementation and evaluation (pp. 25–27)
- Appendix D.1–D.5.8: scale, masking, qualitative rollouts, transfer and LASI (pp. 27–42)
- Appendix E.1–E.2: limitations and interpretation (pp. 42–44)
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Separate supplemental material availability has not been fully verified.
- The verified edition is arXiv:2608.26200v1, 25 August 2026. No other revision was supplied or compared. Catalog title and four authors match; its affiliation snapshot omits LIGHTSPEED and Independent Researcher.
- Text extraction does not reconstruct figure images. Visual inspection covers Figures 2–5 and Tables 1–2, 5–7; remaining figure images were not visually inspected, although all supplied captions and appendix text were read.
- Separate supplemental material availability has not been fully verified; no separate supplement was supplied.
- Code, project links, datasets and checkpoints were not inspected. No experiments were reproduced.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
identityPDF p. 1, title, author/affiliation block and arXiv margin
GameWAM: A World Action Model for Video Games; Yuncheng Guo, Zhanqiu Zhang, Yiwen Guo, Weijia Li; arXiv:2608.26200v1 [cs.AI], 25 Aug 2026. Affiliations are Fudan University, LIGHTSPEED, Independent Researcher, and Tsinghua Shenzhen International Graduate School, respectively. Title and authors agree with the catalog; its affiliation string omits the middle two affiliations.
Go to primary source ↓problemPDF pp. 1–3, Abstract and Section 1
The authors motivate native control through the complementary limitations of direct policies and externally action-conditioned world models. They propose joint visual/action learning, mode-specific action generation, short execution commitments and persistent history; first-WAM status is a novelty claim.
Go to primary source ↓architecturePDF p. 4, Figure 2; p. 5, Section 4.1, Eqs. (1)–(5)
Separate Video and Action DiTs receive visual history, current observations, task text and optional state. Joint flow training interpolates clean targets and Gaussian sources; future video and native actions are prediction targets. Binary decisions are decoded after generation.
Go to primary source ↓maskPDF pp. 20–21, Section A.4, Eqs. (28)–(32)
Modality-specific streams share clean-prefix K/V states produced by the Video DiT; both losses backpropagate through that context pathway. Default masking excludes interaction between simultaneously noisy future video and actions. Action-only deployment retains Video-DiT prefix encoding but omits iterative future-video denoising. The joint alternative requires both future streams.
Go to primary source ↓routingPDF p. 6, Section 4.1, Eq. (6); p. 21, Section A.5; pp. 22–23, Tables 3–4 and Section B.2
Observed per-step gameplay/GUI mode selects the supervised flow head; a sigmoid threshold of one half selects the entire native-action prediction at rollout. Continuous normalization is mode-specific. Minecraft has 22 coordinates, including two continuous channels; hotbar selection permits at most one slot. ViZDoom uses a masked 9-D interface and no GUI regime.
Go to primary source ↓controlPDF p. 6, Figure 3 and Section 4.2, Eq. (7); pp. 16–17, Section A.1
A P-action forecast commits only its first E actions. Overlapping training targets are anchored every E steps with causal teacher forcing. After execution the old suffix is discarded; only realized observations become context. Longer P adds targets and losses, even without lengthening the commitment chain.
Go to primary source ↓historyPDF pp. 6–7, Eqs. (8)–(9); pp. 17–18, Sections A.2–A.3; p. 25, Section C.2
Transient K/V state resets at cycle boundaries. Executed visual segments are VAE-encoded and Conv3D-compressed into recent FIFO memory; overflow updates gated attention-pooled long-term slots. Configuration: two 16-token recent segments and eight long-term tokens, half-lives of two and four cycles, at most 40 cross-cycle tokens.
Go to primary source ↓objectivePDF p. 7, Section 4.3, Eqs. (10)–(14); pp. 21–22, Section A.6; p. 26, Sections C.4–C.5
The loss combines video flow error, separately reduced valid continuous/discrete action errors, masked router BCE, and history prediction of a stop-gradient visual feature using cosine and squared error. Video/action/mode/history weights are 1/1/0.05/0.5; continuous/discrete groups each have weight 1. Flow times are independently sampled with a shifted schedule.
Go to primary source ↓dataPDF p. 5, Section 3; pp. 22–26, Sections B.1–B.6 and C.3
Pre-action alignment and null-action filtering precede construction. Minecraft sample mixture is 80% Event-Anchored VPT, 5% Regular VPT and 15% scripted GUI. Event construction uses a 200,000-window budget and 96-frame windows [t−87,t+8], start separation at least 64 and same-event anchor separation greater than 96 frames. Training stride is 2 near a radius-8 event neighborhood and 16 elsewhere; Regular VPT/GUI use 32/8. ViZDoom demonstrations come from APPO experts.
Go to primary source ↓configurationPDF p. 25, Sections C.1–C.2; p. 27, Table 5, backbone scale rows
Wan2.2-TI2V-5B initializes the visual branch; the 1B Action DiT adapts its weights to width 1024 versus 3072, retaining 30 layers and 24 heads. VAE/text encoder are frozen; DiTs/history/state modules train. Observations: 224×224, one per two actions; P=16, E=8, three blocks per 24-action cycle.
Go to primary source ↓trainingPDF p. 26, Sections C.4–C.6; p. 27, Section D.1
Training uses two epochs, 21,900 AdamW steps, batch 352, BF16, ZeRO-2 and eight H200s for approximately 22 hours. Peak learning rate is 4×10⁻⁵ after 1,095 warmup steps, decaying to 4×10⁻⁷. Standard inference uses ten first-order steps and fresh action sources per replan; final second-epoch checkpoints are used without validation/MCU selection. The 2.79B token presentations include overlapping targets.
Go to primary source ↓protocolPDF p. 8, Table 1 and Figure 4 captions; pp. 26–27, Section C.6
MCU Mini: 30 tasks, ten each mining/crafting/combat, ten runs/task; All: five runs/task. Success follows environment completion signals. Steps count executed native actions, averaged only over successes. Standard deviations summarize per-task success rates. Systems differ in data, supervision, model class, action representation and inputs. ViZDoom averages 50 episodes/map. Separate frequency measurement uses one H200 and ten episodes, excluding simulation/execution time.
Go to primary source ↓mcuPDF p. 8, Table 1, GameWAM, OpenHA and Game-TARS rows, category and Avg columns
GameWAM Mini/All ASRs: 50.7/46.6; OpenHA: 36.8/31.5; Game-TARS All: 42.5, Mini absent. GameWAM All Embodied/GUI/Combat: 47.5±36.0, 60.0±38.6, 32.2±30.2; successful-episode steps: 138/155/203. Game-TARS All category ASRs: 50.4/39.1/38.1; steps: 373/406/372.
Go to primary source ↓vizdoomPDF p. 8, Figure 4, labeled bars for Battle 1, Battle 2, Defend Line and Defend Center
Visually inspected GameWAM reward labels: 43.12, 19.02, 23.52, 15.36; corresponding Game-TARS: 18.87, 10.04, 15.24, 14.00. Figure reports 50 episodes/map without uncertainty intervals.
Go to primary source ↓ablationsPDF p. 9, Table 2, all rows; p. 8, Section 5.4
Mini average ASR: full 50.7; no video supervision 35.7; coarser sampling 36.7; no event-anchored sampling 38.0; unified action distribution 38.3; P=E 41.3; no history 46.7. Removing history raises Embodied 70.0→75.0 but lowers GUI 43.0→31.0 and Combat 39.0→34.0.
Go to primary source ↓mask-resultsPDF p. 28, Table 6 and Section D.2; p. 27, Section C.6, Online execution frequency
Default/joint masking: Mini ASR 50.7/46.3, All 46.6/39.6, frequency 12.51/8.12 Hz. Joint masking has GUI Mini 45.0 versus 43.0 and fewer successful-episode GUI/Combat steps. Training/evaluation settings are otherwise matched; frequency is measured separately from task performance.
Go to primary source ↓lasiPDF p. 9, Figure 5 and Section 5.5; pp. 37–38, Sections D.5.1–D.5.3, Eqs. (36)–(39)
Final-checkpoint diagnostics use 24 conditions × 16 sources. Eight-step camera chunks undergo orthonormal temporal DCT; modes 0–2 are low frequency. Yaw DCT0: r=0.890, 95% CI [0.835,0.936]; donor-follow 94.8%, CI [92.2,97.1]%; zeroing removes 99.25% of within-condition source-induced variance, CI [98.93,99.50]%. Donor-follow means closer to donor than base output. CIs use 10,000 condition-cluster bootstrap resamples where available; trajectory-resolved diagnostics use 20 integration steps.
Go to primary source ↓transferPDF pp. 36–37, Section D.4 and Table 7
Unchanged Minecraft checkpoint in VoxeLibre, deterministic control mapping, no target-game training. Six basic gameplay tasks × 20 episodes: 71/120 successes, 59.2% micro-average, task range 10% Kill Cow to 85% Chop Tree. GUI excluded; related voxel-world semantics; no comparative baseline.
Go to primary source ↓limitationsPDF pp. 42–44, Sections E.1–E.2; pp. 41–42, Section D.5.8
GameWAM is a low-level controller without symbolic plans/recipes. History is predominantly visual without task-retrieval supervision. Evaluation concerns digital games and largely atomic tasks. LASI does not identify a unique internal cause or eliminate conditioning influence. Resampling mitigates accumulated repeated-source bias but leaves sensitivity; training interventions trade suppression against action learning or retain residual sensitivity.
Go to primary source ↓releasePDF p. 10, Reproducibility Statement; pp. 23–26, Sections B.2 and C.1–C.6
The paper promises future release of code, processing scripts, datasets, weights and configurations subject to third-party restrictions. Text describes normalization/thresholding without fitted statistics or numeric binary thresholds, and adapting pretrained widths without an exact weight-mapping recipe.
Go to primary source ↓8.5 Primary sources
GameWAM: A World Action Model for Video Games ↗
PDF · 21,030 extracted words
Source fingerprint
0dc18fa7f9b64e069b98f604576f7910186478ce2fedb1b3f9c40b8261632255