Craftax: A Lightning-Fast Benchmark for Open-Ended Reinforcement Learning
1. Paper overview
In one sentence: Craftax makes long-budget exploration experiments inexpensive, but its tested policies still struggle to turn local reward collection into deeper progression. e-coree-mechanicse-protocole-classic-resulte-maine-baselines
| At a glance | What to know |
|---|---|
| Research problem | Author claim Open-ended RL benchmarks trade complexity against affordable experimentation. The authors argue that slow survival games constrain research, while fast simple environments under-test exploration. Craftax separates a large-budget challenge from a small-budget sample-efficiency challenge. e-coree-protocol |
| Core mechanism | Source description Craftax-Classic reimplements Crafter in JAX; Craftax adds interconnected floors, richer combat, randomized potion meanings and attribute allocation. The contribution is an executable benchmark with baseline evidence. e-coree-mechanics |
| A key reported result | Craftax-1B exploration performance: PPO-RNN approximately 15–16% at the endpoint, read from Figure 3. Reward as percentage of the stated maximum 226. Craftax-Symbolic, 1B interactions, ten seeds; one-standard-error shading. PPO/ICM/RND near 12%; E3B near 11%. These are graph estimates, not exact table values. Memory improves reward, but Advanced and Very Advanced achievements remain unachieved. Dungeon entry improves without robust gnomish-mines exploration. e-maine-diagnostice-protocol |
| Reading caution | Reader analysis Symbolic observations remove much visual representation learning. These experiments do not establish pixel-based control, physical execution or indefinite improvement beyond the finite achievement inventory. e-interfacee-rewardse-protocol |
Core contributions
- Source description
Craftax-Classic reimplements Crafter in JAX; Craftax adds interconnected floors, richer combat, randomized potion meanings and attribute allocation. The contribution is an executable benchmark with baseline evidence. e-coree-mechanics
- Source description
Symbolic interfaces, compiled learning and optimistic resets make large experiments practical. Classic has documented differences from Crafter, so their scores are not strictly interchangeable. e-interfacee-speede-resetse-classic-differences
Figure 25. The interface exposes local terrain and inventory; the experiments use their symbolic counterparts. Original paper, p. 22 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the three panels from Crafter to Classic to the expanded Craftax environment. Each combines a map around the player with a lower status/inventory area. These are examples of the optional pixel renderer, not screenshots of the symbolic tensors used in the reported benchmarks. Appendix D.4 explains the corresponding information flow: encode visible blocks, items and creatures, include lighting, then append player statistics and inventory. Craftax uses a 9-by-11 local map view, so the policy does not directly observe the entire generated world. The source masks map contents below light level 0.05. Those observation restrictions remain even when representation learning from pixels is removed. e-interfacee-protocole-mechanics
What it supports. The benchmark separates visual recognition from strategic decision-making. Symbolic input makes a fast baseline practical while preserving a local view, resource management and action prerequisites. This figure helps identify what information the agent receives, but the numerical gains in this report come from the symbolic evaluation protocol.
Where the evidence stops. These panels are interface examples, not a pixel-policy evaluation or an architecture diagram. They do not show that an agent has discovered the hidden potion mapping, remembered earlier floors or learned a planning model.
2. Motivation
2.1 The problem and the proposed response
Open-ended RL benchmarks trade complexity against affordable experimentation. The authors argue that slow survival games constrain research, while fast simple environments under-test exploration. Craftax separates a large-budget challenge from a small-budget sample-efficiency challenge. e-coree-protocol
2.2 What this reading follows
A fast simulator changes which reinforcement-learning questions are practical to ask. Craftax combines procedural survival-game mechanics with a compiled JAX pipeline, then distinguishes a billion-interaction exploration challenge from a million-interaction sample-efficiency challenge. Its easier Classic variant approaches reward saturation, motivating a substantially harder environment with interconnected floors, richer combat and episode-specific potion meanings. This reading follows the agent’s information, the weighted objective and the resulting behavior before examining curriculum evaluation. The evidence supports faster experimentation and a remaining exploration gap; it does not establish a learned world-action architecture or a solved game. e-coree-mechanicse-protocole-classic-resulte-maine-baselines
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | Benchmarks & simulators |
| Architecture | Not applicable |
| Prediction paradigm | Not applicable |
| Quadrant | Not applicable |
3.1 Evidence-based assessment
Supports the recorded classification
The benchmark/simulator category and RL subcategory are supported. Architecture, prediction paradigm and WAM quadrant are not applicable: simulator baselines do not constitute a proposed One Model system. Physics simulation is not established by these grid-based game dynamics and merits separate catalog review. ICM prediction losses do not change the resource classification. e-coree-mechanicse-interfacee-baselines
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Follow the observation–action loop before thinking about a world model
Start with a procedurally generated world whose full simulator state is available to the environment machinery. The policy receives a restricted local observation instead: visible terrain, items, creatures and lighting, together with inventory and player statistics. A baseline network turns that observation into one of the available actions; Craftax applies the game rules and returns new observations and rewards. Actions with unmet prerequisites still consume time. PPO-RNN supplements the current observation with recurrent state, which can retain information unavailable in a single frame. The ICM baseline learns predictive features to supply an exploration reward during training. Its presence does not make the simulator a learned dynamics model or demonstrate inference-time rollout planning. Figure 25 illustrates the interface, while the experiments use the faster symbolic version. e-interfacee-coree-baselines
Table 6. Weighted achievements define the measured objective and reveal a count discrepancy in the text. Original paper, p. 32 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Follow each row across ID, name and category. The number in parentheses is the reward for first completing that achievement in an episode. For example, COLLECT_WOOD is Basic (1), ENTER_DUNGEON is Intermediate (3), ENTER_SEWERS is Advanced (5), and ENTER_GRAVEYARD is Very Advanced (8). The row order is an identifier list, not a dependency graph or guaranteed acquisition sequence. Section 3.3 says the authors increased rewards for harder categories because a flat reward encouraged agents to avoid dangerous dungeons. Section 4.3 normalizes return by a stated maximum achievement reward of 226. Damage and recovery also contribute separate small reward terms. e-rewardse-diagnostic
What it supports. Completing more achievements and maximizing reward can favor different behavior. Moving underground may improve weighted return while reducing the frequency of easy overworld achievements. The table therefore provides the context needed to read the learning curves and understand why individual achievement diagnostics matter alongside the aggregate score.
Where the evidence stops. Section 3.3 says Craftax contains 65 achievements, but this faithful table lists 67 IDs, from 0 through 66. The supplied source does not reconcile the count; a reproduction must check which inventory its implementation uses.
5.2 Why cheaper transitions do not automatically solve exploration
The engineering contribution connects two costs: producing transitions and generating new worlds. Compiling simulation and policy learning together removes repeated CPU–GPU transfer, while fixed-size state representations fit accelerator execution. Optimistic resets address expensive procedural generation by producing a small pool of fresh worlds for the workers that actually terminate. Appendix C’s probability argument estimates when that pool overflows; it relies on a binomial model rather than a measured guarantee for every policy. The resulting speed makes large interaction budgets accessible. Yet the learning evidence separates computational access from competence: Classic approaches reward saturation, while the harder Craftax remains shallow even in the appendix’s longer run. As a reader interpretation, this shifts attention toward exploration incentives, memory and skill acquisition once raw transition availability becomes less restrictive. e-coree-classic-differencese-resetse-speede-classic-resulte-long
5.3 Separate curriculum learning from learning to exploit a curriculum
PLR chooses previously generated levels using estimated regret, whereas ACCEL also edits levels. These procedures change the distribution of training experience, so a reward increase on replayed levels does not by itself demonstrate better performance on ordinary worlds. Figure 8 makes this distinction visible, and Figure 7 evaluates checkpoints on a common set of normally generated levels. There is a second distinction: robust PLR skips gradient updates on newly sampled levels, so matching environment interactions leaves update exposure unequal. Appendix G also changes rollout organization and lists different replay rates and learning rates. Reader analysis therefore treats the ranking as an outcome of the specified complete training systems. To identify curation’s causal benefit, one would need matched-update controls and separately reported replay, ordinary-generator and fixed-level results. e-ued-methode-ued-resultse-shifte-ued-config
5.4 Training and inference
During training
Exploration baselines use four-layer, width-512 policy/value MLPs; PPO-RNN adds a GRU. The 1B PPO setup uses 1024 workers, 64-step rollouts, eight minibatches, four epochs and learning rate 0.0002 with annealing. Its sweep tunes parameters individually on one seed. e-baselinese-training
RND matches a fixed random target, ICM rewards world-model error, and E3B rewards episodic novelty. These auxiliary objectives change exploration during learning. The 1M random-search configuration instead uses 256 workers, 16-step rollouts and learning rate 0.0003. e-baselinese-training
During inference
A policy maps observations, and recurrent state when present, to simulator actions; the next observation closes the loop. ICM is an intrinsic-reward baseline, not evidence of inference-time imagined-rollout planning or a unified future-and-action predictor. e-interfacee-baselines
The main challenges permit 1B or 1M symbolic-environment interactions. UED evaluates checkpoints on twenty fixed normally generated levels. This tests in-distribution procedural generalization, not transfer to novel game rules. e-protocole-ued-results
5.5 Implementation flow
- Generate and observe
Nine floors create related exploration contexts. A 9-by-11 local view encodes blocks, items, creatures and light, followed by inventory/statistics. Cells below light level 0.05 are masked. The reported flat symbolic observation sizes are 8268 for Craftax and 1345 for Classic. e-mechanicse-interface
- Act and receive feedback
Craftax has 43 discrete actions; Classic has 17. Unmet prerequisites turn an attempted action into a no-op that still advances time. First achievement completion earns its category reward, with additional damage/recovery terms. e-interfacee-rewards
- Compile collection and resets
Simulation and learning run on the accelerator. Fixed-capacity entity arrays permit compilation. Optimistic resets generate one fresh world per sixteen workers and allocate these to terminated episodes; overflow can duplicate initial states. e-coree-classic-differencese-resets
- Choose the training distribution
Ordinary training samples generated worlds. PLR prioritizes buffered levels by estimated regret; robust PLR updates only from replay. ACCEL adds mutations. UED separates short policy-learning rollouts from longer level-scoring trajectories. e-ued-methode-ued-config
6. Experiments & results
Craftax makes a procedurally generated survival game fast enough for billion-step reinforcement-learning studies. Its JAX implementation combines symbolic observations with accelerator-resident simulation and training. The harder environment exposes a gap between collecting easy rewards and exploring deeper floors: recurrent PPO helps, but the tested exploration and curriculum methods leave substantial challenges unsolved.
Craftax is an environment/benchmark paper, and its method is documented through game mechanics, interface specifications and implementation prose rather than a neural architecture diagram. Figure 25 and Table 6 therefore illustrate the mechanism. The source offers algorithm comparisons and achievement/distribution diagnostics, but no controlled neural-architecture ablation; Figures 5 and 8 serve as diagnostics without being presented as causal ablations. e-coree-interfacee-rewardse-baselinese-diagnostice-shift
6.1 Read the original evidence
Table 1. Craftax achieves high learning-pipeline throughput under the paper’s single-machine comparison. Original paper, p. 14 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read throughput together with the worker column: the fastest point for each environment uses a different amount of parallelism. Classic reaches 405,618 steps per second and Craftax 266,961, both with 4096 workers; Crafter reaches 1,580 with 1024. Appendix B uses one RTX 4090, an i9-13900K and 32 GB RAM, and measures the whole learning pipeline. The Craftax rows use symbolic observations and compiled PureJaxRL training. Other rows use different implementations, including NetHack IMPALA and stable-baselines3 PPO for Crafter and Procgen. MineRL is the exception: the authors could not run a baseline and timed random actions instead. e-speede-core
What it supports. The measured throughput supports affordable large-interaction experiments on Craftax. Figure 2 summarizes the Classic and Craftax advantages over Crafter as approximately 257-fold and 169-fold. These gains motivate asking how algorithms behave after much more experience, rather than assuming the original small interaction budget captures their eventual capabilities.
Where the evidence stops. This is a practical system comparison with different environments, observations and implementations. It does not isolate JAX as the sole cause. No uncertainty is tabulated, and MineRL’s random-action timing is not an end-to-end learning measurement.
Figure 3. Recurrent PPO gains reward, but the resulting score remains far from benchmark completion. Original paper, p. 5 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. The horizontal axis counts millions of environment interactions; the final tick is the Craftax-1B budget. The vertical axis expresses reward as a percentage of the paper’s maximum 226, not the probability of solving the game. Follow the blue PPO-RNN curve against PPO and the three intrinsic-reward baselines. Every curve averages ten seeds, with shading for one standard error. The recurrent policy separates most clearly late in training. Figure 4 and the achievement panels are needed to interpret that separation: gains concentrate in easier categories and dungeon-related behavior. The reward plot itself cannot reveal how far through the floor sequence the policy reliably travels. e-maine-baselinese-protocole-long
What it supports. Visually, PPO-RNN finishes around 15–16% of maximum reward, compared with roughly 12% for PPO, ICM and RND and roughly 11% for E3B. These rounded readings support a recurrent-policy advantage under this protocol. They also show that the tested novelty bonuses do not improve aggregate reward over PPO.
Where the evidence stops. The endpoints are approximate readings, not tabulated statistics. Late acceleration alone does not prove continued deep exploration: the separate four-seed 10B study remains shallow. Memory, parameter capacity and learning dynamics are not isolated by this comparison.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| End-to-end environment throughput One RTX 4090, i9-13900K, 32 GB RAM; each environment uses its best worker count. | Craftax: 266,961 at 4096 workers; Classic: 405,618 at 4096. Environment steps per second; higher is faster | Crafter: 1,580 at 1024; Figure 2 reports approximately 169× and 257× speedups respectively. Measured pipeline throughput, not equal-architecture algorithmic speed. MineRL alone uses random actions; no timing uncertainty is tabulated. e-speed |
| Craftax-1B exploration performance Craftax-Symbolic, 1B interactions, ten seeds; one-standard-error shading. | PPO-RNN approximately 15–16% at the endpoint, read from Figure 3. Reward as percentage of the stated maximum 226 | PPO/ICM/RND near 12%; E3B near 11%. These are graph estimates, not exact table values. Memory improves reward, but Advanced and Very Advanced achievements remain unachieved. Dungeon entry improves without robust gnomish-mines exploration. e-maine-diagnostice-protocol |
| Craftax-1M sample efficiency Craftax-Symbolic, 1M interactions, ten seeds; one-standard-error shading. | All five methods finish near 2%, approximately read from Figure 6. Reward as percentage of maximum | Little separation among PPO, PPO-RNN, ICM, E3B and RND. Short-budget results mostly measure basic achievement acquisition rather than deeper exploration. e-smalle-protocol |
| Longer training on Craftax-Extended Appendix F.2 condition, PPO-RNN, 10B interactions, four seeds. | Approximately 17% reward in Figure 22; no gnomish-mines entry in Figure 21. Normalized reward and achievement success | Modest progress beyond the separate 1B experiment; seed counts differ. More interactions do not establish deeper exploration. The learning-rate explanation for late gains is an author hypothesis. e-longe-main |
| Craftax-Classic baseline saturation Appendix F.3, 1B interactions, ten seeds; plots identify PPO-RNN. | About 90%, consistent with the rounded headline; Figure 23 stays below full completion. Reward as percentage of maximum | Diamonds and eating plants remain unreliable. The introduction calls the 51-minute run PPO; the appendix labels it PPO-RNN. Near-saturation motivates Craftax; universal achievement completion and an unambiguous feedforward-PPO result are not established. e-classic-result |
| Craftax-1B curriculum evaluation 1B training interactions; twenty fixed normally generated evaluation levels; ten runs. | PLR ranks highest; unrestricted-swap ACCEL ranks lowest in Figure 7. Evaluation reward as percentage of maximum | DR remains competitive. Swapping curricula perform better on replay than normally generated levels. Curriculum return cannot substitute for evaluation return. Equal interactions do not equalize gradient updates or hyperparameters. e-ued-resultse-shifte-ued-config |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Figure 5. Achievement-level diagnostics distinguish higher return from broader exploration. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. First read the title and vertical scale of every panel: identical plot heights do not mean comparable success rates. Wood-pickaxe success approaches the top of a percentage scale, whereas the gnomish-mines panel uses fractions of one percent. PPO-RNN reaches the dungeon much more frequently, while its zombie success declines. The authors explain that stronger agents spend more time underground and pass up overworld rewards. The EAT_PLANT panel shows E3B exploring a behavior that recurrent PPO largely ignores. Use this figure’s own legend: E3B is orange here, whereas Figure 3 uses red, and the PPO color also changes between figures. e-diagnostice-maine-rewards
What it supports. The recurrent policy’s reward advantage corresponds to more dungeon entry, not reliable exploration of the next floor. E3B’s stronger plant-eating behavior despite lower aggregate reward shows that an exploration bonus can change the repertoire without improving the benchmark objective. Both observations argue for retaining achievement profiles alongside a single score.
Where the evidence stops. This is a behavioral diagnostic, not a controlled ablation proving why memory works. The panels have different scales and noisy rare-event traces; their peaks should not be treated as exact final success rates or evidence of reliable progression.
Figure 8. Success on a curated curriculum can exceed success on normally generated worlds. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Compare each method across the left Replay panel and the right DR panel, using the legend below both. Replay means levels supplied by the curriculum buffer; DR means levels from the normal procedural generator. The two swapping-based ACCEL methods rise higher on replay levels, but that ranking does not carry over to the DR panel. Interpret the vertical quantity as reward following the visible axis, Reward (% of Max), and the preceding paragraph’s average-return description. The original caption instead calls the quantity average episode length. That conflict is preserved explicitly; the image does not support extracting episode lengths. e-shifte-ued-methode-ued-resultse-ued-config
What it supports. The paired plots demonstrate that the source of evaluation levels matters. Higher replay reward can reflect how a curriculum rearranges access to resources, rather than a generally stronger policy. The paper’s separate fixed-level evaluation ranks PLR highest, while these panels diagnose a gap between curriculum and normal-generator experience.
Where the evidence stops. The caption/axis disagreement remains unresolved. These are distribution-conditioned training diagnostics, not the fixed twenty-level evaluation of Figure 7. The authors’ easier-level explanation is plausible analysis, not a causal result established by this plot alone.
7. Analysis & limitations
7.1 What the evidence leaves open
Symbolic observations remove much visual representation learning. These experiments do not establish pixel-based control, physical execution or indefinite improvement beyond the finite achievement inventory. e-interfacee-rewardse-protocol
Recurrent PPO trades overworld zombie rewards for underground achievements. E3B improves plant eating while lowering aggregate reward. Return alone therefore incompletely describes exploration coverage. e-diagnostic
Discrepancies remain: the text says 65 achievements but Table 6 lists 67. Appendix D gives a 100,000-step cap, whereas G.1 says 4096 instead of a full 10,000. Figure 8 plots reward according to its axis/prose, although its caption says episode length. e-rewardse-horizone-shift
The authors suggest intrinsic rewards distract from sufficiently dense extrinsic reward. The comparisons and limited 1B tuning do not isolate that explanation; UED comparisons also differ in update count and settings. e-maine-traininge-ued-resultse-ued-config
7.2 Questions for discussion
- Would reward gains survive per-floor achievement reporting and equal-update curriculum controls?
- Which dynamics require memory once policy capacity and training budget are controlled?
8. Reproducibility audit
8.1 Requirements and known gaps
Reconstruction requires matching Craftax, PureJaxRL/JaxUED, observation/reward settings and separate Tables 7–14 configurations. The paper links code and specifies hardware but does not pin an executable revision or package versions. Resolve achievement-count and episode-cap discrepancies before comparing scores. e-reproe-rewardse-horizon
Proposed check: compare full and optimistic resets with matched policies, logging overflow, duplicated worlds, throughput and achievements; correlate terminations to challenge the binomial assumption. This has not been run. e-resets
Proposed check: compare DR, PLR and robust PLR at equal interactions, then equal gradient-update budgets, with common evaluation levels and aligned learning rates. A disappearing advantage would implicate update exposure. This has not been run. e-ued-resultse-ued-config
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Stress-test optimistic resets under synchronized terminations
Reader-proposed, not run: reproduce the Appendix C example with 1024 workers, comparing a fresh reset candidate for every worker against a 64-world pool. Hold policy checkpoints, observation settings and sampled action streams fixed where possible. Measure throughput, overflow frequency, duplicate starting states and per-achievement returns across seeds. Include ordinary episodes and a deliberately synchronized-termination stress condition; the latter is a robustness test, not a benchmark-score comparison. The paper’s benign-overflow argument would be supported by negligible duplication and unchanged learning under ordinary episodes. Frequent overflow or systematic return changes under ordinary operation would challenge it; stress-condition failure would delimit the independence assumption. e-resetse-interfacee-rewards
Check 2: Disentangle level curation from gradient-update exposure
Reader-proposed, not run: compare DR, PLR and robust PLR on the same twenty normally generated evaluation levels over ten training seeds. First preserve the paper’s equal-interaction protocol and Table 14 settings. Then add controlled comparisons with a common optimizer schedule and matched gradient-update counts, recording the additional interactions required by each method. Keep architecture and observation encoding fixed, and report reward together with dungeon and gnomish-mines success. If PLR’s advantage disappears after aligning updates and learning rates, the original ranking would not isolate a curation benefit. If it persists on fixed levels while replay-only gains disappear, curation would have stronger support. e-ued-resultse-ued-confige-diagnostic
8.3 Reading coverage
Visual audit: The title/author page, interface and baseline descriptions, timing hardware, reset argument, reward specification, exploration and UED results, all cited appendix configurations, and all six final crops were visually inspected. Figure 5 was inspected at higher crop resolution to verify its changed color palette. Figure 8 retains the original caption-versus-axis discrepancy; Table 6 and the two horizon descriptions retain their conflicting counts. No neural architecture diagram is supplied; the method visuals show the environment interface and reward definition. Figures 9–18 and Tables 2–3 were text-read but are outside this visual pass. Code and external supplements were not inspected.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 12, 14, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 30, 31, 32, 33, 34. Appendix coverage: reviewed.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Abstract; Sections 1–2: motivation and background
- Section 3: Classic, Craftax mechanics, interface and evaluation
- Section 4: exploration/UED baselines and 1B/1M experiments
- Sections 5–6; acknowledgements, impact statement and references
- Appendices A–D: implementation, timing, resets, environment and interfaces
- Appendices E–F: tuning, all achievement results, 10B and Classic
- Appendix G: UED implementation, qualitative results and hyperparameters; all supplied tables through p. 34
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Separate supplemental material availability has not been fully verified.
- All eight supplied text chunks were read individually. Separate supplemental material availability has not been fully verified; no separate supplement was supplied.
- Text extraction does not reconstruct figure images. Retained claims were checked against original PDF pages and six final crops. Figures 9–18 and Tables 2–3 were read through supplied text but not visually inspected.
- No code, external project site or other external source was inspected; no experiments were reproduced.
- Identity/version: the observed title exactly matches the catalog and all seven authors agree. The title page uses Matthew Jackson and Jakob Foerster where the catalog expands them to Matthew Thomas Jackson and Jakob Nicolaus Foerster. Catalog BibTeX includes Michael T. Matthews; this title page says Michael Matthews. No added author or title change was observed. The artifact identifies ICML 2024/PMLR 235; no finer revision date or revision chain was established.
- The same PDF calls its 10B appendix condition Craftax-Extended. This label is retained without asserting a separately verified software edition.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e-identityPDF p. 1, title, author block, affiliation and proceedings footnotes
The matching title credits Michael Matthews, Michael Beukman, Benjamin Ellis, Mikayel Samvelyan, Matthew Jackson, Samuel Coward and Jakob Foerster. Affiliations are University of Oxford and University College London. The artifact identifies ICML 2024, PMLR 235; no finer revision identifier is printed here.
Go to primary source ↓e-corePDF pp. 1–3, Abstract, Sections 1, 2.1–2.2 and 3.1
A JAX rewrite of Crafter and a harder Craftax benchmark combine accelerator-based collection and training. Symbolic observations prioritize exploration research over pixel representation learning.
Go to primary source ↓e-mechanicsPDF pp. 3–4, Section 3.2, Multiple Floors through Boss Floor
Nine procedurally generated floors connected by ladders, expanded combat, per-episode potion-effect permutations, experience allocation and a final boss are intended to challenge exploration, memory and adaptation.
Go to primary source ↓e-interfacePDF pp. 4–5, Section 3.3; p. 17, Appendix D.4–D.5; p. 22, Figure 25; pp. 30–31, Tables 4–5
Reported symbolic observation sizes are 1345 for Classic and 8268 for Craftax; action counts are 17 and 43. Craftax observes a 9-by-11 map area, categorical block/item/creature features, light and inventory. Cells below light 0.05 are masked. Unsupported actions consume a timestep as no-ops. Figure 25 depicts optional pixel interfaces.
Go to primary source ↓e-rewardsPDF p. 5, Section 3.3, Reward; p. 6, Section 4.3; p. 32, Table 6
Achievements pay once per episode, with category rewards 1/3/5/8 and damage/recovery terms of minus/plus 0.1 per point. Section 3.3 states 65 achievements, but Table 6 lists IDs 0–66. Section 4.3 uses maximum achievement reward 226.
Go to primary source ↓e-protocolPDF p. 5, Section 3.4 and Figure 3 caption; p. 7, Figure 6 caption
Craftax-1B and Craftax-1M use symbolic observations and one billion or one million interactions. Exploration curves use ten seeds with one-standard-error shading.
Go to primary source ↓e-baselinesPDF pp. 5–6, Section 4.1; p. 33, Tables 8–10
PureJaxRL PPO uses four-layer width-512 policy/value MLPs; PPO-RNN adds a GRU. RND distills a random network; ICM uses world-model error as intrinsic reward; E3B rewards episodic novelty. Table 8 lists forward and inverse model losses. No inference-time imagined-rollout planner is introduced.
Go to primary source ↓e-trainingPDF p. 17, Appendix E.1–E.2; pp. 33–34, Tables 7–13
The 1B PPO sweep tunes parameters individually on one seed. Selected settings include 1024 workers, 64 steps, eight minibatches, four epochs and learning rate 0.0002. PPO-RNN inherits PPO settings and width-sized recurrent state. The 1M random-search baseline uses 256 workers, 16 steps and learning rate 0.0003. Both anneal learning rate.
Go to primary source ↓e-classic-differencesPDF p. 12, Appendix A.1–A.2
Classic uses fixed creature/plant capacities and inactive-entity masks, and replaces Crafter Simplex noise with Perlin noise.
Go to primary source ↓e-speedPDF p. 12, Appendix B; p. 14, Table 1, all rows; p. 2, Figure 2 caption
Best-case throughput is 405618 steps/s for Classic and 266961 for Craftax at 4096 workers, versus 1580 for Crafter at 1024. Hardware is one RTX 4090 with 24 GB VRAM, i9-13900K and 32 GB RAM. Learning pipelines are measured, except MineRL uses random actions; implementations and architectures are not identical.
Go to primary source ↓e-resetsPDF p. 14, Appendix C
Generate M fresh worlds for N workers with M much smaller than N, using one world per sixteen workers. Overflow can duplicate initial states. The illustrative binomial calculation uses 1024 workers, mean episode length 200 and 64 new states; an approximately twofold speedup is reported.
Go to primary source ↓e-mainPDF p. 5, Figure 3; p. 6, Figure 4 and Section 4.3
At 1B interactions PPO-RNN is near 15–16% of maximum reward, PPO/ICM/RND near 12%, and E3B near 11%. These are approximate visual readings, not tabulated endpoints. Figure 4 shows no progress in Advanced or Very Advanced categories.
Go to primary source ↓e-diagnosticPDF pp. 6–7, Section 4.3 and Figure 5; p. 18, Figure 19
PPO-RNN enters the dungeon more frequently while zombie completion declines, interpreted by the authors as a tradeoff between overworld and underground rewards. Gnomish-mines entry remains rare. E3B improves EAT_PLANT despite lower overall reward. Figure 5 uses a different palette from Figure 3.
Go to primary source ↓e-smallPDF p. 7, Section 4.4 and Figure 6; p. 19, Figure 20
The 1M methods have similar returns, approximately 2% of maximum by visual reading. Achievement curves show mostly basic progress and isolated dungeon entries.
Go to primary source ↓e-longPDF p. 19, Appendix F.2; p. 20, Figure 21; p. 21, Figure 22
The 10B appendix condition is called Craftax-Extended. Four PPO-RNN seeds reach roughly 17% reward, with no gnomish-mines entry. The authors associate late acceleration with the learning-rate schedule.
Go to primary source ↓e-classic-resultPDF pp. 1–2, Abstract and Introduction; p. 21, Appendix F.3 and Figure 23; p. 22, Figure 24
The headline describes about 90% of optimal Classic reward and a 51-minute PPO run. Appendix F.3 and Figures 23–24 explicitly identify PPO-RNN, ten seeds and 1B interactions. Diamonds and eating plants remain unreliable; all achievements are not consistently completed.
Go to primary source ↓e-ued-methodPDF p. 6, Section 4.2; p. 7, Section 4.5 mutation operators
PLR curates levels by estimated regret; robust PLR trains only on replay. ACCEL also mutates levels through noise, unrestricted swaps or restricted swaps. Domain randomization samples normally generated levels.
Go to primary source ↓e-ued-resultsPDF pp. 7–8, Section 4.5 and Figure 7; p. 26, Figure 34
UED evaluates checkpoints on twenty fixed normally generated levels over ten runs. PLR has the highest reward and unrestricted-swap ACCEL the lowest. Robust PLR performs fewer updates at equal interactions. Evaluation uses the normal generator, not a separately shifted test distribution.
Go to primary source ↓e-shiftPDF p. 8, Distribution Shift and Figure 8; pp. 24–25, Figures 32–33
Swapping curricula show higher replay reward than reward on normally generated levels, with achievement differences such as diamond collection. Figure 8 has a Reward (% of Max) axis and average-return prose, but an average-episode-length caption. This discrepancy remains unresolved.
Go to primary source ↓e-ued-configPDF p. 22, Appendix G.1; p. 23, Appendix G.2–G.3 and Figures 26–31; p. 27, Table 14
UED separates short learning rollouts from longer regret scoring. Table 14 gives 64 inner and 64 outer rollout steps, 1024 environments, and replay rates 0.5 for PLR/ACCEL and 0.8 for robust PLR. ACCEL updates on DR but not mutated levels. Learning rates and mutation counts differ between algorithms.
Go to primary source ↓e-horizonPDF p. 14, opening of Appendix D; p. 22, final sentence of Appendix G.1
Appendix D states a maximum episode length of 100000. Appendix G.1 says UED uses 4096 instead of a full 10000. The descriptions of the full horizon conflict.
Go to primary source ↓e-reproPDF p. 1, code footnote; p. 12, Appendix B; p. 17, Appendix E; p. 27, Table 14; pp. 33–34, Tables 7–13
The paper provides a code link, hardware and hyperparameters, but no pinned executable revision or package-version environment in these reproducibility details.
Go to primary source ↓8.5 Primary sources
Craftax: A Lightning-Fast Benchmark for Open-Ended Reinforcement Learning ↗
PDF · 14,564 extracted words
Source fingerprint
20805946500a5e63af78d0bcd2322e195f312c59d180f219f06786ba1762df00