FAST: Efficient Action Tokenization for Vision-Language-Action Models
1. Paper overview
In one sentence: Compressing action chunks into quantized frequency coefficients and BPE tokens makes autoregressive robot policies easier to train, but the resulting decoder remains slower than the compared flow-matching policy at inference. e01e02e03e04e13e14e19
| At a glance | What to know |
|---|---|
| Research problem | Author claim Per-dimension, per-timestep binning produces long action-token sequences with highly correlated neighbors. The authors argue that copying recent tokens can yield a weak learning signal without learning the conditioned trajectory. Their cubic-spline case study increases sampling from 25 to 800 timesteps while retaining the underlying curves: naive-token prediction deteriorates, whereas DCT-based prediction remains accurate. e02 |
| Core mechanism | |
| A key reported result | Generalist VLA training efficiency: Authors report 5× fewer GPU hours with comparable overall performance to diffusion π0; Figure 11 supplies means and 95% confidence intervals, without printed numerical means. Training GPU hours and task-specific evaluation scores. π0 cross-embodied mixture: 903M internal timesteps, with open datasets comprising 9.1% of the mixture. Task protocols differ; laundry includes fine-tuning. Figure 1 tracks bussing and shirt-folding checkpoints. Figure 15 compares four tasks against compute-matched diffusion π0, with higher FAST average but a lower toaster-task bar. This is an overall training-efficiency result, not uniform task dominance or wholly zero-shot laundry performance. e14e19 |
| Reading caution | Source description FAST+ compression tests cover unseen datasets and additional morphologies, but executed policies were tested on static manipulators. Offline compression on hands, humanoids or mobile platforms does not establish policy success there. FAST+ training also includes data from the real-robot policy-evaluation tasks. e05e11e15 |
Core contributions
Figure 4. FAST concentrates temporal information before learning a discrete compression dictionary. Original paper, p. 5 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Follow the top row left to right, then the long flattening arrow back to the lower-left panel. The first traces are normalized action channels. DCT changes their representation to frequency coefficients; quantization creates the many zeros in panel 3. Each matrix row is an action dimension. Panel 4 reads down the first column before moving to the next frequency: 124, −86, 344, −45 and 178 therefore precede 12. This agrees with the frequency-first ordering in Section V-B. Finally, panel 5 groups adjacent coefficient values into BPE tokens. Those token IDs, rather than the original scalar actions, become policy prediction targets. e03e04e09
What it supports. The two compression stages do different jobs. Quantization trades away small coefficient detail, while BPE losslessly shortens the resulting integer sequence. The source learns only the BPE dictionary within the tokenizer; the DCT does not require a trained neural encoder. This distinction explains both the simplicity and the reconstruction tradeoff.
Where the evidence stops. Reverse decoding cannot undo rounding loss. Algorithm 1 also inconsistently omits quantization bars on later entries of its flattening line; Figure 4 and Section V-B clearly describe flattening the quantized matrix. The diagram's arrows agree with that sequence.
2. Motivation
2.1 The problem and the proposed response
Per-dimension, per-timestep binning produces long action-token sequences with highly correlated neighbors. The authors argue that copying recent tokens can yield a weak learning signal without learning the conditioned trajectory. Their cubic-spline case study increases sampling from 25 to 800 timesteps while retaining the underlying curves: naive-token prediction deteriorates, whereas DCT-based prediction remains accurate. e02
2.2 What this reading follows
A robot moving smoothly at high frequency can produce many action samples that say almost the same thing. FAST changes the prediction target: instead of asking a transformer to emit every sampled action coordinate, it asks for a compact encoding of the whole action chunk. The encoding combines an analytical cosine transform with quantization and a learned BPE dictionary. This reading follows the supplied January 2025 arXiv v1 from that mechanism to real manipulation results. The central distinctions are between compression and reconstruction fidelity, tokenizer transfer and policy transfer, and training efficiency and deployment latency. Appendix qualifications, including laundry fine-tuning, remain part of the result. e01e02e03e04e13e14e19
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | Foundational work |
| Architecture | Not applicable |
| Prediction paradigm | Not applicable |
| Quadrant | Not applicable |
This table preserves the labels recorded at reading time. The current major category is Components of WAMs. View the current classification.
3.1 Evidence-based assessment
Conflicts with the recorded classification
The foundational action-policy placement is supported, and the world-action architecture, prediction-paradigm and quadrant fields appropriately remain not applicable: FAST predicts action tokens without future-world prediction or inverse dynamics. The narrower ‘Latent action pretraining’ tag is questionable if it means inferring latent actions from videos: FAST learns a BPE vocabulary from recorded action trajectories. This is a local classification disagreement, not uncertainty about the inspected architecture. e02e04e05e06
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Predict the shape of a chunk before its small temporal changes
Start from the paper's observation-conditioned action chunk. With naive binning, increasing the sampling rate lengthens the token stream even when the physical trajectory is unchanged. The spline experiment exposes this issue without changing the underlying curves. FAST first changes basis: a smooth trajectory can concentrate much of its information in a few frequency coefficients. Quantization removes insignificant detail, and BPE makes the sparse coefficient sequence short. The flattening order then presents the lowest-frequency terms across all action dimensions before finer components, which the authors say stabilizes rollouts. Reader interpretation: the representation reorganizes what each next-token decision must explain. It does not predict a future camera image or simulate the consequences of candidate actions; the final decoded object is still a sequence of robot commands. e02e03e04e06
5.2 Separate learning the tokenizer from learning the controller
FAST contains a learned dictionary, but that dictionary is not itself a robot policy. Its BPE training observes sequences of quantized action coefficients and merges recurring patterns. FAST+ performs this step on a diverse collection of approximately one million one-second chunks, with the appendix describing the mixture and action padding. Policy training then uses the resulting action vocabulary as output targets for a pretrained vision-language backbone. Camera encodings, language and proprioception condition the transformer, and all policy weights are fine-tuned. At deployment, autoregressive tokens are decoded back into actions. This separation matters when reading transfer claims: offline compression on a new morphology tests the tokenizer, while successful control on that morphology would additionally require a trained and evaluated policy. The paper explicitly leaves several such policy tests for future work. e04e05e06e07e11e15e16
5.3 Follow the efficiency claim all the way to robot execution
The policy results establish that compressed targets can improve learning, but three quantities must remain separate: tokens per action chunk, compute needed to train a useful model, and time required to generate a new chunk. Table I addresses the first; the generalist convergence comparison addresses the second. The inference discussion gives the counterweight: FAST takes approximately 750 milliseconds per one-second chunk on the stated GPU, whereas diffusion π0 typically takes at most 100 milliseconds. DROID executes a selected portion of the predicted chunk open-loop, so generation delay and execution horizon affect opportunities to incorporate new observations. Reader interpretation: a deployment comparison should measure feedback timing alongside task score. The static manipulation experiments support the reported setting; they do not settle how the same tradeoff behaves when objects or targets move rapidly. e08e09e13e14e15
5.4 Training and inference
During training
The tokenizer learns only its BPE vocabulary. Policy training replaces little-used VLM tokens with action tokens and fine-tunes all weights using next-token prediction. The main backbone is PaliGemma-3B; the OpenVLA comparison uses Prismatic 7B. e04e06
The policy encodes two or three 224×224 images separately and concatenates their tokens. Language and proprioception enter as text; proprioceptive values are first discretized into 256 bins. Training uses a 1,000-step warm-up, then learning rate 5×10⁻⁵, AdamW with b1=0.9 and b2=0.95, no weight decay, gradient clipping at 1, and EMA weight 0.999. e06e07
During inference
Action tokens are predicted autoregressively and decoded into low-level commands. Decoding is greedy except on the three bimanual tasks, where temperature β=0.7 helps escape stationary initial behavior. There is no visual rollout, candidate-trajectory scoring or inverse-dynamics action extraction. e04e06e07
DROID predicts 15-step chunks of joint velocity and absolute gripper position, executing either 8 or 15 steps open-loop. Observation conditioning operates between action predictions; continuous feedback during each open-loop chunk is not established. e08
5.5 Implementation flow
- Normalize each action channel
Map the training distribution's 1st and 99th percentiles to −1 and 1. This aligns scales across action channels and limits outlier influence; it is not a claim that all samples fall inside that interval. e03
- Transform and quantize
Apply DCT independently along time for each action dimension. Scale and round coefficients, making many small components zero. The scale controls the compression–reconstruction tradeoff. e03e04e17
- Order and compress
Flatten by frequency first: emit the lowest-frequency coefficient across action dimensions before advancing to the next frequency. BPE merges common coefficient sequences and repeated zeros. Single-dataset experiments use rounding scale 10 and vocabulary size 1024; these defaults should not be silently assumed to specify every FAST+ configuration. e04
- Decode actions
Reverse BPE and coefficient layout, undo scaling, apply inverse DCT and reverse normalization. This reconstructs an approximation to the original action chunk: rounding is lossy even though BPE itself is lossless. e03e04e09
6. Experiments & results
FAST makes continuous robot action chunks easier to learn with next-token prediction by converting them into quantized frequency coefficients and compressing those coefficients with byte-pair encoding. Its main benefit is efficient policy training on smooth, high-frequency action data. The paper demonstrates executed manipulation skills and reports faster training than a flow-matching VLA, while acknowledging substantially slower autoregressive inference. FAST+ reuses a tokenizer learned across embodiments; this is action representation learning, with no predicted visual world state.
6.1 Read the original evidence
Table I. For one-second chunks, FAST's token count grows much less than naive timestep-wise binning. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the action dimension and control frequency before comparing the two token-count columns. Every row uses a one-second chunk, so naive token counts grow with both the number of channels and the number of timesteps. The green column is FAST's average token count; the last column reports naive count divided by FAST count. Shirt Fold is the clearest example: 14 channels sampled at 50 Hz require 700 naive tokens, versus 53 FAST tokens. DROID moves from 105 to 29. Section VI-B states that these default configurations have comparable reconstruction accuracy, which is essential context for interpreting the reductions. e09
What it supports. The table supports a large representation-size advantage on the tested high-frequency data: 13.2× for Shirt Fold and 5.0× for Bussing, compared with 1.75× for BridgeV2. It is consistent with removing temporal redundancy rather than retaining one token for every sampled coordinate, though the rows also differ in task and embodiment.
Where the evidence stops. This is a token-count comparison, not a timing benchmark or a policy-success table. Comparable reconstruction accuracy is stated in the prose; this table does not print reconstruction errors or uncertainty on the average counts.
Figure 6. Compressed action targets improve executed-policy performance, especially on the harder high-frequency manipulation tasks. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Use the legend to distinguish naive binning, FSQ, dataset-specific FAST and universal FAST+. First read the metric printed above each task: LIBERO and shirt folding use success, while DROID and bussing use task progress. The source caption identifies the error bars as 95% confidence intervals. Then compare methods within a task. The near-zero naive bars on bussing and shirt folding contrast with substantial FAST performance; FSQ improves but remains below the FAST variants. The two green bars are often close, showing that a shared tokenizer can support policy training comparably to a tokenizer fitted separately for the dataset. e05e10e18
What it supports. The evidence goes beyond reconstructing trajectories: these are policy evaluations, including real robot execution. The strongest qualitative separation is on bussing and shirt folding. FAST+ tracks the dataset-specific variant across the plotted tasks, supporting a reusable tokenizer without establishing identical performance in every condition.
Where the evidence stops. The average panel is labeled success, although its inputs mix success and progress. It is not a pooled binary success rate. FAST+ training includes data from the real-robot evaluation tasks; this comparison does not establish tokenizer transfer to wholly unseen task datasets.
Figure 1, upper training-convergence panel. FAST reaches strong manipulation scores earlier in training; the headline GPU-hour claim needs the main-text protocol. Original paper, p. 1 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Track the green and yellow checkpoint curves against the horizontal training-iteration axis. The vertical axis is evaluation score, not training loss. Section VI-F explains that this plot averages table bussing and T-shirt folding; it is not the complete five-task generalist evaluation. The arrow above the curves carries the paper's fivefold training-speed claim. To interpret it, read the accompanying Section VI-F statement about GPU hours, rather than treating the plotted iteration axis as elapsed time. Figure 11 gives the wider task comparison, and Appendix Figure 15 adds a compute-matched control whose advantage is strongest on some tasks rather than universal. e13e14e19
What it supports. The authors report a fivefold reduction in training GPU hours while maintaining comparable overall generalist performance. This supports efficient training of an autoregressive VLA from compressed actions. It does not imply that every task improves: the inspected compute-matched plot still favors diffusion on the toaster task.
Where the evidence stops. Training iterations and GPU hours are different quantities; the fivefold number comes from the source's compute claim. FAST inference is approximately 750 ms per chunk versus diffusion's typical ≤100 ms. Appendix E also specifies laundry fine-tuning despite broader zero-shot wording.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| Action-chunk compression One-second dataset chunks; default tokenizers at source-reported comparable reconstruction accuracy. | BridgeV2: 20 tokens, 1.75×; DROID: 29, 3.6×; Bussing: 28, 5.0×; Shirt Fold: 53, 13.2×. Average tokens per chunk; naive/FAST compression ratio | Naive counts are respectively 35, 105, 140 and 700. Control frequencies are 5, 15, 20 and 50 Hz; dimensions are 7, 7, 7 and 14. Compression grows particularly strongly in the high-frequency bimanual example. These ratios measure token count, not robot speed or success. e09 |
| Policy learning with alternative tokenizers LIBERO simulation plus DROID, table bussing and T-shirt folding; Figure 6 reports means and 95% confidence intervals. | FAST and FAST+ outperform naive tokenization; naive policies make no progress on bussing and shirt folding. Exact bar values are not tabulated. Task-specific success or task progress | FSQ improves over naive tokenization but trails FAST most clearly on the dexterous real-robot tasks. Executed-policy evidence supports the tokenization choice. The displayed average combines progress and success metrics and is not a pooled binary success rate. e10e18 |
| BPE removal ablation Table bussing and T-shirt folding; DCT-based tokenizer with versus without BPE. | Removing BPE lowers both plotted scores; numerical means are not printed. Bussing task progress; shirt-folding success | DCT without BPE still improves over naive tokenization according to Section VI-D. The result supports compressing redundant coefficients, but changes sequence length and decoding burden together; it does not isolate one causal explanation. e12 |
| Generalist VLA training efficiency π0 cross-embodied mixture: 903M internal timesteps, with open datasets comprising 9.1% of the mixture. Task protocols differ; laundry includes fine-tuning. | Authors report 5× fewer GPU hours with comparable overall performance to diffusion π0; Figure 11 supplies means and 95% confidence intervals, without printed numerical means. Training GPU hours and task-specific evaluation scores | Figure 1 tracks bussing and shirt-folding checkpoints. Figure 15 compares four tasks against compute-matched diffusion π0, with higher FAST average but a lower toaster-task bar. This is an overall training-efficiency result, not uniform task dominance or wholly zero-shot laundry performance. e14e19 |
| Action-chunk inference latency One-second action chunks on an NVIDIA 4090 GPU in Section VI-E. | Approximately 750 ms for π0 with FAST. Inference time per chunk | Diffusion π0 typically predicts within 100 ms; FAST uses about 30–60 autoregressive tokens versus 10 diffusion steps. Training efficiency does not imply low deployment latency. The authors found the delay acceptable for their static manipulation evaluations. e13e15 |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Figure 12. Compression should be judged at the reconstruction fidelity the controller needs. Original paper, p. 16 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Each panel plots reconstruction error vertically against the number of output tokens horizontally, both on logarithmic scales. At a fixed token budget, lower is better; at a fixed error level, farther left is better. Green curves sweep FAST's rounding scale, yellow curves vary FSQ's latent-token count, and blue curves vary naive temporal subsampling. Black stars mark the fixed FAST+ tokenizer, while black dots mark naive tokenization without subsampling. Vocabulary size is held constant across tokenizers. Compare curves within each dataset: the point is how the tradeoff changes as greater precision is requested, not which dataset has the smallest absolute error. e17
What it supports. FAST is not uniformly the best choice at every fidelity. The caption acknowledges that learned quantization can be more efficient at coarse reconstruction, while FAST scales better toward high fidelity. This helps explain why a tokenizer that looks adequate under a loose reconstruction budget can still be weak for fine-grained manipulation.
Where the evidence stops. These are offline reconstruction measurements, not robot rollouts. The caption does not define the statistical meaning of the green shaded band, so it should not be read as a specified confidence interval. No exact curve values are inferred here.
Section VI-D, unnumbered BPE ablation plot. Keeping DCT while removing BPE weakens downstream control on both tested tasks. Original paper, p. 9 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. The green bars use the complete FAST pipeline; yellow removes its last BPE compression step. This leaves the frequency transform in place, so it probes what happens after information has already been concentrated into coefficients. Read the two groups separately: the left score measures bussing progress and the right measures successful shirt folding. Both decline without BPE, with a particularly large visual gap on folding. Section VI-D explains that a quantized DCT matrix can still contain many repeated zeros; BPE removes much of this sequential redundancy. The plot therefore complements the pipeline figure by testing the utility of its final stage. e04e12
What it supports. DCT alone is useful according to the accompanying text, but the full tokenizer gives better rollout performance. A sparse coefficient matrix is not automatically a short autoregressive sequence. BPE converts that sparsity into fewer predicted tokens, making its role relevant to both learning and decoding.
Where the evidence stops. Removing BPE changes sequence length, token statistics and decoding cost together. The experiment does not isolate which change causes the gain. This small plot does not print exact means or define its error bars; numerical values and confidence levels are not assigned here.
7. Analysis & limitations
7.1 What the evidence leaves open
FAST+ compression tests cover unseen datasets and additional morphologies, but executed policies were tested on static manipulators. Offline compression on hands, humanoids or mobile platforms does not establish policy success there. FAST+ training also includes data from the real-robot policy-evaluation tasks. e05e11e15
Protocol inconsistencies remain: Section VI-F broadly says zero-shot, but Appendix E specifies laundry fine-tuning. DROID prose says 16 tasks while Table II lists 17 rows totaling 44 trials. The main text names LIBERO-10 where the appendix names LIBERO-Long; DROID's appendix also prints ‘≈3 episodes’ where Appendix D specifies three epochs. These are not silently reconciled. e08e18e19
Three-campus DROID tests are qualitative and have no measured success rates. Several rollout plots show broad uncertainty, and exact means are unavailable in tables. The paper attributes improved DROID language following to the autoregressive variant but leaves a controlled investigation for future work. e10e13e18
7.2 Questions for discussion
- At matched reconstruction error, how much of FAST's benefit comes from shorter sequences versus frequency-first ordering?
- Would FAST's training advantage survive a deployment comparison that fixes feedback delay as well as model compute?
- How does tokenizer transfer change when every evaluation embodiment is excluded from BPE training?
8. Reproducibility audit
8.1 Requirements and known gaps
A DROID reproduction requires the 75k successful episodes, removal of all-zero idle actions, randomized external views and language annotations, and the stated action parameterization. The source reports 240k iterations, batch size 256, approximately four days on eight H100 GPUs. These are author-reported requirements, not a run performed here. e08
Preserve normalization statistics, DCT conventions, frequency-first layout and the BPE dictionary. Universal-tokenizer training pads actions to 32 dimensions and uses the Appendix A mixture. The paper does not fully specify DCT normalization, coefficient-to-BPE serialization, all software versions, or the quantity of laundry fine-tuning data; the printed release link cannot resolve those gaps within this reading. e03e04e05e16e19
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Does BPE help after reconstruction fidelity is held fixed?
Reader-proposed check: repeat table-bussing policy training with identical normalized chunks, quantized DCT coefficients, frequency-first order, backbone, data splits and initialization seeds. Compare BPE encoding against the uncompressed coefficient sequence, verifying identical decoded reconstruction before training. Evaluate both at matched optimizer updates and matched GPU hours; record token count, rollout progress and chunk-generation latency separately. A gain confined to the compute-matched comparison would favor an efficiency explanation, while a persistent update-matched gain would motivate investigating token statistics. Failure to reproduce either rollout advantage would weaken the interpretation of the BPE ablation. No such experiment was run here. e03e04e09e12e17e18
Check 2: Does deployment delay change the FAST–diffusion comparison?
Reader-proposed check: evaluate FAST and diffusion policies in matched DROID scenes using the same instructions, observations, action parameterization and explicit task-progress rubric. First reconcile the prose's 16 tasks with Table II's 17 listed tasks, retaining its 44-trial allocation. Compare 8-step and 15-step open-loop execution, recording actual observation-to-command delay. Add a control that delays the faster decoder to match the slower decoder's measured delay. If a performance gap disappears under matched delay, timing is a plausible contributor; if it remains, investigate policy quality and language grounding separately. Report per-task uncertainty and failure modes. This is a proposed test, not a reproduction claim. e08e13e18
8.3 Reading coverage
Visual audit: Visually inspected the title/version/author page, all Figures 1–15, both unnumbered Section VI-D ablation plots, Algorithm 1, Tables I–III and the Appendix A mixture table. Method and training prose on pages 3–7 and 16–17, result protocols on pages 7–10 and 17–18, and limitations on page 11 were checked against the text. All six final original crops were individually viewed with axes, legends and table headers retained. The narrow BPE crop was rendered at 600 DPI and is readable at its native 826-pixel width. Figure 4's arrows and column-first ordering agree with Section V-B; Algorithm 1's inconsistent quantization bars are disclosed. Figure 6's mixed metrics, Figure 1's iteration-versus-GPU-hour distinction and the appendix protocol inconsistencies are preserved. Reference-only pages 12–15 were read as text; external videos, code and separate supplements were not inspected.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 16, 17, 18, 19. Appendix coverage: reviewed.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Title, authors, affiliations, version and abstract (p. 1)
- I. Introduction (pp. 1–2)
- II. Related Work (pp. 2–3)
- III. Preliminaries (p. 3)
- IV. Case Study: How Does Tokenization Affect VLA Training? (pp. 3–4)
- V. Efficient Action Tokenization via Time-Series Compression, A–C (pp. 4–6)
- VI. Experiments, A–F (pp. 6–10)
- VII. Discussion and Future Work (pp. 10–11)
- Acknowledgements and References (pp. 11–15)
- Appendix A. Data Mixture for Training Universal Tokenizer (p. 16)
- Appendix B. Trading off Between Compression and Reconstruction (p. 16)
- Appendix C. Policy Training (pp. 16–17)
- Appendix D. DROID Policy Setup (p. 17)
- Appendix E. Evaluation Tasks and Training Datasets, including Tables II–III (pp. 17–19)
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Separate supplemental material availability has not been fully verified.
- All eight supplied text chunks were read individually, covering all 19 PDF pages. The title page identifies arXiv:2501.09747v1 [cs.RO], 16 January 2025; the title and all nine authors agree with the catalog. No different revision or edition was supplied or compared.
- Text extraction does not reconstruct figure images; this was addressed by inspecting the retained PDF on pages 1–11 and 16–19, including all scientific figures and tables. Reference-only pages 12–15 were read as text.
- Separate supplemental material availability has not been fully verified. No separate supplement was supplied.
- Linked code, model artifacts, external videos, datasets and cited papers were not inspected. No experiments were reproduced. Code examples printed in the paper were read only.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e01PDF p. 1, title block and arXiv margin
Title matches the supplied observed title. The nine authors are Karl Pertsch, Kyle Stachowicz, Brian Ichter, Danny Driess, Suraj Nair, Quan Vuong, Oier Mees, Chelsea Finn and Sergey Levine. Affiliations are Physical Intelligence, UC Berkeley and Stanford; version is 2501.09747v1, 16 January 2025.
Go to primary source ↓e02PDF pp. 2–4, Introduction; Section III; Section IV and Figure 3
Defines observation-conditioned action chunks, variable-length action tokenization and naive per-timestep binning. The spline experiment samples the same underlying curves at 25–800 timesteps and contrasts rising naive prediction error with stable DCT prediction. The authors explain this through redundant next-token targets.
Go to primary source ↓e03PDF p. 4, Sections V-A and V-B
Actions are normalized using the 1st and 99th quantiles, transformed independently per dimension with DCT, then scaled and rounded. Small coefficients become zero; scaling trades fidelity against compression.
Go to primary source ↓e04PDF p. 5, Figure 4, Algorithm 1, Sections V-B and V-C
The diagram and prose agree on frequency-first flattening across dimensions, followed by BPE. BPE is the only learned tokenizer component and is lossless over quantized coefficients. Single-dataset defaults are scale 10 and vocabulary 1024. The source describes reverse decoding, while quantization prevents lossless recovery of original actions.
Go to primary source ↓e05PDF pp. 5–6, Section V-C; p. 7, Section VI-A, Comparisons
FAST+ uses approximately one million one-second action chunks from diverse embodiments and action spaces. The paper describes a pretrained tokenizer release. Its training mixture includes data from real-robot policy-evaluation tasks; it is separate from the held-out compression test described later.
Go to primary source ↓e06PDF p. 6, Section VI-A, Policy implementation; p. 16, Appendix C
Most policies use π0 with PaliGemma-3B, with additional OpenVLA/Prismatic-7B experiments. Action tokens overwrite little-used vocabulary entries; all weights are fine-tuned. Separate camera encodings are concatenated, with text instructions and binned proprioception as conditioning.
Go to primary source ↓e07PDF pp. 16–17, Appendix C, Policy Training
Specifies 224×224 images, 256 proprioceptive bins, 1k-step warm-up, learning rate 5e-5, AdamW b1=0.9/b2=0.95 without weight decay, gradient clip 1 and EMA 0.999. Inference is greedy except bimanual tasks, which use temperature β=0.7.
Go to primary source ↓e08PDF p. 17, Appendix D, DROID Policy Setup
DROID uses randomized external views and language annotations, a wrist view, no camera calibration, joint velocity and absolute gripper position, 15 predicted steps and 8 or 15 executed open-loop steps. It retains 75k successful episodes, removes all-zero idle actions, and trains three epochs/240k iterations with batch 256, taking about four days on eight H100 GPUs.
Go to primary source ↓e09PDF p. 7, Table I, all rows and Avg. Token/Compression columns; Section VI-B
At comparable reconstruction accuracy, naive/FAST average token counts for one-second chunks are BridgeV2 35/20, DROID 105/29, Bussing 140/28 and Shirt Fold 700/53, with ratios 1.75, 3.6, 5.0 and 13.2. Frequencies are 5/15/20/50 Hz and action dimensions 7/7/7/14.
Go to primary source ↓e10PDF pp. 7–8, Section VI-B and Figure 6
Naive, FSQ, FAST and FAST+ are compared on LIBERO, DROID, bussing and shirt folding. Naive policies fail to progress on the latter two; FAST exceeds FSQ most visibly there. Figure 6 reports mean and 95% CI, but mixes success and task-progress panels and provides no exact numeric means.
Go to primary source ↓e11PDF pp. 8–9, Section VI-C and Figure 8; p. 19, Table III
The paper describes a separate unseen-dataset compression test spanning single arms, dexterous hands, UMI, humanoids and navigation. Table III enumerates platforms, action spaces, dimensions and control frequencies. These measurements concern token compression, not executed policy performance on all listed morphologies.
Go to primary source ↓e12PDF p. 9, Section VI-D and its two unnumbered plots
OpenVLA is adapted to multiple images and one-second chunks; FAST+ improves its shirt folding. Removing BPE reduces bussing progress and folding success while retaining DCT; the authors attribute this to redundant zero tokens and longer decoding. Exact means and an uncertainty definition for these small plots are not printed.
Go to primary source ↓e13PDF p. 9, Section VI-E and Figure 9
Reports faster convergence on bussing and stronger DROID language following, with the latter left for detailed future study. On NVIDIA 4090, diffusion π0 typically generates a one-second chunk within 100 ms versus approximately 750 ms for FAST, using 10 diffusion steps versus 30–60 autoregressive tokens and a smaller action expert versus the full language backbone.
Go to primary source ↓e14PDF p. 1, Figure 1 upper plot; p. 10, Section VI-F and Figure 11; p. 18, Figure 15
Generalist training uses 903M internal timesteps plus a mixture with 9.1% open-source data. Authors report comparable overall performance with fivefold fewer GPU hours. Figure 1 has a training-iteration axis and averages bussing and shirt folding; Figure 11 shows five task outcomes with 95% CIs. Figure 15's four-task compute-matched comparison favors FAST on average but diffusion on the toaster task.
Go to primary source ↓e15PDF p. 11, Section VII, Action tokenizers and Inference speed
Executed-policy experiments concern static manipulators. Testing policies on mobile robots, hands and humanoids remains future work despite offline compression evidence. Autoregressive latency motivates further work on dynamic tasks.
Go to primary source ↓e16PDF p. 16, Appendix A and unnumbered mixture table
The FAST+ mixture includes joint, end-effector world-frame and camera-frame representations and lists morphology/frequency/weight combinations. Actions are padded to 32 dimensions for tokenizer training; public datasets are included in their original form.
Go to primary source ↓e17PDF p. 16, Appendix B, Figure 12 and caption
Six panels compare reconstruction error against token count at fixed vocabulary size. Sweeps vary FAST rounding scale, naive subsampling frequency and FSQ latent-token count. FAST scales well toward high fidelity, while FSQ can be more efficient at low fidelity; FAST+ is shown as fixed star markers. The shaded band's statistical meaning is not defined in the caption.
Go to primary source ↓e18PDF p. 6, Section VI-A, LIBERO; pp. 17–18, Appendix E; p. 18, Table II and Figure 14
Appendix E defines task-specific scoring: binary LIBERO episodes, correctly sorted bussing objects, human-rated shirt folding and rubric-based DROID progress. DROID has 44 quantitative trials; prose says 16 tasks, but Table II has 17 task rows. Campus tests have no measured success rates. Main text calls the final LIBERO suite Libero-10, appendix Libero-Long. Appendix E prints approximately three episodes while Appendix D says three epochs.
Go to primary source ↓e19PDF p. 10, Section VI-F; p. 18, Appendix E, Grocery Bagging, Toast out of Toaster and Laundry Folding
The main generalist section broadly describes zero-shot evaluation. The appendix specifies out-of-the-box grocery/toaster evaluation but fine-tuning on a small amount of high-quality task-specific data for laundry. Laundry is scored by successful folding and stacking; the fine-tuning data quantity is not given.
Go to primary source ↓8.5 Primary sources
FAST: Efficient Action Tokenization for Vision-Language-Action Models ↗
PDF · 12,714 extracted words
Source fingerprint
3739b31f5fecdde371509ff5bb13619979734e894a255a9b264253f4cc53934a