Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success
1. Paper overview
In one sentence: OpenVLA-OFT turns an autoregressive VLA into a fast continuous action-chunk policy, while OFT+ uses language-conditioned vision to improve instruction following in bimanual tasks. problemarchitectureaction-objectivesfilmlibero-successlibero-speed
| At a glance | What to know |
|---|---|
| Research problem | Source description The paper asks how to adapt an autoregressive VLA to new robots when its original token-by-token recipe is too slow and unreliable for dexterous bimanual control. OpenVLA supplies a common pretrained backbone so decoding strategy, action representation, and objective can be compared during offline imitation learning. problemparallel |
| Core mechanism | Source description OFT combines parallel decoding, action chunking, continuous actions, and L1 regression. Its central empirical contribution is a staged comparison showing that a simple regression objective can match diffusion on the tested LIBERO demonstrations. action-objectiveslibero-success |
| A key reported result | LIBERO four-suite success with additional inputs: 97.1 average; Spatial 97.6, Object 98.4, Goal 97.9, Long 94.5. Mean task success rate (%). Separate suite policies; filtered training data; third-person and wrist images, proprioception, instruction; 500 evaluation episodes per suite. π0: 94.2 average in the same table block; original single-image OpenVLA: 76.5 in a different block. The 2.9 percentage-point lead over π0 is a reported benchmark comparison, not a matched-pretraining experiment. Comparing 97.1 with 76.5 additionally changes inputs. libero-protocollibero-success |
| Reading caution | Source description The authors test focused demonstrations with consistent strategies. L1 may fail on genuinely multimodal action distributions; suitability for large-scale pretraining remains untested. The cause of weak language grounding in ALOHA but not LIBERO is unresolved. limitations |
Core contributions
- Source description
OFT combines parallel decoding, action chunking, continuous actions, and L1 regression. Its central empirical contribution is a staged comparison showing that a simple regression objective can match diffusion on the tested LIBERO demonstrations. action-objectiveslibero-success
- Source description
OFT+ adds language-conditioned FiLM inside both vision transformers, enabling adaptation to three-camera ALOHA control and substantially improving target selection in two language-dependent tasks. architecturefilmlanguage-results
Figure 1. One integrated policy converts current observations into a bimanual action chunk. Original paper, p. 2 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start at the lower left: the third-person, left-wrist, and right-wrist images pass through the shared SigLIP–DINOv2 vision backbone. The pink input represents current joint state. The task description has two roles: it supplies language embeddings to the decoder and, through the green FiLM arrow, conditions visual feature extraction. On the right, yellow empty action embeddings mark output positions. Read the purple outputs across the top as 25 successive timesteps, each containing 14 action coordinates. Parallel decoding fills those positions in one pass. Appendix A supplies the continuous MLP head that this overview compresses into the decoder output path. architectureparallelaloha-setuptraining-alohatraining-libero
What it supports. The architecture explains how OFT+ accommodates more cameras and a different robot action space without introducing a second learned controller. Its predicted outputs are executable joint targets. Visual and language features support those predictions; the diagram contains no future-image decoder or explicit world-state rollout.
Where the evidence stops. This figure shows the ALOHA OFT+ configuration. Main LIBERO OFT uses a different action space, an eight-step chunk, and no FiLM. The robot executes a chunk open-loop before the next observation-conditioned prediction.
2. Motivation
2.1 The problem and the proposed response
The paper asks how to adapt an autoregressive VLA to new robots when its original token-by-token recipe is too slow and unreliable for dexterous bimanual control. OpenVLA supplies a common pretrained backbone so decoding strategy, action representation, and objective can be compared during offline imitation learning. problemparallel
2.2 What this reading follows
Adapting a robot foundation model changes more than its weights: the way actions are represented and decoded can determine whether the policy is usable on a new robot. This paper tests that premise with OpenVLA. Its optimized recipe replaces sequential action tokens with parallel continuous predictions and a simple L1 loss. The ALOHA extension adds FiLM so instructions influence visual features before action decoding. Read the evidence along two axes: what improves manipulation quality, and what reduces the time needed to produce a chunk. The strongest LIBERO success number uses extra inputs, while the largest quoted LIBERO speedup uses a single image. problemarchitectureaction-objectivesfilmlibero-successlibero-speed
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | VLA |
| Architecture | One Model |
| Prediction paradigm | Not applicable |
| Quadrant | Not applicable |
3.1 Evidence-based assessment
Supports the recorded classification
The recorded VLA and One Model classification fits a single integrated vision-language decoder with action head and optional FiLM, directly producing low-level actions. Multiple encoders are components of that policy, not a separate predictive world model. The architecture predicts neither future observations nor actions inferred from predicted futures, supporting Not applicable for the world/action prediction paradigm and quadrant. architectureparallelaction-objectives
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Replace sequential action generation without inventing a future world
Original OpenVLA treats action coordinates like successive language tokens. During training, shifted ground-truth action tokens provide the previous-token context; at inference, predicted tokens supply that context. A longer action chunk therefore multiplies sequential decoder work. OFT changes the interface: action positions begin as empty embeddings with positional information, and bidirectional attention computes their hidden states together. A continuous head then turns those states into normalized actions. This is still imitation learning from observed expert actions, not prediction of future camera frames followed by planning. Reader interpretation: the main efficiency gain comes from reorganizing the policy computation around a whole control segment. The resulting architecture supports the catalog's One Model VLA classification, while its lack of predicted world states leaves the world/action prediction quadrant inapplicable. parallelaction-objectivesarchitecture
Figure 8. FiLM makes visual processing depend on the instruction before action decoding. Original paper, p. 15 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Follow the numbered steps on the left. Language token embeddings are averaged into a sentence representation, then learned projections produce gamma and beta for each transformer block. These vectors have the visual transformer's hidden width, so each coefficient affects its corresponding hidden dimension across all image patches. Now follow the central block from bottom to top: normalized features pass through self-attention and its residual addition, then FiLM, then normalization and the feedforward network. The equation uses a scale of one plus gamma and an additive beta. The right-hand green arrow summarizes the language-to-vision direction used in both constituent vision transformers. filmlanguage-resultsmultisuite
What it supports. The identity offset matters during adaptation: near-zero gamma and beta initially leave pretrained visual activations approximately unchanged. Language conditioning can then develop through training. The paper reports stronger ALOHA language following with this design, while the combined-suite LIBERO experiment shows only a small difference, 96.8% without FiLM versus 97.0% with it.
Where the evidence stops. Step 4's prose says multiplication by gamma, but the graphic's equations, Section IV-C, and Appendix C specify one plus gamma. The formula governs this explanation. The reported patch-wise alternative lacks a quantitative comparison in the PDF.
5.2 Read L1's success as conditional on the demonstration distribution
The objective comparison becomes informative after fixing the parallel chunked interface. In that setting, continuous L1 reaches 95.3% average LIBERO success and diffusion reaches 95.4%. The deployment difference remains large because diffusion repeatedly refines noisy actions, whereas the L1 head produces them directly. Reducing diffusion's test steps improves speed but eventually damages task performance. These findings support a simpler fine-tuning recipe for the datasets tested. They do not establish that regression represents every useful action distribution. The authors explicitly describe demonstrations with consistent strategies and leave genuinely multimodal behavior unresolved. Reader interpretation: the discriminating future test would place multiple valid, incompatible action sequences behind similar observations, then ask whether a deterministic regression policy remains executable rather than merely having low prediction error. libero-successlibero-speedaction-objectiveslimitations
5.3 Separate instruction selection, motor execution, and feedback
ALOHA makes three requirements visible. First, the policy must identify which target the instruction specifies. FiLM addresses this by conditioning visual features throughout the vision backbone. Second, it must complete the physical stages after selecting that target; the pot task's 79.2% target-approach rate coexists with only 51.25 completion points. Third, the controller must receive actions at a useful cadence. OFT+ supplies 25 actions per query and the robot executes the entire chunk before replanning. At the stated 25 Hz controller rate, that chunk represents one second of commanded motion, calculated from the reported settings. Reader interpretation: high action throughput makes such segments practical but does not give the model continuous visual correction inside them. A disturbance-recovery test would therefore probe a different property from the paper's query-speed measurement. filmlanguage-resultsaloha-rubricstraining-alohaaloha-setupaloha-speed
5.4 Training and inference
During training
LIBERO has four suites with 10 tasks and 500 demonstrations each. Policies are normally fine-tuned separately by suite; modified datasets remove unsuccessful demonstrations and near-zero actions. The full OFT configuration uses rank-32 LoRA, batch 64, eight A100/H100 80GB GPUs, and learning rate 5×10⁻⁴. Spatial, Object, and Long train for 150K steps with a tenfold decay after 100K; Goal uses 50K steps. Checkpoints are evaluated every 50K steps and the best is reported. libero-protocoltraining-libero
ALOHA tasks contain 20/30/45/300 demonstrations; appendix training/validation splits are 19/1, 29/1, 42/3, and 285/15. OFT+ uses rank-32 LoRA, batch 32, eight A100/H100 80GB GPUs, and task-specific 100K/70K/50K/100K steps. The stated convergence target is normalized L1 loss below 0.01; image augmentation uses 90% crops and color jitter. aloha-setuptraining-aloha
During inference
LIBERO executes all eight predicted actions before obtaining a new chunk. ALOHA executes all 25 at a 25 Hz controller rate. Observation feedback therefore changes the policy output between chunks, with open-loop execution inside each chunk; the model predicts actions rather than future images or world states. paralleltraining-liberotraining-alohaaloha-setup
ALOHA OFT+ generates 77.9 actions/s with 0.321 s chunk latency on an A100 over 100 queries, versus original OpenVLA's 1.8 actions/s and 0.543 s for single actions. Other tested policies are faster; π0 uses JAX while the others use PyTorch. Throughput is not the robot's feedback frequency. aloha-speedtraining-aloha
5.5 Implementation flow
- Fuse observations
The shared SigLIP–DINOv2 backbone extracts and fuses patch features for each camera. A three-layer GELU projector maps them to the Llama-2 embedding space; a two-layer projector encodes proprioception. Visual, language, and state embeddings enter the same decoder sequence. architecture
- Replace autoregression
Empty action embeddings, distinguished by position, replace shifted ground-truth action-token inputs. Bidirectional attention predicts all action coordinates together. With chunk length K and action dimensionality D, this replaces KD sequential decoder passes with one pass over the chunk's output positions. parallel
- Regress continuous actions
A four-layer ReLU MLP maps final decoder hidden states directly to normalized continuous actions. Training minimizes mean absolute prediction error. The comparison diffusion variant predicts added noise and uses iterative DDIM denoising; this variant is an alternative to the selected OFT recipe. action-objectivesarchitecture
- Condition vision with FiLM
OFT+ averages task-language embeddings and uses block-specific affine projections to produce scale and shift vectors. Each vector acts on hidden dimensions across every patch, after self-attention and before the feedforward sublayer, in both vision transformers. film
6. Experiments & results
OpenVLA-OFT adapts a pretrained robot policy by predicting continuous action chunks in one decoder pass and training with L1 regression. OFT+ additionally conditions visual features on language through FiLM. The experiments support faster action generation and strong task performance under focused demonstrations, while leaving multimodal behavior and broad pretraining unresolved. The headline 97.1% LIBERO success and 26× throughput describe different input configurations (evidence: libero-success, libero-speed).
6.1 Read the original evidence
Table I, single-image and filtered-data block. With the same input block, parallel chunks and continuous regression raise average success to 95.3%. Original paper, p. 6 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the banner before comparing numbers: all rows in this crop use a third-person image and an instruction, and the table groups them under modified training data. The four suite columns test spatial, object, goal, and long-horizon behavior. Focus on the OpenVLA rows: the original recipe is followed by parallel decoding plus action chunking (PD&AC), then continuous diffusion and continuous L1. The strongest mechanistic comparison is between these variants of the same base model. The caption specifies 500 evaluation episodes per suite for this work's OpenVLA results; several other baseline rows are imported from earlier reports. libero-successlibero-protocol
What it supports. Average success rises from 76.5% to 90.2% with PD&AC, then to 95.3% with L1. Diffusion reaches 95.4%, a nearly identical reported average. The Long column improves especially strongly, from 53.7% to 90.7% for L1, motivating the paper's emphasis on temporally extended action prediction.
Where the evidence stops. PD and chunking change together, so this comparison cannot assign the gain to either alone. The headline 97.1% belongs to the additional-input block outside this crop. No reported uncertainty interval establishes statistical equivalence between 95.3% and 95.4%.
Table II. Generating more actions per call improves throughput without making every action a fresh feedback decision. Original paper, p. 6 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Follow the first four rows to separate parallel decoding from chunking. PD alone reduces latency because the decoder no longer generates each action coordinate sequentially. Adding AC increases the output horizon to eight timesteps: throughput jumps even though chunk latency rises slightly. The diffusion rows hold training diffusion steps at 50 while changing test steps; inspect the success column alongside speed. Finally, the bottom row adds a wrist image and proprioception, changing the input workload. Measurements average 100 A100 queries. The arrow beside throughput points upward for better performance, while the latency arrow points downward; latency is explicitly in seconds. libero-speedparallelsource-unitstraining-libero
What it supports. Single-image L1 OFT generates 109.7 actions/s at 0.0729 s per chunk, versus original OpenVLA's 4.2 actions/s. Expanded inputs reduce throughput to 71.4 actions/s. Diffusion can approach L1 throughput with one denoising step, but the corresponding Long success collapses to 0.0%, exposing a speed–quality tradeoff.
Where the evidence stops. These are model-query measurements, not closed-loop robot frequencies. Related Work prints analogous latency values in milliseconds; Tables II–III specify seconds. This edition follows the tables and does not combine the 26× single-image speedup with the additional-input configuration.
Figure 4. Strong folding and scooping coexist with much lower performance on object placement into a pot. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Use the legend to track OFT+ in red and the strongest average baseline, π0, in green. The boxed group on the left averages the four task scores; the remaining groups reveal where that average comes from. Demonstration totals appear under task names, but Appendix F reserves part of each dataset for validation. All methods saturate the shorts task. The scoop and pot groups are more discriminating. Read the vertical axis as completion score: Appendix F and Tables X–XIII award cumulative points for stages, with penalties on selected tasks. The plot's rounded values should not be interpreted as counts of fully successful trials. aloha-scorealoha-rubricsaloha-setupinitial-statesbaseline-protocol
What it supports. OFT+ averages 87.8 points, compared with 83.9 for π0 and 72.3 for ACT. It reaches 100 on folding and scooping, but its exact pot-task total is 51.25. The strongest-baseline gap is approximately four points; the larger roughly fifteen-point gap concerns ACT.
Where the evidence stops. Pot results pool 12 in-distribution and 12 distractor trials. The reported figure does not separate these outcomes. Baselines also differ in pretraining, architecture, and training configuration, so their ranking does not isolate the effect of the L1 objective.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| LIBERO four-suite success with additional inputs Separate suite policies; filtered training data; third-person and wrist images, proprioception, instruction; 500 evaluation episodes per suite. | 97.1 average; Spatial 97.6, Object 98.4, Goal 97.9, Long 94.5. Mean task success rate (%) | π0: 94.2 average in the same table block; original single-image OpenVLA: 76.5 in a different block. The 2.9 percentage-point lead over π0 is a reported benchmark comparison, not a matched-pretraining experiment. Comparing 97.1 with 76.5 additionally changes inputs. libero-protocollibero-success |
| LIBERO controlled fine-tuning design comparison Filtered data, one third-person image and instruction; chunked variants predict and execute eight steps. | Discrete PD+AC: 90.2; continuous L1: 95.3; continuous diffusion: 95.4. Four-suite average success rate (%) | Original OpenVLA: 76.5. PD+AC gives a calculated 13.7-point gain; L1 adds 5.1 points. PD and chunking are changed jointly, so their individual success contributions are not isolated. No uncertainty intervals establish equivalence of L1 and diffusion. libero-success |
| LIBERO action-generation efficiency 100 A100 queries; one 224×224 image, instruction, 7-D actions; K=8 for chunked variants. | L1 OFT: 109.7 Hz and 0.0729 s; expanded inputs: 71.4 Hz and 0.1120 s. Action throughput (Hz); chunk latency (seconds) | Original: 4.2 Hz, 0.2396 s. Diffusion with 50/10/5 test steps: 4.2/19.3/35.1 Hz. The roughly 26× throughput comparison applies to single-image OFT. Reducing diffusion to one test step gives 109.4 Hz but 0.0% LIBERO-Long success. libero-speed |
| ALOHA four-task completion Separate task policies; 10 shorts, 10 shirt, 12 scoop, and 24 pot trials; pot combines 12 in-distribution and 12 distractor trials. | 87.8 overall; exact task totals 100, 100, 100, and 51.25. Average staged completion score, out of 100 | Figure 4 averages: π0 83.9, RDT-1B 78.4, Diffusion Policy 77.5, ACT 72.3. These are rubric scores with partial credit and penalties, not binary success rates. OFT+ leads the strongest baseline by about four score points, with substantial remaining failures on the pot task. aloha-setupaloha-scorealoha-rubricsinitial-states |
| ALOHA language-following FiLM ablation Approach the instructed target in scoop (12 trials) and pot (24 trials). | OFT+: 100.0 scoop, 79.2 pot, 89.6 task-average. Target-approach success rate (%) | Without FiLM: 33.3 for each task. FiLM improves instruction-dependent selection in this setup; these scores do not measure completed manipulation. language-resultsaloha-setup |
| LIBERO robot-pretraining ablation Full additional-input OFT versus direct fine-tuning from the underlying pretrained Prismatic VLM. | 97.1 with robot pretraining. Four-suite average success rate (%) | 91.9 without OpenVLA robot pretraining. Robot pretraining remains beneficial by 5.2 points. The table's 'scratch' label does not mean randomly initialized visual and language backbones. pretraining-ablation |
| Scaling fine-tuning to BridgeData V2 50,365 WidowX demonstrations; selected original OpenVLA evaluation tasks; 10 trials per task; no FiLM. | 69.2. Average original-suite task score | Public OpenVLA checkpoint: 65.8. OFT improves the average but regresses on several rows. Bridge data was already included in base-model pretraining, so this is not evidence of transfer to an unseen dataset. bridge |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Figure 5. FiLM improves approaching the instructed target, an intermediate requirement for successful manipulation. Original paper, p. 9 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. First distinguish this plot from the completion figure: success here means approaching the object or ingredient named by the instruction. Compare the pale-purple no-FiLM bars with the red OFT+ bars within each task. Both tasks offer three possible instructed targets, making 33.3% the reference level for a random choice. The scoop trials and pot trials have different counts, but the boxed average is the mean of task rates. Check the other VLA bars as context: several methods select the scoop target perfectly while still obtaining lower completion scores. Correct selection is necessary for these tasks, but it does not guarantee grasping, pouring, or placement. language-resultsaloha-setupaloha-rubricstraining-alohalimitationsmultisuite
What it supports. Removing FiLM drops both task rates to 33.3%. OFT+ reaches 100.0% on scoop and 79.2% on pot, averaging 89.6%. This is direct ablation evidence that the added conditioning helps language-dependent target approach in the tested ALOHA setup, complementing the mechanism diagram.
Where the evidence stops. The ablation also changes model capacity: Table V attributes 456M trainable parameters to FiLM projectors. It does not by itself isolate language conditioning from added capacity, nor resolve why the same problem is much weaker in LIBERO.
7. Analysis & limitations
7.1 What the evidence leaves open
The authors test focused demonstrations with consistent strategies. L1 may fail on genuinely multimodal action distributions; suitability for large-scale pretraining remains untested. The cause of weak language grounding in ALOHA but not LIBERO is unresolved. limitations
Comparisons mix pretrained models, training budgets, observation histories, implementations, and checkpoint-selection procedures. Small ALOHA trial counts and absent uncertainty intervals limit claims about statistical reliability or general superiority of regression. libero-protocollibero-successaloha-setupbaseline-protocolaloha-speed
Source inconsistency: Related Work labels approximately 0.07 and 0.321 latency values in milliseconds, whereas Tables II–III explicitly give seconds. This report follows the tables. Main-text ALOHA training's broad step range is also less precise than Table V's task-specific settings, which are retained here. source-unitstraining-aloha
7.2 Questions for discussion
- Would matched multimodal demonstrations reveal a task-success advantage for diffusion that focused demonstrations conceal?
- Does FiLM's advantage persist after matching added parameter count and balancing camera cues across instructions?
- How does shortening open-loop execution affect recovery from disturbances at fixed inference hardware?
8. Reproducibility audit
8.1 Requirements and known gaps
Reconstruction requires the pretrained OpenVLA artifact, matching filtered LIBERO data, camera/state projections, parallel attention and empty embeddings, action normalization, and the appendix schedules. Full LIBERO OFT trains 279M parameters; ALOHA OFT+ trains 853M, including 456M FiLM parameters. Added conditioning is therefore a material training cost. architectureparallellibero-successtraining-liberotraining-aloha
The PDF specifies hardware, augmentation, splits, rubrics, and selected checkpoints but does not provide a complete pinned software environment, random-seed protocol, or standalone numerical near-zero-action filtering threshold. Bridge scoring refers back to the original OpenVLA suite. These details require confirmation before a faithful reproduction. libero-successtraining-liberotraining-alohabaseline-protocolaloha-rubricsbridge
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Stress the L1–diffusion comparison with incompatible valid trajectories
Reader-proposed check, not performed: fine-tune matched OpenVLA backbones with the same inputs, eight-action chunks, data split, LoRA rank, and checkpoint-selection rule. Compare L1 with diffusion at 50, 10, and 5 test steps. Use a consistent-strategy control dataset and a matched-size dataset in which identical starting scenes admit two distinct collision-free routes. Balance route frequencies. Measure executed task success, invalid intermediate trajectories, latency, and variation across seeds. A diffusion advantage confined to the two-route condition would support the authors' multimodality caveat; no reliable interaction would weaken that explanation. action-objectiveslibero-speedtraining-liberolimitations
Check 2: Separate FiLM's language signal from its added capacity
Reader-proposed check, not performed: repeat the two ALOHA language tasks with no FiLM, true-instruction FiLM, and an equal-parameter FiLM branch receiving a constant instruction embedding. Match demonstrations, training budget, action head, initialization protocol, and camera inputs. Balance target instructions within the repeated initial layouts and evaluate held-out layouts with and without the pot distractors separately. Record target approach, full completion, and rubric score across repeated training runs. If true-language FiLM exceeds both controls while the constant-input branch does not, the evidence favors instruction conditioning over added capacity alone. filmlanguage-resultstraining-alohaaloha-setupinitial-statesaloha-rubrics
8.3 Reading coverage
Visual audit: Inspected the title/author/version page, all original Figures 1–13 and Tables I–XVI, method and limitation pages, appendix training configurations, evaluation layouts, rubrics, and additional experiments. All six final crops were individually viewed; Table I is deliberately limited to its complete single-image block, and the additional-input results were read on the full page. Figure 8's arrows, residual path, and identity-offset equation were checked against Sections IV-C and Appendix C. References on pages 11–13 were read as text, not rendered. Separate supplements, linked code, and rollout videos were not inspected. This edition covers the supplied v2 PDF, not an independently compared earlier version.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24. Appendix coverage: reviewed.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Abstract; I Introduction; II Related Work; III Preliminaries
- IV Studying Key VLA Fine-Tuning Design Decisions, A–C
- V LIBERO experiments, A–F
- VI ALOHA experiments, A–D; VII Discussion; VIII Limitations
- Acknowledgments and References
- Appendix A Architecture; B Implementation; C FiLM; D OFT Training; E Baseline Training
- Appendix F ALOHA Evaluation; G Additional Experiments; Tables IV–XVI
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Identity/version scope: the inspected title page identifies arXiv:2502.19645v2, 28 April 2025, with the exact catalog title and authors Moo Jin Kim, Chelsea Finn, and Percy Liang, all Stanford University. The catalog submission date is 27 February 2025; that earlier edition was not supplied, and revision-by-revision content differences cannot be established.
- All nine supplied text chunks were read, covering the complete 24-page PDF and appendix. Reference-only pages 11–13 were read as text; pages 1–10 and 14–24 were also visually inspected.
- The acquisition notes that text extraction does not reconstruct figure images; the retained PDF was inspected to resolve figures, equations, and tables.
- Separate supplemental material availability has not been fully verified.
- Linked code, checkpoints, project pages, and rollout videos were not inspected. No training, installation, or robot experiments were reproduced.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
identityPDF p. 1, title, author block, affiliation footnote, and arXiv margin stamp
Exact supplied title; Moo Jin Kim, Chelsea Finn, Percy Liang; Stanford University; arXiv:2502.19645v2, 28 April 2025.
Go to primary source ↓problemPDF pp. 1–3, Abstract and Sections I–III
Study of adapting pretrained OpenVLA through offline imitation learning; original autoregressive decoding limits high-frequency bimanual control.
Go to primary source ↓architecturePDF p. 2, Figure 1; p. 4, Section IV-B; p. 14, Appendix A and B3
Shared SigLIP–DINOv2 image encoding, Llama-2 decoder, visual/state projectors, continuous action head, multiple cameras, and optional FiLM form one integrated policy.
Go to primary source ↓parallelPDF p. 3, Figure 2 and Section III; p. 4, Section IV-B; p. 14, Appendix B1
Parallel decoding replaces teacher-forced action inputs with empty positional embeddings and causal masking with bidirectional attention; KD action coordinates are predicted in one pass.
Go to primary source ↓action-objectivesPDF pp. 3–4, Sections IV-A/B; p. 14, Appendix B2
Discrete 256-bin action prediction is compared with continuous L1 regression and diffusion; four-layer MLP heads, DDIM, and the squared-cosine schedule are specified.
Go to primary source ↓filmPDF pp. 4–5, Section IV-C and unnumbered FiLM equation; p. 14, Appendix C; p. 15, Figure 8
Average language embeddings produce block-specific gamma/beta vectors shared across patches; modulation uses (1+gamma) times F plus beta in both vision transformers. Figure 8 Step 4 wording omits the identity offset that its equations retain.
Go to primary source ↓libero-protocolPDF p. 5, Section V-A; p. 6, Table I caption
Four suites each supply 500 demonstrations over 10 tasks; separate fine-tuning, periodic best-checkpoint selection, full eight-action execution, and 500 evaluation episodes per suite.
Go to primary source ↓libero-successPDF p. 6, Table I, all three input/data blocks and caption
Single-image averages are 76.5 original, 90.2 PD+AC, 95.4 continuous diffusion, 95.3 continuous L1. Additional-input filtered-data OFT averages 97.1 versus π0 94.2; unfiltered OFT averages 94.5. Baselines and data protocols differ across blocks.
Go to primary source ↓libero-speedPDF p. 6, Table II and caption; p. 7, Section V-D
A100 100-query timings: original 4.2 Hz/0.2396 s, PD 15.9/0.0629, PD+AC 108.8/0.0735, L1 109.7/0.0729, expanded inputs 71.4/0.1120. Diffusion's 50/10/5/2/1 steps yield Long success 91.1/91.0/90.0/85.7/0.0.
Go to primary source ↓aloha-setupPDF pp. 7–8, Section VI-A; pp. 15–16, Appendix F1
Two-arm ALOHA operates at 25 Hz with three views and joint state/targets. Shorts/shirt/scoop/pot data total 20/30/45/300, with splits 19+1/29+1/42+3/285+15; evaluations use 10/10/12/24 trials.
Go to primary source ↓aloha-scorePDF p. 8, Figure 4 and Section VI-C
The graph reports average percent-completion scores: OFT+ 87.8, π0 83.9, RDT-1B 78.4, Diffusion Policy 77.5, ACT 72.3. The metric includes partial credit.
Go to primary source ↓language-resultsPDF p. 9, Figure 5, caption, and FiLM ablation paragraph
Target-approach rates for OFT+ are 100.0 scoop and 79.2 pot; no-FiLM gives 33.3 each. OFT+ task-average is 89.6. These are language-following metrics.
Go to primary source ↓aloha-speedPDF pp. 9–10, Section VI-D; p. 10, Table III and caption
A100 100-query measurements give OFT+ 77.9 Hz/0.321 s and original OpenVLA 1.8 Hz/0.543 s. Other methods are faster; chunk sizes and software implementations differ.
Go to primary source ↓limitationsPDF p. 10, Section VIII
Authors leave multimodal demonstrations, OFT pretraining applicability, and the cause of inconsistent cross-platform language grounding open.
Go to primary source ↓training-liberoPDF p. 15, Appendix D; p. 17, Table IV
Full OFT uses rank-32 LoRA, eight A100/H100 80GB GPUs, batch 64, learning rate 5e-4, 279M trainable parameters, eight-action chunks, no history/FiLM, stated task schedules, 90% crop/color augmentation, and an L1 convergence criterion below 0.01.
Go to primary source ↓training-alohaPDF p. 8, Section VI-A; p. 15, Appendix D; p. 17, Table V
OFT+ uses rank-32 LoRA, batch 32, eight A100/H100 80GB GPUs, 25-action chunks, FiLM and three views, 853M trainable parameters including 456M FiLM parameters. Table V gives 100K/70K/50K/100K steps and tenfold decay after 50K where applicable.
Go to primary source ↓aloha-rubricsPDF p. 23, Tables X–XIII, stage headers, totals and captions
Cumulative stage scores include shirt and scoop penalties. OFT+ totals are 100/100/100/51.25; no-FiLM scoop/pot totals are 35.00/31.67. Pot results pool in-distribution and out-of-distribution trials.
Go to primary source ↓initial-statesPDF pp. 19–22, Figures 9–13 and captions
Initial-state grids specify cloth, bowl, spoon and object placement variation; pot out-of-distribution trials add unseen green and orange bowls as distractors, with repeated layouts across instructions.
Go to primary source ↓pretraining-ablationPDF p. 16, Appendix G3; p. 23, Table XV
Ablation removes OpenVLA robot pretraining but retains the underlying pretrained Prismatic VLM; average success drops from 97.1 to 91.9.
Go to primary source ↓multisuitePDF p. 16, Appendix G1–G2; p. 23, Table XIV
One policy across all four suites averages 96.8 without FiLM and 97.0 with FiLM, versus separate-suite policies at 97.1.
Go to primary source ↓bridgePDF p. 16, Appendix G4 and footnote; p. 24, Table XVI and caption
Fine-tuning on 50,365 BridgeData V2 demonstrations gives 69.2 average score versus OpenVLA 65.8 on a selected evaluation subset, 10 trials per task. Base OpenVLA had already seen Bridge data; individual task scores both rise and fall.
Go to primary source ↓baseline-protocolPDF pp. 8, 15, Section VI-B and Appendix E; pp. 18–19, Tables VI–IX
Baseline architectures, language conditioning, training schedules, history, samplers and execution horizons differ. RDT selects an earlier scoop checkpoint; baseline configurations do not constitute a compute-matched objective-only comparison.
Go to primary source ↓source-unitsPDF p. 2, Section II, final paragraph of right column; p. 6, Table II; p. 10, Table III
Related Work prints 0.07 ms and 0.321 ms, but the timing tables label their corresponding latency values in seconds; report values follow the explicit tables.
Go to primary source ↓8.5 Primary sources
Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success ↗
PDF · 14,033 extracted words
Source fingerprint
b860aa1206b6cfb0ce8be177f961379dd6a133d52cc74ac346636e0f4952a596