OpenVLA: An Open-Source Vision-Language-Action Model
1. Paper overview
In one sentence: OpenVLA adapts a pretrained vision-language model into a direct action-token policy, gaining broad manipulation ability while remaining sensitive to training diversity, scoring rules and controller timing. e-architecturee-actionse-datae-bridge-rubrice-efficiencye-blockinge-libero-results
| At a glance | What to know |
|---|---|
| Research problem | Source description How can Internet-pretrained visual and language knowledge become an accessible, adaptable manipulation policy? The paper targets generalization beyond narrow demonstrations and the cost of adapting large policies to new robot setups. e-problem |
| Core mechanism | Source description A generalist VLA combines DINOv2, SigLIP and Llama 2 with action tokenization and 970k real-world robot demonstrations. e-architecturee-actionse-data |
| A key reported result | WidowX generalization and language grounding: OpenVLA 70.6 ± 3.2% Mean task score reported as success rate ± standard error; partial credit permitted. Direct evaluation; 17 tasks, 170 rollouts per policy, matched initial states. RT-2-X 50.6 ± 3.5%; Octo 20.0 ± 2.6%; RT-1-X 18.5 ± 2.7%. The 20.0 percentage-point gap over RT-2-X concerns this suite. RT-2-X retains the higher semantic-generalization score. Scores are not uniformly binary completion rates. e-evale-bridgee-bridge-rubric |
| Reading caution | Source description Single-image sensing and slow sequential generation restrict temporal and high-frequency control. Typical success remains below 90%; model scaling and Internet-data co-training remain underexplored. e-limitations |
Core contributions
- Source description
A generalist VLA combines DINOv2, SigLIP and Llama 2 with action tokenization and 970k real-world robot demonstrations. e-architecturee-actionse-data
- Author claim
The authors release weights and a training/adaptation pipeline, and investigate LoRA and quantized serving. Openness applies to the added robot-training stack, not the entire pretraining lineage. e-releasee-probleme-efficiency
Figure 2. A vocabulary interface connects visual-language representations to robot control. Original paper, p. 4 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start at the image on the left and follow its branching arrows into DINOv2 and SigLIP. Section 3.1 specifies channel-wise concatenation of their patch features, followed by the MLP projector; the colored patch tokens then enter Llama 2. The instruction takes a separate path through a task prompt and the Llama tokenizer. Above the language model, purple action-token glyphs feed the de-tokenizer, whose output goes to the robot. These glyphs are schematic: the output is explicitly labeled seven-dimensional. Section 3.2 supplies the training detail absent from the drawing: discrete action tokens receive the next-token cross-entropy loss. e-architecturee-actionse-adaptation-protocole-limitations
What it supports. The key reuse is the multimodal model’s input and output interface. Image features and instruction tokens jointly condition control-token generation, while de-tokenization turns those predictions into robot commands. The learned model is a direct control policy; the figure does not contain a future-image generator, dynamics rollout or planning search.
Where the evidence stops. The diagram is not a complete control-loop timing specification. It shows one image and one action; observation history, proprioception and action chunking are absent from the evaluated OpenVLA interface. See the controller-specific experiments before inferring deployment speed.
2. Motivation
2.1 The problem and the proposed response
How can Internet-pretrained visual and language knowledge become an accessible, adaptable manipulation policy? The paper targets generalization beyond narrow demonstrations and the cost of adapting large policies to new robot setups. e-problem
2.2 What this reading follows
OpenVLA asks whether the machinery that connects images to language can also connect instructions to robot motion. Its answer is a compact interface change: represent continuous controls as vocabulary tokens, then train a pretrained multimodal model on diverse robot demonstrations. This reading follows the information flow, examines what the real-robot comparisons measure, and separates architectural evidence from deployment effects. The appendices are essential: they expose partial-credit scoring, a smaller variant used for efficiency experiments, and a blocking-control test that changes the apparent cost of quantization. They also provide a simulation adaptation study and reveal limits to simple leaderboard summaries. e-architecturee-actionse-datae-bridge-rubrice-efficiencye-blockinge-libero-results
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | VLA |
| Architecture | One Model |
| Prediction paradigm | Not applicable |
| Quadrant | Not applicable |
3.1 Evidence-based assessment
Supports the recorded classification
VLA / Autoregressive VLA / One Model is supported by the integrated image-and-language-to-action architecture. The two encoders are components of one policy, not separate world and action models. It predicts neither future observations nor inverse dynamics conditioned on a future goal observation; the recorded world-action prediction paradigm and quadrant are appropriately not applicable. e-architecturee-actionse-limitations
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
5. Method in detail
5.1 Turn language prediction into a motor interface
The architectural move is to keep the pretrained multimodal pathway and change what its output tokens mean. Prismatic has already connected image features to a language model. OpenVLA supplies robot demonstrations whose continuous actions are discretized separately by dimension, with quantile-based bounds limiting the effect of extreme values. Those discrete values replace the least-used vocabulary entries, and the learning signal is restricted to action-token prediction. The visual encoder is also adapted, so this is more than attaching a fixed semantic recognizer to a controller. At deployment, generated tokens are decoded into one robot action and the next observation closes the feedback loop. No future scene is generated for a planner to evaluate: the prediction target is the action itself, which explains the autoregressive VLA classification. e-architecturee-actionse-traininge-limitations
5.2 Separate visual knowledge from robot experience
A pretrained model may recognize an eggplant without knowing how this robot should approach it from an unfamiliar start state. The Bridge evaluation deliberately combines changed objects, camera placement and reaching requirements, making motor generalization part of the challenge. The reader’s interpretation of Table 9 is that robot-data coverage matters at least as much as the choice of visual features in this setting: reducing the training mixture creates a much larger aggregate gap than removing DINOv2 within Bridge-only training. Yet those comparisons answer different causal questions. The mixture intervention changes both data amount and diversity, while the encoder intervention is tested in a restricted regime. Likewise, successful target selection can earn partial credit before manipulation succeeds. Reading the scoring rubric prevents recognition, reaching and full task completion from collapsing into one notion of competence. e-bridge-rubrice-ablation
5.3 Treat the controller as part of the experiment
The int8 result is a useful example of why a robot policy cannot be evaluated solely as a neural network. Under non-blocking control, a slower prediction stream changes how commands interact with ongoing movement. The source reports lower int8 throughput on the evaluation GPU and a lower rollout score. Blocking control then makes action completion precede the next prediction, and the precision-dependent score gap largely disappears. This supports a timing explanation, although the changed control regime does not prove that action predictions are numerically identical. The same concern appears in Appendix C: dynamically re-querying RT-2-X introduces interruptions, and removing demonstration no-ops reduces OpenVLA freezing. The reader’s practical deduction is to record preprocessing and command timing alongside weights, precision and task scores when assessing reproducibility. e-quantizatione-blockinge-preprocessing
5.4 Training and inference
During training
Prismatic supplies vision-language initialization. Robot training applies action-token cross-entropy and updates vision alongside the remaining network. The final run uses 27 epochs, learning rate 2e-5, batch 2048, and 64 A100 GPUs for 14 days (reported as 21,500 A100-hours). e-architecturee-actionse-training
The OpenX mixture harmonizes third-person vision and single-arm control. DROID starts at 10% sampling weight but is removed for the final third, with its weight redistributed. Bridge preprocessing drops each demonstration’s first transition to reduce no-op freezing. e-datae-preprocessing
Full downstream fine-tuning uses 10–150 demonstrations per task. LoRA adapts linear layers with low-rank updates; the efficiency study recommends rank 32 and reports 10–15 hours on one A100. That study uses a smaller-mixture, SigLIP-only variant. e-adaptation-protocole-efficiency
During inference
Predict one action, send it to the controller, and repeat with visual feedback. The policy has no observation history, proprioceptive input, action chunk or future-image output. Main-model inference is reported at roughly 6 Hz on RTX 4090. e-architecturee-adaptation-protocole-limitationse-training
Quantization affects timing as well as memory. A5000 int8 inference runs at 1.2 Hz versus about 3 Hz for int4, changing the relationship to the data-collection controller. e-quantization
5.5 Implementation flow
- Encode the current scene
DINOv2 and SigLIP process image patches separately. Channel-concatenated features pass through a two-layer MLP into the language embedding space. These represent the current observation, not predicted future states. e-architecturee-training
- Condition and generate
Llama-tokenized instructions and projected image tokens feed Llama 2. Autoregressive action tokens are de-tokenized into robot controls. Figure 2 depicts a policy-to-controller interface, not learned dynamics rollouts. e-architecturee-actions
- Make actions representable
Each action dimension uses 256 uniform bins between its first and ninety-ninth training-data quantiles. OpenVLA overwrites the last 256 vocabulary tokens because too few new special tokens are reserved. Quantile bounds reduce sensitivity to outliers. e-actions
6. Experiments & results
OpenVLA turns a pretrained vision-language model into a direct robot policy by predicting discretized action tokens from one image and an instruction. Its practical strengths are generalization and adaptation across manipulation settings. The evidence also exposes important boundaries: some scores include partial credit, efficiency tests use a smaller variant, and control latency changes apparent quantization quality.
6.1 Read the original evidence
Figure 3. The overall advantage has a clear exception in semantic generalization. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the legend first: red is OpenVLA, green RT-2-X, orange Octo and blue RT-1-X. The shaded group on the left is the overall average; the five groups to its right distinguish visual, motion, physical and semantic generalization from language grounding. The photographs illustrate tasks rather than additional trials or independent measurements. Compare red and green within each category before looking at the average: the semantic group reverses their usual order. The caption defines the uncertainty as standard error over the evaluation rollouts; the detailed Table 4 and task rubrics explain how those scores were accumulated. e-evale-bridgee-bridge-rubrice-preprocessing
What it supports. Across 170 WidowX rollouts, OpenVLA’s reported average is 70.6%, compared with 50.6% for RT-2-X. The advantage is not universal: semantic generalization is 36.3% versus 38.8%. Language-grounding and visual-generalization gains therefore should not be recast as uniformly stronger Internet-derived semantic knowledge.
Where the evidence stops. Some tasks award half credit merely for approaching the named object, so these bars are not pure completion fractions. The RT-2-X comparison also uses its documented second-most-likely-action workaround; training and inference conditions differ between policies.
Table 7. Task-level adaptation results reveal different strengths behind the aggregate. Original paper, p. 32 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read each in-distribution row together with the OOD row immediately below it. Their unequal trial counts matter: OpenVLA’s 75.0% over twelve Move <object> onto Plate trials and 58.3% over six OOD trials yield approximately 69.4% when weighted, agreeing with Figure 5. The upper block uses Franka-Tabletop; the lower block uses the faster Franka-DROID Wipe Table setup. The two Diffusion Policy columns distinguish its richer interface from the single-image, single-relative-action matched baseline. OpenVLA (scratch) still begins from the pretrained Prismatic vision-language model; it removes large-scale robot pretraining. Compare the shaded environment averages after checking pouring, object selection and towel covering, where the relative strengths change. e-adaptatione-adaptation-protocole-franka-rubrice-adaptation-aggregation
What it supports. Table 7 reports OpenVLA at 67.2 ± 4.0% on Franka-Tabletop versus 48.5 ± 4.9% for Diffusion Policy. On Franka-DROID the corresponding scores are 58.3 ± 7.2% and 35.0 ± 8.0%. These averages favor OpenVLA while individual task rows still favor other methods.
Where the evidence stops. Scores include partial credit, with a separate 0–2 rubric for Wipe Table. Keep trial counts and scoring definitions attached to each average: neither an unweighted mean of the displayed rows nor a strict completion fraction represents the same statistic.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| WidowX generalization and language grounding Direct evaluation; 17 tasks, 170 rollouts per policy, matched initial states. | OpenVLA 70.6 ± 3.2% Mean task score reported as success rate ± standard error; partial credit permitted | RT-2-X 50.6 ± 3.5%; Octo 20.0 ± 2.6%; RT-1-X 18.5 ± 2.7%. The 20.0 percentage-point gap over RT-2-X concerns this suite. RT-2-X retains the higher semantic-generalization score. Scores are not uniformly binary completion rates. e-evale-bridgee-bridge-rubric |
| Google-robot direct manipulation 12 tasks, 60 binary-scored trials per policy, including in-distribution and OOD conditions. | OpenVLA 85.0 ± 4.6% Success rate ± standard error | RT-2-X 78.3 ± 5.4%; Octo 26.7 ± 5.8%. The authors describe the two VLA policies as comparable on this suite. Their standard-error bars overlap; that alone does not establish statistical equivalence. e-evale-google |
| Franka-Tabletop adaptation Six tasks, 99 in-distribution/OOD trials; identical target demonstration sets across methods. | OpenVLA 67.2 ± 4.0% Table 7 mean task score ± standard error | Diffusion Policy 48.5 ± 4.9%; Octo 43.4 ± 4.4%; OpenVLA without OpenX robot pretraining 43.4 ± 4.6%. OpenVLA leads the aggregate, but Diffusion Policy is stronger on pouring and Octo on towel covering. Standard Diffusion Policy has richer inputs and action chunks; a matched baseline is also supplied. e-adaptatione-adaptation-protocole-franka-rubric |
| Franka-DROID Wipe Table adaptation 70 demonstrations; 18 in-distribution and 12 distractor/OOD trials; 15 Hz non-blocking controller. | OpenVLA 58.3 ± 7.2% Normalized 0–2 task score ± standard error | Diffusion Policy 35.0 ± 8.0%; Octo 38.3 ± 8.5%. Partial sweeping earns one point, complete sweeping two. This is not the percentage of fully completed trials. e-adaptatione-franka-rubrice-adaptation-protocol |
| Parameter-efficient adaptation SigLIP-only variant; two Franka tasks with OOD variants, 33 trials; VRAM at batch 16. | LoRA rank 32: 68.2 ± 7.5%, 59.7 GB, 97.6 million parameters Task score ± standard error; training VRAM; trainable parameters | Full fine-tuning: 69.7 ± 7.2%, 163.3 GB sharded across two GPUs. Scores are close with broad uncertainty. A small trainable-parameter count does not imply equally small activation memory. e-efficiencye-efficiency-tasks |
| Robot-data and vision-encoder ablations Eight Bridge tasks, ten trials each; full OpenX versus Bridge-only training, then removal of DINOv2. | OpenVLA 76.3 ± 4.8%; Bridge-only 45.6 ± 5.6%; Bridge-only SigLIP 40.6 ± 5.5% Mean task score ± standard error | Mean gaps: 30.7 percentage points for the mixture change and 5.0 for the encoder change. The encoder effect is conditional on Bridge-only training; it does not establish the same effect at full OpenX scale. e-ablatione-bridge-rubric |
| Quantized inference with non-blocking control SigLIP-only efficiency variant; eight Bridge tasks, 80 trials per precision. | int4: 71.9 ± 4.7%, 7.0 GB Mean task score ± standard error; inference VRAM | bfloat16: 71.3 ± 4.8%, 16.8 GB; int8: 58.1 ± 5.1%, 10.2 GB. The int8 deficit combines precision and slower execution, rather than isolating quantization error. e-quantization |
| Quantized inference with blocking control Appendix D.4 follow-up; eight Bridge tasks, 80 trials per precision; each action completes before the next prediction. | bfloat16 70.0 ± 5.1%; int8 74.4 ± 4.9%; int4 68.8 ± 5.2% Mean task score ± standard error | All reported standard-error bars overlap. The vanished int8 deficit supports the latency explanation. Blocking control is a distinct protocol, not a direct improvement over non-blocking scores. e-blockinge-quantization |
| LIBERO suite adaptation Separate supervised fine-tuning per suite; cleaned demonstrations; static third-person images; three seeds with 500 trials per suite per seed. | OpenVLA LoRA rank 32: 76.5 ± 0.6% Average success rate ± reported standard error across four suites | Octo 75.1 ± 0.6%; Diffusion Policy 72.4 ± 0.7%. OpenVLA leads Spatial and Long, not Object or Goal. This is simulation adaptation, not lifelong learning or physical deployment. e-libero-setupe-libero-results |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Table 1. LoRA reduces trainable parameters substantially, but training memory remains material. Original paper, p. 10 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read across a row to keep success, trainable parameters and memory attached to the same strategy. The parameter column is in millions, and the VRAM column fixes batch size at 16. The retained asterisk explains why full fine-tuning and frozen-vision memory are listed across two GPUs. Compare LoRA rank 32 with full fine-tuning for the main efficiency claim, then compare the two LoRA rows to see what doubling rank changes. Last-layer-only and frozen-vision results show that choosing which weights adapt matters. The source’s nearby footnote is crucial: these experiments use the SigLIP-only variant with the smaller robot-data mixture. e-efficiencye-efficiency-tasks
What it supports. Rank-32 LoRA reports 68.2 ± 7.5% against 69.7 ± 7.2% for full fine-tuning, while training 97.6 million parameters and using 59.7 GB rather than 163.3 GB at the stated batch size. This supports an attractive measured tradeoff, not an uncertainty-free claim of equal performance.
Where the evidence stops. Only two tasks and their OOD variants contribute 33 trials. The memory measurements are configuration-specific, and the tested model differs from the dual-encoder main model; these numbers should not be generalized to every checkpoint or batch size.
Table 9. The training-mixture change has a larger observed effect than removing DINOv2. Original paper, p. 34 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Begin with the model headers, because adjacent comparisons answer different questions. OpenVLA versus OpenVLA-Bridge changes the robot training mixture while retaining the dual visual encoder. OpenVLA-Bridge versus OpenVLA-Bridge-SigLIP retains Bridge-only training while removing DINOv2. Each task has ten trials, and fractional successes reflect the source’s partial-credit rules. Inspect the task rows before the shaded average: the dual encoder does not improve every row, and the language-grounding row does not follow the large aggregate data-mixture gap. This table is therefore a controlled local encoder comparison within one training regime, not a complete factorial study of data and vision. e-ablatione-bridge-rubric
What it supports. The aggregate falls from 76.3 ± 4.8% with OpenX training to 45.6 ± 5.6% with Bridge-only training. Removing DINOv2 within Bridge-only training yields 40.6 ± 5.5%. The respective mean gaps are 30.7 and 5.0 percentage points, making the mixture effect larger in this evaluation.
Where the evidence stops. The mixture comparison changes data scale and diversity together. The encoder gap has overlapping uncertainty and is measured only under Bridge-only training; the table cannot establish its size, or even persistence, at full OpenX scale.
Table 11. Changing the controller removes the earlier int8 performance deficit. Original paper, p. 35 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read this as a follow-up intervention on execution timing. Appendix D.4 changes the controller so that an action finishes before the policy predicts the next one. That differs from the non-blocking experiments summarized in Table 2, where prediction latency affects robot motion. The same three precision labels therefore do not denote the same full experimental conditions across the two tables. Within this table, compare task rows and the mean together: precision changes redistribute individual successes without producing the earlier aggregate int8 deficit. There are ten trials per task, and half-success entries retain the Bridge rubric rather than indicating fractional physical rollouts. e-blockinge-quantizatione-bridge-rubrice-limitations
What it supports. Under blocking control, bfloat16 reaches 70.0 ± 5.1%, int8 74.4 ± 4.9%, and int4 68.8 ± 5.2%. Their overlapping standard errors contrast with the non-blocking int8 deficit. The experiment supports the authors’ explanation that inference timing contributed substantially to the earlier apparent precision penalty.
Where the evidence stops. Overlapping standard-error bars do not prove statistical equivalence. Blocking control changes the deployment regime and does not demonstrate that low precision is harmless for every high-frequency task, GPU, or controller.
7. Analysis & limitations
7.1 What the evidence leaves open
Single-image sensing and slow sequential generation restrict temporal and high-frequency control. Typical success remains below 90%; model scaling and Internet-data co-training remain underexplored. e-limitations
The RT-2-X comparison changes training mixture, visual architecture and preprocessing together. Its Bridge evaluation uses a second-most-likely-action workaround, so the overall advantage cannot be attributed solely to DINOv2. e-preprocessinge-ablation
Adaptation averages combine unequal in-distribution/OOD trial counts and task-specific partial credit. They should not be interpreted as uniformly binary completion probabilities or compared without retaining the scoring rubric. e-adaptatione-franka-rubric
The five-point dual-encoder gap has overlapping uncertainty. Early frozen-vision tests use different configurations and include discontinued evaluations; neither establishes a universal architectural ranking. e-ablatione-frozen-vision
7.2 Questions for discussion
- Does the dual encoder help under matched compute at full OpenX scale?
- Would equal Internet-data exposure change semantic generalization?
- Would strict completion scoring change language-grounding rankings?
8. Reproducibility audit
8.1 Requirements and known gaps
Reproduction requires the correct model variant, token mapping, normalization statistics, mixture schedule and controller timing. The PDF names AMP, FlashAttention and FSDP but does not give an exact dependency/version lock or complete per-task fine-tuning/checkpoint-selection recipe. e-actionse-datae-traininge-releasee-efficiencye-adaptation-protocol
LIBERO requires re-rendering, the source’s orientation correction, failed-replay and no-op filtering, and original test initial states. The near-zero filtering threshold is not numerically specified. All compared methods must receive identical cleaned demonstrations. e-libero-setup
Reader-proposed checks: match execution timing across precisions, and repeat dual/single-encoder comparisons under both robot-data regimes to test dependence on training diversity. e-quantizatione-blockinge-ablation
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Test whether the int8 deficit survives matched timing
Reader-proposed experiment, not performed: use one fixed checkpoint, GPU and task set for bfloat16, int8 and int4. Repeat paired initial states under native non-blocking timing and under a shared command schedule chosen to accommodate the slowest mode; also reproduce the blocking condition. Hold image preprocessing, action scaling and episode limits fixed. Record prediction latency, command timestamps, strict completion and the original partial-credit score separately. If int8 remains worse when command timing is matched, latency alone is insufficient; if the deficit appears only at native non-blocking timing, the source’s controller explanation gains support. e-quantizatione-blockinge-bridge-rubric
Check 2: Complete the missing data-by-encoder comparison
Reader-proposed experiment, not performed: compare dual and SigLIP-only encoders under both Bridge-only and full OpenX robot training. Use the same action tokenization, preprocessing, image resolution, trial initial states and evaluation rubric. Report parameter counts and training compute, and repeat with multiple seeds; preserve training-schedule differences explicitly rather than hiding them in an average. The missing full-mixture SigLIP comparison tests whether Table 9’s encoder advantage transfers beyond Bridge-only training. If the dual encoder helps only in the restricted regime, its benefit is conditional; if it improves both regimes consistently, the architectural claim becomes stronger. e-ablatione-actionse-traininge-datae-preprocessing
8.3 Reading coverage
Visual audit: Visually inspected the title/authors/version, all main-paper and appendix figures (1–11), all tables (1–12), and the method, training, evaluation, rubric and reproduction evidence on the declared pages. Figure 2’s branches and action interface were cross-checked against Sections 3.1–3.2. All six final crops were viewed, including their labels, uncertainty and the Table 1 sharding footnote. Figure 5’s OpenVLA Move <object> onto Plate bar reads 69.4%, consistent with the corresponding trial-weighted Table 7 rows. Reference pages 12–20 were read in text only. Separate supplements, linked videos and code remain outside the supplied visual pass.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37. Appendix coverage: reviewed.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Abstract and title block
- 1 Introduction
- 2 Related Work
- 3 The OpenVLA Model, Sections 3.1–3.5
- 4 The OpenVLA Codebase
- 5 Experiments, Sections 5.1–5.4
- 6 Discussion and Limitations
- Acknowledgments and References
- A Data Mixture Details
- B Evaluation Tasks and Detailed Results, Sections B.1–B.3
- C RT-2-X vs. OpenVLA in BridgeData V2 Evaluations
- D Additional Experiments and Ablations, Sections D.1–D.4
- E LIBERO Simulation Experiments, Sections E.1–E.2
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Separate supplemental material availability has not been fully verified.
- All 12 supplied text chunks were read. Pages 1–11 and 21–37 were visually inspected, resolving the extraction-only figure limitation; reference pages 12–20 were read as text.
- Identity/version note: the artifact is arXiv:2406.09246v3, 5 September 2024, with the exact catalog title. Its author list includes Grace Lam, absent from the supplied catalog, and credits Ethan Foster and Pannag Sanketi without catalog middle names. Grace Lam has no affiliation marker. The catalog cites CoRL proceedings with differing year fields; that edition and other revisions were not supplied or compared.
- Code, linked project media, original training datasets and separate supplements were not inspected; experiments were not reproduced.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e-identityPDF p. 1, title, author block, affiliation footnote and arXiv margin
The matching title identifies arXiv:2406.09246v3, 5 September 2024. The authors include Grace Lam, Ethan Foster and Pannag Sanketi; Grace Lam has no affiliation superscript.
Go to primary source ↓e-problemPDF p. 2, Section 1 and footnote 1
The paper motivates accessible generalist policies and efficient adaptation. Base components have open weights, but their original training code and data are not open.
Go to primary source ↓e-architecturePDF p. 4, Figure 2 and Section 3.1
DINOv2 and SigLIP patch features are concatenated channel-wise and projected by a two-layer MLP into Llama 2. Image and instruction tokens condition action tokens, decoded into seven-dimensional robot control.
Go to primary source ↓e-actionsPDF p. 5, Section 3.2
Each action dimension uses 256 uniform bins between its first and ninety-ninth training-data quantiles. The final 256 vocabulary tokens are overwritten; cross-entropy applies only to action tokens.
Go to primary source ↓e-dataPDF p. 4, Section 3; p. 5, Section 3.3; p. 21, Table 3 and footnote 6
The mixture contains 970k demonstrations with third-person vision and single-arm end-effector control. DROID initially has 10% weight, removed for the last third of training and redistributed.
Go to primary source ↓e-trainingPDF p. 6, Sections 3.4–3.5
The chosen resolution is 224 by 224 pixels, and vision weights are fine-tuned. Training uses 27 epochs, fixed learning rate 2e-5, batch 2048, and 64 A100 GPUs for 14 days, reported as 21,500 A100-hours. Main-model inference runs at roughly 6 Hz on RTX 4090.
Go to primary source ↓e-releasePDF pp. 6–7, Section 4; p. 2, footnote 1
The authors describe released weights and a PyTorch stack with AMP, FlashAttention, FSDP, HuggingFace integration, LoRA and quantization. Original base-component training data and code remain undisclosed.
Go to primary source ↓e-evalPDF pp. 7–8, Section 5.1
Direct evaluation uses 17 WidowX tasks with ten trials each and 12 Google-robot tasks with five trials each. Methods share tasks and initial-state sets in A/B evaluations.
Go to primary source ↓e-bridgePDF p. 7, Figure 3; p. 26, Table 4, Mean Success Rate row
WidowX scores are OpenVLA 70.6±3.2%, RT-2-X 50.6±3.5%, Octo 20.0±2.6%, and RT-1-X 18.5±2.7%. Semantic-generalization scores are 36.3% for OpenVLA and 38.8% for RT-2-X.
Go to primary source ↓e-bridge-rubricPDF pp. 23–24, Appendix B.1.1, task rubrics; pp. 24–25, Appendix B.1.2
Several tasks permit 0.5 partial credit, including approaching the named object in language-grounding tasks. Robot, camera, lighting and objects differ from training; most test starts require reaching.
Go to primary source ↓e-googlePDF p. 26, Appendix B.2.1; p. 28, Table 6, Mean Success Rate row
Google-robot trials use binary success. OpenVLA scores 85.0±4.6%, RT-2-X 78.3±5.4%, RT-1-X 33.3±6.1%, and Octo 26.7±5.8%; the two VLA error bars overlap.
Go to primary source ↓e-adaptation-protocolPDF pp. 8–9, Section 5.2 and footnote 3; p. 29, Appendix B.3.1
Adaptation uses 10–150 demonstrations per task, with 5 Hz Franka-Tabletop and 15 Hz Franka-DROID non-blocking controllers. Standard Diffusion Policy uses history, proprioception, action chunks and absolute positions; its matched variant uses one image and one relative action.
Go to primary source ↓e-adaptationPDF p. 32, Table 7, task rows and environment Average rows
OpenVLA scores 67.2±4.0% on Franka-Tabletop and 58.3±7.2% on Franka-DROID; Diffusion Policy scores 48.5±4.9% and 35.0±8.0%. OpenVLA does not win every task.
Go to primary source ↓e-franka-rubricPDF pp. 30–31, Appendix B.3.1, task descriptions and Wipe Table scoring
Some tabletop tasks award 0.5 partial credit. Wipe Table uses 70 demonstrations and scores trials 0, 1 or 2 according to objects swept; 18 in-distribution and 12 OOD trials give a 60-point maximum.
Go to primary source ↓e-efficiencyPDF p. 10, Section 5.3, Table 1 and footnote 4
LoRA rank 32 gives 68.2±7.5%, 97.6 million trainable parameters and 59.7 GB VRAM at batch 16. Full fine-tuning gives 69.7±7.2% and 163.3 GB sharded across two GPUs. Sections 5.3–5.4 use a smaller-mixture, SigLIP-only variant.
Go to primary source ↓e-efficiency-tasksPDF p. 31, Appendix B.3.2 final paragraph; p. 32, Table 8
Efficiency evaluation covers Put Carrot in Bowl and Move <object> onto Plate with OOD variants, 33 trials total, and 50/150 demonstrations respectively.
Go to primary source ↓e-quantizationPDF p. 10, Table 2 and footnote 4; p. 11, Section 5.4; p. 26, Table 5
Under non-blocking control, bfloat16/int8/int4 scores are 71.3±4.8/58.1±5.1/71.9±4.7%, with 16.8/10.2/7.0 GB VRAM. On A5000, int8 runs at 1.2 Hz and int4 at about 3 Hz.
Go to primary source ↓e-preprocessingPDF pp. 32–33, Appendix C
OpenVLA drops each Bridge demonstration's first transition to reduce no-op freezing. RT-2-X is evaluated by always querying its second-most-likely action; dynamic re-querying performed worse.
Go to primary source ↓e-ablationPDF pp. 33–34, Appendices D.1–D.2; p. 34, Table 9
On eight Bridge tasks, OpenVLA scores 76.3±4.8%, Bridge-only dual-encoder training 45.6±5.6%, and Bridge-only SigLIP training 40.6±5.5%. The dual/single comparison shares Bridge-only robot training.
Go to primary source ↓e-frozen-visionPDF pp. 34–35, Appendix D.3 and Table 10
Early experiments favor fine-tuning vision encoders, but use different initial configurations. Some frozen-vision evaluations were discontinued; those cells are missing, not zero.
Go to primary source ↓e-blockingPDF pp. 34–35, Appendix D.4; p. 35, Table 11
Blocking control completes each action before predicting the next. Across eight tasks with ten trials each, bfloat16/int8/int4 score 70.0±5.1/74.4±4.9/68.8±5.2%, with overlapping standard-error bars.
Go to primary source ↓e-limitationsPDF p. 11, Section 6
The model supports a single image, lacks proprioception and observation history, needs faster inference for high-frequency tasks, and typically remains below 90% success. Scaling, Internet-data co-training and visual-feature choices remain underexplored.
Go to primary source ↓e-libero-setupPDF pp. 36–37, Appendix E.1, dataset modifications and evaluation details
Suites start with ten tasks and 50 demonstrations each. The authors re-render at 256 pixels, filter no-ops and failed replays, rotate third-person images 180 degrees on their hardware, and use only that camera. Policies train per suite, with three seeds and 500 evaluation trials per seed.
Go to primary source ↓e-libero-resultsPDF p. 37, Appendix E.2 and Table 12, Average SR and suite columns
Average LIBERO success is 76.5±0.6% for LoRA-fine-tuned OpenVLA, 75.1±0.6% for Octo and 72.4±0.7% for Diffusion Policy. OpenVLA leads Spatial and Long, but not Object or Goal.
Go to primary source ↓e-adaptation-aggregationPDF p. 9, Figure 5, OpenVLA Move <object> onto Plate bar; p. 32, Table 7, corresponding OpenVLA rows and trial counts
Figure 5 labels OpenVLA at 69.4% for Move <object> onto Plate. Table 7 reports 75.0% across twelve in-distribution trials and 58.3% across six OOD trials; weighting these rounded rates by trial count gives approximately 69.4%, consistent with the bar. The nearby 66.7% label belongs to Octo on Knock <object> Over.
Go to primary source ↓8.5 Primary sources
OpenVLA: An Open-Source Vision-Language-Action Model ↗
PDF · 21,623 extracted words
Source fingerprint
353c37df34458f12f969b14dfd8b77175b727b9cddea7bb891759beddeefe1be