OASIS: Observation-Action Space Alignment via SE(3) Trajectory Prediction for Robotic Manipulation
1. Paper overview
In one sentence: Supervising future end-effector poses makes decoder features useful for rigid-body control, but successful execution still depends on a learned action decoder and feedback. e-alignmente-encodere-trajectorye-decodere-ablatione-closed-forme-oode-uncertainty
| At a glance | What to know |
|---|---|
| Research problem | Author claim The authors argue that visual features and predicted future images leave rigid-body pose recovery implicit inside an action decoder. They propose an intermediate with an explicit horizon-indexed SE(3) pose readout, up to a fixed frame transform. This is a design principle, expressly not a sufficiency theorem for control. e-alignment |
| Core mechanism | Source description A three-stage policy combines vision-language and metric-depth encoding, camera-frame pose prediction, and learned action decoding. Demonstration-derived trajectory supervision shapes the actual decoder-conditioning pathway. e-encodere-trajectorye-decoder |
| A key reported result | LIBERO four-suite manipulation: 99.0 / 98.8 / 97.4 / 95.2; average 97.6 Success rate (%). Spatial/Object/Goal/Long; 10 tasks per suite, 50 demonstrations and 50 evaluation episodes per task; three-seed averages. Baselines imported from their papers. Unified-VLA average 95.5; OASIS leads by 2.1 percentage points and ties the best Object result. Strong benchmark performance, with heterogeneous pretraining rather than an equal-compute comparison. e-sim-protocole-libero |
| Reading caution | Source description Evaluation covers single-arm tabletop manipulation. Cup hanging and pot placement expose orientation/placement errors, while drawer-opening errors propagate into later grasping. Mobile and dexterous extensions remain future work. e-limitse-failures |
Core contributions
- Source description
A three-stage policy combines vision-language and metric-depth encoding, camera-frame pose prediction, and learned action decoding. Demonstration-derived trajectory supervision shapes the actual decoder-conditioning pathway. e-encodere-trajectorye-decoder
- Source description
Experiments distinguish metric versus relative depth, auxiliary versus decoder-routed pose supervision, position versus full pose, reference frame, rotation parameterization and geometric versus learned execution. e-ablatione-rotatione-closed-form
Figure 2. Trajectory hidden states carry supervised geometry into a separate action decoder. Original paper, p. 5 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read left to right. Language and image observations enter Qwen, while the image also supplies DA3 metric-depth features. Their projected tokens form the predictor's cross-attention context. Follow the green trajectory hidden states along two branches: a linear pose readout provides trajectory supervision, and a separate arrow carries the hidden states into the orange decoder's cross-attention. The robot-state encoder supplies the other decoder context. Along the bottom, expert demonstrations supervise actions directly and supervise poses after world-to-camera conversion. The flame and snowflake agree with LoRA adaptation of the VLM and frozen DA3. Equation (13) weights the trajectory loss by 0.1, a detail omitted by the schematic sum. e-encodere-trajectorye-decodere-objectivese-traininge-figure-indexe-ablation
What it supports. The decoder uses the pose-supervised hidden states, which retain task context beyond the explicit pose vectors. The pose head makes geometry readable and trainable, while the action head learns execution. This routing is the mechanism later tested by the auxiliary-branch ablation; the figure alone does not establish its benefit.
Where the evidence stops. The action-chunk label uses t+h, whereas the formulation and Eq. (12) use t+h−1. Follow the equations for indexing. The schematic's L_train corresponds to the weighted L_total objective; neither notation difference is silently repaired in the crop.
2. Motivation
2.1 The problem and the proposed response
The authors argue that visual features and predicted future images leave rigid-body pose recovery implicit inside an action decoder. They propose an intermediate with an explicit horizon-indexed SE(3) pose readout, up to a fixed frame transform. This is a design principle, expressly not a sufficiency theorem for control. e-alignment
2.2 What this reading follows
A robot can recognize the right object while still misjudging how its gripper should approach, rotate and release. OASIS addresses that gap by making future end-effector geometry readable from the features that guide its action decoder. The policy first combines language, RGB and inferred metric-depth features, then predicts a camera-frame pose trajectory and uses the predictor's hidden states to generate an action chunk. The most revealing experiments ask whether those supervised states actually reach the decoder and whether explicit geometry alone can execute the task. Read the benchmark gains alongside those controls, the limited camera-shift test and the appendix's finite-trial uncertainty. e-alignmente-encodere-trajectorye-decodere-ablatione-closed-forme-oode-uncertainty
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | VLA |
| Architecture | Dual-system |
| Prediction paradigm | Not applicable |
| Quadrant | Not applicable |
3.1 Evidence-based assessment
Conflicts with the recorded classification
VLA and future-representation assistance fit the architecture. A separate predictor and action decoder support a multi-module or dual-system interpretation, not a single shared action/world predictor. However, the listed diffusion/flow-matching subcategory conflicts with direct transformer decoding and L1 supervision: no denoising or flow objective is described. This is pose-conditioned control, not future-image prediction or an explicitly formulated inverse-dynamics model; the WAM prediction/quadrant fields can remain not applicable. e-trajectorye-decodere-objectives
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Make the future pose readable before decoding actions
Begin with the distinction between knowing what should happen and recovering a rigid-body motion that can make it happen. Section 3 defines an aligned intermediate through a pose readout at each future horizon step, allowing a fixed transform between that readout and world coordinates. OASIS implements this with eight camera-frame poses rather than a predicted scene image. The explicit readout contains position and axis-angle rotation, while the decoder consumes the hidden states that produce it. This preserves information useful for execution without requiring every hidden dimension to be a pose coordinate. Reader interpretation: the design constrains what the representation must explain, yet leaves the decoder freedom to use additional context. It should therefore be assessed through both pose-related controls and executed task success, not through the existence of an interpretable trajectory alone. e-alignmente-trajectorye-decodere-closed-form
5.2 Separate supervision geometry from deployment calibration
At training time, demonstration poses in the world frame are converted into the third-person camera frame with an extrinsic matrix. The trajectory loss then compares position and axis-angle coordinates using L1 error, while a second L1 objective supervises relative actions and gripper commands. Their trajectory weight is 0.1. At deployment, the policy estimates metric-depth features from RGB and conditions action decoding on trajectory hidden states and robot state; it does not run the privileged geometric pipeline used in Appendix B.3. Thus RGB-only observations and demonstration-derived labels can coexist with a training-time calibration requirement. Reader interpretation: camera-frame targets may simplify learning, but the measured frame advantage is not a general invariance guarantee. The claimed canonical axis-angle range also belongs to the reported tabletop setting; the paper does not specify an explicit mechanism enforcing that range for arbitrary novel orientations. e-traininge-objectivese-encodere-rotatione-closed-forme-ablatione-ood
5.3 Use ablations to interpret the benchmark lead
The LIBERO and CALVIN tables establish reported policy performance, but their baselines differ in pretraining and were imported from other papers. The more direct mechanism test keeps a trajectory branch yet prevents its hidden states from reaching the decoder: Long success stays at 89.7%, compared with OASIS's 95.2%. This supports the importance of the conditioning path beyond auxiliary supervision alone. The hardcoded decoder test adds a second boundary: using predicted poses does not itself deliver successful control. Reader interpretation: together these controls motivate the architecture more strongly than an overall ranking does, although neither disentangles every information and control difference. On real robots, precise cup/pot placement and multi-stage error propagation remain failure modes. Read the OOD result as robustness to specified perturbations and the low-data result as evidence from one drawer task, with uncertainty retained. e-liberoe-calvine-sim-protocole-ablatione-closed-forme-failurese-oode-scalinge-uncertainty
5.4 Training and inference
During training
World-frame demonstration poses are transformed using the third-person camera extrinsic to create trajectory labels. Training therefore requires extrinsic information despite RGB-only policy inputs. Mean L1 trajectory and action losses are combined with trajectory weight 0.1. e-objectivese-training
LoRA adapts the VLM; predictor, decoder and projections are trained from scratch while DA3 stays frozen. The model has 1.73B total and 0.18B trainable parameters. Training uses 50,000 steps, four A800 GPUs, global batch 64, AdamW at 2 × 10⁻⁴, cosine scheduling and 5,000 warm-up steps, averaging three seeds. No large-scale robotic pretraining does not mean no pretrained models. e-training
During inference
Live RGB and robot state produce a pose-conditioned action chunk for sequential execution. The authors describe feedback-based retargeting after bowl displacement. Appendix C.2 reports about 0.05 s per inference cycle, roughly 20 Hz, and 4.5 GB memory on an unloaded RTX 4090; this does not specify the full sensor-to-actuator latency or exact replanning schedule. e-oode-latency
5.5 Implementation flow
- Encode semantics and metric structure
A Prismatic-derived VLM combines Qwen2.5-0.5B with DINOv2 and SigLIP, pretrained on LLaVA-1.5-Instruct. Frozen DA3METRIC-LARGE estimates metric-depth features from RGB. Projected language, visual and depth tokens are concatenated; depth is inferred, not supplied by a depth sensor. e-encodere-training
- Predict camera-frame poses
Eight learned queries pass through four transformer blocks with RoPE self-attention and cross-attention to the fused features. A linear readout predicts position and axis-angle rotation. Rodrigues conversion produces valid rotations; the claimed canonical-chart bound relies on the tabletop orientation range. e-trajectorye-rotation
- Decode executable actions
Two transformer blocks cross-attend to trajectory hidden states concatenated with an embedded current robot state. The decoder receives richer hidden states rather than only six-dimensional pose readouts. It learns action and gripper outputs, including practical residuals beyond pose differencing. e-decodere-closed-form
6. Experiments & results
OASIS guides a visuomotor policy with hidden states supervised to predict future camera-frame end-effector poses. Metric-depth features support this intermediate, and a separate learned decoder produces executable action chunks. Results favor the design on LIBERO, CALVIN and two real robot platforms, while routing ablations provide the clearest mechanism evidence. Pose prediction remains a control aid, not a guarantee of successful execution (e-alignment, e-trajectory, e-libero, e-calvin, e-real-results, e-ablation).
6.1 Read the original evidence
Table 1. OASIS leads the reported LIBERO average and ties the best Object score. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start with the four suite columns, then use Average to compare overall performance. Every suite contains ten tasks; OASIS uses fifty demonstrations and fifty evaluation rollouts per task, with results averaged across three seeds. The Intermediate column describes what conditions each action decoder. Pretrain has a narrower meaning than its label might suggest: the checkmark denotes large-scale robotic pretraining. OASIS's cross does not remove its pretrained vision-language and depth components. Compare OASIS with Unified-VLA for the strongest reported average, and notice that Object has multiple tied entries. The baseline values were taken from their original papers rather than regenerated under one training budget. e-liberoe-sim-protocole-traininge-ablation
What it supports. OASIS's average is 97.6% against Unified-VLA's 95.5%, a 2.1-percentage-point lead. Its 95.2% Long result exceeds the best listed prior Long score of 94.0%. These are task-execution outcomes in simulation, supporting policy effectiveness without directly measuring trajectory accuracy or establishing why the representation helps.
Where the evidence stops. The table supplies no seed dispersion, and models differ in pretraining and architecture. Its ranking is a benchmark comparison; the within-OASIS ablations are more informative about the proposed mechanism than cross-model success differences alone.
Figure 6. Robustness holds for three tested Goal perturbations with a fixed wrist camera. Original paper, p. 9 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Use the legend to compare blue OASIS, pink π0.5 and green RDT within each perturbation group. Backgrounds adds unfamiliar objects and a tabletop appearance. Perspectives moves only the third-person camera by roughly fifteen centimeters; the wrist view stays fixed and no policy is recalibrated or adapted at test time. Interference relocates the destination bowl while the robot carries the object. Appendix Figure 10 shows these scenarios, and Section 5.4 describes regenerating trajectory guidance to retarget. These definitions are essential: the three groups test different disturbances, and the camera group contains one specific displacement rather than a broad camera-pose sweep. e-oode-real-protocol
What it supports. OASIS reaches 94.8%, 91.2% and 86.5% on background, viewpoint and interference tests, respectively. Corresponding π0.5 results are 88.6%, 76.4% and 81.5%. The strongest separation from π0.5 in this chart occurs under the tested camera displacement, while interference still reduces OASIS's success.
Where the evidence stops. The source explicitly says this test does not establish camera-pose invariance. The unchanged wrist view may support robustness, and this chart includes neither error bars nor an independent ablation proving that camera-frame prediction causes the OOD lead.
Table 15. Higher real-world point estimates coexist with finite-trial uncertainty. Original paper, p. 22 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Each cell contains a success percentage followed by a Wilson 95% interval. Compare the two methods within a column. Goal and Spatial pool six subtasks with sixty trials each, giving 360 trials per suite; Long has sixty trials on one drawer-opening, banana-retrieval and bowl-placement task. These are descriptive pooled intervals rather than variation across independent training seeds or a paired test of methods. The rates match the main real-world comparison, whose reported three-suite averages are 89.2% for OASIS and 81.6% for π0.5. The interval widths help explain why the source describes the Long-task improvement cautiously despite its higher point estimate. e-uncertaintye-real-resultse-real-protocol
What it supports. For Long, OASIS reports 83.3% with [71.7, 90.7], while π0.5 reports 71.6% with [58.9, 81.6]. Both Goal and Spatial also favor OASIS numerically. These observed leads support further evaluation, while the intervals preserve uncertainty that the main success table alone cannot show.
Where the evidence stops. The following source paragraph claims clear Goal/Spatial interval separation, but both overlap slightly: 96.6–96.9 and 81.8–82.3, respectively. Preserve the table values; neither overlap nor its absence alone replaces a properly specified comparison test.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| LIBERO four-suite manipulation Spatial/Object/Goal/Long; 10 tasks per suite, 50 demonstrations and 50 evaluation episodes per task; three-seed averages. Baselines imported from their papers. | 99.0 / 98.8 / 97.4 / 95.2; average 97.6 Success rate (%) | Unified-VLA average 95.5; OASIS leads by 2.1 percentage points and ties the best Object result. Strong benchmark performance, with heterogeneous pretraining rather than an equal-compute comparison. e-sim-protocole-libero |
| CALVIN ABC→D sequential manipulation Train A/B/C, test unseen D; 1,000 five-instruction chains without scene resets; three-seed averages. | 4.57; 83.3% Average completed sequence length; five-task success (%) | DreamVLA: 4.44; 78.1%. Five-task gap: 5.2 percentage points. Better long-chain completion; reduced error accumulation is an author interpretation, not directly measured. The included 3D Diffuser Actor uses multi-view RGB-D. e-calvine-sim-protocol |
| Trajectory supervision and decoder routing LIBERO-Long and Spatial ablations using the reported training protocol. | OASIS 95.2 / 99.0 Success rate (%), Long / Spatial | No trajectory: 89.5 / 91.6; parallel auxiliary trajectory: 89.7 / 91.9; world-frame pose: 93.2 / 96.5. Routing supervised states to the decoder matters more than merely adding the auxiliary branch. The no-trajectory row removes predictor capacity. e-ablation |
| Learned versus geometric action execution Same predicted trajectory; geometric pipeline receives known extrinsics and privileged simulator gripper information. | Learned: 99.0 / 95.2 Success rate (%), Spatial / Long | Hardcoded: 12.4 / 0.0. This pipeline fails despite privileged inputs; the comparison does not isolate calibration, contact and timing errors individually. e-closed-form |
| Real-world Goal, Spatial and Long manipulation Franka Research 3 and Kinova Gen3; 50 demonstrations per task; three runs of 20 trials per task. Goal and Spatial each contain six evaluated subtasks. | Goal 98.6; Spatial 85.8; Long 83.3; reported suite average 89.2 Success rate (%) | π0.5: 95.0 / 78.3 / 71.6; average 81.6. Average lead is 7.6 percentage points. Long Wilson intervals overlap: OASIS [71.7, 90.7], π0.5 [58.9, 81.6]; no significant Long-task advantage is established. e-real-protocole-real-resultse-uncertainty |
| Real-world Goal under OOD perturbations Unseen backgrounds, one approximately 15 cm third-person camera displacement without recalibration, and bowl relocation during transport; wrist view unchanged. | 94.8 / 91.2 / 86.5 Success rate (%) in that order | π0.5: 88.6 / 76.4 / 81.5. Supports tested perturbation robustness; neither full camera-pose invariance nor a separate OOD confidence interval is supplied. e-ood |
| Real-world Long demonstration scaling 10, 25 or 50 demonstrations; 60 evaluation trials per configuration on the drawer-to-bowl task. | 35.0 / 55.0 / 83.3 Success rate (%) | π0.5: 15.0 / 35.0 / 71.6. OASIS with 10 demonstrations matches π0.5 with 25 on this task; no universal data-efficiency multiplier follows. e-scaling |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Figure 4. The auxiliary trajectory branch helps little when its hidden states do not condition actions. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Use the printed values because the vertical axis begins at 89%, magnifying visual differences. The first two alternatives remove metric depth or replace it with Depth Anything 2 relative depth. The central comparison is w/o Traj. versus w/ AuxTraj: the latter retains a parallel trajectory predictor and loss, but the decoder sees the fused encoder features instead of trajectory hidden states. Next, w/ 2D, w/ 3D Pos. and w/ World increase the supervision target's geometric content, ending with world-frame full pose. OASIS uses camera-frame full pose. Appendix Table 10 repeats these comparisons on Spatial and explains the predictor-capacity difference between the first two trajectory controls. e-ablatione-trajectorye-decoder
What it supports. On Long, the parallel auxiliary branch reaches 89.7%, close to 89.5% without the predictor, while OASIS reaches 95.2%. World-frame pose reaches 93.2%. The results support decoder access to supervised trajectory states and an empirical camera-frame advantage; they do not show that any auxiliary pose loss is sufficient.
Where the evidence stops. Table 10's matched-budget caption conflicts with its prose: w/o Traj. removes predictor parameters. Relative-depth replacement also changes the depth model, so it does not isolate metric scale alone. No error bars justify significance claims for small gaps.
Table 11. A meaningful predicted trajectory does not make geometric execution sufficient. Original paper, p. 18 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read this as a change in the execution interface. Both rows use the same predicted camera-frame trajectory, but the hardcoded row transforms explicit poses with known extrinsics and generates relative actions. It also receives privileged simulator information for gripper opening and closing, unavailable to the learned policy. The OASIS decoder instead conditions on trajectory hidden states and the current robot state. Compare each column vertically rather than mixing Spatial and Long tasks. Appendix B.3 says the geometric pipeline often moves toward the right object yet fails during precise execution; the authors attribute this to prediction noise, contact effects and gripper timing. e-closed-forme-decodere-alignment
What it supports. The geometric pipeline falls to 12.4% on Spatial and 0.0% on Long, versus 99.0% and 95.2% with learned decoding. The practical result is decisive for this replacement: readable rigid-body geometry remains useful supervision, while a learned controller converts its richer conditioning features into successful actions.
Where the evidence stops. Privileged extrinsics and gripper information do not make every aspect of the hardcoded pipeline stronger. OASIS also receives richer hidden-state context. This comparison does not isolate each residual error source or rule out better geometric feedback controllers.
7. Analysis & limitations
7.1 What the evidence leaves open
Evaluation covers single-arm tabletop manipulation. Cup hanging and pot placement expose orientation/placement errors, while drawer-opening errors propagate into later grasping. Mobile and dexterous extensions remain future work. e-limitse-failures
Source inconsistencies matter: Figure 2 indexes actions at t+h, while the formulation uses t+h−1. Table 10 claims identical budgets although adjacent prose says the no-trajectory variant removes predictor parameters. The action explanation follows the equations and the ablation analysis preserves that asymmetry. e-figure-indexe-ablation
C.3 claims clear Goal/Spatial interval separation, but Table 15 intervals overlap slightly in both suites. Interval overlap alone is not a significance test. Simulation tables and ablations omit seed dispersion, limiting uncertainty assessment. e-uncertaintye-liberoe-calvine-ablation
7.2 Questions for discussion
- How much useful decoder context remains in trajectory hidden states beyond their explicit pose readout?
- Would camera-frame prediction retain its advantage across broader extrinsic and orientation changes?
8. Reproducibility audit
8.1 Requirements and known gaps
Reproduction requires the pretrained VLM/depth components, RLDS demonstrations, camera extrinsics and robot control conventions. Appendix A specifies 896-wide, eight-head transformers but omits LoRA rank/targets, action normalization and exact software versions; deployment also leaves chunk replacement timing unclear. e-traininge-trajectorye-decodere-latency
Proposed checks: repeat the routed-versus-parallel trajectory comparison with matched capacity and paired seeds; separately sweep camera translations/rotations while controlling wrist-view availability. Measure execution success alongside pose error and replanning latency. e-ablatione-oode-latency
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Does routing supervised states help at matched capacity?
Reader-proposed, not run: train three otherwise matched LIBERO-Long/Spatial variants using the same data, initialization seeds and training budget. Retain the predictor in every variant. Compare OASIS routing with a parallel auxiliary branch whose hidden states do not reach the decoder, and with routed hidden states but no trajectory loss. Keep decoder context dimensions and trainable parameter counts equal. Report paired task success, pose translation/rotation errors and seed dispersion. The mechanism predicts that routed pose supervision improves execution over both controls. If a parallel branch matches it, or routing helps equally without pose supervision, the specific alignment explanation weakens. Log the conditioning tensors to verify the intended routes. e-ablatione-trajectorye-decodere-objectivese-traininge-sim-protocol
Check 2: Where does the camera-frame advantage stop?
Reader-proposed, not run: compare camera-frame OASIS and the world-frame pose variant after matched training, then sweep held-out third-person translations and rotations without recalibration or test-time adaptation. Include the nominal view and the paper's roughly 15 cm displacement as controls. Repeat with the wrist image available and occluded, applying identical conditions to both policies. Measure success, pose error in a common evaluation frame, sensor-to-action latency and recovery after a controlled bowl move. A persistent camera-frame lead would support broader robustness; a lead that disappears without the wrist view or under modest rotations would locate its dependence on the tested observation setup. Evaluation calibration should measure errors without becoming a policy input. e-ablatione-oode-traininge-latencye-real-protocol
8.3 Reading coverage
Visual audit: Actually rendered and visually inspected the title/author page, all main-text pages and all appendix pages. This includes Figures 1–11, Tables 1–15, method equations, training configuration, evaluation definitions, hardware/latency statements, failure analysis and Wilson intervals. All six final original crops were individually viewed and checked for readable labels, complete boundaries and agreement with the page images. Figure 2's hidden-state routing agrees with Sections 4.1–4.4; its action-index and loss-label differences are disclosed. The Table 10 capacity-caption conflict and Table 15 interval/prose conflict are also disclosed. All seven text chunks were read individually, including reference pages 10–14, which were not rendered. Separate supplements, project website, code and experiments remain outside this reading. No visual limitation prevents the six-visual edition.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9, 15, 16, 17, 18, 19, 20, 21, 22, 23. Appendix coverage: reviewed.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Abstract
- 1 Introduction
- 2 Related Work
- 3.1 Problem Formulation
- 3.2 Aligned Intermediate Representations of Visuomotor Policies
- 4.1 Overview of OASIS
- 4.2 3D-aware Feature Encoder
- 4.3 SE(3) Trajectory Predictor
- 4.4 Action Decoder
- 5.1 Implementation Details
- 5.2 Simulation Experiments
- 5.3 Ablation Study
- 5.4 Real-world Experiments
- 6 Conclusion and limitations
- References
- A.1 OASIS architecture
- A.2 Training details
- B.1 Simulation benchmarks and settings
- B.2 Additional ablation results on LIBERO-Spatial
- B.3 Closed-form decoder ablation
- C.1 Real-world experimental settings
- C.2 Additional real-world experimental results
- C.3 Wilson 95% confidence intervals for real-world results
- D Broader impacts
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- The preceding supplied extraction limitation was addressed by inspecting PDF pages 1–9 and 15–23 and all six final crops. Reference pages 10–14 were read as text.
- Separate supplemental material availability has not been fully verified.
- All seven supplied text chunks were read individually, covering the complete 23-page artifact. Code and the project website were not inspected; experiments were not reproduced.
- Identity: the inspected title page states arXiv:2605.25829v1 [cs.RO], 25 May 2026, Preprint. Its title and eight authors match the catalog. No alternative revision was supplied or compared; no revision or title discrepancy was observed. Appendix C.3 is present despite its absence from the manifest's section list.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e-identityPDF p. 1, title block, affiliation lines and arXiv margin
Exact OASIS title; Xinzhe Chen, Sihua Ren, Liqi Huang, Haowen Sun, Mingyang Li, Xingyu Chen, Zeyang Liu and Xuguang Lan; Xi’an Jiaotong University affiliation; arXiv:2605.25829v1, 25 May 2026.
Go to primary source ↓e-alignmentPDF pp. 3–4, Sections 3.1–3.2, Eqs. (1)–(4)
Defines relative rigid-body actions, separate gripper commands and a pose-readable aligned intermediate up to a fixed rigid transform. Explicitly labels alignment a design principle rather than a sufficiency theorem.
Go to primary source ↓e-encoderPDF p. 5, Section 4.2, Eq. (8); p. 15, Appendix A.1, Vision-language model and Metric depth module
Concatenates language, visual and metric-depth features; frozen DA3METRIC-LARGE; Prismatic-derived Qwen backbone with DINOv2/SigLIP and LLaVA instruction pretraining.
Go to primary source ↓e-trajectoryPDF pp. 4–5, Eqs. (5)–(10), Figure 2 and Section 4.3; p. 15, Appendix A.1 and Table 5
Pose-supervised hidden states feed the decoder; linear position/axis-angle readout; four predictor blocks, eight queries, hidden size 896, eight attention heads.
Go to primary source ↓e-decoderPDF p. 6, Section 4.4 and Eq. (12); p. 15, Appendix A.1 and Table 5
Two-block action decoder attends to trajectory hidden states plus robot-state embedding and directly predicts relative action/gripper chunks.
Go to primary source ↓e-objectivesPDF p. 6, Eqs. (11)–(13)
Mean L1 losses supervise camera-frame pose vectors and executable action/gripper vectors; total trajectory-loss weight is 0.1.
Go to primary source ↓e-trainingPDF p. 6, Section 5.1; pp. 15–16, Appendix A.1–A.2, Tables 5–7
Reports 1.73B total/0.18B trainable parameters; RGB views at 224 × 224, seven-dimensional state, extrinsic-derived camera-frame labels, LoRA, frozen depth, four A800 GPUs, batch 64, AdamW, 50k steps, learning rate 2e-4, cosine schedule, 5k warm-up and three seeds. LoRA rank, normalization and software versions are not specified here.
Go to primary source ↓e-sim-protocolPDF p. 7, Section 5.2, evaluation protocol; pp. 16–17, Appendix B.1
LIBERO has four ten-task suites, 50 demonstrations and 50 evaluation rollouts per task. CALVIN tests D after A/B/C training over 1,000 instruction chains. OASIS uses RGB/language/state, trains 50k steps per suite and averages three seeds; baseline results come from original papers.
Go to primary source ↓e-liberoPDF p. 7, Table 1, OASIS and Unified-VLA rows, all suite/Average columns
OASIS reports 99.0, 98.8, 97.4, 95.2 and 97.6 average; Unified-VLA average is 95.5. Pretrain marks refer specifically to large-scale robotic pretraining; no dispersion is tabulated.
Go to primary source ↓e-calvinPDF p. 7, Table 2, OASIS/DreamVLA rows and RGB-D footnote; Section 5.2 results
OASIS 4.57 average sequence length and 83.3% five-task completion; DreamVLA 4.44 and 78.1%. 3D Diffuser Actor uses multi-view RGB-D. Authors interpret the long-chain lead as less error accumulation.
Go to primary source ↓e-ablationPDF p. 8, Figure 4 and Section 5.3; pp. 17–18, Appendix B.2 and Table 10
Long/Spatial: no trajectory 89.5/91.6, auxiliary branch 89.7/91.9, 2D 90.7/93.3, 3D position 92.3/95.4, world pose 93.2/96.5, OASIS 95.2/99.0, no metric 91.8/93.4. Relative-depth Long is 92.0. Table 10 caption claims matched budgets; adjacent prose acknowledges the no-trajectory variant lacks predictor parameters. No seed dispersion is supplied.
Go to primary source ↓e-rotationPDF pp. 5–6, Section 4.3 and Eq. (10); p. 8, Table 3
Axis-angle is converted by Rodrigues to valid rotations; authors assert orientations stay below the pi chart boundary in this setting. Long success: axis-angle 95.2%, quaternion 91.6%, Euler 92.2%.
Go to primary source ↓e-closed-formPDF p. 18, Appendix B.3 and Table 11
With the same predicted trajectory, known camera extrinsics and privileged gripper information, the geometric pipeline yields 12.4% Spatial and 0.0% Long versus learned 99.0% and 95.2%. Authors attribute failures to prediction noise, contact and timing.
Go to primary source ↓e-real-protocolPDF pp. 8–9, Section 5.4 and Figure 5; pp. 19–20, Figure 9 and Table 12; p. 22, Appendix C.3
Franka Research 3 and Kinova Gen3; policies fine-tuned on 50 demonstrations per task and evaluated in three runs of 20 trials. Six Goal and six Spatial subtasks, one Long drawer/banana/bowl task; pooled n=360 for Goal/Spatial and n=60 for Long.
Go to primary source ↓e-real-resultsPDF p. 9, Table 4; p. 22, Table 13
OASIS Goal/Spatial/Long/average: 98.6/85.8/83.3/89.2%; π0.5: 95.0/78.3/71.6/81.6%. Table 13 lists individual OASIS subtasks.
Go to primary source ↓e-oodPDF p. 9, Figure 6 and OOD results analysis; p. 20, Appendix C.1 OOD settings; p. 21, Figure 10
Background/viewpoint/interference rates: OASIS 94.8/91.2/86.5, π0.5 88.6/76.4/81.5, RDT 71.6/58.4/41.2. Viewpoint test shifts only third-person camera by roughly 15 cm without recalibration/adaptation. Wrist camera stays fixed. Interference moves the bowl during transport; authors describe trajectory regeneration and retargeting.
Go to primary source ↓e-failuresPDF p. 20, Appendix C.2 Failure analysis; p. 21, Figure 11; p. 22, Table 13
Cup-hanging rotation errors, pot support misplacement and drawer-stage errors propagating into later grasps are identified; success is 76.6%, 83.3% and 83.3%, respectively.
Go to primary source ↓e-latencyPDF pp. 20–21, Appendix C.2 Inference speed analysis
Remote WebSocket inference receives images up to 30 Hz and robot state. An unloaded RTX 4090 uses about 4.5 GB; each inference predicts eight actions with about 0.05 s response, roughly 20 Hz. Exact controller timing and software versions are not supplied.
Go to primary source ↓e-scalingPDF p. 21, Appendix C.2 Data-scaling; p. 22, Table 14
At 10/25/50 demonstrations OASIS gives 35.0/55.0/83.3%, π0.5 gives 15.0/35.0/71.6%, with 60 trials per configuration. Authors limit the data-efficiency claim to this one Long task.
Go to primary source ↓e-uncertaintyPDF p. 22, Appendix C.3, Table 15 and following paragraph
Reported Wilson 95% intervals: Goal OASIS [96.6,99.5], π0.5 [92.4,96.9]; Spatial [81.8,88.9] versus [73.7,82.3]; Long [71.7,90.7] versus [58.9,81.6]. The following prose claims clear Goal/Spatial separation and describes Long as only a numerical lead.
Go to primary source ↓e-limitsPDF p. 9, Section 6 limitations; pp. 22–23, Appendix D
Current method focuses on single-arm tabletop manipulation; mobile and dexterous action spaces are future directions. Broader deployment and safety benefits are proposed impacts, not evaluated outcomes.
Go to primary source ↓e-figure-indexPDF p. 5, Figure 2 Action chunk label; pp. 3 and 6, Eqs. (2), (12) and action-chunk definition
Figure 2 displays action/gripper subscripts t+h, whereas the action formulation and supervision use t+h−1 for horizon h. The figure also labels the combined loss L_train while Eq. (13) calls it L_total.
Go to primary source ↓8.5 Primary sources
OASIS: Observation-Action Space Alignment via SE(3) Trajectory Prediction for Robotic Manipulation ↗
PDF · 10,324 extracted words
Source fingerprint
06a165ba642b0bc971c6de13be555a8c87e8842d7fc9af7a6589323068e5ed6e