3DFlowAction: Learning Cross-Embodiment Manipulation from 3D Flow World Model
1. Paper overview
In one sentence: Predicting object motion supplies a transferable manipulation target, while rendered-goal checks and robot-specific optimization handle the gap between an imagined trajectory and executable actions. e02e03e04e05e06e07e10e12e16
| At a glance | What to know |
|---|---|
| Research problem | Source description Robot demonstrations use incompatible action spaces. The paper instead represents how an object should move, aiming to reuse actionless videos across embodiments while retaining depth and rotation that image-plane flow can miss. e02 |
| Core mechanism | |
| A key reported result | Four foundational tasks: world-model comparison: 70.0%; task counts 6/10, 7/10, 5/10 and 10/10, respectively. Task success rate. Physical XTrainer evaluation; pouring, pen insertion, mug hanging and drawer opening; ten randomized-pose trials per task. AVDC 20.0%; Rekep 20.0%; Im2Flow2Act* 25.0%. The star replaces its learned policy with optimization. Favors the complete pipeline in this setup. Differing predictors and pretraining prevent isolating depth as the sole cause. e10e11 |
| Reading caution |
Core contributions
Figure 2. Object tracks become the supervision and prediction target for the flow world model. Original paper, p. 4 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start at panel I: the detector identifies the gripper, sampled points are filtered, and CoTracker3 locates the moving object before tracking it again. Section 3.1 adds the depth-prediction step that lifts these tracks into 3D. In panel II, RGB features, the instruction and object points feed the conditioning sequence above the U-Net. The legend marks the gray SD v1.5 layers with ice, and the orange LoRA and blue motion layers with flames; this matches the training description on page 5. Panel III identifies the mixture of human and robot video sources. Its percentages describe dataset composition, not task success. e03e04
What it supports. The model learns from 110k extracted motion instances. Its four output channels encode two image coordinates, depth and visibility; bypassing the image VAE preserves direct access to depth information. This provides an object-motion interface that can be passed to different robot solvers, without making the predictor itself an action policy.
Where the evidence stops. The detection, tracking and monocular-depth stages generate supervision rather than measured robot actions. The source does not provide a detailed label-error analysis, depth-scale calibration recipe or dataset partition. Frozen/trainable markers establish module treatment, not the completeness of training specifications.
2. Motivation
2.1 The problem and the proposed response
Robot demonstrations use incompatible action spaces. The paper instead represents how an object should move, aiming to reuse actionless videos across embodiments while retaining depth and rotation that image-plane flow can miss. e02
2.2 What this reading follows
To pour from a teapot, a robot must move the spout over the cup and then rotate the pot, while keeping its grasp feasible. 3DFlowAction learns such object trajectories from human and robot videos, using a common flow representation instead of robot action labels. Its diffusion model proposes motion; separate modules judge the rendered endpoint, choose a grasp and solve robot poses. The experiments report physical task success, including deployment on two arms, after task-specific human-video fine-tuning. Read the figures as a chain of distinct requirements: useful supervision, plausible object motion, reachable grasps and successful execution. The rigid-grasp assumption and small evaluation remain central boundaries. e02e03e04e05e06e07e10e12e16
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | Not assigned |
| Architecture | Not assigned |
| Prediction paradigm | Not assigned |
| Quadrant | Not assigned |
This table preserves the labels recorded at reading time. The current major category is WAMs. View the current classification.
3.1 Evidence-based assessment
Insufficient evidence to decide
The recorded taxonomy is entirely unassigned, so there is no substantive label to confirm. Architecture evidence indicates a flow predictor followed by separate VLM/grasp components and a numerical action solver. It does not demonstrate One Model joint future/action prediction or a learned inverse-dynamics decoder; the world prediction is used during inference. e04e05e06e07
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Learn a motion interface before learning robot actions
Begin with a demonstration video rather than an action log. The moving-object pipeline excludes the gripper, finds the moving region, tracks its points and attaches predicted depth. The resulting supervision describes where object points travel, not which robot joints produced that motion. The generator then conditions on the current RGB image, task instruction and initial points to predict this representation. Its four channels contain image coordinates, depth and visibility, and the image VAE is bypassed because the authors report poor depth encoding. Figure 2 separates frozen SD layers from trainable LoRA and motion modules. Large-scale pretraining is followed by task-level adaptation from 30 human demonstrations per task in the experiments. That adaptation is essential context: action-label-free learning still uses demonstrations of the intended tasks. e03e04e10
Figure 3. Endpoint verification and grasp feasibility bridge predicted object motion to robot execution. Original paper, p. 5 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Follow the orange panel from the initial scene to the flow proposal, rendering and verification box. The left-pointing IF FAIL arrow returns to generation; Section 4.1 confirms that GPT-4o judges the rendered endpoint against the instruction. Next, the blue panel asks which teapot part to grasp. AnyGrasp proposes candidates, the predicted rigid transform moves each candidate to its goal pose, and red IK Error examples are rejected. The green tick marks a reachable candidate. Finally, the green panel sends flow constraints into an optimizer. These arrows separate predicting a desired object trajectory from solving the end-effector trajectory that would realize it. e05e06e07e16
What it supports. The same predicted rigid transformation supports two different decisions: whether the final object placement appears task-consistent, and whether a candidate grasp would remain reachable at that placement. The accepted flow then guides a sequence of robot poses. This modular design is the mechanism behind the claimed embodiment transfer.
Where the evidence stops. The loop checks a rendered prediction, not a measured completed action. Endpoint semantic approval and final-pose reachability do not establish intermediate-path feasibility. Appendix B assumes rigid attachment; the paper explicitly warns that non-rigid objects can defeat the action solver.
5.2 Turn a flow proposal into a checkable goal and grasp
For pouring, the desired endpoint alone is not an action command. Section 4.1 first fits a transformation between the initial and final flow-point sets, then applies it to the object point cloud and renders a candidate final scene. GPT-4o evaluates whether this image matches the instruction; the failure arrow in Figure 3 requests a new prediction. Separately, the model identifies a task-relevant grasp region and AnyGrasp proposes poses. Transforming those poses to the predicted goal lets inverse kinematics reject candidates that cannot reach it. Reader interpretation: this splits semantic plausibility from geometric feasibility, with different checks for each. Neither check certifies the entire trajectory. The paper’s pouring description also requires keeping the pot level during transport before tilting, illustrating why intermediate motion matters even when an endpoint looks acceptable. e05e06e10
5.3 Follow the object trajectory through the robot solver
Appendix B samples object-surface keypoints and uses their predicted future positions as constraints. The printed objective sums squared point distances but leaves the optimization variable implicit; its continuation explains that a rigid transformation carries both the grasped object and gripper. Appendix C makes the robot-side variables concrete: position and Euler angles, bounded by workspace and orientation restrictions. The first solve combines Dual Annealing and SLSQP, while later solves reuse the previous solution. Reader interpretation: embodiment independence resides in the motion target, while feasible actions remain tied to robot geometry and rigid attachment. Table 2 supports this separation through two-platform deployment, but does not eliminate calibration or kinematic requirements. The authors’ deformable-object limitation is therefore mechanistically consistent with their solver rather than merely a gap in the benchmark. e07e08e12e16
5.4 Training and inference
During training
The Stable Diffusion backbone is frozen in Figure 2; LoRA layers and newly trained motion modules adapt it. Pretraining uses ManiFlow-110k. Experiments fine-tune with 30 actionless human demonstrations per task, collected in roughly ten minutes per task. e03e04e10
Table 6 lists AdamW, learning rate 0.0001, batch size 512, 500 epochs, weight decay 0.01 and epsilon 1e-8. Appendix D reports approximately two days on an “8x8 V100”; this hardware wording is retained without inferring its topology. e09
During inference
Detected initial points undergo the paper’s “corrosion” filtering to exclude boundary interference. Flow prediction, endpoint verification and grasp feasibility precede action solving; no learned robot-action decoder is specified. e04e05e06e07
SciPy optimizes one pose represented by position and Euler angles, with workspace and downward-facing orientation bounds normalized to [-1,1]. The first solve uses Dual Annealing followed by SLSQP, taking approximately one second; subsequent solves warm-start SLSQP. This is solver timing, not end-to-end latency. e08
The described feedback loop checks a predicted final image. It does not specify continuous execution-time visual feedback or guarantee that an accepted endpoint implies a safe intermediate trajectory. e05e07
5.5 Implementation flow
- Construct object-centric supervision
Grounding-SAM2 masks the gripper; sampled points inside it are removed. CoTracker3 tracks remaining points, and significantly moving points define an object bounding box. Object tracks are extracted again, camera motion is removed when needed, and DepthAnythingV2 depth lifts image tracks into 3D. This yields 110k instances. e03
- Predict flow rather than RGB video
The AnimateDiff-based generator conditions on CLIP encodings of RGB and text plus sinusoidal encodings of initial points. Its channels store image coordinates, depth and visibility. The U-Net receives flow directly because the authors found the image VAE poor at preserving depth. e04
- Verify the predicted endpoint
SVD estimates a rigid transformation from first to final flow points. The transformed object point cloud is added to the current scene and reprojected into an image. GPT-4o judges instruction alignment and can request another flow prediction. e05
- Choose a grasp and solve actions
GPT-4o identifies the relevant object part; AnyGrasp supplies candidate grasps. Candidates are transformed to the predicted goal and checked with inverse kinematics. Farthest-point-sampled flow constraints then guide successive end-effector poses, with IK and collision considerations. e06e07
6. Experiments & results
3DFlowAction learns instruction-conditioned object trajectories from human and robot videos, checks a rendered endpoint with GPT-4o, and converts accepted flow into robot poses through grasp selection and optimization. Its four-task physical evaluation reports 70% success, but transfer depends on rigid grasp geometry and follows task-specific human-video fine-tuning.
6.1 Read the original evidence
Table 1. The full pipeline leads the reported four-task physical comparison. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read each task row as successes out of ten randomized-pose trials, then compare the Total row across systems. The rightmost column gives 6/10 for pouring, 7/10 for insertion, 5/10 for mug hanging and 10/10 for drawer opening. Preserve the asterisk: the short original caption explains that Im2Flow2Act’s learned action policy has been replaced by optimization. Appendix A confirms this modification. The baseline is therefore distinct from the original Im2Flow2Act reported in Table 3. Section 5.1 supplies the XTrainer platform and task-specific human-video fine-tuning context; this table measures executed manipulation, rather than generated-video quality. e10e11
What it supports. 3DFlowAction reports 70.0% success against 20.0% for AVDC, 20.0% for Rekep and 25.0% for Im2Flow2Act*. The per-task rows also show substantial variation: drawer opening succeeds in every reported trial, while mug hanging succeeds in half. The aggregate advantage should be read together with those task-level counts.
Where the evidence stops. Only ten trials per task are reported, without confidence intervals. These systems differ in prediction machinery and prior training; replacing one baseline’s policy does not turn the comparison into a controlled ablation of depth alone.
Table 2. The same task-conditioned flow approach is evaluated on two robot platforms. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Move across each row to compare platforms on the same named task. Franka records 7/10, 7/10, 4/10 and 9/10; XTrainer records 6/10, 7/10, 5/10 and 10/10. Section 5.3 says deployment involves no robot-related fine-tuning or new annotations. Keep that claim separate from Section 5.1’s earlier collection of 30 human demonstrations per task. The transferable component is the object trajectory; the grasp reachability test and action solver still use each robot’s kinematics. The bottom row compresses the four task counts into one rate for each arm, so inspect the individual rows before interpreting similarity. e06e07e10e12
What it supports. Success totals of 67.5% on Franka and 70.0% on XTrainer support the feasibility of transferring the task-level motion representation between these two platforms. The row pattern is similar, including lower success on hanging than on drawer opening, but the table does not establish statistical equivalence.
Where the evidence stops. The evidence covers two arms and four already-adapted tasks. It does not test arbitrary hardware, hands or unseen task instructions, nor eliminate robot-specific geometry and calibration requirements. No uncertainty estimates accompany the platform comparison.
Table 4. Object and background changes are tested as separate transfer settings. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the two grouped column blocks independently. The left block changes target objects; the right block changes backgrounds. Within either block, each method has ten trials for each of the same four tasks. Figure 7 on page 15 shows example object changes, while Figure 8 shows background changes. In the object block, 3DFlowAction ties PI0 on insertion at 6/10; in the background block, it ties PI0 on pouring at 4/10. These ties matter because the aggregate advantage does not mean a strict improvement on every row. The zero-shot label here concerns the changed conditions, after the original task adaptation. e10e11e14
What it supports. 3DFlowAction reaches 55.0% for object changes versus PI0’s 40.0% and AVDC’s 15.0%. For background changes it reaches 50.0%, versus 32.5% and 0.0%. These results support transfer on the tested variations, while both 3DFlowAction rates remain below its 70.0% in-domain result.
Where the evidence stops. The table does not jointly vary objects and backgrounds, establish task generalization, or quantify the breadth of possible visual shifts. The qualitative examples are selected demonstrations; they cannot establish the distribution of failures behind the success counts.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| Four foundational tasks: world-model comparison Physical XTrainer evaluation; pouring, pen insertion, mug hanging and drawer opening; ten randomized-pose trials per task. | 70.0%; task counts 6/10, 7/10, 5/10 and 10/10, respectively. Task success rate | AVDC 20.0%; Rekep 20.0%; Im2Flow2Act* 25.0%. The star replaces its learned policy with optimization. Favors the complete pipeline in this setup. Differing predictors and pretraining prevent isolating depth as the sole cause. e10e11 |
| Cross-embodiment transfer on four tasks Direct deployment on Franka and XTrainer without robot-related fine-tuning; ten trials per task and platform. | Franka 67.5%; XTrainer 70.0%. Task success rate | Franka counts: 7, 7, 4, 9 successes; XTrainer: 6, 7, 5, 10, each out of ten. Evidence spans two arms after task-level human demonstrations; it is not zero-shot transfer to unseen tasks or proof of platform equivalence. e10e12 |
| Four foundational tasks: imitation-learning comparison Baselines fine-tuned using 30 teleoperated demonstrations per task; 3DFlowAction uses 30 human demonstrations. | 3DFlowAction 70.0%. Task success rate | PI0 50.0%; original Im2Flow2Act 27.5%. Demonstration counts are matched, but modalities, labels and prior training differ. Original Im2Flow2Act is distinct from the starred optimizer variant. e10e13 |
| Zero-shot object generalization on four tasks Changed target objects for the same tasks; ten trials per task. | 55.0%; counts 4/10, 6/10, 4/10, 8/10. Task success rate | AVDC 15.0%; PI0 40.0%. Transfer to tested object changes remains imperfect; pen insertion ties PI0. e10e14 |
| Zero-shot background generalization on four tasks Changed backgrounds for the same tasks; ten trials per task. | 50.0%; counts 4/10, 4/10, 4/10, 8/10. Task success rate | AVDC 0.0%; PI0 32.5%. Supports robustness on tested changes, while falling below in-domain success; pouring ties PI0. e10e14 |
| Rendering verification and pretraining ablations The same four manipulation tasks, ten trials each; Table 5 variants. | Full model 70.0%; no rendering verification 50.0%; no large-scale pretraining 30.0%. Task success rate | Removing verification loses 20 percentage points; removing pretraining loses 40 percentage points. Both components contribute in the reported runs. No neither-component variant, repeated-seed uncertainty or verification-only accuracy is reported. e15 |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Table 5. Removing either pretraining or rendering verification lowers reported task success. Original paper, p. 9 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. First use the checkmark columns to identify each intervention. Variant 1 retains large-scale pretraining and removes the rendering machine; Variant 2 retains rendering and removes large-scale pretraining. The final row includes both. Then compare Success Rate and the four task-count columns. Section 5.6 confirms this interpretation, resolving the compact layout. The original header’s typo in the cup-hanging task is preserved in the crop. Read the differences as percentage points: 70 to 50 is a 20-point decrease, and 70 to 30 is a 40-point decrease. Each aggregate summarizes the four reported sets of ten trials. e15e05
What it supports. The full model’s 70.0% exceeds the 50.0% without rendering verification and the 30.0% without pretraining. Pouring drops from 6/10 to 3/10 in either ablation; drawer opening drops from 10/10 to 9/10 without verification and to 4/10 without pretraining. Thus the effects differ by task.
Where the evidence stops. The table lacks a variant with both components removed and does not report repeated seeds. It therefore cannot estimate their interaction. Verification is also not compared against an equal-budget resampling control, leaving selection quality versus additional attempts unresolved.
7. Analysis & limitations
7.1 What the evidence leaves open
Flexible objects introduce severe occlusion and complex motion; non-rigid deformation can invalidate downstream action generation. This matches the rigid object–gripper connection assumed in Appendix B. e16e07
Ten trials per setting provide limited precision, and the tables give no confidence intervals. Four selected tasks and two robot platforms do not establish broad manipulation coverage. Qualitative flow/rendering examples are not quantitative dynamics-accuracy tests. e10e11e12e14e17
The paper reports over 80% moving-object detection accuracy on BridgeV2 without a detailed evaluation protocol. Motion thresholds, depth-scale calibration and dataset train/test partitioning are not specified in the supplied construction description. e03
7.2 Questions for discussion
- Would endpoint verification still improve executed success if flow-sampling count and compute were matched?
- How much transfer survives controlled depth-scale errors or violation of the rigid-grasp assumption?
8. Reproducibility audit
8.1 Requirements and known gaps
A reproduction needs the video-to-flow pipeline, diffusion backbone and adapters, GPT-4o verification/affordance prompts, AnyGrasp, calibrated scene geometry and robot IK/collision support. The reported setup uses one Femto Bolt camera facing the XTrainer workspace. e03e04e05e06e07e10
Missing details include flow horizon/resolution, denoising loss and schedule, LoRA configuration, separate fine-tuning settings, verifier prompts/retry limits, constraint weights and objective-evaluation budget. The printed optimization objective also leaves its variable implicit. Code/data are promised in the abstract, which does not establish release availability. e01e04e05e07e08e09
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Separate verifier quality from extra flow samples
Reader-proposed check, not performed: freeze the pretrained and task-adapted generator, grasps and solver, and generate a fixed-size pool of flow candidates for each randomized initial scene. Compare the first candidate, uniform selection from the same pool, and GPT-4o selection using rendered endpoints. Keep the maximum candidate count identical and record actual generation count, verifier calls and wall-clock time. Independently label endpoint task consistency, then measure physical task success and intermediate collisions. A consistent gain over uniform selection at the same sample budget would support useful verification; improvement only over the first candidate would leave extra sampling as an alternative explanation. Use the four paper tasks and report task-wise counts and uncertainty. e05e06e07e10e15
Check 2: Test whether depth variation helps the same action solver
Reader-proposed check, not performed: use one fixed 3DFlowAction model and the same scenes, predicted image-plane tracks, visibility, sampled keypoints and solver settings. Compare its complete flow against a controlled intervention that holds every tracked point’s depth at its initial value. Keep grasp choice fixed where both conditions are feasible, and report infeasible cases separately. Evaluate pouring and insertion, which require alignment and rotation, alongside drawer opening. Measure 3D keypoint residuals, IK failures and executed success; independently measured object motion can assess whether lower residuals reflect physical accuracy. A loss concentrated in depth-sensitive motions would support the value of predicted depth within this pipeline. This tests downstream use of depth, not the whole 2D-versus-3D training claim. e04e07e08e10e11
8.3 Reading coverage
Visual audit: The title/byline and revision on page 1, all nine original figures, all six tables, method equations, evaluation setup, limitations, and Appendices A–G were visually inspected on the declared pages. Figure 2 trainability markers and Figure 3 failure/IK branches were cross-checked against Sections 3.2 and 4.1–4.3. Appendix pages 14–17 supply optimizer, training, hardware wording and task details. All six final original crops were separately viewed; Table 1 retains its asterisk explanation and the narrow Table 2 was rendered at 600 DPI. References on pages 11–13 were read in the complete text but not rendered. External code, separate supplements and videos were not inspected.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 14, 15, 16, 17. Appendix coverage: reviewed.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Abstract; 1 Introduction; 2 Related works
- 3 3D Flow World Model for Manipulation, including 3.1–3.2
- 4 Flow-Guided Action Planning, including 4.1–4.3
- 5 Experiments, including 5.1–5.6
- 6 Conclusion and Limitations
- References
- Appendix A Baselines
- Appendix B Flow-Based Action Generation
- Appendix C Implementation Details of Optimization Procedure
- Appendix D Training Details of 3DFlowAction
- Appendix E More Manipultion Visualization
- Appendix F Flow World Model Visualization
- Appendix G Experimental Tasks
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Identity: the inspected title page states arXiv:2506.06199v1 [cs.RO], 6 June 2025, and Preprint. Under review. Its title and seven authors match the catalog. No different revision or edition was supplied or compared.
- Text extraction does not reconstruct figure images; the retained PDF was inspected separately for figures and equation/table layout.
- Separate supplemental material availability has not been fully verified.
- Code, external datasets and linked resources were not inspected; experiments were not reproduced. Reference entries were read as bibliography, without consulting their cited works.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e01PDF p. 1, title/byline, arXiv margin and Abstract
Title and all seven authors match the catalog; v1 is dated 6 June 2025 and marked Preprint. Under review. Four affiliations are printed. The abstract states code and data will be available.
Go to primary source ↓e02PDF pp. 1–3, Abstract and Section 1; p. 3, Section 3 introduction
The motivation is incompatible robot action representations and object-centric, embodiment-agnostic motion prediction that captures rotation and depth.
Go to primary source ↓e03PDF p. 4, Section 3.1 and Figure 2(I, III)
Gripper masking, point filtering/tracking, moving-object localization, depth lifting and optional camera-motion removal produce ManiFlow-110k. The text reports over 80% moving-object detection accuracy; Figure 2 gives the seven-source composition.
Go to primary source ↓e04PDF pp. 4–5, Section 3.2; p. 4, Figure 2(II)
RGB/text CLIP features and encoded initial points condition AnimateDiff. Flow channels are two image coordinates, depth and visibility. The VAE is bypassed. Figure 2 marks SD v1.5 frozen and LoRA/motion modules trainable; text says motion modules train from scratch and describes initial-point filtering.
Go to primary source ↓e05PDF p. 5, Section 4.1, unnumbered SVD equation and Figure 3(I)
Initial/final point sets define an SVD-estimated transformation. The transformed object is rendered into the scene; GPT-4o assesses the image with the instruction. Figure 3 routes verification failure back to flow generation.
Go to primary source ↓e06PDF pp. 5–6, Section 4.2; p. 5, Figure 3(II)
GPT-4o selects an object part, AnyGrasp produces candidates, and the predicted rigid transformation maps grasps to target poses for IK reachability filtering. The diagram rejects IK-error candidates.
Go to primary source ↓e07PDF p. 6, Section 4.3; pp. 14–15, Appendix B and its unnumbered objective
Farthest-point-sampled object keypoints supply flow constraints for SE(3) end-effector actions. The squared-distance expression leaves the minimization variable implicit; prose assumes rigid object–gripper attachment and optimizes transformations, adding IK and collision considerations.
Go to primary source ↓e08PDF pp. 15–16, Appendix C
SciPy pose optimization uses position/Euler angles, workspace and lower-hemisphere orientation bounds, and normalization to [-1,1]. Initial Dual Annealing plus SLSQP takes approximately one second; later solves warm-start local optimization. The numerical evaluation budget is unspecified.
Go to primary source ↓e09PDF p. 16, Appendix D and Table 6, all parameter rows
Training settings are learning rate 0.0001, batch 512, epochs 500, AdamW, weight decay 0.01 and epsilon 1e-8. The text reports an “8x8 V100” for approximately two days without clarifying the hardware layout.
Go to primary source ↓e10PDF pp. 6–7, Section 5.1 and Figure 4; p. 17, Appendix G
The physical setup uses a Dobot XTrainer and one third-person Femto Bolt camera. Four tasks use ten randomized-pose trials per setting; 30 human demonstrations per task fine-tune the model. Appendix G explains task geometry, including aligning a teapot spout to an approximately 5 cm cup opening before tilting.
Go to primary source ↓e11PDF p. 7, Table 1, all task rows and Total; p. 14, Appendix A, Im2Flow2Act* paragraph
Table 1 reports AVDC/Rekep/Im2Flow2Act*/3DFlowAction totals of 20/20/25/70%. Their pouring counts are 1/2/2/6, insertion 2/1/2/7, hanging 0/3/0/5 and drawer 5/2/6/10, out of ten. The starred baseline substitutes optimization for diffusion policy.
Go to primary source ↓e12PDF p. 7, Table 2, all rows; p. 8, Section 5.3
Franka versus XTrainer counts are 7/6, 7/7, 4/5 and 9/10 successes out of ten; totals 67.5/70%. Section 5.3 says deployment requires no robot-related fine-tuning or new annotation.
Go to primary source ↓e13PDF p. 7, Table 3, all rows; p. 8, Section 5.4; p. 14, Appendix A
PI0/original Im2Flow2Act/3DFlowAction totals are 50/27.5/70%. The imitation baselines receive 30 teleoperated demonstrations per task. Appendix A distinguishes the original diffusion-policy method from its starred replacement.
Go to primary source ↓e14PDF p. 8, Table 4, all rows, and Section 5.5 continuing on p. 9; p. 15, Figures 7–8
Object-change totals are AVDC 15%, PI0 40%, 3DFlowAction 55%; background-change totals are 0%, 32.5%, 50%. Ours has object counts 4/6/4/8 and background counts 4/4/4/8 out of ten. Figure 7 shows object changes; Figure 8 shows background changes.
Go to primary source ↓e15PDF p. 9, Table 5, checkmark columns, all task rows and Section 5.6
Variant 1 retains pretraining only: 50%, with counts 3/5/3/9. Variant 2 retains rendering only: 30%, with counts 3/3/2/4. Full model has both: 70%, with counts 6/7/5/10. Each task denominator is ten.
Go to primary source ↓e16PDF p. 10, Limitations
The authors identify occlusions/complex motion for flexible objects and failure of action generation under non-rigid deformation.
Go to primary source ↓e17PDF p. 8, Figure 5; p. 9, Figure 6; p. 15, Figures 7–8; pp. 16–17, Figure 9 and Appendices E–F
The figures show selected planning/execution sequences and in-domain flow/endpoint renderings. Appendix F interprets these as learned motion patterns but gives no quantitative flow-error benchmark.
Go to primary source ↓8.5 Primary sources
3DFlowAction: Learning Cross-Embodiment Manipulation from 3D Flow World Model ↗
PDF · 7,939 extracted words
Source fingerprint
b81a3cef309b83d25ca8febed99ce45a6f4707aae7ad73e199a31cb911c42a74