Beyond Data Scaling: Representation-Centric Continued Pre-training for Vision-Language-Action Models

English reading report: Method, equations, original figures, experiments and reproducibility.

Authors: Senqiao Yang, Chengyao Wang, Yuxin Chen, Zixuan Wang, Longxiang Tang, Haokun Gui, Jinhui Ye, Changsheng Lu, Xiaoyang Wu, Mingkang Zhu, Pengguang Chen, Shu Liu, Zhuotao Tian, Hengshuang Zhao, Bei Yu, Jiaya Jia

Source: 2608.27550 ↗ · Project page ↗ · Catalog record

Reading: 38 / 558 · 6 original figures & tables · ~23 min ·

1. Paper overview

In one sentence: VLAct preserves early vision-language features and trains several action decoders against shared targets, producing a backbone that transfers to fresh downstream heads without deploying the pre-training ensemble. e02 e03 e04 e05 e10 e20 e28

At a glance	What to know
Research problem	Author claim How can continued pre-training turn limited, heterogeneous robot demonstrations into a reusable backbone? The authors identify three obstacles: erosion of broad vision-language priors, specialization to one action decoder, and incompatible embodiment-specific action coordinates. e02
Core mechanism	Source description A backbone-centered training recipe combines partial freezing, caption supervision, and multiple action heads; downstream evaluation discards the pre-training heads. e03 e04 e08
A key reported result	LIBERO-Plus robustness: 82.6% Total task success rate. Train on standard LIBERO; evaluate perturbed test conditions with an OFT downstream head. Qwen3VL-OFT: 75.0%; Abot-M0: 80.5%. The matched in-house gain is 7.6 percentage points. Improvement is not uniform: language-perturbation success is 81.5%, below Qwen3VL-OFT's 87.0%. e10
Reading caution	Reader analysis Decoder lock-in and preservation of VLM priors are interpretations of adaptation results. The supplied evidence does not directly establish geometric representation collapse or preserved general vision-language benchmark performance. e02 e19 e20 e24

Core contributions

Source description
A backbone-centered training recipe combines partial freezing, caption supervision, and multiple action heads; downstream evaluation discards the pre-training heads. e03 e04 e08
Source description
A partially shared action layout preserves different arm-control conventions while sharing compatible gripper supervision. Periodic joint targets and prediction residuals receive angular wrapping. e05 e06
Reader analysis
The clearest evidence for head generality is improved PI adaptation after pre-training with OFT and GR00T while excluding PI. Gains with all three heads also reflect direct exposure to the downstream decoder family. e20 e21

Pre-training diagram freezes the vision encoder and shallow language layers, mixes caption and robot inputs, and trains OFT, GR00T and PI heads; fine-tuning unfreezes the backbone and initializes one head. — **Figure 3.** Several decoders shape the backbone; one fresh decoder adapts it to the task. Original paper, p. 6 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.

2. Motivation

2.1 The problem and the proposed response

Author claim

How can continued pre-training turn limited, heterogeneous robot demonstrations into a reusable backbone? The authors identify three obstacles: erosion of broad vision-language priors, specialization to one action decoder, and incompatible embodiment-specific action coordinates. e02

2.2 What this reading follows

Robot pre-training can make a policy accurate yet leave its backbone difficult to reuse. VLAct studies that distinction by changing how action supervision reaches a pretrained vision-language model. It freezes early layers, mixes caption supervision, asks several continuous action heads to decode the same features, and aligns only compatible action coordinates across robots. Downstream adaptation keeps the backbone but starts the chosen action head afresh. The figures below explain that training recipe; the tables test robustness and the accessibility of action information to another decoder. The strongest interpretation is improved policy initialization under the reported protocols. The paper does not introduce a future-observation model, and decoder-independent representation quality remains an experimental claim rather than a measured geometric property. e02 e03 e04 e05 e10 e20 e28

3. Research context

We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.

Catalog dimension	Recorded classification
Major category	VLA
Architecture	Not applicable
Prediction paradigm	Not applicable
Quadrant	Not applicable

3.1 Evidence-based assessment

Supports the recorded classification

Reader analysis

The recorded VLA category and continued-training umbrella fit the described contribution. Training predicts demonstrated action chunks, and downstream inference uses an action decoder; the method specifies no future-observation model, inverse-dynamics bridge, or world-model planning loop. Joint supervision by three action heads does not establish a One Model WAM architecture. The recorded Not applicable architecture, prediction-paradigm, and quadrant fields are therefore supported as outside the WAM quadrant scheme, rather than unverified. Data augmentation is secondary to the core continued-pre-training recipe. e03 e04 e05 e07 e25

This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.

4. Problem formulation

4.1 Inputs and outputs

Inputs	Outputs
Camera observations and a language task instruction, encoded by Qwen3-VL-4B. [e03, e10] Demonstrated future action chunks, embodiment-dependent dimension masks, and auxiliary image-caption pairs during continued pre-training. [e04, e05, e08] Robot state as an additional conditioning input in the GR00T-family formulation described in Appendix I.1. [e07]	Reusable backbone weights after continued pre-training. [e03] Continuous action chunks from a selected downstream head; the pre-training layout contains 20 coordinates per action step. [e05, e07]

4.2 Equations and their role

\begin{aligned}\mathcal L_{\mathrm{action}}&=\mathcal L_{\mathrm{OFT}}+\mathcal L_{\mathrm{PI}}+\mathcal L_{\mathrm{GR00T}},\\\mathcal L_{\mathrm{total}}&=\mathcal L_{\mathrm{action}}+0.5\mathcal L_{\mathrm{VLM\text{-}CE}}.\end{aligned}

Section 3.3 sums decoder objectives; Appendix D adds auxiliary language-token cross-entropy with weight 0.5. Appendix F separately augments each decoder objective with the angular penalty. e04 e08 e06

\begin{aligned}a_{\mathrm{wrap}}&=((a+\pi)\bmod 2\pi)-\pi,\\\delta_{\mathrm{wrap}}&=(((\hat a-a)+\pi)\bmod 2\pi)-\pi,\\\mathcal L_{\mathrm{wrap}}&=|\delta_{\mathrm{wrap}}|.\end{aligned}

Equations (1)–(3) canonicalize the target angle a and measure the shortest periodic residual from prediction â. Equation (4) adds this penalty to the original head loss, avoiding large regression errors across the angular boundary. e06

\begin{aligned}A_t^\tau&=(1-\tau)\epsilon+\tau A_t,\qquad\epsilon\sim\mathcal N(0,I),\\\mathcal L_{\mathrm{PI}}&=\mathbb E\!\left[\lVert v_\phi(A_t^\tau,\tau;H_t)-(A_t-\epsilon)\rVert_2^2\right].\end{aligned}

A_t is a demonstrated chunk, ε is Gaussian noise, τ is flow time, and H_t is the backbone representation. The decoder learns the velocity transporting noise toward actions. GR00T uses the same objective family with additional conditioning and a separate motor module. e07

5. Method in detail

5.1 Protect a useful visual prior while making its upper layers action-aware

Reader analysis

Continued pre-training starts from Qwen3-VL-4B, rather than training a foundation model from scratch. Robot trajectories come from DROID and MolmoAct for Franka and from InternData-A1 and RoboCoin for AgileX. Because that interaction data has narrower coverage than the original vision-language corpus, VLAct freezes the vision encoder and lower half of the language layers. The upper layers remain trainable and receive both robot-action losses and caption cross-entropy, with the auxiliary loss weighted by 0.5. Full-model updating resumes during downstream fine-tuning. The freezing ablation supports this recipe through higher policy success. My interpretation is that preservation is a plausible explanation, but the paper does not directly measure retained general vision-language capability. Caption-source naming and baseline wording also need clarification before the auxiliary-data contribution can be reproduced precisely. e03 e08 e09 e10 e19 e24 e31

Three panels contrast separate AgileX and Franka heads, a padded shared head, and a 20-dimensional layout with separate arm coordinates and an aligned gripper coordinate. — **Figure 4.** Share a gripper meaning without pretending two robots have the same arm coordinates. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.

5.2 Use disagreement between decoder families as a training constraint

Source description

A single backbone forward pass produces features that all three continuous heads must use to predict the same demonstrated action chunk. OFT performs direct regression, while PI and GR00T use iterative conditional flow matching in separate action modules. Their native losses are summed, so the backbone must support several decoding procedures. At downstream adaptation, the pre-training heads are discarded and a fresh selected head is trained with the backbone. This rules out simply carrying over already-adapted head weights as the source of the in-house gain. It does not rule out a benefit from having seen that decoder family before. Table 8 addresses that distinction by excluding PI during pre-training: the OFT-plus-GR00T representation gives a modest improvement when a new PI head is introduced. That comparison is more diagnostic of reusable features than the all-head result. e03 e04 e07 e20

5.3 Align action meanings before comparing transfer results

Reader analysis

Sharing an output dimension is a claim about physical meaning. VLAct therefore keeps twelve AgileX joint coordinates separate from six Franka end-effector coordinates and shares only the compatible gripper slot. Masks remove inactive coordinates from each sample’s loss. Data cleaning also converts delta actions using frame rates, normalizes gripper ranges and wraps joint angles; the wrap-aware penalty handles residuals across the angular boundary. These operations make the supervision more consistent without assuming identical kinematics. Downstream experiments then ask different questions: LIBERO-Plus tests perturbation robustness, RoboTwin separates limited clean demonstrations from added randomized demonstrations, and RoboCasa-GR1 tests adaptation to an embodiment excluded from pre-training. My reading is that these jointly support a useful VLA initialization. They do not establish a world-model prediction mechanism, nor eliminate the need for target-robot data and controller configuration. e05 e06 e09 e10 e11 e12 e22

5.4 Training and inference

During training

Source description

The main robot mixture uses DROID and MolmoAct for Franka, and InternData-A1 and RoboCoin for AgileX. Continued pre-training uses a 4B backbone and a reported 16-GPU setup. A separate extension adds 20K UMI-style trajectories; these are not part of the default mixture. e09 e10 e25

Source description

The wrap-aware penalty is added to each head's native objective and evaluated on its final predicted action, including the generated sample for iterative heads. It applies only to absolute joint angles, excluding grippers and delta end-effector actions. e06

Source description

Physical evaluation trains separate single-arm and dual-arm policies for 50,000 steps on eight H800 GPUs, using 50 demonstrations per single-arm task and 100 per dual-arm task. e16

During inference

Source description

OFT predicts an entire chunk in one forward pass. PI and GR00T generate continuous chunks iteratively from noise, conditioned on backbone features. Pre-training does not require deploying an ensemble of all three heads. e03 e07

Reader analysis

Robot execution is assessed through simulator episodes and physical rollouts. The supplied method does not specify a complete deployment schedule for chunk execution, observation refresh, or low-level feedback, so closed-loop timing cannot be reconstructed. e07 e11 e16

5.5 Implementation flow

Clean and align demonstrations
Reject invalid task labels, mask abnormal action steps, and discard heavily corrupted chunks. Delta end-effector actions are converted using dataset frame rates; joint angles are wrapped; grippers are normalized per dataset. e09
Protect the inherited representation
Freeze the vision encoder and lower half of the language-model layers. Update upper layers with robot-action supervision and caption cross-entropy; each minibatch contains robot and auxiliary samples. e03 e08
Expose shared features to diverse decoders
A shared backbone forward pass produces latent z. OFT, PI, and GR00T predict the same demonstrated action chunk from this representation, and their losses are summed. OFT regresses actions directly; Appendix I describes both PI and GR00T as flow-matching generators. e04 e07
Share only compatible action coordinates
Coordinates 1–12 encode AgileX arm joints, 13–18 encode Franka delta end-effector pose, 19 shares Franka and AgileX-left gripper supervision, and 20 represents the AgileX-right gripper. Inactive coordinates are masked. Each decoder family shares its output layout across embodiments; the recipe still uses three decoder families. e04 e05
Transfer and specialize
Discard the pre-training heads and caption stream, attach a fresh task-specific head, and unfreeze the full backbone. The in-house comparisons hold downstream data, head initialization, optimizer, and training budget fixed. e03 e16

6. Experiments & results

VLAct changes how a pretrained vision-language backbone learns from robot trajectories: protect early representations, train several continuous action decoders on shared features, and align compatible action coordinates across robots. The resulting backbone is transferred to a freshly initialized downstream action head. Controlled comparisons support better policy adaptation; an ablation excluding the downstream PI head provides narrower evidence for decoder-independent transfer. These are action-policy experiments, with no future-video prediction component. The supplied identifier, title, and authors match the catalog; the observed version is arXiv v1, dated August 27, 2026.

6.1 Read the original evidence

Eleven model rows compare success under camera, robot, language, lighting, background, noise and layout perturbations, followed by a total score; VLAct is highlighted. — **Table 1.** The overall robustness gain includes a language-perturbation regression. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.

RoboCasa-GR1 plot shows VLAct success at 10%, 20%, 50% and 100% downstream data, with vertical full-data reference lines for π0.5, GR00T-N1.6 and Qwen3VL-OFT. — **Figure 5(d).** A held-out robot can benefit from the backbone after supervised adaptation. Original paper, p. 12 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.

6.2 Results and evaluation conditions

Task & protocol	Reported result	Comparison & interpretation
LIBERO-Plus robustness Train on standard LIBERO; evaluate perturbed test conditions with an OFT downstream head.	82.6% Total task success rate	Qwen3VL-OFT: 75.0%; Abot-M0: 80.5%. The matched in-house gain is 7.6 percentage points. Improvement is not uniform: language-perturbation success is 81.5%, below Qwen3VL-OFT's 87.0%. e10
RoboTwin 2.0 limited-data manipulation Base: 50 clean demonstrations per task; 50 tasks and 100 evaluation episodes per task in each regime.	80.5% / 41.5% Clean / Random success rate	Qwen3VL-OFT: 61.7% / 10.5%. Reported OFT results support improved clean-to-random transfer without randomized downstream demonstrations. e11
RoboTwin 2.0 with additional demonstrations Data Scaling adds 500 randomized expert demonstrations per task to the Base data.	VLAct-OFT: 92.5% / 90.8% Clean / Random success rate	Qwen3VL-OFT: 88.2% / 88.3%; HoloBrain-0-QW: 91.9% / 92.3%. Competitive across both regimes, but not best in every column. VLAct-PI reaches 93.0% Clean; external training compute is not matched. e11
VLA-Arena behavioral generalization Category scores average difficulty levels L0/L1/L2; the final score uses official weighting over 11 suites.	54.8% Weighted average success rate	Qwen3-VL-OFT: 33.4%; π0.5: 44.3%. The in-house improvement is 21.4 percentage points; the final average should not be recomputed as an unweighted mean of the four displayed categories. e14
DOMINO dynamic manipulation One policy evaluated on all 35 tasks under clean dynamic conditions.	18.50 / 34.20 Success Rate / Manipulation Score, in the table's reported scales	Qwen3VL-OFT: 10.86 / 30.49. Both metrics improve, although substantial difficulty remains. Manipulation Score is distinct from complete-task success. e15
Held-out GR-1 embodiment adaptation RoboCasa-GR1 downstream fine-tuning; GR-1 absent from continued pre-training.	49.5% with 20% of downstream trajectories; 54.0% with all trajectories. Task success rate	Full-data Qwen3VL-OFT: 48.8%; full-data GR00T-N1.6: 47.6%. Supports sample-efficient adaptation to a new embodiment, rather than zero-shot control. Values are stated explicitly in Section 4.4. e12
Held-out ARX X5 simulation RoboDojo: 42 tasks, 50 episodes per task; August 24, 2026 leaderboard snapshot.	10.66 / 7.60% Average partial-progress score / binary success rate	X-WAM: 7.69 / 3.83%; VLAct ranks eighth by score and sixth by success among 35 policies. These aggregate scores exceed all four explicitly designated WAM entries in this snapshot. They do not establish an architecture-level advantage under matched resources. e13
Physical Franka manipulation Matched demonstrations and fine-tuning; 10 fixed initial configurations per task shared across models.	Single-arm short-horizon: 92.5%; dual-arm coordination: 72.0%. Average binary completion rate, separately by task family	Qwen3VL-4B-OFT: 77.5% and 44.0%, respectively. These are physical execution results. Small per-task rollout counts limit precision. e16 e17
Physical long-horizon manipulation Table cleaning and scooping beans; scoring rewards completed task steps.	Table cleaning: 86.6%; scooping beans: 80.0%. Normalized partial-completion score	Baseline: 73.3% and 33.3%, respectively. Despite the paper's weighted-success terminology, these values are not binary episode-completion rates. e18
Transfer to an unseen action-head family Appendix E.1 PI fine-tuning diagnostic; PI excluded from OFT+GR00T pre-training.	63.1% Downstream success rate	No additional pre-training: 60.5%; OFT-only pre-training: 55.1%; all three heads including PI: 77.0%. The unseen-head gain over scratch is 2.6 percentage points. The much larger all-head result also includes direct PI-family supervision. e20
Shallow-layer protection ablation Table 6 compares pre-training update strategies on LIBERO-Plus and RoboTwin 2.0.	Freeze vision encoder and lower half of LLM: 82.6% / 80.5%. Success rate on the two benchmarks	Update full backbone: 78.9% / 77.1%; freeze vision only: 81.3% / 79.3%. Supports the freezing recipe through downstream performance; it does not directly quantify retained vision-language capability. e19
Angular parameterization ablation RoboTwin Base, Clean evaluation; 50 tasks with 50 training trajectories each.	Target wrapping plus residual wrapping: 80.5%. Task success rate	Raw-angle baseline: 75.5%; target wrapping alone: 78.6%. Controlled comparisons support both canonicalizing targets and correcting boundary residuals. e22

6.3 Ablations and diagnostic examples

Read component removals and qualitative examples within their stated evaluation conditions.

Four pre-training conditions report whether PI was seen, PI fine-tuning success and change from scratch: none, OFT, OFT plus GR00T, and all three heads. — **Table 8.** The unseen-head comparison is the 63.1 row, not the 77.0 row. Original paper, p. 26 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.

Three RoboTwin configurations successively enable canonical joint-angle targets and a wrap-aware residual loss, with success values 75.5, 78.6 and 80.5. — **Table 10.** Correcting periodic targets and correcting their residuals are separate changes. Original paper, p. 28 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.

7. Analysis & limitations

7.1 What the evidence leaves open

Reader analysis

Decoder lock-in and preservation of VLM priors are interpretations of adaptation results. The supplied evidence does not directly establish geometric representation collapse or preserved general vision-language benchmark performance. e02 e19 e20 e24

Reader analysis

Published-system comparisons do not normalize training compute, optimization schedules, or checkpoint selection. They provide competitive context but weaker causal evidence than the matched in-house comparisons. e11 e13

Source description

The study covers a 4B backbone; larger-model behavior is untested. RoboDojo Memory performance is particularly weak at 0.66 partial-progress score and 0.56% success. e26 e13

Reader analysis

Section 4.3 describes physical dual-arm transfer as following exclusively single-arm pre-training, while Section 4.4 and Appendix H describe a Franka-plus-bimanual-AgileX mixture. The physical checkpoint's precise pre-training subset is unresolved. e16 e12 e09

Reader analysis

The physical study uses only 10 rollouts per task and reports no confidence intervals. Its long-horizon scores award partial credit, preventing direct comparison with binary short-horizon success. e16 e18

Reader analysis

The caption ablation has inconsistent baseline wording: Figure 8 separates Baseline 75.0, + Pretrain 79.6 and + Image Caption 82.6, while Appendix D.3 describes 75.0 as the robot-only baseline. The plot also labels a condition + Code where the prose discusses pure language instructions. The full 7.6-point improvement cannot be assigned to caption mixing alone. e10 e24 e31

Reader analysis

The pilot Figure 2 reports OFT-to-OFT success of 75.8 on RoboTwin, while Table 9 reports 78.8 for matched OFT single-head pre-training. The paper does not clearly identify the recipe difference, so the two values should not be merged into a single baseline. e32

7.2 Questions for discussion

Would the modest unseen-PI gain persist with matched compute and repeated seeds? [e20]
Which direct representation measurements could distinguish preservation of VLM knowledge from improved optimization? [e19, e24]
Does sharing the gripper coordinate remain beneficial when embodiments have different grasp mechanics or command semantics? [e05, e23]
Can temporal conditioning improve RoboDojo Memory performance without sacrificing the backbone's transfer advantages? [e13]

8. Reproducibility audit

8.1 Requirements and known gaps

Source description

Reproduction requires the Qwen3-VL-4B initialization, cleaned robot mixture, auxiliary captions, three continuous heads, masks, and angular loss. Cleaning specifies delta-action thresholds of 0.5 translation and 1.0 rotation after FPS conversion, and discards chunks with invalid-step ratio above 0.5; the gripper outlier cutoff remains unspecified. e03 e04 e05 e09 e10

Reader analysis

The supplied paper does not provide a complete continued-pre-training configuration: retained dataset sizes, sampling ratios, optimizer hyperparameters, total steps, chunk horizon, generative integration steps, and random-seed uncertainty remain unresolved. GPU count alone does not establish total training cost. e07 e08 e09 e10

Reader analysis

Resolve caption provenance before replication: Table 7 lists LLaVA OneVision alongside LLaVA-ReCap-CC3M, whereas the accompanying caption-data paragraph names ShareGPT4V alongside ReCap. e24

Reader analysis

A focused mechanism test would repeat the unseen-PI comparison with matched robot samples, backbone updates, downstream initialization, and measured compute, reporting variation across seeds. Keep PI absent from pre-training to distinguish decoder transfer from direct head-family exposure. e04 e20

Reader analysis

The abstract claims an open release while Section 4 uses future-release language; the paper’s release statements alone do not establish that the specified checkpoints and training configuration are available. e27

8.2 Proposed reproduction checks

The following checks are proposals motivated by the paper. They have not been run as part of this reading.

Check 1: Does head diversity help an excluded decoder under matched resources?

Reader-proposed experiment: repeat the no-pre-training, OFT-only and OFT-plus-GR00T conditions with PI strictly excluded from continued pre-training. Match robot samples, backbone update counts, downstream PI initialization, hyperparameter selection and evaluation episodes across repeated seeds. Log wall-clock compute as well as success. A consistent improvement of OFT-plus-GR00T over both controls would support reusable decoder compatibility; disappearance after equalizing optimization or seed variance would weaken that explanation. Treat the all-three-head condition as a separate family-exposure control. e03 e04 e20 e21

Check 2: Do captions preserve broad capability or mainly improve policy optimization?

Reader-proposed experiment: first resolve the caption-source list and Figure 8 baseline definitions, then cross caption mixing on/off with shallow-layer protection on/off while holding robot data and backbone update budget fixed. Evaluate a held-out vision-language suite alongside LIBERO-Plus, retaining the language-perturbation column separately. If caption mixing preserves the original suite performance while improving control, it supports the preservation explanation. If control improves without retained vision-language performance, the gain needs a narrower optimization or task-transfer explanation. This test has not been run. e03 e08 e10 e19 e24 e31

8.3 Reading coverage

Visual audit: Read the complete 37-page source, including references and Appendices A–K. Visually inspected the title, p. 12 author list, all fourteen numbered figures and all twelve tables, then inspected all six final crops. Figure 5(d) is explicitly a panel excerpt of the simulated embodiment-transfer result; physical panels use different scoring conventions. Original pages use 200 DPI, with 400 DPI for the narrow Figure 5(d) crop and 300 DPI for Table 8. Figure 2 versus Table 9 and Figure 8 versus its prose contain unresolved comparison details recorded in the base report. External code, weights, datasets and linked references were not inspected; no experiments were run.

PDF pages inspected for this edition: 1, 3, 4, 6, 7, 8, 9, 11, 12, 21, 22, 23, 24, 25, 26, 28, 29, 31, 32, 35, 36, 37. Appendix coverage: reviewed.

Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.

Text reading scope & known omissions

Abstract and highlights
1 Introduction
2 Pilot Study: How Action Supervision Shapes VLA Backbones
3 VLAct, including Sections 3.1–3.4
4 Experiments, including Sections 4.1–4.4
5 Related Work
6 Conclusion
Authors
References
A Additional Related Work
B Additional Main Results, including B.1–B.2
C Effectiveness of Shallow-Layer Protection During Pre-training, including C.1–C.2
D VLM Co-training for Representation Preservation, including D.1–D.3
E Representation Effects of Head-Diverse Pre-training, including E.1–E.2
F Unified Joint Space and Wrap-Aware Loss, including F.1–F.2
G Adding Heterogeneous UMI Data to Continued Pre-training
H Details of Data Cleaning
I Details of Action Heads, including I.1–I.2
J Details of Real-world Experiments, including J.1–J.5
K Future Work and Broader Impact
Original diagrams, plots, tables and appendix image sequences visually inspected after complete text reading

Outside the original text pass

Separate supplemental material availability was not verified.
External references, project links, source code, model checkpoints and datasets were not accessed.
No paper experiment was reproduced; visual inspection covers the supplied PDF and its retained figure/table excerpts.
Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
Separate supplemental material availability has not been fully verified.

The visual audit above records the subsequent illustrated pass.

8.4 Traceable evidence

e01PDF p. 1, title and arXiv version line; p. 12, AuthorsInspect

The identifier is 2608.27550v1, dated 27 August 2026. The title and listed contributors and advisors match the supplied catalog identity.

1. Paper overview

Core contributions

2. Motivation

2.1 The problem and the proposed response

2.2 What this reading follows

3. Research context

3.1 Evidence-based assessment

4. Problem formulation

4.1 Inputs and outputs

4.2 Equations and their role

5. Method in detail

5.1 Protect a useful visual prior while making its upper layers action-aware

5.2 Use disagreement between decoder families as a training constraint

5.3 Align action meanings before comparing transfer results

5.4 Training and inference

During training

During inference

5.5 Implementation flow

6. Experiments & results

6.1 Read the original evidence

6.2 Results and evaluation conditions

6.3 Ablations and diagnostic examples

7. Analysis & limitations

7.1 What the evidence leaves open

7.2 Questions for discussion

8. Reproducibility audit

8.1 Requirements and known gaps

8.2 Proposed reproduction checks

Check 1: Does head diversity help an excluded decoder under matched resources?

Check 2: Do captions preserve broad capability or mainly improve policy optimization?

8.3 Reading coverage

Outside the original text pass

8.4 Traceable evidence

8.5 Primary sources

Beyond Data Scaling: Representation-Centric Continued Pre-training for Vision-Language-Action Models ↗