Kairos: A Regret-Aware Native World-Action Model Stack for Physical AI
1. Paper overview
In one sentence: Kairos trains video and action transformers together, retaining an action-only inference path, but its benchmark gains do not yet establish closed-loop regret reduction. regretarchitecturemaskliberotraining-ablationsgeneration-latencyevaluation-boundary
| At a glance | What to know |
|---|---|
| Research problem | Source description A robot needs to retain contact, object state and task progress from partially observed histories. Kairos frames a useful latent state as one that preserves decisions with low physical cost. Representation-induced regret compares decisions available from the compressed state with those available from full history. This motivates the stack but is not its measured training objective. regret |
| Core mechanism | Source description A native mixture of Video DiT and Action DiT learns future video and actions within a shared stack, with an inference path that predicts actions without generating future frames. architecturemask |
| A key reported result | LIBERO-Plus robustness: Kairos 89.0%; Kairos-joint 90.8%. Average success rate ↑. Seven perturbation categories; standard action-only versus joint future-video/action inference. ACoT-VLA 88.0%; π₀.₅ 85.7%. Joint inference adds 1.8 percentage points, but lighting and noise scores decline; its latency cost is unreported. libero |
| Reading caution | Source description The source explicitly leaves imagined-versus-real rollout agreement, counterfactual action validation, failure prediction, safety filtering, recovery and policy improvement from imagination for future tests. No direct representation-induced regret reduction is measured. evaluation-boundary |
Core contributions
- Source description
A native mixture of Video DiT and Action DiT learns future video and actions within a shared stack, with an inference path that predicts actions without generating future frames. architecturemask
- Source description
Sliding-window, dilated-window and gated linear attention divide temporal modeling across local, intermediate and persistent context. Conditional necessity and sufficiency results motivate this factorization. memorymemory-theory
- Source description
A Cross-Embodiment Data Curriculum moves from general visual observation through human activity to robot grounding; deployment work combines few-step distillation and inference optimizations. curriculuminference-efficiency
Figure 4. Two prediction streams share grounding, while the mask determines whether generated futures can influence actions. Original paper, p. 14 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start at the left: the image enters both the VAE and vision-language model, while robot state enters a separate encoder. Green arrows carry conditioning into cross-attention. Snowflakes mark the frozen VAE and VLM components; the flame marks the trainable backbone. Follow the separate visual and action columns through mixed attention and modality-specific gates and feed-forward blocks. Now inspect the small training mask: v₀ denotes observed visual context; future visual rows and action rows each access that context, but their off-diagonal cross-modality cells are blank. The inference mask removes future visual tokens. The shared-attention box therefore needs to be read together with the actual mask. architecturemaskliberoevaluation-boundary
What it supports. The architecture supports learning from video while predicting actions without decoding a future movie. Its visual and action branches retain distinct processing and outputs. The standard mask explains how joint training and action-only deployment coexist; later Kairos-joint experiments deliberately change the future-video access available to actions.
Where the evidence stops. The pink modulation arrows do not establish unrestricted cross-modal attention. Figure 4 and the page 18 mask description support separated future groups; the Kairos-joint inference variant on pages 69–70 permits additional future-video access. Neither diagram proves counterfactual rollout accuracy.
2. Motivation
2.1 The problem and the proposed response
A robot needs to retain contact, object state and task progress from partially observed histories. Kairos frames a useful latent state as one that preserves decisions with low physical cost. Representation-induced regret compares decisions available from the compressed state with those available from full history. This motivates the stack but is not its measured training objective. regret
2.2 What this reading follows
Kairos asks what a robot must remember to choose useful actions under partial observation. Its answer combines visual generation, language grounding, action prediction and memory at several temporal scales. The key reading distinction is between learning with future video and consulting generated video during control: the standard policy does the former, while the Kairos-joint variant also does the latter. The visuals below trace that distinction through the architecture, manipulation results and ablations, then examine the measured cost of video generation. They also locate the evidence boundary: plausible videos and successful benchmark actions are prerequisites for the paper's regret-aware ambition, not a direct measurement of it. regretarchitecturemaskliberotraining-ablationsgeneration-latencyevaluation-boundary
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | WAMs |
| Architecture | Dual-system |
| Prediction paradigm | Joint prediction |
| Quadrant | Q3 · Dual-system × Joint prediction |
3.1 Evidence-based assessment
Supports the recorded classification
Video DiT and a smaller Action DiT support the recorded Dual-system architecture; synchronized future-video/action objectives support Joint prediction. Hybrid attention and generation acceleration support the three subcategories. Joint training does not make this a single undifferentiated model, and action-only inference qualifies how joint prediction is deployed. architecturemaskjoint-trainingmemoryinference-efficiency
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Follow a training target through to an action
Begin with a synchronized demonstration: observed images and an instruction provide context, while future video and action chunks supply training targets. Flow matching corrupts the targets with noise and teaches velocity predictors to recover their structure. The joint loss adds video and action terms, with a balancing coefficient whose numerical value is not given. Crucially, the standard attention mask lets actions use observed history without reading future-video tokens. Reader interpretation: a video objective can shape useful shared representations even when the deployed action path never renders a movie. This explains why the training ablation and Kairos-joint inference comparison answer different questions. The former asks whether visual prediction helps learn actions; the latter asks whether giving actions access to generated futures adds further value at test time. architecturemaskflowjoint-traininglibero
Figure 5. Local windows, spaced temporal links and a recurrent memory provide different routes through history. Original paper, p. 19 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the center stack first: two sliding-window blocks are followed by one dilated-window block and one gated-linear block, and the group repeats. The upper-right map is a band around the diagonal, retaining nearby temporal relationships. The lower map places connections farther apart, representing the dilated pathway. These are centered temporal masks, not a strictly lower-triangular autoregressive mask; the local-window equation on page 22 agrees with that reading. The left panel places these operators inside a diffusion block with conditioning, timestep modulation and residual additions. Global history is handled by gated linear attention, whose explicit recurrent update appears on pages 20–21 rather than in the two window maps. memorymemory-updatememory-theorylong-horizonfuture-memory
What it supports. This factorization assigns dense nearby interactions to local attention and longer retention to a compact recurrent state. It offers a mechanism for preserving information beyond a fixed recent window. The appendix explains why such information can be necessary, but the figure itself does not establish which branch causes the reported long-video advantage.
Where the evidence stops. The sufficiency theorem assumes an appropriate predictor factorization, Lipschitz decoding and uniform contraction; these are not verified checkpoint properties. The source proposes branch-specific memory ablations as future work, so the 15-second generation result cannot isolate GLA's contribution.
5.2 Understand what persistent memory can—and cannot—guarantee
Imagine two histories that end in the same recent view but differ in where an object was placed earlier. This is a reader's example of the source's partial-observation problem: a window-only predictor cannot distinguish histories once the relevant event leaves its window. The appendix formalizes the resulting excess squared-error risk through a difference in conditional expectations. Kairos's gated memory offers a route to retain earlier information: it reads the old value associated with a key, discounts the state and writes a correction. Local and dilated attention handle nearer dependencies. The sufficiency theorem then requires the true predictor to admit this factorization and the memory to evolve contractively under additional assumptions. It does not show that a trained checkpoint actually remembers the right object or reduces execution cost. regretmemorymemory-updatememory-theoryfuture-memory
5.3 Build an evidence ladder from videos to control
Start with Table 7: higher instruction-following and physics scores show that generated robot videos satisfy benchmark criteria. Move to LIBERO-Plus: action success under perturbations is stronger evidence about control, and Tables 14–15 connect that performance to specific training choices. Now ask whether either evaluation measures the paper's physical-cost regret. Neither compares decisions from the learned state against decisions from full history under matched real outcomes. The source explicitly proposes imagined-versus-real comparisons, alternative actions from the same initial state and failure/recovery evaluations for that next step. Reader interpretation: the present results support useful components of a regret-aware stack, while the defining closed-loop claim remains open. Generation latency adds a separate engineering dimension and cannot fill the missing decision-quality evidence. wmb-robotliberotraining-ablationsgeneration-latencyevaluation-boundary
5.4 Training and inference
During training
Stage I trains latent video flow matching, progressing from 256P images through 256P/480P/720P video to 241-frame, approximately 15-second clips. AdamW learning rates decrease across stages; domain SFT and preference refinement follow. Stage II trains Video DiT on human/robot clips and shifts detailed captions toward instructions; complete robot action grounding is deferred to Stage III. flowcurriculumtraining-settings
Stage III describes execution-preference alignment using failures, recovery and safety-related comparisons, then synchronized video-action flow matching. Action DiT is initialized by interpolating video weights. The action branch uses a fixed timestep shift while video uses a dynamic shift. Joint loss weighting is symbolic; its numerical coefficient is unspecified. joint-training
The data engine mixes public sources including Koala-36M, OpenHumanVid, VidGen, AgiBotWorld-Beta and DROID with proprietary web and human recordings. It filters, deduplicates and captions clips. Raw collection scale is not an exact training mixture; explicit control-information-density scoring remains a future extension. data
During inference
Standard control inference removes future-video tokens and predicts actions from observed history. Kairos-joint instead permits actions to attend to generated future-video tokens and jointly denoises both streams. The report gives no end-to-end action-only control latency. maskliberodeployment-boundary
Generation deployment combines four-step distillation, parallelism, caching, compilation and quantization. Proxy self-evolution iteratively rewrites prompts using generated-video rewards; it is not demonstrated robot-policy improvement. Distillation quality is mainly qualitative, with systematic teacher comparisons left open. inference-efficiencyproxy-loop
5.5 Implementation flow
- Encode and ground
A frozen VAE encodes observations; a frozen VLM encodes image/instruction context, alongside a robot-state encoder. The resulting visual tokens and conditioning features implement the proposed state representation. Their control sufficiency is an intended property, not a separately verified state variable. architectureregret
- Predict through modality-specific transformers
Video DiT supplies a pretrained visual backbone; the smaller Action DiT uses modality-specific processing within mixed attention. Figure 4's standard mask lets both future groups use history but blocks future-video/action cross-attention. Shared training therefore does not imply that actions read generated futures. architecturemask
- Maintain temporal context
Repeated groups contain two sliding-window blocks, one dilated sliding-window block and one gated linear block. Dilations of 6 and 12 cover intermediate dependencies; only the gated linear branch provides global attention. Gated DeltaNet retrieves an old value, decays memory and writes a value correction. memorymemory-update
6. Experiments & results
Kairos couples a video diffusion transformer with an action transformer, sharing visual history and language grounding. Joint video-action training improves manipulation results, while hybrid attention supports longer video histories and action-only inference avoids generating future video. The evidence establishes benchmark capabilities and generation efficiency; it does not measure representation-induced regret or validate a closed-loop robot learning system.
6.1 Read the original evidence
Table 7. A small composite-score lead reflects several video-quality dimensions, not a direct measure of physical control. Original paper, p. 63 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Locate the gray Kairos row, then read across rather than jumping immediately to Total Score. Instruction Following measures compliance; Physics Adherence separates Newtonian, deformation, fluid, penetration and gravity checks before reporting its overall score. Common Sense adds frame and temporal measures. Kairos and Cosmos3-Nano tie at 2.36 instruction following and 4.96 physics; their temporal entries differ, at 1.00 and 0.96. The total consequently favors Kairos, 9.30 to 9.26. Compare GigaWorld-0 as a useful counterexample: its 4.98 physics score exceeds Kairos's despite a lower overall score. Retain the asterisk note: these baseline runs were reproduced by the paper's team, not by this reading. wmb-robotevaluation-boundary
What it supports. Kairos-4B achieves the highest total in this robot-video table while matching a larger model on instruction following and aggregate physics. This supports a favorable parameter/performance comparison under the reported protocol. It also shows why describing Kairos as uniquely best at physics would overstate what the component columns say.
Where the evidence stops. This is text-and-image-conditioned video evaluation, not executed robot success or a calibrated regret estimate. No uncertainty interval accompanies the 0.04 total-score margin; the composite's numerical advantage should not be promoted into demonstrated safety or causal dynamics fidelity.
Table 13. Generating future video at inference raises average robustness, with uneven gains across perturbations. Original paper, p. 69 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read the seven perturbation columns before comparing the Average column. The bottom two shaded rows distinguish standard Kairos from Kairos-joint, which the surrounding text defines by allowing action tokens to attend to generated future visual tokens during joint denoising. Language changes from 86.8 to 95.3, while robot and layout scores also rise. Lighting falls from 97.7 to 97.1 and noise from 96.8 to 95.4, so the improvement is not uniform. Compare the baselines by category as well: π₀.₅ leads the robot and layout columns even though its average is lower. Dashes in other rows remain missing category results and must not be read as zero. liberotraining-ablationsdeployment-boundary
What it supports. The average rises from 89.0% to 90.8%, a 1.8-percentage-point gain when generated future video becomes available during inference. This is evidence that the inference variant can improve this benchmark. It is a different comparison from training with a video loss, whose much larger ablation gain appears in Tables 14–15.
Where the evidence stops. The table does not report the latency or memory cost of joint inference, nor confidence intervals. Its average therefore cannot determine whether consulting generated video is worthwhile under a particular robot's control deadline or whether the small margin is statistically stable.
Table 5. Video generation becomes much faster, but the table does not time an action-only robot policy. Original paper, p. 58 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Use the protocol on pages 58–59 to anchor this crop: the comparison generates five-second, 720P text-and-image-conditioned video on NVIDIA A800 hardware. The memory and complexity columns have different units from the final two latency columns. Compare Kairos with the similarly sized Wan2.2-5B row first: latency falls from 201 to 43 seconds on one GPU and from 85 to 9 seconds on four, while memory is nearly unchanged at 23.4 versus 23.5 GB. The much larger models occupy the upper rows. Do not mix these numbers with neighboring Table 4, which measures a distilled robot model at 480P and includes RTX 5090 hardware. generation-latencydistilled-latencydeployment-boundary
What it supports. The table supports a substantial generation-latency advantage in the stated setting, alongside lower reported computational complexity. It does not show a universal memory advantage: Wan uses slightly less. The four-GPU time of nine seconds refers to producing a five-second video, with a different workload from predicting an action chunk.
Where the evidence stops. These timings measure video generation. The paper explicitly leaves end-to-end action-only latency unreported, and it provides no joint-inference control timing. Inferring a robot control rate from seconds per generated video would combine incompatible workloads.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| LIBERO-Plus robustness Seven perturbation categories; standard action-only versus joint future-video/action inference. | Kairos 89.0%; Kairos-joint 90.8%. Average success rate ↑ | ACoT-VLA 88.0%; π₀.₅ 85.7%. Joint inference adds 1.8 percentage points, but lighting and noise scores decline; its latency cost is unreported. libero |
| LIBERO-Plus training ablations Human pretraining and joint branch training are separate comparisons. | 83.0% → 89.0% with human pretraining; 65.8% → 89.0% with joint Video/Action DiT training. Average success rate ↑ | Differences are +6.0 and +23.2 percentage points. The latter changes training branches/objectives; it is distinct from enabling future-video attention at inference. training-ablations |
| RoboTwin 2.0 manipulation Bimanual benchmark; clean and randomized settings. | 96.9% clean; 95.2% random; 96.1% average. Success rate ↑ | MotuBrain: 95.8%, 96.1%, 96.0%. Kairos leads the reported average by 0.1 point but trails on randomization; no uncertainty is supplied. robotwin |
| WorldModelBench robot video Robot subset; text-and-image-conditioned video generation. | 9.30 / 2.36 / 4.96. Total / instruction-following / physics score ↑ | Cosmos-3-Nano-16B: 9.26 / 2.36 / 4.96; GigaWorld-0 physics: 4.98. Higher composite score does not mean uniquely best physics or measured robot success. wmb-robot |
| DreamGen robot video Novel objects, behaviors and environments with GR1 examples. | 0.618. Average aggregate score ↑ | Wan2.2-14B: 0.611; instruction following 0.703 versus Kairos 0.698. Overall improvement coexists with slightly lower instruction following. dreamgen |
| PAI-Bench robot video Robot subset; text-and-image-conditioned video. | 82.57. Final score ↑ | Wan2.2-5B 78.63; Cosmos-3-Nano-16B 82.62; ABot-PhysWorld + DPO 84.91. Competitive parameter efficiency, not the highest overall score. pai-robot |
| General world-model generation Separate PAI-Bench, WorldModelBench and VideoPhy protocols. | PAI 80.8; WMB TI2V 8.89 and T2V 8.99; VideoPhy 45.55. Reported overall scores ↑ | Cosmos-14B: 81.0; 8.95 and 9.09; 45.16, respectively. Kairos does not lead every benchmark. VideoPhy uses 344 prompts and model-specific prompt enhancement; these scores are not interchangeable. general-evalvideophy |
| Long-horizon PAI-Bench generation 15-second text-and-image-conditioned clips; compare corresponding five-second results. | 79.9 at 15 seconds, versus 80.8 at five seconds. Final score ↑ | 15-second Wan2.2-5B 77.8; Cosmos-2B 77.2; Cosmos-14B 76.2. The 0.9-point decline supports temporal video consistency, not task memory over minutes or an isolated GLA benefit. long-horizon |
| Video generation efficiency Table 5: 720P five-second TI2V on NVIDIA A800 hardware. | Kairos: 43 s on one GPU, 9 s on four; 23.5 GB; 2.3 PFlops. Latency ↓; memory footprint ↓ | Wan2.2-5B: 201/85 s, 23.4 GB, 16.6 PFlops. Large measured generation speedup, with slightly higher memory than Wan; not an action-control timing measurement. generation-latency |
| Distilled robot video generation Table 4: 480P five-second distilled robot model. | A800: 11.7 s on one GPU, 3.0 s on four. Generation latency ↓ | RTX 5090: 11.4 s on one GPU, 5.7 s on four. Faster-than-duration generation on four A800s does not establish a closed-loop robot control frequency. distilled-latency |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Tables 14 and 15. Human pretraining and joint video-action training test two distinct contributors to manipulation performance. Original paper, p. 70 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Treat the left and right tables as separate experiments. Table 14 removes or includes human-centric pretraining, and the surrounding text states that other settings are fixed; the average rises from 83.0 to 89.0. Table 15 contrasts training only Action DiT with jointly training Video DiT and Action DiT, yielding 65.8 and 89.0. Its row label 'Video Generation & Action Prediction' names the training configuration, not a requirement to generate a video every time the robot acts. Read the gain entries as percentage-point differences. Both tables end at the standard 89.0 configuration, which is distinct from the 90.8 Kairos-joint inference result. training-ablationsjoint-traininglibero
What it supports. The reported gains are +6.0 points for human-centered pretraining and +23.2 points for joint generation/prediction training. These are the strongest local evidence that human-centered pretraining and the video objective contribute to downstream action performance. They make the architecture's training rationale more concrete than a leaderboard comparison alone.
Where the evidence stops. Joint training changes active branches and objectives, so this ablation does not independently equalize representational capacity or compute. The tables provide no uncertainty estimates or complete matching recipe; reproducing the gain requires controlling those factors rather than merely copying the row names.
7. Analysis & limitations
7.1 What the evidence leaves open
The source explicitly leaves imagined-versus-real rollout agreement, counterfactual action validation, failure prediction, safety filtering, recovery and policy improvement from imagination for future tests. No direct representation-induced regret reduction is measured. evaluation-boundary
Memory sufficiency assumes a suitable Bayes-predictor factorization, Lipschitz decoding and uniformly contractive gated updates. The appendix's key-norm and contraction conditions are not empirically verified for the checkpoint. Fifteen-second generation does not establish persistent task memory. memory-theorylong-horizonfuture-memory
Benchmarks provide no confidence intervals for the highlighted small margins. The human-data comparison holds the encoder fixed; the separate encoder comparison changes the conditioning model. Figure 19 labels Lingbot-14B while its surrounding comparison text names Lingbot-28B; that baseline identity is unresolved. human-encoderhuman-labelrobotwinlibero
Proprietary data, incomplete mixture specification and caption/filter limitations restrict reconstruction. Execution-preference alignment and proxy self-evolution lack controlled evidence of their separate contribution to physical regret. datajoint-trainingproxy-loopevaluation-boundary
7.2 Questions for discussion
- Does the joint-training advantage survive matched data, compute and action-branch capacity?
- When does the 1.8-point joint-inference gain justify generating future video?
- Which retained state variables predict physical cost beyond what video-quality metrics capture?
- Would GLA improve delayed task-state decisions when recent visual observations are identical?
8. Reproducibility audit
8.1 Requirements and known gaps
A faithful implementation needs matched VAE/VLM and video/action initializations, synchronization and token masks, action representation and chunk horizon, loss weight, data mixture and optimization budgets. The report supplies staged learning rates but not a complete executable recipe, total training compute, or all task-specific settings. architecturetraining-settingsjoint-trainingdatalibero
Proposed checks should separate joint training from future-video use at inference and test memory under delayed partial observation. Report matched seeds, task-level success and control latency; do not use generation timing as a substitute. The illustrated edition specifies two falsifiable comparisons. masktraining-ablationsdeployment-boundaryfuture-memory
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Separate joint training from future-video use at inference
Proposed check, not performed: compare an action-only-trained model with a jointly trained model, then evaluate the latter with future-video access disabled and enabled. Hold the VLM/VAE, action-branch capacity, demonstrations, splits, random seeds and evaluation resets fixed. Run a matched-update comparison and a separately matched-compute comparison, documenting the joint loss weight. Measure success in all seven LIBERO-Plus perturbations, paired uncertainty, peak memory and end-to-end action-chunk latency on the same hardware. A persistent joint-training advantage would support the reported mechanism; disappearance under controlled budgets would weaken that attribution. A joint-inference gain that breaches the control deadline would expose a deployment tradeoff hidden by the average-success table. architecturemaskjoint-trainingtraining-ablationsliberodeployment-boundary
Check 2: Test whether gated memory preserves delayed task state
Proposed check, not performed: construct paired tasks whose recent observations are matched but whose earlier object placements or completed subtasks differ. Increase the delay until the relevant event lies beyond the local and dilated windows. Compare full Kairos with a model retrained without GLA under matched data and compute, retaining the same local-window settings; use an always-visible condition as a control. Separately reset the full model's recurrent state at test time as a diagnostic, reporting that this creates a distribution shift. Measure object/subtask-state probe accuracy, action success and latency by delay. A selective, growing full-model advantage after the informative event leaves the windows would support persistent-memory utility; equal degradation would challenge that explanation of the long-horizon result. memorymemory-updatememory-theorylong-horizonfuture-memory
8.3 Reading coverage
Visual audit: All 31 supplied text chunks covering PDF pp. 1–119 were read individually. The declared pages were rendered from the supplied PDF and actually viewed, including identity, architecture/masks, training/evaluation protocols, all retained result tables, deployment boundaries and the appendix conditions used in the interpretation. Six final original PNG crops were each viewed for complete labels and legibility. Figure 4's frozen-module markers and masks were checked against pages 14–16 and 18 and the changed inference mask described on pages 69–70. Figure 5's centered windows were checked against page 22; recurrent updates and conditional theoretical claims were checked against pages 20–21 and Appendix B. Table 7 retains its author-reproduction asterisk note; the other crops retain full relevant headers and rows. Page 75's generated sequences were visually inspected despite little extracted text. Figure 19's Lingbot-14B label conflicts with the surrounding Lingbot-28B text, as disclosed in the base report. Images on pages outside this list were not claimed as visually reviewed. No external supplements or executable artifacts were inspected. Training progression was checked in Table 1 on page 31; AdamW settings were checked in prose on pages 32 and 34. Table 2 on page 44 contains domain tags. DreamGen baseline labels and values were checked against pages 63–64, which identify Wan2.2-14B. Pages 42–46 were inspected for filtering, deduplication, domain tags and the captioning pipeline.
PDF pages inspected for this edition: 1, 5, 6, 14, 15, 16, 18, 19, 20, 21, 22, 24, 27, 29, 31, 32, 33, 34, 36, 37, 40, 41, 42, 43, 44, 45, 46, 49, 52, 53, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 78, 79, 85, 87, 104, 108, 111, 113, 115, 116, 119. Appendix coverage: reviewed.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Title, abstract and overview; complete supplied text, PDF pp. 1–119
- 1 Introduction
- 2 Model Architecture, including world understanding, generation, prediction, hybrid attention and theoretical analysis
- 3 Native Pretraining Paradigm for Physical AI: Cross-Embodiment Data Curriculum, all three stages, objectives and parallelism
- 4 Data Engine: collection, filtering, annotation, synthesis and limitations
- 5 Inference: proxy self-evolution, distillation, acceleration and deployment boundaries
- 6 Evaluation: embodied and general world models, WAM benchmarks, ablations, long-horizon generation and missing closed-loop tests
- 7 Related Work
- 8 Conclusion and Future Works, including the outlook
- References
- Appendix A Contributors
- Appendix B Theoretical Analysis, including problem setup, necessity proof and approximate-sufficiency proof
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Separate supplemental material availability has not been fully verified.
- Pages with little extracted text require visual inspection: 75
- All 31 supplied text chunks were read individually without truncation. Selected original PDF pages and all six final crops were visually inspected; page 75 was inspected and contains video-sequence imagery. This resolves that page's extraction caution, but does not imply every figure on all 119 pages was visually inspected.
- The inspected PDF is arXiv:2606.16533v3. Page 1 shows the arXiv date 3 Jul 2026 and a separate document date July 7, 2026; the source does not reconcile those dates. Its main title is Kairos: A Regret-Aware Native World-Action Model Stack for Physical AI, with the subtitle Learning, Maintaining, and Deploying Control-Sufficient World States.
- The title-page byline is Kairos Team. Page 104 separately lists 23 individuals under advisor, project-lead, core-contributor and contributor roles; acknowledgments are separate. The team byline is preserved as the verified author string. ACE Robotics branding is visible on page 1, but individual affiliations are not specified.
- The acquisition record identifies a previous HTML source as the same v3 revision, with SHA-256 ac1cd4a5163d6a7beb3232d84832f18dae9a920c99124d42eb191e8aa6413769. This reading uses only the supplied PDF, whose raw hash differs; the previous HTML and earlier revisions were not independently compared.
- Code, model weights, linked resources and external supplements were not inspected; no paper code or experiments were executed.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
identityPDF p. 1, title block and arXiv sidebar; p. 104, Appendix A Contributors
Page 1 displays the exact main title Kairos: A Regret-Aware Native World-Action Model Stack for Physical AI, the subtitle Learning, Maintaining, and Deploying Control-Sufficient World States, and the byline Kairos Team. The sidebar identifies arXiv:2606.16533v3, 3 Jul 2026; the document date is July 7, 2026. ACE Robotics branding is visible. Page 104 lists 23 individuals across advisor, project-lead, core-contributor and contributor roles, separately from acknowledgments; individual affiliations are not specified.
Go to primary source ↓regretPDF pp. 5–6, Section 1, control-sufficient state and representation-induced regret
A state compresses observation-action history for task-conditioned decisions. Physical cost includes failure, safety, recovery and time. Regret compares achievable decisions from compressed state and full history; the formulation motivates the system.
Go to primary source ↓architecturePDF pp. 14–16, Figure 4 and Section 2.1; p. 18, Section 2.1.3
The diagram shows frozen VAE encoding/decoding and VLM conditioning, robot-state encoding, visual/action token groups and a trainable mixed transformer backbone. Text distinguishes Video DiT from Action DiT and the structured latent state.
Go to primary source ↓maskPDF p. 14, Figure 4 attention masks; p. 18, Section 2.1.3; p. 37, action-branch initialization
Standard training masks share observed history while separating future-video and future-action attention. The smaller Action DiT is initialized by interpolating Video DiT weights. Action-only inference removes the future-video branch.
Go to primary source ↓memoryPDF pp. 19–22, Figure 5 and Section 2.2, Eq. (7)
Hybrid groups have two SWA blocks, one DSWA block and one GLA block. Local window masks are centered bands; DSWA uses dilations 6 and 12. GLA is the global-attention component.
Go to primary source ↓memory-updatePDF pp. 20–21, Eqs. (2)–(6), Figure 6
Gated DeltaNet projects keys, values and queries, retrieves S_(t−1)k_t, decays state through α_t and writes a β_t-scaled value correction. Figure 6 shows gates, normalization and recurrent-state structure.
Go to primary source ↓memory-theoryPDF p. 24, Section 2.3; pp. 108, 111, 113, 115–116, 119, Appendix B, Theorems 3–4 and Lemma 3
Window-restricted excess risk equals the squared conditional-mean gap. Approximate sufficiency assumes shared/local/mid/global factorization and Lipschitz decoding. Lemma 3 requires a small-key norm relative to the decay gate; the proof assumes a uniform contraction factor below one. Remark 10 denies a checkpoint performance guarantee.
Go to primary source ↓curriculumPDF p. 27, Section 3 overview; pp. 31–34, Stages I–II
Curriculum proceeds through broad visual observation, human-centered activity and robot grounding. Stage I increases spatial/temporal scale; Stage II trains Video DiT on human/robot clips and transitions toward robot instructions.
Go to primary source ↓flowPDF p. 29, Section 3.1, Training Objective, Equations 21–23
Clean VAE latents interpolate linearly to Gaussian noise; the target velocity is noise minus clean latent. The conditioned velocity network minimizes squared error.
Go to primary source ↓training-settingsPDF p. 31, Table 1; p. 32, Stage I optimizer prose; pp. 33–34, Stage II descriptions and optimizer prose
Table 1 lists task type, stage, resolution and maximum frames: image training begins at 256P, mixed video reaches 720P with 81 frames, and continual training, domain SFT and preference training use 241 frames. AdamW settings are reported in prose, not in the table: Stage I rates progress from 5e-5 for images through 4e-5, 3e-5 and 2e-5 for increasing mixed-video resolutions to 1e-5 for continual training; weight decay changes from 1e-3 to zero. Stage II uses learning rates 1e-5, 5e-6 and 1e-6 with zero weight decay.
Go to primary source ↓joint-trainingPDF pp. 36–37, execution-preference alignment and joint video-action training, Eq. (28)
Stage III describes preference comparisons involving execution outcomes. Joint flow matching sums video loss and λ-weighted action loss, with interpolated video-to-action initialization and different timestep-shift strategies. A numerical λ and complete execution-preference recipe are not supplied here.
Go to primary source ↓dataPDF pp. 40–41, Section 4.1; pp. 42–43, filtering and deduplication; pp. 45–46, captioning; p. 49, data-engine limitations
Public video/robot datasets are mixed with proprietary sources. Scene filtering, quality selection and deduplication form the pipeline. Explicit control-information-density selection is prospective; heterogeneous data and imperfect captions remain limitations.
Go to primary source ↓proxy-loopPDF pp. 52–53, Figure 14 and Sections 5.1–5.2; p. 56, distillation quality boundary
Proxy self-evolution loops through prompt refinement, generation, reward and feedback. It operates on generation quality and task/physics proxies; controlled real-policy improvement is not shown. Distillation quality assessment remains mainly qualitative.
Go to primary source ↓inference-efficiencyPDF pp. 53, 56–58, Sections 5.3.1–5.3.3
The generation stack uses four-step distillation plus sequence/tensor parallelism, caching, compilation, kernel optimization and quantization. These are deployment techniques for video generation.
Go to primary source ↓generation-latencyPDF p. 58, Table 5; p. 59, benchmark hardware and protocol
For 720P five-second TI2V on NVIDIA A800 servers, Kairos-4B reports 23.5 GB, 2.3 PFlops and 43/9 seconds on one/four GPUs. Wan2.2-5B reports 23.4 GB, 16.6 PFlops and 201/85 seconds; larger baselines are slower.
Go to primary source ↓distilled-latencyPDF p. 58, Table 4
For distilled 480P five-second robot generation, Kairos reports 11.7/3.0 seconds on one/four A800 GPUs, and 11.4/5.7 seconds on one/four RTX 5090 GPUs. Memory is 23.5 GB and 13.9 GB, respectively.
Go to primary source ↓deployment-boundaryPDF p. 60, Section 5.4
Video-generation acceleration is distinct from robot-control latency. End-to-end action-only inference latency and complete closed-loop deployment evaluation are not reported.
Go to primary source ↓wmb-robotPDF pp. 61–63, evaluation protocol and Table 7; p. 71, TI2V protocol clarification
Kairos-robot-4B scores IF 2.36, physics 4.96, frame 0.98, temporal 1.00 and total 9.30. Cosmos-3-Nano-16B totals 9.26 with the same IF/physics but temporal 0.96. GigaWorld-0 has physics 4.98. Table stars denote results reproduced by the paper's authors.
Go to primary source ↓dreamgenPDF pp. 63–64, DreamGen protocol and Table 8
Kairos averages physical adherence 0.538, instruction following 0.698 and aggregate 0.618 across novel-object/behavior/environment evaluations. Wan2.2-14B averages 0.519, 0.703 and 0.611. Table sources mix reported and author-reproduced results.
Go to primary source ↓pai-robotPDF pp. 64–65, PAI-Bench robot protocol and Table 9
Kairos final score is 82.57 with domain score 88.59. Wan2.2-5B scores 78.63, Cosmos-3-Nano-16B 82.62 and ABot-PhysWorld + DPO 84.91. The final metric combines quality and domain scores.
Go to primary source ↓human-encoderPDF pp. 66–68, Table 10 and human-data/encoder discussions
With Qwen2.5-VL-7B fixed, human pretraining raises robot-WMB IF 2.10 to 2.33 and total 9.08 to 9.25. The Qwen3.5-2B variant reaches IF 2.36 and total 9.30, changing the encoder as well.
Go to primary source ↓human-labelPDF pp. 65–67, human evaluation text and Figure 19
The source describes ten volunteers, anonymized/randomized presentation and averaged rankings. Text names Lingbot-28B, whereas Figure 19's comparison label says lingbot-14B; the discrepancy is unresolved.
Go to primary source ↓robotwinPDF pp. 67–68, RoboTwin 2.0 description and Table 12
Kairos reports clean/random/average success 96.9/95.2/96.1 percent. MotuBrain reports 95.8/96.1/96.0; Kairos does not lead the random setting. Task-level uncertainty is not shown.
Go to primary source ↓liberoPDF pp. 68–70, LIBERO-Plus protocol, Table 13 and inference-variant discussion
Seven perturbation categories yield Kairos 89.0 and Kairos-joint 90.8 percent average success. The joint inference variant permits actions to attend to generated future visual tokens. Lighting falls 97.7 to 97.1 and noise 96.8 to 95.4; language rises 86.8 to 95.3. ACoT-VLA averages 88.0 and π₀.₅ 85.7.
Go to primary source ↓training-ablationsPDF pp. 69–70, LIBERO-Plus ablation descriptions and Tables 14–15
Human pretraining raises success from 83.0 to 89.0 percent with other settings described as fixed. Training only Action DiT scores 65.8; jointly training Video DiT and Action DiT scores 89.0. These are training comparisons, separate from Kairos-joint inference.
Go to primary source ↓general-evalPDF pp. 70–72, Tables 16–17 and protocols
Five-second general PAI scores are Kairos 80.8, Wan 80.4 and Cosmos 81.0. WorldModelBench TI2V scores are Kairos 8.89 versus Cosmos-14B 8.95; T2V scores are 8.99 versus 9.09.
Go to primary source ↓videophyPDF p. 73, Section 6.4.3 and Table 18
VideoPhy evaluates 344 test prompts with VIDEOCON-PHYSICS and counts simultaneous semantic adherence and physical commonsense. Kairos scores 45.55 versus Cosmos-14B 45.16. Best results with/without model-specific prompt enhancement are used; enhancement models differ.
Go to primary source ↓long-horizonPDF pp. 74–76, Section 6.5, Figure 24 and Table 19; p. 70, Table 16 five-second reference
The 15-second PAI evaluation scores Kairos 79.9, Wan 77.8, Cosmos-2B 77.2 and Cosmos-14B 76.2. Kairos's corresponding five-second score is 80.8. The sequence figure is qualitative generated video, not a measured robot rollout.
Go to primary source ↓evaluation-boundaryPDF pp. 61–62, Table 6; pp. 78–79, Section 6.8 and Table 20
Current benchmarks test component/proxy capabilities. Direct rollout correlation, alternative-action validation, failure prediction, safety filtering, recovery, policy improvement and calibration are proposed future evaluations, not measured representation-induced regret.
Go to primary source ↓future-memoryPDF p. 87, Section 8.1.4
The source calls for memory evaluation beyond 15-second generation, including object permanence, subtask state, delayed effects and long-horizon action success, with separate SWA/DSWA/GLA ablations.
Go to primary source ↓8.5 Primary sources
Kairos: A Regret-Aware Native World-Action Model Stack for Physical AI ↗
PDF · 50,939 extracted words
Source fingerprint
ba46b4ca75dd3800d6ae338fcbb76840d5ce8ee4e414bb4788c39e2e1ee70c6a