PhyAI: Real-Time Physical AI at the Edge, Scalable Rollouts in the Cloud
1. Paper overview
In one sentence: PhyAI shares an execution runtime across existing VLA and WAM policies, accelerating model inference while leaving control deadlines and end-to-end rollout gains dependent on the surrounding system. e-purposee-architecturee-batch-definitione-phase-diagnostice-parallele-groot-successe-rl-profilee-rl-intro
| At a glance | What to know |
|---|---|
| Research problem | Source description A checkpoint may pass through evaluation, RL rollout, edge serving and onboard execution, each with a separate implementation. PhyAI addresses this duplication while accommodating reusable VLA conditions and evolving WAM latents. Its design target is a consistent inference path across deployment settings. e-purposee-architecture |
| Core mechanism | Source description A scheduler–runner–Layers ownership hierarchy shares execution services while preserving architecture-specific preprocessing, cache validity, solver order and action conversion. e-architecture |
| A key reported result | Single-request Cosmos3-Nano-Policy-DROID inference: 1180 ms; 2.08× speedup. Latency and official-to-PhyAI speedup. Eight H20 GPUs; CFG parallel degree 2, TP degree 4; model-runner latency. Official path: 2460 ms. A matched-hardware runtime improvement; no matched FlashRT result is available for this configuration. e-latencye-settings |
| Reading caution | Reader analysis Internal v3 differences remain unresolved: the introduction instead describes an eight-A100 rollout simulation projecting 26.5% lower latency and 1.36× throughput; Section 5.4 uses four A800 GPUs. Section 6.3 still calls usable RLinf integration future work. These accounts must not be merged. e-rl-introe-rl-profile |
Core contributions
- Source description
A scheduler–runner–Layers ownership hierarchy shares execution services while preserving architecture-specific preprocessing, cache validity, solver order and action conversion. e-architecture
- Source description
The control-time Roofline relates inference speed to the next action handoff; phase profiles explain why batching and within-request parallelism suit different workloads. e-controle-phase-diagnostic
Figure 3. Request state belongs to runners; execution primitives are shared beneath the model-specific computation. Original paper, p. 8 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Start at the scheduler and follow its downward branches into the two example runners. The scheduler chooses placement and parallel policy; each runner owns the KV cache, CUDA Graph buckets and buffers shown inside its box. The smaller modeling boxes are explicitly stateless. Follow their arrows into Layers, where distributed and fused operators share a selector. Its rules choose either phyai-kernel or framework/library kernels before execution reaches hardware. The left-hand quantizer has a dashed offline connection, whereas preprocessing and post-processing are utility services. This is an ownership diagram: the arrows do not describe a learned perception-to-action network. e-architecturee-optimizations
What it supports. The useful abstraction is separation of policy semantics from execution choices. A model can keep its conditioning and solver procedure while the runtime changes kernel selection or placement. Stateful runner ownership also gives cached tensors and graph buffers a defined lifetime, which is essential for safely reusing work.
Where the evidence stops. Shared runtime services do not imply shared model weights or one WAM architecture. The diagram specifies responsibilities, but does not by itself demonstrate numerical equivalence across kernels, dtypes or distributed configurations.
2. Motivation
2.1 The problem and the proposed response
A checkpoint may pass through evaluation, RL rollout, edge serving and onboard execution, each with a separate implementation. PhyAI addresses this duplication while accommodating reusable VLA conditions and evolving WAM latents. Its design target is a consistent inference path across deployment settings. e-purposee-architecture
2.2 What this reading follows
A robot policy does more than call a transformer: it prepares observations, reuses conditions, iterates a solver and converts outputs into actions. PhyAI organizes those responsibilities so the same model path can serve onboard robots, shared accelerators and cloud rollouts. Read this paper through three boundaries: model semantics versus runtime services, batch throughput versus request wait, and inference speedup versus whole-system improvement. The figures explain why a small VLA action expert can dominate latency, while a WAM needs parallelism inside one request. The appendices add simulation checks, but the RL projections and internal differences in this v3 source require care. e-purposee-architecturee-batch-definitione-phase-diagnostice-parallele-groot-successe-rl-profilee-rl-intro
3. Research context
We place the paper in the collection through its world–action interface. The catalog labels and the reading’s assessment are shown separately.
| Catalog dimension | Recorded classification |
|---|---|
| Major category | Foundational work |
| Architecture | Not applicable |
| Prediction paradigm | Not applicable |
| Quadrant | Not applicable |
This table preserves the labels recorded at reading time. The current major category is Related resources. View the current classification.
3.1 Evidence-based assessment
Supports the recorded classification
The technical-resource placement and architecture/prediction/quadrant 'Not applicable' fit an inference infrastructure paper supporting heterogeneous policies. Shared runtime code does not establish a One Model WAM. The training-optimization tag applies indirectly through rollout acceleration; PhyAI proposes no distillation method. e-purposee-architecturee-optimizationse-rl-profile
This is the collection’s architectural analysis, not a new related-work survey. Benchmark comparisons and their protocols appear in Section 6.
4. Problem formulation
4.1 Inputs and outputs
| Inputs | Outputs |
|---|---|
|
|
4.2 Equations and their role
5. Method in detail
5.1 Follow one π0.5 request through cache reuse and graph replay
The adapter first determines the observation preprocessing, conditioning and action-conversion rules. The runner then owns the tensors that survive across model calls. For the standard π0.5 path, the vision-language prefix is fixed throughout ten Euler steps while the action tokens evolve. Computing that prefix once avoids repeating invariant work; caching its attention metadata and precomputing timestep and modulation tables also avoids repeated setup. Once shapes, buffers and control flow are stable, CUDA Graph replay executes the captured loop. This reuse has a precise boundary: a new observation or condition invalidates the prefix before the next prediction. The source separately describes fused projections and activations beneath Layers. Its MiniCPM-Robot experiment adds two kernels to an existing graph path, so that local gain should not be attributed to graph replay alone. e-architecturee-optimizationse-kernel-ablation
Figure 4. Cosmos3 parallelizes both guidance branches and transformer computation within each branch. Original paper, p. 10 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read a row from left to right through the repeated denoising region. GPUs zero through three form the conditional TP group; four through seven form the unconditional group. The CFG exchange connects matching TP ranks across these groups. Equation (10) then combines the two velocities using guidance scale gamma, and every rank advances its local video and action UniPC states. The bottom return arrow leads back to the next denoising timestep. Only after the final step does the optional right-hand region decode a row-major two-by-four set of VAE tiles and gather them for stitching. e-parallele-latencye-cosmos-batching
What it supports. Action-only output can stop before VAE decoding, but it still uses the jointly evolving video/action GEN sequence. The eight-H20 latency result uses two CFG branches and four TP shards per branch. This distinction explains why suppressing a video output is not equivalent to removing world-state computation from inference.
Where the evidence stops. CFG degree two counts parallel branches; it is not the guidance scale, which is 3.0 in the separate batch sweep. This schematic and one latency pair do not establish a multi-GPU scaling curve.
5.2 Turn a throughput number into a controller's actual wait
Reader interpretation: the batch result becomes useful only after choosing what the application must optimize. A rollout pool can value the Hopper π0.5 capacity of 100.02 chunks per second at batch 32, but a synchronized client waits 319.94 milliseconds for that batch. Dividing the batch time by 32 gives an accounting cost, not a deadline guarantee. The control-time Roofline adds a second constraint: the next chunk must arrive while the current execution window remains open. Actual deployment must fit observation, transport, queueing and handoff into that window too. Appendix D makes this concrete with a maximum 20-millisecond batching wait and padding to a fixed graph shape. Longer executed horizons can create more time, but the paper warns that the actions then depend on older observations. e-pi-batchinge-batch-definitione-controle-dp-demo
Figure 2. Finishing before the next handoff creates timing margin under ideal overlap. Original paper, p. 7 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Panel (a) divides environment time by inference time on the horizontal axis. Moving right means inference is faster relative to environment execution. The vertical axis divides environment time by the control period. The dashed sequential curve follows X divided by one plus X; the solid roof follows the smaller of X and one. The four suite comparisons labeled RTX PRO 6000 lie to the right of the breakpoint. Panel (b) supplies the timing interpretation: synchronous inference leaves an idle gap, while RTC computes chunk B during execution of chunk A and can finish before the nominal handoff. e-controle-policy
What it supports. The source's four RTX PRO 6000 suite comparisons are environment-bound in this inference-only analysis. Additional acceleration can increase deadline margin without raising the ideal overlap ceiling. The timing schematic describes a possible schedule; the measured points themselves use the sequential sum of inference and environment time.
Where the evidence stops. The separate AGX Orin star is left of one, so the graph does not support a universal environment-bound π0.5 claim. Transport, queues and controller handoff are excluded, and the star's detailed protocol is not given here.
5.3 Apply the RL speedup only to work on the critical path
Reader interpretation: a fast predict function does not accelerate actor training or environment interaction automatically. Figure 9 profiles concurrent rollout ranks, so adding their predict durations would overcount serial work. The source instead uses the largest rank total, 152.4 seconds within a 955.8-second step, to characterize the available opportunity. It measures a 2.55-fold reduction in mean predict time and projects an 863.2-second step only under fixed non-predict work, preserved critical-path placement and zero integration overhead. The introduction's separate eight-A100 simulation gives another estimate under different settings; it is not corroboration of the four-A800 projection. Section 6.3 still identifies usable RLinf integration as future work. A convincing follow-up must time the complete backend replacement and inspect task outcomes as well as predict duration. e-rl-profilee-rl-intro
5.4 Training and inference
During training
PhyAI introduces no policy-learning objective or distillation procedure. It executes existing checkpoints; offline round-to-nearest quantization prepares optional low-precision artifacts. The reported phase/batch profiles use BF16. RLinf retains environment orchestration and policy optimization; PhyAI targets its inference-backend boundary. e-purposee-optimizationse-rl-profile
During inference
A request ends when its complete action chunk is ready. Controller handoff and physical execution are separate stages, and the executed horizon can be shorter than the predicted horizon. Skipping Cosmos3 video decoding still retains joint video/action latent evolution during generation. e-policye-controle-parallel
5.5 Implementation flow
- Separate request state from computation
The scheduler assigns requests and device groups without owning model tensors. Stateful runners manage KV caches, buffers, graph buckets, solver updates and action assembly. Stateless modeling modules express computation through Layers; the operator selector chooses kernels using shape, dtype, accelerator and execution configuration. e-architecture
- Reuse only valid conditions
For standard π0.5, compute the fixed vision-language prefix once across ten Euler steps. Cache its KV tensors and attention metadata, precompute timestep/modulation tables, and capture the stable loop in a CUDA Graph. New observations or conditions invalidate that prefix. Fused projections, activations and normalization reduce launches and intermediate traffic. e-optimizations
- Distribute the appropriate unit of work
π0.5 data-parallel workers process independent requests. Cosmos3 keeps video and action tokens in one GEN sequence, shards transformer computation within each guidance branch, then exchanges conditional/unconditional velocities between matching TP ranks. Each rank applies guidance and advances local video/action UniPC states. Optional tiled VAE decoding follows the last update. e-parallel
6. Experiments & results
PhyAI is a shared inference runtime for existing vision-language-action and world-action policies. Model adapters retain policy semantics while runners, graph replay, fused kernels and parallel services accelerate execution. The experiments establish model-runner speedups and model-dependent batching behavior; appendices add simulation success checks. Control-rate ceilings and RL speedup projections require separate interpretation from measured latency.
The source provides batch/phase diagnostics and a textual two-kernel increment, but no dedicated figure or table decomposing the complete acceleration stack into controlled component ablations. Figure 5 therefore serves as the diagnostic visual. Appendix B supplies RoboTwin settings without numerical success, and Appendix D supplies serving topology without aggregate performance; neither is presented as a missing result invented for this edition. e-kernel-ablatione-pi-batchinge-phase-diagnostice-robotwine-dp-demo
6.1 Read the original evidence
Table 5. Each speedup compares PhyAI with the official implementation on the row's hardware. Original paper, p. 11 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read across a single model/device row before comparing rows. The third and fourth columns are action-chunk model-runner times in milliseconds; the last column divides official latency by PhyAI latency. For example, π0.5 on RTX 5090 changes from 52.076 to 28.613 ms, yielding the reported 1.82-fold speedup. The Cosmos3 row uses eight H20 GPUs and a different generation workload, so its absolute latency cannot rank WAM quality against the VLA rows. Section 5.1 defines the timing boundary, while Section 5.2 and Figure 1 provide the omitted specialized-runtime comparisons and precision caveats. e-latencye-settings
What it supports. Every listed official-path comparison favors PhyAI, with speedups from 1.40-fold for GR00T on Thor to 4.65-fold for MiniCPM-Robot on H100. Cosmos3 improves from 2.46 to 1.18 seconds. The evidence supports broad acceleration over official implementations while preserving the model/device conditions attached to each result.
Where the evidence stops. The table has no uncertainty column and does not establish fastest-in-class execution. Specialized runtimes sometimes beat PhyAI, some comparisons use FP8, and runner latency excludes observation capture, transport, queueing and controller handoff.
Table 7. A matched LIBERO-10 comparison checks task behavior alongside runtime performance. Original paper, p. 22 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Begin with the equal episode counts, then compare the success counts before reading percentages. Appendix A uses all ten LIBERO-10 tasks with fifty fixed initial states each, seeds 42 through 91, and the same checkpoint, task order and simulator settings for both runtimes. Inference runs on A40 at batch one, taking two camera views. The simulator executes the first eight returned actions and checks success after every step, up to a 720-step cap. These controls make this table a more direct behavior comparison than comparing unrelated latency rows or different appendix protocols. e-groot-successe-latency
What it supports. PhyAI records 459 successes against the official implementation's 456, giving a difference of three episodes or 0.6 percentage points. The result supplies evidence of similar aggregate simulated task performance in this configuration. It complements the runtime table without establishing that faster kernels improve the learned policy.
Where the evidence stops. No paired outcome breakdown, confidence interval or significance test is reported. The small aggregate difference cannot establish a robust quality gain, universal numerical equivalence, or physical-robot performance across other models and deployments.
6.2 Results and evaluation conditions
| Task & protocol | Reported result | Comparison & interpretation |
|---|---|---|
| Single-request Cosmos3-Nano-Policy-DROID inference Eight H20 GPUs; CFG parallel degree 2, TP degree 4; model-runner latency. | 1180 ms; 2.08× speedup. Latency and official-to-PhyAI speedup | Official path: 2460 ms. A matched-hardware runtime improvement; no matched FlashRT result is available for this configuration. e-latencye-settings |
| Official-path latency across VLA models Table 5 pairs π0, π0.5, GR00T N1.7 and MiniCPM-Robot with their listed devices. | 1.40×–4.65×; π0.5 on RTX 5090: 28.613 ms. Single-request speedup | π0.5 official: 52.076 ms. Range endpoints are GR00T/Thor and MiniCPM-Robot/H100. Specialized runtimes sometimes win; Figure 1 includes explicitly marked FP8 comparisons. e-latency |
| π0.5 static-batch scaling Hopper-series GPU; BF16, three cameras, 50 actions, ten Euler steps, CUDA Graphs. | Batch 32: 100.02 samples/s and 319.94 ms. Throughput and synchronized latency | Batch 1: 44.26 samples/s and 22.59 ms. The 10.00 ms amortized cost at batch 32 is not request latency. Expert time share falls from 57.2% to 13.5%. e-pi-batchinge-phase-diagnostic |
| Cosmos3 static-batch scaling Hopper-series GPU; eager BF16, four steps, guidance scale 3.0, 32 actions, 33 frames at 480 × 832; optional video decode excluded. | Batch 16: 1.010 chunks/s; 15.85 s batch latency. Action-chunk throughput | Batch 1: 0.883 chunks/s; 1.132 s. Only 14.3% throughput gain; generation already uses compute heavily. This is separate from the eight-H20 comparison. e-cosmos-batchinge-phase-diagnostic |
| GR00T batch saturation RTX 5090, BF16; two 256 × 256 views, action shape 40 × 132, four flow steps, CUDA Graphs. | Batch 32: 138.8 samples/s; batch 8 reaches 92.2% of that. Throughput | Batch 1: 48.8 samples/s. Backbone becomes dominant from batch four; further batching increasingly raises synchronized wait. e-settingse-groot-batching |
| MiniCPM-Robot kernel increment H20, BF16, CUDA Graph path; add two fused kernels. | 36.77 Hz; reported 10.5% gain. Throughput | 33.28 Hz before the two kernels. A local combined-kernel ablation, without separate attribution to each kernel. e-kernel-ablation |
| GR00T-N1.7 matched LIBERO-10 success A40; same checkpoint, 50 initial states/task, seeds 42–91, batch 1, first eight actions executed, 720-step cap. | PhyAI: 459/500 (91.8%). Successful episodes | NVIDIA Isaac-GR00T: 456/500 (91.2%). Three more successes, or 0.6 percentage points; no significance test or uncertainty is reported. e-groot-success |
| π0.5 success across four LIBERO suites 500 episodes per suite; synchronous ten-action chunks, BF16 and CUDA Graphs. | 1949/2000 (97.45%); spatial 97.8%, object 99.8%, goal 98.0%, LIBERO-10 94.2%. Successful episodes | No matched baseline in Table 9. Author-reported simulation results; inference means are 36.10–36.33 ms, with GPU model unspecified in Appendix C. e-pi-success |
| π0 success on LIBERO spatial and object Batch 1; ten episodes/task; BF16; execute each 50-action chunk before querying again. | 143/200 (71.5%): spatial 71/100, object 72/100. Successful episodes | No matched baseline in Table 11. Different policy, horizon and episode count prevent direct comparison with Appendix C. e-pi0-successe-pi-success |
| Projected GRPO RL-step acceleration LIBERO-10 baseline on four A800 80 GB GPUs without NVLink; 32 environments, four denoising steps. | Measured predict speedup: 2.55×. Projected step: 863.2 s, 9.7% lower, 1.11× faster. Mean predict speedup and projected step time | Baseline step: 955.8 s; maximum-rank predict time: 152.4 s. Amdahl projection holds non-predict work fixed, assumes accelerated calls remain critical and adds no integration overhead; not measured end-to-end. e-rl-profile |
6.3 Ablations and diagnostic examples
Read component removals and qualitative examples within their stated evaluation conditions.
Figure 5. Batching raises capacity while moving the bottleneck and increasing each request's synchronized wait. Original paper, p. 12 ↗
Excerpt from the authors’ paper; cropped without altering the figure or table.
How to read it. Read panels (a) and (b) together: both use batch size on the horizontal axis, but one measures completed chunks per second and the other measures the time until the entire batch finishes. The blue Hopper curve gains throughput while its batch latency rises. The purple Thor curve plateaus, with a marked high-variance region at batch four. Then inspect panel (c), using its legend to follow the dark action-expert share as vision and language take more of the profiled time. The adjacent text fixes BF16, three cameras, a 50-action chunk, ten Euler steps and CUDA Graphs. e-pi-batchinge-phase-diagnostice-batch-definitione-settings
What it supports. On Hopper, batch 32 provides 100.02 samples/s while every synchronized request waits 319.94 ms. The accompanying phase analysis reports that the expert's share falls from 57.2% at batch one to 13.5% at batch 32. Capacity gains therefore change which phase is most worth optimizing; they do not make every robot response faster.
Where the evidence stops. This is a batch/phase diagnostic, not an ablation isolating cache reuse, fusion and graph replay. Static batches exclude formation delay and networking; phase arithmetic-intensity explanations use modeled logical traffic rather than measured DRAM traffic.
7. Analysis & limitations
7.1 What the evidence leaves open
Internal v3 differences remain unresolved: the introduction instead describes an eight-A100 rollout simulation projecting 26.5% lower latency and 1.36× throughput; Section 5.4 uses four A800 GPUs. Section 6.3 still calls usable RLinf integration future work. These accounts must not be merged. e-rl-introe-rl-profile
Section 5.5 excludes closed-loop quality and tail latency from its summary, although Appendices A/C/E report simulation success and Figure 6 plots runner P99. The narrower reading is warranted: full deployment tail latency, physical-robot quality and integrated RL gains remain unestablished. e-boundariese-groot-batchinge-groot-successe-pi-successe-pi0-success
Logical arithmetic intensity uses modeled tensor traffic, not measured DRAM transfers. Phase profiles diagnose batch behavior without isolating all optimizations or measuring distributed scaling. The Hopper-series device is unnamed. e-settingse-phase-diagnostic
7.2 Questions for discussion
- Can exact cache reuse and graph replay preserve episode outcomes under changing observations and partial batches?
- How much of the projected RL gain survives backend synchronization and queueing?
8. Reproducibility audit
8.1 Requirements and known gaps
Match checkpoint, precision, camera order, solver steps and executed horizon before timing. Profiles specify PyTorch 2.11.0/CUDA 13.0; π0.5 uses ten warm-ups and 50 timed iterations. Appendix A separately specifies A40, PhyAI 0.1.0, Python 3.12.11 and MuJoCo 3.2.3. e-settingse-pi-batchinge-groot-success
RoboTwin requires matching q01/q99 normalization and swapping wrist views into head/left/right order. Appendix B describes 100 episodes but supplies no success number or exact archived revision. Appendix C omits its GPU model; Appendix A mentions a pinned image without printing its digest. e-robotwine-pi-successe-groot-success
Appendix D specifies eight H20 workers, 32 clients, local graph batch four, padding and up-to-20-ms batch formation, using one-action chunks. It supplies no aggregate throughput or success. A reproduction must measure full request wait as well as compute time. e-dp-demoe-batch-definition
8.2 Proposed reproduction checks
The following checks are proposals motivated by the paper. They have not been run as part of this reading.
Check 1: Separate cache correctness from graph speed
Reader-proposed, not run: use one π0.5 checkpoint with the Table 4 BF16 configuration, fixed observations and initial noise, and test batches 1, 8 and 32. Compare reference inference with cached-prefix execution, graph replay and their combination, holding solver steps and outputs fixed. Include unchanged-condition repeats and deliberately changed camera/state/instruction inputs. Measure action-tensor error, prefix recomputation and complete chunk latency after the prescribed warm-up. A speedup accompanied by stale-condition outputs would falsify correct cache invalidation; a gain only at small batches would support the launch-overhead explanation. Report the comparison tolerances explicitly. e-optimizationse-settingse-pi-batchinge-phase-diagnostic
Check 2: Replace the RL backend and test the Amdahl projection
Reader-proposed, not run: repeat the four-A800 LIBERO-10 GRPO configuration from Table 6, including 32 environments, four denoising steps and identical actor-update settings, using reference and PhyAI backends. Keep evaluation, checkpoint writing and video capture disabled for the timing comparison as in the source. Trace complete steady-state steps, maximum-rank predict time, queueing, weight synchronization and actor training; separately compare trajectory outcomes under matched evaluation. Test whether a similar predict speedup produces the projected 9.7% step reduction. A smaller gain with added synchronization or off-critical-path calls would reject the idealized end-to-end estimate for that integration. e-rl-profilee-rl-intro
8.3 Reading coverage
Visual audit: Visually inspected the title/author/version page, every main-body page, all nine figures, all eleven tables and Appendices A–E. Figure 3 ownership arrows, Figure 4 guidance/loop arrows and optional-decode boundary, and Figure 2 axes and schedule were cross-checked against Sections 3–4 and Equations (6)–(10). All six final crops were viewed; the narrow Table 7 crop was rendered directly at higher DPI. References on pages 18–21 were read as text but were not visually inspected. No separate supplement, code, archived run or external source was inspected. The RL setup differences across pages 3, 15 and 16 and the broader appendix scope relative to Section 5.5 are preserved in the report.
PDF pages inspected for this edition: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 22, 23, 24, 25. Appendix coverage: reviewed.
Original figures and tables remain the work of the source’s authors. Extractions preserve their scientific content; any HTML wrapper layout is disclosed with each figure. The surrounding reading notes are our own.
Text reading scope & known omissions
- Abstract; Sections 1–2.3 (Introduction and Related Work)
- Sections 3.1–3.4 (task, control-time Roofline, deployment and batching)
- Sections 4.1–4.3 (architecture, optimizations and parallel execution)
- Sections 5.1–5.5 (measurement settings, latency, phase/batch analysis and RL)
- Sections 6.1–6.5 and 7.1–7.3 (future work, discussion and conclusion)
- References, PDF pp. 18–21
- Appendices A–E, PDF pp. 22–25
Outside the original text pass
- Text extraction does not reconstruct figure images; inspect the retained PDF for figures and equation/table layout.
- Separate supplemental material availability has not been fully verified.
- The figure-extraction limitation was addressed by inspecting original PDF pages and six faithful crops. All eight supplied text chunks were read individually, including references and Appendices A–E.
- Identity: the supplied title and 26-author list match the catalog. The inspected artifact is arXiv:2608.03682v3, dated 14 August 2026. The catalog URL is unversioned; earlier revisions were not supplied, and their differences or original submission date cannot be established.
- Code, benchmark harnesses, archived runs and separate supplements were not inspected; no experiments were reproduced. Availability statements about these resources remain author claims.
The visual audit above records the subsequent illustrated pass.
8.4 Traceable evidence
e-identityPDF p. 1, title, author block and arXiv margin
The title and all 26 authors match the catalog; the PDF is arXiv:2608.03682v3, dated 14 August 2026. Six institutional affiliations are printed.
Go to primary source ↓e-purposePDF pp. 2–3, Section 1; p. 4, Section 2.2
PhyAI shares an inference path across deployment settings while leaving training and RL orchestration to existing systems.
Go to primary source ↓e-policyPDF pp. 5–6, Section 3.1, Equations (1)–(2)
Camera observations, instruction and robot state produce an action chunk. Observation-to-handoff latency includes observation, transfer, queueing, inference and actuation, but excludes execution of the returned actions.
Go to primary source ↓e-controlPDF p. 6, Section 3.2, Equations (3)–(8); p. 7, Figure 2 and Sections 3.3–3.4
Sequential control adds inference and environment time; ideal overlap takes their maximum. The plotted timing omits other request overheads. Figure 2 shows four LIBERO-suite comparisons on RTX PRO 6000 and a separate AGX Orin star.
Go to primary source ↓e-batch-definitionPDF p. 7, Section 3.4, Equation (9)
Throughput is batch size divided by synchronized batch time; amortized per-sample time is not the wait experienced by a request.
Go to primary source ↓e-architecturePDF p. 8, Sections 4–4.2 and Figure 3
Adapters define policy semantics; the scheduler places requests; runners own buffers, caches and graph buckets; stateless modeling modules call shared Layers and shape-dependent kernel dispatch.
Go to primary source ↓e-optimizationsPDF p. 9, Section 4.2, Kernel fusion, Graph replay, Profiled operator selection and Quantization
The π0.5 prefix is reused within ten Euler steps and invalidated for new conditions. Fixed timestep/modulation tables and graph capture reduce repeated work. Offline round-to-nearest quantization is separate from BF16 batch profiles.
Go to primary source ↓e-kernel-ablationPDF p. 9, Section 4.2, Kernel fusion, MiniCPM-Robot paragraph
On H20 in BF16, adding two fused kernels to the CUDA Graph path changes throughput from 33.28 to 36.77 Hz, a reported 10.5% gain; kernels are checked against FP32/BF16 PyTorch references.
Go to primary source ↓e-parallelPDF pp. 9–10, Section 4.3, Figure 4 and Equation (10)
Independent π0.5 requests use DP. Cosmos3 jointly evolves video/action tokens using TP within conditional and unconditional branches, matching-rank CFG all-gather, rank-local guidance and UniPC. Video VAE decoding is optional after the final step.
Go to primary source ↓e-settingsPDF pp. 10–11, Section 5.1, Table 4 and Equations (11)–(13)
Measurements concern model-runner action-chunk time. Phase/batch profiles use BF16; logical tensor traffic is modeled rather than measured DRAM traffic. Profiles use PyTorch 2.11.0 and CUDA 13.0; the Hopper-series GPU model is not specified.
Go to primary source ↓e-latencyPDF p. 11, Section 5.2 and Table 5, all model/device rows; p. 1, Figure 1
All 11 official-to-PhyAI pairs in Table 5 improve by 1.40×–4.65×. Cosmos3 on eight H20 GPUs takes 2460 versus 1180 ms with CFG degree two and TP degree four. Specialized runtimes can be faster and available comparisons are not fully precision matched.
Go to primary source ↓e-pi-batchingPDF p. 12, Section 5.3, PI0.5 paragraph and Figure 5
BF16 π0.5 with three views, 50 actions, ten Euler steps and CUDA Graphs uses ten warm-ups and 50 timed iterations. Hopper throughput rises from 44.26 to 100.02 samples/s between batches one and 32, while full-batch time rises from 22.59 to 319.94 ms. Thor batch four is explicitly noisy.
Go to primary source ↓e-groot-batchingPDF pp. 12–13, Section 5.3, GR00T paragraph and Figure 6
RTX 5090 GR00T BF16 throughput rises from 48.8 to 138.8 samples/s from batch one to 32; batch eight reaches 92.2% of batch-32 throughput. Backbone overtakes Action Head from batch four. The figure includes runner-latency P99 measurements.
Go to primary source ↓e-cosmos-batchingPDF pp. 12–13, Section 5.3, Cosmos3 paragraph and Figure 7; p. 10, Table 4
The eager BF16 sweep uses four denoising steps, guidance scale 3.0, 32 actions at 15 Hz and 33 frames at 480 × 832, excluding optional video decode. Hopper throughput rises from 0.883 to 1.010 chunks/s as batch grows one to 16; full-batch time reaches 15.85 s. RTX 5090 runs out of memory.
Go to primary source ↓e-phase-diagnosticPDF p. 14, Figure 8 and Section 5.3 analysis; p. 11, Section 5.1
At batch one the Hopper π0.5 expert has 8.8% of estimated FLOPs but 57.2% of profiled time; at batch 32 its time share is 13.5%. Cosmos3 generation is compute-heavy in the measured logical-intensity diagnostic, which does not establish TP/CFG scaling.
Go to primary source ↓e-rl-profilePDF pp. 14–16, Section 5.4; p. 15, Table 6 and Figure 9
A four-A800-80-GB GRPO LIBERO-10 baseline step takes 955.8 s, with maximum-rank predict time 152.4 s. A measured 2.55× mean-predict speedup motivates an idealized 863.2 s step projection; the 1.11× end-to-end speedup is not measured.
Go to primary source ↓e-rl-introPDF p. 3, Section 1, simulated RL rollout paragraph; p. 16, Section 6.3
The introduction separately reports an eight-A100 simulation with batch 40 and 41 policy calls per RL step, estimating 26.5% lower step latency and about 1.36× throughput. Section 6.3 still describes usable RLinf backend integration as future work.
Go to primary source ↓e-boundariesPDF p. 16, Sections 5.5 and 6.1–6.4; p. 17, Sections 6.5–7.3
The main evaluation summary limits runtime claims and excludes full deployment overheads. Persistent kernels, kernel-agent tuning, broader support and a production serving protocol are future work. Hardware-cost and control-budget trades are proposed rather than quantified.
Go to primary source ↓e-groot-successPDF p. 22, Appendix A, Inference server, Evaluation protocol, Results and Table 7
GR00T-N1.7-LIBERO/libero_10 is evaluated on A40 with two 256 × 256 views, batch one, first eight actions executed, 720-step cap, 50 initial states per task and seeds 42–91. Matched official/PhyAI trials yield 456/500 (91.2%) versus 459/500 (91.8%).
Go to primary source ↓e-robotwinPDF pp. 22–23, Appendix B and Table 8
RoboTwin requires the motus-robotics/pi0.5_robotwin2 checkpoint, matched q01/q99 normalization, reordered head/left/right views, BF16, ten denoising steps and 32 × 14 outputs; ten actions are executed at 30 Hz. A 100-episode evaluation is described without a numerical success result.
Go to primary source ↓e-pi-successPDF p. 23, Appendix C, Model configuration, Simulation protocol, Results and Table 9
Four LIBERO suites each use 500 episodes, synchronous ten-action chunks, BF16, FlashInfer and CUDA Graphs. Success counts are 489, 499, 490 and 471, totaling 1949/2000; mean inference ranges 36.10–36.33 ms. No matched baseline or GPU model is listed here.
Go to primary source ↓e-dp-demoPDF p. 24, Appendix D and Table 10
An eight-H20 demonstration serves 32 clients with maximum server batch 32, local graph batch four, up-to-20-ms batching wait and one-action chunks. Padding handles incomplete batches; no aggregate throughput or success result is supplied.
Go to primary source ↓e-pi0-successPDF p. 25, Appendix E and Table 11
BF16 π0 evaluates spatial and object suites with 100 episodes each, batch one and 50-action chunks exhausted before replanning. Results are 71/100 and 72/100, totaling 143/200 (71.5%), without a matched runtime baseline.
Go to primary source ↓8.5 Primary sources
PhyAI: Real-Time Physical AI at the Edge, Scalable Rollouts in the Cloud ↗
PDF · 14,355 extracted words
Source fingerprint
3e23c2ab7adce5f2f50a126ff4ab9c71e4f15fbe4e9f1ed56fe7e8690802a42b