RESEARCH PAPERYear 2023

RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control

View all authors

Brianna Zitkovich; Tianhe Yu; Sichun Xu; Peng Xu; Ted Xiao; Fei Xia; Jialin Wu; Paul Wohlhart; Stefan Welker; Ayzaan Wahid; Quan Vuong; Vincent Vanhoucke; Huong T. Tran; Radu Soricut; Anikait Singh; Jaspiar Singh; Pierre Sermanet; Pannag R. Sanketi; Grecia Salazar; Michael S. Ryoo; Krista Reymann; Kanishka Rao; Karl Pertsch; Igor Mordatch; Henryk Michalewski; Yao Lu; Sergey Levine; Lisa Lee; Tsang-Wei Edward Lee; Isabel Leal; Yuheng Kuang; Dmitry Kalashnikov; Ryan Julian; Nikhil J. Joshi; Alex Irpan; Brian Ichter; Jasmine Hsu; Alexander Herzog; Karol Hausman; Keerthana Gopalakrishnan; Chuyuan Fu; Pete Florence; Chelsea Finn; Kumar Avinava Dubey; Danny Driess; Tianli Ding; Krzysztof Marcin Choromanski; Xi Chen; Yevgen Chebotar; Justice Carbajal; Noah Brown; Anthony Brohan; Montserrat Gonzalez Arenas; Kehang Han

Read paper PDF

Classification

View four quadrants

Major category: VLA
Quadrant: Not applicable
Architecture: One Model
Prediction paradigm: Not applicable
Subcategories: Autoregressive VLA
Source review status: Verified from primary sources

AT A GLANCE

Contribution

A contribution summary has not been added yet.

Abstract

An abstract has not been added yet.

Affiliations

Not listed in the collection.

BibTeX

@inproceedings{rt2,
  author       = {Brianna Zitkovich and
                  Tianhe Yu and
                  Sichun Xu and
                  Peng Xu and
                  Ted Xiao and
                  Fei Xia and
                  Jialin Wu and
                  Paul Wohlhart and
                  Stefan Welker and
                  Ayzaan Wahid and
                  Quan Vuong and
                  Vincent Vanhoucke and
                  Huong T. Tran and
                  Radu Soricut and
                  Anikait Singh and
                  Jaspiar Singh and
                  Pierre Sermanet and
                  Pannag R. Sanketi and
                  Grecia Salazar and
                  Michael S. Ryoo and
                  Krista Reymann and
                  Kanishka Rao and
                  Karl Pertsch and
                  Igor Mordatch and
                  Henryk Michalewski and
                  Yao Lu and
                  Sergey Levine and
                  Lisa Lee and
                  Tsang{-}Wei Edward Lee and
                  Isabel Leal and
                  Yuheng Kuang and
                  Dmitry Kalashnikov and
                  Ryan Julian and
                  Nikhil J. Joshi and
                  Alex Irpan and
                  Brian Ichter and
                  Jasmine Hsu and
                  Alexander Herzog and
                  Karol Hausman and
                  Keerthana Gopalakrishnan and
                  Chuyuan Fu and
                  Pete Florence and
                  Chelsea Finn and
                  Kumar Avinava Dubey and
                  Danny Driess and
                  Tianli Ding and
                  Krzysztof Marcin Choromanski and
                  Xi Chen and
                  Yevgen Chebotar and
                  Justice Carbajal and
                  Noah Brown and
                  Anthony Brohan and
                  Montserrat Gonzalez Arenas and
                  Kehang Han},
  title        = {{RT-2:} Vision-Language-Action Models Transfer Web Knowledge to Robotic
                  Control},
  booktitle    = {CoRL},
  pages        = {2165--2183},
  year         = {2023},

}

Classification

Contribution

Abstract

Affiliations

BibTeX

More in this category

Beyond Data Scaling: Representation-Centric Continued Pre-training for Vision-Language-Action Models

What Matters for Latent Actions in Robot Learning

SLIM-0.5B: Learning Action-Grounded Predictive Latents for Robot Manipulation